From dakota at brokenpipe.ru Thu May 2 00:23:22 2013 From: dakota at brokenpipe.ru (Marat Dakota) Date: Thu, 2 May 2013 04:23:22 +0400 Subject: proxy_pass takes Content-Length from another subrequest Message-ID: Hi. My module makes a few subrequests with body (I populate sr->request_body and set Content-Length in sr-headers_in). These subrequests are landing to a locations that look like: location /backend { internal; rewrite /backend(.+) $1 break; proxy_pass http://mybackend; } location /backend2 { internal; rewrite /backend2(.+) $1 break; proxy_pass http://mybackend2; } The problem is that proxy_pass takes Content-Length from the first subrequest and uses it for the second subrequest. So, when the first subrequest is longer than the second one, the second one never ends, and when the first one is shorter, the second backend (mybackend2) receives incomplete body. I've spent quite a time tracing the problem and for the moment I've figured out that Content-Length comes from these lines in ngx_http_proxy_module.c: while (*(uintptr_t *) e.ip) { code = *(ngx_http_script_code_pt *) e.ip; code((ngx_http_script_engine_t *) &e); } I haven't managed to go deeper at the moment. So, question is ? am I doing something wrong or is it a bug? Thanks. -- Marat -------------- next part -------------- An HTML attachment was scrubbed... URL: From piotr at cloudflare.com Thu May 2 10:32:51 2013 From: piotr at cloudflare.com (Piotr Sikora) Date: Thu, 2 May 2013 03:32:51 -0700 Subject: [PATCH] PCRE: retain input pattern for all regular expressions Message-ID: # HG changeset patch # User Piotr Sikora # Date 1367490396 25200 # Node ID c6434a8634386be8b9bfb90be47c4902a009b36b # Parent 886800caf36081e1fcbecd0483abeefbafc781b3 PCRE: retain input pattern for all regular expressions. Previously, input pattern was kept only for regular expressions with named captures, which resulted in error log entries without input pattern for PCRE errors that occured while processing regular expressions without them. Signed-off-by: Piotr Sikora diff -r 886800caf360 -r c6434a863438 src/http/ngx_http_variables.c --- a/src/http/ngx_http_variables.c Mon Apr 29 18:58:58 2013 +0400 +++ b/src/http/ngx_http_variables.c Thu May 02 03:26:36 2013 -0700 @@ -2257,6 +2257,7 @@ re->regex = rc->regex; re->ncaptures = rc->captures; + re->name = rc->pattern; cmcf = ngx_http_conf_get_module_main_conf(cf, ngx_http_core_module); cmcf->ncaptures = ngx_max(cmcf->ncaptures, re->ncaptures); @@ -2274,7 +2275,6 @@ re->variables = rv; re->nvariables = n; - re->name = rc->pattern; size = rc->name_size; From fdasilvayy at gmail.com Thu May 2 14:01:29 2013 From: fdasilvayy at gmail.com (Filipe Da Silva) Date: Thu, 2 May 2013 16:01:29 +0200 Subject: [PATCH] ngx_mail_http_auth_module Message-ID: Hello, Nginx developers, I been working recently on NGinx, as part of my job . And I see some small issues as this one : This call to *ngx_close_connection* is useless as it was already made 18 lines before in the code . Same code is present in 1.2 Regards, Filipe ---- diff -r 8222ca034980 src/mail/ngx_mail_auth_http_module.c --- a/src/mail/ngx_mail_auth_http_module.c Tue Apr 02 12:34:39 2013 +0000 +++ b/src/mail/ngx_mail_auth_http_module.c Thu May 02 15:46:42 2013 +0200 @@ -696,7 +696,6 @@ p = ngx_pnalloc(s->connection->pool, ctx->err.len); if (p == NULL) { - ngx_close_connection(ctx->peer.connection); ngx_destroy_pool(ctx->pool); ngx_mail_session_internal_server_error(s); return; -------------- next part -------------- An HTML attachment was scrubbed... URL: From leixunen at gmail.com Fri May 3 02:51:58 2013 From: leixunen at gmail.com (xunen) Date: Fri, 3 May 2013 10:51:58 +0800 Subject: [PATCH] Add Range support with gzip filter output Message-ID: Hello, in case of gzip filter output for range header request(Range: start-end), range module is not work, attached patch fixes this. Thanks, Zhou.Lei diff -urN nginx-1.4.0/auto/modules nginx-1.4.0-patched/auto/modules --- nginx-1.4.0/auto/modules 2013-03-20 18:36:57.000000000 +0800 +++ nginx-1.4.0-patched/auto/modules 2013-05-03 10:29:33.000000000 +0800 @@ -125,8 +125,6 @@ HTTP_FILTER_MODULES="$HTTP_FILTER_MODULES $HTTP_SPDY_FILTER_MODULE" fi -HTTP_FILTER_MODULES="$HTTP_FILTER_MODULES $HTTP_RANGE_HEADER_FILTER_MODULE" - if [ $HTTP_GZIP = YES ]; then have=NGX_HTTP_GZIP . auto/have USE_ZLIB=YES @@ -134,6 +132,8 @@ HTTP_SRCS="$HTTP_SRCS $HTTP_GZIP_SRCS" fi +HTTP_FILTER_MODULES="$HTTP_FILTER_MODULES $HTTP_RANGE_HEADER_FILTER_MODULE" + if [ $HTTP_POSTPONE = YES ]; then HTTP_FILTER_MODULES="$HTTP_FILTER_MODULES $HTTP_POSTPONE_FILTER_MODULE" HTTP_SRCS="$HTTP_SRCS $HTTP_POSTPONE_FILTER_SRCS" diff -urN nginx-1.4.0/src/http/modules/ngx_http_gzip_filter_module.c nginx-1.4.0-patched/src/http/modules/ngx_http_gzip_filter_module.c --- nginx-1.4.0/src/http/modules/ngx_http_gzip_filter_module.c 2013-03-11 19:19:58.000000000 +0800 +++ nginx-1.4.0-patched/src/http/modules/ngx_http_gzip_filter_module.c 2013-05-03 10:31:14.000000000 +0800 @@ -245,6 +245,7 @@ if (!conf->enable || (r->headers_out.status != NGX_HTTP_OK + && r->headers_out.status != NGX_HTTP_PARTIAL_CONTENT && r->headers_out.status != NGX_HTTP_FORBIDDEN && r->headers_out.status != NGX_HTTP_NOT_FOUND) || (r->headers_out.content_encoding -------------- next part -------------- An HTML attachment was scrubbed... URL: From zls.sogou at gmail.com Fri May 3 03:20:46 2013 From: zls.sogou at gmail.com (lanshun zhou) Date: Fri, 3 May 2013 11:20:46 +0800 Subject: [BUG] Wrong Content-Length shared between main request and subrequests in http proxy module Message-ID: Nginx uses a variable "$proxy_internal_body_length" for the "Content-Length" header in http proxy module, which is cacheable and shared between the main request and all subrequests by default. Although ctx->internal_body_length is calculated for each request/subrequest, the string format is cached after the first generation, and this may be wrong for other subrequests. The following simple config can reproduce the problem, where xxxxx is a place that can receive post data. curl --data-binary "xxxxxxxxxxxxxxx" localhost/test.html will cause the client hang. location = /test.html { add_after_body /vpost; proxy_pass http://xxxxx; } location /vpost { proxy_set_body "in vpost"; proxy_pass http://xxxxx; } -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: proxy_internal_body_length.patch Type: application/octet-stream Size: 551 bytes Desc: not available URL: From zls.sogou at gmail.com Fri May 3 03:26:11 2013 From: zls.sogou at gmail.com (lanshun zhou) Date: Fri, 3 May 2013 11:26:11 +0800 Subject: proxy_pass takes Content-Length from another subrequest In-Reply-To: References: Message-ID: You can either use separated variables between the main request and subrequests by creating new ones for subrequest after calling ngx_http_subrequest, or mark variable "$proxy_internal_body_length" as no_cacheable. See http://mailman.nginx.org/pipermail/nginx-devel/2013-May/003669.html for more details. 2013/5/2 Marat Dakota > Hi. > > My module makes a few subrequests with body (I populate sr->request_body > and set Content-Length in sr-headers_in). > > These subrequests are landing to a locations that look like: > > location /backend { > internal; > rewrite /backend(.+) $1 break; > proxy_pass http://mybackend; > } > > location /backend2 { > internal; > rewrite /backend2(.+) $1 break; > proxy_pass http://mybackend2; > } > > The problem is that proxy_pass takes Content-Length from the first > subrequest and uses it for the second subrequest. So, when the first > subrequest is longer than the second one, the second one never ends, and > when the first one is shorter, the second backend (mybackend2) receives > incomplete body. > > I've spent quite a time tracing the problem and for the moment I've > figured out that Content-Length comes from these lines in > ngx_http_proxy_module.c: > > while (*(uintptr_t *) e.ip) { > code = *(ngx_http_script_code_pt *) e.ip; > code((ngx_http_script_engine_t *) &e); > } > > I haven't managed to go deeper at the moment. > > So, question is ? am I doing something wrong or is it a bug? > > Thanks. > > -- > Marat > > _______________________________________________ > nginx-devel mailing list > nginx-devel at nginx.org > http://mailman.nginx.org/mailman/listinfo/nginx-devel > -------------- next part -------------- An HTML attachment was scrubbed... URL: From dakota at brokenpipe.ru Fri May 3 21:29:41 2013 From: dakota at brokenpipe.ru (Marat Dakota) Date: Sat, 4 May 2013 01:29:41 +0400 Subject: proxy_pass takes Content-Length from another subrequest In-Reply-To: References: Message-ID: Thanks, I've reset subrequest's variables and it's ok now. On Fri, May 3, 2013 at 7:26 AM, lanshun zhou wrote: > You can either use separated variables between the main request and > subrequests by creating new ones for subrequest after calling > ngx_http_subrequest, or mark variable "$proxy_internal_body_length" as > no_cacheable. > > See http://mailman.nginx.org/pipermail/nginx-devel/2013-May/003669.html for > more details. > > > 2013/5/2 Marat Dakota > >> Hi. >> >> My module makes a few subrequests with body (I populate sr->request_body >> and set Content-Length in sr-headers_in). >> >> These subrequests are landing to a locations that look like: >> >> location /backend { >> internal; >> rewrite /backend(.+) $1 break; >> proxy_pass http://mybackend; >> } >> >> location /backend2 { >> internal; >> rewrite /backend2(.+) $1 break; >> proxy_pass http://mybackend2; >> } >> >> The problem is that proxy_pass takes Content-Length from the first >> subrequest and uses it for the second subrequest. So, when the first >> subrequest is longer than the second one, the second one never ends, and >> when the first one is shorter, the second backend (mybackend2) receives >> incomplete body. >> >> I've spent quite a time tracing the problem and for the moment I've >> figured out that Content-Length comes from these lines in >> ngx_http_proxy_module.c: >> >> while (*(uintptr_t *) e.ip) { >> code = *(ngx_http_script_code_pt *) e.ip; >> code((ngx_http_script_engine_t *) &e); >> } >> >> I haven't managed to go deeper at the moment. >> >> So, question is ? am I doing something wrong or is it a bug? >> >> Thanks. >> >> -- >> Marat >> >> _______________________________________________ >> nginx-devel mailing list >> nginx-devel at nginx.org >> http://mailman.nginx.org/mailman/listinfo/nginx-devel >> > > > _______________________________________________ > nginx-devel mailing list > nginx-devel at nginx.org > http://mailman.nginx.org/mailman/listinfo/nginx-devel > -------------- next part -------------- An HTML attachment was scrubbed... URL: From dakota at brokenpipe.ru Fri May 3 21:52:12 2013 From: dakota at brokenpipe.ru (Marat Dakota) Date: Sat, 4 May 2013 01:52:12 +0400 Subject: Reorder output chain when subrequests are used Message-ID: Hi. Let's suppose we've called ngx_http_output_filter(r, "aaa") for the main request, then we've called ngx_http_subrequest() and then ngx_http_output_filter(r, "bbb") for the main request. Let's suppose the subrequest's handler has called ngx_http_output_filter(sr, "ccc"). So, the result will be "aaacccbbb". What is the proper way to reorder main request's output chain to get "aaabbbccc" (i.e. how to move subrequest's output chain to the end of the main output chain)? -- Marat -------------- next part -------------- An HTML attachment was scrubbed... URL: From mdounin at mdounin.ru Fri May 3 22:03:44 2013 From: mdounin at mdounin.ru (Maxim Dounin) Date: Sat, 4 May 2013 02:03:44 +0400 Subject: Reorder output chain when subrequests are used In-Reply-To: References: Message-ID: <20130503220343.GC69760@mdounin.ru> Hello! On Sat, May 04, 2013 at 01:52:12AM +0400, Marat Dakota wrote: > Hi. > > Let's suppose we've called ngx_http_output_filter(r, "aaa") for the main > request, then we've called ngx_http_subrequest() and then > ngx_http_output_filter(r, "bbb") for the main request. > > Let's suppose the subrequest's handler has called > ngx_http_output_filter(sr, "ccc"). > > So, the result will be "aaacccbbb". > > What is the proper way to reorder main request's output chain to get > "aaabbbccc" (i.e. how to move subrequest's output chain to the end of the > main output chain)? Proper way is to output "aaa", then "bbb", and then call a subrequest. -- Maxim Dounin http://nginx.org/en/donation.html From dakota at brokenpipe.ru Fri May 3 22:28:57 2013 From: dakota at brokenpipe.ru (Marat Dakota) Date: Sat, 4 May 2013 02:28:57 +0400 Subject: Reorder output chain when subrequests are used In-Reply-To: <20130503220343.GC69760@mdounin.ru> References: <20130503220343.GC69760@mdounin.ru> Message-ID: Ok, I need improper way then. Just some way. The problem is that I know "bbb" only after the subrequest is made. And subrequest's body is being completely filtered by body filter. So, in fact I will not have "ccc", just "aaabbb", but I can't flush "bbb" out because it waits for the subrequest to be done. On Sat, May 4, 2013 at 2:03 AM, Maxim Dounin wrote: > Hello! > > On Sat, May 04, 2013 at 01:52:12AM +0400, Marat Dakota wrote: > > > Hi. > > > > Let's suppose we've called ngx_http_output_filter(r, "aaa") for the main > > request, then we've called ngx_http_subrequest() and then > > ngx_http_output_filter(r, "bbb") for the main request. > > > > Let's suppose the subrequest's handler has called > > ngx_http_output_filter(sr, "ccc"). > > > > So, the result will be "aaacccbbb". > > > > What is the proper way to reorder main request's output chain to get > > "aaabbbccc" (i.e. how to move subrequest's output chain to the end of the > > main output chain)? > > Proper way is to output "aaa", then "bbb", and then call a > subrequest. > > -- > Maxim Dounin > http://nginx.org/en/donation.html > > _______________________________________________ > nginx-devel mailing list > nginx-devel at nginx.org > http://mailman.nginx.org/mailman/listinfo/nginx-devel > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mdounin at mdounin.ru Fri May 3 22:35:04 2013 From: mdounin at mdounin.ru (Maxim Dounin) Date: Sat, 4 May 2013 02:35:04 +0400 Subject: [PATCH] Add Range support with gzip filter output In-Reply-To: References: Message-ID: <20130503223503.GD69760@mdounin.ru> Hello! On Fri, May 03, 2013 at 10:51:58AM +0800, xunen wrote: > Hello, > > in case of gzip filter output for range header request(Range: start-end), > range module is not work, attached patch fixes this. This is expected behaviour. With gzipping on the fly via gzip filter result is not something stable - it may change depending on various settings, as well as various timing factors. Consider e.g. proxied response with proxy_buffering set to off - each buffer passed to the gzip filter will have flush flag set, and gzipped response will depend on speed of a network to an upstream server, and on an nginx server load. This makes gzip filter results unsuitable for byterange requests. Additionally, moving range filter in the filter chain as your patch does is very bad idea from performance point of view. -- Maxim Dounin http://nginx.org/en/donation.html From agentzh at gmail.com Sat May 4 18:42:04 2013 From: agentzh at gmail.com (agentzh) Date: Sat, 4 May 2013 11:42:04 -0700 Subject: Reorder output chain when subrequests are used In-Reply-To: References: <20130503220343.GC69760@mdounin.ru> Message-ID: Hello! On Fri, May 3, 2013 at 3:28 PM, Marat Dakota wrote: > Ok, I need improper way then. Just some way. > > The problem is that I know "bbb" only after the subrequest is made. And > subrequest's body is being completely filtered by body filter. So, in fact I > will not have "ccc", just "aaabbb", but I can't flush "bbb" out because it > waits for the subrequest to be done. > Well, I ran into this problem while working on the "light thread" model [1] for my ngx_lua module months ago. The default Nginx subrequest setup ensures the output order by means of ngx_http_postpone_filter_module and r->postponed. So we need workarounds here. The solution I end up with is to write my own version of the ngx_http_subrequest function in ngx_lua and temporarily switch the current active request (i.e., r->connection->data) to the parent request every time the parent request is trying to output something when there's still pending subrequests running (so as to bypass ngx_http_postpone_filter_module). This works pretty well. You can check out the ngx_lua module's source for more details https://github.com/chaoslawful/lua-nginx-module especially the file src/ngx_http_lua_subrequest.c. Among other things, maybe you want to consider directly using ngx_lua instead of writing your own Nginx C module for this task. Best regards, -agentzh [1] http://wiki.nginx.org/HttpLuaModule#ngx.thread.spawn From roy at adallom.com Sun May 5 05:50:06 2013 From: roy at adallom.com (Roy Reznik) Date: Sun, 5 May 2013 08:50:06 +0300 Subject: [PATCH] Add directive to always gunzip content Message-ID: Use this by adding "gunzip_always on;" to configuration. This was in the modules to-do list. Cheers. @@ -16,6 +16,7 @@ typedef struct { ngx_flag_t enable; ngx_bufs_t bufs; + ngx_flag_t always; } ngx_http_gunzip_conf_t; @@ -77,6 +78,13 @@ NGX_HTTP_LOC_CONF_OFFSET, offsetof(ngx_http_gunzip_conf_t, bufs), NULL }, + + { ngx_string("gunzip_always"), + NGX_HTTP_MAIN_CONF|NGX_HTTP_SRV_CONF|NGX_HTTP_LOC_CONF|NGX_CONF_FLAG, + ngx_conf_set_flag_slot, + NGX_HTTP_LOC_CONF_OFFSET, + offsetof(ngx_http_gunzip_conf_t, always), + NULL }, ngx_null_command }; @@ -126,7 +134,7 @@ conf = ngx_http_get_module_loc_conf(r, ngx_http_gunzip_filter_module); /* TODO support multiple content-codings */ - /* TODO always gunzip - due to configuration or module request */ + /* TODO always gunzip - due to module request */ /* TODO ignore content encoding? */ if (!conf->enable @@ -140,13 +148,16 @@ r->gzip_vary = 1; - if (!r->gzip_tested) { - if (ngx_http_gzip_ok(r) == NGX_OK) { + /* if configured to gunzip always, don't check request headers */ + if (!conf->always) { + if (!r->gzip_tested) { + if (ngx_http_gzip_ok(r) == NGX_OK) { + return ngx_http_next_header_filter(r); + } + + } else if (r->gzip_ok) { return ngx_http_next_header_filter(r); } - - } else if (r->gzip_ok) { - return ngx_http_next_header_filter(r); } ctx = ngx_pcalloc(r->pool, sizeof(ngx_http_gunzip_ctx_t)); @@ -644,6 +655,7 @@ */ conf->enable = NGX_CONF_UNSET; + conf->always = NGX_CONF_UNSET; return conf; } @@ -656,6 +668,7 @@ ngx_http_gunzip_conf_t *conf = child; ngx_conf_merge_value(conf->enable, prev->enable, 0); + ngx_conf_merge_value(conf->always, prev->always, 0); ngx_conf_merge_bufs_value(conf->bufs, prev->bufs, (128 * 1024) / ngx_pagesize, ngx_pagesize); -- *Roy Reznik, VP R&D* Adallom, 1 Ha'barzel st., Tel-Aviv, Israel Mobile: +972 (54) 4524066 *roy at adallom.com, www.adallom.com* -------------- next part -------------- An HTML attachment was scrubbed... URL: From dakota at brokenpipe.ru Sun May 5 21:08:09 2013 From: dakota at brokenpipe.ru (Marat Dakota) Date: Mon, 6 May 2013 01:08:09 +0400 Subject: Reorder output chain when subrequests are used In-Reply-To: References: <20130503220343.GC69760@mdounin.ru> Message-ID: That's helped. Although, I'm not quite sure that I completely understand what I've done. But it is working as expected now. Thank you! -- Marat On Sat, May 4, 2013 at 10:42 PM, agentzh wrote: > Hello! > > On Fri, May 3, 2013 at 3:28 PM, Marat Dakota wrote: > > Ok, I need improper way then. Just some way. > > > > The problem is that I know "bbb" only after the subrequest is made. And > > subrequest's body is being completely filtered by body filter. So, in > fact I > > will not have "ccc", just "aaabbb", but I can't flush "bbb" out because > it > > waits for the subrequest to be done. > > > > Well, I ran into this problem while working on the "light thread" > model [1] for my ngx_lua module months ago. > > The default Nginx subrequest setup ensures the output order by means > of ngx_http_postpone_filter_module and r->postponed. So we need > workarounds here. > > The solution I end up with is to write my own version of the > ngx_http_subrequest function in ngx_lua and temporarily switch the > current active request (i.e., r->connection->data) to the parent > request every time the parent request is trying to output something > when there's still pending subrequests running (so as to bypass > ngx_http_postpone_filter_module). This works pretty well. > > You can check out the ngx_lua module's source for more details > > https://github.com/chaoslawful/lua-nginx-module > > especially the file src/ngx_http_lua_subrequest.c. > > Among other things, maybe you want to consider directly using ngx_lua > instead of writing your own Nginx C module for this task. > > Best regards, > -agentzh > > [1] http://wiki.nginx.org/HttpLuaModule#ngx.thread.spawn > > _______________________________________________ > nginx-devel mailing list > nginx-devel at nginx.org > http://mailman.nginx.org/mailman/listinfo/nginx-devel > -------------- next part -------------- An HTML attachment was scrubbed... URL: From maxim at nginx.com Mon May 6 14:37:03 2013 From: maxim at nginx.com (Maxim Konovalov) Date: Mon, 06 May 2013 18:37:03 +0400 Subject: [PATCH] PCRE: retain input pattern for all regular expressions In-Reply-To: References: Message-ID: <5187C00F.3070109@nginx.com> Hi Piotr, On 5/2/13 2:32 PM, Piotr Sikora wrote: > # HG changeset patch > # User Piotr Sikora > # Date 1367490396 25200 > # Node ID c6434a8634386be8b9bfb90be47c4902a009b36b > # Parent 886800caf36081e1fcbecd0483abeefbafc781b3 > PCRE: retain input pattern for all regular expressions. > [...] I asked Valentin to review your patch. He is on vacation currently so it takes some time. Thanks for the patch! -- Maxim Konovalov +7 (910) 4293178 http://nginx.com/services.html From ykirpichev at gmail.com Tue May 7 10:49:07 2013 From: ykirpichev at gmail.com (Yury Kirpichev) Date: Tue, 7 May 2013 14:49:07 +0400 Subject: TCP_CORK option is never turned off Message-ID: Hi, I've got an issue when trying to use SPDY over HTTP for nginx. I'm using nginx 1.4.0. The issue is that there is 200ms delay between request is sent to server and response with data is received. After analysing this issue a little bit, it turned out that TCP_CORK option is set before sendfile is called and never reset afterwards. 9910 0.000165 writev(3, [{"\200\2\0\4\1\0\0\f\0\0\0\1\4\0\0\1\0\0\0d", 20}, {"\200\2\0\2\0\0\0\370\0\0\0\1\0\00080\337\242Q\262\0\342\0\35\377\0\t\0\7ver"..., 256}], 2) = 276 9910 0.000121 setsockopt(3, SOL_TCP, TCP_CORK, [1], 4) = 0 9910 0.000040 writev(3, [{"\0\0\0\1\1\0\0L", 8}], 1) = 8 9910 0.000052 sendfile(3, 14, [0], 76) = 76 9910 0.000054 write(5, "84.201.173.232 - - [07/May/2013:"..., 203) = 203 9910 0.000043 close(14) = 0 9910 0.000162 recvfrom(3, 0x7fddc6670010, 262112, 0, 0, 0) = -1 EAGAIN (Resource temporarily unavailable) 9910 0.000253 epoll_wait(12, {{EPOLLIN, {u32=3328906192, u64=140590493406160}}}, 512, 180000) = 1 9910 0.218032 recvfrom(3, "\200\2\0\1\1\0\0u\0\0\0\3\0\0\0\0\200\0B\212bf`d\r\252h\"*6\0041"..., 262112, 0, NULL, NULL) = 125 9910 0.000391 open("/etc/nginx/html/favicon.ico", O_RDONLY|O_NONBLOCK) = 14 9910 0.000353 fstat(14, {st_mode=S_IFREG|0644, st_size=0, ...}) = 0 9910 0.000273 writev(3, [{"\200\2\0\2\1\0\0\256\0\0\0\3\0\0\0\236\0a\377\0\6\0\7version\0\10"..., 182}], 1) = 182 9910 0.000267 write(5, "84.201.173.232 - - [07/May/2013:"..., 213) = 213 9910 0.000300 close(14) = 0 I tried to disable tcp_nopush using config file, but result is always the same. Could you please help me to resolve this issue? As temporary solution I just commented out the following code in my local source: int ngx_tcp_nopush(ngx_socket_t s) { #if 0 int cork; cork = 1; return setsockopt(s, IPPROTO_TCP, TCP_CORK, (const void *) &cork, sizeof(int)); #else return 0; #endif } And issue disappeared after that. However, I need a permanent solution for this problem. BR/ Yury -------------- next part -------------- An HTML attachment was scrubbed... URL: From vbart at nginx.com Tue May 7 14:05:06 2013 From: vbart at nginx.com (Valentin V. Bartenev) Date: Tue, 7 May 2013 18:05:06 +0400 Subject: TCP_CORK option is never turned off In-Reply-To: References: Message-ID: <201305071805.06714.vbart@nginx.com> On Tuesday 07 May 2013 14:49:07 Yury Kirpichev wrote: > Hi, > > I've got an issue when trying to use SPDY over HTTP for nginx. > > I'm using nginx 1.4.0. > > The issue is that there is 200ms delay between request is sent to server > and response with data is received. > > > After analysing this issue a little bit, it turned out that TCP_CORK option > is set before sendfile is called and never reset afterwards. > > 9910 0.000165 writev(3, > [{"\200\2\0\4\1\0\0\f\0\0\0\1\4\0\0\1\0\0\0d", 20}, > {"\200\2\0\2\0\0\0\370\0\0\0\1\0\00080\337\242Q\262\0\342\0\35\377\0\t\0\7v > er"..., 256}], 2) = 276 > 9910 0.000121 setsockopt(3, SOL_TCP, TCP_CORK, [1], 4) = 0 > 9910 0.000040 writev(3, [{"\0\0\0\1\1\0\0L", 8}], 1) = 8 > 9910 0.000052 sendfile(3, 14, [0], 76) = 76 > 9910 0.000054 write(5, "84.201.173.232 - - [07/May/2013:"..., 203) = > 203 > 9910 0.000043 close(14) = 0 > 9910 0.000162 recvfrom(3, 0x7fddc6670010, 262112, 0, 0, 0) = -1 > EAGAIN (Resource temporarily unavailable) > 9910 0.000253 epoll_wait(12, {{EPOLLIN, {u32=3328906192, > u64=140590493406160}}}, 512, 180000) = 1 > 9910 0.218032 recvfrom(3, > "\200\2\0\1\1\0\0u\0\0\0\3\0\0\0\0\200\0B\212bf`d\r\252h\"*6\0041"..., > 262112, 0, NULL, NULL) = 125 > 9910 0.000391 open("/etc/nginx/html/favicon.ico", > O_RDONLY|O_NONBLOCK) = 14 > 9910 0.000353 fstat(14, {st_mode=S_IFREG|0644, st_size=0, ...}) = 0 > 9910 0.000273 writev(3, > [{"\200\2\0\2\1\0\0\256\0\0\0\3\0\0\0\236\0a\377\0\6\0\7version\0\10"..., > 182}], 1) = 182 > 9910 0.000267 write(5, "84.201.173.232 - - [07/May/2013:"..., 213) = > 213 > 9910 0.000300 close(14) = 0 > > I tried to disable tcp_nopush using config file, but result is always the > same. > > Could you please help me to resolve this issue? Thank you for the report. I will look for the right solution. wbr, Valentin V. Bartenev > > As temporary solution I just commented out the following code in my local > source: > int > ngx_tcp_nopush(ngx_socket_t s) > { > #if 0 > int cork; > > cork = 1; > > return setsockopt(s, IPPROTO_TCP, TCP_CORK, > (const void *) &cork, sizeof(int)); > #else > return 0; > #endif > } > And issue disappeared after that. > However, I need a permanent solution for this problem. > > BR/ Yury From vbart at nginx.com Wed May 8 01:08:17 2013 From: vbart at nginx.com (Valentin V. Bartenev) Date: Wed, 8 May 2013 05:08:17 +0400 Subject: [PATCH] PCRE: retain input pattern for all regular expressions In-Reply-To: References: Message-ID: <201305080508.17422.vbart@nginx.com> On Thursday 02 May 2013 14:32:51 Piotr Sikora wrote: > # HG changeset patch > # User Piotr Sikora > # Date 1367490396 25200 > # Node ID c6434a8634386be8b9bfb90be47c4902a009b36b > # Parent 886800caf36081e1fcbecd0483abeefbafc781b3 > PCRE: retain input pattern for all regular expressions. > > Previously, input pattern was kept only for regular expressions > with named captures, which resulted in error log entries without > input pattern for PCRE errors that occured while processing > regular expressions without them. > > Signed-off-by: Piotr Sikora > Nice catch, thank you. I will push it after Maxim Dounin's acknowledgment. wbr, Valentin V. Bartenev > diff -r 886800caf360 -r c6434a863438 src/http/ngx_http_variables.c > --- a/src/http/ngx_http_variables.c Mon Apr 29 18:58:58 2013 +0400 > +++ b/src/http/ngx_http_variables.c Thu May 02 03:26:36 2013 -0700 > @@ -2257,6 +2257,7 @@ > > re->regex = rc->regex; > re->ncaptures = rc->captures; > + re->name = rc->pattern; > > cmcf = ngx_http_conf_get_module_main_conf(cf, ngx_http_core_module); > cmcf->ncaptures = ngx_max(cmcf->ncaptures, re->ncaptures); > @@ -2274,7 +2275,6 @@ > > re->variables = rv; > re->nvariables = n; > - re->name = rc->pattern; > > size = rc->name_size; From vbart at nginx.com Wed May 8 14:22:02 2013 From: vbart at nginx.com (Valentin V. Bartenev) Date: Wed, 8 May 2013 18:22:02 +0400 Subject: [PATCH] ngx_mail_http_auth_module In-Reply-To: References: Message-ID: <201305081822.02337.vbart@nginx.com> On Thursday 02 May 2013 18:01:29 Filipe Da Silva wrote: > Hello, Nginx developers, > > I been working recently on NGinx, as part of my job . > > And I see some small issues as this one : > > This call to *ngx_close_connection* is useless as it was already made 18 > lines before in the code . > Same code is present in 1.2 > > Regards, > Filipe > > ---- > > diff -r 8222ca034980 src/mail/ngx_mail_auth_http_module.c > --- a/src/mail/ngx_mail_auth_http_module.c Tue Apr 02 12:34:39 2013 > +0000 +++ b/src/mail/ngx_mail_auth_http_module.c Thu May 02 15:46:42 > 2013 +0200 @@ -696,7 +696,6 @@ > > p = ngx_pnalloc(s->connection->pool, ctx->err.len); > if (p == NULL) { > - ngx_close_connection(ctx->peer.connection); > ngx_destroy_pool(ctx->pool); > ngx_mail_session_internal_server_error(s); > return; The patch looks good for me. It would be nice if you provide it as a mercurial changeset for pushing. wbr, Valentin V. Bartenev From fdasilvayy at gmail.com Fri May 10 14:39:35 2013 From: fdasilvayy at gmail.com (Filipe Da Silva) Date: Fri, 10 May 2013 16:39:35 +0200 Subject: [PATCH] ngx_mail_http_auth_module In-Reply-To: <201305081822.02337.vbart@nginx.com> References: <201305081822.02337.vbart@nginx.com> Message-ID: Hi, Please find attach this patch as a mercurial changeset, It's based on the current trunk version. Regards, Filipe 2013/5/8 Valentin V. Bartenev > On Thursday 02 May 2013 18:01:29 Filipe Da Silva wrote: > > Hello, Nginx developers, > > > > I been working recently on NGinx, as part of my job . > > > > And I see some small issues as this one : > > > > This call to *ngx_close_connection* is useless as it was already made 18 > > lines before in the code . > > Same code is present in 1.2 > > > > Regards, > > Filipe > > > > ---- > > > > diff -r 8222ca034980 src/mail/ngx_mail_auth_http_module.c > > --- a/src/mail/ngx_mail_auth_http_module.c Tue Apr 02 12:34:39 2013 > > +0000 +++ b/src/mail/ngx_mail_auth_http_module.c Thu May 02 15:46:42 > > 2013 +0200 @@ -696,7 +696,6 @@ > > > > p = ngx_pnalloc(s->connection->pool, ctx->err.len); > > if (p == NULL) { > > - ngx_close_connection(ctx->peer.connection); > > ngx_destroy_pool(ctx->pool); > > ngx_mail_session_internal_server_error(s); > > return; > > The patch looks good for me. It would be nice if you provide it as > a mercurial changeset for pushing. > > wbr, Valentin V. Bartenev > -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: 5198.patch Type: application/octet-stream Size: 804 bytes Desc: not available URL: From mdounin at mdounin.ru Sat May 11 17:28:50 2013 From: mdounin at mdounin.ru (Maxim Dounin) Date: Sat, 11 May 2013 21:28:50 +0400 Subject: [BUG] Wrong Content-Length shared between main request and subrequests in http proxy module In-Reply-To: References: Message-ID: <20130511172850.GQ69760@mdounin.ru> Hello! On Fri, May 03, 2013 at 11:20:46AM +0800, lanshun zhou wrote: > Nginx uses a variable "$proxy_internal_body_length" for the > "Content-Length" header in http proxy module, which is cacheable and shared > between the main request and all subrequests by default. > Although ctx->internal_body_length is calculated for each > request/subrequest, the string format is cached after the first generation, > and this may be wrong for other subrequests. > > The following simple config can reproduce the problem, where xxxxx is a > place that can receive post data. curl --data-binary "xxxxxxxxxxxxxxx" > localhost/test.html will cause the client hang. > > location = /test.html { > add_after_body /vpost; > > proxy_pass http://xxxxx; > } > > location /vpost { > proxy_set_body "in vpost"; > > proxy_pass http://xxxxx; > } Patch committed, thanks. Added tests, too. http://hg.nginx.org/nginx/rev/bfb99b75833b http://hg.nginx.org/nginx-tests/rev/36d24870ccb2 -- Maxim Dounin http://nginx.org/en/donation.html From mdounin at mdounin.ru Sat May 11 17:30:39 2013 From: mdounin at mdounin.ru (Maxim Dounin) Date: Sat, 11 May 2013 21:30:39 +0400 Subject: [PATCH] PCRE: retain input pattern for all regular expressions In-Reply-To: <201305080508.17422.vbart@nginx.com> References: <201305080508.17422.vbart@nginx.com> Message-ID: <20130511173039.GR69760@mdounin.ru> Hello! On Wed, May 08, 2013 at 05:08:17AM +0400, Valentin V. Bartenev wrote: > On Thursday 02 May 2013 14:32:51 Piotr Sikora wrote: > > # HG changeset patch > > # User Piotr Sikora > > # Date 1367490396 25200 > > # Node ID c6434a8634386be8b9bfb90be47c4902a009b36b > > # Parent 886800caf36081e1fcbecd0483abeefbafc781b3 > > PCRE: retain input pattern for all regular expressions. > > > > Previously, input pattern was kept only for regular expressions > > with named captures, which resulted in error log entries without > > input pattern for PCRE errors that occured while processing > > regular expressions without them. > > > > Signed-off-by: Piotr Sikora > > > > Nice catch, thank you. I will push it after Maxim Dounin's acknowledgment. Looks fine, please push. -- Maxim Dounin http://nginx.org/en/donation.html From breno.silva at gmail.com Sat May 11 19:23:00 2013 From: breno.silva at gmail.com (Breno Silva) Date: Sat, 11 May 2013 16:23:00 -0300 Subject: Nginx eats 100% cpu in ngx_event_pipe_write_to_downstream Message-ID: Hello list, We are porting ModSecurity to NGINX. However we are seeing sometimes an issue. Nginx eats 100% of cpu and when i use gdb i see: gdb -p 8645 ngx_event_pipe_write_to_downstream (p=0x9bbc720, do_write=0) at src/event/ngx_event_pipe.c:551 551 if (cl->buf->recycled) { (gdb) Looks like it is happening when we call ngx_http_modsecurity_body_filter() then go to this conditions; rc = move_chain_to_brigade(in, ctx->brigade, r->pool, 0); if (rc != NGX_OK) { r->buffered |= NGX_HTTP_SSI_BUFFERED; return rc; } move_chainto_brigade is defined as: ngx_int_t move_chain_to_brigade(ngx_chain_t *chain, apr_bucket_brigade *bb, ngx_pool_t *pool, ngx_int_t last_buf) { apr_bucket *e; ngx_chain_t *cl; while (chain) { e = ngx_buf_to_apr_bucket(chain->buf, bb->p, bb->bucket_alloc); if (e == NULL) { return NGX_ERROR; } APR_BRIGADE_INSERT_TAIL(bb, e); if (chain->buf->last_buf) { e = apr_bucket_eos_create(bb->bucket_alloc); APR_BRIGADE_INSERT_TAIL(bb, e); chain->buf->last_buf = 0; return NGX_OK; } cl = chain; chain = chain->next; ngx_free_chain(pool, cl); } if (last_buf) { e = apr_bucket_eos_create(bb->bucket_alloc); APR_BRIGADE_INSERT_TAIL(bb, e); return NGX_OK; } return NGX_AGAIN; } Let me know if you guys can help us understanding why sometimes we trigger this issue Thanks Breno -------------- next part -------------- An HTML attachment was scrubbed... URL: From ne at vbart.ru Sat May 11 22:16:55 2013 From: ne at vbart.ru (Valentin V. Bartenev) Date: Sun, 12 May 2013 02:16:55 +0400 Subject: [PATCH] PCRE: retain input pattern for all regular expressions In-Reply-To: <20130511173039.GR69760@mdounin.ru> References: <201305080508.17422.vbart@nginx.com> <20130511173039.GR69760@mdounin.ru> Message-ID: <201305120216.55582.ne@vbart.ru> On Saturday 11 May 2013 21:30:39 Maxim Dounin wrote: > Hello! > > On Wed, May 08, 2013 at 05:08:17AM +0400, Valentin V. Bartenev wrote: > > On Thursday 02 May 2013 14:32:51 Piotr Sikora wrote: > > > # HG changeset patch > > > # User Piotr Sikora > > > # Date 1367490396 25200 > > > # Node ID c6434a8634386be8b9bfb90be47c4902a009b36b > > > # Parent 886800caf36081e1fcbecd0483abeefbafc781b3 > > > PCRE: retain input pattern for all regular expressions. > > > > > > Previously, input pattern was kept only for regular expressions > > > with named captures, which resulted in error log entries without > > > input pattern for PCRE errors that occured while processing > > > regular expressions without them. > > > > > > Signed-off-by: Piotr Sikora > > > > Nice catch, thank you. I will push it after Maxim Dounin's > > acknowledgment. > > Looks fine, please push. Done. http://hg.nginx.org/nginx/rev/a64c8a5da336 wbr, Valentin V. Bartenev From mdounin at mdounin.ru Sat May 11 22:56:34 2013 From: mdounin at mdounin.ru (Maxim Dounin) Date: Sun, 12 May 2013 02:56:34 +0400 Subject: Nginx eats 100% cpu in ngx_event_pipe_write_to_downstream In-Reply-To: References: Message-ID: <20130511225634.GS69760@mdounin.ru> Hello! On Sat, May 11, 2013 at 04:23:00PM -0300, Breno Silva wrote: > Hello list, > > We are porting ModSecurity to NGINX. However we are seeing sometimes an > issue. Nginx eats 100% of cpu and when i use gdb i see: > > gdb -p 8645 > > ngx_event_pipe_write_to_downstream (p=0x9bbc720, do_write=0) at > src/event/ngx_event_pipe.c:551 > > 551 if (cl->buf->recycled) { > (gdb) > > Looks like it is happening when we call ngx_http_modsecurity_body_filter() > then go to this conditions; > > rc = move_chain_to_brigade(in, ctx->brigade, r->pool, 0); > > if (rc != NGX_OK) { > > r->buffered |= NGX_HTTP_SSI_BUFFERED; > > return rc; > > } > > move_chainto_brigade is defined as: > > > ngx_int_t > > move_chain_to_brigade(ngx_chain_t *chain, apr_bucket_brigade *bb, > ngx_pool_t *pool, ngx_int_t last_buf) { > > apr_bucket *e; > > ngx_chain_t *cl; > > > while (chain) { > > e = ngx_buf_to_apr_bucket(chain->buf, bb->p, bb->bucket_alloc); > > if (e == NULL) { > > return NGX_ERROR; > > } > > > APR_BRIGADE_INSERT_TAIL(bb, e); > > if (chain->buf->last_buf) { > > e = apr_bucket_eos_create(bb->bucket_alloc); > > APR_BRIGADE_INSERT_TAIL(bb, e); > > chain->buf->last_buf = 0; > > return NGX_OK; > > } > > cl = chain; > > chain = chain->next; > > ngx_free_chain(pool, cl); > > } > > > if (last_buf) { > > e = apr_bucket_eos_create(bb->bucket_alloc); > > APR_BRIGADE_INSERT_TAIL(bb, e); > > return NGX_OK; > > } > > return NGX_AGAIN; > > } > Let me know if you guys can help us understanding why sometimes we trigger > this issue It looks like your code modify chain links (ngx_chain_t structures) as got by your response body filter. This is not something filters are allowed to do, and results are undefined. If a filter needs to modify chain, it should allocate it's own chain links. In the code quoted you probably don't want to touch chain links at all. -- Maxim Dounin http://nginx.org/en/donation.html From breno.silva at gmail.com Sat May 11 23:09:59 2013 From: breno.silva at gmail.com (Breno Silva) Date: Sat, 11 May 2013 20:09:59 -0300 Subject: Nginx eats 100% cpu in ngx_event_pipe_write_to_downstream In-Reply-To: <20130511225634.GS69760@mdounin.ru> References: <20130511225634.GS69760@mdounin.ru> Message-ID: Hello Maxim, We are sending chains data into a brigade/bucket structure (APR). It sometimes works fine.. sometimes trigger the cpu issue If i call: rc = move_chain_to_brigade(in, ctx->brigade, r->pool, 1); <--- change from 0 to 1 Then i will get only the first X bytes of the response body. This works great... but i need the entire buffer :) Could you give us more details how we could patch this function to fix the issue ? Thanks On Sat, May 11, 2013 at 7:56 PM, Maxim Dounin wrote: > Hello! > > On Sat, May 11, 2013 at 04:23:00PM -0300, Breno Silva wrote: > > > Hello list, > > > > We are porting ModSecurity to NGINX. However we are seeing sometimes an > > issue. Nginx eats 100% of cpu and when i use gdb i see: > > > > gdb -p 8645 > > > > ngx_event_pipe_write_to_downstream (p=0x9bbc720, do_write=0) at > > src/event/ngx_event_pipe.c:551 > > > > 551 if (cl->buf->recycled) { > > (gdb) > > > > Looks like it is happening when we call > ngx_http_modsecurity_body_filter() > > then go to this conditions; > > > > rc = move_chain_to_brigade(in, ctx->brigade, r->pool, 0); > > > > if (rc != NGX_OK) { > > > > r->buffered |= NGX_HTTP_SSI_BUFFERED; > > > > return rc; > > > > } > > > > move_chainto_brigade is defined as: > > > > > > ngx_int_t > > > > move_chain_to_brigade(ngx_chain_t *chain, apr_bucket_brigade *bb, > > ngx_pool_t *pool, ngx_int_t last_buf) { > > > > apr_bucket *e; > > > > ngx_chain_t *cl; > > > > > > while (chain) { > > > > e = ngx_buf_to_apr_bucket(chain->buf, bb->p, bb->bucket_alloc); > > > > if (e == NULL) { > > > > return NGX_ERROR; > > > > } > > > > > > APR_BRIGADE_INSERT_TAIL(bb, e); > > > > if (chain->buf->last_buf) { > > > > e = apr_bucket_eos_create(bb->bucket_alloc); > > > > APR_BRIGADE_INSERT_TAIL(bb, e); > > > > chain->buf->last_buf = 0; > > > > return NGX_OK; > > > > } > > > > cl = chain; > > > > chain = chain->next; > > > > ngx_free_chain(pool, cl); > > > > } > > > > > > if (last_buf) { > > > > e = apr_bucket_eos_create(bb->bucket_alloc); > > > > APR_BRIGADE_INSERT_TAIL(bb, e); > > > > return NGX_OK; > > > > } > > > > return NGX_AGAIN; > > > > } > > Let me know if you guys can help us understanding why sometimes we > trigger > > this issue > > It looks like your code modify chain links (ngx_chain_t > structures) as got by your response body filter. This is not > something filters are allowed to do, and results are undefined. > If a filter needs to modify chain, it should allocate it's own > chain links. > > In the code quoted you probably don't want to touch chain links at > all. > > -- > Maxim Dounin > http://nginx.org/en/donation.html > > _______________________________________________ > nginx-devel mailing list > nginx-devel at nginx.org > http://mailman.nginx.org/mailman/listinfo/nginx-devel > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mdounin at mdounin.ru Sun May 12 00:06:40 2013 From: mdounin at mdounin.ru (Maxim Dounin) Date: Sun, 12 May 2013 04:06:40 +0400 Subject: Nginx eats 100% cpu in ngx_event_pipe_write_to_downstream In-Reply-To: References: <20130511225634.GS69760@mdounin.ru> Message-ID: <20130512000640.GU69760@mdounin.ru> Hello! On Sat, May 11, 2013 at 08:09:59PM -0300, Breno Silva wrote: > Hello Maxim, > > We are sending chains data into a brigade/bucket structure (APR). It > sometimes works fine.. sometimes trigger the cpu issue > > If i call: > rc = move_chain_to_brigade(in, ctx->brigade, r->pool, 1); <--- change > from 0 to 1 > Then i will get only the first X bytes of the response body. This works > great... but i need the entire buffer :) > > Could you give us more details how we could patch this function to fix the > issue ? Remove the ngx_free_chain(pool, cl); call from the move_chain_to_brigade() function, it should fix you CPU hog issues. > > Thanks > > > On Sat, May 11, 2013 at 7:56 PM, Maxim Dounin wrote: > > > Hello! > > > > On Sat, May 11, 2013 at 04:23:00PM -0300, Breno Silva wrote: > > > > > Hello list, > > > > > > We are porting ModSecurity to NGINX. However we are seeing sometimes an > > > issue. Nginx eats 100% of cpu and when i use gdb i see: > > > > > > gdb -p 8645 > > > > > > ngx_event_pipe_write_to_downstream (p=0x9bbc720, do_write=0) at > > > src/event/ngx_event_pipe.c:551 > > > > > > 551 if (cl->buf->recycled) { > > > (gdb) > > > > > > Looks like it is happening when we call > > ngx_http_modsecurity_body_filter() > > > then go to this conditions; > > > > > > rc = move_chain_to_brigade(in, ctx->brigade, r->pool, 0); > > > > > > if (rc != NGX_OK) { > > > > > > r->buffered |= NGX_HTTP_SSI_BUFFERED; > > > > > > return rc; > > > > > > } > > > > > > move_chainto_brigade is defined as: > > > > > > > > > ngx_int_t > > > > > > move_chain_to_brigade(ngx_chain_t *chain, apr_bucket_brigade *bb, > > > ngx_pool_t *pool, ngx_int_t last_buf) { > > > > > > apr_bucket *e; > > > > > > ngx_chain_t *cl; > > > > > > > > > while (chain) { > > > > > > e = ngx_buf_to_apr_bucket(chain->buf, bb->p, bb->bucket_alloc); > > > > > > if (e == NULL) { > > > > > > return NGX_ERROR; > > > > > > } > > > > > > > > > APR_BRIGADE_INSERT_TAIL(bb, e); > > > > > > if (chain->buf->last_buf) { > > > > > > e = apr_bucket_eos_create(bb->bucket_alloc); > > > > > > APR_BRIGADE_INSERT_TAIL(bb, e); > > > > > > chain->buf->last_buf = 0; > > > > > > return NGX_OK; > > > > > > } > > > > > > cl = chain; > > > > > > chain = chain->next; > > > > > > ngx_free_chain(pool, cl); > > > > > > } > > > > > > > > > if (last_buf) { > > > > > > e = apr_bucket_eos_create(bb->bucket_alloc); > > > > > > APR_BRIGADE_INSERT_TAIL(bb, e); > > > > > > return NGX_OK; > > > > > > } > > > > > > return NGX_AGAIN; > > > > > > } > > > Let me know if you guys can help us understanding why sometimes we > > trigger > > > this issue > > > > It looks like your code modify chain links (ngx_chain_t > > structures) as got by your response body filter. This is not > > something filters are allowed to do, and results are undefined. > > If a filter needs to modify chain, it should allocate it's own > > chain links. > > > > In the code quoted you probably don't want to touch chain links at > > all. > > > > -- > > Maxim Dounin > > http://nginx.org/en/donation.html > > > > _______________________________________________ > > nginx-devel mailing list > > nginx-devel at nginx.org > > http://mailman.nginx.org/mailman/listinfo/nginx-devel > > > _______________________________________________ > nginx-devel mailing list > nginx-devel at nginx.org > http://mailman.nginx.org/mailman/listinfo/nginx-devel -- Maxim Dounin http://nginx.org/en/donation.html From vadim.lazovskiy at gmail.com Sun May 12 09:37:18 2013 From: vadim.lazovskiy at gmail.com (Vadim Lazovskiy) Date: Sun, 12 May 2013 13:37:18 +0400 Subject: [PATCH] PCRE: retain input pattern for all regular expressions In-Reply-To: <201305120216.55582.ne@vbart.ru> References: <201305080508.17422.vbart@nginx.com> <20130511173039.GR69760@mdounin.ru> <201305120216.55582.ne@vbart.ru> Message-ID: Hello. It is a pity that commits are no longer sent to the mailing list. 2013/5/12 Valentin V. Bartenev > On Saturday 11 May 2013 21:30:39 Maxim Dounin wrote: > > Hello! > > > > On Wed, May 08, 2013 at 05:08:17AM +0400, Valentin V. Bartenev wrote: > > > On Thursday 02 May 2013 14:32:51 Piotr Sikora wrote: > > > > # HG changeset patch > > > > # User Piotr Sikora > > > > # Date 1367490396 25200 > > > > # Node ID c6434a8634386be8b9bfb90be47c4902a009b36b > > > > # Parent 886800caf36081e1fcbecd0483abeefbafc781b3 > > > > PCRE: retain input pattern for all regular expressions. > > > > > > > > Previously, input pattern was kept only for regular expressions > > > > with named captures, which resulted in error log entries without > > > > input pattern for PCRE errors that occured while processing > > > > regular expressions without them. > > > > > > > > Signed-off-by: Piotr Sikora > > > > > > Nice catch, thank you. I will push it after Maxim Dounin's > > > acknowledgment. > > > > Looks fine, please push. > > Done. > > http://hg.nginx.org/nginx/rev/a64c8a5da336 > > > wbr, Valentin V. Bartenev > > _______________________________________________ > nginx-devel mailing list > nginx-devel at nginx.org > http://mailman.nginx.org/mailman/listinfo/nginx-devel > -- Best Regards, Vadim Lazovskiy -------------- next part -------------- An HTML attachment was scrubbed... URL: From breno.silva at gmail.com Sun May 12 14:09:26 2013 From: breno.silva at gmail.com (Breno Silva) Date: Sun, 12 May 2013 11:09:26 -0300 Subject: Nginx eats 100% cpu in ngx_event_pipe_write_to_downstream In-Reply-To: <20130512000640.GU69760@mdounin.ru> References: <20130511225634.GS69760@mdounin.ru> <20130512000640.GU69760@mdounin.ru> Message-ID: Hello, Yes removing he ngx_free_chain fix the cpu issue. Looks like ther modules needs the chain. I detected another issue.. sometimes when loading big response bodies (like images.. +32K) it does not load it entirely or does it slowly. Do you have any idea ? Thanks Breno On Sat, May 11, 2013 at 9:06 PM, Maxim Dounin wrote: > Hello! > > On Sat, May 11, 2013 at 08:09:59PM -0300, Breno Silva wrote: > > > Hello Maxim, > > > > We are sending chains data into a brigade/bucket structure (APR). It > > sometimes works fine.. sometimes trigger the cpu issue > > > > If i call: > > rc = move_chain_to_brigade(in, ctx->brigade, r->pool, 1); <--- change > > from 0 to 1 > > Then i will get only the first X bytes of the response body. This works > > great... but i need the entire buffer :) > > > > Could you give us more details how we could patch this function to fix > the > > issue ? > > Remove the > > ngx_free_chain(pool, cl); > > call from the move_chain_to_brigade() function, it should fix you > CPU hog issues. > > > > > > Thanks > > > > > > On Sat, May 11, 2013 at 7:56 PM, Maxim Dounin > wrote: > > > > > Hello! > > > > > > On Sat, May 11, 2013 at 04:23:00PM -0300, Breno Silva wrote: > > > > > > > Hello list, > > > > > > > > We are porting ModSecurity to NGINX. However we are seeing sometimes > an > > > > issue. Nginx eats 100% of cpu and when i use gdb i see: > > > > > > > > gdb -p 8645 > > > > > > > > ngx_event_pipe_write_to_downstream (p=0x9bbc720, do_write=0) at > > > > src/event/ngx_event_pipe.c:551 > > > > > > > > 551 if (cl->buf->recycled) { > > > > (gdb) > > > > > > > > Looks like it is happening when we call > > > ngx_http_modsecurity_body_filter() > > > > then go to this conditions; > > > > > > > > rc = move_chain_to_brigade(in, ctx->brigade, r->pool, 0); > > > > > > > > if (rc != NGX_OK) { > > > > > > > > r->buffered |= NGX_HTTP_SSI_BUFFERED; > > > > > > > > return rc; > > > > > > > > } > > > > > > > > move_chainto_brigade is defined as: > > > > > > > > > > > > ngx_int_t > > > > > > > > move_chain_to_brigade(ngx_chain_t *chain, apr_bucket_brigade *bb, > > > > ngx_pool_t *pool, ngx_int_t last_buf) { > > > > > > > > apr_bucket *e; > > > > > > > > ngx_chain_t *cl; > > > > > > > > > > > > while (chain) { > > > > > > > > e = ngx_buf_to_apr_bucket(chain->buf, bb->p, > bb->bucket_alloc); > > > > > > > > if (e == NULL) { > > > > > > > > return NGX_ERROR; > > > > > > > > } > > > > > > > > > > > > APR_BRIGADE_INSERT_TAIL(bb, e); > > > > > > > > if (chain->buf->last_buf) { > > > > > > > > e = apr_bucket_eos_create(bb->bucket_alloc); > > > > > > > > APR_BRIGADE_INSERT_TAIL(bb, e); > > > > > > > > chain->buf->last_buf = 0; > > > > > > > > return NGX_OK; > > > > > > > > } > > > > > > > > cl = chain; > > > > > > > > chain = chain->next; > > > > > > > > ngx_free_chain(pool, cl); > > > > > > > > } > > > > > > > > > > > > if (last_buf) { > > > > > > > > e = apr_bucket_eos_create(bb->bucket_alloc); > > > > > > > > APR_BRIGADE_INSERT_TAIL(bb, e); > > > > > > > > return NGX_OK; > > > > > > > > } > > > > > > > > return NGX_AGAIN; > > > > > > > > } > > > > Let me know if you guys can help us understanding why sometimes we > > > trigger > > > > this issue > > > > > > It looks like your code modify chain links (ngx_chain_t > > > structures) as got by your response body filter. This is not > > > something filters are allowed to do, and results are undefined. > > > If a filter needs to modify chain, it should allocate it's own > > > chain links. > > > > > > In the code quoted you probably don't want to touch chain links at > > > all. > > > > > > -- > > > Maxim Dounin > > > http://nginx.org/en/donation.html > > > > > > _______________________________________________ > > > nginx-devel mailing list > > > nginx-devel at nginx.org > > > http://mailman.nginx.org/mailman/listinfo/nginx-devel > > > > > > _______________________________________________ > > nginx-devel mailing list > > nginx-devel at nginx.org > > http://mailman.nginx.org/mailman/listinfo/nginx-devel > > > -- > Maxim Dounin > http://nginx.org/en/donation.html > > _______________________________________________ > nginx-devel mailing list > nginx-devel at nginx.org > http://mailman.nginx.org/mailman/listinfo/nginx-devel > -------------- next part -------------- An HTML attachment was scrubbed... URL: From breno.silva at gmail.com Sun May 12 20:47:08 2013 From: breno.silva at gmail.com (Breno Silva) Date: Sun, 12 May 2013 17:47:08 -0300 Subject: Nginx eats 100% cpu in ngx_event_pipe_write_to_downstream In-Reply-To: References: <20130511225634.GS69760@mdounin.ru> <20130512000640.GU69760@mdounin.ru> Message-ID: Looks like we never see the chain->buf->last_buf, so never get the buffer entirely ? Breno On Sun, May 12, 2013 at 11:09 AM, Breno Silva wrote: > Hello, > > Yes removing he ngx_free_chain fix the cpu issue. Looks like ther modules > needs the chain. > I detected another issue.. sometimes when loading big response bodies > (like images.. +32K) it does not load it entirely or does it slowly. > > Do you have any idea ? > Thanks > > Breno > > > On Sat, May 11, 2013 at 9:06 PM, Maxim Dounin wrote: > >> Hello! >> >> On Sat, May 11, 2013 at 08:09:59PM -0300, Breno Silva wrote: >> >> > Hello Maxim, >> > >> > We are sending chains data into a brigade/bucket structure (APR). It >> > sometimes works fine.. sometimes trigger the cpu issue >> > >> > If i call: >> > rc = move_chain_to_brigade(in, ctx->brigade, r->pool, 1); <--- change >> > from 0 to 1 >> > Then i will get only the first X bytes of the response body. This works >> > great... but i need the entire buffer :) >> > >> > Could you give us more details how we could patch this function to fix >> the >> > issue ? >> >> Remove the >> >> ngx_free_chain(pool, cl); >> >> call from the move_chain_to_brigade() function, it should fix you >> CPU hog issues. >> >> >> > >> > Thanks >> > >> > >> > On Sat, May 11, 2013 at 7:56 PM, Maxim Dounin >> wrote: >> > >> > > Hello! >> > > >> > > On Sat, May 11, 2013 at 04:23:00PM -0300, Breno Silva wrote: >> > > >> > > > Hello list, >> > > > >> > > > We are porting ModSecurity to NGINX. However we are seeing >> sometimes an >> > > > issue. Nginx eats 100% of cpu and when i use gdb i see: >> > > > >> > > > gdb -p 8645 >> > > > >> > > > ngx_event_pipe_write_to_downstream (p=0x9bbc720, do_write=0) at >> > > > src/event/ngx_event_pipe.c:551 >> > > > >> > > > 551 if (cl->buf->recycled) { >> > > > (gdb) >> > > > >> > > > Looks like it is happening when we call >> > > ngx_http_modsecurity_body_filter() >> > > > then go to this conditions; >> > > > >> > > > rc = move_chain_to_brigade(in, ctx->brigade, r->pool, 0); >> > > > >> > > > if (rc != NGX_OK) { >> > > > >> > > > r->buffered |= NGX_HTTP_SSI_BUFFERED; >> > > > >> > > > return rc; >> > > > >> > > > } >> > > > >> > > > move_chainto_brigade is defined as: >> > > > >> > > > >> > > > ngx_int_t >> > > > >> > > > move_chain_to_brigade(ngx_chain_t *chain, apr_bucket_brigade *bb, >> > > > ngx_pool_t *pool, ngx_int_t last_buf) { >> > > > >> > > > apr_bucket *e; >> > > > >> > > > ngx_chain_t *cl; >> > > > >> > > > >> > > > while (chain) { >> > > > >> > > > e = ngx_buf_to_apr_bucket(chain->buf, bb->p, >> bb->bucket_alloc); >> > > > >> > > > if (e == NULL) { >> > > > >> > > > return NGX_ERROR; >> > > > >> > > > } >> > > > >> > > > >> > > > APR_BRIGADE_INSERT_TAIL(bb, e); >> > > > >> > > > if (chain->buf->last_buf) { >> > > > >> > > > e = apr_bucket_eos_create(bb->bucket_alloc); >> > > > >> > > > APR_BRIGADE_INSERT_TAIL(bb, e); >> > > > >> > > > chain->buf->last_buf = 0; >> > > > >> > > > return NGX_OK; >> > > > >> > > > } >> > > > >> > > > cl = chain; >> > > > >> > > > chain = chain->next; >> > > > >> > > > ngx_free_chain(pool, cl); >> > > > >> > > > } >> > > > >> > > > >> > > > if (last_buf) { >> > > > >> > > > e = apr_bucket_eos_create(bb->bucket_alloc); >> > > > >> > > > APR_BRIGADE_INSERT_TAIL(bb, e); >> > > > >> > > > return NGX_OK; >> > > > >> > > > } >> > > > >> > > > return NGX_AGAIN; >> > > > >> > > > } >> > > > Let me know if you guys can help us understanding why sometimes we >> > > trigger >> > > > this issue >> > > >> > > It looks like your code modify chain links (ngx_chain_t >> > > structures) as got by your response body filter. This is not >> > > something filters are allowed to do, and results are undefined. >> > > If a filter needs to modify chain, it should allocate it's own >> > > chain links. >> > > >> > > In the code quoted you probably don't want to touch chain links at >> > > all. >> > > >> > > -- >> > > Maxim Dounin >> > > http://nginx.org/en/donation.html >> > > >> > > _______________________________________________ >> > > nginx-devel mailing list >> > > nginx-devel at nginx.org >> > > http://mailman.nginx.org/mailman/listinfo/nginx-devel >> > > >> >> > _______________________________________________ >> > nginx-devel mailing list >> > nginx-devel at nginx.org >> > http://mailman.nginx.org/mailman/listinfo/nginx-devel >> >> >> -- >> Maxim Dounin >> http://nginx.org/en/donation.html >> >> _______________________________________________ >> nginx-devel mailing list >> nginx-devel at nginx.org >> http://mailman.nginx.org/mailman/listinfo/nginx-devel >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mdounin at mdounin.ru Mon May 13 07:01:19 2013 From: mdounin at mdounin.ru (Maxim Dounin) Date: Mon, 13 May 2013 11:01:19 +0400 Subject: Nginx eats 100% cpu in ngx_event_pipe_write_to_downstream In-Reply-To: References: <20130511225634.GS69760@mdounin.ru> <20130512000640.GU69760@mdounin.ru> Message-ID: <20130513070119.GW69760@mdounin.ru> Hello! On Sun, May 12, 2013 at 11:09:26AM -0300, Breno Silva wrote: > Hello, > > Yes removing he ngx_free_chain fix the cpu issue. Looks like ther modules > needs the chain. > I detected another issue.. sometimes when loading big response bodies (like > images.. +32K) it does not load it entirely or does it slowly. > > Do you have any idea ? > Thanks It looks like you don't release buffers you got in your filter, and this results buffers exhaustion (output_buffers, proxy_buffers) and transfer stall. If your code can't work with a data stream and needs full response before it can start it's work, you have to copy needed data from buffers and release buffers got (by setting b->pos = b->last, like if a buffer was actually sent). To test things you may try the following in config: output_buffers 1 1k; Then request a static file larger than 1k. > > Breno > > > On Sat, May 11, 2013 at 9:06 PM, Maxim Dounin wrote: > > > Hello! > > > > On Sat, May 11, 2013 at 08:09:59PM -0300, Breno Silva wrote: > > > > > Hello Maxim, > > > > > > We are sending chains data into a brigade/bucket structure (APR). It > > > sometimes works fine.. sometimes trigger the cpu issue > > > > > > If i call: > > > rc = move_chain_to_brigade(in, ctx->brigade, r->pool, 1); <--- change > > > from 0 to 1 > > > Then i will get only the first X bytes of the response body. This works > > > great... but i need the entire buffer :) > > > > > > Could you give us more details how we could patch this function to fix > > the > > > issue ? > > > > Remove the > > > > ngx_free_chain(pool, cl); > > > > call from the move_chain_to_brigade() function, it should fix you > > CPU hog issues. > > > > > > > > > > Thanks > > > > > > > > > On Sat, May 11, 2013 at 7:56 PM, Maxim Dounin > > wrote: > > > > > > > Hello! > > > > > > > > On Sat, May 11, 2013 at 04:23:00PM -0300, Breno Silva wrote: > > > > > > > > > Hello list, > > > > > > > > > > We are porting ModSecurity to NGINX. However we are seeing sometimes > > an > > > > > issue. Nginx eats 100% of cpu and when i use gdb i see: > > > > > > > > > > gdb -p 8645 > > > > > > > > > > ngx_event_pipe_write_to_downstream (p=0x9bbc720, do_write=0) at > > > > > src/event/ngx_event_pipe.c:551 > > > > > > > > > > 551 if (cl->buf->recycled) { > > > > > (gdb) > > > > > > > > > > Looks like it is happening when we call > > > > ngx_http_modsecurity_body_filter() > > > > > then go to this conditions; > > > > > > > > > > rc = move_chain_to_brigade(in, ctx->brigade, r->pool, 0); > > > > > > > > > > if (rc != NGX_OK) { > > > > > > > > > > r->buffered |= NGX_HTTP_SSI_BUFFERED; > > > > > > > > > > return rc; > > > > > > > > > > } > > > > > > > > > > move_chainto_brigade is defined as: > > > > > > > > > > > > > > > ngx_int_t > > > > > > > > > > move_chain_to_brigade(ngx_chain_t *chain, apr_bucket_brigade *bb, > > > > > ngx_pool_t *pool, ngx_int_t last_buf) { > > > > > > > > > > apr_bucket *e; > > > > > > > > > > ngx_chain_t *cl; > > > > > > > > > > > > > > > while (chain) { > > > > > > > > > > e = ngx_buf_to_apr_bucket(chain->buf, bb->p, > > bb->bucket_alloc); > > > > > > > > > > if (e == NULL) { > > > > > > > > > > return NGX_ERROR; > > > > > > > > > > } > > > > > > > > > > > > > > > APR_BRIGADE_INSERT_TAIL(bb, e); > > > > > > > > > > if (chain->buf->last_buf) { > > > > > > > > > > e = apr_bucket_eos_create(bb->bucket_alloc); > > > > > > > > > > APR_BRIGADE_INSERT_TAIL(bb, e); > > > > > > > > > > chain->buf->last_buf = 0; > > > > > > > > > > return NGX_OK; > > > > > > > > > > } > > > > > > > > > > cl = chain; > > > > > > > > > > chain = chain->next; > > > > > > > > > > ngx_free_chain(pool, cl); > > > > > > > > > > } > > > > > > > > > > > > > > > if (last_buf) { > > > > > > > > > > e = apr_bucket_eos_create(bb->bucket_alloc); > > > > > > > > > > APR_BRIGADE_INSERT_TAIL(bb, e); > > > > > > > > > > return NGX_OK; > > > > > > > > > > } > > > > > > > > > > return NGX_AGAIN; > > > > > > > > > > } > > > > > Let me know if you guys can help us understanding why sometimes we > > > > trigger > > > > > this issue > > > > > > > > It looks like your code modify chain links (ngx_chain_t > > > > structures) as got by your response body filter. This is not > > > > something filters are allowed to do, and results are undefined. > > > > If a filter needs to modify chain, it should allocate it's own > > > > chain links. > > > > > > > > In the code quoted you probably don't want to touch chain links at > > > > all. > > > > > > > > -- > > > > Maxim Dounin > > > > http://nginx.org/en/donation.html > > > > > > > > _______________________________________________ > > > > nginx-devel mailing list > > > > nginx-devel at nginx.org > > > > http://mailman.nginx.org/mailman/listinfo/nginx-devel > > > > > > > > > _______________________________________________ > > > nginx-devel mailing list > > > nginx-devel at nginx.org > > > http://mailman.nginx.org/mailman/listinfo/nginx-devel > > > > > > -- > > Maxim Dounin > > http://nginx.org/en/donation.html > > > > _______________________________________________ > > nginx-devel mailing list > > nginx-devel at nginx.org > > http://mailman.nginx.org/mailman/listinfo/nginx-devel > > > _______________________________________________ > nginx-devel mailing list > nginx-devel at nginx.org > http://mailman.nginx.org/mailman/listinfo/nginx-devel -- Maxim Dounin http://nginx.org/en/donation.html From mdounin at mdounin.ru Mon May 13 11:29:06 2013 From: mdounin at mdounin.ru (Maxim Dounin) Date: Mon, 13 May 2013 11:29:06 +0000 Subject: [nginx] Version bump. Message-ID: details: http://hg.nginx.com/nginx/rev/6082e0ab3d89 branches: stable-1.2 changeset: 5205:6082e0ab3d89 user: Maxim Dounin date: Mon May 13 13:18:31 2013 +0400 description: Version bump. diffstat: src/core/nginx.h | 4 ++-- src/http/modules/perl/nginx.pm | 2 +- 2 files changed, 3 insertions(+), 3 deletions(-) diffs (26 lines): diff --git a/src/core/nginx.h b/src/core/nginx.h --- a/src/core/nginx.h +++ b/src/core/nginx.h @@ -9,8 +9,8 @@ #define _NGINX_H_INCLUDED_ -#define nginx_version 1002008 -#define NGINX_VERSION "1.2.8" +#define nginx_version 1002009 +#define NGINX_VERSION "1.2.9" #define NGINX_VER "nginx/" NGINX_VERSION #define NGINX_VAR "NGINX" diff --git a/src/http/modules/perl/nginx.pm b/src/http/modules/perl/nginx.pm --- a/src/http/modules/perl/nginx.pm +++ b/src/http/modules/perl/nginx.pm @@ -50,7 +50,7 @@ our @EXPORT = qw( HTTP_INSUFFICIENT_STORAGE ); -our $VERSION = '1.2.8'; +our $VERSION = '1.2.9'; require XSLoader; XSLoader::load('nginx', $VERSION); From mdounin at mdounin.ru Mon May 13 11:29:06 2013 From: mdounin at mdounin.ru (Maxim Dounin) Date: Mon, 13 May 2013 11:29:06 +0000 Subject: [nginx] nginx-1.2.9-RELEASE Message-ID: details: http://hg.nginx.com/nginx/rev/0e80c5bf5e1b branches: stable-1.2 changeset: 5207:0e80c5bf5e1b user: Maxim Dounin date: Mon May 13 14:41:51 2013 +0400 description: nginx-1.2.9-RELEASE diffstat: docs/xml/nginx/changes.xml | 18 ++++++++++++++++++ 1 files changed, 18 insertions(+), 0 deletions(-) diffs (28 lines): diff --git a/docs/xml/nginx/changes.xml b/docs/xml/nginx/changes.xml --- a/docs/xml/nginx/changes.xml +++ b/docs/xml/nginx/changes.xml @@ -5,6 +5,24 @@ + + + + +?????????? ?????? ???????? ???????? ????? ???? ?????????? ???????, +???? HTTP-?????? ????????? ?????????? ????????? ????? (CVE-2013-2070); +?????? ????????? ? 1.1.4. + + +contents of worker process memory might be sent to a client +if HTTP backend returned specially crafted response (CVE-2013-2070); +the bug had appeared in 1.1.4. + + + + + + From mdounin at mdounin.ru Mon May 13 11:29:06 2013 From: mdounin at mdounin.ru (Maxim Dounin) Date: Mon, 13 May 2013 11:29:06 +0000 Subject: [nginx] Fixed chunk size parsing. Message-ID: details: http://hg.nginx.com/nginx/rev/abfe9e6e72cb branches: stable-1.2 changeset: 5206:abfe9e6e72cb user: Maxim Dounin date: Mon May 13 13:19:28 2013 +0400 description: Fixed chunk size parsing. diffstat: src/http/modules/ngx_http_proxy_module.c | 4 ++++ 1 files changed, 4 insertions(+), 0 deletions(-) diffs (14 lines): diff --git a/src/http/modules/ngx_http_proxy_module.c b/src/http/modules/ngx_http_proxy_module.c --- a/src/http/modules/ngx_http_proxy_module.c +++ b/src/http/modules/ngx_http_proxy_module.c @@ -1865,6 +1865,10 @@ data: } + if (ctx->size < 0 || ctx->length < 0) { + goto invalid; + } + return rc; done: From mdounin at mdounin.ru Mon May 13 11:29:06 2013 From: mdounin at mdounin.ru (Maxim Dounin) Date: Mon, 13 May 2013 11:29:06 +0000 Subject: [nginx] release-1.2.9 tag Message-ID: details: http://hg.nginx.com/nginx/rev/9c3c460f8a05 branches: stable-1.2 changeset: 5208:9c3c460f8a05 user: Maxim Dounin date: Mon May 13 14:43:06 2013 +0400 description: release-1.2.9 tag diffstat: .hgtags | 1 + 1 files changed, 1 insertions(+), 0 deletions(-) diffs (8 lines): diff --git a/.hgtags b/.hgtags --- a/.hgtags +++ b/.hgtags @@ -343,3 +343,4 @@ d763d5c9a13395fb78100f7c2d63c2323541f210 eb1043eaedacddb1bbada27822527049b99bde6d release-1.2.6 a58e268f6c081f671667c0c929f0c5cec3e80958 release-1.2.7 d50f390fa97eb9871622666b40703d497d65925e release-1.2.8 +0e80c5bf5e1bb42a6491fea5340a16e51cd37bb8 release-1.2.9 From ibobrik at gmail.com Mon May 13 11:59:20 2013 From: ibobrik at gmail.com (ivan babrou) Date: Mon, 13 May 2013 15:59:20 +0400 Subject: [nginx] Version bump. In-Reply-To: References: Message-ID: Hi! Details link doesn't work: http://hg.nginx.com/nginx/rev/6082e0ab3d89 On 13 May 2013 15:29, Maxim Dounin wrote: > details: http://hg.nginx.com/nginx/rev/6082e0ab3d89 > branches: stable-1.2 > changeset: 5205:6082e0ab3d89 > user: Maxim Dounin > date: Mon May 13 13:18:31 2013 +0400 > description: > Version bump. > > diffstat: > > src/core/nginx.h | 4 ++-- > src/http/modules/perl/nginx.pm | 2 +- > 2 files changed, 3 insertions(+), 3 deletions(-) > > diffs (26 lines): > > diff --git a/src/core/nginx.h b/src/core/nginx.h > --- a/src/core/nginx.h > +++ b/src/core/nginx.h > @@ -9,8 +9,8 @@ > #define _NGINX_H_INCLUDED_ > > > -#define nginx_version 1002008 > -#define NGINX_VERSION "1.2.8" > +#define nginx_version 1002009 > +#define NGINX_VERSION "1.2.9" > #define NGINX_VER "nginx/" NGINX_VERSION > > #define NGINX_VAR "NGINX" > diff --git a/src/http/modules/perl/nginx.pm b/src/http/modules/perl/ > nginx.pm > --- a/src/http/modules/perl/nginx.pm > +++ b/src/http/modules/perl/nginx.pm > @@ -50,7 +50,7 @@ our @EXPORT = qw( > HTTP_INSUFFICIENT_STORAGE > ); > > -our $VERSION = '1.2.8'; > +our $VERSION = '1.2.9'; > > require XSLoader; > XSLoader::load('nginx', $VERSION); > > _______________________________________________ > nginx-devel mailing list > nginx-devel at nginx.org > http://mailman.nginx.org/mailman/listinfo/nginx-devel > -- Regards, Ian Babrou http://bobrik.name http://twitter.com/ibobrik skype:i.babrou -------------- next part -------------- An HTML attachment was scrubbed... URL: From mdounin at mdounin.ru Mon May 13 12:10:35 2013 From: mdounin at mdounin.ru (Maxim Dounin) Date: Mon, 13 May 2013 16:10:35 +0400 Subject: [nginx] Version bump. In-Reply-To: References: Message-ID: <20130513121035.GQ69760@mdounin.ru> Hello! On Mon, May 13, 2013 at 03:59:20PM +0400, ivan babrou wrote: > Hi! Details link doesn't work: http://hg.nginx.com/nginx/rev/6082e0ab3d89 Thanks for pointing this. An obvious workaround is to change "com" to "org". -- Maxim Dounin http://nginx.org/en/donation.html From breno.silva at gmail.com Mon May 13 12:17:23 2013 From: breno.silva at gmail.com (Breno Silva) Date: Mon, 13 May 2013 09:17:23 -0300 Subject: Nginx eats 100% cpu in ngx_event_pipe_write_to_downstream In-Reply-To: <20130513070119.GW69760@mdounin.ru> References: <20130511225634.GS69760@mdounin.ru> <20130512000640.GU69760@mdounin.ru> <20130513070119.GW69760@mdounin.ru> Message-ID: Hello Maxim, Yes, my module cannot work with streams, we need full response body before it starts working. So what you suggest it something like this in my move_chain_to_brigado() ? while (chain) { buf = ngx_calloc_buf(pool); if (buf == NULL) { return NGX_ERROR; } len = chain->buf->end - chain->buf->start; buf->start = ngx_palloc(pool, len); ngx_memcpy(buf->start, chain->buf->start, len); buf->pos = buf->start; buf->end = buf->last = buf->pos + len; e = ngx_buf_to_apr_bucket(buf, bb->p, bb->bucket_alloc); if (e == NULL) { return NGX_ERROR; } APR_BRIGADE_INSERT_TAIL(bb, e); if (chain->buf->last_buf) { e = apr_bucket_eos_create(bb->bucket_alloc); APR_BRIGADE_INSERT_TAIL(bb, e); chain->buf->last_buf = 0; chain->buf->pos = chain->buf->last; return NGX_OK; } chain->buf->pos = chain->buf->last; chain = chain->next; Thanks Breno On Mon, May 13, 2013 at 4:01 AM, Maxim Dounin wrote: > Hello! > > On Sun, May 12, 2013 at 11:09:26AM -0300, Breno Silva wrote: > > > Hello, > > > > Yes removing he ngx_free_chain fix the cpu issue. Looks like ther modules > > needs the chain. > > I detected another issue.. sometimes when loading big response bodies > (like > > images.. +32K) it does not load it entirely or does it slowly. > > > > Do you have any idea ? > > Thanks > > It looks like you don't release buffers you got in your filter, > and this results buffers exhaustion (output_buffers, > proxy_buffers) and transfer stall. > > If your code can't work with a data stream and needs full response > before it can start it's work, you have to copy needed data from > buffers and release buffers got (by setting b->pos = b->last, like > if a buffer was actually sent). > > To test things you may try the following in config: > > output_buffers 1 1k; > > Then request a static file larger than 1k. > > > > > > Breno > > > > > > On Sat, May 11, 2013 at 9:06 PM, Maxim Dounin > wrote: > > > > > Hello! > > > > > > On Sat, May 11, 2013 at 08:09:59PM -0300, Breno Silva wrote: > > > > > > > Hello Maxim, > > > > > > > > We are sending chains data into a brigade/bucket structure (APR). It > > > > sometimes works fine.. sometimes trigger the cpu issue > > > > > > > > If i call: > > > > rc = move_chain_to_brigade(in, ctx->brigade, r->pool, 1); <--- > change > > > > from 0 to 1 > > > > Then i will get only the first X bytes of the response body. This > works > > > > great... but i need the entire buffer :) > > > > > > > > Could you give us more details how we could patch this function to > fix > > > the > > > > issue ? > > > > > > Remove the > > > > > > ngx_free_chain(pool, cl); > > > > > > call from the move_chain_to_brigade() function, it should fix you > > > CPU hog issues. > > > > > > > > > > > > > > Thanks > > > > > > > > > > > > On Sat, May 11, 2013 at 7:56 PM, Maxim Dounin > > > wrote: > > > > > > > > > Hello! > > > > > > > > > > On Sat, May 11, 2013 at 04:23:00PM -0300, Breno Silva wrote: > > > > > > > > > > > Hello list, > > > > > > > > > > > > We are porting ModSecurity to NGINX. However we are seeing > sometimes > > > an > > > > > > issue. Nginx eats 100% of cpu and when i use gdb i see: > > > > > > > > > > > > gdb -p 8645 > > > > > > > > > > > > ngx_event_pipe_write_to_downstream (p=0x9bbc720, do_write=0) at > > > > > > src/event/ngx_event_pipe.c:551 > > > > > > > > > > > > 551 if (cl->buf->recycled) { > > > > > > (gdb) > > > > > > > > > > > > Looks like it is happening when we call > > > > > ngx_http_modsecurity_body_filter() > > > > > > then go to this conditions; > > > > > > > > > > > > rc = move_chain_to_brigade(in, ctx->brigade, r->pool, 0); > > > > > > > > > > > > if (rc != NGX_OK) { > > > > > > > > > > > > r->buffered |= NGX_HTTP_SSI_BUFFERED; > > > > > > > > > > > > return rc; > > > > > > > > > > > > } > > > > > > > > > > > > move_chainto_brigade is defined as: > > > > > > > > > > > > > > > > > > ngx_int_t > > > > > > > > > > > > move_chain_to_brigade(ngx_chain_t *chain, apr_bucket_brigade *bb, > > > > > > ngx_pool_t *pool, ngx_int_t last_buf) { > > > > > > > > > > > > apr_bucket *e; > > > > > > > > > > > > ngx_chain_t *cl; > > > > > > > > > > > > > > > > > > while (chain) { > > > > > > > > > > > > e = ngx_buf_to_apr_bucket(chain->buf, bb->p, > > > bb->bucket_alloc); > > > > > > > > > > > > if (e == NULL) { > > > > > > > > > > > > return NGX_ERROR; > > > > > > > > > > > > } > > > > > > > > > > > > > > > > > > APR_BRIGADE_INSERT_TAIL(bb, e); > > > > > > > > > > > > if (chain->buf->last_buf) { > > > > > > > > > > > > e = apr_bucket_eos_create(bb->bucket_alloc); > > > > > > > > > > > > APR_BRIGADE_INSERT_TAIL(bb, e); > > > > > > > > > > > > chain->buf->last_buf = 0; > > > > > > > > > > > > return NGX_OK; > > > > > > > > > > > > } > > > > > > > > > > > > cl = chain; > > > > > > > > > > > > chain = chain->next; > > > > > > > > > > > > ngx_free_chain(pool, cl); > > > > > > > > > > > > } > > > > > > > > > > > > > > > > > > if (last_buf) { > > > > > > > > > > > > e = apr_bucket_eos_create(bb->bucket_alloc); > > > > > > > > > > > > APR_BRIGADE_INSERT_TAIL(bb, e); > > > > > > > > > > > > return NGX_OK; > > > > > > > > > > > > } > > > > > > > > > > > > return NGX_AGAIN; > > > > > > > > > > > > } > > > > > > Let me know if you guys can help us understanding why sometimes > we > > > > > trigger > > > > > > this issue > > > > > > > > > > It looks like your code modify chain links (ngx_chain_t > > > > > structures) as got by your response body filter. This is not > > > > > something filters are allowed to do, and results are undefined. > > > > > If a filter needs to modify chain, it should allocate it's own > > > > > chain links. > > > > > > > > > > In the code quoted you probably don't want to touch chain links at > > > > > all. > > > > > > > > > > -- > > > > > Maxim Dounin > > > > > http://nginx.org/en/donation.html > > > > > > > > > > _______________________________________________ > > > > > nginx-devel mailing list > > > > > nginx-devel at nginx.org > > > > > http://mailman.nginx.org/mailman/listinfo/nginx-devel > > > > > > > > > > > > _______________________________________________ > > > > nginx-devel mailing list > > > > nginx-devel at nginx.org > > > > http://mailman.nginx.org/mailman/listinfo/nginx-devel > > > > > > > > > -- > > > Maxim Dounin > > > http://nginx.org/en/donation.html > > > > > > _______________________________________________ > > > nginx-devel mailing list > > > nginx-devel at nginx.org > > > http://mailman.nginx.org/mailman/listinfo/nginx-devel > > > > > > _______________________________________________ > > nginx-devel mailing list > > nginx-devel at nginx.org > > http://mailman.nginx.org/mailman/listinfo/nginx-devel > > > -- > Maxim Dounin > http://nginx.org/en/donation.html > > _______________________________________________ > nginx-devel mailing list > nginx-devel at nginx.org > http://mailman.nginx.org/mailman/listinfo/nginx-devel > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mdounin at mdounin.ru Mon May 13 12:30:41 2013 From: mdounin at mdounin.ru (Maxim Dounin) Date: Mon, 13 May 2013 16:30:41 +0400 Subject: [PATCH] PCRE: retain input pattern for all regular expressions In-Reply-To: References: <201305080508.17422.vbart@nginx.com> <20130511173039.GR69760@mdounin.ru> <201305120216.55582.ne@vbart.ru> Message-ID: <20130513123041.GR69760@mdounin.ru> Hello! On Sun, May 12, 2013 at 01:37:18PM +0400, Vadim Lazovskiy wrote: > It is a pity that commits are no longer sent to the mailing list. This was a fallout from migrating to Mercurial. Should be fixed now. -- Maxim Dounin http://nginx.org/en/donation.html From ru at nginx.com Mon May 13 12:45:49 2013 From: ru at nginx.com (Ruslan Ermilov) Date: Mon, 13 May 2013 12:45:49 +0000 Subject: [nginx] Upstream: allow to intercept responses with status 300. Message-ID: details: http://hg.nginx.com/nginx/rev/07e515e65984 branches: changeset: 5209:07e515e65984 user: Ruslan Ermilov date: Mon May 13 14:10:22 2013 +0400 description: Upstream: allow to intercept responses with status 300. This fixes an omission made in 9e7926763f87 where all 3XX statuses were allowed for "error_page". diffstat: src/http/ngx_http_upstream.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diffs (12 lines): diff -r a64c8a5da336 -r 07e515e65984 src/http/ngx_http_upstream.c --- a/src/http/ngx_http_upstream.c Thu May 02 03:26:36 2013 -0700 +++ b/src/http/ngx_http_upstream.c Mon May 13 14:10:22 2013 +0400 @@ -1660,7 +1660,7 @@ ngx_http_upstream_process_header(ngx_htt /* rc == NGX_OK */ - if (u->headers_in.status_n > NGX_HTTP_SPECIAL_RESPONSE) { + if (u->headers_in.status_n >= NGX_HTTP_SPECIAL_RESPONSE) { if (r->subrequest_in_memory) { u->buffer.last = u->buffer.pos; From mdounin at mdounin.ru Mon May 13 13:05:50 2013 From: mdounin at mdounin.ru (Maxim Dounin) Date: Mon, 13 May 2013 17:05:50 +0400 Subject: Nginx eats 100% cpu in ngx_event_pipe_write_to_downstream In-Reply-To: References: <20130511225634.GS69760@mdounin.ru> <20130512000640.GU69760@mdounin.ru> <20130513070119.GW69760@mdounin.ru> Message-ID: <20130513130550.GS69760@mdounin.ru> Hello! On Mon, May 13, 2013 at 09:17:23AM -0300, Breno Silva wrote: > Hello Maxim, > > Yes, my module cannot work with streams, we need full response body before > it starts working. > So what you suggest it something like this in my move_chain_to_brigado() ? > > while (chain) { > > buf = ngx_calloc_buf(pool); > if (buf == NULL) { > return NGX_ERROR; > } > > len = chain->buf->end - chain->buf->start; > buf->start = ngx_palloc(pool, len); > ngx_memcpy(buf->start, chain->buf->start, len); > > buf->pos = buf->start; > buf->end = buf->last = buf->pos + len; > > e = ngx_buf_to_apr_bucket(buf, bb->p, bb->bucket_alloc); > if (e == NULL) { > return NGX_ERROR; > } > > APR_BRIGADE_INSERT_TAIL(bb, e); > if (chain->buf->last_buf) { > e = apr_bucket_eos_create(bb->bucket_alloc); > APR_BRIGADE_INSERT_TAIL(bb, e); > chain->buf->last_buf = 0; Just a side note: resetting last_buf looks useless. > chain->buf->pos = chain->buf->last; > return NGX_OK; > } > > chain->buf->pos = chain->buf->last; > chain = chain->next; This looks suboptimal, but should work. > > Thanks > > Breno > > > On Mon, May 13, 2013 at 4:01 AM, Maxim Dounin wrote: > > > Hello! > > > > On Sun, May 12, 2013 at 11:09:26AM -0300, Breno Silva wrote: > > > > > Hello, > > > > > > Yes removing he ngx_free_chain fix the cpu issue. Looks like ther modules > > > needs the chain. > > > I detected another issue.. sometimes when loading big response bodies > > (like > > > images.. +32K) it does not load it entirely or does it slowly. > > > > > > Do you have any idea ? > > > Thanks > > > > It looks like you don't release buffers you got in your filter, > > and this results buffers exhaustion (output_buffers, > > proxy_buffers) and transfer stall. > > > > If your code can't work with a data stream and needs full response > > before it can start it's work, you have to copy needed data from > > buffers and release buffers got (by setting b->pos = b->last, like > > if a buffer was actually sent). > > > > To test things you may try the following in config: > > > > output_buffers 1 1k; > > > > Then request a static file larger than 1k. > > > > > > > > > > Breno > > > > > > > > > On Sat, May 11, 2013 at 9:06 PM, Maxim Dounin > > wrote: > > > > > > > Hello! > > > > > > > > On Sat, May 11, 2013 at 08:09:59PM -0300, Breno Silva wrote: > > > > > > > > > Hello Maxim, > > > > > > > > > > We are sending chains data into a brigade/bucket structure (APR). It > > > > > sometimes works fine.. sometimes trigger the cpu issue > > > > > > > > > > If i call: > > > > > rc = move_chain_to_brigade(in, ctx->brigade, r->pool, 1); <--- > > change > > > > > from 0 to 1 > > > > > Then i will get only the first X bytes of the response body. This > > works > > > > > great... but i need the entire buffer :) > > > > > > > > > > Could you give us more details how we could patch this function to > > fix > > > > the > > > > > issue ? > > > > > > > > Remove the > > > > > > > > ngx_free_chain(pool, cl); > > > > > > > > call from the move_chain_to_brigade() function, it should fix you > > > > CPU hog issues. > > > > > > > > > > > > > > > > > > Thanks > > > > > > > > > > > > > > > On Sat, May 11, 2013 at 7:56 PM, Maxim Dounin > > > > wrote: > > > > > > > > > > > Hello! > > > > > > > > > > > > On Sat, May 11, 2013 at 04:23:00PM -0300, Breno Silva wrote: > > > > > > > > > > > > > Hello list, > > > > > > > > > > > > > > We are porting ModSecurity to NGINX. However we are seeing > > sometimes > > > > an > > > > > > > issue. Nginx eats 100% of cpu and when i use gdb i see: > > > > > > > > > > > > > > gdb -p 8645 > > > > > > > > > > > > > > ngx_event_pipe_write_to_downstream (p=0x9bbc720, do_write=0) at > > > > > > > src/event/ngx_event_pipe.c:551 > > > > > > > > > > > > > > 551 if (cl->buf->recycled) { > > > > > > > (gdb) > > > > > > > > > > > > > > Looks like it is happening when we call > > > > > > ngx_http_modsecurity_body_filter() > > > > > > > then go to this conditions; > > > > > > > > > > > > > > rc = move_chain_to_brigade(in, ctx->brigade, r->pool, 0); > > > > > > > > > > > > > > if (rc != NGX_OK) { > > > > > > > > > > > > > > r->buffered |= NGX_HTTP_SSI_BUFFERED; > > > > > > > > > > > > > > return rc; > > > > > > > > > > > > > > } > > > > > > > > > > > > > > move_chainto_brigade is defined as: > > > > > > > > > > > > > > > > > > > > > ngx_int_t > > > > > > > > > > > > > > move_chain_to_brigade(ngx_chain_t *chain, apr_bucket_brigade *bb, > > > > > > > ngx_pool_t *pool, ngx_int_t last_buf) { > > > > > > > > > > > > > > apr_bucket *e; > > > > > > > > > > > > > > ngx_chain_t *cl; > > > > > > > > > > > > > > > > > > > > > while (chain) { > > > > > > > > > > > > > > e = ngx_buf_to_apr_bucket(chain->buf, bb->p, > > > > bb->bucket_alloc); > > > > > > > > > > > > > > if (e == NULL) { > > > > > > > > > > > > > > return NGX_ERROR; > > > > > > > > > > > > > > } > > > > > > > > > > > > > > > > > > > > > APR_BRIGADE_INSERT_TAIL(bb, e); > > > > > > > > > > > > > > if (chain->buf->last_buf) { > > > > > > > > > > > > > > e = apr_bucket_eos_create(bb->bucket_alloc); > > > > > > > > > > > > > > APR_BRIGADE_INSERT_TAIL(bb, e); > > > > > > > > > > > > > > chain->buf->last_buf = 0; > > > > > > > > > > > > > > return NGX_OK; > > > > > > > > > > > > > > } > > > > > > > > > > > > > > cl = chain; > > > > > > > > > > > > > > chain = chain->next; > > > > > > > > > > > > > > ngx_free_chain(pool, cl); > > > > > > > > > > > > > > } > > > > > > > > > > > > > > > > > > > > > if (last_buf) { > > > > > > > > > > > > > > e = apr_bucket_eos_create(bb->bucket_alloc); > > > > > > > > > > > > > > APR_BRIGADE_INSERT_TAIL(bb, e); > > > > > > > > > > > > > > return NGX_OK; > > > > > > > > > > > > > > } > > > > > > > > > > > > > > return NGX_AGAIN; > > > > > > > > > > > > > > } > > > > > > > Let me know if you guys can help us understanding why sometimes > > we > > > > > > trigger > > > > > > > this issue > > > > > > > > > > > > It looks like your code modify chain links (ngx_chain_t > > > > > > structures) as got by your response body filter. This is not > > > > > > something filters are allowed to do, and results are undefined. > > > > > > If a filter needs to modify chain, it should allocate it's own > > > > > > chain links. > > > > > > > > > > > > In the code quoted you probably don't want to touch chain links at > > > > > > all. > > > > > > > > > > > > -- > > > > > > Maxim Dounin > > > > > > http://nginx.org/en/donation.html > > > > > > > > > > > > _______________________________________________ > > > > > > nginx-devel mailing list > > > > > > nginx-devel at nginx.org > > > > > > http://mailman.nginx.org/mailman/listinfo/nginx-devel > > > > > > > > > > > > > > > _______________________________________________ > > > > > nginx-devel mailing list > > > > > nginx-devel at nginx.org > > > > > http://mailman.nginx.org/mailman/listinfo/nginx-devel > > > > > > > > > > > > -- > > > > Maxim Dounin > > > > http://nginx.org/en/donation.html > > > > > > > > _______________________________________________ > > > > nginx-devel mailing list > > > > nginx-devel at nginx.org > > > > http://mailman.nginx.org/mailman/listinfo/nginx-devel > > > > > > > > > _______________________________________________ > > > nginx-devel mailing list > > > nginx-devel at nginx.org > > > http://mailman.nginx.org/mailman/listinfo/nginx-devel > > > > > > -- > > Maxim Dounin > > http://nginx.org/en/donation.html > > > > _______________________________________________ > > nginx-devel mailing list > > nginx-devel at nginx.org > > http://mailman.nginx.org/mailman/listinfo/nginx-devel > > > _______________________________________________ > nginx-devel mailing list > nginx-devel at nginx.org > http://mailman.nginx.org/mailman/listinfo/nginx-devel -- Maxim Dounin http://nginx.org/en/donation.html From mdounin at mdounin.ru Mon May 13 14:25:44 2013 From: mdounin at mdounin.ru (Maxim Dounin) Date: Mon, 13 May 2013 14:25:44 +0000 Subject: [nginx] Fixed lingering_time check. Message-ID: details: http://hg.nginx.org/nginx/rev/ea2ba6dbe361 branches: changeset: 5210:ea2ba6dbe361 user: Maxim Dounin date: Mon May 13 17:39:45 2013 +0400 description: Fixed lingering_time check. There are two significant changes in this patch: 1) The <= 0 comparison is done with a signed type. This fixes the case of ngx_time() being larger than r->lingering_time. 2) Calculation of r->lingering_time - ngx_time() is now always done in the ngx_msec_t type. This ensures the calculation is correct even if time_t is unsigned and differs in size from ngx_msec_t. Thanks to Lanshun Zhou. diffstat: src/http/ngx_http_request.c | 4 ++-- src/http/ngx_http_request_body.c | 4 ++-- 2 files changed, 4 insertions(+), 4 deletions(-) diffs (29 lines): diff --git a/src/http/ngx_http_request.c b/src/http/ngx_http_request.c --- a/src/http/ngx_http_request.c +++ b/src/http/ngx_http_request.c @@ -3166,8 +3166,8 @@ ngx_http_lingering_close_handler(ngx_eve return; } - timer = (ngx_msec_t) (r->lingering_time - ngx_time()); - if (timer <= 0) { + timer = (ngx_msec_t) r->lingering_time - (ngx_msec_t) ngx_time(); + if ((ngx_msec_int_t) timer <= 0) { ngx_http_close_request(r, 0); return; } diff --git a/src/http/ngx_http_request_body.c b/src/http/ngx_http_request_body.c --- a/src/http/ngx_http_request_body.c +++ b/src/http/ngx_http_request_body.c @@ -570,9 +570,9 @@ ngx_http_discarded_request_body_handler( } if (r->lingering_time) { - timer = (ngx_msec_t) (r->lingering_time - ngx_time()); + timer = (ngx_msec_t) r->lingering_time - (ngx_msec_t) ngx_time(); - if (timer <= 0) { + if ((ngx_msec_int_t) timer <= 0) { r->discard_body = 0; r->lingering_close = 0; ngx_http_finalize_request(r, NGX_ERROR); From mdounin at mdounin.ru Mon May 13 14:26:34 2013 From: mdounin at mdounin.ru (Maxim Dounin) Date: Mon, 13 May 2013 18:26:34 +0400 Subject: [PATCH] use signed value when comparing timer with 0 and check lingering_time setting In-Reply-To: References: Message-ID: <20130513142634.GU69760@mdounin.ru> Hello! On Sat, Apr 27, 2013 at 12:43:21PM +0800, lanshun zhou wrote: > In ngx_http_lingering_close_handler > and ngx_http_discarded_request_body_handler, there's risk > that r->lingering_time is smaller than ngx_time(), then comparing timer > which is a unsigned value with zero will never return true. This can cause > long time connection for some kind of requests (For example, lingering_time > is set smaller than lingering_timeout) I've pushed slightly different patch which addresses both the original problem and concerns about systems with unsigned time_t: http://hg.nginx.org/nginx/rev/ea2ba6dbe361 Part of the patch which added configuration time checks was dropped, as it looks unneded. Thanks! -- Maxim Dounin http://nginx.org/en/donation.html From mdounin at mdounin.ru Wed May 15 09:21:41 2013 From: mdounin at mdounin.ru (Maxim Dounin) Date: Wed, 15 May 2013 09:21:41 +0000 Subject: [nginx] Proxy: clear script engine used to calculate lengths. Message-ID: details: http://hg.nginx.org/nginx/rev/ecd762770729 branches: changeset: 5211:ecd762770729 user: Maxim Dounin date: Wed May 15 12:23:44 2013 +0400 description: Proxy: clear script engine used to calculate lengths. Previous code is believed to be safe, but might access uninitialized memory (e.g., e->quote). diffstat: src/http/modules/ngx_http_proxy_module.c | 2 ++ 1 files changed, 2 insertions(+), 0 deletions(-) diffs (12 lines): diff --git a/src/http/modules/ngx_http_proxy_module.c b/src/http/modules/ngx_http_proxy_module.c --- a/src/http/modules/ngx_http_proxy_module.c +++ b/src/http/modules/ngx_http_proxy_module.c @@ -993,6 +993,8 @@ ngx_http_proxy_create_request(ngx_http_r len += uri_len; + ngx_memzero(&le, sizeof(ngx_http_script_engine_t)); + ngx_http_script_flush_no_cacheable_variables(r, plcf->flushes); if (plcf->body_set_len) { From ru at nginx.com Wed May 15 11:05:43 2013 From: ru at nginx.com (Ruslan Ermilov) Date: Wed, 15 May 2013 11:05:43 +0000 Subject: [nginx] Removed vestiges of SVN. Message-ID: details: http://hg.nginx.org/nginx/rev/09dbd363050a branches: changeset: 5212:09dbd363050a user: Ruslan Ermilov date: Thu Apr 25 17:41:45 2013 +0400 description: Removed vestiges of SVN. diffstat: misc/GNUmakefile | 31 ++----------------------------- misc/README | 3 --- 2 files changed, 2 insertions(+), 32 deletions(-) diffs (64 lines): diff -r ecd762770729 -r 09dbd363050a misc/GNUmakefile --- a/misc/GNUmakefile Wed May 15 12:23:44 2013 +0400 +++ b/misc/GNUmakefile Thu Apr 25 17:41:45 2013 +0400 @@ -3,7 +3,6 @@ VER = $(shell grep 'define NGINX_VERSIO | sed -e 's/^.*"\(.*\)".*/\1/') NGINX = nginx-$(VER) TEMP = tmp -REPO = $(shell svn info | sed -n 's/^Repository Root: //p') OBJS = objs.msvc8 OPENSSL = openssl-1.0.1e @@ -38,40 +37,14 @@ release: export export: rm -rf $(TEMP) - - if [ -d .svn ]; then \ - svn export -rHEAD . $(TEMP)/$(NGINX); \ - else \ - hg archive -X '.hg*' $(TEMP)/$(NGINX); \ - fi + hg archive -X '.hg*' $(TEMP)/$(NGINX) RELEASE: - if [ -d .svn ]; then \ - $(MAKE) -f misc/GNUmakefile RELEASE.svn; \ - else \ - $(MAKE) -f misc/GNUmakefile RELEASE.hg; \ - fi - - $(MAKE) -f misc/GNUmakefile release - - -RELEASE.hg: hg ci -m nginx-$(VER)-RELEASE hg tag -m "release-$(VER) tag" release-$(VER) - -RELEASE.svn: - test -d $(TEMP) || mkdir -p $(TEMP) - - echo "nginx-$(VER)-RELEASE" > $(TEMP)/message - svn ci -F $(TEMP)/message - - echo "release-$(VER) tag" > $(TEMP)/message - svn copy $(REPO)/trunk $(REPO)/tags/release-$(VER) \ - -F $(TEMP)/message - - svn up + $(MAKE) -f misc/GNUmakefile release win32: diff -r ecd762770729 -r 09dbd363050a misc/README --- a/misc/README Wed May 15 12:23:44 2013 +0400 +++ b/misc/README Thu Apr 25 17:41:45 2013 +0400 @@ -1,6 +1,3 @@ - -GNUmakefile, in svn it is available since 0.4.0 only. - make -f misc/GNUmakefile release From ru at nginx.com Wed May 15 11:05:43 2013 From: ru at nginx.com (Ruslan Ermilov) Date: Wed, 15 May 2013 11:05:43 +0000 Subject: [nginx] Upstream keepalive: slightly simplified code. Message-ID: details: http://hg.nginx.org/nginx/rev/822b82191940 branches: changeset: 5213:822b82191940 user: Ruslan Ermilov date: Wed May 15 15:04:49 2013 +0400 description: Upstream keepalive: slightly simplified code. diffstat: src/http/modules/ngx_http_upstream_keepalive_module.c | 7 ++----- 1 files changed, 2 insertions(+), 5 deletions(-) diffs (31 lines): diff -r 09dbd363050a -r 822b82191940 src/http/modules/ngx_http_upstream_keepalive_module.c --- a/src/http/modules/ngx_http_upstream_keepalive_module.c Thu Apr 25 17:41:45 2013 +0400 +++ b/src/http/modules/ngx_http_upstream_keepalive_module.c Wed May 15 15:04:49 2013 +0400 @@ -81,7 +81,7 @@ static ngx_command_t ngx_http_upstream_ { ngx_string("keepalive"), NGX_HTTP_UPS_CONF|NGX_CONF_TAKE12, ngx_http_upstream_keepalive, - 0, + NGX_HTTP_SRV_CONF_OFFSET, 0, NULL }, @@ -481,7 +481,7 @@ static char * ngx_http_upstream_keepalive(ngx_conf_t *cf, ngx_command_t *cmd, void *conf) { ngx_http_upstream_srv_conf_t *uscf; - ngx_http_upstream_keepalive_srv_conf_t *kcf; + ngx_http_upstream_keepalive_srv_conf_t *kcf = conf; ngx_int_t n; ngx_str_t *value; @@ -489,9 +489,6 @@ ngx_http_upstream_keepalive(ngx_conf_t * uscf = ngx_http_conf_get_module_srv_conf(cf, ngx_http_upstream_module); - kcf = ngx_http_conf_upstream_srv_conf(uscf, - ngx_http_upstream_keepalive_module); - if (kcf->original_init_upstream) { return "is duplicate"; } From vbart at nginx.com Wed May 15 20:36:00 2013 From: vbart at nginx.com (Valentin Bartenev) Date: Wed, 15 May 2013 20:36:00 +0000 Subject: [nginx] Mail: removed surplus ngx_close_connection() call. Message-ID: details: http://hg.nginx.org/nginx/rev/2220de0521ca branches: changeset: 5214:2220de0521ca user: Filipe Da Silva date: Thu May 09 10:54:28 2013 +0200 description: Mail: removed surplus ngx_close_connection() call. It is already called for a peer connection a few lines above. diffstat: src/mail/ngx_mail_auth_http_module.c | 1 - 1 files changed, 0 insertions(+), 1 deletions(-) diffs (11 lines): diff -r 822b82191940 -r 2220de0521ca src/mail/ngx_mail_auth_http_module.c --- a/src/mail/ngx_mail_auth_http_module.c Wed May 15 15:04:49 2013 +0400 +++ b/src/mail/ngx_mail_auth_http_module.c Thu May 09 10:54:28 2013 +0200 @@ -699,7 +699,6 @@ ngx_mail_auth_http_process_headers(ngx_m p = ngx_pnalloc(s->connection->pool, ctx->err.len); if (p == NULL) { - ngx_close_connection(ctx->peer.connection); ngx_destroy_pool(ctx->pool); ngx_mail_session_internal_server_error(s); return; From piotr at cloudflare.com Thu May 16 22:42:01 2013 From: piotr at cloudflare.com (Piotr Sikora) Date: Thu, 16 May 2013 15:42:01 -0700 Subject: [PATCH] OCSP stapling: fix error logging of successful OCSP responses. Message-ID: changeset: 5215:cfab1e7e4ac2 user: Piotr Sikora date: Thu May 16 15:37:13 2013 -0700 files: src/event/ngx_event_openssl_stapling.c description: OCSP stapling: fix error logging of successful OCSP responses. Due to a bad argument list, nginx worker would crash (SIGSEGV) while trying to log the fact that it received OCSP response with "revoked" or "unknown" certificate status. While there, fix similar (but non-crashing) error a few lines above. Signed-off-by: Piotr Sikora diff -r 2220de0521ca -r cfab1e7e4ac2 src/event/ngx_event_openssl_stapling.c --- a/src/event/ngx_event_openssl_stapling.c Thu May 09 10:54:28 2013 +0200 +++ b/src/event/ngx_event_openssl_stapling.c Thu May 16 15:37:13 2013 -0700 @@ -611,15 +611,14 @@ != 1) { ngx_log_error(NGX_LOG_ERR, ctx->log, 0, - "certificate status not found in the OCSP response", - n, OCSP_response_status_str(n)); + "certificate status not found in the OCSP response"); goto error; } if (n != V_OCSP_CERTSTATUS_GOOD) { ngx_log_error(NGX_LOG_ERR, ctx->log, 0, "certificate status \"%s\" in the OCSP response", - n, OCSP_cert_status_str(n)); + OCSP_cert_status_str(n)); goto error; } From piotr at cloudflare.com Thu May 16 22:42:39 2013 From: piotr at cloudflare.com (Piotr Sikora) Date: Thu, 16 May 2013 15:42:39 -0700 Subject: [PATCH] OCSP stapling: better handling of successful OCSP responses. Message-ID: changeset: 5216:4fb8fac2b2f5 user: Piotr Sikora date: Thu May 16 15:37:24 2013 -0700 files: src/event/ngx_event_openssl_stapling.c description: OCSP stapling: better handling of successful OCSP responses. All successful OCSP responseses, regardless of the certificate status, should be cached and used for OCSP stapling. While there, log the certificate's common name and revocation reason, because certificate status alone isn't very useful information. Signed-off-by: Piotr Sikora diff -r cfab1e7e4ac2 -r 4fb8fac2b2f5 src/event/ngx_event_openssl_stapling.c --- a/src/event/ngx_event_openssl_stapling.c Thu May 16 15:37:13 2013 -0700 +++ b/src/event/ngx_event_openssl_stapling.c Thu May 16 15:37:24 2013 -0700 @@ -529,7 +529,7 @@ const #endif u_char *p; - int n; + int n, r, idx; size_t len; ngx_str_t response; X509_STORE *store; @@ -539,6 +539,10 @@ OCSP_BASICRESP *basic; ngx_ssl_stapling_t *staple; ASN1_GENERALIZEDTIME *thisupdate, *nextupdate; + X509_NAME *name; + X509_NAME_ENTRY *entry; + ASN1_STRING *str; + ngx_str_t s; staple = ctx->data; ocsp = NULL; @@ -606,7 +610,7 @@ goto error; } - if (OCSP_resp_find_status(basic, id, &n, NULL, NULL, + if (OCSP_resp_find_status(basic, id, &n, &r, NULL, &thisupdate, &nextupdate) != 1) { @@ -615,19 +619,43 @@ goto error; } - if (n != V_OCSP_CERTSTATUS_GOOD) { - ngx_log_error(NGX_LOG_ERR, ctx->log, 0, - "certificate status \"%s\" in the OCSP response", - OCSP_cert_status_str(n)); - goto error; - } - if (OCSP_check_validity(thisupdate, nextupdate, 300, -1) != 1) { ngx_ssl_error(NGX_LOG_ERR, ctx->log, 0, "OCSP_check_validity() failed"); goto error; } + if (n != V_OCSP_CERTSTATUS_GOOD) { + ngx_str_set(&s, "unknown"); + + if (ctx->cert) { + name = X509_get_subject_name(ctx->cert); + if (name) { + idx = X509_NAME_get_index_by_NID(name, NID_commonName, -1); + if (idx != -1) { + entry = X509_NAME_get_entry(name, idx); + if (entry) { + str = X509_NAME_ENTRY_get_data(entry); + s.data = ASN1_STRING_data(str); + s.len = ASN1_STRING_length(str); + } + } + } + } + + if (n == V_OCSP_CERTSTATUS_REVOKED && r != -1) { + ngx_log_error(NGX_LOG_WARN, ctx->log, 0, + "certificate status \"%s\" (reason: \"%s\") in the " + "OCSP response for \"%V\"", + OCSP_cert_status_str(n), OCSP_crl_reason_str(r), &s); + + } else { + ngx_log_error(NGX_LOG_WARN, ctx->log, 0, + "certificate status \"%s\" in the OCSP response " + "for \"%V\"", OCSP_cert_status_str(n), &s); + } + } + OCSP_CERTID_free(id); OCSP_BASICRESP_free(basic); OCSP_RESPONSE_free(ocsp); From piotr at cloudflare.com Thu May 16 23:10:26 2013 From: piotr at cloudflare.com (Piotr Sikora) Date: Thu, 16 May 2013 16:10:26 -0700 Subject: [PATCH] OCSP stapling: fix error logging of successful OCSP responses. In-Reply-To: References: Message-ID: Erm, "hg export" patch attached, sorry about that. Best regards, Piotr Sikora # HG changeset patch # User Piotr Sikora # Date 1368743833 25200 # Node ID cfab1e7e4ac2f0d17199ee1d49ac4647b63746d3 # Parent 2220de0521ca2c0b664a8ea1e201ce1cb90fd7a2 OCSP stapling: fix error logging of successful OCSP responses. Due to a bad argument list, nginx worker would crash (SIGSEGV) while trying to log the fact that it received OCSP response with "revoked" or "unknown" certificate status. While there, fix similar (but non-crashing) error a few lines above. Signed-off-by: Piotr Sikora diff -r 2220de0521ca -r cfab1e7e4ac2 src/event/ngx_event_openssl_stapling.c --- a/src/event/ngx_event_openssl_stapling.c Thu May 09 10:54:28 2013 +0200 +++ b/src/event/ngx_event_openssl_stapling.c Thu May 16 15:37:13 2013 -0700 @@ -611,15 +611,14 @@ != 1) { ngx_log_error(NGX_LOG_ERR, ctx->log, 0, - "certificate status not found in the OCSP response", - n, OCSP_response_status_str(n)); + "certificate status not found in the OCSP response"); goto error; } if (n != V_OCSP_CERTSTATUS_GOOD) { ngx_log_error(NGX_LOG_ERR, ctx->log, 0, "certificate status \"%s\" in the OCSP response", - n, OCSP_cert_status_str(n)); + OCSP_cert_status_str(n)); goto error; } From piotr at cloudflare.com Thu May 16 23:10:33 2013 From: piotr at cloudflare.com (Piotr Sikora) Date: Thu, 16 May 2013 16:10:33 -0700 Subject: [PATCH] OCSP stapling: better handling of successful OCSP responses. In-Reply-To: References: Message-ID: Erm, "hg export" patch attached, sorry about that. Best regards, Piotr Sikora # HG changeset patch # User Piotr Sikora # Date 1368743844 25200 # Node ID 4fb8fac2b2f58f8946c120a3da9743c4af8dd6ba # Parent cfab1e7e4ac2f0d17199ee1d49ac4647b63746d3 OCSP stapling: better handling of successful OCSP responses. All successful OCSP responseses, regardless of the certificate status, should be cached and used for OCSP stapling. While there, log the certificate's common name and revocation reason, because certificate status alone isn't very useful information. Signed-off-by: Piotr Sikora diff -r cfab1e7e4ac2 -r 4fb8fac2b2f5 src/event/ngx_event_openssl_stapling.c --- a/src/event/ngx_event_openssl_stapling.c Thu May 16 15:37:13 2013 -0700 +++ b/src/event/ngx_event_openssl_stapling.c Thu May 16 15:37:24 2013 -0700 @@ -529,7 +529,7 @@ const #endif u_char *p; - int n; + int n, r, idx; size_t len; ngx_str_t response; X509_STORE *store; @@ -539,6 +539,10 @@ OCSP_BASICRESP *basic; ngx_ssl_stapling_t *staple; ASN1_GENERALIZEDTIME *thisupdate, *nextupdate; + X509_NAME *name; + X509_NAME_ENTRY *entry; + ASN1_STRING *str; + ngx_str_t s; staple = ctx->data; ocsp = NULL; @@ -606,7 +610,7 @@ goto error; } - if (OCSP_resp_find_status(basic, id, &n, NULL, NULL, + if (OCSP_resp_find_status(basic, id, &n, &r, NULL, &thisupdate, &nextupdate) != 1) { @@ -615,19 +619,43 @@ goto error; } - if (n != V_OCSP_CERTSTATUS_GOOD) { - ngx_log_error(NGX_LOG_ERR, ctx->log, 0, - "certificate status \"%s\" in the OCSP response", - OCSP_cert_status_str(n)); - goto error; - } - if (OCSP_check_validity(thisupdate, nextupdate, 300, -1) != 1) { ngx_ssl_error(NGX_LOG_ERR, ctx->log, 0, "OCSP_check_validity() failed"); goto error; } + if (n != V_OCSP_CERTSTATUS_GOOD) { + ngx_str_set(&s, "unknown"); + + if (ctx->cert) { + name = X509_get_subject_name(ctx->cert); + if (name) { + idx = X509_NAME_get_index_by_NID(name, NID_commonName, -1); + if (idx != -1) { + entry = X509_NAME_get_entry(name, idx); + if (entry) { + str = X509_NAME_ENTRY_get_data(entry); + s.data = ASN1_STRING_data(str); + s.len = ASN1_STRING_length(str); + } + } + } + } + + if (n == V_OCSP_CERTSTATUS_REVOKED && r != -1) { + ngx_log_error(NGX_LOG_WARN, ctx->log, 0, + "certificate status \"%s\" (reason: \"%s\") in the " + "OCSP response for \"%V\"", + OCSP_cert_status_str(n), OCSP_crl_reason_str(r), &s); + + } else { + ngx_log_error(NGX_LOG_WARN, ctx->log, 0, + "certificate status \"%s\" in the OCSP response " + "for \"%V\"", OCSP_cert_status_str(n), &s); + } + } + OCSP_CERTID_free(id); OCSP_BASICRESP_free(basic); OCSP_RESPONSE_free(ocsp); From mdounin at mdounin.ru Fri May 17 13:12:47 2013 From: mdounin at mdounin.ru (Maxim Dounin) Date: Fri, 17 May 2013 17:12:47 +0400 Subject: [PATCH] OCSP stapling: fix error logging of successful OCSP responses. In-Reply-To: References: Message-ID: <20130517131247.GW69760@mdounin.ru> Hello! On Thu, May 16, 2013 at 04:10:26PM -0700, Piotr Sikora wrote: > Erm, "hg export" patch attached, sorry about that. > > Best regards, > Piotr Sikora > > > # HG changeset patch > # User Piotr Sikora > # Date 1368743833 25200 > # Node ID cfab1e7e4ac2f0d17199ee1d49ac4647b63746d3 > # Parent 2220de0521ca2c0b664a8ea1e201ce1cb90fd7a2 > OCSP stapling: fix error logging of successful OCSP responses. > > Due to a bad argument list, nginx worker would crash (SIGSEGV) while > trying to log the fact that it received OCSP response with "revoked" > or "unknown" certificate status. > > While there, fix similar (but non-crashing) error a few lines above. > > Signed-off-by: Piotr Sikora This one queued, thanks. -- Maxim Dounin http://nginx.org/en/donation.html From mdounin at mdounin.ru Fri May 17 13:20:26 2013 From: mdounin at mdounin.ru (Maxim Dounin) Date: Fri, 17 May 2013 17:20:26 +0400 Subject: [PATCH] OCSP stapling: better handling of successful OCSP responses. In-Reply-To: References: Message-ID: <20130517132026.GX69760@mdounin.ru> Hello! On Thu, May 16, 2013 at 04:10:33PM -0700, Piotr Sikora wrote: > Erm, "hg export" patch attached, sorry about that. > > Best regards, > Piotr Sikora > > > # HG changeset patch > # User Piotr Sikora > # Date 1368743844 25200 > # Node ID 4fb8fac2b2f58f8946c120a3da9743c4af8dd6ba > # Parent cfab1e7e4ac2f0d17199ee1d49ac4647b63746d3 > OCSP stapling: better handling of successful OCSP responses. > > All successful OCSP responseses, regardless of the certificate status, > should be cached and used for OCSP stapling. Presenting a certificate and a non-good certificate status to a user looks like "bees against honey" for me. I would rather not. -- Maxim Dounin http://nginx.org/en/donation.html From mdounin at mdounin.ru Fri May 17 13:41:56 2013 From: mdounin at mdounin.ru (Maxim Dounin) Date: Fri, 17 May 2013 13:41:56 +0000 Subject: [nginx] OCSP stapling: fix error logging of successful OCSP resp... Message-ID: details: http://hg.nginx.org/nginx/rev/cfab1e7e4ac2 branches: changeset: 5215:cfab1e7e4ac2 user: Piotr Sikora date: Thu May 16 15:37:13 2013 -0700 description: OCSP stapling: fix error logging of successful OCSP responses. Due to a bad argument list, nginx worker would crash (SIGSEGV) while trying to log the fact that it received OCSP response with "revoked" or "unknown" certificate status. While there, fix similar (but non-crashing) error a few lines above. Signed-off-by: Piotr Sikora diffstat: src/event/ngx_event_openssl_stapling.c | 5 ++--- 1 files changed, 2 insertions(+), 3 deletions(-) diffs (21 lines): diff --git a/src/event/ngx_event_openssl_stapling.c b/src/event/ngx_event_openssl_stapling.c --- a/src/event/ngx_event_openssl_stapling.c +++ b/src/event/ngx_event_openssl_stapling.c @@ -611,15 +611,14 @@ ngx_ssl_stapling_ocsp_handler(ngx_ssl_oc != 1) { ngx_log_error(NGX_LOG_ERR, ctx->log, 0, - "certificate status not found in the OCSP response", - n, OCSP_response_status_str(n)); + "certificate status not found in the OCSP response"); goto error; } if (n != V_OCSP_CERTSTATUS_GOOD) { ngx_log_error(NGX_LOG_ERR, ctx->log, 0, "certificate status \"%s\" in the OCSP response", - n, OCSP_cert_status_str(n)); + OCSP_cert_status_str(n)); goto error; } From piotr at cloudflare.com Fri May 17 23:32:12 2013 From: piotr at cloudflare.com (Piotr Sikora) Date: Fri, 17 May 2013 16:32:12 -0700 Subject: [PATCH] OCSP stapling: better handling of successful OCSP responses. In-Reply-To: <20130517132026.GX69760@mdounin.ru> References: <20130517132026.GX69760@mdounin.ru> Message-ID: Hey Maxim, > Presenting a certificate and a non-good certificate status to a > user looks like "bees against honey" for me. I would rather not. While I agree that it looks kind of iffy, by not caching OCSP responses with "revoked" or "unknown" certificate status, we're loosing all of the OCSP stapling advantages (offloading CA's OCSP responders, improving user's privacy and perceived performance), while not changing anything for the user - he'll still receive exactly the same certificate status directly from CA's OCSP responder, just a few hundred milliseconds later. Best regards, Piotr Sikora From dp at highloadlab.com Sun May 19 21:34:26 2013 From: dp at highloadlab.com (Dmitry Popov) Date: Mon, 20 May 2013 01:34:26 +0400 Subject: [BUG?]fail_timeout/max_fails: code doesn't do what doc says Message-ID: <20130520013426.2ca62487f4d4aa6d6d7de494@highloadlab.com> Hi. http://wiki.nginx.org/HttpUpstreamModule says max_fails = NUMBER - number of unsuccessful attempts at communicating with the server within the time period (assigned by parameter fail_timeout) after which it is considered inoperative ... fail_timeout = TIME - the time during which must occur *max_fails* number of unsuccessful attempts at communication with the server that would cause the server to be considered inoperative ... However, as we may see from code (ngx_http_upstream_get_peer and ngx_http_upstream_free_round_robin_peer from src/http/ngx_http_upstream_round_robin.c) the logic is not as described: (simplified code) get_peer: if (fails >= max_fails && now <= fail_timeout + checked) skip ... checked = now free_peer: if (request_failed) fails++ accessed = now checked = now else if (accessed < checked) fails = 0 1) So, fail_timeout is never used while peer is gaining fails (until fails >= max_fails); 2) This algorithm always resets fails count if first request inside new second succeeds; it always increases fails count if first request fails. So, a lot depends on first (inside a second) request; I don't think it's a desired behaviour. 3) I'm not sure if "accessed" is a good name for a field that contains last fail timestamp. I don't know where an error is (in doc or code) and I don't know how you (nginx devs) wanted it to work so I don't have any constructive ideas, sorry. -- Dmitry Popov Highloadlab From mdounin at mdounin.ru Mon May 20 10:56:43 2013 From: mdounin at mdounin.ru (Maxim Dounin) Date: Mon, 20 May 2013 14:56:43 +0400 Subject: [PATCH] OCSP stapling: better handling of successful OCSP responses. In-Reply-To: References: <20130517132026.GX69760@mdounin.ru> Message-ID: <20130520105643.GZ69760@mdounin.ru> Hello! On Fri, May 17, 2013 at 04:32:12PM -0700, Piotr Sikora wrote: > Hey Maxim, > > > Presenting a certificate and a non-good certificate status to a > > user looks like "bees against honey" for me. I would rather not. > > While I agree that it looks kind of iffy, by not caching OCSP > responses with "revoked" or "unknown" certificate status, we're > loosing all of the OCSP stapling advantages (offloading CA's OCSP > responders, improving user's privacy and perceived performance), while > not changing anything for the user - he'll still receive exactly the > same certificate status directly from CA's OCSP responder, just a few > hundred milliseconds later. Yes, but this only happens if client actually does an OCSP request (and the user will query the same ocsp server, and response will be the same). >From site owner point of view - returning a non-good certificate status reduces probability of a client being served, and thus isn't a good idea. As this is exceptional situation - I don't think that site owner will care much about few milliseconds delay before a user sees a "bad certificate" browser's page. On the other hand, percentage of users who won't see the page (and will be able to work normally) usually matters. I understand that from a caching provider point of view this might look different though. You may consider adding a control for this, e.g., it looks like Apache has something similar called SSLStaplingReturnResponderErrors: https://httpd.apache.org/docs/trunk/mod/mod_ssl.html#sslstaplingreturnrespondererrors -- Maxim Dounin http://nginx.org/en/donation.html From fdasilvayy at gmail.com Mon May 20 16:40:14 2013 From: fdasilvayy at gmail.com (Filipe Da Silva) Date: Mon, 20 May 2013 18:40:14 +0200 Subject: [PATCH] ngx_mail_ssl_module / ecdh_curve Message-ID: Hello , Please find attach a two lines patch about ecdh_curve setting in mail_ssl_module. This setting is not applied, although it is declared and parsed from nginx.conf file I see this lack in 1.2.0 version too. Rgds, Filipe DA SILVA -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: 5199.patch Type: application/octet-stream Size: 753 bytes Desc: not available URL: From mdounin at mdounin.ru Mon May 20 17:11:03 2013 From: mdounin at mdounin.ru (Maxim Dounin) Date: Mon, 20 May 2013 21:11:03 +0400 Subject: [BUG?]fail_timeout/max_fails: code doesn't do what doc says In-Reply-To: <20130520013426.2ca62487f4d4aa6d6d7de494@highloadlab.com> References: <20130520013426.2ca62487f4d4aa6d6d7de494@highloadlab.com> Message-ID: <20130520171103.GG69760@mdounin.ru> Hello! On Mon, May 20, 2013 at 01:34:26AM +0400, Dmitry Popov wrote: > Hi. > > http://wiki.nginx.org/HttpUpstreamModule says > max_fails = NUMBER - number of unsuccessful attempts at communicating with the > server within the time period (assigned by parameter fail_timeout) after which > it is considered inoperative ... > fail_timeout = TIME - the time during which must occur *max_fails* number of > unsuccessful attempts at communication with the server that would cause the > server to be considered inoperative ... > > However, as we may see from code (ngx_http_upstream_get_peer and > ngx_http_upstream_free_round_robin_peer > from src/http/ngx_http_upstream_round_robin.c) the logic is not as described: > (simplified code) > get_peer: > if (fails >= max_fails && now <= fail_timeout + checked) > skip > ... > checked = now > free_peer: > if (request_failed) > fails++ > accessed = now > checked = now > else > if (accessed < checked) > fails = 0 > > 1) So, fail_timeout is never used while peer is gaining fails (until > fails >= max_fails); > 2) This algorithm always resets fails count if first request inside new second > succeeds; it always increases fails count if first request fails. So, a lot > depends on first (inside a second) request; I don't think it's a desired > behaviour. This looks like a bug introduced in 1.3.0, thanks. The peer->fails counter should be only reset after fail_timeout passed either since previous check or since previous failure. This was previously achieved by only bumping peer->checked once per fail_timeout, but was accedentally omitted in revision c90801720a0c. Correct code is still here in the ip_hash balancer. Patch: diff --git a/src/http/modules/ngx_http_upstream_least_conn_module.c b/src/http/modules/ngx_http_upstream_least_conn_module.c --- a/src/http/modules/ngx_http_upstream_least_conn_module.c +++ b/src/http/modules/ngx_http_upstream_least_conn_module.c @@ -282,7 +282,10 @@ ngx_http_upstream_get_least_conn_peer(ng } best->current_weight -= total; - best->checked = now; + + if (now - best->checked > best->fail_timeout) { + best->checked = now; + } pc->sockaddr = best->sockaddr; pc->socklen = best->socklen; diff --git a/src/http/ngx_http_upstream_round_robin.c b/src/http/ngx_http_upstream_round_robin.c --- a/src/http/ngx_http_upstream_round_robin.c +++ b/src/http/ngx_http_upstream_round_robin.c @@ -523,7 +523,10 @@ ngx_http_upstream_get_peer(ngx_http_upst rrp->tried[n] |= m; best->current_weight -= total; - best->checked = now; + + if (now - best->checked > best->fail_timeout) { + best->checked = now; + } return best; } -- Maxim Dounin http://nginx.org/en/donation.html From mdounin at mdounin.ru Mon May 20 17:25:16 2013 From: mdounin at mdounin.ru (Maxim Dounin) Date: Mon, 20 May 2013 21:25:16 +0400 Subject: [PATCH] ngx_mail_ssl_module / ecdh_curve In-Reply-To: References: Message-ID: <20130520172516.GH69760@mdounin.ru> Hello! On Mon, May 20, 2013 at 06:40:14PM +0200, Filipe Da Silva wrote: > Hello , > > Please find attach a two lines patch about ecdh_curve setting in > mail_ssl_module. > > This setting is not applied, although it is declared and parsed from > nginx.conf file > > I see this lack in 1.2.0 version too. Nice catch, thanks. > > Rgds, > Filipe DA SILVA > # HG changeset patch > # User fds Could you please use a proper name here? Just "fds" looks wierd. > # Date 1368197625 -7200 > # Node ID f6c3e219648f47eaa65ab43a8189af31f7babc69 > # Parent 7ffe7f3be4045dbc7478248370b473271660e473 > Missing call to ngx_ssl_ecdh_curve in mail_ssl_module Please use trailing dot and "Mail:" prefix for mail-related changes. That is, I would like to see something like this here: Mail: missing ngx_ssl_ecdh_curve() call. > > diff -r 7ffe7f3be404 -r f6c3e219648f src/mail/ngx_mail_ssl_module.c > --- a/src/mail/ngx_mail_ssl_module.c Thu May 09 10:54:28 2013 +0200 > +++ b/src/mail/ngx_mail_ssl_module.c Fri May 10 16:53:45 2013 +0200 > @@ -308,6 +308,10 @@ > return NGX_CONF_ERROR; > } > > + if (ngx_ssl_ecdh_curve(cf, &conf->ssl, &conf->ecdh_curve) != NGX_OK) { > + return NGX_CONF_ERROR; > + } > + > ngx_conf_merge_value(conf->builtin_session_cache, > prev->builtin_session_cache, NGX_SSL_NONE_SCACHE); > -- Maxim Dounin http://nginx.org/en/donation.html From fdasilvayy at gmail.com Mon May 20 21:37:39 2013 From: fdasilvayy at gmail.com (Filipe Da Silva) Date: Mon, 20 May 2013 23:37:39 +0200 Subject: [PATCH] ngx_mail_ssl_module / ecdh_curve In-Reply-To: <20130520172516.GH69760@mdounin.ru> References: <20130520172516.GH69760@mdounin.ru> Message-ID: Hi, Here is the fixed version. Rgds. Filipe DA SILVA 2013/5/20 Maxim Dounin > Hello! > > On Mon, May 20, 2013 at 06:40:14PM +0200, Filipe Da Silva wrote: > > > Hello , > > > > Please find attach a two lines patch about ecdh_curve setting in > > mail_ssl_module. > > > > This setting is not applied, although it is declared and parsed from > > nginx.conf file > > > > I see this lack in 1.2.0 version too. > > Nice catch, thanks. > > > > > Rgds, > > Filipe DA SILVA > > > # HG changeset patch > > # User fds > > Could you please use a proper name here? Just "fds" looks wierd. > > > # Date 1368197625 -7200 > > # Node ID f6c3e219648f47eaa65ab43a8189af31f7babc69 > > # Parent 7ffe7f3be4045dbc7478248370b473271660e473 > > Missing call to ngx_ssl_ecdh_curve in mail_ssl_module > > Please use trailing dot and "Mail:" prefix for mail-related > changes. > > That is, I would like to see something like this here: > > Mail: missing ngx_ssl_ecdh_curve() call. > > > > > diff -r 7ffe7f3be404 -r f6c3e219648f src/mail/ngx_mail_ssl_module.c > > --- a/src/mail/ngx_mail_ssl_module.c Thu May 09 10:54:28 2013 +0200 > > +++ b/src/mail/ngx_mail_ssl_module.c Fri May 10 16:53:45 2013 +0200 > > @@ -308,6 +308,10 @@ > > return NGX_CONF_ERROR; > > } > > > > + if (ngx_ssl_ecdh_curve(cf, &conf->ssl, &conf->ecdh_curve) != > NGX_OK) { > > + return NGX_CONF_ERROR; > > + } > > + > > ngx_conf_merge_value(conf->builtin_session_cache, > > prev->builtin_session_cache, > NGX_SSL_NONE_SCACHE); > > > > -- > Maxim Dounin > http://nginx.org/en/donation.html > -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: 5216.patch Type: application/octet-stream Size: 783 bytes Desc: not available URL: From fdasilvayy at gmail.com Mon May 20 21:55:48 2013 From: fdasilvayy at gmail.com (Filipe Da Silva) Date: Mon, 20 May 2013 23:55:48 +0200 Subject: [PATCH] Limit req: pointer init order issue Message-ID: Hello, It's a tiny issue as only the next test code about 'shm' is broken. That's my last patch for the moment. Regards. Filipe DA SILVA -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: 5219.patch Type: application/octet-stream Size: 1021 bytes Desc: not available URL: From mdounin at mdounin.ru Mon May 20 22:28:33 2013 From: mdounin at mdounin.ru (Maxim Dounin) Date: Tue, 21 May 2013 02:28:33 +0400 Subject: [PATCH] Limit req: pointer init order issue In-Reply-To: References: Message-ID: <20130520222833.GP69760@mdounin.ru> Hello! On Mon, May 20, 2013 at 11:55:48PM +0200, Filipe Da Silva wrote: > Hello, > > It's a tiny issue as only the next test code about 'shm' is broken. > > That's my last patch for the moment. > > Regards. > Filipe DA SILVA > # HG changeset patch > # User F. da Silva > # Date 1368198460 -7200 > # Fri May 10 17:07:40 2013 +0200 > # Node ID 44310812509300443b296e4ff0292e0934305a55 > # Parent 8abca47e7ca0537000bbbd307eefdd033a9971e7 > Limit req: pointer init order issue. > > diff -r 8abca47e7ca0 -r 443108125093 src/http/modules/ngx_http_limit_req_module.c > --- a/src/http/modules/ngx_http_limit_req_module.c Fri May 10 16:53:45 2013 +0200 > +++ b/src/http/modules/ngx_http_limit_req_module.c Fri May 10 17:07:40 2013 +0200 > @@ -936,9 +936,7 @@ > return NGX_CONF_ERROR; > } > > - limits = lrcf->limits.elts; > - > - if (limits == NULL) { > + if (lrcf->limits.elts == NULL) { > if (ngx_array_init(&lrcf->limits, cf->pool, 1, > sizeof(ngx_http_limit_req_limit_t)) > != NGX_OK) > @@ -947,6 +945,8 @@ > } > } > > + limits = lrcf->limits.elts; > + > for (i = 0; i < lrcf->limits.nelts; i++) { > if (shm_zone == limits[i].shm_zone) { > return "is duplicate"; There is no problem here: if lrcf->limits.elts == NULL (aka limits == NULL), then lrcf->limits.nelts == 0 and the limits pointer is never dereferenced. The patch is not needed. -- Maxim Dounin http://nginx.org/en/donation.html From mdounin at mdounin.ru Mon May 20 23:31:39 2013 From: mdounin at mdounin.ru (Maxim Dounin) Date: Tue, 21 May 2013 03:31:39 +0400 Subject: [PATCH] ngx_mail_ssl_module / ecdh_curve In-Reply-To: References: <20130520172516.GH69760@mdounin.ru> Message-ID: <20130520233139.GQ69760@mdounin.ru> Hello! On Mon, May 20, 2013 at 11:37:39PM +0200, Filipe Da Silva wrote: [...] > > > # HG changeset patch > > > # User fds > > > > Could you please use a proper name here? Just "fds" looks wierd. > > > > > # Date 1368197625 -7200 > > > # Node ID f6c3e219648f47eaa65ab43a8189af31f7babc69 > > > # Parent 7ffe7f3be4045dbc7478248370b473271660e473 > > > Missing call to ngx_ssl_ecdh_curve in mail_ssl_module > > > > Please use trailing dot and "Mail:" prefix for mail-related > > changes. > > > > That is, I would like to see something like this here: > > > > Mail: missing ngx_ssl_ecdh_curve() call. [...] > # HG changeset patch > # User F. da Silva At least space after a name seems to be missing. What about "Filipe Da Silva ", as used in the From address of your messages? > # Date 1368197625 -7200 > # Fri May 10 16:53:45 2013 +0200 > # Node ID 24369ba28ede7baea634de2dfbab899925830f34 > # Parent cfab1e7e4ac2f0d17199ee1d49ac4647b63746d3 > Mail: Missing ngx_ssl_ecdh_curve call. No capitalization after after a prefix, please. Parentheses after a function name is also generally good idea to explicitly mark functions. That is, I would like to see what was previously suggested: Mail: missing ngx_ssl_ecdh_curve() call. (Please let me know if you'll prefer me to fix it myself. It would be better if you'll fix it yourself though - especially if you are planning to submit more patches.) (Please also consider switching off html in your messages to nginx-devel at . Thank you.) -- Maxim Dounin http://nginx.org/en/donation.html From mdounin at mdounin.ru Tue May 21 00:26:33 2013 From: mdounin at mdounin.ru (Maxim Dounin) Date: Tue, 21 May 2013 04:26:33 +0400 Subject: [PATCH] Add directive to always gunzip content In-Reply-To: References: Message-ID: <20130521002632.GR69760@mdounin.ru> Hello! On Sun, May 05, 2013 at 08:50:06AM +0300, Roy Reznik wrote: > Use this by adding "gunzip_always on;" to configuration. > This was in the modules to-do list. > Cheers. [...] > @@ -140,13 +148,16 @@ > > r->gzip_vary = 1; > > - if (!r->gzip_tested) { > - if (ngx_http_gzip_ok(r) == NGX_OK) { > + /* if configured to gunzip always, don't check request headers */ What's the point of adding Vary header then? See also here for more details: http://mailman.nginx.org/pipermail/nginx-devel/2013-January/003284.html -- Maxim Dounin http://nginx.org/en/donation.html From pingu at anchor.net.au Tue May 21 03:08:43 2013 From: pingu at anchor.net.au (Christian Marie) Date: Tue, 21 May 2013 13:08:43 +1000 Subject: [PATCH] New variable: $ssl_sni_host Message-ID: <20130521030843.GA12890@cucumber.bridge.anchor.net.au> Patch attached adds a new variable, $ssl_sni_host. I would find this quite useful as there is no other way of knowing for sure which host a request is directed at (at the SSL layer), as the HTTP HOST header can be wrong. Possibly somewhat related to: http://trac.nginx.org/nginx/ticket/229 I should mention that I don't intend for this to be a drop in replacement for $http_host, though that could very well work with proxy_pass. -------------- next part -------------- # HG changeset patch # User Christian Marie # Date 1369104447 -36000 # Tue May 21 12:47:27 2013 +1000 # Node ID aad7765348785a6e3958ea7594dc77e67a1119f5 # Parent cfab1e7e4ac2f0d17199ee1d49ac4647b63746d3 Add new $ssl_sni_host ngx_http_ssl_variable. Useful when using SNI and multiple vhosts or proxy backends, as $http_host can be entirely arbitrary. diff -r cfab1e7e4ac2 -r aad776534878 src/event/ngx_event_openssl.c --- a/src/event/ngx_event_openssl.c Thu May 16 15:37:13 2013 -0700 +++ b/src/event/ngx_event_openssl.c Tue May 21 12:47:27 2013 +1000 @@ -2496,6 +2496,33 @@ } +#ifdef SSL_CTRL_SET_TLSEXT_HOSTNAME +ngx_int_t +ngx_ssl_get_sni_host(ngx_connection_t *c, ngx_pool_t *pool, ngx_str_t *s) +{ + const char *host; + + host = SSL_get_servername(c->ssl->connection, TLSEXT_NAMETYPE_host_name); + if (host == NULL) { + return NGX_ERROR; + } + + s->len = ngx_strlen(host); + if (s->len == 0) { + return NGX_ERROR; + } + + s->data = ngx_pnalloc(pool, s->len); + if(s->data == NULL) { + return NGX_ERROR; + } + + ngx_memcpy(s->data, host, s->len); + return NGX_OK; +} +#endif + + static void * ngx_openssl_create_conf(ngx_cycle_t *cycle) { diff -r cfab1e7e4ac2 -r aad776534878 src/event/ngx_event_openssl.h --- a/src/event/ngx_event_openssl.h Thu May 16 15:37:13 2013 -0700 +++ b/src/event/ngx_event_openssl.h Tue May 21 12:47:27 2013 +1000 @@ -153,6 +153,10 @@ ngx_str_t *s); ngx_int_t ngx_ssl_get_client_verify(ngx_connection_t *c, ngx_pool_t *pool, ngx_str_t *s); +#ifdef SSL_CTRL_SET_TLSEXT_HOSTNAME +ngx_int_t ngx_ssl_get_sni_host(ngx_connection_t *c, ngx_pool_t *pool, + ngx_str_t *s); +#endif ngx_int_t ngx_ssl_handshake(ngx_connection_t *c); diff -r cfab1e7e4ac2 -r aad776534878 src/http/modules/ngx_http_ssl_module.c --- a/src/http/modules/ngx_http_ssl_module.c Thu May 16 15:37:13 2013 -0700 +++ b/src/http/modules/ngx_http_ssl_module.c Tue May 21 12:47:27 2013 +1000 @@ -260,6 +260,11 @@ { ngx_string("ssl_client_verify"), NULL, ngx_http_ssl_variable, (uintptr_t) ngx_ssl_get_client_verify, NGX_HTTP_VAR_CHANGEABLE, 0 }, +#ifdef SSL_CTRL_SET_TLSEXT_HOSTNAME + { ngx_string("ssl_sni_host"), NULL, ngx_http_ssl_variable, + (uintptr_t) ngx_ssl_get_sni_host, NGX_HTTP_VAR_CHANGEABLE, 0 }, +#endif + { ngx_null_string, NULL, NULL, 0, 0, 0 } }; From jincm at neusoft.com Tue May 21 05:13:22 2013 From: jincm at neusoft.com (=?gb2312?B?vfm0usPP?=) Date: Tue, 21 May 2013 13:13:22 +0800 Subject: unsubscribe References: <2013052113104741948050@neusoft.com> Message-ID: <2013052113132200292552@neusoft.com> unsubscribe --------------------------------------------------------------------------------------------------- Confidentiality Notice: The information contained in this e-mail and any accompanying attachment(s) is intended only for the use of the intended recipient and may be confidential and/or privileged of Neusoft Corporation, its subsidiaries and/or its affiliates. If any reader of this communication is not the intended recipient, unauthorized use, forwarding, printing, storing, disclosure or copying is strictly prohibited, and may be unlawful.If you have received this communication in error,please immediately notify the sender by return e-mail, and delete the original message and all copies from your system. Thank you. --------------------------------------------------------------------------------------------------- -------------- next part -------------- An HTML attachment was scrubbed... URL: From jincm at neusoft.com Tue May 21 05:14:57 2013 From: jincm at neusoft.com (=?gb2312?B?vfm0usPP?=) Date: Tue, 21 May 2013 13:14:57 +0800 Subject: unsubscribe Message-ID: <2013052113145659466654@neusoft.com> unsubscribe ??? --------------------------------------------------------------------------------------------------- Confidentiality Notice: The information contained in this e-mail and any accompanying attachment(s) is intended only for the use of the intended recipient and may be confidential and/or privileged of Neusoft Corporation, its subsidiaries and/or its affiliates. If any reader of this communication is not the intended recipient, unauthorized use, forwarding, printing, storing, disclosure or copying is strictly prohibited, and may be unlawful.If you have received this communication in error,please immediately notify the sender by return e-mail, and delete the original message and all copies from your system. Thank you. --------------------------------------------------------------------------------------------------- -------------- next part -------------- An HTML attachment was scrubbed... URL: From fdasilvayy at gmail.com Tue May 21 06:39:45 2013 From: fdasilvayy at gmail.com (Filipe Da Silva) Date: Tue, 21 May 2013 08:39:45 +0200 Subject: [PATCH] ngx_mail_ssl_module / ecdh_curve In-Reply-To: <20130520233139.GQ69760@mdounin.ru> References: <20130520172516.GH69760@mdounin.ru> <20130520233139.GQ69760@mdounin.ru> Message-ID: Hi. I fix it. If it's still not the case, please do. Thank for your patience. Rgds. Filipe 2013/5/21 Maxim Dounin : > Hello! > > On Mon, May 20, 2013 at 11:37:39PM +0200, Filipe Da Silva wrote: > > [...] > >> > > # HG changeset patch >> > > # User fds >> > >> > Could you please use a proper name here? Just "fds" looks wierd. >> > >> > > # Date 1368197625 -7200 >> > > # Node ID f6c3e219648f47eaa65ab43a8189af31f7babc69 >> > > # Parent 7ffe7f3be4045dbc7478248370b473271660e473 >> > > Missing call to ngx_ssl_ecdh_curve in mail_ssl_module >> > >> > Please use trailing dot and "Mail:" prefix for mail-related >> > changes. >> > >> > That is, I would like to see something like this here: >> > >> > Mail: missing ngx_ssl_ecdh_curve() call. > > [...] > >> # HG changeset patch >> # User F. da Silva > > At least space after a name seems to be missing. > > What about "Filipe Da Silva ", as used in > the From address of your messages? > >> # Date 1368197625 -7200 >> # Fri May 10 16:53:45 2013 +0200 >> # Node ID 24369ba28ede7baea634de2dfbab899925830f34 >> # Parent cfab1e7e4ac2f0d17199ee1d49ac4647b63746d3 >> Mail: Missing ngx_ssl_ecdh_curve call. > > No capitalization after after a prefix, please. Parentheses after > a function name is also generally good idea to explicitly mark > functions. That is, I would like to see what was previously > suggested: > > Mail: missing ngx_ssl_ecdh_curve() call. > > (Please let me know if you'll prefer me to fix it myself. It > would be better if you'll fix it yourself though - especially if > you are planning to submit more patches.) > > (Please also consider switching off html in your messages to > nginx-devel at . Thank you.) > > -- > Maxim Dounin > http://nginx.org/en/donation.html -------------- next part -------------- A non-text attachment was scrubbed... Name: 5216-2.patch Type: application/octet-stream Size: 786 bytes Desc: not available URL: From ru at nginx.com Tue May 21 08:59:37 2013 From: ru at nginx.com (Ruslan Ermilov) Date: Tue, 21 May 2013 08:59:37 +0000 Subject: [nginx] Upstream: made the assignment more obvious. Message-ID: details: http://hg.nginx.org/nginx/rev/4a40163772a1 branches: changeset: 5216:4a40163772a1 user: Ruslan Ermilov date: Tue May 21 12:54:26 2013 +0400 description: Upstream: made the assignment more obvious. No functional changes. diffstat: src/http/ngx_http_upstream.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diffs (12 lines): diff -r cfab1e7e4ac2 -r 4a40163772a1 src/http/ngx_http_upstream.c --- a/src/http/ngx_http_upstream.c Thu May 16 15:37:13 2013 -0700 +++ b/src/http/ngx_http_upstream.c Tue May 21 12:54:26 2013 +0400 @@ -1701,7 +1701,7 @@ ngx_http_upstream_process_header(ngx_htt n = u->buffer.last - u->buffer.pos; if (n) { - u->buffer.last -= n; + u->buffer.last = u->buffer.pos; u->state->response_length += n; From ru at nginx.com Tue May 21 08:59:37 2013 From: ru at nginx.com (Ruslan Ermilov) Date: Tue, 21 May 2013 08:59:37 +0000 Subject: [nginx] Upstream: slightly optimized ngx_http_upstream_process_h... Message-ID: details: http://hg.nginx.org/nginx/rev/ddba4e308ecc branches: changeset: 5217:ddba4e308ecc user: Ruslan Ermilov date: Tue May 21 12:54:27 2013 +0400 description: Upstream: slightly optimized ngx_http_upstream_process_header(). diffstat: src/http/ngx_http_upstream.c | 10 +++++----- 1 files changed, 5 insertions(+), 5 deletions(-) diffs (20 lines): diff -r 4a40163772a1 -r ddba4e308ecc src/http/ngx_http_upstream.c --- a/src/http/ngx_http_upstream.c Tue May 21 12:54:26 2013 +0400 +++ b/src/http/ngx_http_upstream.c Tue May 21 12:54:27 2013 +0400 @@ -1709,11 +1709,11 @@ ngx_http_upstream_process_header(ngx_htt ngx_http_upstream_finalize_request(r, u, NGX_ERROR); return; } - - if (u->length == 0) { - ngx_http_upstream_finalize_request(r, u, 0); - return; - } + } + + if (u->length == 0) { + ngx_http_upstream_finalize_request(r, u, 0); + return; } u->read_event_handler = ngx_http_upstream_process_body_in_memory; From dp at highloadlab.com Tue May 21 12:16:05 2013 From: dp at highloadlab.com (Dmitry Popov) Date: Tue, 21 May 2013 16:16:05 +0400 Subject: [BUG?]fail_timeout/max_fails: code doesn't do what doc says In-Reply-To: <20130520171103.GG69760@mdounin.ru> References: <20130520013426.2ca62487f4d4aa6d6d7de494@highloadlab.com> <20130520171103.GG69760@mdounin.ru> Message-ID: <20130521161605.fbd3e3d1c8cb30db69215823@highloadlab.com> On Mon, 20 May 2013 21:11:03 +0400 Maxim Dounin wrote: > Patch: > > diff --git a/src/http/modules/ngx_http_upstream_least_conn_module.c b/src/http/modules/ngx_http_upstream_least_conn_module.c > --- a/src/http/modules/ngx_http_upstream_least_conn_module.c > +++ b/src/http/modules/ngx_http_upstream_least_conn_module.c > @@ -282,7 +282,10 @@ ngx_http_upstream_get_least_conn_peer(ng > } > > best->current_weight -= total; > - best->checked = now; > + > + if (now - best->checked > best->fail_timeout) { > + best->checked = now; > + } > > pc->sockaddr = best->sockaddr; > pc->socklen = best->socklen; > diff --git a/src/http/ngx_http_upstream_round_robin.c b/src/http/ngx_http_upstream_round_robin.c > --- a/src/http/ngx_http_upstream_round_robin.c > +++ b/src/http/ngx_http_upstream_round_robin.c > @@ -523,7 +523,10 @@ ngx_http_upstream_get_peer(ngx_http_upst > rrp->tried[n] |= m; > > best->current_weight -= total; > - best->checked = now; > + > + if (now - best->checked > best->fail_timeout) { > + best->checked = now; > + } > > return best; > } Still a bug exists because of this code: if (state & NGX_PEER_FAILED) { /* ... */ peer->fails++; peer->accessed = now; peer->checked = now; /* .... */ } else { if (peer->accessed < peer->checked) { peer->fails = 0; } } Consider upstream which fails at least once each second (actually it's enough to fail each fail_timeout seconds). This upstream will be disabled as soon as it will fail max_fails times no matter what fail_timeout is. I propose the following solution: let's have two bits per peer: 1) avail - set if this peer is available for usage (fails < max_fails) 2) avail_check - set if this peer was disabled, but its fail_timeout is expired and we sent one request to check if it's alive and two timestamps: 3) first_fail - we'll count number of fails only inside [first_fail, first_fail + fail_timeout) interval 4) unavail_from - peer is unavailable inside [unavail_from, unavail_from + fail_timeout) interval first_fail will only be used if avail == 1, unavail_from only if avail == 0, so we may use union here. Simplified code: get: if (!avail && (now < unavail_from + fail_timeout || avail_check)) { continue; } if (!avail) avail_check =1; free: if (avail && now >= first_fail + fail_timeout) fails = 0; if (request_failed) { if (fails == 0) first_fail = now; fails++; if (fails >= max_fails) { avail = 0; unavail_from = now; } avail_check = 0; } else { if (avail_check) { fails = 0; avail = 1; avail_check = 0; } } Also I think it's a good idea to split fail_timeout to fail_timeout and disable_time: we'll count fails during fail_timeout and disable upstream for disable_time. patch with bugfix: Upstreams bugfix: fail_timeout could be ignored Upstream could be disabled no matter what fail_timeout is (if requests fail each fail_timeout seconds). Signed-off-by: Dmitry Popov diff -ur old/src/http/ngx_http_upstream_round_robin.h new/src/http/ngx_http_upstream_round_robin.h --- old/src/http/ngx_http_upstream_round_robin.h 2013-05-21 13:22:09.478903537 +0400 +++ new/src/http/ngx_http_upstream_round_robin.h 2013-05-21 15:38:05.488236353 +0400 @@ -24,13 +24,17 @@ ngx_int_t weight; ngx_uint_t fails; - time_t accessed; - time_t checked; + union { + time_t first_fail; + time_t unavail_from; + }; ngx_uint_t max_PCRE: retain input pattern for all regular expressions. Previously, input pattern was kept only for regular expressions with named captures, which resulted in error log entries without input pattern for PCRE errors that occured while processing regular expressions without them. Signed-off-by: Piotr Sikora fails; time_t fail_timeout; - ngx_uint_t down; /* unsigned down:1; */ + unsigned avail:1; + unsigned avail_check:1; + unsigned down:1; #if (NGX_HTTP_SSL) ngx_ssl_session_t *ssl_session; /* local to a process */ diff -ur old/src/http/modules/ngx_http_upstream_ip_hash_module.c new/src/http/modules/ngx_http_upstream_ip_hash_module.c --- old/src/http/modules/ngx_http_upstream_ip_hash_module.c 2013-05-21 13:22:09.475570203 +0400 +++ new/src/http/modules/ngx_http_upstream_ip_hash_module.c 2013-05-21 15:44:22.671564821 +0400 @@ -208,12 +208,14 @@ if (!peer->down) { - if (peer->max_fails == 0 || peer->fails < peer->max_fails) { + if (peer->avail) { break; } - if (now - peer->checked > peer->fail_timeout) { - peer->checked = now; + if (now - peer->unavail_from >= peer->fail_timeout + && !peer->avail_check) + { + peer->avail_check = 1; break; } } diff -ur old/src/http/modules/ngx_http_upstream_least_conn_module.c new/src/http/modules/ngx_http_upstream_least_conn_module.c --- old/src/http/modules/ngx_http_upstream_least_conn_module.c 2013-05-21 13:22:09.472236869 +0400 +++ new/src/http/modules/ngx_http_upstream_least_conn_module.c 2013-05-21 15:45:35.254897219 +0400 @@ -203,9 +203,9 @@ continue; } - if (peer->max_fails - && peer->fails >= peer->max_fails - && now - peer->checked <= peer->fail_timeout) + if (!peer->avail + && (now - peer->unavail_from < peer->fail_timeout + || peer->avail_check)) { continue; } @@ -260,9 +260,9 @@ continue; } - if (peer->max_fails - && peer->fails >= peer->max_fails - && now - peer->checked <= peer->fail_timeout) + if (!peer->avail + && (now - peer->unavail_from < peer->fail_timeout + || peer->avail_check)) { continue; } @@ -282,7 +282,9 @@ } best->current_weight -= total; - best->checked = now; + if (!best->avail) { + best->avail_check = 1; + } pc->sockaddr = best->sockaddr; pc->socklen = best->socklen; @@ -331,6 +333,9 @@ for (i = 0; i < peers->number; i++) { peers->peer[i].fails = 0; + peers->peer[i].avail = 1; + peers->peer[i].avail_check = 0; + /* peers->peer[i].first_fail = 0; */ } pc->name = peers->name; diff -ur old/src/http/ngx_http_upstream_round_robin.c new/src/http/ngx_http_upstream_round_robin.c --- old/src/http/ngx_http_upstream_round_robin.c 2013-05-21 13:22:09.472236869 +0400 +++ new/src/http/ngx_http_upstream_round_robin.c 2013-05-21 15:46:07.171563474 +0400 @@ -85,6 +85,8 @@ peers->peer[n].weight = server[i].weight; peers->peer[n].effective_weight = server[i].weight; peers->peer[n].current_weight = 0; + peers->peer[n].avail = 1; + peers->peer[n].avail_check = 0; n++; } } @@ -139,6 +141,8 @@ backup->peer[n].max_fails = server[i].max_fails; backup->peer[n].fail_timeout = server[i].fail_timeout; backup->peer[n].down = server[i].down; + backup->peer[n].avail = 1; + backup->peer[n].avail_check = 0; n++; } } @@ -196,6 +200,8 @@ peers->peer[i].current_weight = 0; peers->peer[i].max_fails = 1; peers->peer[i].fail_timeout = 10; + peers->peer[i].avail = 1; + peers->peer[i].avail_check = 0; } us->peer.data = peers; @@ -301,6 +307,8 @@ peers->peer[0].current_weight = 0; peers->peer[0].max_fails = 1; peers->peer[0].fail_timeout = 10; + peers->peer[0].avail = 1; + peers->peer[0].avail_check = 0; } else { @@ -334,6 +342,8 @@ peers->peer[i].current_weight = 0; peers->peer[i].max_fails = 1; peers->peer[i].fail_timeout = 10; + peers->peer[i].avail = 1; + peers->peer[i].avail_check = 0; } } @@ -451,6 +461,9 @@ for (i = 0; i < peers->number; i++) { peers->peer[i].fails = 0; + peers->peer[i].avail = 1; + peers->peer[i].avail_check = 0; + /* peers->peer[i].first_fail = 0; */ } /* ngx_unlock_mutex(peers->mutex); */ @@ -490,9 +503,9 @@ continue; } - if (peer->max_fails - && peer->fails >= peer->max_fails - && now - peer->checked <= peer->fail_timeout) + if (!peer->avail + && (now - peer->unavail_from < peer->fail_timeout + || peer->avail_check)) { continue; } @@ -523,7 +536,9 @@ rrp->tried[n] |= m; best->current_weight -= total; - best->checked = now; + if (!best->avail) { + best->avail_check = 1; + } return best; } @@ -550,35 +565,49 @@ peer = &rrp->peers->peer[rrp->current]; - if (state & NGX_PEER_FAILED) { + if (peer->max_fails) { now = ngx_time(); - /* ngx_lock_mutex(rrp->peers->mutex); */ + if (peer->avail && now - peer->first_fail >= peer->fail_timeout) { + peer->fails = 0; + } + + if (state & NGX_PEER_FAILED) { + /* ngx_lock_mutex(rrp->peers->mutex); */ + + if (peer->fails++ == 0) { + peer->first_fail = now; + } + + if (peer->fails >= peer->max_fails) { + peer->avail = 0; + peer->unavail_from = now; + } - peer->fails++; - peer->accessed = now; - peer->checked = now; + peer->avail_check = 0; - if (peer->max_fails) { peer->effective_weight -= peer->weight / peer->max_fails; - } - ngx_log_debug2(NGX_LOG_DEBUG_HTTP, pc->log, 0, - "free rr peer failed: %ui %i", - rrp->current, peer->effective_weight); + ngx_log_debug2(NGX_LOG_DEBUG_HTTP, pc->log, 0, + "free rr peer failed: %ui %i", + rrp->current, peer->effective_weight); - if (peer->effective_weight < 0) { - peer->effective_weight = 0; - } + if (peer->effective_weight < 0) { + peer->effective_weight = 0; + } - /* ngx_unlock_mutex(rrp->peers->mutex); */ + /* ngx_unlock_mutex(rrp->peers->mutex); */ - } else { + } else { - /* mark peer live if check passed */ + /* mark peer live if check passed */ - if (peer->accessed < peer->checked) { - peer->fails = 0; + if (peer->avail_check) { + peer->fails = 0; + peer->avail = 1; + peer->avail_check = 0; + /* peers->peer[i].first_fail = 0; */ + } } } -- Dmitry Popov Highloadlab From dp at highloadlab.com Tue May 21 12:43:31 2013 From: dp at highloadlab.com (Dmitry Popov) Date: Tue, 21 May 2013 16:43:31 +0400 Subject: [BUG?]fail_timeout/max_fails: code doesn't do what doc says In-Reply-To: <20130521161605.fbd3e3d1c8cb30db69215823@highloadlab.com> References: <20130520013426.2ca62487f4d4aa6d6d7de494@highloadlab.com> <20130520171103.GG69760@mdounin.ru> <20130521161605.fbd3e3d1c8cb30db69215823@highloadlab.com> Message-ID: <20130521164331.b02b1edce5f725fb818faa12@highloadlab.com> Sorry, clipboard issue, corrupted patch send, here's a good one: Upstream bugfix: fail_timeout could be ignored Upstream could be disabled no matter what fail_timeout is (if requests fail each fail_timeout seconds). Signed-off-by: Dmitry Popov diff -ur old/src/http/ngx_http_upstream_round_robin.h new/src/http/ngx_http_upstream_round_robin.h --- old/src/http/ngx_http_upstream_round_robin.h 2013-05-21 13:22:09.478903537 +0400 +++ new/src/http/ngx_http_upstream_round_robin.h 2013-05-21 15:38:05.488236353 +0400 @@ -24,13 +24,17 @@ ngx_int_t weight; ngx_uint_t fails; - time_t accessed; - time_t checked; + union { + time_t first_fail; + time_t unavail_from; + }; ngx_uint_t max_fails; time_t fail_timeout; - ngx_uint_t down; /* unsigned down:1; */ + unsigned avail:1; + unsigned avail_check:1; + unsigned down:1; #if (NGX_HTTP_SSL) ngx_ssl_session_t *ssl_session; /* local to a process */ diff -ur old/src/http/modules/ngx_http_upstream_ip_hash_module.c new/src/http/modules/ngx_http_upstream_ip_hash_module.c --- old/src/http/modules/ngx_http_upstream_ip_hash_module.c 2013-05-21 13:22:09.475570203 +0400 +++ new/src/http/modules/ngx_http_upstream_ip_hash_module.c 2013-05-21 15:44:22.671564821 +0400 @@ -208,12 +208,14 @@ if (!peer->down) { - if (peer->max_fails == 0 || peer->fails < peer->max_fails) { + if (peer->avail) { break; } - if (now - peer->checked > peer->fail_timeout) { - peer->checked = now; + if (now - peer->unavail_from >= peer->fail_timeout + && !peer->avail_check) + { + peer->avail_check = 1; break; } } diff -ur old/src/http/modules/ngx_http_upstream_least_conn_module.c new/src/http/modules/ngx_http_upstream_least_conn_module.c --- old/src/http/modules/ngx_http_upstream_least_conn_module.c 2013-05-21 13:22:09.472236869 +0400 +++ new/src/http/modules/ngx_http_upstream_least_conn_module.c 2013-05-21 15:45:35.254897219 +0400 @@ -203,9 +203,9 @@ continue; } - if (peer->max_fails - && peer->fails >= peer->max_fails - && now - peer->checked <= peer->fail_timeout) + if (!peer->avail + && (now - peer->unavail_from < peer->fail_timeout + || peer->avail_check)) { continue; } @@ -260,9 +260,9 @@ continue; } - if (peer->max_fails - && peer->fails >= peer->max_fails - && now - peer->checked <= peer->fail_timeout) + if (!peer->avail + && (now - peer->unavail_from < peer->fail_timeout + || peer->avail_check)) { continue; } @@ -282,7 +282,9 @@ } best->current_weight -= total; - best->checked = now; + if (!best->avail) { + best->avail_check = 1; + } pc->sockaddr = best->sockaddr; pc->socklen = best->socklen; @@ -331,6 +333,9 @@ for (i = 0; i < peers->number; i++) { peers->peer[i].fails = 0; + peers->peer[i].avail = 1; + peers->peer[i].avail_check = 0; + /* peers->peer[i].first_fail = 0; */ } pc->name = peers->name; diff -ur old/src/http/ngx_http_upstream_round_robin.c new/src/http/ngx_http_upstream_round_robin.c --- old/src/http/ngx_http_upstream_round_robin.c 2013-05-21 13:22:09.472236869 +0400 +++ new/src/http/ngx_http_upstream_round_robin.c 2013-05-21 15:46:07.171563474 +0400 @@ -85,6 +85,8 @@ peers->peer[n].weight = server[i].weight; peers->peer[n].effective_weight = server[i].weight; peers->peer[n].current_weight = 0; + peers->peer[n].avail = 1; + peers->peer[n].avail_check = 0; n++; } } @@ -139,6 +141,8 @@ backup->peer[n].max_fails = server[i].max_fails; backup->peer[n].fail_timeout = server[i].fail_timeout; backup->peer[n].down = server[i].down; + backup->peer[n].avail = 1; + backup->peer[n].avail_check = 0; n++; } } @@ -196,6 +200,8 @@ peers->peer[i].current_weight = 0; peers->peer[i].max_fails = 1; peers->peer[i].fail_timeout = 10; + peers->peer[i].avail = 1; + peers->peer[i].avail_check = 0; } us->peer.data = peers; @@ -301,6 +307,8 @@ peers->peer[0].current_weight = 0; peers->peer[0].max_fails = 1; peers->peer[0].fail_timeout = 10; + peers->peer[0].avail = 1; + peers->peer[0].avail_check = 0; } else { @@ -334,6 +342,8 @@ peers->peer[i].current_weight = 0; peers->peer[i].max_fails = 1; peers->peer[i].fail_timeout = 10; + peers->peer[i].avail = 1; + peers->peer[i].avail_check = 0; } } @@ -451,6 +461,9 @@ for (i = 0; i < peers->number; i++) { peers->peer[i].fails = 0; + peers->peer[i].avail = 1; + peers->peer[i].avail_check = 0; + /* peers->peer[i].first_fail = 0; */ } /* ngx_unlock_mutex(peers->mutex); */ @@ -490,9 +503,9 @@ continue; } - if (peer->max_fails - && peer->fails >= peer->max_fails - && now - peer->checked <= peer->fail_timeout) + if (!peer->avail + && (now - peer->unavail_from < peer->fail_timeout + || peer->avail_check)) { continue; } @@ -523,7 +536,9 @@ rrp->tried[n] |= m; best->current_weight -= total; - best->checked = now; + if (!best->avail) { + best->avail_check = 1; + } return best; } @@ -550,35 +565,49 @@ peer = &rrp->peers->peer[rrp->current]; - if (state & NGX_PEER_FAILED) { + if (peer->max_fails) { now = ngx_time(); - /* ngx_lock_mutex(rrp->peers->mutex); */ + if (peer->avail && now - peer->first_fail >= peer->fail_timeout) { + peer->fails = 0; + } + + if (state & NGX_PEER_FAILED) { + /* ngx_lock_mutex(rrp->peers->mutex); */ + + if (peer->fails++ == 0) { + peer->first_fail = now; + } + + if (peer->fails >= peer->max_fails) { + peer->avail = 0; + peer->unavail_from = now; + } - peer->fails++; - peer->accessed = now; - peer->checked = now; + peer->avail_check = 0; - if (peer->max_fails) { peer->effective_weight -= peer->weight / peer->max_fails; - } - ngx_log_debug2(NGX_LOG_DEBUG_HTTP, pc->log, 0, - "free rr peer failed: %ui %i", - rrp->current, peer->effective_weight); + ngx_log_debug2(NGX_LOG_DEBUG_HTTP, pc->log, 0, + "free rr peer failed: %ui %i", + rrp->current, peer->effective_weight); - if (peer->effective_weight < 0) { - peer->effective_weight = 0; - } + if (peer->effective_weight < 0) { + peer->effective_weight = 0; + } - /* ngx_unlock_mutex(rrp->peers->mutex); */ + /* ngx_unlock_mutex(rrp->peers->mutex); */ - } else { + } else { - /* mark peer live if check passed */ + /* mark peer live if check passed */ - if (peer->accessed < peer->checked) { - peer->fails = 0; + if (peer->avail_check) { + peer->fails = 0; + peer->avail = 1; + peer->avail_check = 0; + /* peers->peer[i].first_fail = 0; */ + } } } From mdounin at mdounin.ru Tue May 21 13:23:08 2013 From: mdounin at mdounin.ru (Maxim Dounin) Date: Tue, 21 May 2013 17:23:08 +0400 Subject: [BUG?]fail_timeout/max_fails: code doesn't do what doc says In-Reply-To: <20130521161605.fbd3e3d1c8cb30db69215823@highloadlab.com> References: <20130520013426.2ca62487f4d4aa6d6d7de494@highloadlab.com> <20130520171103.GG69760@mdounin.ru> <20130521161605.fbd3e3d1c8cb30db69215823@highloadlab.com> Message-ID: <20130521132308.GW69760@mdounin.ru> Hello! On Tue, May 21, 2013 at 04:16:05PM +0400, Dmitry Popov wrote: > On Mon, 20 May 2013 21:11:03 +0400 > Maxim Dounin wrote: > > > Patch: > > > > diff --git a/src/http/modules/ngx_http_upstream_least_conn_module.c b/src/http/modules/ngx_http_upstream_least_conn_module.c > > --- a/src/http/modules/ngx_http_upstream_least_conn_module.c > > +++ b/src/http/modules/ngx_http_upstream_least_conn_module.c > > @@ -282,7 +282,10 @@ ngx_http_upstream_get_least_conn_peer(ng > > } > > > > best->current_weight -= total; > > - best->checked = now; > > + > > + if (now - best->checked > best->fail_timeout) { > > + best->checked = now; > > + } > > > > pc->sockaddr = best->sockaddr; > > pc->socklen = best->socklen; > > diff --git a/src/http/ngx_http_upstream_round_robin.c b/src/http/ngx_http_upstream_round_robin.c > > --- a/src/http/ngx_http_upstream_round_robin.c > > +++ b/src/http/ngx_http_upstream_round_robin.c > > @@ -523,7 +523,10 @@ ngx_http_upstream_get_peer(ngx_http_upst > > rrp->tried[n] |= m; > > > > best->current_weight -= total; > > - best->checked = now; > > + > > + if (now - best->checked > best->fail_timeout) { > > + best->checked = now; > > + } > > > > return best; > > } > > Still a bug exists because of this code: > > if (state & NGX_PEER_FAILED) { > /* ... */ > peer->fails++; > peer->accessed = now; > peer->checked = now; > /* .... */ > } else { > if (peer->accessed < peer->checked) { > peer->fails = 0; > } > } > > Consider upstream which fails at least once each second (actually it's enough > to fail each fail_timeout seconds). This upstream will be disabled as soon as it > will fail max_fails times no matter what fail_timeout is. This is expected behaviour. Documentation is a bit simplified here, and fail_timeout is used like session time limit - the peer->fails counter is reset once there are no failures within fail_timeout. While this might be non-ideal for some use cases, it's certainly not a bug. > I propose the following solution: > let's have two bits per peer: > 1) avail - set if this peer is available for usage (fails < max_fails) > 2) avail_check - set if this peer was disabled, but its fail_timeout is expired and > we sent one request to check if it's alive > and two timestamps: > 3) first_fail - we'll count number of fails only inside > [first_fail, first_fail + fail_timeout) interval > 4) unavail_from - peer is unavailable inside > [unavail_from, unavail_from + fail_timeout) interval > > first_fail will only be used if avail == 1, unavail_from only if avail == 0, > so we may use union here. > > Simplified code: > get: > if (!avail > && (now < unavail_from + fail_timeout || avail_check)) > { > continue; > } > > if (!avail) > avail_check =1; > free: > if (avail && now >= first_fail + fail_timeout) > fails = 0; > > if (request_failed) { > if (fails == 0) > first_fail = now; > fails++; > if (fails >= max_fails) { > avail = 0; > unavail_from = now; > } > avail_check = 0; > } else { > if (avail_check) { > fails = 0; > avail = 1; > avail_check = 0; > } > } Such algorithm forget everything about previous failures once per fail_timeout, and won't detect bursts of failures split across two fail_timeout intervals. Consider: - max_fails = 3, fail_timeout = 10s - failure at 0s - failure at 9s - at 10s peer->fails counter is reset - failure at 11s - failure at 12s While 3 failures are only 3 seconds away from each other, this is not detected due to granularity introduced by the algorithm. -- Maxim Dounin http://nginx.org/en/donation.html From pluknet at nginx.com Tue May 21 13:35:15 2013 From: pluknet at nginx.com (Sergey Kandaurov) Date: Tue, 21 May 2013 13:35:15 +0000 Subject: [nginx] Fixed error logging. Message-ID: details: http://hg.nginx.org/nginx/rev/f026adb935ad branches: changeset: 5218:f026adb935ad user: Sergey Kandaurov date: Tue May 21 17:30:19 2013 +0400 description: Fixed error logging. The provided argument list didn't follow a used format string. diffstat: src/http/modules/ngx_http_autoindex_module.c | 2 +- src/http/ngx_http_postpone_filter_module.c | 6 ++---- 2 files changed, 3 insertions(+), 5 deletions(-) diffs (35 lines): diff -r ddba4e308ecc -r f026adb935ad src/http/modules/ngx_http_autoindex_module.c --- a/src/http/modules/ngx_http_autoindex_module.c Tue May 21 12:54:27 2013 +0400 +++ b/src/http/modules/ngx_http_autoindex_module.c Tue May 21 17:30:19 2013 +0400 @@ -357,7 +357,7 @@ ngx_http_autoindex_handler(ngx_http_requ if (ngx_close_dir(&dir) == NGX_ERROR) { ngx_log_error(NGX_LOG_ALERT, r->connection->log, ngx_errno, - ngx_close_dir_n " \"%s\" failed", &path); + ngx_close_dir_n " \"%V\" failed", &path); } escape_html = ngx_escape_html(NULL, r->uri.data, r->uri.len); diff -r ddba4e308ecc -r f026adb935ad src/http/ngx_http_postpone_filter_module.c --- a/src/http/ngx_http_postpone_filter_module.c Tue May 21 12:54:27 2013 +0400 +++ b/src/http/ngx_http_postpone_filter_module.c Tue May 21 17:30:19 2013 +0400 @@ -70,8 +70,7 @@ ngx_http_postpone_filter(ngx_http_reques #if 0 /* TODO: SSI may pass NULL */ ngx_log_error(NGX_LOG_ALERT, c->log, 0, - "http postpone filter NULL inactive request", - &r->uri, &r->args); + "http postpone filter NULL inactive request"); #endif return NGX_OK; @@ -108,8 +107,7 @@ ngx_http_postpone_filter(ngx_http_reques if (pr->out == NULL) { ngx_log_error(NGX_LOG_ALERT, c->log, 0, - "http postpone filter NULL output", - &r->uri, &r->args); + "http postpone filter NULL output"); } else { ngx_log_debug2(NGX_LOG_DEBUG_HTTP, c->log, 0, From dp at highloadlab.com Tue May 21 13:51:40 2013 From: dp at highloadlab.com (Dmitry Popov) Date: Tue, 21 May 2013 17:51:40 +0400 Subject: [BUG?]fail_timeout/max_fails: code doesn't do what doc says In-Reply-To: <20130521132308.GW69760@mdounin.ru> References: <20130520013426.2ca62487f4d4aa6d6d7de494@highloadlab.com> <20130520171103.GG69760@mdounin.ru> <20130521161605.fbd3e3d1c8cb30db69215823@highloadlab.com> <20130521132308.GW69760@mdounin.ru> Message-ID: <20130521175140.915f301d5c7093e11fe119d8@highloadlab.com> On Tue, 21 May 2013 17:23:08 +0400 Maxim Dounin wrote: > > This is expected behaviour. Documentation is a bit simplified > here, and fail_timeout is used like session time limit - the > peer->fails counter is reset once there are no failures within > fail_timeout. > > While this might be non-ideal for some use cases, it's certainly > not a bug. > Well, it really hurts. Upstreams which fail in ~1% of requests is not a rare case, and we can't use max_fails+fail_timeout for them (because round-robin is thrashed for them and ip_hash is completely useless). Moreover, it is very hard to debug because of wiki. > Such algorithm forget everything about previous failures once per > fail_timeout, and won't detect bursts of failures split across > two fail_timeout intervals. > > Consider: > > - max_fails = 3, fail_timeout = 10s > - failure at 0s > - failure at 9s > - at 10s peer->fails counter is reset > - failure at 11s > - failure at 12s > > While 3 failures are only 3 seconds away from each other, this > is not detected due to granularity introduced by the algorithm. Yes, I know this case, sorry, forgot to mention. However, I think it will extend detection period to 2-3 fail_timeouts in real life (in theory up to max_fails fail_timeouts, yes, but it's almost improbable). If we want correct implementation we need per-second array (with fail_timeout elements), that's an overkill in my opinion. By the way, leaky bucket approach (like limit_req but with fails per second) might work well here, what do you think? -- Dmitry Popov Highloadlab From mdounin at mdounin.ru Tue May 21 18:52:08 2013 From: mdounin at mdounin.ru (Maxim Dounin) Date: Tue, 21 May 2013 18:52:08 +0000 Subject: [nginx] Mail: missing ngx_ssl_ecdh_curve() call. Message-ID: details: http://hg.nginx.org/nginx/rev/32fe021911c9 branches: changeset: 5219:32fe021911c9 user: F. da Silva date: Fri May 10 16:53:45 2013 +0200 description: Mail: missing ngx_ssl_ecdh_curve() call. diffstat: src/mail/ngx_mail_ssl_module.c | 4 ++++ 1 files changed, 4 insertions(+), 0 deletions(-) diffs (14 lines): diff --git a/src/mail/ngx_mail_ssl_module.c b/src/mail/ngx_mail_ssl_module.c --- a/src/mail/ngx_mail_ssl_module.c +++ b/src/mail/ngx_mail_ssl_module.c @@ -308,6 +308,10 @@ ngx_mail_ssl_merge_conf(ngx_conf_t *cf, return NGX_CONF_ERROR; } + if (ngx_ssl_ecdh_curve(cf, &conf->ssl, &conf->ecdh_curve) != NGX_OK) { + return NGX_CONF_ERROR; + } + ngx_conf_merge_value(conf->builtin_session_cache, prev->builtin_session_cache, NGX_SSL_NONE_SCACHE); From mdounin at mdounin.ru Tue May 21 18:52:08 2013 From: mdounin at mdounin.ru (Maxim Dounin) Date: Tue, 21 May 2013 18:52:08 +0000 Subject: [nginx] Upstream: fixed fail_timeout and max_fails > 1. Message-ID: details: http://hg.nginx.org/nginx/rev/1d68b502088c branches: changeset: 5220:1d68b502088c user: Maxim Dounin date: Tue May 21 21:47:50 2013 +0400 description: Upstream: fixed fail_timeout and max_fails > 1. Due to peer->checked always set since rev. c90801720a0c (1.3.0) by round-robin and least_conn balancers (ip_hash not affected), the code in ngx_http_upstream_free_round_robin_peer() function incorrectly reset peer->fails too often. Reported by Dmitry Popov, http://mailman.nginx.org/pipermail/nginx-devel/2013-May/003720.html diffstat: src/http/modules/ngx_http_upstream_least_conn_module.c | 5 ++++- src/http/ngx_http_upstream_round_robin.c | 5 ++++- 2 files changed, 8 insertions(+), 2 deletions(-) diffs (30 lines): diff --git a/src/http/modules/ngx_http_upstream_least_conn_module.c b/src/http/modules/ngx_http_upstream_least_conn_module.c --- a/src/http/modules/ngx_http_upstream_least_conn_module.c +++ b/src/http/modules/ngx_http_upstream_least_conn_module.c @@ -282,7 +282,10 @@ ngx_http_upstream_get_least_conn_peer(ng } best->current_weight -= total; - best->checked = now; + + if (now - best->checked > best->fail_timeout) { + best->checked = now; + } pc->sockaddr = best->sockaddr; pc->socklen = best->socklen; diff --git a/src/http/ngx_http_upstream_round_robin.c b/src/http/ngx_http_upstream_round_robin.c --- a/src/http/ngx_http_upstream_round_robin.c +++ b/src/http/ngx_http_upstream_round_robin.c @@ -523,7 +523,10 @@ ngx_http_upstream_get_peer(ngx_http_upst rrp->tried[n] |= m; best->current_weight -= total; - best->checked = now; + + if (now - best->checked > best->fail_timeout) { + best->checked = now; + } return best; } From mdounin at mdounin.ru Tue May 21 18:52:40 2013 From: mdounin at mdounin.ru (Maxim Dounin) Date: Tue, 21 May 2013 22:52:40 +0400 Subject: [PATCH] ngx_mail_ssl_module / ecdh_curve In-Reply-To: References: <20130520172516.GH69760@mdounin.ru> <20130520233139.GQ69760@mdounin.ru> Message-ID: <20130521185240.GA69760@mdounin.ru> Hello! On Tue, May 21, 2013 at 08:39:45AM +0200, Filipe Da Silva wrote: > Hi. > I fix it. If it's still not the case, please do. > > Thank for your patience. Committed, thanks. -- Maxim Dounin http://nginx.org/en/donation.html From sorin.v.manole at gmail.com Tue May 21 19:27:21 2013 From: sorin.v.manole at gmail.com (Sorin Manole) Date: Tue, 21 May 2013 22:27:21 +0300 Subject: HttpAccessModule and unix domain sockets Message-ID: Hi all, It seems that when using HttpAccessModule directives to deny requests, they don't seem to work if the server is listening on a unix domain socket. Even when using deny all. Can someone confirm and it's not just me making some stupid mistake ? Now if that is the case, would it be a good idea to add this functionality to the module ? Maybe add a new parameter like "deny unix" or something ? Or was this left out on purpose for a reason or another ? Thank you. -------------- next part -------------- An HTML attachment was scrubbed... URL: From piotr at cloudflare.com Tue May 21 23:11:36 2013 From: piotr at cloudflare.com (Piotr Sikora) Date: Tue, 21 May 2013 16:11:36 -0700 Subject: [PATCH] SNI: better server name handling. Message-ID: # HG changeset patch # User Piotr Sikora # Date 1369177319 25200 # Node ID 4b277448dfd56751c7c88477e78b2ba3cf6ae472 # Parent 1d68b502088c9d6e6603e9699354e36d03d77f9c SNI: better server name handling. Acknowledge acceptance of SNI server name to the OpenSSL library, which in turn lets the client know that it was accepted (by sending "server_name" TLS extension in the "ServerHello" handshake message, as suggested by RFC4366). Previously, this would happen only in case when requested server name was on the "server_name" list and either: there were multiple virtual servers defined for the same listening port or there was at least one regular expression with captures in the "server_name" directive. As a consequence, this change also: 1. Preserves requested SNI server name for future use. 2. Avoids unnecessary setting of SSL options if the virtual server didn't change. 3. Avoids unnecessary lookup of virtual server later on if requested HTTP server name is the same as requested SNI server name. Signed-off-by: Piotr Sikora diff -r 1d68b502088c -r 4b277448dfd5 src/http/ngx_http_request.c --- a/src/http/ngx_http_request.c Tue May 21 21:47:50 2013 +0400 +++ b/src/http/ngx_http_request.c Tue May 21 16:01:59 2013 -0700 @@ -773,6 +773,7 @@ ngx_http_ssl_srv_conf_t *sscf; ngx_http_core_loc_conf_t *clcf; ngx_http_core_srv_conf_t *cscf; + ngx_int_t rc; servername = SSL_get_servername(ssl_conn, TLSEXT_NAMETYPE_host_name); @@ -799,10 +800,10 @@ hc = c->data; - if (ngx_http_find_virtual_server(c, hc->addr_conf->virtual_names, &host, - NULL, &cscf) - != NGX_OK) - { + rc = ngx_http_find_virtual_server(c, hc->addr_conf->virtual_names, &host, + NULL, &cscf); + + if (rc == NGX_ERROR) { return SSL_TLSEXT_ERR_NOACK; } @@ -813,6 +814,10 @@ *hc->ssl_servername = host; + if (rc == NGX_DECLINED || hc->conf_ctx == cscf->ctx) { + return SSL_TLSEXT_ERR_OK; + } + hc->conf_ctx = cscf->ctx; clcf = ngx_http_get_module_loc_conf(hc->conf_ctx, ngx_http_core_module); From piotr at cloudflare.com Tue May 21 23:12:07 2013 From: piotr at cloudflare.com (Piotr Sikora) Date: Tue, 21 May 2013 16:12:07 -0700 Subject: [PATCH] SNI: store server name in the ngx_ssl_connection_t structure. Message-ID: # HG changeset patch # User Piotr Sikora # Date 1369177330 25200 # Node ID 8646199ded31a725bea599aeafc581f9c969872d # Parent 4b277448dfd56751c7c88477e78b2ba3cf6ae472 SNI: store server name in the ngx_ssl_connection_t structure. SNI server name is a property of the SSL connection and there is no good reason to store it elsewhere. Also, this makes the stored value accessible by non-HTTP modules. Signed-off-by: Piotr Sikora diff -r 4b277448dfd5 -r 8646199ded31 src/event/ngx_event_openssl.h --- a/src/event/ngx_event_openssl.h Tue May 21 16:01:59 2013 -0700 +++ b/src/event/ngx_event_openssl.h Tue May 21 16:02:10 2013 -0700 @@ -43,6 +43,13 @@ ngx_event_handler_pt saved_read_handler; ngx_event_handler_pt saved_write_handler; +#ifdef SSL_CTRL_SET_TLSEXT_HOSTNAME + ngx_str_t *servername; +#if (NGX_PCRE) + void *servername_regex; +#endif +#endif + unsigned handshaked:1; unsigned renegotiation:1; unsigned buffer:1; diff -r 4b277448dfd5 -r 8646199ded31 src/http/ngx_http_request.c --- a/src/http/ngx_http_request.c Tue May 21 16:01:59 2013 -0700 +++ b/src/http/ngx_http_request.c Tue May 21 16:02:10 2013 -0700 @@ -807,12 +807,12 @@ return SSL_TLSEXT_ERR_NOACK; } - hc->ssl_servername = ngx_palloc(c->pool, sizeof(ngx_str_t)); - if (hc->ssl_servername == NULL) { + c->ssl->servername = ngx_palloc(c->pool, sizeof(ngx_str_t)); + if (c->ssl->servername == NULL) { return SSL_TLSEXT_ERR_NOACK; } - *hc->ssl_servername = host; + *c->ssl->servername = host; if (rc == NGX_DECLINED || hc->conf_ctx == cscf->ctx) { return SSL_TLSEXT_ERR_OK; @@ -1954,23 +1954,24 @@ ngx_http_set_virtual_server(ngx_http_request_t *r, ngx_str_t *host) { ngx_int_t rc; + ngx_connection_t *c; ngx_http_connection_t *hc; ngx_http_core_loc_conf_t *clcf; ngx_http_core_srv_conf_t *cscf; - hc = r->http_connection; + c = r->connection; #if (NGX_HTTP_SSL && defined SSL_CTRL_SET_TLSEXT_HOSTNAME) - if (hc->ssl_servername) { - if (hc->ssl_servername->len == host->len - && ngx_strncmp(hc->ssl_servername->data, + if (c->ssl && c->ssl->servername) { + if (c->ssl->servername->len == host->len + && ngx_strncmp(c->ssl->servername->data, host->data, host->len) == 0) { #if (NGX_PCRE) - if (hc->ssl_servername_regex - && ngx_http_regex_exec(r, hc->ssl_servername_regex, - hc->ssl_servername) != NGX_OK) + if (c->ssl->servername_regex + && ngx_http_regex_exec(r, c->ssl->servername_regex, + c->ssl->servername) != NGX_OK) { ngx_http_close_request(r, NGX_HTTP_INTERNAL_SERVER_ERROR); return NGX_ERROR; @@ -1982,8 +1983,9 @@ #endif - rc = ngx_http_find_virtual_server(r->connection, - hc->addr_conf->virtual_names, + hc = r->http_connection; + + rc = ngx_http_find_virtual_server(c, hc->addr_conf->virtual_names, host, r, &cscf); if (rc == NGX_ERROR) { @@ -1993,7 +1995,7 @@ #if (NGX_HTTP_SSL && defined SSL_CTRL_SET_TLSEXT_HOSTNAME) - if (hc->ssl_servername) { + if (c->ssl && c->ssl->servername) { ngx_http_ssl_srv_conf_t *sscf; if (rc == NGX_DECLINED) { @@ -2004,7 +2006,7 @@ sscf = ngx_http_get_module_srv_conf(cscf->ctx, ngx_http_ssl_module); if (sscf->verify) { - ngx_log_error(NGX_LOG_INFO, r->connection->log, 0, + ngx_log_error(NGX_LOG_INFO, c->log, 0, "client attempted to request the server name " "different from that one was negotiated"); ngx_http_finalize_request(r, NGX_HTTP_BAD_REQUEST); @@ -2023,7 +2025,7 @@ clcf = ngx_http_get_module_loc_conf(r, ngx_http_core_module); - ngx_http_set_connection_log(r->connection, clcf->error_log); + ngx_http_set_connection_log(c, clcf->error_log); return NGX_OK; } @@ -2060,8 +2062,7 @@ #if (NGX_HTTP_SSL && defined SSL_CTRL_SET_TLSEXT_HOSTNAME) - if (r == NULL) { - ngx_http_connection_t *hc; + if (r == NULL && c->ssl) { for (i = 0; i < virtual_names->nregex; i++) { @@ -2072,8 +2073,7 @@ } if (n >= 0) { - hc = c->data; - hc->ssl_servername_regex = sn[i].regex; + c->ssl->servername_regex = sn[i].regex; *cscfp = sn[i].server; return NGX_OK; diff -r 4b277448dfd5 -r 8646199ded31 src/http/ngx_http_request.h --- a/src/http/ngx_http_request.h Tue May 21 16:01:59 2013 -0700 +++ b/src/http/ngx_http_request.h Tue May 21 16:02:10 2013 -0700 @@ -295,13 +295,6 @@ ngx_http_addr_conf_t *addr_conf; ngx_http_conf_ctx_t *conf_ctx; -#if (NGX_HTTP_SSL && defined SSL_CTRL_SET_TLSEXT_HOSTNAME) - ngx_str_t *ssl_servername; -#if (NGX_PCRE) - ngx_http_regex_t *ssl_servername_regex; -#endif -#endif - ngx_buf_t **busy; ngx_int_t nbusy; From piotr at cloudflare.com Tue May 21 23:12:41 2013 From: piotr at cloudflare.com (Piotr Sikora) Date: Tue, 21 May 2013 16:12:41 -0700 Subject: [PATCH] SNI: add $ssl_servername variable. Message-ID: # HG changeset patch # User Piotr Sikora # Date 1369177341 25200 # Node ID 4d617cb445673c8e3c43d75c240a7d401b394ee8 # Parent 8646199ded31a725bea599aeafc581f9c969872d SNI: add $ssl_servername variable. Signed-off-by: Piotr Sikora diff -r 8646199ded31 -r 4d617cb44567 src/event/ngx_event_openssl.c --- a/src/event/ngx_event_openssl.c Tue May 21 16:02:10 2013 -0700 +++ b/src/event/ngx_event_openssl.c Tue May 21 16:02:21 2013 -0700 @@ -2221,6 +2221,21 @@ ngx_int_t +ngx_ssl_get_servername(ngx_connection_t *c, ngx_pool_t *pool, ngx_str_t *s) +{ +#ifdef SSL_CTRL_SET_TLSEXT_HOSTNAME + if (c->ssl->servername) { + *s = *c->ssl->servername; + return NGX_OK; + } +#endif + + s->len = 0; + return NGX_OK; +} + + +ngx_int_t ngx_ssl_get_session_id(ngx_connection_t *c, ngx_pool_t *pool, ngx_str_t *s) { int len; diff -r 8646199ded31 -r 4d617cb44567 src/event/ngx_event_openssl.h --- a/src/event/ngx_event_openssl.h Tue May 21 16:02:10 2013 -0700 +++ b/src/event/ngx_event_openssl.h Tue May 21 16:02:21 2013 -0700 @@ -146,6 +146,8 @@ ngx_str_t *s); ngx_int_t ngx_ssl_get_cipher_name(ngx_connection_t *c, ngx_pool_t *pool, ngx_str_t *s); +ngx_int_t ngx_ssl_get_servername(ngx_connection_t *c, ngx_pool_t *pool, + ngx_str_t *s); ngx_int_t ngx_ssl_get_session_id(ngx_connection_t *c, ngx_pool_t *pool, ngx_str_t *s); ngx_int_t ngx_ssl_get_raw_certificate(ngx_connection_t *c, ngx_pool_t *pool, diff -r 8646199ded31 -r 4d617cb44567 src/http/modules/ngx_http_ssl_module.c --- a/src/http/modules/ngx_http_ssl_module.c Tue May 21 16:02:10 2013 -0700 +++ b/src/http/modules/ngx_http_ssl_module.c Tue May 21 16:02:21 2013 -0700 @@ -238,6 +238,9 @@ { ngx_string("ssl_cipher"), NULL, ngx_http_ssl_static_variable, (uintptr_t) ngx_ssl_get_cipher_name, NGX_HTTP_VAR_CHANGEABLE, 0 }, + { ngx_string("ssl_servername"), NULL, ngx_http_ssl_variable, + (uintptr_t) ngx_ssl_get_servername, NGX_HTTP_VAR_CHANGEABLE, 0 }, + { ngx_string("ssl_session_id"), NULL, ngx_http_ssl_variable, (uintptr_t) ngx_ssl_get_session_id, NGX_HTTP_VAR_CHANGEABLE, 0 }, From agentzh at gmail.com Tue May 21 23:29:36 2013 From: agentzh at gmail.com (agentzh) Date: Tue, 21 May 2013 16:29:36 -0700 Subject: [PATCH] Making $invalid_referer accessible in Perl (and Lua) Message-ID: Hello! The $invalid_referer variable is always created with the NGX_HTTP_VAR_NOHASH flag in ngx_http_referer_module, which unfortunately prohibits its use in embedded dynamic languages like Perl and Lua (through the ngx_http_get_variable function). Below attaches a patch that removes this flag. This issue was originally reported by Fry-kun. Thanks! -agentzh --- nginx-1.5.0/src/http/modules/ngx_http_referer_module.c 2013-05-06 03:27:10.000000000 -0700 +++ nginx-1.5.0-patched/src/http/modules/ngx_http_referer_module.c 2013-05-21 16:04:49.340286168 -0700 @@ -396,8 +396,7 @@ ngx_http_valid_referers(ngx_conf_t *cf, ngx_str_set(&name, "invalid_referer"); - var = ngx_http_add_variable(cf, &name, - NGX_HTTP_VAR_CHANGEABLE|NGX_HTTP_VAR_NOHASH); + var = ngx_http_add_variable(cf, &name, NGX_HTTP_VAR_CHANGEABLE); if (var == NULL) { return NGX_CONF_ERROR; } -------------- next part -------------- A non-text attachment was scrubbed... Name: nginx-1.5.0-invalid_referer_hash.patch Type: application/octet-stream Size: 562 bytes Desc: not available URL: From piotr at cloudflare.com Wed May 22 00:17:52 2013 From: piotr at cloudflare.com (Piotr Sikora) Date: Tue, 21 May 2013 17:17:52 -0700 Subject: [PATCH] OCSP stapling: better handling of successful OCSP responses. In-Reply-To: <20130520105643.GZ69760@mdounin.ru> References: <20130517132026.GX69760@mdounin.ru> <20130520105643.GZ69760@mdounin.ru> Message-ID: Hey Maxim, > You may consider adding a control for this Updated changeset attached below. > e.g., it looks like Apache has something similar called > SSLStaplingReturnResponderErrors: > > https://httpd.apache.org/docs/trunk/mod/mod_ssl.html#sslstaplingreturnrespondererrors I didn't test it, but both: directive's name and description suggest that this is different behavior from the one that I suggested, i.e. with this turned on Apache would return all OCSP responses, regardless of _their_ status (malformed responses, expired responses, etc). Best regards, Piotr Sikora # HG changeset patch # User Piotr Sikora # Date 1369181290 25200 # Node ID 7f52714a0b6a4229ab2fa9e640edbf80060eaf34 # Parent 1d68b502088c9d6e6603e9699354e36d03d77f9c OCSP stapling: add ssl_stapling_strict directive. When turned on (default), nginx will cache only OCSP responeses with "good" certificate status. When turned off, nginx will cache all successful OCSP responseses, regardless of the certificate status. While there, log certificate's common name and revocation reason, because certificate status alone isn't very useful information. Signed-off-by: Piotr Sikora diff -r 1d68b502088c -r 7f52714a0b6a src/event/ngx_event_openssl.h --- a/src/event/ngx_event_openssl.h Tue May 21 21:47:50 2013 +0400 +++ b/src/event/ngx_event_openssl.h Tue May 21 17:08:10 2013 -0700 @@ -106,7 +106,8 @@ ngx_str_t *cert, ngx_int_t depth); ngx_int_t ngx_ssl_crl(ngx_conf_t *cf, ngx_ssl_t *ssl, ngx_str_t *crl); ngx_int_t ngx_ssl_stapling(ngx_conf_t *cf, ngx_ssl_t *ssl, - ngx_str_t *file, ngx_str_t *responder, ngx_uint_t verify); + ngx_str_t *file, ngx_str_t *responder, ngx_uint_t verify, + ngx_uint_t strict); ngx_int_t ngx_ssl_stapling_resolver(ngx_conf_t *cf, ngx_ssl_t *ssl, ngx_resolver_t *resolver, ngx_msec_t resolver_timeout); RSA *ngx_ssl_rsa512_key_callback(SSL *ssl, int is_export, int key_length); diff -r 1d68b502088c -r 7f52714a0b6a src/event/ngx_event_openssl_stapling.c --- a/src/event/ngx_event_openssl_stapling.c Tue May 21 21:47:50 2013 +0400 +++ b/src/event/ngx_event_openssl_stapling.c Tue May 21 17:08:10 2013 -0700 @@ -34,6 +34,7 @@ time_t valid; unsigned verify:1; + unsigned strict:1; unsigned loading:1; } ngx_ssl_stapling_t; @@ -116,7 +117,7 @@ ngx_int_t ngx_ssl_stapling(ngx_conf_t *cf, ngx_ssl_t *ssl, ngx_str_t *file, - ngx_str_t *responder, ngx_uint_t verify) + ngx_str_t *responder, ngx_uint_t verify, ngx_uint_t strict) { ngx_int_t rc; ngx_pool_cleanup_t *cln; @@ -146,6 +147,7 @@ staple->ssl_ctx = ssl->ctx; staple->timeout = 60000; staple->verify = verify; + staple->strict = strict; if (file->len) { /* use OCSP response from the file */ @@ -529,7 +531,7 @@ const #endif u_char *p; - int n; + int n, r, idx; size_t len; ngx_str_t response; X509_STORE *store; @@ -539,6 +541,10 @@ OCSP_BASICRESP *basic; ngx_ssl_stapling_t *staple; ASN1_GENERALIZEDTIME *thisupdate, *nextupdate; + X509_NAME *name; + X509_NAME_ENTRY *entry; + ASN1_STRING *str; + ngx_str_t s; staple = ctx->data; ocsp = NULL; @@ -606,7 +612,7 @@ goto error; } - if (OCSP_resp_find_status(basic, id, &n, NULL, NULL, + if (OCSP_resp_find_status(basic, id, &n, &r, NULL, &thisupdate, &nextupdate) != 1) { @@ -615,19 +621,47 @@ goto error; } - if (n != V_OCSP_CERTSTATUS_GOOD) { - ngx_log_error(NGX_LOG_ERR, ctx->log, 0, - "certificate status \"%s\" in the OCSP response", - OCSP_cert_status_str(n)); - goto error; - } - if (OCSP_check_validity(thisupdate, nextupdate, 300, -1) != 1) { ngx_ssl_error(NGX_LOG_ERR, ctx->log, 0, "OCSP_check_validity() failed"); goto error; } + if (n != V_OCSP_CERTSTATUS_GOOD) { + ngx_str_set(&s, "unknown"); + + if (ctx->cert) { + name = X509_get_subject_name(ctx->cert); + if (name) { + idx = X509_NAME_get_index_by_NID(name, NID_commonName, -1); + if (idx != -1) { + entry = X509_NAME_get_entry(name, idx); + if (entry) { + str = X509_NAME_ENTRY_get_data(entry); + s.data = ASN1_STRING_data(str); + s.len = ASN1_STRING_length(str); + } + } + } + } + + if (n == V_OCSP_CERTSTATUS_REVOKED && r != -1) { + ngx_log_error(NGX_LOG_ERR, ctx->log, 0, + "certificate status \"%s\" (reason: \"%s\") in the " + "OCSP response for \"%V\"", + OCSP_cert_status_str(n), OCSP_crl_reason_str(r), &s); + + } else { + ngx_log_error(NGX_LOG_ERR, ctx->log, 0, + "certificate status \"%s\" in the OCSP response " + "for \"%V\"", OCSP_cert_status_str(n), &s); + } + + if (staple->strict) { + goto error; + } + } + OCSP_CERTID_free(id); OCSP_BASICRESP_free(basic); OCSP_RESPONSE_free(ocsp); @@ -1729,7 +1763,7 @@ ngx_int_t ngx_ssl_stapling(ngx_conf_t *cf, ngx_ssl_t *ssl, ngx_str_t *file, - ngx_str_t *responder, ngx_uint_t verify) + ngx_str_t *responder, ngx_uint_t verify, ngx_uint_t strict) { ngx_log_error(NGX_LOG_WARN, ssl->log, 0, "\"ssl_stapling\" ignored, not supported"); diff -r 1d68b502088c -r 7f52714a0b6a src/http/modules/ngx_http_ssl_module.c --- a/src/http/modules/ngx_http_ssl_module.c Tue May 21 21:47:50 2013 +0400 +++ b/src/http/modules/ngx_http_ssl_module.c Tue May 21 17:08:10 2013 -0700 @@ -195,6 +195,13 @@ offsetof(ngx_http_ssl_srv_conf_t, stapling_verify), NULL }, + { ngx_string("ssl_stapling_strict"), + NGX_HTTP_MAIN_CONF|NGX_HTTP_SRV_CONF|NGX_CONF_FLAG, + ngx_conf_set_flag_slot, + NGX_HTTP_SRV_CONF_OFFSET, + offsetof(ngx_http_ssl_srv_conf_t, stapling_strict), + NULL }, + ngx_null_command }; @@ -423,6 +430,7 @@ sscf->session_timeout = NGX_CONF_UNSET; sscf->stapling = NGX_CONF_UNSET; sscf->stapling_verify = NGX_CONF_UNSET; + sscf->stapling_strict = NGX_CONF_UNSET; return sscf; } @@ -478,6 +486,7 @@ ngx_conf_merge_value(conf->stapling, prev->stapling, 0); ngx_conf_merge_value(conf->stapling_verify, prev->stapling_verify, 0); + ngx_conf_merge_value(conf->stapling_strict, prev->stapling_strict, 1); ngx_conf_merge_str_value(conf->stapling_file, prev->stapling_file, ""); ngx_conf_merge_str_value(conf->stapling_responder, prev->stapling_responder, ""); @@ -624,8 +633,9 @@ if (conf->stapling) { - if (ngx_ssl_stapling(cf, &conf->ssl, &conf->stapling_file, - &conf->stapling_responder, conf->stapling_verify) + if (ngx_ssl_stapling(cf, &conf->ssl,&conf->stapling_file, + &conf->stapling_responder, conf->stapling_verify, + conf->stapling_strict) != NGX_OK) { return NGX_CONF_ERROR; diff -r 1d68b502088c -r 7f52714a0b6a src/http/modules/ngx_http_ssl_module.h --- a/src/http/modules/ngx_http_ssl_module.h Tue May 21 21:47:50 2013 +0400 +++ b/src/http/modules/ngx_http_ssl_module.h Tue May 21 17:08:10 2013 -0700 @@ -44,6 +44,7 @@ ngx_flag_t stapling; ngx_flag_t stapling_verify; + ngx_flag_t stapling_strict; ngx_str_t stapling_file; ngx_str_t stapling_responder; From piotr at cloudflare.com Wed May 22 00:44:37 2013 From: piotr at cloudflare.com (Piotr Sikora) Date: Tue, 21 May 2013 17:44:37 -0700 Subject: [PATCH] Style: cleanup. Message-ID: # HG changeset patch # User Piotr Sikora # Date 1369183181 25200 # Node ID 8f2e7bd395bae3c496639ba48d4683b2a6b796d5 # Parent 1d68b502088c9d6e6603e9699354e36d03d77f9c Style: cleanup. Remove unnecessary references to HTTP from non-HTTP modules and replace SSL * with ngx_ssl_conn_t. diff -r 1d68b502088c -r 8f2e7bd395ba src/core/ngx_conf_file.h --- a/src/core/ngx_conf_file.h Tue May 21 21:47:50 2013 +0400 +++ b/src/core/ngx_conf_file.h Tue May 21 17:39:41 2013 -0700 @@ -5,8 +5,8 @@ */ -#ifndef _NGX_HTTP_CONF_FILE_H_INCLUDED_ -#define _NGX_HTTP_CONF_FILE_H_INCLUDED_ +#ifndef _NGX_CONF_FILE_H_INCLUDED_ +#define _NGX_CONF_FILE_H_INCLUDED_ #include @@ -337,4 +337,4 @@ extern ngx_module_t *ngx_modules[]; -#endif /* _NGX_HTTP_CONF_FILE_H_INCLUDED_ */ +#endif /* _NGX_CONF_FILE_H_INCLUDED_ */ diff -r 1d68b502088c -r 8f2e7bd395ba src/event/ngx_event_openssl.c --- a/src/event/ngx_event_openssl.c Tue May 21 21:47:50 2013 +0400 +++ b/src/event/ngx_event_openssl.c Tue May 21 17:39:41 2013 -0700 @@ -15,7 +15,7 @@ } ngx_openssl_conf_t; -static int ngx_http_ssl_verify_callback(int ok, X509_STORE_CTX *x509_store); +static int ngx_ssl_verify_callback(int ok, X509_STORE_CTX *x509_store); static void ngx_ssl_info_callback(const ngx_ssl_conn_t *ssl_conn, int where, int ret); static void ngx_ssl_handshake_handler(ngx_event_t *ev); @@ -342,7 +342,7 @@ { STACK_OF(X509_NAME) *list; - SSL_CTX_set_verify(ssl->ctx, SSL_VERIFY_PEER, ngx_http_ssl_verify_callback); + SSL_CTX_set_verify(ssl->ctx, SSL_VERIFY_PEER, ngx_ssl_verify_callback); SSL_CTX_set_verify_depth(ssl->ctx, depth); @@ -457,7 +457,7 @@ static int -ngx_http_ssl_verify_callback(int ok, X509_STORE_CTX *x509_store) +ngx_ssl_verify_callback(int ok, X509_STORE_CTX *x509_store) { #if (NGX_DEBUG) char *subject, *issuer; @@ -517,7 +517,7 @@ RSA * -ngx_ssl_rsa512_key_callback(SSL *ssl, int is_export, int key_length) +ngx_ssl_rsa512_key_callback(ngx_ssl_conn_t *ssl, int is_export, int key_length) { static RSA *key; diff -r 1d68b502088c -r 8f2e7bd395ba src/event/ngx_event_openssl.h --- a/src/event/ngx_event_openssl.h Tue May 21 21:47:50 2013 +0400 +++ b/src/event/ngx_event_openssl.h Tue May 21 17:39:41 2013 -0700 @@ -109,7 +109,8 @@ ngx_str_t *file, ngx_str_t *responder, ngx_uint_t verify); ngx_int_t ngx_ssl_stapling_resolver(ngx_conf_t *cf, ngx_ssl_t *ssl, ngx_resolver_t *resolver, ngx_msec_t resolver_timeout); -RSA *ngx_ssl_rsa512_key_callback(SSL *ssl, int is_export, int key_length); +RSA *ngx_ssl_rsa512_key_callback(ngx_ssl_conn_t *ssl, int is_export, + int key_length); ngx_int_t ngx_ssl_dhparam(ngx_conf_t *cf, ngx_ssl_t *ssl, ngx_str_t *file); ngx_int_t ngx_ssl_ecdh_curve(ngx_conf_t *cf, ngx_ssl_t *ssl, ngx_str_t *name); ngx_int_t ngx_ssl_session_cache(ngx_ssl_t *ssl, ngx_str_t *sess_ctx, diff -r 1d68b502088c -r 8f2e7bd395ba src/mail/ngx_mail_ssl_module.c --- a/src/mail/ngx_mail_ssl_module.c Tue May 21 21:47:50 2013 +0400 +++ b/src/mail/ngx_mail_ssl_module.c Tue May 21 17:39:41 2013 -0700 @@ -25,7 +25,7 @@ void *conf); -static ngx_conf_enum_t ngx_http_starttls_state[] = { +static ngx_conf_enum_t ngx_mail_starttls_state[] = { { ngx_string("off"), NGX_MAIL_STARTTLS_OFF }, { ngx_string("on"), NGX_MAIL_STARTTLS_ON }, { ngx_string("only"), NGX_MAIL_STARTTLS_ONLY }, @@ -58,7 +58,7 @@ ngx_mail_ssl_starttls, NGX_MAIL_SRV_CONF_OFFSET, offsetof(ngx_mail_ssl_conf_t, starttls), - ngx_http_starttls_state }, + ngx_mail_starttls_state }, { ngx_string("ssl_certificate"), NGX_MAIL_MAIN_CONF|NGX_MAIL_SRV_CONF|NGX_CONF_TAKE1, diff -r 1d68b502088c -r 8f2e7bd395ba src/os/unix/ngx_linux_sendfile_chain.c --- a/src/os/unix/ngx_linux_sendfile_chain.c Tue May 21 21:47:50 2013 +0400 +++ b/src/os/unix/ngx_linux_sendfile_chain.c Tue May 21 17:39:41 2013 -0700 @@ -181,7 +181,7 @@ } else { c->tcp_nodelay = NGX_TCP_NODELAY_UNSET; - ngx_log_debug0(NGX_LOG_DEBUG_HTTP, c->log, 0, + ngx_log_debug0(NGX_LOG_DEBUG_EVENT, c->log, 0, "no tcp_nodelay"); } } From piotr at cloudflare.com Wed May 22 01:51:14 2013 From: piotr at cloudflare.com (Piotr Sikora) Date: Tue, 21 May 2013 18:51:14 -0700 Subject: [PATCH] Style: cleanup. In-Reply-To: References: Message-ID: Actually, after thinking about it, it makes much more sense to do this as a two separate patches with better descriptions. While there, I've also renamed "ssl" to "ssl_conn", to stay consistent with the rest of the code. Best regards, Piotr Sikora # HG changeset patch # User Piotr Sikora # Date 1369187023 25200 # Node ID 5511739dc78991a6cdcc98053d533c773d829fda # Parent 1d68b502088c9d6e6603e9699354e36d03d77f9c Style: remove unnecessary references to HTTP from non-HTTP modules. No functional changes. Signed-off-by: Piotr Sikora diff -r 1d68b502088c -r 5511739dc789 src/core/ngx_conf_file.h --- a/src/core/ngx_conf_file.h Tue May 21 21:47:50 2013 +0400 +++ b/src/core/ngx_conf_file.h Tue May 21 18:43:43 2013 -0700 @@ -5,8 +5,8 @@ */ -#ifndef _NGX_HTTP_CONF_FILE_H_INCLUDED_ -#define _NGX_HTTP_CONF_FILE_H_INCLUDED_ +#ifndef _NGX_CONF_FILE_H_INCLUDED_ +#define _NGX_CONF_FILE_H_INCLUDED_ #include @@ -337,4 +337,4 @@ extern ngx_module_t *ngx_modules[]; -#endif /* _NGX_HTTP_CONF_FILE_H_INCLUDED_ */ +#endif /* _NGX_CONF_FILE_H_INCLUDED_ */ diff -r 1d68b502088c -r 5511739dc789 src/event/ngx_event_openssl.c --- a/src/event/ngx_event_openssl.c Tue May 21 21:47:50 2013 +0400 +++ b/src/event/ngx_event_openssl.c Tue May 21 18:43:43 2013 -0700 @@ -15,7 +15,7 @@ } ngx_openssl_conf_t; -static int ngx_http_ssl_verify_callback(int ok, X509_STORE_CTX *x509_store); +static int ngx_ssl_verify_callback(int ok, X509_STORE_CTX *x509_store); static void ngx_ssl_info_callback(const ngx_ssl_conn_t *ssl_conn, int where, int ret); static void ngx_ssl_handshake_handler(ngx_event_t *ev); @@ -342,7 +342,7 @@ { STACK_OF(X509_NAME) *list; - SSL_CTX_set_verify(ssl->ctx, SSL_VERIFY_PEER, ngx_http_ssl_verify_callback); + SSL_CTX_set_verify(ssl->ctx, SSL_VERIFY_PEER, ngx_ssl_verify_callback); SSL_CTX_set_verify_depth(ssl->ctx, depth); @@ -457,7 +457,7 @@ static int -ngx_http_ssl_verify_callback(int ok, X509_STORE_CTX *x509_store) +ngx_ssl_verify_callback(int ok, X509_STORE_CTX *x509_store) { #if (NGX_DEBUG) char *subject, *issuer; diff -r 1d68b502088c -r 5511739dc789 src/mail/ngx_mail_ssl_module.c --- a/src/mail/ngx_mail_ssl_module.c Tue May 21 21:47:50 2013 +0400 +++ b/src/mail/ngx_mail_ssl_module.c Tue May 21 18:43:43 2013 -0700 @@ -25,7 +25,7 @@ void *conf); -static ngx_conf_enum_t ngx_http_starttls_state[] = { +static ngx_conf_enum_t ngx_mail_starttls_state[] = { { ngx_string("off"), NGX_MAIL_STARTTLS_OFF }, { ngx_string("on"), NGX_MAIL_STARTTLS_ON }, { ngx_string("only"), NGX_MAIL_STARTTLS_ONLY }, @@ -58,7 +58,7 @@ ngx_mail_ssl_starttls, NGX_MAIL_SRV_CONF_OFFSET, offsetof(ngx_mail_ssl_conf_t, starttls), - ngx_http_starttls_state }, + ngx_mail_starttls_state }, { ngx_string("ssl_certificate"), NGX_MAIL_MAIN_CONF|NGX_MAIL_SRV_CONF|NGX_CONF_TAKE1, diff -r 1d68b502088c -r 5511739dc789 src/os/unix/ngx_linux_sendfile_chain.c --- a/src/os/unix/ngx_linux_sendfile_chain.c Tue May 21 21:47:50 2013 +0400 +++ b/src/os/unix/ngx_linux_sendfile_chain.c Tue May 21 18:43:43 2013 -0700 @@ -181,7 +181,7 @@ } else { c->tcp_nodelay = NGX_TCP_NODELAY_UNSET; - ngx_log_debug0(NGX_LOG_DEBUG_HTTP, c->log, 0, + ngx_log_debug0(NGX_LOG_DEBUG_EVENT, c->log, 0, "no tcp_nodelay"); } } # HG changeset patch # User Piotr Sikora # Date 1369187107 25200 # Node ID a485b47aae9c3d1781360ea2f9a8a2b04458045a # Parent 5511739dc78991a6cdcc98053d533c773d829fda Style: replace SSL *ssl with ngx_ssl_conn_t *ssl_conn. No functional changes. Signed-off-by: Piotr Sikora diff -r 5511739dc789 -r a485b47aae9c src/event/ngx_event_openssl.c --- a/src/event/ngx_event_openssl.c Tue May 21 18:43:43 2013 -0700 +++ b/src/event/ngx_event_openssl.c Tue May 21 18:45:07 2013 -0700 @@ -517,7 +517,8 @@ RSA * -ngx_ssl_rsa512_key_callback(SSL *ssl, int is_export, int key_length) +ngx_ssl_rsa512_key_callback(ngx_ssl_conn_t *ssl_conn, int is_export, + int key_length) { static RSA *key; diff -r 5511739dc789 -r a485b47aae9c src/event/ngx_event_openssl.h --- a/src/event/ngx_event_openssl.h Tue May 21 18:43:43 2013 -0700 +++ b/src/event/ngx_event_openssl.h Tue May 21 18:45:07 2013 -0700 @@ -109,7 +109,8 @@ ngx_str_t *file, ngx_str_t *responder, ngx_uint_t verify); ngx_int_t ngx_ssl_stapling_resolver(ngx_conf_t *cf, ngx_ssl_t *ssl, ngx_resolver_t *resolver, ngx_msec_t resolver_timeout); -RSA *ngx_ssl_rsa512_key_callback(SSL *ssl, int is_export, int key_length); +RSA *ngx_ssl_rsa512_key_callback(ngx_ssl_conn_t *ssl_conn, int is_export, + int key_length); ngx_int_t ngx_ssl_dhparam(ngx_conf_t *cf, ngx_ssl_t *ssl, ngx_str_t *file); ngx_int_t ngx_ssl_ecdh_curve(ngx_conf_t *cf, ngx_ssl_t *ssl, ngx_str_t *name); ngx_int_t ngx_ssl_session_cache(ngx_ssl_t *ssl, ngx_str_t *sess_ctx, From mdounin at mdounin.ru Wed May 22 13:49:41 2013 From: mdounin at mdounin.ru (Maxim Dounin) Date: Wed, 22 May 2013 17:49:41 +0400 Subject: HttpAccessModule and unix domain sockets In-Reply-To: References: Message-ID: <20130522134941.GE69760@mdounin.ru> Hello! On Tue, May 21, 2013 at 10:27:21PM +0300, Sorin Manole wrote: > Hi all, > > It seems that when using HttpAccessModule directives to deny requests, they > don't seem to work if the server is listening on a unix domain socket. Even > when using deny all. > Can someone confirm and it's not just me making some stupid mistake ? Yes, access module allow/deny directives currently only able to limit ipv4 and ipv6 addresses. > Now if that is the case, would it be a good idea to add this functionality > to the module ? Maybe add a new parameter like "deny unix" or something ? > Or was this left out on purpose for a reason or another ? It probably should be expanded to support "unix:" special address like set_real_ip_from does (see http://nginx.org/r/set_real_ip_from). -- Maxim Dounin http://nginx.org/en/donation.html From mdounin at mdounin.ru Wed May 22 16:57:08 2013 From: mdounin at mdounin.ru (Maxim Dounin) Date: Wed, 22 May 2013 20:57:08 +0400 Subject: [BUG?]fail_timeout/max_fails: code doesn't do what doc says In-Reply-To: <20130521175140.915f301d5c7093e11fe119d8@highloadlab.com> References: <20130520013426.2ca62487f4d4aa6d6d7de494@highloadlab.com> <20130520171103.GG69760@mdounin.ru> <20130521161605.fbd3e3d1c8cb30db69215823@highloadlab.com> <20130521132308.GW69760@mdounin.ru> <20130521175140.915f301d5c7093e11fe119d8@highloadlab.com> Message-ID: <20130522165708.GO69760@mdounin.ru> Hello! On Tue, May 21, 2013 at 05:51:40PM +0400, Dmitry Popov wrote: > On Tue, 21 May 2013 17:23:08 +0400 > Maxim Dounin wrote: > > > > > This is expected behaviour. Documentation is a bit simplified > > here, and fail_timeout is used like session time limit - the > > peer->fails counter is reset once there are no failures within > > fail_timeout. > > > > While this might be non-ideal for some use cases, it's certainly > > not a bug. > > > > Well, it really hurts. Upstreams which fail in ~1% of requests is not a rare > case, and we can't use max_fails+fail_timeout for them (because round-robin is > thrashed for them and ip_hash is completely useless). Moreover, it is very hard > to debug because of wiki. Well, in normal world if an upstream constantly fails ~1% of requests - it's not healthy and should not be used. I understand that your use case is a bit special though. > > Such algorithm forget everything about previous failures once per > > fail_timeout, and won't detect bursts of failures split across > > two fail_timeout intervals. > > > > Consider: > > > > - max_fails = 3, fail_timeout = 10s > > - failure at 0s > > - failure at 9s > > - at 10s peer->fails counter is reset > > - failure at 11s > > - failure at 12s > > > > While 3 failures are only 3 seconds away from each other, this > > is not detected due to granularity introduced by the algorithm. > > Yes, I know this case, sorry, forgot to mention. However, I think it will > extend detection period to 2-3 fail_timeouts in real life (in theory up to > max_fails fail_timeouts, yes, but it's almost improbable). If we want correct > implementation we need per-second array (with fail_timeout elements), that's an > overkill in my opinion. Sure, per-second array isn't a solution. > By the way, leaky bucket approach (like limit_req but > with fails per second) might work well here, what do you think? Yes, leaky/token bucket should work. That's actually what I think about if I think about changing the above algorithm to something strictly bound to fail_timeout period. -- Maxim Dounin http://nginx.org/en/donation.html From mdounin at mdounin.ru Wed May 22 17:32:37 2013 From: mdounin at mdounin.ru (Maxim Dounin) Date: Wed, 22 May 2013 21:32:37 +0400 Subject: [PATCH] New variable: $ssl_sni_host In-Reply-To: <20130521030843.GA12890@cucumber.bridge.anchor.net.au> References: <20130521030843.GA12890@cucumber.bridge.anchor.net.au> Message-ID: <20130522173237.GP69760@mdounin.ru> Hello! On Tue, May 21, 2013 at 01:08:43PM +1000, Christian Marie wrote: > Patch attached adds a new variable, $ssl_sni_host. > > I would find this quite useful as there is no other way of knowing for sure > which host a request is directed at (at the SSL layer), as the HTTP HOST header > can be wrong. > > Possibly somewhat related to: http://trac.nginx.org/nginx/ticket/229 > > I should mention that I don't intend for this to be a drop in replacement for > $http_host, though that could very well work with proxy_pass. E.g. with SPDY, server name as indicated using SNI in it's turn is _expected_ to be wrong, at least with Chrome. Using such a variable with proxy_pass doesn't looks like a good idea. Such a variable might make sense from logging point of view though. But probably solution suggested by Piotr Sikora is a bit better, see here: http://mailman.nginx.org/pipermail/nginx-devel/2013-May/003745.html http://mailman.nginx.org/pipermail/nginx-devel/2013-May/003746.html http://mailman.nginx.org/pipermail/nginx-devel/2013-May/003747.html -- Maxim Dounin http://nginx.org/en/donation.html From mdounin at mdounin.ru Wed May 22 18:54:05 2013 From: mdounin at mdounin.ru (Maxim Dounin) Date: Wed, 22 May 2013 18:54:05 +0000 Subject: [nginx] Referer module: added $invalid_referer to variables hash. Message-ID: details: http://hg.nginx.org/nginx/rev/8f74cf107137 branches: changeset: 5221:8f74cf107137 user: Maxim Dounin date: Wed May 22 22:31:53 2013 +0400 description: Referer module: added $invalid_referer to variables hash. This makes it accessible via dynamic lookup with ngx_http_get_variable() from Perl, SSI, etc. Patch by Yichun Zhang (agentzh). diffstat: src/http/modules/ngx_http_referer_module.c | 3 +-- 1 files changed, 1 insertions(+), 2 deletions(-) diffs (13 lines): diff --git a/src/http/modules/ngx_http_referer_module.c b/src/http/modules/ngx_http_referer_module.c --- a/src/http/modules/ngx_http_referer_module.c +++ b/src/http/modules/ngx_http_referer_module.c @@ -396,8 +396,7 @@ ngx_http_valid_referers(ngx_conf_t *cf, ngx_str_set(&name, "invalid_referer"); - var = ngx_http_add_variable(cf, &name, - NGX_HTTP_VAR_CHANGEABLE|NGX_HTTP_VAR_NOHASH); + var = ngx_http_add_variable(cf, &name, NGX_HTTP_VAR_CHANGEABLE); if (var == NULL) { return NGX_CONF_ERROR; } From mdounin at mdounin.ru Wed May 22 18:54:37 2013 From: mdounin at mdounin.ru (Maxim Dounin) Date: Wed, 22 May 2013 22:54:37 +0400 Subject: [PATCH] Making $invalid_referer accessible in Perl (and Lua) In-Reply-To: References: Message-ID: <20130522185437.GR69760@mdounin.ru> Hello! On Tue, May 21, 2013 at 04:29:36PM -0700, agentzh wrote: > Hello! > > The $invalid_referer variable is always created with the > NGX_HTTP_VAR_NOHASH flag in ngx_http_referer_module, which > unfortunately prohibits its use in embedded dynamic languages like > Perl and Lua (through the ngx_http_get_variable function). > > Below attaches a patch that removes this flag. > > This issue was originally reported by Fry-kun. > > Thanks! > -agentzh > > --- nginx-1.5.0/src/http/modules/ngx_http_referer_module.c 2013-05-06 > 03:27:10.000000000 -0700 > +++ nginx-1.5.0-patched/src/http/modules/ngx_http_referer_module.c > 2013-05-21 16:04:49.340286168 -0700 > @@ -396,8 +396,7 @@ ngx_http_valid_referers(ngx_conf_t *cf, > > ngx_str_set(&name, "invalid_referer"); > > - var = ngx_http_add_variable(cf, &name, > - NGX_HTTP_VAR_CHANGEABLE|NGX_HTTP_VAR_NOHASH); > + var = ngx_http_add_variable(cf, &name, NGX_HTTP_VAR_CHANGEABLE); > if (var == NULL) { > return NGX_CONF_ERROR; > } Committed, thanks. -- Maxim Dounin http://nginx.org/en/donation.html From mdounin at mdounin.ru Wed May 22 19:02:05 2013 From: mdounin at mdounin.ru (Maxim Dounin) Date: Wed, 22 May 2013 19:02:05 +0000 Subject: [nginx] Style: remove unnecessary references to HTTP from non-HT... Message-ID: details: http://hg.nginx.org/nginx/rev/23a186e8ca45 branches: changeset: 5222:23a186e8ca45 user: Piotr Sikora date: Tue May 21 18:43:43 2013 -0700 description: Style: remove unnecessary references to HTTP from non-HTTP modules. No functional changes. Signed-off-by: Piotr Sikora diffstat: src/core/ngx_conf_file.h | 6 +++--- src/event/ngx_event_openssl.c | 6 +++--- src/mail/ngx_mail_ssl_module.c | 4 ++-- src/os/unix/ngx_linux_sendfile_chain.c | 2 +- 4 files changed, 9 insertions(+), 9 deletions(-) diffs (83 lines): diff --git a/src/core/ngx_conf_file.h b/src/core/ngx_conf_file.h --- a/src/core/ngx_conf_file.h +++ b/src/core/ngx_conf_file.h @@ -5,8 +5,8 @@ */ -#ifndef _NGX_HTTP_CONF_FILE_H_INCLUDED_ -#define _NGX_HTTP_CONF_FILE_H_INCLUDED_ +#ifndef _NGX_CONF_FILE_H_INCLUDED_ +#define _NGX_CONF_FILE_H_INCLUDED_ #include @@ -337,4 +337,4 @@ extern ngx_uint_t ngx_max_module; extern ngx_module_t *ngx_modules[]; -#endif /* _NGX_HTTP_CONF_FILE_H_INCLUDED_ */ +#endif /* _NGX_CONF_FILE_H_INCLUDED_ */ diff --git a/src/event/ngx_event_openssl.c b/src/event/ngx_event_openssl.c --- a/src/event/ngx_event_openssl.c +++ b/src/event/ngx_event_openssl.c @@ -15,7 +15,7 @@ typedef struct { } ngx_openssl_conf_t; -static int ngx_http_ssl_verify_callback(int ok, X509_STORE_CTX *x509_store); +static int ngx_ssl_verify_callback(int ok, X509_STORE_CTX *x509_store); static void ngx_ssl_info_callback(const ngx_ssl_conn_t *ssl_conn, int where, int ret); static void ngx_ssl_handshake_handler(ngx_event_t *ev); @@ -342,7 +342,7 @@ ngx_ssl_client_certificate(ngx_conf_t *c { STACK_OF(X509_NAME) *list; - SSL_CTX_set_verify(ssl->ctx, SSL_VERIFY_PEER, ngx_http_ssl_verify_callback); + SSL_CTX_set_verify(ssl->ctx, SSL_VERIFY_PEER, ngx_ssl_verify_callback); SSL_CTX_set_verify_depth(ssl->ctx, depth); @@ -457,7 +457,7 @@ ngx_ssl_crl(ngx_conf_t *cf, ngx_ssl_t *s static int -ngx_http_ssl_verify_callback(int ok, X509_STORE_CTX *x509_store) +ngx_ssl_verify_callback(int ok, X509_STORE_CTX *x509_store) { #if (NGX_DEBUG) char *subject, *issuer; diff --git a/src/mail/ngx_mail_ssl_module.c b/src/mail/ngx_mail_ssl_module.c --- a/src/mail/ngx_mail_ssl_module.c +++ b/src/mail/ngx_mail_ssl_module.c @@ -25,7 +25,7 @@ static char *ngx_mail_ssl_session_cache( void *conf); -static ngx_conf_enum_t ngx_http_starttls_state[] = { +static ngx_conf_enum_t ngx_mail_starttls_state[] = { { ngx_string("off"), NGX_MAIL_STARTTLS_OFF }, { ngx_string("on"), NGX_MAIL_STARTTLS_ON }, { ngx_string("only"), NGX_MAIL_STARTTLS_ONLY }, @@ -58,7 +58,7 @@ static ngx_command_t ngx_mail_ssl_comma ngx_mail_ssl_starttls, NGX_MAIL_SRV_CONF_OFFSET, offsetof(ngx_mail_ssl_conf_t, starttls), - ngx_http_starttls_state }, + ngx_mail_starttls_state }, { ngx_string("ssl_certificate"), NGX_MAIL_MAIN_CONF|NGX_MAIL_SRV_CONF|NGX_CONF_TAKE1, diff --git a/src/os/unix/ngx_linux_sendfile_chain.c b/src/os/unix/ngx_linux_sendfile_chain.c --- a/src/os/unix/ngx_linux_sendfile_chain.c +++ b/src/os/unix/ngx_linux_sendfile_chain.c @@ -181,7 +181,7 @@ ngx_linux_sendfile_chain(ngx_connection_ } else { c->tcp_nodelay = NGX_TCP_NODELAY_UNSET; - ngx_log_debug0(NGX_LOG_DEBUG_HTTP, c->log, 0, + ngx_log_debug0(NGX_LOG_DEBUG_EVENT, c->log, 0, "no tcp_nodelay"); } } From mdounin at mdounin.ru Wed May 22 19:02:05 2013 From: mdounin at mdounin.ru (Maxim Dounin) Date: Wed, 22 May 2013 19:02:05 +0000 Subject: [nginx] Style: replace SSL *ssl with ngx_ssl_conn_t *ssl_conn. Message-ID: details: http://hg.nginx.org/nginx/rev/71d85de7b53b branches: changeset: 5223:71d85de7b53b user: Piotr Sikora date: Tue May 21 18:45:07 2013 -0700 description: Style: replace SSL *ssl with ngx_ssl_conn_t *ssl_conn. No functional changes. Signed-off-by: Piotr Sikora diffstat: src/event/ngx_event_openssl.c | 3 ++- src/event/ngx_event_openssl.h | 3 ++- 2 files changed, 4 insertions(+), 2 deletions(-) diffs (26 lines): diff --git a/src/event/ngx_event_openssl.c b/src/event/ngx_event_openssl.c --- a/src/event/ngx_event_openssl.c +++ b/src/event/ngx_event_openssl.c @@ -517,7 +517,8 @@ ngx_ssl_info_callback(const ngx_ssl_conn RSA * -ngx_ssl_rsa512_key_callback(SSL *ssl, int is_export, int key_length) +ngx_ssl_rsa512_key_callback(ngx_ssl_conn_t *ssl_conn, int is_export, + int key_length) { static RSA *key; diff --git a/src/event/ngx_event_openssl.h b/src/event/ngx_event_openssl.h --- a/src/event/ngx_event_openssl.h +++ b/src/event/ngx_event_openssl.h @@ -109,7 +109,8 @@ ngx_int_t ngx_ssl_stapling(ngx_conf_t *c ngx_str_t *file, ngx_str_t *responder, ngx_uint_t verify); ngx_int_t ngx_ssl_stapling_resolver(ngx_conf_t *cf, ngx_ssl_t *ssl, ngx_resolver_t *resolver, ngx_msec_t resolver_timeout); -RSA *ngx_ssl_rsa512_key_callback(SSL *ssl, int is_export, int key_length); +RSA *ngx_ssl_rsa512_key_callback(ngx_ssl_conn_t *ssl_conn, int is_export, + int key_length); ngx_int_t ngx_ssl_dhparam(ngx_conf_t *cf, ngx_ssl_t *ssl, ngx_str_t *file); ngx_int_t ngx_ssl_ecdh_curve(ngx_conf_t *cf, ngx_ssl_t *ssl, ngx_str_t *name); ngx_int_t ngx_ssl_session_cache(ngx_ssl_t *ssl, ngx_str_t *sess_ctx, From mdounin at mdounin.ru Wed May 22 19:02:21 2013 From: mdounin at mdounin.ru (Maxim Dounin) Date: Wed, 22 May 2013 23:02:21 +0400 Subject: [PATCH] Style: cleanup. In-Reply-To: References: Message-ID: <20130522190221.GS69760@mdounin.ru> Hello! On Tue, May 21, 2013 at 06:51:14PM -0700, Piotr Sikora wrote: > Actually, after thinking about it, it makes much more sense to do this > as a two separate patches with better descriptions. > > While there, I've also renamed "ssl" to "ssl_conn", to stay consistent > with the rest of the code. [...] Pushed, thanks. -- Maxim Dounin http://nginx.org/en/donation.html From sorin.v.manole at gmail.com Wed May 22 19:17:22 2013 From: sorin.v.manole at gmail.com (Sorin Manole) Date: Wed, 22 May 2013 22:17:22 +0300 Subject: HttpAccessModule and unix domain sockets In-Reply-To: <20130522134941.GE69760@mdounin.ru> References: <20130522134941.GE69760@mdounin.ru> Message-ID: Hi, Thanks for the quick response! I would like to implement this feature and submit the patch for review. Just want some quick clarifications: Once there is support for unix domain sockets, should "deny all" limit them too ? (I suppose yes?) Also "deny unix:" should limit connections that come through all unix domain sockets ? Thank you. 2013/5/22 Maxim Dounin > Hello! > > On Tue, May 21, 2013 at 10:27:21PM +0300, Sorin Manole wrote: > > > Hi all, > > > > It seems that when using HttpAccessModule directives to deny requests, > they > > don't seem to work if the server is listening on a unix domain socket. > Even > > when using deny all. > > Can someone confirm and it's not just me making some stupid mistake ? > > Yes, access module allow/deny directives currently only able to > limit ipv4 and ipv6 addresses. > > > Now if that is the case, would it be a good idea to add this > functionality > > to the module ? Maybe add a new parameter like "deny unix" or something ? > > Or was this left out on purpose for a reason or another ? > > It probably should be expanded to support "unix:" special address > like set_real_ip_from does (see http://nginx.org/r/set_real_ip_from). > > -- > Maxim Dounin > http://nginx.org/en/donation.html > > _______________________________________________ > nginx-devel mailing list > nginx-devel at nginx.org > http://mailman.nginx.org/mailman/listinfo/nginx-devel > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mdounin at mdounin.ru Wed May 22 19:32:26 2013 From: mdounin at mdounin.ru (Maxim Dounin) Date: Wed, 22 May 2013 23:32:26 +0400 Subject: HttpAccessModule and unix domain sockets In-Reply-To: References: <20130522134941.GE69760@mdounin.ru> Message-ID: <20130522193226.GU69760@mdounin.ru> Hello! On Wed, May 22, 2013 at 10:17:22PM +0300, Sorin Manole wrote: > Hi, > > Thanks for the quick response! I would like to implement this feature and > submit the patch for review. > Just want some quick clarifications: > Once there is support for unix domain sockets, should "deny all" limit them > too ? (I suppose yes?) Yes. > Also "deny unix:" should limit connections that come through all unix > domain sockets ? Yes. > > Thank you. > > > 2013/5/22 Maxim Dounin > > > Hello! > > > > On Tue, May 21, 2013 at 10:27:21PM +0300, Sorin Manole wrote: > > > > > Hi all, > > > > > > It seems that when using HttpAccessModule directives to deny requests, > > they > > > don't seem to work if the server is listening on a unix domain socket. > > Even > > > when using deny all. > > > Can someone confirm and it's not just me making some stupid mistake ? > > > > Yes, access module allow/deny directives currently only able to > > limit ipv4 and ipv6 addresses. > > > > > Now if that is the case, would it be a good idea to add this > > functionality > > > to the module ? Maybe add a new parameter like "deny unix" or something ? > > > Or was this left out on purpose for a reason or another ? > > > > It probably should be expanded to support "unix:" special address > > like set_real_ip_from does (see http://nginx.org/r/set_real_ip_from). > > > > -- > > Maxim Dounin > > http://nginx.org/en/donation.html > > > > _______________________________________________ > > nginx-devel mailing list > > nginx-devel at nginx.org > > http://mailman.nginx.org/mailman/listinfo/nginx-devel > > > _______________________________________________ > nginx-devel mailing list > nginx-devel at nginx.org > http://mailman.nginx.org/mailman/listinfo/nginx-devel -- Maxim Dounin http://nginx.org/en/donation.html From maxim at nginx.com Thu May 23 07:25:07 2013 From: maxim at nginx.com (Maxim Konovalov) Date: Thu, 23 May 2013 11:25:07 +0400 Subject: contributing changes article Message-ID: <519DC453.1070207@nginx.com> Hello, we recently added a short article to help people to contribute code to nginx: http://nginx.org/en/docs/contributing_changes.html Hope this helps a bit. -- Maxim Konovalov +7 (910) 4293178 http://nginx.com/services.html From ru at nginx.com Thu May 23 08:11:38 2013 From: ru at nginx.com (Ruslan Ermilov) Date: Thu, 23 May 2013 08:11:38 +0000 Subject: [nginx] Configure: fixed test of OS X atomic(3). Message-ID: details: http://hg.nginx.org/nginx/rev/09d7faaef16f branches: changeset: 5224:09d7faaef16f user: Ruslan Ermilov date: Thu May 23 10:23:21 2013 +0400 description: Configure: fixed test of OS X atomic(3). diffstat: auto/os/darwin | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diffs (10 lines): diff -r 71d85de7b53b -r 09d7faaef16f auto/os/darwin --- a/auto/os/darwin Tue May 21 18:45:07 2013 -0700 +++ b/auto/os/darwin Thu May 23 10:23:21 2013 +0400 @@ -112,5 +112,5 @@ ngx_feature_incs="#include details: http://hg.nginx.org/nginx/rev/15a7deeaa19a branches: changeset: 5225:15a7deeaa19a user: Sergey Kandaurov date: Thu May 23 15:47:58 2013 +0400 description: Use "void" for functions with empty parameter list. diffstat: src/http/ngx_http_spdy.c | 2 +- src/http/ngx_http_spdy.h | 2 +- src/os/unix/ngx_darwin_init.c | 2 +- src/os/unix/ngx_freebsd_init.c | 2 +- src/os/unix/ngx_freebsd_rfork_thread.c | 2 +- src/os/unix/ngx_freebsd_rfork_thread.h | 4 ++-- 6 files changed, 7 insertions(+), 7 deletions(-) diffs (81 lines): diff -r 09d7faaef16f -r 15a7deeaa19a src/http/ngx_http_spdy.c --- a/src/http/ngx_http_spdy.c Thu May 23 10:23:21 2013 +0400 +++ b/src/http/ngx_http_spdy.c Thu May 23 15:47:58 2013 +0400 @@ -2176,7 +2176,7 @@ ngx_http_spdy_handle_request_header(ngx_ void -ngx_http_spdy_request_headers_init() +ngx_http_spdy_request_headers_init(void) { ngx_uint_t i; ngx_http_spdy_request_header_t *h; diff -r 09d7faaef16f -r 15a7deeaa19a src/http/ngx_http_spdy.h --- a/src/http/ngx_http_spdy.h Thu May 23 10:23:21 2013 +0400 +++ b/src/http/ngx_http_spdy.h Thu May 23 15:47:58 2013 +0400 @@ -186,7 +186,7 @@ ngx_http_spdy_queue_blocked_frame(ngx_ht void ngx_http_spdy_init(ngx_event_t *rev); -void ngx_http_spdy_request_headers_init(); +void ngx_http_spdy_request_headers_init(void); ngx_int_t ngx_http_spdy_read_request_body(ngx_http_request_t *r, ngx_http_client_body_handler_pt post_handler); diff -r 09d7faaef16f -r 15a7deeaa19a src/os/unix/ngx_darwin_init.c --- a/src/os/unix/ngx_darwin_init.c Thu May 23 10:23:21 2013 +0400 +++ b/src/os/unix/ngx_darwin_init.c Thu May 23 15:47:58 2013 +0400 @@ -59,7 +59,7 @@ sysctl_t sysctls[] = { void -ngx_debug_init() +ngx_debug_init(void) { #if (NGX_DEBUG_MALLOC) diff -r 09d7faaef16f -r 15a7deeaa19a src/os/unix/ngx_freebsd_init.c --- a/src/os/unix/ngx_freebsd_init.c Thu May 23 10:23:21 2013 +0400 +++ b/src/os/unix/ngx_freebsd_init.c Thu May 23 15:47:58 2013 +0400 @@ -72,7 +72,7 @@ sysctl_t sysctls[] = { void -ngx_debug_init() +ngx_debug_init(void) { #if (NGX_DEBUG_MALLOC) diff -r 09d7faaef16f -r 15a7deeaa19a src/os/unix/ngx_freebsd_rfork_thread.c --- a/src/os/unix/ngx_freebsd_rfork_thread.c Thu May 23 10:23:21 2013 +0400 +++ b/src/os/unix/ngx_freebsd_rfork_thread.c Thu May 23 15:47:58 2013 +0400 @@ -271,7 +271,7 @@ ngx_init_threads(int n, size_t size, ngx ngx_tid_t -ngx_thread_self() +ngx_thread_self(void) { ngx_int_t tid; diff -r 09d7faaef16f -r 15a7deeaa19a src/os/unix/ngx_freebsd_rfork_thread.h --- a/src/os/unix/ngx_freebsd_rfork_thread.h Thu May 23 10:23:21 2013 +0400 +++ b/src/os/unix/ngx_freebsd_rfork_thread.h Thu May 23 15:47:58 2013 +0400 @@ -57,7 +57,7 @@ extern size_t ngx_thread_stack_size; static ngx_inline ngx_int_t -ngx_gettid() +ngx_gettid(void) { char *sp; @@ -83,7 +83,7 @@ ngx_gettid() } -ngx_tid_t ngx_thread_self(); +ngx_tid_t ngx_thread_self(void); typedef ngx_uint_t ngx_tls_key_t; From ru at nginx.com Thu May 23 13:18:25 2013 From: ru at nginx.com (Ruslan Ermilov) Date: Thu, 23 May 2013 13:18:25 +0000 Subject: [nginx] Memcached: stricten header validation. Message-ID: details: http://hg.nginx.org/nginx/rev/a30ea5c6451d branches: changeset: 5226:a30ea5c6451d user: Ruslan Ermilov date: Thu May 23 16:26:10 2013 +0400 description: Memcached: stricten header validation. An invalid memcached reply that started with '\n' could cause segmentation fault. An invalid memcached reply "VALUE / 0 2\r?ok\r\nEND\r\n" was considered as a valid response. In addition, if memcached reports that the key was not found, set u->headers_in.content_length_n to 0. This ensures that ngx_http_memcached_filter() will not be called while previous code relied on always intercepting 404. Initialization of ctx->rest was moved to where it belongs. diffstat: src/http/modules/ngx_http_memcached_module.c | 26 +++++++++++++++++--------- 1 files changed, 17 insertions(+), 9 deletions(-) diffs (75 lines): diff -r 15a7deeaa19a -r a30ea5c6451d src/http/modules/ngx_http_memcached_module.c --- a/src/http/modules/ngx_http_memcached_module.c Thu May 23 15:47:58 2013 +0400 +++ b/src/http/modules/ngx_http_memcached_module.c Thu May 23 16:26:10 2013 +0400 @@ -197,7 +197,6 @@ ngx_http_memcached_handler(ngx_http_requ return NGX_HTTP_INTERNAL_SERVER_ERROR; } - ctx->rest = NGX_HTTP_MEMCACHED_END; ctx->request = r; ngx_http_set_ctx(r, ctx, ngx_http_memcached_module); @@ -309,10 +308,15 @@ ngx_http_memcached_process_header(ngx_ht found: + line.data = u->buffer.pos; + line.len = p - u->buffer.pos; + + if (line.len == 0 || *(p - 1) != CR) { + goto no_valid; + } + *p = '\0'; - - line.len = p - u->buffer.pos - 1; - line.data = u->buffer.pos; + line.len--; ngx_log_debug1(NGX_LOG_DEBUG_HTTP, r->connection->log, 0, "memcached: \"%V\"", &line); @@ -387,10 +391,9 @@ found: length: start = p; + p = line.data + line.len; - while (*p && *p++ != CR) { /* void */ } - - u->headers_in.content_length_n = ngx_atoof(start, p - start - 1); + u->headers_in.content_length_n = ngx_atoof(start, p - start); if (u->headers_in.content_length_n == -1) { ngx_log_error(NGX_LOG_ERR, r->connection->log, 0, "memcached sent invalid length in response \"%V\" " @@ -401,7 +404,7 @@ found: u->headers_in.status_n = 200; u->state->status = 200; - u->buffer.pos = p + 1; + u->buffer.pos = p + sizeof(CRLF) - 1; return NGX_OK; } @@ -410,8 +413,10 @@ found: ngx_log_error(NGX_LOG_INFO, r->connection->log, 0, "key: \"%V\" was not found by memcached", &ctx->key); + u->headers_in.content_length_n = 0; u->headers_in.status_n = 404; u->state->status = 404; + u->buffer.pos = p + sizeof("END" CRLF) - 1; u->keepalive = 1; return NGX_OK; @@ -435,7 +440,10 @@ ngx_http_memcached_filter_init(void *dat u = ctx->request->upstream; - u->length += NGX_HTTP_MEMCACHED_END; + if (u->headers_in.status_n != 404) { + u->length += NGX_HTTP_MEMCACHED_END; + ctx->rest = NGX_HTTP_MEMCACHED_END; + } return NGX_OK; } From mdounin at mdounin.ru Thu May 23 15:24:49 2013 From: mdounin at mdounin.ru (Maxim Dounin) Date: Thu, 23 May 2013 19:24:49 +0400 Subject: [PATCH] OCSP stapling: better handling of successful OCSP responses. In-Reply-To: References: <20130517132026.GX69760@mdounin.ru> <20130520105643.GZ69760@mdounin.ru> Message-ID: <20130523152449.GC69760@mdounin.ru> Hello! On Tue, May 21, 2013 at 05:17:52PM -0700, Piotr Sikora wrote: [...] > # HG changeset patch > # User Piotr Sikora > # Date 1369181290 25200 > # Node ID 7f52714a0b6a4229ab2fa9e640edbf80060eaf34 > # Parent 1d68b502088c9d6e6603e9699354e36d03d77f9c > OCSP stapling: add ssl_stapling_strict directive. > > When turned on (default), nginx will cache only OCSP responeses > with "good" certificate status. > > When turned off, nginx will cache all successful OCSP responseses, > regardless of the certificate status. Both directive name and flag name looks Directive name doesn't looks good. > While there, log certificate's common name and revocation reason, > because certificate status alone isn't very useful information. You may want to submit this as a separate patch. It might be also good idea to print certificate's information in ngx_ssl_ocsp_log_error(). It might be also good idea to use certificate file name instead of certificate subject (the same subject can be used for more than one certificate... on the other hand, there may be more than one certificate in a file - but only first one will be used for stapling). [...] > @@ -615,19 +621,47 @@ > goto error; > } > > - if (n != V_OCSP_CERTSTATUS_GOOD) { > - ngx_log_error(NGX_LOG_ERR, ctx->log, 0, > - "certificate status \"%s\" in the OCSP response", > - OCSP_cert_status_str(n)); > - goto error; > - } > - I don't think that changing check order is a good idea. > if (OCSP_check_validity(thisupdate, nextupdate, 300, -1) != 1) { > ngx_ssl_error(NGX_LOG_ERR, ctx->log, 0, > "OCSP_check_validity() failed"); > goto error; > } > > + if (n != V_OCSP_CERTSTATUS_GOOD) { > + ngx_str_set(&s, "unknown"); > + > + if (ctx->cert) { > + name = X509_get_subject_name(ctx->cert); > + if (name) { > + idx = X509_NAME_get_index_by_NID(name, NID_commonName, -1); > + if (idx != -1) { > + entry = X509_NAME_get_entry(name, idx); > + if (entry) { > + str = X509_NAME_ENTRY_get_data(entry); > + s.data = ASN1_STRING_data(str); > + s.len = ASN1_STRING_length(str); > + } > + } > + } > + } See above, it probably should be moved to ngx_ssl_ocsp_log_error(), and changed to print certificate file name instead. > + > + if (n == V_OCSP_CERTSTATUS_REVOKED && r != -1) { > + ngx_log_error(NGX_LOG_ERR, ctx->log, 0, > + "certificate status \"%s\" (reason: \"%s\") in the " > + "OCSP response for \"%V\"", > + OCSP_cert_status_str(n), OCSP_crl_reason_str(r), &s); > + > + } else { > + ngx_log_error(NGX_LOG_ERR, ctx->log, 0, > + "certificate status \"%s\" in the OCSP response " > + "for \"%V\"", OCSP_cert_status_str(n), &s); > + } Is printing revocation reason is really needed in error logs? It looks like too many lines of code just to print an additional minor detail. [...] -- Maxim Dounin http://nginx.org/en/donation.html From vbart at nginx.com Thu May 23 16:32:26 2013 From: vbart at nginx.com (Valentin Bartenev) Date: Thu, 23 May 2013 16:32:26 +0000 Subject: [nginx] Core: strengthen configuration syntax checker. Message-ID: details: http://hg.nginx.org/nginx/rev/ea41bba49e8a branches: changeset: 5227:ea41bba49e8a user: Valentin Bartenev date: Thu May 23 20:30:27 2013 +0400 description: Core: strengthen configuration syntax checker. It is now a syntax error if tokens passed to a custom configuration handler are terminated by "{". The following incorrect configuration is now properly rejected: map $v $v2 { a b { c d { e f { } diffstat: src/core/ngx_conf_file.c | 5 +++++ 1 files changed, 5 insertions(+), 0 deletions(-) diffs (15 lines): diff -r a30ea5c6451d -r ea41bba49e8a src/core/ngx_conf_file.c --- a/src/core/ngx_conf_file.c Thu May 23 16:26:10 2013 +0400 +++ b/src/core/ngx_conf_file.c Thu May 23 20:30:27 2013 +0400 @@ -225,6 +225,11 @@ ngx_conf_parse(ngx_conf_t *cf, ngx_str_t * "types { ... }" directive */ + if (rc == NGX_CONF_BLOCK_START) { + ngx_conf_log_error(NGX_LOG_EMERG, cf, 0, "unexpected \"{\""); + goto failed; + } + rv = (*cf->handler)(cf, NULL, cf->handler_conf); if (rv == NGX_CONF_OK) { continue; From ru at nginx.com Thu May 23 19:52:24 2013 From: ru at nginx.com (Ruslan Ermilov) Date: Thu, 23 May 2013 23:52:24 +0400 Subject: HttpAccessModule and unix domain sockets In-Reply-To: <20130522134941.GE69760@mdounin.ru> References: <20130522134941.GE69760@mdounin.ru> Message-ID: <20130523195224.GL42801@lo0.su> On Wed, May 22, 2013 at 05:49:41PM +0400, Maxim Dounin wrote: > On Tue, May 21, 2013 at 10:27:21PM +0300, Sorin Manole wrote: > > > Hi all, > > > > It seems that when using HttpAccessModule directives to deny requests, they > > don't seem to work if the server is listening on a unix domain socket. Even > > when using deny all. > > Can someone confirm and it's not just me making some stupid mistake ? > > Yes, access module allow/deny directives currently only able to > limit ipv4 and ipv6 addresses. > > > Now if that is the case, would it be a good idea to add this functionality > > to the module ? Maybe add a new parameter like "deny unix" or something ? > > Or was this left out on purpose for a reason or another ? > > It probably should be expanded to support "unix:" special address > like set_real_ip_from does (see http://nginx.org/r/set_real_ip_from). # HG changeset patch # User Ruslan Ermilov # Date 1369338540 -14400 # Node ID d26c24c812846f2993947a0514efa9556d31f404 # Parent a30ea5c6451dcae3ce1e6d9eabe718c0222e5d9f Access: support for UNIX-domain client addresses (ticket #359). diff --git a/src/http/modules/ngx_http_access_module.c b/src/http/modules/ngx_http_access_module.c --- a/src/http/modules/ngx_http_access_module.c +++ b/src/http/modules/ngx_http_access_module.c @@ -26,11 +26,22 @@ typedef struct { #endif +#if (NGX_HAVE_UNIX_DOMAIN) + +typedef struct { + ngx_uint_t deny; /* unsigned deny:1; */ +} ngx_http_access_rule_un_t; + +#endif + typedef struct { ngx_array_t *rules; /* array of ngx_http_access_rule_t */ #if (NGX_HAVE_INET6) ngx_array_t *rules6; /* array of ngx_http_access_rule6_t */ #endif +#if (NGX_HAVE_UNIX_DOMAIN) + ngx_array_t *rules_un; /* array of ngx_http_access_rule_un_t */ +#endif } ngx_http_access_loc_conf_t; @@ -41,6 +52,10 @@ static ngx_int_t ngx_http_access_inet(ng static ngx_int_t ngx_http_access_inet6(ngx_http_request_t *r, ngx_http_access_loc_conf_t *alcf, u_char *p); #endif +#if (NGX_HAVE_UNIX_DOMAIN) +static ngx_int_t ngx_http_access_unix(ngx_http_request_t *r, + ngx_http_access_loc_conf_t *alcf); +#endif static ngx_int_t ngx_http_access_found(ngx_http_request_t *r, ngx_uint_t deny); static char *ngx_http_access_rule(ngx_conf_t *cf, ngx_command_t *cmd, void *conf); @@ -145,6 +160,15 @@ ngx_http_access_handler(ngx_http_request } #endif + +#if (NGX_HAVE_UNIX_DOMAIN) + + case AF_UNIX: + if (alcf->rules_un) { + return ngx_http_access_unix(r, alcf); + } + +#endif } return NGX_DECLINED; @@ -221,6 +245,25 @@ ngx_http_access_inet6(ngx_http_request_t #endif +#if (NGX_HAVE_UNIX_DOMAIN) + +static ngx_int_t +ngx_http_access_unix(ngx_http_request_t *r, ngx_http_access_loc_conf_t *alcf) +{ + ngx_uint_t i; + ngx_http_access_rule_un_t *rule_un; + + rule_un = alcf->rules_un->elts; + for (i = 0; i < alcf->rules_un->nelts; i++) { + return ngx_http_access_found(r, rule_un[i].deny); + } + + return NGX_DECLINED; +} + +#endif + + static ngx_int_t ngx_http_access_found(ngx_http_request_t *r, ngx_uint_t deny) { @@ -246,13 +289,16 @@ ngx_http_access_rule(ngx_conf_t *cf, ngx { ngx_http_access_loc_conf_t *alcf = conf; - ngx_int_t rc; - ngx_uint_t all; - ngx_str_t *value; - ngx_cidr_t cidr; - ngx_http_access_rule_t *rule; + ngx_int_t rc; + ngx_uint_t all; + ngx_str_t *value; + ngx_cidr_t cidr; + ngx_http_access_rule_t *rule; #if (NGX_HAVE_INET6) - ngx_http_access_rule6_t *rule6; + ngx_http_access_rule6_t *rule6; +#endif +#if (NGX_HAVE_UNIX_DOMAIN) + ngx_http_access_rule_un_t *rule_un; #endif ngx_memzero(&cidr, sizeof(ngx_cidr_t)); @@ -263,7 +309,19 @@ ngx_http_access_rule(ngx_conf_t *cf, ngx if (!all) { +#if (NGX_HAVE_UNIX_DOMAIN) + + if (value[1].len == 5 && ngx_strcmp(value[1].data, "unix:") == 0) { + cidr.family = AF_UNIX; + rc = NGX_OK; + + } else { + rc = ngx_ptocidr(&value[1], &cidr); + } + +#else rc = ngx_ptocidr(&value[1], &cidr); +#endif if (rc == NGX_ERROR) { ngx_conf_log_error(NGX_LOG_EMERG, cf, 0, @@ -307,6 +365,32 @@ ngx_http_access_rule(ngx_conf_t *cf, ngx /* "all" passes through */ #endif +#if (NGX_HAVE_UNIX_DOMAIN) + case AF_UNIX: + case 0: /* all */ + + if (alcf->rules_un == NULL) { + alcf->rules_un = ngx_array_create(cf->pool, 1, + sizeof(ngx_http_access_rule_un_t)); + if (alcf->rules_un == NULL) { + return NGX_CONF_ERROR; + } + } + + rule_un = ngx_array_push(alcf->rules_un); + if (rule_un == NULL) { + return NGX_CONF_ERROR; + } + + rule_un->deny = (value[0].data[0] == 'd') ? 1 : 0; + + if (!all) { + break; + } + + /* "all" passes through */ +#endif + default: /* AF_INET */ if (alcf->rules == NULL) { From faskiri.devel at gmail.com Fri May 24 13:39:58 2013 From: faskiri.devel at gmail.com (Fasih) Date: Fri, 24 May 2013 19:09:58 +0530 Subject: Socket leak Message-ID: Hi all I have been seeing slow but steady socket leak in nginx ever since I upgraded from 1.0.5 to 1.2.6. I have my custom module in nginx which I was sure what was the leak. This is how I went about investigating: 1. Configure nginx with one worker 2. strace on the worker process, tracing read/readv/write/writev/close/shutdown calls 3. Every now and then, for all the open fds (from ls -l /proc//fd), check the socket that is not available in netstat -pane 4. What I saw was, the leaking socket always had the last operation as writev which returned an error. 5. Increased the nginx log level to info and verified that nginx was getting ECONNRESET or EPIPE on writev failure. Which was OK. 6. Traced back in code to see how it is handled, the error translates to CHAIN_ERROR and eventually causes ngx_http_finalize_request to be called. This in turn calls ngx_http_terminate_request. However, in this function, the request is not terminated if r->write_event_handler is set. This seems to be set if the request handler is a user module. I think the rationale for the check is, if there is a module who is handling the request, dont terminate yet, wait for a write event on the socket and then terminate it (which is why I thought it is setting r->write_event_handler to ngx_http_terminate_handler). I tried to repro this w/ empty_gif_handler however, it sends header and body in one call to writev which I cant get to fail in my test environment. To reproduce the bug, if I replace the call to ngx_http_send_response with ngx_http_send_header and ngx_http_output_filter (as used by ngx_upstream or other modules which dont have the headers and body together), I could reproduce the leak. I have a client that sends a request and closes the socket immediately, nginx sees the error, prints the info log, and then it doesnt close the socket. I have a small patch attached, the fix I did is basically saying that if there is a connection error, there is no point setting write_event_handler as there wont be any activity on the socket, so just terminate it immediately. I could be very wrong in the understanding of the code flow. My patch just fixes this and I am not very sure if this is the right fix. Please let me know. I will try to add a testcase to reproduce this in the nginx test framework. Thank you for your patience. Regards +Fasih -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- diff --git a/nginx-1.2.6/src/http/ngx_http_request.c b/nginx-1.2.6/src/http/ngx_http_request.c --- a/nginx-1.2.6/src/http/ngx_http_request.c +++ b/nginx-1.2.6/src/http/ngx_http_request.c @@ -2172,7 +2172,7 @@ ngx_http_terminate_request(ngx_http_request_t *r, ngx_int_t rc) "http terminate cleanup count:%d blk:%d", mr->count, mr->blocked); - if (mr->write_event_handler) { + if (mr->write_event_handler && !c->error) { if (mr->blocked) { return; From faskiri.devel at gmail.com Fri May 24 14:22:14 2013 From: faskiri.devel at gmail.com (Fasih) Date: Fri, 24 May 2013 19:52:14 +0530 Subject: Socket leak In-Reply-To: References: Message-ID: Sorry. Attached the wrong file. On Fri, May 24, 2013 at 7:09 PM, Fasih wrote: > Hi all > > I have been seeing slow but steady socket leak in nginx ever since I > upgraded from 1.0.5 to 1.2.6. I have my custom module in nginx which I was > sure what was the leak. This is how I went about investigating: > 1. Configure nginx with one worker > 2. strace on the worker process, tracing > read/readv/write/writev/close/shutdown calls > 3. Every now and then, for all the open fds (from ls -l /proc//fd), > check the socket that is not available in netstat -pane > 4. What I saw was, the leaking socket always had the last operation as > writev which returned an error. > 5. Increased the nginx log level to info and verified that nginx was > getting ECONNRESET or EPIPE on writev failure. Which was OK. > 6. Traced back in code to see how it is handled, the error translates to > CHAIN_ERROR and eventually causes ngx_http_finalize_request to be called. > This in turn calls ngx_http_terminate_request. > > However, in this function, the request is not terminated if > r->write_event_handler is set. This seems to be set if the request handler > is a user module. I think the rationale for the check is, if there is a > module who is handling the request, dont terminate yet, wait for a write > event on the socket and then terminate it (which is why I thought it is > setting r->write_event_handler to ngx_http_terminate_handler). > > I tried to repro this w/ empty_gif_handler however, it sends header and > body in one call to writev which I cant get to fail in my test environment. > To reproduce the bug, if I replace the call to ngx_http_send_response with > ngx_http_send_header and ngx_http_output_filter (as used by ngx_upstream or > other modules which dont have the headers and body together), I could > reproduce the leak. I have a client that sends a request and closes the > socket immediately, nginx sees the error, prints the info log, and then it > doesnt close the socket. > > I have a small patch attached, the fix I did is basically saying that if > there is a connection error, there is no point setting write_event_handler > as there wont be any activity on the socket, so just terminate it > immediately. > > I could be very wrong in the understanding of the code flow. My patch just > fixes this and I am not very sure if this is the right fix. Please let me > know. > > I will try to add a testcase to reproduce this in the nginx test framework. > > Thank you for your patience. > > Regards > +Fasih > -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- diff --git a/nginx-1.2.6/src/http/ngx_http_request.c b/nginx-1.2.6/src/http/ngx_http_request.c --- a/nginx-1.2.6/src/http/ngx_http_request.c +++ b/nginx-1.2.6/src/http/ngx_http_request.c @@ -2172,7 +2172,7 @@ ngx_http_terminate_request(ngx_http_request_t *r, ngx_int_t rc) "http terminate cleanup count:%d blk:%d", mr->count, mr->blocked); - if (mr->write_event_handler) { + if (mr->write_event_handler && !r->connection->error) { if (mr->blocked) { return; From mdounin at mdounin.ru Fri May 24 14:54:50 2013 From: mdounin at mdounin.ru (Maxim Dounin) Date: Fri, 24 May 2013 18:54:50 +0400 Subject: Socket leak In-Reply-To: References: Message-ID: <20130524145450.GI69760@mdounin.ru> Hello! On Fri, May 24, 2013 at 07:09:58PM +0530, Fasih wrote: > Hi all > > I have been seeing slow but steady socket leak in nginx ever since I > upgraded from 1.0.5 to 1.2.6. I have my custom module in nginx which I was > sure what was the leak. This is how I went about investigating: > 1. Configure nginx with one worker > 2. strace on the worker process, tracing > read/readv/write/writev/close/shutdown calls > 3. Every now and then, for all the open fds (from ls -l /proc//fd), > check the socket that is not available in netstat -pane > 4. What I saw was, the leaking socket always had the last operation as > writev which returned an error. > 5. Increased the nginx log level to info and verified that nginx was > getting ECONNRESET or EPIPE on writev failure. Which was OK. > 6. Traced back in code to see how it is handled, the error translates to > CHAIN_ERROR and eventually causes ngx_http_finalize_request to be called. > This in turn calls ngx_http_terminate_request. > > However, in this function, the request is not terminated if > r->write_event_handler is set. This seems to be set if the request handler > is a user module. I think the rationale for the check is, if there is a > module who is handling the request, dont terminate yet, wait for a write > event on the socket and then terminate it (which is why I thought it is > setting r->write_event_handler to ngx_http_terminate_handler). Rationale is to make sure there are no functions on stack which assume request object is here and will try to access it after we'll free request data. The r->write_event_handler (that is, ngx_http_terminate_handler()) is expected to be called by a ngx_http_run_posted_requests() which in turn is called by low-level event handling functions (notably, ngx_http_request_handler()). > I tried to repro this w/ empty_gif_handler however, it sends header and > body in one call to writev which I cant get to fail in my test environment. > To reproduce the bug, if I replace the call to ngx_http_send_response with > ngx_http_send_header and ngx_http_output_filter (as used by ngx_upstream or > other modules which dont have the headers and body together), I could > reproduce the leak. I have a client that sends a request and closes the > socket immediately, nginx sees the error, prints the info log, and then it > doesnt close the socket. > > I have a small patch attached, the fix I did is basically saying that if > there is a connection error, there is no point setting write_event_handler > as there wont be any activity on the socket, so just terminate it > immediately. > > I could be very wrong in the understanding of the code flow. My patch just > fixes this and I am not very sure if this is the right fix. Please let me > know. > > I will try to add a testcase to reproduce this in the nginx test framework. The patch looks wrong, see above. Could you please show a backtrace up to ngx_http_terminate_request() with mr->write_event_handler and c->error set (i.e. where you think leak happens)? You may also want to upgrade to a more recent version, e.g. 1.5.0, to make sure the problem you are facing isn't already fixed. -- Maxim Dounin http://nginx.org/en/donation.html From faskiri.devel at gmail.com Fri May 24 16:49:57 2013 From: faskiri.devel at gmail.com (Fasih) Date: Fri, 24 May 2013 22:19:57 +0530 Subject: Socket leak In-Reply-To: <20130524145450.GI69760@mdounin.ru> References: <20130524145450.GI69760@mdounin.ru> Message-ID: Hello Thanks for the really quick reply. The ngx_http_run_posted_requests totally made sense and explained the bit that I was missing. I get the bug when writev called in the context of a request handler gets an error. The repro I had was basically with nginx running on a server and client on my laptop over wireless @ work. I am not @ work now and from my home connection I am unable to repro this. Will send you the backtrace as soon as I get it again. On Fri, May 24, 2013 at 8:24 PM, Maxim Dounin wrote: > Hello! > > On Fri, May 24, 2013 at 07:09:58PM +0530, Fasih wrote: > > > Hi all > > > > I have been seeing slow but steady socket leak in nginx ever since I > > upgraded from 1.0.5 to 1.2.6. I have my custom module in nginx which I > was > > sure what was the leak. This is how I went about investigating: > > 1. Configure nginx with one worker > > 2. strace on the worker process, tracing > > read/readv/write/writev/close/shutdown calls > > 3. Every now and then, for all the open fds (from ls -l /proc//fd), > > check the socket that is not available in netstat -pane > > 4. What I saw was, the leaking socket always had the last operation as > > writev which returned an error. > > 5. Increased the nginx log level to info and verified that nginx was > > getting ECONNRESET or EPIPE on writev failure. Which was OK. > > 6. Traced back in code to see how it is handled, the error translates to > > CHAIN_ERROR and eventually causes ngx_http_finalize_request to be called. > > This in turn calls ngx_http_terminate_request. > > > > However, in this function, the request is not terminated if > > r->write_event_handler is set. This seems to be set if the request > handler > > is a user module. I think the rationale for the check is, if there is a > > module who is handling the request, dont terminate yet, wait for a write > > event on the socket and then terminate it (which is why I thought it is > > setting r->write_event_handler to ngx_http_terminate_handler). > > Rationale is to make sure there are no functions on stack which > assume request object is here and will try to access it after > we'll free request data. > > The r->write_event_handler (that is, ngx_http_terminate_handler()) > is expected to be called by a ngx_http_run_posted_requests() which > in turn is called by low-level event handling functions (notably, > ngx_http_request_handler()). > > > I tried to repro this w/ empty_gif_handler however, it sends header and > > body in one call to writev which I cant get to fail in my test > environment. > > To reproduce the bug, if I replace the call to ngx_http_send_response > with > > ngx_http_send_header and ngx_http_output_filter (as used by ngx_upstream > or > > other modules which dont have the headers and body together), I could > > reproduce the leak. I have a client that sends a request and closes the > > socket immediately, nginx sees the error, prints the info log, and then > it > > doesnt close the socket. > > > > I have a small patch attached, the fix I did is basically saying that if > > there is a connection error, there is no point setting > write_event_handler > > as there wont be any activity on the socket, so just terminate it > > immediately. > > > > I could be very wrong in the understanding of the code flow. My patch > just > > fixes this and I am not very sure if this is the right fix. Please let me > > know. > > > > I will try to add a testcase to reproduce this in the nginx test > framework. > > The patch looks wrong, see above. > > Could you please show a backtrace up to > ngx_http_terminate_request() with mr->write_event_handler and > c->error set (i.e. where you think leak happens)? > > You may also want to upgrade to a more recent version, e.g. 1.5.0, > to make sure the problem you are facing isn't already fixed. > > -- > Maxim Dounin > http://nginx.org/en/donation.html > > _______________________________________________ > nginx-devel mailing list > nginx-devel at nginx.org > http://mailman.nginx.org/mailman/listinfo/nginx-devel > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mdounin at mdounin.ru Fri May 24 18:32:42 2013 From: mdounin at mdounin.ru (Maxim Dounin) Date: Fri, 24 May 2013 18:32:42 +0000 Subject: [nginx] SSI: ssi_last_modified directive. Message-ID: details: http://hg.nginx.org/nginx/rev/cbd4bbe976d4 branches: changeset: 5228:cbd4bbe976d4 user: Maxim Dounin date: Fri May 24 22:27:23 2013 +0400 description: SSI: ssi_last_modified directive. The "ssi_last_modified" directive allows to preserve Last-Modified header in SSI responses. The directive is similar to SSILastModified one available in Apache: http://httpd.apache.org/docs/2.4/mod/mod_include.html#ssilastmodified Patch by Alexey Kolpakov. diffstat: src/http/modules/ngx_http_ssi_filter_module.c | 15 ++++++++++++++- 1 files changed, 14 insertions(+), 1 deletions(-) diffs (55 lines): diff --git a/src/http/modules/ngx_http_ssi_filter_module.c b/src/http/modules/ngx_http_ssi_filter_module.c --- a/src/http/modules/ngx_http_ssi_filter_module.c +++ b/src/http/modules/ngx_http_ssi_filter_module.c @@ -21,6 +21,7 @@ typedef struct { ngx_flag_t enable; ngx_flag_t silent_errors; ngx_flag_t ignore_recycled_buffers; + ngx_flag_t last_modified; ngx_hash_t types; @@ -162,6 +163,13 @@ static ngx_command_t ngx_http_ssi_filte offsetof(ngx_http_ssi_loc_conf_t, types_keys), &ngx_http_html_default_types[0] }, + { ngx_string("ssi_last_modified"), + NGX_HTTP_MAIN_CONF|NGX_HTTP_SRV_CONF|NGX_HTTP_LOC_CONF|NGX_CONF_FLAG, + ngx_conf_set_flag_slot, + NGX_HTTP_LOC_CONF_OFFSET, + offsetof(ngx_http_ssi_loc_conf_t, last_modified), + NULL }, + ngx_null_command }; @@ -359,9 +367,12 @@ ngx_http_ssi_header_filter(ngx_http_requ if (r == r->main) { ngx_http_clear_content_length(r); - ngx_http_clear_last_modified(r); ngx_http_clear_accept_ranges(r); ngx_http_clear_etag(r); + + if (!slcf->last_modified) { + ngx_http_clear_last_modified(r); + } } return ngx_http_next_header_filter(r); @@ -2878,6 +2889,7 @@ ngx_http_ssi_create_loc_conf(ngx_conf_t slcf->enable = NGX_CONF_UNSET; slcf->silent_errors = NGX_CONF_UNSET; slcf->ignore_recycled_buffers = NGX_CONF_UNSET; + slcf->last_modified = NGX_CONF_UNSET; slcf->min_file_chunk = NGX_CONF_UNSET_SIZE; slcf->value_len = NGX_CONF_UNSET_SIZE; @@ -2896,6 +2908,7 @@ ngx_http_ssi_merge_loc_conf(ngx_conf_t * ngx_conf_merge_value(conf->silent_errors, prev->silent_errors, 0); ngx_conf_merge_value(conf->ignore_recycled_buffers, prev->ignore_recycled_buffers, 0); + ngx_conf_merge_value(conf->last_modified, prev->last_modified, 0); ngx_conf_merge_size_value(conf->min_file_chunk, prev->min_file_chunk, 1024); ngx_conf_merge_size_value(conf->value_len, prev->value_len, 255); From mdounin at mdounin.ru Fri May 24 18:32:42 2013 From: mdounin at mdounin.ru (Maxim Dounin) Date: Fri, 24 May 2013 18:32:42 +0000 Subject: [nginx] Sub filter: sub_filter_last_modified directive. Message-ID: details: http://hg.nginx.org/nginx/rev/4c1a604b0285 branches: changeset: 5229:4c1a604b0285 user: Maxim Dounin date: Fri May 24 22:27:30 2013 +0400 description: Sub filter: sub_filter_last_modified directive. Directive is similar to ssi_last_modified introduced by previous commit. Patch by Alexey Kolpakov. diffstat: src/http/modules/ngx_http_sub_filter_module.c | 15 ++++++++++++++- 1 files changed, 14 insertions(+), 1 deletions(-) diffs (54 lines): diff --git a/src/http/modules/ngx_http_sub_filter_module.c b/src/http/modules/ngx_http_sub_filter_module.c --- a/src/http/modules/ngx_http_sub_filter_module.c +++ b/src/http/modules/ngx_http_sub_filter_module.c @@ -17,6 +17,7 @@ typedef struct { ngx_hash_t types; ngx_flag_t once; + ngx_flag_t last_modified; ngx_array_t *types_keys; } ngx_http_sub_loc_conf_t; @@ -89,6 +90,13 @@ static ngx_command_t ngx_http_sub_filte offsetof(ngx_http_sub_loc_conf_t, once), NULL }, + { ngx_string("sub_filter_last_modified"), + NGX_HTTP_MAIN_CONF|NGX_HTTP_SRV_CONF|NGX_HTTP_LOC_CONF|NGX_CONF_FLAG, + ngx_conf_set_flag_slot, + NGX_HTTP_LOC_CONF_OFFSET, + offsetof(ngx_http_sub_loc_conf_t, last_modified), + NULL }, + ngx_null_command }; @@ -167,8 +175,11 @@ ngx_http_sub_header_filter(ngx_http_requ if (r == r->main) { ngx_http_clear_content_length(r); - ngx_http_clear_last_modified(r); ngx_http_clear_etag(r); + + if (!slcf->last_modified) { + ngx_http_clear_last_modified(r); + } } return ngx_http_next_header_filter(r); @@ -674,6 +685,7 @@ ngx_http_sub_create_conf(ngx_conf_t *cf) */ slcf->once = NGX_CONF_UNSET; + slcf->last_modified = NGX_CONF_UNSET; return slcf; } @@ -687,6 +699,7 @@ ngx_http_sub_merge_conf(ngx_conf_t *cf, ngx_conf_merge_value(conf->once, prev->once, 1); ngx_conf_merge_str_value(conf->match, prev->match, ""); + ngx_conf_merge_value(conf->last_modified, prev->last_modified, 0); if (conf->value.value.data == NULL) { conf->value = prev->value; From mdounin at mdounin.ru Fri May 24 18:32:42 2013 From: mdounin at mdounin.ru (Maxim Dounin) Date: Fri, 24 May 2013 18:32:42 +0000 Subject: [nginx] Xslt: xslt_last_modified directive. Message-ID: details: http://hg.nginx.org/nginx/rev/2139768ee404 branches: changeset: 5230:2139768ee404 user: Maxim Dounin date: Fri May 24 22:28:09 2013 +0400 description: Xslt: xslt_last_modified directive. Directive is similar to ssi_last_modified and sub_filter_last_modified directives introduced by previous commits. diffstat: src/http/modules/ngx_http_xslt_filter_module.c | 26 ++++++++++++++++++++++---- 1 files changed, 22 insertions(+), 4 deletions(-) diffs (72 lines): diff --git a/src/http/modules/ngx_http_xslt_filter_module.c b/src/http/modules/ngx_http_xslt_filter_module.c --- a/src/http/modules/ngx_http_xslt_filter_module.c +++ b/src/http/modules/ngx_http_xslt_filter_module.c @@ -58,6 +58,7 @@ typedef struct { ngx_hash_t types; ngx_array_t *types_keys; ngx_array_t *params; /* ngx_http_xslt_param_t */ + ngx_flag_t last_modified; } ngx_http_xslt_filter_loc_conf_t; @@ -150,6 +151,13 @@ static ngx_command_t ngx_http_xslt_filt offsetof(ngx_http_xslt_filter_loc_conf_t, types_keys), &ngx_http_xslt_default_types[0] }, + { ngx_string("xslt_last_modified"), + NGX_HTTP_MAIN_CONF|NGX_HTTP_SRV_CONF|NGX_HTTP_LOC_CONF|NGX_CONF_FLAG, + ngx_conf_set_flag_slot, + NGX_HTTP_LOC_CONF_OFFSET, + offsetof(ngx_http_xslt_filter_loc_conf_t, last_modified), + NULL }, + ngx_null_command }; @@ -300,9 +308,10 @@ static ngx_int_t ngx_http_xslt_send(ngx_http_request_t *r, ngx_http_xslt_filter_ctx_t *ctx, ngx_buf_t *b) { - ngx_int_t rc; - ngx_chain_t out; - ngx_pool_cleanup_t *cln; + ngx_int_t rc; + ngx_chain_t out; + ngx_pool_cleanup_t *cln; + ngx_http_xslt_filter_loc_conf_t *conf; ctx->done = 1; @@ -327,8 +336,13 @@ ngx_http_xslt_send(ngx_http_request_t *r r->headers_out.content_length = NULL; } - ngx_http_clear_last_modified(r); ngx_http_clear_etag(r); + + conf = ngx_http_get_module_loc_conf(r, ngx_http_xslt_filter_module); + + if (!conf->last_modified) { + ngx_http_clear_last_modified(r); + } } rc = ngx_http_next_header_filter(r); @@ -1058,6 +1072,8 @@ ngx_http_xslt_filter_create_conf(ngx_con * conf->params = NULL; */ + conf->last_modified = NGX_CONF_UNSET; + return conf; } @@ -1088,6 +1104,8 @@ ngx_http_xslt_filter_merge_conf(ngx_conf return NGX_CONF_ERROR; } + ngx_conf_merge_value(conf->last_modified, prev->last_modified, 0); + return NGX_CONF_OK; } From alexander_koch_log at lavabit.com Sun May 26 17:40:51 2013 From: alexander_koch_log at lavabit.com (Alex) Date: Sun, 26 May 2013 19:40:51 +0200 Subject: OPEN_FILE_CACHE memory usage Message-ID: <51A24923.50608@lavabit.com> Hi, Per the max number set for open_file_cache size, is it possible to approximate how much memory will be used ? Also what would be a reasonable high number until performance drop in the lookup? Thanks, Alex From faskiri.devel at gmail.com Mon May 27 05:39:20 2013 From: faskiri.devel at gmail.com (Fasih) Date: Mon, 27 May 2013 11:09:20 +0530 Subject: Socket leak In-Reply-To: References: <20130524145450.GI69760@mdounin.ru> Message-ID: Hi Maxim I found the rootcause. This was a problem with my plugin. Your explanation on posted_requests helped a lot in debugging the problem. The issue was, my plugin for some unavoidable reasons holds reference to the ngx_http_request_t and calls finalize once it is done or it sees some error. I didnt call ngx_run_posted_request() like ngx_http_request_handler does. The actual call to writev happens *after* the request_handler returns because of which it doesnt see the c->error or the posted request and hence doesnt clean it up. I will fix my plugin to go with the normal nginx flow soon, but till then this fix (calling run_posted after finalize) fixes my problem. I did see the diff from 1.0.5 -> 1.2.6 and couldnt see what could have caused this. Thanks again for the help and really helpful reply. +Fasih On Fri, May 24, 2013 at 10:19 PM, Fasih wrote: > Hello > > Thanks for the really quick reply. The ngx_http_run_posted_requests > totally made sense and explained the bit that I was missing. > > I get the bug when writev called in the context of a request handler gets > an error. The repro I had was basically with nginx running on a server > and client on my laptop over wireless @ work. I am not @ work now and from > my home connection I am unable to repro this. Will send you the backtrace > as soon as I get it again. > > > On Fri, May 24, 2013 at 8:24 PM, Maxim Dounin wrote: > >> Hello! >> >> On Fri, May 24, 2013 at 07:09:58PM +0530, Fasih wrote: >> >> > Hi all >> > >> > I have been seeing slow but steady socket leak in nginx ever since I >> > upgraded from 1.0.5 to 1.2.6. I have my custom module in nginx which I >> was >> > sure what was the leak. This is how I went about investigating: >> > 1. Configure nginx with one worker >> > 2. strace on the worker process, tracing >> > read/readv/write/writev/close/shutdown calls >> > 3. Every now and then, for all the open fds (from ls -l /proc//fd), >> > check the socket that is not available in netstat -pane >> > 4. What I saw was, the leaking socket always had the last operation as >> > writev which returned an error. >> > 5. Increased the nginx log level to info and verified that nginx was >> > getting ECONNRESET or EPIPE on writev failure. Which was OK. >> > 6. Traced back in code to see how it is handled, the error translates to >> > CHAIN_ERROR and eventually causes ngx_http_finalize_request to be >> called. >> > This in turn calls ngx_http_terminate_request. >> > >> > However, in this function, the request is not terminated if >> > r->write_event_handler is set. This seems to be set if the request >> handler >> > is a user module. I think the rationale for the check is, if there is a >> > module who is handling the request, dont terminate yet, wait for a write >> > event on the socket and then terminate it (which is why I thought it is >> > setting r->write_event_handler to ngx_http_terminate_handler). >> >> Rationale is to make sure there are no functions on stack which >> assume request object is here and will try to access it after >> we'll free request data. >> >> The r->write_event_handler (that is, ngx_http_terminate_handler()) >> is expected to be called by a ngx_http_run_posted_requests() which >> in turn is called by low-level event handling functions (notably, >> ngx_http_request_handler()). >> >> > I tried to repro this w/ empty_gif_handler however, it sends header and >> > body in one call to writev which I cant get to fail in my test >> environment. >> > To reproduce the bug, if I replace the call to ngx_http_send_response >> with >> > ngx_http_send_header and ngx_http_output_filter (as used by >> ngx_upstream or >> > other modules which dont have the headers and body together), I could >> > reproduce the leak. I have a client that sends a request and closes the >> > socket immediately, nginx sees the error, prints the info log, and then >> it >> > doesnt close the socket. >> > >> > I have a small patch attached, the fix I did is basically saying that if >> > there is a connection error, there is no point setting >> write_event_handler >> > as there wont be any activity on the socket, so just terminate it >> > immediately. >> > >> > I could be very wrong in the understanding of the code flow. My patch >> just >> > fixes this and I am not very sure if this is the right fix. Please let >> me >> > know. >> > >> > I will try to add a testcase to reproduce this in the nginx test >> framework. >> >> The patch looks wrong, see above. >> >> Could you please show a backtrace up to >> ngx_http_terminate_request() with mr->write_event_handler and >> c->error set (i.e. where you think leak happens)? >> >> You may also want to upgrade to a more recent version, e.g. 1.5.0, >> to make sure the problem you are facing isn't already fixed. >> >> -- >> Maxim Dounin >> http://nginx.org/en/donation.html >> >> _______________________________________________ >> nginx-devel mailing list >> nginx-devel at nginx.org >> http://mailman.nginx.org/mailman/listinfo/nginx-devel >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mdounin at mdounin.ru Mon May 27 13:25:24 2013 From: mdounin at mdounin.ru (Maxim Dounin) Date: Mon, 27 May 2013 13:25:24 +0000 Subject: [nginx] Upstream: http_403 support in proxy_next_upstream (and f... Message-ID: details: http://hg.nginx.org/nginx/rev/05c53652e7b4 branches: changeset: 5231:05c53652e7b4 user: Maxim Dounin date: Mon May 27 16:54:09 2013 +0400 description: Upstream: http_403 support in proxy_next_upstream (and friends). The parameter is mostly identical to http_404, and is expected to be used in similar situations. The 403 code might be returned by a backend instead of 404 on initial sync of new directories with rsync. See here for feature request and additional details: http://mailman.nginx.org/pipermail/nginx-ru/2013-April/050920.html diffstat: src/http/modules/ngx_http_fastcgi_module.c | 1 + src/http/modules/ngx_http_proxy_module.c | 1 + src/http/modules/ngx_http_scgi_module.c | 1 + src/http/modules/ngx_http_uwsgi_module.c | 1 + src/http/ngx_http_upstream.c | 10 +++++++++- src/http/ngx_http_upstream.h | 10 ++++++---- 6 files changed, 19 insertions(+), 5 deletions(-) diffs (106 lines): diff --git a/src/http/modules/ngx_http_fastcgi_module.c b/src/http/modules/ngx_http_fastcgi_module.c --- a/src/http/modules/ngx_http_fastcgi_module.c +++ b/src/http/modules/ngx_http_fastcgi_module.c @@ -185,6 +185,7 @@ static ngx_conf_bitmask_t ngx_http_fast { ngx_string("invalid_header"), NGX_HTTP_UPSTREAM_FT_INVALID_HEADER }, { ngx_string("http_500"), NGX_HTTP_UPSTREAM_FT_HTTP_500 }, { ngx_string("http_503"), NGX_HTTP_UPSTREAM_FT_HTTP_503 }, + { ngx_string("http_403"), NGX_HTTP_UPSTREAM_FT_HTTP_403 }, { ngx_string("http_404"), NGX_HTTP_UPSTREAM_FT_HTTP_404 }, { ngx_string("updating"), NGX_HTTP_UPSTREAM_FT_UPDATING }, { ngx_string("off"), NGX_HTTP_UPSTREAM_FT_OFF }, diff --git a/src/http/modules/ngx_http_proxy_module.c b/src/http/modules/ngx_http_proxy_module.c --- a/src/http/modules/ngx_http_proxy_module.c +++ b/src/http/modules/ngx_http_proxy_module.c @@ -178,6 +178,7 @@ static ngx_conf_bitmask_t ngx_http_prox { ngx_string("http_502"), NGX_HTTP_UPSTREAM_FT_HTTP_502 }, { ngx_string("http_503"), NGX_HTTP_UPSTREAM_FT_HTTP_503 }, { ngx_string("http_504"), NGX_HTTP_UPSTREAM_FT_HTTP_504 }, + { ngx_string("http_403"), NGX_HTTP_UPSTREAM_FT_HTTP_403 }, { ngx_string("http_404"), NGX_HTTP_UPSTREAM_FT_HTTP_404 }, { ngx_string("updating"), NGX_HTTP_UPSTREAM_FT_UPDATING }, { ngx_string("off"), NGX_HTTP_UPSTREAM_FT_OFF }, diff --git a/src/http/modules/ngx_http_scgi_module.c b/src/http/modules/ngx_http_scgi_module.c --- a/src/http/modules/ngx_http_scgi_module.c +++ b/src/http/modules/ngx_http_scgi_module.c @@ -65,6 +65,7 @@ static ngx_conf_bitmask_t ngx_http_scgi_ { ngx_string("invalid_header"), NGX_HTTP_UPSTREAM_FT_INVALID_HEADER }, { ngx_string("http_500"), NGX_HTTP_UPSTREAM_FT_HTTP_500 }, { ngx_string("http_503"), NGX_HTTP_UPSTREAM_FT_HTTP_503 }, + { ngx_string("http_403"), NGX_HTTP_UPSTREAM_FT_HTTP_403 }, { ngx_string("http_404"), NGX_HTTP_UPSTREAM_FT_HTTP_404 }, { ngx_string("updating"), NGX_HTTP_UPSTREAM_FT_UPDATING }, { ngx_string("off"), NGX_HTTP_UPSTREAM_FT_OFF }, diff --git a/src/http/modules/ngx_http_uwsgi_module.c b/src/http/modules/ngx_http_uwsgi_module.c --- a/src/http/modules/ngx_http_uwsgi_module.c +++ b/src/http/modules/ngx_http_uwsgi_module.c @@ -78,6 +78,7 @@ static ngx_conf_bitmask_t ngx_http_uwsgi { ngx_string("invalid_header"), NGX_HTTP_UPSTREAM_FT_INVALID_HEADER }, { ngx_string("http_500"), NGX_HTTP_UPSTREAM_FT_HTTP_500 }, { ngx_string("http_503"), NGX_HTTP_UPSTREAM_FT_HTTP_503 }, + { ngx_string("http_403"), NGX_HTTP_UPSTREAM_FT_HTTP_403 }, { ngx_string("http_404"), NGX_HTTP_UPSTREAM_FT_HTTP_404 }, { ngx_string("updating"), NGX_HTTP_UPSTREAM_FT_UPDATING }, { ngx_string("off"), NGX_HTTP_UPSTREAM_FT_OFF }, diff --git a/src/http/ngx_http_upstream.c b/src/http/ngx_http_upstream.c --- a/src/http/ngx_http_upstream.c +++ b/src/http/ngx_http_upstream.c @@ -369,6 +369,7 @@ static ngx_http_upstream_next_t ngx_htt { 502, NGX_HTTP_UPSTREAM_FT_HTTP_502 }, { 503, NGX_HTTP_UPSTREAM_FT_HTTP_503 }, { 504, NGX_HTTP_UPSTREAM_FT_HTTP_504 }, + { 403, NGX_HTTP_UPSTREAM_FT_HTTP_403 }, { 404, NGX_HTTP_UPSTREAM_FT_HTTP_404 }, { 0, 0 } }; @@ -3156,8 +3157,11 @@ ngx_http_upstream_next(ngx_http_request_ if (u->peer.sockaddr) { - if (ft_type == NGX_HTTP_UPSTREAM_FT_HTTP_404) { + if (ft_type == NGX_HTTP_UPSTREAM_FT_HTTP_403 + || ft_type == NGX_HTTP_UPSTREAM_FT_HTTP_404) + { state = NGX_PEER_NEXT; + } else { state = NGX_PEER_FAILED; } @@ -3189,6 +3193,10 @@ ngx_http_upstream_next(ngx_http_request_ status = NGX_HTTP_INTERNAL_SERVER_ERROR; break; + case NGX_HTTP_UPSTREAM_FT_HTTP_403: + status = NGX_HTTP_FORBIDDEN; + break; + case NGX_HTTP_UPSTREAM_FT_HTTP_404: status = NGX_HTTP_NOT_FOUND; break; diff --git a/src/http/ngx_http_upstream.h b/src/http/ngx_http_upstream.h --- a/src/http/ngx_http_upstream.h +++ b/src/http/ngx_http_upstream.h @@ -24,10 +24,11 @@ #define NGX_HTTP_UPSTREAM_FT_HTTP_502 0x00000020 #define NGX_HTTP_UPSTREAM_FT_HTTP_503 0x00000040 #define NGX_HTTP_UPSTREAM_FT_HTTP_504 0x00000080 -#define NGX_HTTP_UPSTREAM_FT_HTTP_404 0x00000100 -#define NGX_HTTP_UPSTREAM_FT_UPDATING 0x00000200 -#define NGX_HTTP_UPSTREAM_FT_BUSY_LOCK 0x00000400 -#define NGX_HTTP_UPSTREAM_FT_MAX_WAITING 0x00000800 +#define NGX_HTTP_UPSTREAM_FT_HTTP_403 0x00000100 +#define NGX_HTTP_UPSTREAM_FT_HTTP_404 0x00000200 +#define NGX_HTTP_UPSTREAM_FT_UPDATING 0x00000400 +#define NGX_HTTP_UPSTREAM_FT_BUSY_LOCK 0x00000800 +#define NGX_HTTP_UPSTREAM_FT_MAX_WAITING 0x00001000 #define NGX_HTTP_UPSTREAM_FT_NOLIVE 0x40000000 #define NGX_HTTP_UPSTREAM_FT_OFF 0x80000000 @@ -35,6 +36,7 @@ |NGX_HTTP_UPSTREAM_FT_HTTP_502 \ |NGX_HTTP_UPSTREAM_FT_HTTP_503 \ |NGX_HTTP_UPSTREAM_FT_HTTP_504 \ + |NGX_HTTP_UPSTREAM_FT_HTTP_403 \ |NGX_HTTP_UPSTREAM_FT_HTTP_404) #define NGX_HTTP_UPSTREAM_INVALID_HEADER 40 From vbart at nginx.com Mon May 27 16:51:34 2013 From: vbart at nginx.com (Valentin V. Bartenev) Date: Mon, 27 May 2013 20:51:34 +0400 Subject: [PATCH] SNI: better server name handling. In-Reply-To: References: Message-ID: <201305272051.34349.vbart@nginx.com> On Wednesday 22 May 2013 03:11:36 Piotr Sikora wrote: > # HG changeset patch > # User Piotr Sikora > # Date 1369177319 25200 > # Node ID 4b277448dfd56751c7c88477e78b2ba3cf6ae472 > # Parent 1d68b502088c9d6e6603e9699354e36d03d77f9c > SNI: better server name handling. > > Acknowledge acceptance of SNI server name to the OpenSSL library, > which in turn lets the client know that it was accepted (by sending > "server_name" TLS extension in the "ServerHello" handshake message, > as suggested by RFC4366). > > Previously, this would happen only in case when requested server name > was on the "server_name" list and either: there were multiple virtual > servers defined for the same listening port or there was at least one > regular expression with captures in the "server_name" directive. > Nice catch, but I'm not happy with the solution. With your patch, client will be acknowledged of acceptance even if the server name is not found. I believe such behavior isn't consistent with RFC 4366, and it prevents the client to know that specified virtual host doesn't exist on the server, which effectively makes it useless. Let me propose a better (from my point of view) patch at the end of my message. > As a consequence, this change also: > 1. Preserves requested SNI server name for future use. > > 2. Avoids unnecessary setting of SSL options if the virtual server > didn't change. > While I have nothing against such micro-optimizations, I would like to see them as separate patches. > 3. Avoids unnecessary lookup of virtual server later on if requested > HTTP server name is the same as requested SNI server name. > It also changes behavior a bit when "ssl_verify_client" is used, but I don't think that we care much about that case, especially since the change is needed for $ssl_servername. Probably you should combine it with your patch that adds the $ssl_servername variable (and also rename $ssl_servername to $ssl_server_name). [...] > diff -r 1d68b502088c -r 4b277448dfd5 src/http/ngx_http_request.c > --- a/src/http/ngx_http_request.c Tue May 21 21:47:50 2013 +0400 > +++ b/src/http/ngx_http_request.c Tue May 21 16:01:59 2013 -0700 > @@ -773,6 +773,7 @@ > ngx_http_ssl_srv_conf_t *sscf; > ngx_http_core_loc_conf_t *clcf; > ngx_http_core_srv_conf_t *cscf; > + ngx_int_t rc; > [...] There's a style problem. Please, sort variable declarations in ascending order of the length of their type name (or alphabetically when they are equal). wbr, Valentin V. Bartenev # HG changeset patch # User Valentin Bartenev # Date 1369673389 -14400 # Node ID 3252ce05c7aedd0ad22c9e9db115dbbb5be22807 # Parent 2139768ee404a2e8b1e6164dbae0dc911fe14f1f SNI: always add server names if SSL is enabled for the port. It allows to acknowledge acceptance of SNI server name to the OpenSSL library, which in turn lets the client know that it was accepted (by sending "server_name" TLS extension in the "ServerHello" handshake message, as suggested by RFC4366). Previously, this would happen only in case when requested server name was on the "server_name" list and either: there were multiple virtual servers defined for the same listening port or there was at least one regular expression with captures in the "server_name" directive. diff -r 2139768ee404 -r 3252ce05c7ae src/http/ngx_http.c --- a/src/http/ngx_http.c Fri May 24 22:28:09 2013 +0400 +++ b/src/http/ngx_http.c Mon May 27 20:49:49 2013 +0400 @@ -1444,6 +1444,9 @@ ngx_http_optimize_servers(ngx_conf_t *cf #if (NGX_PCRE) || addr[a].default_server->captures #endif +#if (NGX_HTTP_SSL && defined SSL_CTRL_SET_TLSEXT_HOSTNAME) + || addr[a].opt.ssl +#endif ) { if (ngx_http_server_names(cf, cmcf, &addr[a]) != NGX_OK) { From faskiri.devel at gmail.com Wed May 29 14:47:21 2013 From: faskiri.devel at gmail.com (Fasih) Date: Wed, 29 May 2013 20:17:21 +0530 Subject: SSL error in nginx Message-ID: Hi I see this crash very very infrequently in nginx. Notice the len parameter=3734714755 #12 0x00007f40b8b45975 in sha1_update (c=0x808bdfe3, data=, len=3734714755) at e_aes_cbc_hmac_sha1.c:156 Walking through the openssl source code didnt help. There are two possibilities: 1. Bug in nginx which corrupts some data that openssl crashes on 2. Bug in openssl I will probably post this on the openssl forum also, but wanted to know if someone could shed some light on this. Or give me some pointers on how to debug this. nginx-1.2.6 OpenSSL> version OpenSSL 1.0.1 14 Mar 2012 Ubuntu 12.04 LTS The complete stack trace: #0 sha1_block_data_order_ssse3 () at sha1-x86_64.s:2242 #1 0xca62c1d6ca62c1d6 in ?? () #2 0xca62c1d6ca62c1d6 in ?? () #3 0xca62c1d6ca62c1d6 in ?? () #4 0xca62c1d6ca62c1d6 in ?? () #5 0xca62c1d6ca62c1d6 in ?? () #6 0xca62c1d6ca62c1d6 in ?? () #7 0xca62c1d6ca62c1d6 in ?? () #8 0xca62c1d6ca62c1d6 in ?? () #9 0x000000000fa92011 in ?? () #10 0x000000002e1e7174 in ?? () #11 0xffffffffffffffc0 in ?? () #12 0x00007f40b8b45975 in sha1_update (c=0x808bdfe3, data=, len=3734714755) at e_aes_cbc_hmac_sha1.c:156 #13 0x00007f40b8b45d76 in aesni_cbc_hmac_sha1_cipher (ctx=, out=0xfa9200e "[\200P\303\351\337^\034\336\364:\305\005TeM\356I\232\236\264n\361?\232i\216$,%\026\334\071\375\301!yp\361\214%OFq\355\365\317\354W^\352)\347\376`m\366j'.\316!\027\003\002\002p[lZj\315\a\377Ov\033[/w\247]4\225+\250\356\357\343\311\036e\236?\002\270\001\364\366\362R\363\271[\032\247\220\324\024\017C{b\307N\334\334\022R?b?F\300\225\266g\202\304\336\262\224\265\355\016\374\037(K?/\177\224\257\b?\244\233\314%\260\372\357c\236\001#\271\276\301\027.\377kU\255\016sl&z\340$0\260\253\264w\b\277\201:\265\230M\223]\004?\024\177\261"..., in=, len=) at e_aes_cbc_hmac_sha1.c:260 #14 0x00007f40b8e48bdf in tls1_enc (s=0x11bb9500, send=0) at t1_enc.c:828 #15 0x00007f40b8e406e0 in ssl3_get_record (s=0x11bb9500) at s3_pkt.c:405 #16 ssl3_read_bytes (s=0x11bb9500, type=22, buf=0xe864000 "\024", len=4, peek=0) at s3_pkt.c:997 #17 0x00007f40b8e421f8 in ssl3_get_message (s=0x11bb9500, st1=, stn=8641, mt=20, max=64, ok=0x7fff5b563d2c) at s3_both.c:449 #18 0x00007f40b8e41b47 in ssl3_get_finished (s=0x11bb9500, a=, b=) at s3_both.c:238 #19 0x00007f40b8e367a2 in ssl3_accept (s=0x11bb9500) at s3_srvr.c:701 #20 0x000000000049448c in ngx_ssl_handshake (c=0xd3acdc0) at src/event/ngx_event_openssl.c:607 #21 0x00000000004946a9 in ngx_ssl_handshake_handler (ev=) at src/event/ngx_event_openssl.c:747 #22 0x000000000048aade in ngx_event_process_posted (cycle=, posted=0x103b060) at src/event/ngx_event_posted.c:40 #23 0x000000000048a6a7 in ngx_process_events_and_timers (cycle=0x3e1a050) at src/event/ngx_event.c:290 #24 0x0000000000490d42 in ngx_worker_process_cycle (cycle=0x3e1a050, data=) at src/os/unix/ngx_process_cycle.c:895 #25 0x000000000048f437 in ngx_spawn_process (cycle=0x3e1a050, proc=0x490bad , data=0x3, name=0xb49798 "worker process", respawn=-4) at src/os/unix/ngx_process.c:198 #26 0x0000000000490173 in ngx_start_worker_processes (cycle=0x3e1a050, n=4, type=-4) at src/os/unix/ngx_process_cycle.c:404 #27 0x0000000000491c6d in ngx_master_process_cycle (cycle=0x3e1a050) at src/os/unix/ngx_process_cycle.c:290 #28 0x00000000004748df in main (argc=5, argv=0x7fff5b5643a8) at src/core/nginx.c:436 -------------- next part -------------- An HTML attachment was scrubbed... URL: From mdounin at mdounin.ru Wed May 29 15:18:43 2013 From: mdounin at mdounin.ru (Maxim Dounin) Date: Wed, 29 May 2013 15:18:43 +0000 Subject: [nginx] Win32: added missing reset of wev->ready on WSAEWOULDBLOCK. Message-ID: details: http://hg.nginx.org/nginx/rev/53eb1e67e432 branches: changeset: 5232:53eb1e67e432 user: Maxim Dounin date: Wed May 29 19:18:22 2013 +0400 description: Win32: added missing reset of wev->ready on WSAEWOULDBLOCK. This fixes connection hang with websockets proxy, and likely some other places as well. diffstat: src/os/win32/ngx_wsasend.c | 1 + 1 files changed, 1 insertions(+), 0 deletions(-) diffs (11 lines): diff --git a/src/os/win32/ngx_wsasend.c b/src/os/win32/ngx_wsasend.c --- a/src/os/win32/ngx_wsasend.c +++ b/src/os/win32/ngx_wsasend.c @@ -54,6 +54,7 @@ ngx_wsasend(ngx_connection_t *c, u_char if (err == WSAEWOULDBLOCK) { ngx_log_debug0(NGX_LOG_DEBUG_EVENT, c->log, err, "WSASend() not ready"); + wev->ready = 0; return NGX_AGAIN; } From ru at nginx.com Thu May 30 14:24:54 2013 From: ru at nginx.com (Ruslan Ermilov) Date: Thu, 30 May 2013 14:24:54 +0000 Subject: [nginx] Access: support for UNIX-domain client addresses (ticket... Message-ID: details: http://hg.nginx.org/nginx/rev/00dbfac67e48 branches: changeset: 5233:00dbfac67e48 user: Ruslan Ermilov date: Thu May 30 18:23:05 2013 +0400 description: Access: support for UNIX-domain client addresses (ticket #359). diffstat: src/http/modules/ngx_http_access_module.c | 170 +++++++++++++++++++++-------- 1 files changed, 123 insertions(+), 47 deletions(-) diffs (248 lines): diff -r 53eb1e67e432 -r 00dbfac67e48 src/http/modules/ngx_http_access_module.c --- a/src/http/modules/ngx_http_access_module.c Wed May 29 19:18:22 2013 +0400 +++ b/src/http/modules/ngx_http_access_module.c Thu May 30 18:23:05 2013 +0400 @@ -26,11 +26,22 @@ typedef struct { #endif +#if (NGX_HAVE_UNIX_DOMAIN) + +typedef struct { + ngx_uint_t deny; /* unsigned deny:1; */ +} ngx_http_access_rule_un_t; + +#endif + typedef struct { ngx_array_t *rules; /* array of ngx_http_access_rule_t */ #if (NGX_HAVE_INET6) ngx_array_t *rules6; /* array of ngx_http_access_rule6_t */ #endif +#if (NGX_HAVE_UNIX_DOMAIN) + ngx_array_t *rules_un; /* array of ngx_http_access_rule_un_t */ +#endif } ngx_http_access_loc_conf_t; @@ -41,6 +52,10 @@ static ngx_int_t ngx_http_access_inet(ng static ngx_int_t ngx_http_access_inet6(ngx_http_request_t *r, ngx_http_access_loc_conf_t *alcf, u_char *p); #endif +#if (NGX_HAVE_UNIX_DOMAIN) +static ngx_int_t ngx_http_access_unix(ngx_http_request_t *r, + ngx_http_access_loc_conf_t *alcf); +#endif static ngx_int_t ngx_http_access_found(ngx_http_request_t *r, ngx_uint_t deny); static char *ngx_http_access_rule(ngx_conf_t *cf, ngx_command_t *cmd, void *conf); @@ -144,6 +159,19 @@ ngx_http_access_handler(ngx_http_request return ngx_http_access_inet6(r, alcf, p); } + break; + +#endif + +#if (NGX_HAVE_UNIX_DOMAIN) + + case AF_UNIX: + if (alcf->rules_un) { + return ngx_http_access_unix(r, alcf); + } + + break; + #endif } @@ -221,6 +249,25 @@ ngx_http_access_inet6(ngx_http_request_t #endif +#if (NGX_HAVE_UNIX_DOMAIN) + +static ngx_int_t +ngx_http_access_unix(ngx_http_request_t *r, ngx_http_access_loc_conf_t *alcf) +{ + ngx_uint_t i; + ngx_http_access_rule_un_t *rule_un; + + rule_un = alcf->rules_un->elts; + for (i = 0; i < alcf->rules_un->nelts; i++) { + return ngx_http_access_found(r, rule_un[i].deny); + } + + return NGX_DECLINED; +} + +#endif + + static ngx_int_t ngx_http_access_found(ngx_http_request_t *r, ngx_uint_t deny) { @@ -246,13 +293,16 @@ ngx_http_access_rule(ngx_conf_t *cf, ngx { ngx_http_access_loc_conf_t *alcf = conf; - ngx_int_t rc; - ngx_uint_t all; - ngx_str_t *value; - ngx_cidr_t cidr; - ngx_http_access_rule_t *rule; + ngx_int_t rc; + ngx_uint_t all; + ngx_str_t *value; + ngx_cidr_t cidr; + ngx_http_access_rule_t *rule; #if (NGX_HAVE_INET6) - ngx_http_access_rule6_t *rule6; + ngx_http_access_rule6_t *rule6; +#endif +#if (NGX_HAVE_UNIX_DOMAIN) + ngx_http_access_rule_un_t *rule_un; #endif ngx_memzero(&cidr, sizeof(ngx_cidr_t)); @@ -263,7 +313,19 @@ ngx_http_access_rule(ngx_conf_t *cf, ngx if (!all) { +#if (NGX_HAVE_UNIX_DOMAIN) + + if (value[1].len == 5 && ngx_strcmp(value[1].data, "unix:") == 0) { + cidr.family = AF_UNIX; + rc = NGX_OK; + + } else { + rc = ngx_ptocidr(&value[1], &cidr); + } + +#else rc = ngx_ptocidr(&value[1], &cidr); +#endif if (rc == NGX_ERROR) { ngx_conf_log_error(NGX_LOG_EMERG, cf, 0, @@ -277,37 +339,7 @@ ngx_http_access_rule(ngx_conf_t *cf, ngx } } - switch (cidr.family) { - -#if (NGX_HAVE_INET6) - case AF_INET6: - case 0: /* all */ - - if (alcf->rules6 == NULL) { - alcf->rules6 = ngx_array_create(cf->pool, 4, - sizeof(ngx_http_access_rule6_t)); - if (alcf->rules6 == NULL) { - return NGX_CONF_ERROR; - } - } - - rule6 = ngx_array_push(alcf->rules6); - if (rule6 == NULL) { - return NGX_CONF_ERROR; - } - - rule6->mask = cidr.u.in6.mask; - rule6->addr = cidr.u.in6.addr; - rule6->deny = (value[0].data[0] == 'd') ? 1 : 0; - - if (!all) { - break; - } - - /* "all" passes through */ -#endif - - default: /* AF_INET */ + if (cidr.family == AF_INET || all) { if (alcf->rules == NULL) { alcf->rules = ngx_array_create(cf->pool, 4, @@ -327,6 +359,48 @@ ngx_http_access_rule(ngx_conf_t *cf, ngx rule->deny = (value[0].data[0] == 'd') ? 1 : 0; } +#if (NGX_HAVE_INET6) + if (cidr.family == AF_INET6 || all) { + + if (alcf->rules6 == NULL) { + alcf->rules6 = ngx_array_create(cf->pool, 4, + sizeof(ngx_http_access_rule6_t)); + if (alcf->rules6 == NULL) { + return NGX_CONF_ERROR; + } + } + + rule6 = ngx_array_push(alcf->rules6); + if (rule6 == NULL) { + return NGX_CONF_ERROR; + } + + rule6->mask = cidr.u.in6.mask; + rule6->addr = cidr.u.in6.addr; + rule6->deny = (value[0].data[0] == 'd') ? 1 : 0; + } +#endif + +#if (NGX_HAVE_UNIX_DOMAIN) + if (cidr.family == AF_UNIX || all) { + + if (alcf->rules_un == NULL) { + alcf->rules_un = ngx_array_create(cf->pool, 1, + sizeof(ngx_http_access_rule_un_t)); + if (alcf->rules_un == NULL) { + return NGX_CONF_ERROR; + } + } + + rule_un = ngx_array_push(alcf->rules_un); + if (rule_un == NULL) { + return NGX_CONF_ERROR; + } + + rule_un->deny = (value[0].data[0] == 'd') ? 1 : 0; + } +#endif + return NGX_CONF_OK; } @@ -351,21 +425,23 @@ ngx_http_access_merge_loc_conf(ngx_conf_ ngx_http_access_loc_conf_t *prev = parent; ngx_http_access_loc_conf_t *conf = child; + if (conf->rules == NULL #if (NGX_HAVE_INET6) - - if (conf->rules == NULL && conf->rules6 == NULL) { + && conf->rules6 == NULL +#endif +#if (NGX_HAVE_UNIX_DOMAIN) + && conf->rules_un == NULL +#endif + ) { conf->rules = prev->rules; +#if (NGX_HAVE_INET6) conf->rules6 = prev->rules6; +#endif +#if (NGX_HAVE_UNIX_DOMAIN) + conf->rules_un = prev->rules_un; +#endif } -#else - - if (conf->rules == NULL) { - conf->rules = prev->rules; - } - -#endif - return NGX_CONF_OK; } From radford at galvanix.com Thu May 30 21:34:34 2013 From: radford at galvanix.com (Jim Radford) Date: Thu, 30 May 2013 14:34:34 -0700 Subject: [PATCH] SPDY: Allow returning the full status line Message-ID: <20130530213434.GA9044@home.blackbean.org> # HG changeset patch # User Jim Radford # Date 1369948357 25200 # Node ID 5b75a45ba4deae4b0047357d9bdad7472a83ea3d # Parent 00dbfac67e48a8fe20802287b6fca50950178b8b SPDY: Allow returning the full status line diff -r 00dbfac67e48 -r 5b75a45ba4de src/http/ngx_http_spdy_filter_module.c --- a/src/http/ngx_http_spdy_filter_module.c Thu May 30 18:23:05 2013 +0400 +++ b/src/http/ngx_http_spdy_filter_module.c Thu May 30 14:12:37 2013 -0700 @@ -304,8 +304,14 @@ last = ngx_http_spdy_nv_write_val(last, "HTTP/1.1"); last = ngx_http_spdy_nv_write_name(last, "status"); - last = ngx_spdy_frame_write_uint16(last, 3); - last = ngx_sprintf(last, "%03ui", r->headers_out.status); + if (r->headers_out.status_line.len) { + last = ngx_http_spdy_nv_write_vlen(last, r->headers_out.status_line.len); + last = ngx_cpymem(last, r->headers_out.status_line.data, + r->headers_out.status_line.len); + } else { + last = ngx_spdy_frame_write_uint16(last, 3); + last = ngx_sprintf(last, "%03ui", r->headers_out.status); + } count = 2; From radford at galvanix.com Thu May 30 22:24:50 2013 From: radford at galvanix.com (Jim Radford) Date: Thu, 30 May 2013 15:24:50 -0700 Subject: [PATCH/v2] SPDY: Allow returning the full status line In-Reply-To: <20130530213434.GA9044@home.blackbean.org> References: <20130530213434.GA9044@home.blackbean.org> Message-ID: <20130530222450.GA10588@home.blackbean.org> This is a replacement to my previous patch which actaully includes the buffer length handling. # HG changeset patch # User Jim Radford # Date 1369952377 25200 # Node ID 52d7b6082129c90275579fa3667cce3f537cbd09 # Parent 00dbfac67e48a8fe20802287b6fca50950178b8b SPDY: Allow returning the full status line diff -r 00dbfac67e48 -r 52d7b6082129 src/http/ngx_http_spdy_filter_module.c --- a/src/http/ngx_http_spdy_filter_module.c Thu May 30 18:23:05 2013 +0400 +++ b/src/http/ngx_http_spdy_filter_module.c Thu May 30 15:19:37 2013 -0700 @@ -162,7 +162,9 @@ + ngx_http_spdy_nv_nsize("version") + ngx_http_spdy_nv_vsize("HTTP/1.1") + ngx_http_spdy_nv_nsize("status") - + ngx_http_spdy_nv_vsize("418"); + + (r->headers_out.status_line.len + ? NGX_SPDY_NV_VLEN_SIZE + r->headers_out.status_line.len + : ngx_http_spdy_nv_vsize("418")); clcf = ngx_http_get_module_loc_conf(r, ngx_http_core_module); @@ -304,8 +306,14 @@ last = ngx_http_spdy_nv_write_val(last, "HTTP/1.1"); last = ngx_http_spdy_nv_write_name(last, "status"); - last = ngx_spdy_frame_write_uint16(last, 3); - last = ngx_sprintf(last, "%03ui", r->headers_out.status); + if (r->headers_out.status_line.len) { + last = ngx_http_spdy_nv_write_vlen(last, r->headers_out.status_line.len); + last = ngx_cpymem(last, r->headers_out.status_line.data, + r->headers_out.status_line.len); + } else { + last = ngx_spdy_frame_write_uint16(last, 3); + last = ngx_sprintf(last, "%03ui", r->headers_out.status); + } count = 2; From vbart at nginx.com Fri May 31 00:21:45 2013 From: vbart at nginx.com (Valentin V. Bartenev) Date: Fri, 31 May 2013 04:21:45 +0400 Subject: [PATCH/v2] SPDY: Allow returning the full status line In-Reply-To: <20130530222450.GA10588@home.blackbean.org> References: <20130530213434.GA9044@home.blackbean.org> <20130530222450.GA10588@home.blackbean.org> Message-ID: <201305310421.45269.vbart@nginx.com> On Friday 31 May 2013 02:24:50 Jim Radford wrote: > This is a replacement to my previous patch which actaully includes the > buffer length handling. > > # HG changeset patch > # User Jim Radford > # Date 1369952377 25200 > # Node ID 52d7b6082129c90275579fa3667cce3f537cbd09 > # Parent 00dbfac67e48a8fe20802287b6fca50950178b8b > SPDY: Allow returning the full status line > [...] Could you clarify a bit the purpose of this change? wbr, Valentin V. Bartenev From radford at galvanix.com Fri May 31 04:52:41 2013 From: radford at galvanix.com (Jim Radford) Date: Thu, 30 May 2013 21:52:41 -0700 Subject: [PATCH/v2] SPDY: Allow returning the full status line In-Reply-To: <201305310421.45269.vbart@nginx.com> References: <20130530213434.GA9044@home.blackbean.org> <20130530222450.GA10588@home.blackbean.org> <201305310421.45269.vbart@nginx.com> Message-ID: <20130531045241.GA12348@home.blackbean.org> On Fri, May 31, 2013 at 04:21:45AM +0400, Valentin V. Bartenev wrote: > On Friday 31 May 2013 02:24:50 Jim Radford wrote: > > This is a replacement to my previous patch which actaully includes the > > buffer length handling. > > > > # HG changeset patch > > # User Jim Radford > > # Date 1369952377 25200 > > # Node ID 52d7b6082129c90275579fa3667cce3f537cbd09 > > # Parent 00dbfac67e48a8fe20802287b6fca50950178b8b > > SPDY: Allow returning the full status line > > > [...] > > Could you clarify a bit the purpose of this change? When using nginx to proxy to http, the response status (code and text) are passed though unchanged if the request comes in via HTTP or HTTP over SSL; the text however is currently stripped when the request is made via SPDY. For us this meant that enabling SPDY broke our application which expected to receive our custom status text. While we could easily work around this, we think that it better for nginx to be request transport agnostic as much as possible. Applications that prefer a curt status may still provide one and it will be passed through unmolested. -Jim From ru at nginx.com Fri May 31 09:30:56 2013 From: ru at nginx.com (Ruslan Ermilov) Date: Fri, 31 May 2013 09:30:56 +0000 Subject: [nginx] OCSP stapling: fixed incorrect debug level. Message-ID: details: http://hg.nginx.org/nginx/rev/a855ae7e6377 branches: changeset: 5234:a855ae7e6377 user: Ruslan Ermilov date: Fri May 31 13:30:37 2013 +0400 description: OCSP stapling: fixed incorrect debug level. diffstat: src/event/ngx_event_openssl_stapling.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diffs (12 lines): diff -r 00dbfac67e48 -r a855ae7e6377 src/event/ngx_event_openssl_stapling.c --- a/src/event/ngx_event_openssl_stapling.c Thu May 30 18:23:05 2013 +0400 +++ b/src/event/ngx_event_openssl_stapling.c Fri May 31 13:30:37 2013 +0400 @@ -822,7 +822,7 @@ ngx_ssl_ocsp_resolve_handler(ngx_resolve ngx_uint_t i; struct sockaddr_in *sin; - ngx_log_debug0(NGX_LOG_ALERT, ctx->log, 0, + ngx_log_debug0(NGX_LOG_DEBUG_EVENT, ctx->log, 0, "ssl ocsp resolve handler"); if (resolve->state) { From mdounin at mdounin.ru Fri May 31 12:12:09 2013 From: mdounin at mdounin.ru (Maxim Dounin) Date: Fri, 31 May 2013 12:12:09 +0000 Subject: [nginx] Win32: accept_mutex now always disabled (ticket #362). Message-ID: details: http://hg.nginx.org/nginx/rev/c9fe549b127b branches: changeset: 5235:c9fe549b127b user: Maxim Dounin date: Fri May 31 14:59:26 2013 +0400 description: Win32: accept_mutex now always disabled (ticket #362). Use of accept mutex on win32 may result in a deadlock if there are multiple worker_processes configured and the mutex is grabbed by a process which can't accept connections. diffstat: src/event/ngx_event.c | 11 +++++++++++ 1 files changed, 11 insertions(+), 0 deletions(-) diffs (21 lines): diff --git a/src/event/ngx_event.c b/src/event/ngx_event.c --- a/src/event/ngx_event.c +++ b/src/event/ngx_event.c @@ -607,6 +607,17 @@ ngx_event_process_init(ngx_cycle_t *cycl ngx_use_accept_mutex = 0; } +#if (NGX_WIN32) + + /* + * disable accept mutex on win32 as it may cause deadlock if + * grabbed by a process which can't accept connections + */ + + ngx_use_accept_mutex = 0; + +#endif + #if (NGX_THREADS) ngx_posted_events_mutex = ngx_mutex_init(cycle->log, 0); if (ngx_posted_events_mutex == NULL) { From zorceta at gmail.com Fri May 31 15:02:49 2013 From: zorceta at gmail.com (Zorceta Moshak) Date: Fri, 31 May 2013 23:02:49 +0800 Subject: Function in Filter/Handler Chain Doing Actual Sending? Message-ID: <619978250139700803@unknownmsgid> Hi there, I'm about to put some traffic stats code into the nginx core, as I'm going to let the traffic go into specific user's "iptables" statistics by setuid() rather than creating a filter counting bytes,for other toys are already going thru iptables. The problem is,that I can't figure out where's the point......Evan Miller's articles?Read,but still not clear about how to go thru the chain.In "ngx_http_write_handler" found the struct "connection" has a pointer to function "send_chain" that might be,but can't find it anywhere.I'm not that kind waiting for a LMGIFY link(although Google doesn't help either),but I gracefully appreciate if you can tell it.Thanks a looooooooooooot!! Yours, Zorceta Moshak -------------- next part -------------- An HTML attachment was scrubbed... URL: