From mat999 at gmail.com Fri Aug 1 01:42:09 2014 From: mat999 at gmail.com (SplitIce) Date: Fri, 1 Aug 2014 11:42:09 +1000 Subject: [PATCH] Config: enhancing nginx default config file with added security options In-Reply-To: References: <20140731122543.GI1849@mdounin.ru> Message-ID: expires -1; Does not create an "Expires: -1" header. It should create: Expires: Thu, 01 Jan 1970 00:00:00 GMT A time in the ?Expires? field is computed as a sum of the current time and time specified in the directive. If the modified parameter is used (0.7.0, 0.6.32) then time is computed as a sum of the file?s modification time and time specified in the directive. On Fri, Aug 1, 2014 at 5:06 AM, Kristian Erik Hermansen < kristian.hermansen at gmail.com> wrote: > ??????! > > On Thu, Jul 31, 2014 at 5:25 AM, Maxim Dounin wrote: > > We intentionally avoid various "security recommendations" except > > via providing appropriate defaults. > > > > People tend to have different ideas of what security is, and how > > it should be achieved. Additionally, all such recommendations > > tend to become stale in a very short period of time. > > How do you define "very short period of time"? These are standards > that will remain effectively indefinitely. > > > Goal of the sample configuration file is to show how to configure > > things, not to give any recommendations. > > And I thought that it was useful to be secure by default, rather than > insecure by default. If nginx would like to take the stance that > security should be avoided while preferring ease of use, well OK then, > but state that publicly here and take ownership of that stance so that > I can reference your lack of commitment. > > > Cache-related headers are either invalid (Expires syntax doesn't > > allow "-1" as a valid value, and "Pragma: no-cache" behaviour is > > unspecified when used in a response) or just silly (Cache-Control > > in question disables caching, which is irrelevant for security in > > most cases, but will make things much slower). > > If you don't agree that "Expires '-1'" is valid, then maybe you should > update your own internal documentation and stop recommending it, but I > think your stance is incorrect. It is not only valid, but recommended. > > http://nginx.org/en/docs/http/ngx_http_headers_module.html > > The Pragma / Cache-Control options are actually very relevant, > especially in corporate environments. For instance, most corporations > force outbound connections via an internal web proxy. By caching > content served over HTTPS, an internal attacker can infer content via > the proxy cache, which is a security issue. Sensitive content should > not be cached, I hope we agree. And I request you consult RFC2616 if > you think the behavior is "unspecified" as you surely aren't > considering the same RFCs I am referencing. > > http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html > > > Moreover, there is the "expires" directive to control > > cache-related headers, and it should be used in a proper nginx > > configuration instead, see http://nginx.org/r/expires. > > Great. Again, see my comments above regarding using it. You contradict > yourself... > -- > Regards, > > Kristian Erik Hermansen > https://www.linkedin.com/in/kristianhermansen > https://google.com/+KristianHermansen > > _______________________________________________ > nginx-devel mailing list > nginx-devel at nginx.org > http://mailman.nginx.org/mailman/listinfo/nginx-devel > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mdounin at mdounin.ru Fri Aug 1 02:07:16 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Fri, 1 Aug 2014 06:07:16 +0400 Subject: [PATCH] Config: enhancing nginx default config file with added security options In-Reply-To: References: <20140731122543.GI1849@mdounin.ru> Message-ID: <20140801020716.GM1849@mdounin.ru> Hello! On Thu, Jul 31, 2014 at 12:06:59PM -0700, Kristian Erik Hermansen wrote: > ??????! > > On Thu, Jul 31, 2014 at 5:25 AM, Maxim Dounin wrote: > > We intentionally avoid various "security recommendations" except > > via providing appropriate defaults. > > > > People tend to have different ideas of what security is, and how > > it should be achieved. Additionally, all such recommendations > > tend to become stale in a very short period of time. > > How do you define "very short period of time"? These are standards > that will remain effectively indefinitely. E.g., when talking about SSL ciphers, things tend to become stale in months. > > Goal of the sample configuration file is to show how to configure > > things, not to give any recommendations. > > And I thought that it was useful to be secure by default, rather than > insecure by default. If nginx would like to take the stance that > security should be avoided while preferring ease of use, well OK then, > but state that publicly here and take ownership of that stance so that > I can reference your lack of commitment. As previously said, we are providing appropriate defaults, reasonably secure by default. And, as previously said, people tend to have different ideas of what security is, what is secure and what is not. And this discussion is an illustration of this claim. > > Cache-related headers are either invalid (Expires syntax doesn't > > allow "-1" as a valid value, and "Pragma: no-cache" behaviour is > > unspecified when used in a response) or just silly (Cache-Control > > in question disables caching, which is irrelevant for security in > > most cases, but will make things much slower). > > If you don't agree that "Expires '-1'" is valid, then maybe you should > update your own internal documentation and stop recommending it, but I > think your stance is incorrect. It is not only valid, but recommended. > > http://nginx.org/en/docs/http/ngx_http_headers_module.html Please re-read the documentation and stop yourself from further false claims. There is a difference between the "expires" directive, and the "Expires" response header line added with the "add_header" directive. > The Pragma / Cache-Control options are actually very relevant, > especially in corporate environments. For instance, most corporations > force outbound connections via an internal web proxy. By caching > content served over HTTPS, an internal attacker can infer content via > the proxy cache, which is a security issue. Sensitive content should > not be cached, I hope we agree. When working though a proxy, clients use the CONNECT method to stablish opaque tunnel to a destination https server, and establish SSL connection via this tunnel. The proxy server can't cache content transferred though the tunnel, much like any other attacker who is able to capture network traffic. If a proxy is able to see https requests, it means that there is a successful MitM attack going on, and you already have zero security. Adding any headers won't help - attackers won't respect headers. Of course there may be sensitive content which shouldn't ever hit persistant storage, even on client's own computer. And because of such cases I didn't write "in all cases". The problem though that most of data isn't that sensitive even if transmitted over https. And disabled caching will likely do the opposite to "better security": people will use http instead, because https will be slow for them due to disabled caching. Therefore, if a particular resource is really sensitive and shouldn't be cached/stored, then appropriate headers should be added to this particular resource, not to all https responses. > And I request you consult RFC2616 if > you think the behavior is "unspecified" as you surely aren't > considering the same RFCs I am referencing. > > http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html I'm not really sure why you are trying to refer to the RFC 2616 without actually reading it. Quote from RFC 2616, "14.32 Pragma", http://tools.ietf.org/html/rfc2616#section-14.32: Note: because the meaning of "Pragma: no-cache as a response header field is not actually specified, it does not provide a reliable replacement for "Cache-Control: no-cache" in a response Though RFC 2616 is obsolete now, and correct reference should be to RFC 7234, "5.4 Pragma", it fixes some typos in the text in question, http://tools.ietf.org/html/rfc7234#section-5.4: Note: Because the meaning of "Pragma: no-cache" in responses is not specified, it does not provide a reliable replacement for "Cache-Control: no-cache" in them. -- Maxim Dounin http://nginx.org/ From mdounin at mdounin.ru Fri Aug 1 03:31:49 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Fri, 01 Aug 2014 03:31:49 +0000 Subject: [nginx] Dav: ngx_http_map_uri_to_path() errors were not checked. Message-ID: details: http://hg.nginx.org/nginx/rev/1f70fe0d9576 branches: changeset: 5781:1f70fe0d9576 user: FengGu date: Wed Jul 30 14:45:08 2014 +0800 description: Dav: ngx_http_map_uri_to_path() errors were not checked. Once error occured, it could lead to use uninitialized variables to log, even more segmentation fault. diffstat: src/http/modules/ngx_http_dav_module.c | 20 ++++++++++++++++---- 1 files changed, 16 insertions(+), 4 deletions(-) diffs (58 lines): diff --git a/src/http/modules/ngx_http_dav_module.c b/src/http/modules/ngx_http_dav_module.c --- a/src/http/modules/ngx_http_dav_module.c +++ b/src/http/modules/ngx_http_dav_module.c @@ -212,7 +212,10 @@ ngx_http_dav_put_handler(ngx_http_reques return; } - ngx_http_map_uri_to_path(r, &path, &root, 0); + if (ngx_http_map_uri_to_path(r, &path, &root, 0) == NULL) { + ngx_http_finalize_request(r, NGX_HTTP_INTERNAL_SERVER_ERROR); + return; + } path.len--; @@ -320,7 +323,9 @@ ngx_http_dav_delete_handler(ngx_http_req ok: - ngx_http_map_uri_to_path(r, &path, &root, 0); + if (ngx_http_map_uri_to_path(r, &path, &root, 0) == NULL) { + return NGX_HTTP_INTERNAL_SERVER_ERROR; + } ngx_log_debug1(NGX_LOG_DEBUG_HTTP, r->connection->log, 0, "http delete filename: \"%s\"", path.data); @@ -488,6 +493,9 @@ ngx_http_dav_mkcol_handler(ngx_http_requ } p = ngx_http_map_uri_to_path(r, &path, &root, 0); + if (p == NULL) { + return NGX_HTTP_INTERNAL_SERVER_ERROR; + } *(p - 1) = '\0'; r->uri.len--; @@ -666,7 +674,9 @@ destination_done: overwrite_done: - ngx_http_map_uri_to_path(r, &path, &root, 0); + if (ngx_http_map_uri_to_path(r, &path, &root, 0) == NULL) { + return NGX_HTTP_INTERNAL_SERVER_ERROR; + } ngx_log_debug1(NGX_LOG_DEBUG_HTTP, r->connection->log, 0, "http copy from: \"%s\"", path.data); @@ -674,7 +684,9 @@ overwrite_done: uri = r->uri; r->uri = duri; - ngx_http_map_uri_to_path(r, ©.path, &root, 0); + if (ngx_http_map_uri_to_path(r, ©.path, &root, 0) == NULL) { + return NGX_HTTP_INTERNAL_SERVER_ERROR; + } r->uri = uri; From mdounin at mdounin.ru Fri Aug 1 03:35:36 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Fri, 1 Aug 2014 07:35:36 +0400 Subject: [PATCH]Dav: ngx_http_map_uri_to_path() errors were not checked. In-Reply-To: <336f6c81.f17.147860fbff6.Coremail.flygoast@126.com> References: <336f6c81.f17.147860fbff6.Coremail.flygoast@126.com> Message-ID: <20140801033536.GN1849@mdounin.ru> Hello! On Wed, Jul 30, 2014 at 02:58:04PM +0800, flygoast wrote: > # HG changeset patch > # User FengGu > # Date 1406702708 -28800 > # Wed Jul 30 14:45:08 2014 +0800 > # Node ID 9a9f7c2783b1afe0e5d374450daa5df6acbfd332 > # Parent 4d092aa2f4637ce50284d2accd99a8e91aae2b4c > Dav: ngx_http_map_uri_to_path() errors were not checked. > > > Once error occured, it could lead to use uninitialized variables to log, > even more segmentation fault. Committed, thanks. > diff -r 4d092aa2f463 -r 9a9f7c2783b1 src/http/modules/ngx_http_dav_module.c > --- a/src/http/modules/ngx_http_dav_module.cMon Jul 28 12:27:57 2014 -0700 > +++ b/src/http/modules/ngx_http_dav_module.cWed Jul 30 14:45:08 2014 +0800 > @@ -212,7 +212,10 @@ > return; > } Two minor notes: - Please post patches in plain text. I had to recover text corruption done by your mail client to commit it. - Please add [diff] showfunc=1 to your ~/.hgrc. This makes patches easier for review. -- Maxim Dounin http://nginx.org/ From mdounin at mdounin.ru Fri Aug 1 03:39:31 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Fri, 01 Aug 2014 03:39:31 +0000 Subject: [nginx] Configure: remove outdated and unused patch.zlib.h. Message-ID: details: http://hg.nginx.org/nginx/rev/428303916425 branches: changeset: 5782:428303916425 user: Piotr Sikora date: Thu Jul 31 03:52:28 2014 -0700 description: Configure: remove outdated and unused patch.zlib.h. Signed-off-by: Piotr Sikora diffstat: auto/lib/zlib/patch.zlib.h | 10 ---------- 1 files changed, 0 insertions(+), 10 deletions(-) diffs (15 lines): diff --git a/auto/lib/zlib/patch.zlib.h b/auto/lib/zlib/patch.zlib.h deleted file mode 100644 --- a/auto/lib/zlib/patch.zlib.h +++ /dev/null @@ -1,10 +0,0 @@ ---- zlib.h Thu Jul 9 20:06:56 1998 -+++ zlib-1.1.3/zlib.h Tue Mar 22 13:41:04 2005 -@@ -709,7 +709,6 @@ - (0 in case of error). - */ - --ZEXTERN int ZEXPORTVA gzprintf OF((gzFile file, const char *format, ...)); - /* - Converts, formats, and writes the args to the compressed file under - control of the format string, as in fprintf. gzprintf returns the number of From mdounin at mdounin.ru Fri Aug 1 03:39:46 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Fri, 1 Aug 2014 07:39:46 +0400 Subject: [PATCH] Configure: remove outdated and unused patch.zlib.h In-Reply-To: <5be611309d6f79838791.1406804000@Piotrs-MacBook-Pro.local> References: <5be611309d6f79838791.1406804000@Piotrs-MacBook-Pro.local> Message-ID: <20140801033945.GO1849@mdounin.ru> Hello! On Thu, Jul 31, 2014 at 03:53:20AM -0700, Piotr Sikora wrote: > # HG changeset patch > # User Piotr Sikora > # Date 1406803948 25200 > # Thu Jul 31 03:52:28 2014 -0700 > # Node ID 5be611309d6f7983879104054d4a68feece64142 > # Parent f1e05e533c8b7028121104740f2ab76e49d9212f > Configure: remove outdated and unused patch.zlib.h. > > Signed-off-by: Piotr Sikora Committed, thanks. -- Maxim Dounin http://nginx.org/ From mdounin at mdounin.ru Fri Aug 1 03:47:15 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Fri, 1 Aug 2014 07:47:15 +0400 Subject: [PATCH] Core: add support for expressing size in gigabytes In-Reply-To: References: Message-ID: <20140801034714.GP1849@mdounin.ru> Hello! On Thu, Jul 31, 2014 at 03:41:30AM -0700, Piotr Sikora wrote: > # HG changeset patch > # User Piotr Sikora > # Date 1406803246 25200 > # Thu Jul 31 03:40:46 2014 -0700 > # Node ID c1aeec0f33fe6a42fde0a1851228a130f5ab12a1 > # Parent e0eaf2d92a8cee90abe592d7ac01d3118cb0853a > Core: add support for expressing size in gigabytes. > > Signed-off-by: Piotr Sikora > > diff -r e0eaf2d92a8c -r c1aeec0f33fe src/core/ngx_parse.c > --- a/src/core/ngx_parse.c Wed Jul 30 04:32:16 2014 -0700 > +++ b/src/core/ngx_parse.c Thu Jul 31 03:40:46 2014 -0700 > @@ -33,6 +33,12 @@ ngx_parse_size(ngx_str_t *line) > scale = 1024 * 1024; > break; > > + case 'G': > + case 'g': > + len--; > + scale = 1024 * 1024 * 1024; > + break; > + > default: > scale = 1; > } Memory sizes in gigabytes are almost always wrong, and we intentionally don't understand gigabytes here as an additional safety belt. I don't think this should be added. -- Maxim Dounin http://nginx.org/ From pdn at cryptopro.ru Fri Aug 1 05:20:02 2014 From: pdn at cryptopro.ru (Dmitrii Pichulin) Date: Fri, 1 Aug 2014 09:20:02 +0400 Subject: [PATCH] allow to use engine keyform for server private key In-Reply-To: <20140731134918.GJ1849@mdounin.ru> References: <20140729154024.GP1849@mdounin.ru> <20140731134918.GJ1849@mdounin.ru> Message-ID: <53DB2382.6040005@cryptopro.ru> On 31.07.2014 17:49, Maxim Dounin wrote: >> + if (engine == NULL) { >> + ngx_ssl_error(NGX_LOG_EMERG, ssl->log, 0, >> + "ENGINE_by_id(\"%s\") failed", p); >> + return NGX_ERROR; >> + } >> + >> + p[last - p] = ':'; >> + >> + if (passwords) { >> + pwd = passwords->elts; >> + >> + ngx_cpystrn(pwd_buf, pwd->data, pwd->len + 1); >> + >> + pwd_data.password = pwd_buf; >> + } else { >> + pwd_data.password = NULL; >> + } >> + pwd_data.prompt_info = NULL; >> + >> + last++; >> + >> + private_key = ENGINE_load_private_key(engine, last, 0, >> + (void *) &pwd_data); > I don't see how it's expected to work. You only pass private data > for UI callbacks, but not callbacks itself. > > Anyway, proper implementation of passing key passwords into an engine > seems to be rather big, and as per my reading of the code under > crypto/engine won't work with most of the engines anyway. It > might be better idea to don't try to do this for now. Maxim, our vision is based on 2 implementations of engines as previously noted: 1) gost_capi ? doesn't support external passwords 2) opensc ? with such code from get_pin function (https://github.com/OpenSC/engine_pkcs11/blob/master/src/engine_pkcs11.c): /* either get the pin code from the supplied callback data, or get the pin * via asking our self. In both cases keep a copy of the pin code in the * pin variable (strdup'ed copy). */ static int get_pin(UI_METHOD * ui_method, void *callback_data) { UI *ui; struct { const void *password; const char *prompt_info; } *mycb = callback_data; /* pin in the call back data, copy and use */ if (mycb != NULL && mycb->password) { pin = (char *)calloc(MAX_PIN_LENGTH, sizeof(char)); if (!pin) return 0; strncpy(pin,mycb->password,MAX_PIN_LENGTH); pin_length = MAX_PIN_LENGTH; return 1; } ... As you can see, there's no need for ui_method if a password is present. We suggest to implement something like this: typedef struct { const void *password; const char *prompt_info; ngx_array_t *passwords; ngx_uint_t position; } ngx_openssl_pw_cb_data_ex; In this case, our ui_read implementation can run through all passwords, while supporting the basics. Or it would be better to pass nothing for now? From cubicdaiya at gmail.com Fri Aug 1 11:12:05 2014 From: cubicdaiya at gmail.com (Tatsuhiko Kubo) Date: Fri, 01 Aug 2014 20:12:05 +0900 Subject: [PATCH] Core: improved ngx_pstrdup() error handling Message-ID: <136ec9b0dcc48104d4f2.1406891525@localhost.localdomain> # HG changeset patch # User Tatsuhiko Kubo # Date 1406891054 -32400 # Fri Aug 01 20:04:14 2014 +0900 # Node ID 136ec9b0dcc48104d4f27aa2433cc78246153ef7 # Parent 428303916425d3b7eee77e7918df11bd85da2db1 Core: improved ngx_pstrdup() error handling. diff -r 428303916425 -r 136ec9b0dcc4 src/core/ngx_conf_file.c --- a/src/core/ngx_conf_file.c Thu Jul 31 03:52:28 2014 -0700 +++ b/src/core/ngx_conf_file.c Fri Aug 01 20:04:14 2014 +0900 @@ -781,6 +781,9 @@ file.len = name.len++; file.data = ngx_pstrdup(cf->pool, &name); + if (file.data == NULL) { + return NGX_CONF_ERROR; + } ngx_log_debug1(NGX_LOG_DEBUG_CORE, cf->log, 0, "include %s", file.data); From pluknet at nginx.com Fri Aug 1 14:37:50 2014 From: pluknet at nginx.com (Sergey Kandaurov) Date: Fri, 01 Aug 2014 14:37:50 +0000 Subject: [nginx] Status: indentation and style, no functional changes. Message-ID: details: http://hg.nginx.org/nginx/rev/dc7c139fca21 branches: changeset: 5783:dc7c139fca21 user: Sergey Kandaurov date: Fri Aug 01 18:36:35 2014 +0400 description: Status: indentation and style, no functional changes. diffstat: src/http/modules/ngx_http_stub_status_module.c | 16 +++++++++------- 1 files changed, 9 insertions(+), 7 deletions(-) diffs (60 lines): diff -r 428303916425 -r dc7c139fca21 src/http/modules/ngx_http_stub_status_module.c --- a/src/http/modules/ngx_http_stub_status_module.c Thu Jul 31 03:52:28 2014 -0700 +++ b/src/http/modules/ngx_http_stub_status_module.c Fri Aug 01 18:36:35 2014 +0400 @@ -10,18 +10,19 @@ #include +static ngx_int_t ngx_http_stub_status_handler(ngx_http_request_t *r); static ngx_int_t ngx_http_stub_status_variable(ngx_http_request_t *r, ngx_http_variable_value_t *v, uintptr_t data); static ngx_int_t ngx_http_stub_status_add_variables(ngx_conf_t *cf); +static char *ngx_http_set_stub_status(ngx_conf_t *cf, ngx_command_t *cmd, + void *conf); -static char *ngx_http_set_status(ngx_conf_t *cf, ngx_command_t *cmd, - void *conf); static ngx_command_t ngx_http_status_commands[] = { { ngx_string("stub_status"), NGX_HTTP_SRV_CONF|NGX_HTTP_LOC_CONF|NGX_CONF_FLAG, - ngx_http_set_status, + ngx_http_set_stub_status, 0, 0, NULL }, @@ -30,7 +31,6 @@ static ngx_command_t ngx_http_status_co }; - static ngx_http_module_t ngx_http_stub_status_module_ctx = { ngx_http_stub_status_add_variables, /* preconfiguration */ NULL, /* postconfiguration */ @@ -80,7 +80,8 @@ static ngx_http_variable_t ngx_http_stu }; -static ngx_int_t ngx_http_status_handler(ngx_http_request_t *r) +static ngx_int_t +ngx_http_stub_status_handler(ngx_http_request_t *r) { size_t size; ngx_int_t rc; @@ -223,12 +224,13 @@ ngx_http_stub_status_add_variables(ngx_c } -static char *ngx_http_set_status(ngx_conf_t *cf, ngx_command_t *cmd, void *conf) +static char * +ngx_http_set_stub_status(ngx_conf_t *cf, ngx_command_t *cmd, void *conf) { ngx_http_core_loc_conf_t *clcf; clcf = ngx_http_conf_get_module_loc_conf(cf, ngx_http_core_module); - clcf->handler = ngx_http_status_handler; + clcf->handler = ngx_http_stub_status_handler; return NGX_CONF_OK; } From mdounin at mdounin.ru Fri Aug 1 16:58:39 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Fri, 1 Aug 2014 20:58:39 +0400 Subject: [PATCH] allow to use engine keyform for server private key In-Reply-To: <53DB2382.6040005@cryptopro.ru> References: <20140729154024.GP1849@mdounin.ru> <20140731134918.GJ1849@mdounin.ru> <53DB2382.6040005@cryptopro.ru> Message-ID: <20140801165839.GT1849@mdounin.ru> Hello! On Fri, Aug 01, 2014 at 09:20:02AM +0400, Dmitrii Pichulin wrote: > On 31.07.2014 17:49, Maxim Dounin wrote: > >> + if (engine == NULL) { > >> + ngx_ssl_error(NGX_LOG_EMERG, ssl->log, 0, > >> + "ENGINE_by_id(\"%s\") failed", p); > >> + return NGX_ERROR; > >> + } > >> + > >> + p[last - p] = ':'; > >> + > >> + if (passwords) { > >> + pwd = passwords->elts; > >> + > >> + ngx_cpystrn(pwd_buf, pwd->data, pwd->len + 1); > >> + > >> + pwd_data.password = pwd_buf; > >> + } else { > >> + pwd_data.password = NULL; > >> + } > >> + pwd_data.prompt_info = NULL; > >> + > >> + last++; > >> + > >> + private_key = ENGINE_load_private_key(engine, last, 0, > >> + (void *) &pwd_data); > > I don't see how it's expected to work. You only pass private data > > for UI callbacks, but not callbacks itself. > > > > Anyway, proper implementation of passing key passwords into an engine > > seems to be rather big, and as per my reading of the code under > > crypto/engine won't work with most of the engines anyway. It > > might be better idea to don't try to do this for now. > > Maxim, our vision is based on 2 implementations of engines as previously > noted: > 1) gost_capi ? doesn't support external passwords > 2) opensc ? with such code from get_pin function > (https://github.com/OpenSC/engine_pkcs11/blob/master/src/engine_pkcs11.c): > > /* either get the pin code from the supplied callback data, or get the pin > * via asking our self. In both cases keep a copy of the pin code in the > * pin variable (strdup'ed copy). */ > static int get_pin(UI_METHOD * ui_method, void *callback_data) > { > UI *ui; > struct { > const void *password; > const char *prompt_info; > } *mycb = callback_data; > > /* pin in the call back data, copy and use */ > if (mycb != NULL && mycb->password) { > pin = (char *)calloc(MAX_PIN_LENGTH, sizeof(char)); > if (!pin) > return 0; > strncpy(pin,mycb->password,MAX_PIN_LENGTH); > pin_length = MAX_PIN_LENGTH; > return 1; > } > ... > > As you can see, there's no need for ui_method if a password is present. The code in the get_pin() function provided looks utterly wrong - it tries to access private data of UI_METHOD, assuming some structure. This may work if stucture match (it seems to be copied from PW_CB_DATA used by openssl apps, see apps/apps.[ch]), but will result in segmentation fault if it doesn't. Proper engine behaviour would be to use provided UI_METHOD's callbacks to ask for a password, using the UI_INPUT_FLAG_DEFAULT_PWD flag, see engines/e_chil.c for an example. This seems to be understood at least by openssl itself and curl, see here: http://curl.haxx.se/mail/lib-2013-08/0265.html > We suggest to implement something like this: > > typedef struct { > const void *password; > const char *prompt_info; > ngx_array_t *passwords; > ngx_uint_t position; > } ngx_openssl_pw_cb_data_ex; > > In this case, our ui_read implementation can run through all passwords, > while supporting the basics. There is no need to provide complex structure as there should be a loop to call ENGINE_load_private_keyi() multiple times anyway. > Or it would be better to pass nothing for now? Given the fact how crappy underlying code in engines is, I seriously doubt it worth supporting. Better to don't try for now. -- Maxim Dounin http://nginx.org/ From mdounin at mdounin.ru Fri Aug 1 18:59:19 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Fri, 1 Aug 2014 22:59:19 +0400 Subject: [PATCH] Mail: send starttls flag value to auth script In-Reply-To: <53D9AAB0.5060501@phpgangsta.de> References: <51fd90f96449c23af007.1394099969@HPC> <20140306162718.GL34696@mdounin.ru> <877FD2F6-57CD-4C14-9F2B-4C9E909C3488@phpgangsta.de> <53D9AAB0.5060501@phpgangsta.de> Message-ID: <20140801185919.GU1849@mdounin.ru> Hello! On Thu, Jul 31, 2014 at 04:32:16AM +0200, Michael Kliewe wrote: > Hi Maxim, > > this is very interesting to have in nginx, and it's a very easy patch. Any > chance this gets into nginx? It helps a lot to migrate users to encrypted > mail connections, allowing some users to still use unencrypted connections, > and log in the auth script which user already uses encrypted connections. > > It would be great if you could add it to nginx. I have this flagged in my inbox, and I hope I'll be able to into this. The functionality itself should be usable, but I'm not happy with the header name and value used. -- Maxim Dounin http://nginx.org/ From info at phpgangsta.de Fri Aug 1 20:14:41 2014 From: info at phpgangsta.de (Michael Kliewe) Date: Fri, 01 Aug 2014 22:14:41 +0200 Subject: [PATCH] Mail: send starttls flag value to auth script In-Reply-To: <20140801185919.GU1849@mdounin.ru> References: <51fd90f96449c23af007.1394099969@HPC> <20140306162718.GL34696@mdounin.ru> <877FD2F6-57CD-4C14-9F2B-4C9E909C3488@phpgangsta.de> <53D9AAB0.5060501@phpgangsta.de> <20140801185919.GU1849@mdounin.ru> Message-ID: <53DBF531.2010308@phpgangsta.de> Hi, you can rename it as you wish, as long as the functionality gets into nginx ;-) Michael Am 01.08.2014 um 20:59 schrieb Maxim Dounin: > Hello! > > On Thu, Jul 31, 2014 at 04:32:16AM +0200, Michael Kliewe wrote: > >> Hi Maxim, >> >> this is very interesting to have in nginx, and it's a very easy patch. Any >> chance this gets into nginx? It helps a lot to migrate users to encrypted >> mail connections, allowing some users to still use unencrypted connections, >> and log in the auth script which user already uses encrypted connections. >> >> It would be great if you could add it to nginx. > I have this flagged in my inbox, and I hope I'll be able to into > this. The functionality itself should be usable, but I'm not > happy with the header name and value used. > From mdounin at mdounin.ru Fri Aug 1 21:50:15 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Fri, 01 Aug 2014 21:50:15 +0000 Subject: [nginx] Core: improved ngx_pstrdup() error handling. Message-ID: details: http://hg.nginx.org/nginx/rev/f87afb46ccd2 branches: changeset: 5784:f87afb46ccd2 user: Tatsuhiko Kubo date: Fri Aug 01 20:04:14 2014 +0900 description: Core: improved ngx_pstrdup() error handling. diffstat: src/core/ngx_conf_file.c | 3 +++ 1 files changed, 3 insertions(+), 0 deletions(-) diffs (13 lines): diff --git a/src/core/ngx_conf_file.c b/src/core/ngx_conf_file.c --- a/src/core/ngx_conf_file.c +++ b/src/core/ngx_conf_file.c @@ -781,6 +781,9 @@ ngx_conf_include(ngx_conf_t *cf, ngx_com file.len = name.len++; file.data = ngx_pstrdup(cf->pool, &name); + if (file.data == NULL) { + return NGX_CONF_ERROR; + } ngx_log_debug1(NGX_LOG_DEBUG_CORE, cf->log, 0, "include %s", file.data); From mdounin at mdounin.ru Fri Aug 1 21:52:36 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Sat, 2 Aug 2014 01:52:36 +0400 Subject: [PATCH] Core: improved ngx_pstrdup() error handling In-Reply-To: <136ec9b0dcc48104d4f2.1406891525@localhost.localdomain> References: <136ec9b0dcc48104d4f2.1406891525@localhost.localdomain> Message-ID: <20140801215236.GV1849@mdounin.ru> Hello! On Fri, Aug 01, 2014 at 08:12:05PM +0900, Tatsuhiko Kubo wrote: > # HG changeset patch > # User Tatsuhiko Kubo > # Date 1406891054 -32400 > # Fri Aug 01 20:04:14 2014 +0900 > # Node ID 136ec9b0dcc48104d4f27aa2433cc78246153ef7 > # Parent 428303916425d3b7eee77e7918df11bd85da2db1 > Core: improved ngx_pstrdup() error handling. > > diff -r 428303916425 -r 136ec9b0dcc4 src/core/ngx_conf_file.c > --- a/src/core/ngx_conf_file.c Thu Jul 31 03:52:28 2014 -0700 > +++ b/src/core/ngx_conf_file.c Fri Aug 01 20:04:14 2014 +0900 > @@ -781,6 +781,9 @@ > > file.len = name.len++; > file.data = ngx_pstrdup(cf->pool, &name); > + if (file.data == NULL) { > + return NGX_CONF_ERROR; > + } > > ngx_log_debug1(NGX_LOG_DEBUG_CORE, cf->log, 0, "include %s", file.data); Committed, thanks. -- Maxim Dounin http://nginx.org/ From s.martin49 at gmail.com Fri Aug 1 23:14:05 2014 From: s.martin49 at gmail.com (Samuel Martin) Date: Sat, 02 Aug 2014 01:14:05 +0200 Subject: [PATCH 0 of 4 v3] Build-system: Cross-compilation support improvement Message-ID: Hi all, Here is the 3rd round of this short series improving nginx build-system support for cross-compilation. This series fixes a couple of issues found in the previous submission. It still tries to be as less intrusive as possible, and intends to make easier integration in cross-compilataion frameworks such as Buildroot [1]. These patches include most of the changes needed for porperly supporting the cross-compilation, except the sys_nerr guessing part [2], which cannot merged in nginx because it is too unix-oriented and based on fragile assumptions [1] http://buildroot.net/ [2] https://github.com/tSed/buildroot/blob/nginx-integration/package/nginx/nginx-0005-auto-unix-make-sys_nerr-guessing-cross-friendly.patch Yours, Samuel From s.martin49 at gmail.com Fri Aug 1 23:14:08 2014 From: s.martin49 at gmail.com (Samuel Martin) Date: Sat, 02 Aug 2014 01:14:08 +0200 Subject: [PATCH 3 of 4 v3] auto/*: set ngx_feature_run_force_result for each feature requiring run test In-Reply-To: References: Message-ID: # HG changeset patch # User Samuel Martin # Date 1406932254 -7200 # Sat Aug 02 00:30:54 2014 +0200 # Node ID a0389d32d970bc5740013805bc50d2f1de52aef6 # Parent 0abfd4903c4b440a18ed2de0d8c68e5b163e94f0 auto/*: set ngx_feature_run_force_result for each feature requiring run test Each feature requiring a run test has a matching preset variable (called ngx_force_*) used to set ngx_feature_run_force_result. These ngx_force_* variables are passed through the environment at configure time. Signed-off-by: Samuel Martin diff -r 0abfd4903c4b -r a0389d32d970 auto/cc/conf --- a/auto/cc/conf Sat Aug 02 00:30:54 2014 +0200 +++ b/auto/cc/conf Sat Aug 02 00:30:54 2014 +0200 @@ -159,6 +159,7 @@ ngx_feature="gcc builtin atomic operations" ngx_feature_name=NGX_HAVE_GCC_ATOMIC ngx_feature_run=yes + ngx_feature_run_force_result="$ngx_force_gcc_have_atomic" ngx_feature_incs= ngx_feature_path= ngx_feature_libs= @@ -179,6 +180,7 @@ ngx_feature="C99 variadic macros" ngx_feature_name="NGX_HAVE_C99_VARIADIC_MACROS" ngx_feature_run=yes + ngx_feature_run_force_result="$ngx_force_c99_have_variadic_macros" ngx_feature_incs="#include #define var(dummy, ...) sprintf(__VA_ARGS__)" ngx_feature_path= @@ -193,6 +195,7 @@ ngx_feature="gcc variadic macros" ngx_feature_name="NGX_HAVE_GCC_VARIADIC_MACROS" ngx_feature_run=yes + ngx_feature_run_force_result="$ngx_force_gcc_have_variadic_macros" ngx_feature_incs="#include #define var(dummy, args...) sprintf(args)" ngx_feature_path= diff -r 0abfd4903c4b -r a0389d32d970 auto/cc/name --- a/auto/cc/name Sat Aug 02 00:30:54 2014 +0200 +++ b/auto/cc/name Sat Aug 02 00:30:54 2014 +0200 @@ -8,6 +8,7 @@ ngx_feature="C compiler" ngx_feature_name= ngx_feature_run=yes + ngx_feature_run_force_result="$ngx_force_c_compiler" ngx_feature_incs= ngx_feature_path= ngx_feature_libs= diff -r 0abfd4903c4b -r a0389d32d970 auto/lib/libatomic/conf --- a/auto/lib/libatomic/conf Sat Aug 02 00:30:54 2014 +0200 +++ b/auto/lib/libatomic/conf Sat Aug 02 00:30:54 2014 +0200 @@ -15,6 +15,7 @@ ngx_feature="atomic_ops library" ngx_feature_name=NGX_HAVE_LIBATOMIC ngx_feature_run=yes + ngx_feature_run_force_result="$ngx_force_have_libatomic" ngx_feature_incs="#define AO_REQUIRE_CAS #include " ngx_feature_path= diff -r 0abfd4903c4b -r a0389d32d970 auto/os/darwin --- a/auto/os/darwin Sat Aug 02 00:30:54 2014 +0200 +++ b/auto/os/darwin Sat Aug 02 00:30:54 2014 +0200 @@ -27,6 +27,7 @@ ngx_feature="kqueue's EVFILT_TIMER" ngx_feature_name="NGX_HAVE_TIMER_EVENT" ngx_feature_run=yes +ngx_feature_run_force_result="$ngx_force_have_timer_event" ngx_feature_incs="#include #include " ngx_feature_path= @@ -57,6 +58,7 @@ ngx_feature="Darwin 64-bit kqueue millisecond timeout bug" ngx_feature_name=NGX_DARWIN_KEVENT_BUG ngx_feature_run=bug +ngx_feature_run_force_result="$ngx_force_kevent_bug" ngx_feature_incs="#include #include " ngx_feature_path= @@ -87,6 +89,7 @@ ngx_feature="sendfile()" ngx_feature_name="NGX_HAVE_SENDFILE" ngx_feature_run=yes +ngx_feature_run_force_result="$ngx_force_have_sendfile" ngx_feature_incs="#include #include #include diff -r 0abfd4903c4b -r a0389d32d970 auto/os/linux --- a/auto/os/linux Sat Aug 02 00:30:54 2014 +0200 +++ b/auto/os/linux Sat Aug 02 00:30:54 2014 +0200 @@ -49,6 +49,7 @@ ngx_feature="epoll" ngx_feature_name="NGX_HAVE_EPOLL" ngx_feature_run=yes +ngx_feature_run_force_result="$ngx_force_have_epoll" ngx_feature_incs="#include " ngx_feature_path= ngx_feature_libs= @@ -106,6 +107,7 @@ ngx_feature="sendfile()" ngx_feature_name="NGX_HAVE_SENDFILE" ngx_feature_run=yes +ngx_feature_run_force_result="$ngx_force_have_sendfile" ngx_feature_incs="#include #include " ngx_feature_path= @@ -127,6 +129,7 @@ ngx_feature="sendfile64()" ngx_feature_name="NGX_HAVE_SENDFILE64" ngx_feature_run=yes +ngx_feature_run_force_result="$ngx_force_have_sendfile64" ngx_feature_incs="#include #include " ngx_feature_path= @@ -145,6 +148,7 @@ ngx_feature="prctl(PR_SET_DUMPABLE)" ngx_feature_name="NGX_HAVE_PR_SET_DUMPABLE" ngx_feature_run=yes +ngx_feature_run_force_result="$ngx_force_have_pr_set_dumpable" ngx_feature_incs="#include " ngx_feature_path= ngx_feature_libs= diff -r 0abfd4903c4b -r a0389d32d970 auto/unix --- a/auto/unix Sat Aug 02 00:30:54 2014 +0200 +++ b/auto/unix Sat Aug 02 00:30:54 2014 +0200 @@ -99,6 +99,7 @@ ngx_feature="kqueue's EVFILT_TIMER" ngx_feature_name="NGX_HAVE_TIMER_EVENT" ngx_feature_run=yes + ngx_feature_run_force_result="$ngx_force_have_timer_event" ngx_feature_incs="#include #include " ngx_feature_path= @@ -565,6 +566,7 @@ ngx_feature="sys_nerr" ngx_feature_name="NGX_SYS_NERR" ngx_feature_run=value +ngx_feature_run_force_result="$ngx_force_sys_nerr" ngx_feature_incs='#include #include ' ngx_feature_path= @@ -579,6 +581,7 @@ ngx_feature="_sys_nerr" ngx_feature_name="NGX_SYS_NERR" ngx_feature_run=value + ngx_feature_run_force_result="$ngx_force_sys_nerr" ngx_feature_incs='#include #include ' ngx_feature_path= @@ -594,6 +597,7 @@ ngx_feature='maximum errno' ngx_feature_name=NGX_SYS_NERR ngx_feature_run=value + ngx_feature_run_force_result="$ngx_force_sys_nerr" ngx_feature_incs='#include #include #include ' @@ -652,6 +656,7 @@ ngx_feature="mmap(MAP_ANON|MAP_SHARED)" ngx_feature_name="NGX_HAVE_MAP_ANON" ngx_feature_run=yes +ngx_feature_run_force_result="$ngx_force_have_map_anon" ngx_feature_incs="#include " ngx_feature_path= ngx_feature_libs= @@ -665,6 +670,7 @@ ngx_feature='mmap("/dev/zero", MAP_SHARED)' ngx_feature_name="NGX_HAVE_MAP_DEVZERO" ngx_feature_run=yes +ngx_feature_run_force_result="$ngx_force_have_map_devzero" ngx_feature_incs="#include #include #include " @@ -680,6 +686,7 @@ ngx_feature="System V shared memory" ngx_feature_name="NGX_HAVE_SYSVSHM" ngx_feature_run=yes +ngx_feature_run_force_result="$ngx_force_have_sysvshm" ngx_feature_incs="#include #include " ngx_feature_path= @@ -694,6 +701,7 @@ ngx_feature="POSIX semaphores" ngx_feature_name="NGX_HAVE_POSIX_SEM" ngx_feature_run=yes +ngx_feature_run_force_result="$ngx_force_have_posix_sem" ngx_feature_incs="#include " ngx_feature_path= ngx_feature_libs= From s.martin49 at gmail.com Fri Aug 1 23:14:06 2014 From: s.martin49 at gmail.com (Samuel Martin) Date: Sat, 02 Aug 2014 01:14:06 +0200 Subject: [PATCH 1 of 4 v3] auto/type/sizeof: rework autotest to be cross-compilation friendly In-Reply-To: References: Message-ID: # HG changeset patch # User Samuel Martin # Date 1406932254 -7200 # Sat Aug 02 00:30:54 2014 +0200 # Node ID e16232205bd4dd2f50173643d133b06b654ed2af # Parent f1e05e533c8b7028121104740f2ab76e49d9212f auto/type/sizeof: rework autotest to be cross-compilation friendly Rework the sizeof test to do the checks at compile time instead of at runtime. This way, it does not break when cross-compiling for a different CPU architecture. Signed-off-by: Samuel Martin diff -r f1e05e533c8b -r e16232205bd4 auto/types/sizeof --- a/auto/types/sizeof Thu Jul 31 13:59:37 2014 +0400 +++ b/auto/types/sizeof Sat Aug 02 00:30:54 2014 +0200 @@ -14,7 +14,7 @@ ngx_size= -cat << END > $NGX_AUTOTEST.c +cat << _EOF > $NGX_AUTOTEST.c #include #include @@ -25,28 +25,42 @@ $NGX_INCLUDE_INTTYPES_H $NGX_INCLUDE_AUTO_CONFIG_H -int main() { - printf("%d", (int) sizeof($ngx_type)); +#if !defined( PASTE) +#define PASTE2( x, y) x##y +#define PASTE( x, y) PASTE2( x, y) +#endif /* PASTE */ + +#define SAY_IF_SIZEOF( typename, type, size) \\ + static char PASTE( PASTE( PASTE( sizeof_, typename), _is_), size) \\ + [(sizeof(type) == (size)) ? 1 : -1] + +SAY_IF_SIZEOF(TEST_TYPENAME, TEST_TYPE, TEST_SIZE); + +int main(void) +{ return 0; } -END +_EOF - -ngx_test="$CC $CC_TEST_FLAGS $CC_AUX_FLAGS \ - -o $NGX_AUTOTEST $NGX_AUTOTEST.c $NGX_LD_OPT $ngx_feature_libs" - -eval "$ngx_test >> $NGX_AUTOCONF_ERR 2>&1" - - -if [ -x $NGX_AUTOTEST ]; then - ngx_size=`$NGX_AUTOTEST` - echo " $ngx_size bytes" -fi - +_ngx_typename=`echo "$ngx_type" | sed 's/ /_/g;s/\*/p/'` +ngx_size="-1" +ngx_size=`for i in 1 2 4 8 16 ; do \ + $CC $CC_TEST_FLAGS $CC_AUX_FLAGS \ + -DTEST_TYPENAME="$_ngx_typename" -DTEST_TYPE="$ngx_type" -DTEST_SIZE="$i" \ + $NGX_AUTOTEST.c -o $NGX_AUTOTEST \ + $NGX_LD_OPT $ngx_feature_libs >/dev/null 2>&1 || continue ;\ + echo $i ; break ; done` rm -rf $NGX_AUTOTEST* +if test -z $ngx_size ; then + ngx_size=-1 +fi + +if [ $ngx_size -gt 0 ]; then + echo " $ngx_size bytes" +fi case $ngx_size in 4) From s.martin49 at gmail.com Fri Aug 1 23:14:07 2014 From: s.martin49 at gmail.com (Samuel Martin) Date: Sat, 02 Aug 2014 01:14:07 +0200 Subject: [PATCH 2 of 4 v3] auto/feature: add mechanism allowing to force feature run test result In-Reply-To: References: Message-ID: <0abfd4903c4b440a18ed.1406934847@bobook> # HG changeset patch # User Samuel Martin # Date 1406932254 -7200 # Sat Aug 02 00:30:54 2014 +0200 # Node ID 0abfd4903c4b440a18ed2de0d8c68e5b163e94f0 # Parent e16232205bd4dd2f50173643d133b06b654ed2af auto/feature: add mechanism allowing to force feature run test result Whenever a feature needs to run a test, the ngx_feature_run_force_result variable can be set to the desired test result, and thus skip the test. Therefore, the generated config.h file will honor these presets. This mechanism aims to make easier cross-compilation support. Signed-off-by: Samuel Martin diff -r e16232205bd4 -r 0abfd4903c4b auto/feature --- a/auto/feature Sat Aug 02 00:30:54 2014 +0200 +++ b/auto/feature Sat Aug 02 00:30:54 2014 +0200 @@ -52,50 +52,88 @@ case "$ngx_feature_run" in yes) - # /bin/sh is used to intercept "Killed" or "Abort trap" messages - if /bin/sh -c $NGX_AUTOTEST >> $NGX_AUTOCONF_ERR 2>&1; then - echo " found" + if test -n "$ngx_feature_run_force_result" ; then + echo " not tested (maybe cross-compiling)" + if test -n "$ngx_feature_name" ; then + if test "$ngx_feature_run_force_result" = "yes" ; then + have=$ngx_have_feature . auto/have + fi + fi ngx_found=yes + else - if test -n "$ngx_feature_name"; then - have=$ngx_have_feature . auto/have + # /bin/sh is used to intercept "Killed" or "Abort trap" messages + if /bin/sh -c $NGX_AUTOTEST >> $NGX_AUTOCONF_ERR 2>&1; then + echo " found" + ngx_found=yes + + if test -n "$ngx_feature_name"; then + have=$ngx_have_feature . auto/have + fi + + else + echo " found but is not working" fi - else - echo " found but is not working" fi ;; value) - # /bin/sh is used to intercept "Killed" or "Abort trap" messages - if /bin/sh -c $NGX_AUTOTEST >> $NGX_AUTOCONF_ERR 2>&1; then - echo " found" + if test -n "$ngx_feature_run_force_result" ; then + echo " not tested (maybe cross-compiling)" + cat << END >> $NGX_AUTO_CONFIG_H + +#ifndef $ngx_feature_name +#define $ngx_feature_name $ngx_feature_run_force_result +#endif + +END ngx_found=yes + else - cat << END >> $NGX_AUTO_CONFIG_H + # /bin/sh is used to intercept "Killed" or "Abort trap" messages + if /bin/sh -c $NGX_AUTOTEST >> $NGX_AUTOCONF_ERR 2>&1; then + echo " found" + ngx_found=yes + + cat << END >> $NGX_AUTO_CONFIG_H #ifndef $ngx_feature_name #define $ngx_feature_name `$NGX_AUTOTEST` #endif END - else - echo " found but is not working" + else + echo " found but is not working" + fi + fi ;; bug) - # /bin/sh is used to intercept "Killed" or "Abort trap" messages - if /bin/sh -c $NGX_AUTOTEST >> $NGX_AUTOCONF_ERR 2>&1; then - echo " not found" + if test -n "$ngx_feature_run_force_result" ; then + echo " not tested (maybe cross-compiling)" + if test -n "$ngx_feature_name"; then + if test "$ngx_feature_run_force_result" = "yes" ; then + have=$ngx_have_feature . auto/have + fi + fi + ngx_found=yes + else - else - echo " found" - ngx_found=yes + # /bin/sh is used to intercept "Killed" or "Abort trap" messages + if /bin/sh -c $NGX_AUTOTEST >> $NGX_AUTOCONF_ERR 2>&1; then + echo " not found" - if test -n "$ngx_feature_name"; then - have=$ngx_have_feature . auto/have + else + echo " found" + ngx_found=yes + + if test -n "$ngx_feature_name"; then + have=$ngx_have_feature . auto/have + fi fi + fi ;; From s.martin49 at gmail.com Fri Aug 1 23:14:09 2014 From: s.martin49 at gmail.com (Samuel Martin) Date: Sat, 02 Aug 2014 01:14:09 +0200 Subject: [PATCH 4 of 4 v3] auto/lib/libxslt/conf: allow to override ngx_feature_path and ngx_feature_libs In-Reply-To: References: Message-ID: <1be88123e98c8b0e7860.1406934849@bobook> # HG changeset patch # User Samuel Martin # Date 1406932255 -7200 # Sat Aug 02 00:30:55 2014 +0200 # Node ID 1be88123e98c8b0e78602eeb3a8c3eb3444c15f3 # Parent a0389d32d970bc5740013805bc50d2f1de52aef6 auto/lib/libxslt/conf: allow to override ngx_feature_path and ngx_feature_libs Because libxml2 headers are not in /usr/include by default, hardcoding the include directory to /usr/include/libxml2 does not play well when cross-compiling, or if libxml2 has been installed somewhere else. This patch allows to define/override the libxslt include directory, and the libxslt libs flags. Being able to override the include location is especially useful when cross-compiling to prevent gcc from complaining about unsafe include location for cross-compilation (-Wpoision-system-directories). So far, this warning is only triggered by libxslt. Signed-off-by: Samuel Martin diff -r a0389d32d970 -r 1be88123e98c auto/lib/libxslt/conf --- a/auto/lib/libxslt/conf Sat Aug 02 00:30:54 2014 +0200 +++ b/auto/lib/libxslt/conf Sat Aug 02 00:30:55 2014 +0200 @@ -12,8 +12,8 @@ #include #include #include " - ngx_feature_path="/usr/include/libxml2" - ngx_feature_libs="-lxml2 -lxslt" + ngx_feature_path="${ngx_feature_path_libxslt:=/usr/include/libxml2}" + ngx_feature_libs="${ngx_feature_libs_libxslt:=-lxml2 -lxslt}" ngx_feature_test="xmlParserCtxtPtr ctxt = NULL; xsltStylesheetPtr sheet = NULL; xmlDocPtr doc; From cubicdaiya at gmail.com Sat Aug 2 03:30:47 2014 From: cubicdaiya at gmail.com (Tatsuhiko Kubo) Date: Sat, 02 Aug 2014 12:30:47 +0900 Subject: [PATCH 1 of 2] Core: simplified error handling in ngx_pstrdup() Message-ID: <21d53d2ef2cd35967df5.1406950247@classic-club.local> # HG changeset patch # User Tatsuhiko Kubo # Date 1406949459 -32400 # Sat Aug 02 12:17:39 2014 +0900 # Node ID 21d53d2ef2cd35967df5f788589e5470d650dad3 # Parent f87afb46ccd26fccc7ed55ca8a7ef89c6256c3f2 Core: simplified error handling in ngx_pstrdup(). diff -r f87afb46ccd2 -r 21d53d2ef2cd src/core/ngx_string.c --- a/src/core/ngx_string.c Fri Aug 01 20:04:14 2014 +0900 +++ b/src/core/ngx_string.c Sat Aug 02 12:17:39 2014 +0900 @@ -59,12 +59,10 @@ ngx_pstrdup(ngx_pool_t *pool, ngx_str_t u_char *dst; dst = ngx_pnalloc(pool, src->len); - if (dst == NULL) { - return NULL; + if (dst) { + ngx_memcpy(dst, src->data, src->len); } - ngx_memcpy(dst, src->data, src->len); - return dst; } From cubicdaiya at gmail.com Sat Aug 2 03:30:48 2014 From: cubicdaiya at gmail.com (Tatsuhiko Kubo) Date: Sat, 02 Aug 2014 12:30:48 +0900 Subject: [PATCH 2 of 2] Core: simplified error handling in ngx_resolver_dup() In-Reply-To: <21d53d2ef2cd35967df5.1406950247@classic-club.local> References: <21d53d2ef2cd35967df5.1406950247@classic-club.local> Message-ID: <894e6e8803a1e580ac4a.1406950248@classic-club.local> # HG changeset patch # User Tatsuhiko Kubo # Date 1406949498 -32400 # Sat Aug 02 12:18:18 2014 +0900 # Node ID 894e6e8803a1e580ac4af17cf10813f1bd20075b # Parent 21d53d2ef2cd35967df5f788589e5470d650dad3 Core: simplified error handling in ngx_resolver_dup(). diff -r 21d53d2ef2cd -r 894e6e8803a1 src/core/ngx_resolver.c --- a/src/core/ngx_resolver.c Sat Aug 02 12:17:39 2014 +0900 +++ b/src/core/ngx_resolver.c Sat Aug 02 12:18:18 2014 +0900 @@ -2888,12 +2888,10 @@ ngx_resolver_dup(ngx_resolver_t *r, void dst = ngx_resolver_alloc(r, size); - if (dst == NULL) { - return dst; + if (dst) { + ngx_memcpy(dst, src, size); } - ngx_memcpy(dst, src, size); - return dst; } From piotr at cloudflare.com Sat Aug 2 03:40:08 2014 From: piotr at cloudflare.com (Piotr Sikora) Date: Fri, 01 Aug 2014 20:40:08 -0700 Subject: [PATCH] Core: exit on ngx_pnalloc() failure Message-ID: # HG changeset patch # User Piotr Sikora # Date 1406950762 25200 # Fri Aug 01 20:39:22 2014 -0700 # Node ID f9c41363826ff73216a6db3ca9062dd2794bfd69 # Parent f87afb46ccd26fccc7ed55ca8a7ef89c6256c3f2 Core: exit on ngx_pnalloc() failure. Signed-off-by: Piotr Sikora diff -r f87afb46ccd2 -r f9c41363826f src/os/unix/ngx_process_cycle.c --- a/src/os/unix/ngx_process_cycle.c Fri Aug 01 20:04:14 2014 +0900 +++ b/src/os/unix/ngx_process_cycle.c Fri Aug 01 20:39:22 2014 -0700 @@ -121,6 +121,10 @@ ngx_master_process_cycle(ngx_cycle_t *cy } title = ngx_pnalloc(cycle->pool, size); + if (title == NULL) { + /* fatal */ + exit(2); + } p = ngx_cpymem(title, master_process, sizeof(master_process) - 1); for (i = 0; i < ngx_argc; i++) { From piotr at cloudflare.com Sat Aug 2 03:40:12 2014 From: piotr at cloudflare.com (Piotr Sikora) Date: Fri, 01 Aug 2014 20:40:12 -0700 Subject: [PATCH] Gzip: avoid unnecessary assignment Message-ID: <78c35623c5fa444e8ce7.1406950812@Piotrs-MacBook-Pro.local> # HG changeset patch # User Piotr Sikora # Date 1406950763 25200 # Fri Aug 01 20:39:23 2014 -0700 # Node ID 78c35623c5fa444e8ce7a3af5999477a117f79be # Parent f87afb46ccd26fccc7ed55ca8a7ef89c6256c3f2 Gzip: avoid unnecessary assignment. This change makes it consistent with other custom memory allocators provided for zlib. Signed-off-by: Piotr Sikora diff -r f87afb46ccd2 -r 78c35623c5fa src/http/modules/ngx_http_gzip_filter_module.c --- a/src/http/modules/ngx_http_gzip_filter_module.c Fri Aug 01 20:04:14 2014 +0900 +++ b/src/http/modules/ngx_http_gzip_filter_module.c Fri Aug 01 20:39:23 2014 -0700 @@ -1013,9 +1013,7 @@ ngx_http_gzip_filter_alloc(void *opaque, "gzip filter failed to use preallocated memory: %ud of %ud", items * size, ctx->allocated); - p = ngx_palloc(ctx->request->pool, items * size); - - return p; + return ngx_palloc(ctx->request->pool, items * size); } From cubicdaiya at gmail.com Sat Aug 2 16:50:17 2014 From: cubicdaiya at gmail.com (Tatsuhiko Kubo) Date: Sun, 03 Aug 2014 01:50:17 +0900 Subject: [PATCH] Style: use ngx_calloc_buf() Message-ID: # HG changeset patch # User Tatsuhiko Kubo # Date 1406996678 -32400 # Sun Aug 03 01:24:38 2014 +0900 # Node ID cca43437e50ab2581e857479b287ceb8c20606a4 # Parent f87afb46ccd26fccc7ed55ca8a7ef89c6256c3f2 Style: use ngx_calloc_buf(). diff -r f87afb46ccd2 -r cca43437e50a src/http/modules/ngx_http_flv_module.c --- a/src/http/modules/ngx_http_flv_module.c Fri Aug 01 20:04:14 2014 +0900 +++ b/src/http/modules/ngx_http_flv_module.c Sun Aug 03 01:24:38 2014 +0900 @@ -203,7 +203,7 @@ ngx_http_flv_handler(ngx_http_request_t } if (i == 0) { - b = ngx_pcalloc(r->pool, sizeof(ngx_buf_t)); + b = ngx_calloc_buf(r->pool); if (b == NULL) { return NGX_HTTP_INTERNAL_SERVER_ERROR; } @@ -217,7 +217,7 @@ ngx_http_flv_handler(ngx_http_request_t } - b = ngx_pcalloc(r->pool, sizeof(ngx_buf_t)); + b = ngx_calloc_buf(r->pool); if (b == NULL) { return NGX_HTTP_INTERNAL_SERVER_ERROR; } diff -r f87afb46ccd2 -r cca43437e50a src/http/modules/ngx_http_gzip_filter_module.c --- a/src/http/modules/ngx_http_gzip_filter_module.c Fri Aug 01 20:04:14 2014 +0900 +++ b/src/http/modules/ngx_http_gzip_filter_module.c Sun Aug 03 01:24:38 2014 +0900 @@ -638,7 +638,7 @@ ngx_http_gzip_filter_gzheader(ngx_http_r static u_char gzheader[10] = { 0x1f, 0x8b, Z_DEFLATED, 0, 0, 0, 0, 0, 0, 3 }; - b = ngx_pcalloc(r->pool, sizeof(ngx_buf_t)); + b = ngx_calloc_buf(r->pool); if (b == NULL) { return NGX_ERROR; } diff -r f87afb46ccd2 -r cca43437e50a src/http/modules/ngx_http_gzip_static_module.c --- a/src/http/modules/ngx_http_gzip_static_module.c Fri Aug 01 20:04:14 2014 +0900 +++ b/src/http/modules/ngx_http_gzip_static_module.c Sun Aug 03 01:24:38 2014 +0900 @@ -248,7 +248,7 @@ ngx_http_gzip_static_handler(ngx_http_re /* we need to allocate all before the header would be sent */ - b = ngx_pcalloc(r->pool, sizeof(ngx_buf_t)); + b = ngx_calloc_buf(r->pool); if (b == NULL) { return NGX_HTTP_INTERNAL_SERVER_ERROR; } diff -r f87afb46ccd2 -r cca43437e50a src/http/modules/ngx_http_image_filter_module.c --- a/src/http/modules/ngx_http_image_filter_module.c Fri Aug 01 20:04:14 2014 +0900 +++ b/src/http/modules/ngx_http_image_filter_module.c Sun Aug 03 01:24:38 2014 +0900 @@ -561,7 +561,7 @@ ngx_http_image_json(ngx_http_request_t * size_t len; ngx_buf_t *b; - b = ngx_pcalloc(r->pool, sizeof(ngx_buf_t)); + b = ngx_calloc_buf(r->pool); if (b == NULL) { return NULL; } @@ -613,7 +613,7 @@ ngx_http_image_asis(ngx_http_request_t * { ngx_buf_t *b; - b = ngx_pcalloc(r->pool, sizeof(ngx_buf_t)); + b = ngx_calloc_buf(r->pool); if (b == NULL) { return NULL; } @@ -997,7 +997,7 @@ transparent: return NULL; } - b = ngx_pcalloc(r->pool, sizeof(ngx_buf_t)); + b = ngx_calloc_buf(r->pool); if (b == NULL) { gdFree(out); return NULL; diff -r f87afb46ccd2 -r cca43437e50a src/http/modules/ngx_http_mp4_module.c --- a/src/http/modules/ngx_http_mp4_module.c Fri Aug 01 20:04:14 2014 +0900 +++ b/src/http/modules/ngx_http_mp4_module.c Sun Aug 03 01:24:38 2014 +0900 @@ -646,7 +646,7 @@ ngx_http_mp4_handler(ngx_http_request_t } if (mp4 == NULL) { - b = ngx_pcalloc(r->pool, sizeof(ngx_buf_t)); + b = ngx_calloc_buf(r->pool); if (b == NULL) { return NGX_HTTP_INTERNAL_SERVER_ERROR; } diff -r f87afb46ccd2 -r cca43437e50a src/http/modules/ngx_http_static_module.c --- a/src/http/modules/ngx_http_static_module.c Fri Aug 01 20:04:14 2014 +0900 +++ b/src/http/modules/ngx_http_static_module.c Sun Aug 03 01:24:38 2014 +0900 @@ -236,7 +236,7 @@ ngx_http_static_handler(ngx_http_request /* we need to allocate all before the header would be sent */ - b = ngx_pcalloc(r->pool, sizeof(ngx_buf_t)); + b = ngx_calloc_buf(r->pool); if (b == NULL) { return NGX_HTTP_INTERNAL_SERVER_ERROR; } diff -r f87afb46ccd2 -r cca43437e50a src/http/modules/ngx_http_xslt_filter_module.c --- a/src/http/modules/ngx_http_xslt_filter_module.c Fri Aug 01 20:04:14 2014 +0900 +++ b/src/http/modules/ngx_http_xslt_filter_module.c Sun Aug 03 01:24:38 2014 +0900 @@ -584,7 +584,7 @@ ngx_http_xslt_apply_stylesheet(ngx_http_ return NULL; } - b = ngx_pcalloc(r->pool, sizeof(ngx_buf_t)); + b = ngx_calloc_buf(r->pool); if (b == NULL) { ngx_free(buf); return NULL; diff -r f87afb46ccd2 -r cca43437e50a src/http/ngx_http_core_module.c --- a/src/http/ngx_http_core_module.c Fri Aug 01 20:04:14 2014 +0900 +++ b/src/http/ngx_http_core_module.c Sun Aug 03 01:24:38 2014 +0900 @@ -1945,7 +1945,7 @@ ngx_http_send_response(ngx_http_request_ return ngx_http_send_header(r); } - b = ngx_pcalloc(r->pool, sizeof(ngx_buf_t)); + b = ngx_calloc_buf(r->pool); if (b == NULL) { return NGX_HTTP_INTERNAL_SERVER_ERROR; } diff -r f87afb46ccd2 -r cca43437e50a src/http/ngx_http_file_cache.c --- a/src/http/ngx_http_file_cache.c Fri Aug 01 20:04:14 2014 +0900 +++ b/src/http/ngx_http_file_cache.c Sun Aug 03 01:24:38 2014 +0900 @@ -1120,7 +1120,7 @@ ngx_http_cache_send(ngx_http_request_t * /* we need to allocate all before the header would be sent */ - b = ngx_pcalloc(r->pool, sizeof(ngx_buf_t)); + b = ngx_calloc_buf(r->pool); if (b == NULL) { return NGX_HTTP_INTERNAL_SERVER_ERROR; } From cubicdaiya at gmail.com Sat Aug 2 18:28:53 2014 From: cubicdaiya at gmail.com (Tatsuhiko Kubo) Date: Sun, 03 Aug 2014 03:28:53 +0900 Subject: [PATCH] Style: use ngx_free() instead of free() Message-ID: # HG changeset patch # User Tatsuhiko Kubo # Date 1407004059 -32400 # Sun Aug 03 03:27:39 2014 +0900 # Node ID fbdab7f38fd639b763f78136535cbe6788bcadf0 # Parent f87afb46ccd26fccc7ed55ca8a7ef89c6256c3f2 Style: use ngx_free() instead of free(). diff -r f87afb46ccd2 -r fbdab7f38fd6 src/os/win32/ngx_files.c --- a/src/os/win32/ngx_files.c Fri Aug 01 20:04:14 2014 +0900 +++ b/src/os/win32/ngx_files.c Sun Aug 03 03:27:39 2014 +0900 @@ -864,7 +864,7 @@ ngx_utf8_to_utf16(u_short *utf16, u_char n = ngx_utf8_decode(&p, 4); if (n > 0x10ffff) { - free(utf16); + ngx_free(utf16); ngx_set_errno(NGX_EILSEQ); return NULL; } From mdounin at mdounin.ru Sun Aug 3 20:22:45 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Mon, 4 Aug 2014 00:22:45 +0400 Subject: [PATCH 1 of 2] Core: simplified error handling in ngx_pstrdup() In-Reply-To: <21d53d2ef2cd35967df5.1406950247@classic-club.local> References: <21d53d2ef2cd35967df5.1406950247@classic-club.local> Message-ID: <20140803202245.GY1849@mdounin.ru> Hello! On Sat, Aug 02, 2014 at 12:30:47PM +0900, Tatsuhiko Kubo wrote: > # HG changeset patch > # User Tatsuhiko Kubo > # Date 1406949459 -32400 > # Sat Aug 02 12:17:39 2014 +0900 > # Node ID 21d53d2ef2cd35967df5f788589e5470d650dad3 > # Parent f87afb46ccd26fccc7ed55ca8a7ef89c6256c3f2 > Core: simplified error handling in ngx_pstrdup(). > > diff -r f87afb46ccd2 -r 21d53d2ef2cd src/core/ngx_string.c > --- a/src/core/ngx_string.c Fri Aug 01 20:04:14 2014 +0900 > +++ b/src/core/ngx_string.c Sat Aug 02 12:17:39 2014 +0900 > @@ -59,12 +59,10 @@ ngx_pstrdup(ngx_pool_t *pool, ngx_str_t > u_char *dst; > > dst = ngx_pnalloc(pool, src->len); > - if (dst == NULL) { > - return NULL; > + if (dst) { > + ngx_memcpy(dst, src->data, src->len); > } > > - ngx_memcpy(dst, src->data, src->len); > - > return dst; > } No, thanks. Explicit error handling is better, even if it requires couple of additional lines. The same for the second patch. -- Maxim Dounin http://nginx.org/ From mdounin at mdounin.ru Sun Aug 3 20:40:23 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Mon, 4 Aug 2014 00:40:23 +0400 Subject: [PATCH] Gzip: avoid unnecessary assignment In-Reply-To: <78c35623c5fa444e8ce7.1406950812@Piotrs-MacBook-Pro.local> References: <78c35623c5fa444e8ce7.1406950812@Piotrs-MacBook-Pro.local> Message-ID: <20140803204022.GZ1849@mdounin.ru> Hello! On Fri, Aug 01, 2014 at 08:40:12PM -0700, Piotr Sikora wrote: > # HG changeset patch > # User Piotr Sikora > # Date 1406950763 25200 > # Fri Aug 01 20:39:23 2014 -0700 > # Node ID 78c35623c5fa444e8ce7a3af5999477a117f79be > # Parent f87afb46ccd26fccc7ed55ca8a7ef89c6256c3f2 > Gzip: avoid unnecessary assignment. > > This change makes it consistent with other custom memory allocators > provided for zlib. > > Signed-off-by: Piotr Sikora > > diff -r f87afb46ccd2 -r 78c35623c5fa src/http/modules/ngx_http_gzip_filter_module.c > --- a/src/http/modules/ngx_http_gzip_filter_module.c Fri Aug 01 20:04:14 2014 +0900 > +++ b/src/http/modules/ngx_http_gzip_filter_module.c Fri Aug 01 20:39:23 2014 -0700 > @@ -1013,9 +1013,7 @@ ngx_http_gzip_filter_alloc(void *opaque, > "gzip filter failed to use preallocated memory: %ud of %ud", > items * size, ctx->allocated); > > - p = ngx_palloc(ctx->request->pool, items * size); > - > - return p; > + return ngx_palloc(ctx->request->pool, items * size); > } I prefer to keep current code as is. In particular, because it allows to add debug logging of the address allocated without additional changes. -- Maxim Dounin http://nginx.org/ From mdounin at mdounin.ru Sun Aug 3 22:07:00 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Mon, 4 Aug 2014 02:07:00 +0400 Subject: [PATCH] Style: use ngx_calloc_buf() In-Reply-To: References: Message-ID: <20140803220700.GA1849@mdounin.ru> Hello! On Sun, Aug 03, 2014 at 01:50:17AM +0900, Tatsuhiko Kubo wrote: > # HG changeset patch > # User Tatsuhiko Kubo > # Date 1406996678 -32400 > # Sun Aug 03 01:24:38 2014 +0900 > # Node ID cca43437e50ab2581e857479b287ceb8c20606a4 > # Parent f87afb46ccd26fccc7ed55ca8a7ef89c6256c3f2 > Style: use ngx_calloc_buf(). > > diff -r f87afb46ccd2 -r cca43437e50a src/http/modules/ngx_http_flv_module.c > --- a/src/http/modules/ngx_http_flv_module.c Fri Aug 01 20:04:14 2014 +0900 > +++ b/src/http/modules/ngx_http_flv_module.c Sun Aug 03 01:24:38 2014 +0900 > @@ -203,7 +203,7 @@ ngx_http_flv_handler(ngx_http_request_t > } > > if (i == 0) { > - b = ngx_pcalloc(r->pool, sizeof(ngx_buf_t)); > + b = ngx_calloc_buf(r->pool); > if (b == NULL) { > return NGX_HTTP_INTERNAL_SERVER_ERROR; > } [...] I don't think that this change is needed. There is almost no difference between ngx_pcalloc(sizeof(ngx_buf_t)) and ngx_calloc_buf() even from style point of view, and both are usable if appropriate. -- Maxim Dounin http://nginx.org/ From mdounin at mdounin.ru Sun Aug 3 22:55:23 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Sun, 03 Aug 2014 22:55:23 +0000 Subject: [nginx] Style: use ngx_free() instead of free(). Message-ID: details: http://hg.nginx.org/nginx/rev/fbdab7f38fd6 branches: changeset: 5785:fbdab7f38fd6 user: Tatsuhiko Kubo date: Sun Aug 03 03:27:39 2014 +0900 description: Style: use ngx_free() instead of free(). diffstat: src/os/win32/ngx_files.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diffs (12 lines): diff --git a/src/os/win32/ngx_files.c b/src/os/win32/ngx_files.c --- a/src/os/win32/ngx_files.c +++ b/src/os/win32/ngx_files.c @@ -864,7 +864,7 @@ ngx_utf8_to_utf16(u_short *utf16, u_char n = ngx_utf8_decode(&p, 4); if (n > 0x10ffff) { - free(utf16); + ngx_free(utf16); ngx_set_errno(NGX_EILSEQ); return NULL; } From mdounin at mdounin.ru Sun Aug 3 22:55:38 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Sun, 03 Aug 2014 22:55:38 +0000 Subject: [nginx] Core: exit on ngx_pnalloc() failure. Message-ID: details: http://hg.nginx.org/nginx/rev/f8764e20fcd7 branches: changeset: 5786:f8764e20fcd7 user: Piotr Sikora date: Fri Aug 01 20:39:22 2014 -0700 description: Core: exit on ngx_pnalloc() failure. Signed-off-by: Piotr Sikora diffstat: src/os/unix/ngx_process_cycle.c | 4 ++++ 1 files changed, 4 insertions(+), 0 deletions(-) diffs (14 lines): diff --git a/src/os/unix/ngx_process_cycle.c b/src/os/unix/ngx_process_cycle.c --- a/src/os/unix/ngx_process_cycle.c +++ b/src/os/unix/ngx_process_cycle.c @@ -121,6 +121,10 @@ ngx_master_process_cycle(ngx_cycle_t *cy } title = ngx_pnalloc(cycle->pool, size); + if (title == NULL) { + /* fatal */ + exit(2); + } p = ngx_cpymem(title, master_process, sizeof(master_process) - 1); for (i = 0; i < ngx_argc; i++) { From mdounin at mdounin.ru Sun Aug 3 22:59:08 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Mon, 4 Aug 2014 02:59:08 +0400 Subject: [PATCH] Core: exit on ngx_pnalloc() failure In-Reply-To: References: Message-ID: <20140803225908.GB1849@mdounin.ru> Hello! On Fri, Aug 01, 2014 at 08:40:08PM -0700, Piotr Sikora wrote: > # HG changeset patch > # User Piotr Sikora > # Date 1406950762 25200 > # Fri Aug 01 20:39:22 2014 -0700 > # Node ID f9c41363826ff73216a6db3ca9062dd2794bfd69 > # Parent f87afb46ccd26fccc7ed55ca8a7ef89c6256c3f2 > Core: exit on ngx_pnalloc() failure. > > Signed-off-by: Piotr Sikora > > diff -r f87afb46ccd2 -r f9c41363826f src/os/unix/ngx_process_cycle.c > --- a/src/os/unix/ngx_process_cycle.c Fri Aug 01 20:04:14 2014 +0900 > +++ b/src/os/unix/ngx_process_cycle.c Fri Aug 01 20:39:22 2014 -0700 > @@ -121,6 +121,10 @@ ngx_master_process_cycle(ngx_cycle_t *cy > } > > title = ngx_pnalloc(cycle->pool, size); > + if (title == NULL) { > + /* fatal */ > + exit(2); > + } > > p = ngx_cpymem(title, master_process, sizeof(master_process) - 1); > for (i = 0; i < ngx_argc; i++) { Committed, thanks. -- Maxim Dounin http://nginx.org/ From mdounin at mdounin.ru Sun Aug 3 22:59:23 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Mon, 4 Aug 2014 02:59:23 +0400 Subject: [PATCH] Style: use ngx_free() instead of free() In-Reply-To: References: Message-ID: <20140803225923.GC1849@mdounin.ru> Hello! On Sun, Aug 03, 2014 at 03:28:53AM +0900, Tatsuhiko Kubo wrote: > # HG changeset patch > # User Tatsuhiko Kubo > # Date 1407004059 -32400 > # Sun Aug 03 03:27:39 2014 +0900 > # Node ID fbdab7f38fd639b763f78136535cbe6788bcadf0 > # Parent f87afb46ccd26fccc7ed55ca8a7ef89c6256c3f2 > Style: use ngx_free() instead of free(). > > diff -r f87afb46ccd2 -r fbdab7f38fd6 src/os/win32/ngx_files.c > --- a/src/os/win32/ngx_files.c Fri Aug 01 20:04:14 2014 +0900 > +++ b/src/os/win32/ngx_files.c Sun Aug 03 03:27:39 2014 +0900 > @@ -864,7 +864,7 @@ ngx_utf8_to_utf16(u_short *utf16, u_char > n = ngx_utf8_decode(&p, 4); > > if (n > 0x10ffff) { > - free(utf16); > + ngx_free(utf16); > ngx_set_errno(NGX_EILSEQ); > return NULL; > } Committed, thanks. -- Maxim Dounin http://nginx.org/ From pdn at cryptopro.ru Mon Aug 4 07:06:35 2014 From: pdn at cryptopro.ru (Dmitrii Pichulin) Date: Mon, 04 Aug 2014 11:06:35 +0400 Subject: [PATCH] allow to use engine keyform for server private key In-Reply-To: <20140801165839.GT1849@mdounin.ru> References: <20140801165839.GT1849@mdounin.ru> Message-ID: # HG changeset patch # User Dmitrii Pichulin # Date 1407135800 -14400 # Mon Aug 04 11:03:20 2014 +0400 # Node ID ea21759f209f468d2fd5035782691ff0d4d12cc6 # Parent f8764e20fcd7f87d98fe97f82b2a8d0a77ed9097 allow to use engine keyform for server private key diff -r f8764e20fcd7 -r ea21759f209f src/event/ngx_event_openssl.c --- a/src/event/ngx_event_openssl.c Fri Aug 01 20:39:22 2014 -0700 +++ b/src/event/ngx_event_openssl.c Mon Aug 04 11:03:20 2014 +0400 @@ -275,6 +275,11 @@ u_long n; ngx_str_t *pwd; ngx_uint_t tries; +#ifndef OPENSSL_NO_ENGINE + char *p, *last; + ENGINE *engine; + EVP_PKEY *private_key; +#endif if (ngx_conf_full_name(cf->cycle, cert, 1) != NGX_OK) { return NGX_ERROR; @@ -357,6 +362,62 @@ BIO_free(bio); + if (ngx_strncmp(key->data, "engine:", sizeof("engine:") - 1) == 0) { + +#ifndef OPENSSL_NO_ENGINE + + p = (char *) key->data + sizeof("engine:") - 1; + last = ngx_strchr(p, ':'); + + if (last == NULL) { + ngx_conf_log_error(NGX_LOG_EMERG, cf, 0, "invalid syntax: %V", key); + return NGX_ERROR; + } + + p[last - p] = '\0'; + + engine = ENGINE_by_id(p); + + if (engine == NULL) { + ngx_ssl_error(NGX_LOG_EMERG, ssl->log, 0, + "ENGINE_by_id(\"%s\") failed", p); + return NGX_ERROR; + } + + p[last - p] = ':'; + last++; + + private_key = ENGINE_load_private_key(engine, last, 0, 0); + + if (private_key == NULL) { + ngx_ssl_error(NGX_LOG_EMERG, ssl->log, 0, + "ENGINE_load_private_key(\"%s\") failed", last); + ENGINE_free(engine); + return NGX_ERROR; + } + + ENGINE_free(engine); + + if (SSL_CTX_use_PrivateKey(ssl->ctx, private_key) == 0) { + ngx_ssl_error(NGX_LOG_EMERG, ssl->log, 0, + "SSL_CTX_use_PrivateKey(\"%s\") failed", last); + EVP_PKEY_free(private_key); + return NGX_ERROR; + } + + EVP_PKEY_free(private_key); + + return NGX_OK; + +#else + + ngx_conf_log_error(NGX_LOG_EMERG, cf, 0, + "\"engine\" is not supported: %V", key); + return NGX_ERROR; + +#endif + } + if (ngx_conf_full_name(cf->cycle, key, 1) != NGX_OK) { return NGX_ERROR; } From kpariani at zimbra.com Tue Aug 5 07:26:59 2014 From: kpariani at zimbra.com (Kunal Pariani) Date: Tue, 5 Aug 2014 02:26:59 -0500 (CDT) Subject: [PATCH] The directives such as user, rlimit_core should also be effective on master process Message-ID: <181368080.86148.1407223619639.JavaMail.zimbra@mbs01.zimbra.com> # HG changeset patch # User Kunal Pariani # Date 1407194790 25200 # Mon Aug 04 16:26:30 2014 -0700 # Node ID f25ab24517bb5e45b1b7fa1a1502b907f2cff213 # Parent f8764e20fcd7f87d98fe97f82b2a8d0a77ed9097 The directives such as user, rlimit_core should also be effective on master process diff -r f8764e20fcd7 -r f25ab24517bb src/os/unix/ngx_process_cycle.c --- a/src/os/unix/ngx_process_cycle.c Fri Aug 01 20:39:22 2014 -0700 +++ b/src/os/unix/ngx_process_cycle.c Mon Aug 04 16:26:30 2014 -0700 @@ -93,6 +93,7 @@ ngx_msec_t delay; ngx_listening_t *ls; ngx_core_conf_t *ccf; + struct rlimit rlmt; sigemptyset(&set); sigaddset(&set, SIGCHLD); @@ -139,6 +140,64 @@ ngx_start_worker_processes(cycle, ccf->worker_processes, NGX_PROCESS_RESPAWN); + + if (ccf->rlimit_nofile != NGX_CONF_UNSET) { + rlmt.rlim_cur = (rlim_t) ccf->rlimit_nofile; + rlmt.rlim_max = (rlim_t) ccf->rlimit_nofile; + + if (setrlimit(RLIMIT_NOFILE, &rlmt) == -1) { + ngx_log_error(NGX_LOG_ALERT, cycle->log, ngx_errno, + "setrlimit(RLIMIT_NOFILE, %i) failed", + ccf->rlimit_nofile); + } + } + + if (ccf->rlimit_core != NGX_CONF_UNSET) { + rlmt.rlim_cur = (rlim_t) ccf->rlimit_core; + rlmt.rlim_max = (rlim_t) ccf->rlimit_core; + + if (setrlimit(RLIMIT_CORE, &rlmt) == -1) { + ngx_log_error(NGX_LOG_ALERT, cycle->log, ngx_errno, + "setrlimit(RLIMIT_CORE, %O) failed", + ccf->rlimit_core); + } + } + +#ifdef RLIMIT_SIGPENDING + if (ccf->rlimit_sigpending != NGX_CONF_UNSET) { + rlmt.rlim_cur = (rlim_t) ccf->rlimit_sigpending; + rlmt.rlim_max = (rlim_t) ccf->rlimit_sigpending; + + if (setrlimit(RLIMIT_SIGPENDING, &rlmt) == -1) { + ngx_log_error(NGX_LOG_ALERT, cycle->log, ngx_errno, + "setrlimit(RLIMIT_SIGPENDING, %i) failed", + ccf->rlimit_sigpending); + } + } +#endif + + if (geteuid() == 0) { + if (setgid(ccf->group) == -1) { + ngx_log_error(NGX_LOG_EMERG, cycle->log, ngx_errno, + "setgid(%d) failed", ccf->group); + /* fatal */ + exit(2); + } + + if (initgroups(ccf->username, ccf->group) == -1) { + ngx_log_error(NGX_LOG_EMERG, cycle->log, ngx_errno, + "initgroups(%s, %d) failed", + ccf->username, ccf->group); + } + + if (setuid(ccf->user) == -1) { + ngx_log_error(NGX_LOG_EMERG, cycle->log, ngx_errno, + "setuid(%d) failed", ccf->user); + /* fatal */ + exit(2); + } + } + ngx_start_cache_manager_processes(cycle, 0); ngx_new_binary = 0; Thanks -Kunal -------------- next part -------------- An HTML attachment was scrubbed... URL: From mdounin at mdounin.ru Tue Aug 5 07:58:19 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Tue, 5 Aug 2014 11:58:19 +0400 Subject: [PATCH] The directives such as user, rlimit_core should also be effective on master process In-Reply-To: <181368080.86148.1407223619639.JavaMail.zimbra@mbs01.zimbra.com> References: <181368080.86148.1407223619639.JavaMail.zimbra@mbs01.zimbra.com> Message-ID: <20140805075819.GY1849@mdounin.ru> Hello! On Tue, Aug 05, 2014 at 02:26:59AM -0500, Kunal Pariani wrote: > # HG changeset patch > # User Kunal Pariani > # Date 1407194790 25200 > # Mon Aug 04 16:26:30 2014 -0700 > # Node ID f25ab24517bb5e45b1b7fa1a1502b907f2cff213 > # Parent f8764e20fcd7f87d98fe97f82b2a8d0a77ed9097 > The directives such as user, rlimit_core should also be effective on master process No, thanks. The master process should remain root to be able to read priveleged configuration files during configuration reload, open listening sockets on priveleged ports and so on. -- Maxim Dounin http://nginx.org/ From piotr at cloudflare.com Tue Aug 5 08:39:23 2014 From: piotr at cloudflare.com (Piotr Sikora) Date: Tue, 5 Aug 2014 01:39:23 -0700 Subject: [PATCH] Perl: NULL-terminate argument list In-Reply-To: References: <290f3fcb9cf552c235b9.1403176643@Piotrs-MacBook-Pro.local> <20140619132641.GA1849@mdounin.ru> <20140620194102.GQ1849@mdounin.ru> Message-ID: Hey Maxim, any progress on this? The original patch fixes consistent and 100% reproducible crashes for me, so it cannot be that bad ;) Best regards, Piotr Sikora From piotr at cloudflare.com Tue Aug 5 08:46:16 2014 From: piotr at cloudflare.com (Piotr Sikora) Date: Tue, 5 Aug 2014 01:46:16 -0700 Subject: [PATCH] Core: add support for expressing size in gigabytes In-Reply-To: <20140801034714.GP1849@mdounin.ru> References: <20140801034714.GP1849@mdounin.ru> Message-ID: Hey Maxim, > Memory sizes in gigabytes are almost always wrong, and we > intentionally don't understand gigabytes here as an additional > safety belt. > > I don't think this should be added. I disagree. For me (and, most likely, others) this just means that I need to replace "1g" or "2g" with "1024m" or "2048m". I doubt that no support for gigabytes makes people rethink their values... IMHO, this is just annoyance and not a safety belt. Best regards, Piotr Sikora From dev.panky2014 at gmail.com Tue Aug 5 08:46:31 2014 From: dev.panky2014 at gmail.com (Pankaj Kaushik) Date: Tue, 5 Aug 2014 14:16:31 +0530 Subject: Regarding using of nginx http load balancer Message-ID: Hi Team, I would like to use nginx http load balancer. I want to parse http uri and based on some key present in uri, i want the backened server to be selected dynamically. I want to use consistent hash mechanism which will hash the key present in uri and gives me the server ip corresponding to that. Could you please help me how can i write my own module for above? Thanks, Panky -------------- next part -------------- An HTML attachment was scrubbed... URL: From piotr at cloudflare.com Tue Aug 5 08:51:04 2014 From: piotr at cloudflare.com (Piotr Sikora) Date: Tue, 5 Aug 2014 01:51:04 -0700 Subject: Regarding using of nginx http load balancer In-Reply-To: References: Message-ID: Hey, > I want to parse http uri and based on some key present in uri, i want the > backened server to be selected dynamically. > > I want to use consistent hash mechanism which will hash the key present in > uri and gives me the server ip corresponding to that. What's wrong with "hash" [1]? [1] http://nginx.org/en/docs/http/ngx_http_upstream_module.html#hash Best regards, Piotr Sikora From piotr at cloudflare.com Tue Aug 5 08:52:59 2014 From: piotr at cloudflare.com (Piotr Sikora) Date: Tue, 05 Aug 2014 01:52:59 -0700 Subject: [PATCH] Style: use ngx_str_set() Message-ID: <42d455aafd7fc9c3ab20.1407228779@Piotrs-MacBook-Pro.local> # HG changeset patch # User Piotr Sikora # Date 1407228679 25200 # Tue Aug 05 01:51:19 2014 -0700 # Node ID 42d455aafd7fc9c3ab204c6bf7f60a8aafb5179c # Parent f8764e20fcd7f87d98fe97f82b2a8d0a77ed9097 Style: use ngx_str_set(). Signed-off-by: Piotr Sikora diff -r f8764e20fcd7 -r 42d455aafd7f src/core/nginx.c --- a/src/core/nginx.c Fri Aug 01 20:39:22 2014 -0700 +++ b/src/core/nginx.c Tue Aug 05 01:51:19 2014 -0700 @@ -493,8 +493,7 @@ ngx_set_environment(ngx_cycle_t *cycle, return NULL; } - var->len = 2; - var->data = (u_char *) "TZ"; + ngx_str_set(var, "TZ"); var = ccf->env.elts; diff -r f8764e20fcd7 -r 42d455aafd7f src/event/ngx_event.c --- a/src/event/ngx_event.c Fri Aug 01 20:39:22 2014 -0700 +++ b/src/event/ngx_event.c Tue Aug 05 01:51:19 2014 -0700 @@ -518,8 +518,7 @@ ngx_event_module_init(ngx_cycle_t *cycle #endif shm.size = size; - shm.name.len = sizeof("nginx_shared_zone"); - shm.name.data = (u_char *) "nginx_shared_zone"; + ngx_str_set(&shm.name, "nginx_shared_zone"); shm.log = cycle->log; if (ngx_shm_alloc(&shm) != NGX_OK) { diff -r f8764e20fcd7 -r 42d455aafd7f src/http/modules/ngx_http_auth_request_module.c --- a/src/http/modules/ngx_http_auth_request_module.c Fri Aug 01 20:39:22 2014 -0700 +++ b/src/http/modules/ngx_http_auth_request_module.c Tue Aug 05 01:51:19 2014 -0700 @@ -367,9 +367,7 @@ ngx_http_auth_request(ngx_conf_t *cf, ng value = cf->args->elts; if (ngx_strcmp(value[1].data, "off") == 0) { - arcf->uri.len = 0; - arcf->uri.data = (u_char *) ""; - + ngx_str_set(&arcf->uri, ""); return NGX_CONF_OK; } diff -r f8764e20fcd7 -r 42d455aafd7f src/http/modules/ngx_http_headers_filter_module.c --- a/src/http/modules/ngx_http_headers_filter_module.c Fri Aug 01 20:39:22 2014 -0700 +++ b/src/http/modules/ngx_http_headers_filter_module.c Tue Aug 05 01:51:19 2014 -0700 @@ -194,7 +194,6 @@ ngx_http_headers_filter(ngx_http_request static ngx_int_t ngx_http_set_expires(ngx_http_request_t *r, ngx_http_headers_conf_t *conf) { - size_t len; time_t now, expires_time, max_age; ngx_uint_t i; ngx_table_elt_t *expires, *cc, **ccp; @@ -214,9 +213,6 @@ ngx_http_set_expires(ngx_http_request_t ngx_str_set(&expires->key, "Expires"); } - len = sizeof("Mon, 28 Sep 1970 06:00:00 GMT"); - expires->value.len = len - 1; - ccp = r->headers_out.cache_control.elts; if (ccp == NULL) { @@ -251,19 +247,21 @@ ngx_http_set_expires(ngx_http_request_t } if (conf->expires == NGX_HTTP_EXPIRES_EPOCH) { - expires->value.data = (u_char *) "Thu, 01 Jan 1970 00:00:01 GMT"; + ngx_str_set(&expires->value, "Thu, 01 Jan 1970 00:00:01 GMT"); ngx_str_set(&cc->value, "no-cache"); return NGX_OK; } if (conf->expires == NGX_HTTP_EXPIRES_MAX) { - expires->value.data = (u_char *) "Thu, 31 Dec 2037 23:55:55 GMT"; + ngx_str_set(&expires->value, "Thu, 31 Dec 2037 23:55:55 GMT"); /* 10 years */ ngx_str_set(&cc->value, "max-age=315360000"); return NGX_OK; } - expires->value.data = ngx_pnalloc(r->pool, len); + expires->value.len = sizeof("Mon, 28 Sep 1970 06:00:00 GMT") - 1; + + expires->value.data = ngx_pnalloc(r->pool, expires->value.len + 1); if (expires->value.data == NULL) { return NGX_ERROR; } diff -r f8764e20fcd7 -r 42d455aafd7f src/http/ngx_http_core_module.c --- a/src/http/ngx_http_core_module.c Fri Aug 01 20:39:22 2014 -0700 +++ b/src/http/ngx_http_core_module.c Tue Aug 05 01:51:19 2014 -0700 @@ -2093,7 +2093,7 @@ ngx_http_auth_basic_user(ngx_http_reques } if (r->headers_in.authorization == NULL) { - r->headers_in.user.data = (u_char *) ""; + ngx_str_set(&r->headers_in.user, ""); return NGX_DECLINED; } @@ -2104,7 +2104,7 @@ ngx_http_auth_basic_user(ngx_http_reques sizeof("Basic ") - 1) != 0) { - r->headers_in.user.data = (u_char *) ""; + ngx_str_set(&r->headers_in.user, ""); return NGX_DECLINED; } @@ -2117,7 +2117,7 @@ ngx_http_auth_basic_user(ngx_http_reques } if (encoded.len == 0) { - r->headers_in.user.data = (u_char *) ""; + ngx_str_set(&r->headers_in.user, ""); return NGX_DECLINED; } @@ -2128,7 +2128,7 @@ ngx_http_auth_basic_user(ngx_http_reques } if (ngx_decode_base64(&auth, &encoded) != NGX_OK) { - r->headers_in.user.data = (u_char *) ""; + ngx_str_set(&r->headers_in.user, ""); return NGX_DECLINED; } @@ -2141,7 +2141,7 @@ ngx_http_auth_basic_user(ngx_http_reques } if (len == 0 || len == auth.len) { - r->headers_in.user.data = (u_char *) ""; + ngx_str_set(&r->headers_in.user, ""); return NGX_DECLINED; } diff -r f8764e20fcd7 -r 42d455aafd7f src/http/ngx_http_script.c --- a/src/http/ngx_http_script.c Fri Aug 01 20:39:22 2014 -0700 +++ b/src/http/ngx_http_script.c Tue Aug 05 01:51:19 2014 -0700 @@ -601,8 +601,7 @@ ngx_http_script_done(ngx_http_script_com if (sc->zero) { - zero.len = 1; - zero.data = (u_char *) "\0"; + ngx_str_set(&zero, "\0"); if (ngx_http_script_add_copy_code(sc, &zero, 0) != NGX_OK) { return NGX_ERROR; From dev.panky2014 at gmail.com Tue Aug 5 08:58:23 2014 From: dev.panky2014 at gmail.com (Pankaj Kaushik) Date: Tue, 5 Aug 2014 14:28:23 +0530 Subject: Regarding using of nginx http load balancer In-Reply-To: References: Message-ID: Hey Piotr, On Tue, Aug 5, 2014 at 2:21 PM, Piotr Sikora wrote: > > What's wrong with "hash" [1]? > > [1] http://nginx.org/en/docs/http/ngx_http_upstream_module.html#hash > > As mentioned in the syntax at above link:- Syntax:*hash* *key* [consistent]; Default:?Context:upstream In my scenario i have to fetch key out of uri and then need to hash that to identify the backend server. Basically i need to fetch the key out of uri during run time. Can you please help in knowing how can i do that? Thanks, Panky -------------- next part -------------- An HTML attachment was scrubbed... URL: From mdounin at mdounin.ru Tue Aug 5 09:11:42 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Tue, 5 Aug 2014 13:11:42 +0400 Subject: [PATCH] Perl: NULL-terminate argument list In-Reply-To: References: <290f3fcb9cf552c235b9.1403176643@Piotrs-MacBook-Pro.local> <20140619132641.GA1849@mdounin.ru> <20140620194102.GQ1849@mdounin.ru> Message-ID: <20140805091142.GZ1849@mdounin.ru> Hello! On Tue, Aug 05, 2014 at 01:39:23AM -0700, Piotr Sikora wrote: > Hey Maxim, > any progress on this? The original patch fixes consistent and 100% > reproducible crashes for me, so it cannot be that bad ;) Somebody have to investigate these crashes to find out where the problem is. As previously suggested, nginx code matches perlembed samples, and it looks like as either perl itself or perlembed bug. I also suspect that the patch fixes things due to some side effect, not because it does something needed. If you have no plans to investigate it yourself, you may want to share a way to reproduce the problem. -- Maxim Dounin http://nginx.org/ From arut at nginx.com Tue Aug 5 09:37:33 2014 From: arut at nginx.com (Roman Arutyunyan) Date: Tue, 5 Aug 2014 13:37:33 +0400 Subject: Regarding using of nginx http load balancer In-Reply-To: References: Message-ID: <591B3BBA-A8A0-4D46-ABFD-EF5DF66C7B84@nginx.com> On 05 Aug 2014, at 12:58, Pankaj Kaushik wrote: > Hey Piotr, > > > > On Tue, Aug 5, 2014 at 2:21 PM, Piotr Sikora wrote: > > What's wrong with "hash" [1]? > > [1] http://nginx.org/en/docs/http/ngx_http_upstream_module.html#hash > > > > As mentioned in the syntax at above link:- > > Syntax: hash key [consistent]; > Default: ? > Context: upstream > > > In my scenario i have to fetch key out of uri and then need to hash that to identify the backend server. > > Basically i need to fetch the key out of uri during run time. > > Can you please help in knowing how can i do that? > You can use $arg_XXX variables to access arguments passed in uri. Or you can use ?map" directive to fetch any part of $uri directly http://nginx.org/ru/docs/http/ngx_http_map_module.html > Thanks, > Panky > > _______________________________________________ > nginx-devel mailing list > nginx-devel at nginx.org > http://mailman.nginx.org/mailman/listinfo/nginx-devel From piotr at cloudflare.com Tue Aug 5 10:33:18 2014 From: piotr at cloudflare.com (Piotr Sikora) Date: Tue, 5 Aug 2014 03:33:18 -0700 Subject: [PATCH] Perl: NULL-terminate argument list In-Reply-To: <20140805091142.GZ1849@mdounin.ru> References: <290f3fcb9cf552c235b9.1403176643@Piotrs-MacBook-Pro.local> <20140619132641.GA1849@mdounin.ru> <20140620194102.GQ1849@mdounin.ru> <20140805091142.GZ1849@mdounin.ru> Message-ID: Hey Maxim, > Somebody have to investigate these crashes to find out where the > problem is. As previously suggested, nginx code matches perlembed > samples, and it looks like as either perl itself or perlembed bug. > I also suspect that the patch fixes things due to some > side effect, not because it does something needed. > > If you have no plans to investigate it yourself, you may want to > share a way to reproduce the problem. I did enough of investigation to be satisfied with the patch I provided and I have no plans to dig into perl's code, especially because I have no interest in the perl module itself. Like I said, this condition can be triggered when using malloc with guard pages, when argv[argc] == pool->d.last == pool->d.end. For me, it's 100% reproducible when running nginx-tests under OSX's guard malloc against fully built nginx built: $ ./objs/nginx -V nginx version: nginx/1.7.4 (f8764e20fcd7) built by clang 3.4.2 (tags/RELEASE_34/dot2-final) TLS SNI support enabled configure arguments: --build=f8764e20fcd7 --with-debug --with-http_ssl_module --with-http_spdy_module --with-debug --with-ipv6 --with-http_ssl_module --with-http_spdy_module --with-http_realip_module --with-http_addition_module --with-http_xslt_module --with-http_image_filter_module --with-http_geoip_module --with-http_sub_module --with-http_dav_module --with-http_flv_module --with-http_mp4_module --with-http_gunzip_module --with-http_gzip_static_module --with-http_auth_request_module --with-http_random_index_module --with-http_secure_link_module --with-http_stub_status_module --with-http_perl_module --with-mail --with-mail_ssl_module --with-cc-opt='-O0 -ggdb' $ env MALLOC_FILL_SPACE=1 MALLOC_STRICT_SIZE=1 DYLD_INSERT_LIBRARIES=/usr/lib/libgmalloc.dylib prove auth_basic.t http_server_name.t auth_basic.t ........ Can't start nginx at lib/Test/Nginx.pm line 227. # Looks like your test exited with 2 before it could output anything. auth_basic.t ........ Dubious, test returned 2 (wstat 512, 0x200) Failed 19/19 subtests http_server_name.t .. Can't start nginx at lib/Test/Nginx.pm line 227. # Looks like your test exited with 2 before it could output anything. http_server_name.t .. Dubious, test returned 2 (wstat 512, 0x200) Failed 9/9 subtests Test Summary Report ------------------- auth_basic.t (Wstat: 512 Tests: 0 Failed: 0) Non-zero exit status: 2 Parse errors: Bad plan. You planned 19 tests but ran 0. http_server_name.t (Wstat: 512 Tests: 0 Failed: 0) Non-zero exit status: 2 Parse errors: Bad plan. You planned 9 tests but ran 0. Files=2, Tests=0, 4 wallclock secs ( 0.11 usr 0.44 sys + 0.76 cusr 2.39 csys = 3.70 CPU) Result: FAIL $ lldb objs/nginx Current executable set to 'objs/nginx' (x86_64). (lldb) env MALLOC_FILL_SPACE=1 (lldb) env MALLOC_STRICT_SIZE=1 (lldb) env DYLD_INSERT_LIBRARIES=/usr/lib/libgmalloc.dylib (lldb) r Process 48754 launched: 'objs/nginx' (x86_64) Process 48754 stopped * thread #1: tid = 0xa7e9a, 0x00000001006c1aaf libperl.dylib`perl_parse + 4148, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, address=0x103279000) frame #0: 0x00000001006c1aaf libperl.dylib`perl_parse + 4148 libperl.dylib`perl_parse + 4148: -> 0x1006c1aaf: movq (%rax), %rcx 0x1006c1ab2: movq %rcx, -0x960(%rbp) 0x1006c1ab9: cmpq $0x0, 0x4f8(%r12) 0x1006c1ac2: movl -0x924(%rbp), %r14d (lldb) bt * thread #1: tid = 0xa7e9a, 0x00000001006c1aaf libperl.dylib`perl_parse + 4148, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, address=0x103279000) * frame #0: 0x00000001006c1aaf libperl.dylib`perl_parse + 4148 frame #1: 0x0000000100101b77 nginx`ngx_http_perl_create_interpreter(cf=0x00007fff5fbff4b0, pmcf=0x00000001030821c8) + 775 at ngx_http_perl_module.c:599 frame #2: 0x000000010010143a nginx`ngx_http_perl_init_interpreter(cf=0x00007fff5fbff4b0, pmcf=0x00000001030821c8) + 298 at ngx_http_perl_module.c:520 frame #3: 0x0000000100102846 nginx`ngx_http_perl_init_main_conf(cf=0x00007fff5fbff4b0, conf=0x00000001030821c8) + 54 at ngx_http_perl_module.c:813 frame #4: 0x00000001000534e6 nginx`ngx_http_block(cf=0x00007fff5fbff4b0, cmd=0x0000000100142cc0, conf=0x000000010307ff28) + 1462 at ngx_http.c:264 frame #5: 0x0000000100022d8c nginx`ngx_conf_handler(cf=0x00007fff5fbff4b0, last=1) + 1068 at ngx_conf_file.c:391 frame #6: 0x0000000100021899 nginx`ngx_conf_parse(cf=0x00007fff5fbff4b0, filename=0x000000010307f1d0) + 1065 at ngx_conf_file.c:247 frame #7: 0x000000010001db5f nginx`ngx_init_cycle(old_cycle=0x00007fff5fbff5a8) + 2719 at ngx_cycle.c:264 frame #8: 0x0000000100000e6e nginx`main(argc=5, argv=0x00007fff5fbff7d0) + 766 at nginx.c:333 (lldb) f 1 frame #1: 0x0000000100101b77 nginx`ngx_http_perl_create_interpreter(cf=0x00007fff5fbff4b0, pmcf=0x00000001030821c8) + 775 at ngx_http_perl_module.c:599 596 embedding[n++] = "-e"; 597 embedding[n++] = "0"; 598 -> 599 n = perl_parse(perl, ngx_http_perl_xs_init, n, embedding, NULL); 600 601 if (n != 0) { 602 ngx_log_error(NGX_LOG_ALERT, cf->log, 0, "perl_parse() failed: %d", n); (lldb) p n (int) $0 = 4 (lldb) p embedding[0] (char *) $1 = 0x000000010012f43c "" (lldb) p embedding[1] (char *) $2 = 0x000000010013c0a4 "-Mnginx" (lldb) p embedding[2] (char *) $3 = 0x000000010013c0ac "-e" (lldb) p embedding[3] (char *) $4 = 0x000000010013c0af "0" (lldb) p embedding[4] error: Couldn't apply expression side effects : Couldn't dematerialize a result variable: couldn't read its memory (lldb) p &embedding[0] (char **) $6 = 0x0000000103278fe0 (lldb) p &embedding[1] (char **) $7 = 0x0000000103278fe8 (lldb) p &embedding[2] (char **) $8 = 0x0000000103278ff0 (lldb) p &embedding[3] (char **) $9 = 0x0000000103278ff8 (lldb) p &embedding[4] (char **) $10 = 0x0000000103279000 As you can see, embedding[4] is the address which triggered EXC_BAD_ACCESS, so perl_parse() must have tried to access it. Let me know if I can be of further assistance. Best regards, Piotr Sikora From dev.panky2014 at gmail.com Tue Aug 5 11:11:08 2014 From: dev.panky2014 at gmail.com (Pankaj Kaushik) Date: Tue, 5 Aug 2014 16:41:08 +0530 Subject: Regarding using of nginx http load balancer In-Reply-To: <591B3BBA-A8A0-4D46-ABFD-EF5DF66C7B84@nginx.com> References: <591B3BBA-A8A0-4D46-ABFD-EF5DF66C7B84@nginx.com> Message-ID: Hey Roman, I do not actually get the idea how to use map in below scenario. Suppose i have two form of uri as mentioned below:- URI 1 : http://abc.10.23.32.22:2000/testFile URI 2 : http://10.20.32.23:2000/def/testFile So i want to fetch out following substring from above mentioned URI's as a key :- Key 1 : abc Key 2 : def Now i want to use them in upstream configuration as :- http { upstream myapp1 { hash Key1 consistent; server 10.23.22.34:8080; } server { listen 80; location / { proxy_pass http://myapp1; } } } How can i do this ? Thanks, Panky On Tue, Aug 5, 2014 at 3:07 PM, Roman Arutyunyan wrote: > > On 05 Aug 2014, at 12:58, Pankaj Kaushik wrote: > > > Hey Piotr, > > > > > > > > On Tue, Aug 5, 2014 at 2:21 PM, Piotr Sikora > wrote: > > > > What's wrong with "hash" [1]? > > > > [1] http://nginx.org/en/docs/http/ngx_http_upstream_module.html#hash > > > > > > > > As mentioned in the syntax at above link:- > > > > Syntax: hash key [consistent]; > > Default: ? > > Context: upstream > > > > > > In my scenario i have to fetch key out of uri and then need to hash that > to identify the backend server. > > > > Basically i need to fetch the key out of uri during run time. > > > > Can you please help in knowing how can i do that? > > > > You can use $arg_XXX variables to access arguments passed in uri. > Or you can use ?map" directive to fetch any part of $uri directly > > http://nginx.org/ru/docs/http/ngx_http_map_module.html > > > Thanks, > > Panky > > > > _______________________________________________ > > nginx-devel mailing list > > nginx-devel at nginx.org > > http://mailman.nginx.org/mailman/listinfo/nginx-devel > > _______________________________________________ > nginx-devel mailing list > nginx-devel at nginx.org > http://mailman.nginx.org/mailman/listinfo/nginx-devel > -------------- next part -------------- An HTML attachment was scrubbed... URL: From arut at nginx.com Tue Aug 5 11:54:17 2014 From: arut at nginx.com (Roman Arutyunyan) Date: Tue, 5 Aug 2014 15:54:17 +0400 Subject: Regarding using of nginx http load balancer In-Reply-To: References: <591B3BBA-A8A0-4D46-ABFD-EF5DF66C7B84@nginx.com> Message-ID: <682C14A7-F138-41BC-919D-ED60E8221D76@nginx.com> On 05 Aug 2014, at 15:11, Pankaj Kaushik wrote: > Hey Roman, > > I do not actually get the idea how to use map in below scenario. > > Suppose i have two form of uri as mentioned below:- > > URI 1 : http://abc.10.23.32.22:2000/testFile > > URI 2 : http://10.20.32.23:2000/def/testFile > > So i want to fetch out following substring from above mentioned URI's as a key :- > > Key 1 : abc > > Key 2 : def > Here?s an example for the second case map $uri $key2 { ~^(?/[^/]+)/ $key; default ""; } Now you can use $key2 in the ?hash? directive. From mdounin at mdounin.ru Tue Aug 5 13:52:02 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Tue, 05 Aug 2014 13:52:02 +0000 Subject: [nginx] Mail: discard pipelined commands after SMTP STARTTLS. Message-ID: details: http://hg.nginx.org/nginx/rev/3f5d0af4e40a branches: changeset: 5787:3f5d0af4e40a user: Maxim Dounin date: Tue Aug 05 12:22:07 2014 +0400 description: Mail: discard pipelined commands after SMTP STARTTLS. The bug had appeared in nginx 1.5.6 (04e43d03e153). Reported by Chris Boulton. diffstat: src/mail/ngx_mail_smtp_handler.c | 3 +++ 1 files changed, 3 insertions(+), 0 deletions(-) diffs (13 lines): diff --git a/src/mail/ngx_mail_smtp_handler.c b/src/mail/ngx_mail_smtp_handler.c --- a/src/mail/ngx_mail_smtp_handler.c +++ b/src/mail/ngx_mail_smtp_handler.c @@ -777,6 +777,9 @@ ngx_mail_smtp_starttls(ngx_mail_session_ ngx_str_null(&s->smtp_from); ngx_str_null(&s->smtp_to); + s->buffer->pos = s->buffer->start; + s->buffer->last = s->buffer->start; + c->read->handler = ngx_mail_starttls_handler; return NGX_OK; } From mdounin at mdounin.ru Tue Aug 5 13:52:13 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Tue, 05 Aug 2014 13:52:13 +0000 Subject: [nginx] nginx-1.7.4-RELEASE Message-ID: details: http://hg.nginx.org/nginx/rev/cbb146b12029 branches: changeset: 5788:cbb146b12029 user: Maxim Dounin date: Tue Aug 05 15:13:04 2014 +0400 description: nginx-1.7.4-RELEASE diffstat: docs/xml/nginx/changes.xml | 129 +++++++++++++++++++++++++++++++++++++++++++++ 1 files changed, 129 insertions(+), 0 deletions(-) diffs (139 lines): diff --git a/docs/xml/nginx/changes.xml b/docs/xml/nginx/changes.xml --- a/docs/xml/nginx/changes.xml +++ b/docs/xml/nginx/changes.xml @@ -5,6 +5,135 @@ + + + + +pipelined-??????? ?? ????????????? +????? ??????? STARTTLS ? SMTP ??????-??????? (CVE-2014-3556); +?????? ????????? ? 1.5.6.
+??????? Chris Boulton. +
+ +pipelined commands were not discarded +after STARTTLS command in SMTP proxy (CVE-2014-3556); +the bug had appeared in 1.5.6.
+Thanks to Chris Boulton. +
+
+ + + +????????????? ???????? ? URI ?????? ?????????? +????????????????? ????? ? ??????? ????????.
+??????? Piotr Sikora. +
+ +URI escaping now uses +uppercase hexadecimal digits.
+Thanks to Piotr Sikora. +
+
+ + + +?????? nginx ????? ??????? ? BoringSSL ? LibreSSL.
+??????? Piotr Sikora. +
+ +now nginx can be build with BoringSSL and LibreSSL.
+Thanks to Piotr Sikora. +
+
+ + + +??????? ????? ????????, ???? ????????????? resolver +? DNS-?????? ????????? ???????????? ?????; +?????? ????????? ? 1.5.8. + + +requests might hang if resolver was used +and a DNS server returned a malformed response; +the bug had appeared in 1.5.8. + + + + + +? ?????? ngx_http_spdy_module.
+??????? Piotr Sikora. +
+ +in the ngx_http_spdy_module.
+Thanks to Piotr Sikora. +
+
+ + + +?????????? $uri ????? ????????? ????? +??? ???????? ?????? ? ????? 400.
+??????? ?????? ???????. +
+ +the $uri variable might contain garbage +when returning errors with code 400.
+Thanks to Sergey Bobrov. +
+
+ + + +? ????????? ?????? ? ????????? proxy_store +? ? ?????? ngx_http_dav_module.
+??????? Feng Gu. +
+ +in error handling in the "proxy_store" directive +and the ngx_http_dav_module.
+Thanks to Feng Gu. +
+
+ + + +??? ???????????? ?????? ? syslog ??? ??????????? segmentation fault; +?????? ????????? ? 1.7.1. + + +a segmentation fault might occur if logging of errors to syslog was used; +the bug had appeared in 1.7.1. + + + + + +?????????? $geoip_latitude, $geoip_longitude, $geoip_dma_code +? $geoip_area_code ????? ?? ????????.
+??????? Yichun Zhang. +
+ +the $geoip_latitude, $geoip_longitude, $geoip_dma_code, +and $geoip_area_code variables might not work.
+Thanks to Yichun Zhang. +
+
+ + + +? ????????? ?????? ????????? ??????.
+??????? Tatsuhiko Kubo ? Piotr Sikora. +
+ +in memory allocation error handling.
+Thanks to Tatsuhiko Kubo and Piotr Sikora. +
+
+ +
+ + From mdounin at mdounin.ru Tue Aug 5 13:52:25 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Tue, 05 Aug 2014 13:52:25 +0000 Subject: [nginx] release-1.7.4 tag Message-ID: details: http://hg.nginx.org/nginx/rev/930ce13f19ab branches: changeset: 5789:930ce13f19ab user: Maxim Dounin date: Tue Aug 05 15:13:05 2014 +0400 description: release-1.7.4 tag diffstat: .hgtags | 1 + 1 files changed, 1 insertions(+), 0 deletions(-) diffs (8 lines): diff --git a/.hgtags b/.hgtags --- a/.hgtags +++ b/.hgtags @@ -373,3 +373,4 @@ d161d68df8be32e5cbf72b07db1a707714827803 0351a6d89c3dbcc7a76295024ba6b70e27b9a497 release-1.7.1 0bd223a546192fdf2e862f33938f4ec2a3b5b283 release-1.7.2 fe7cd01828d5ca7491059f0690bb4453645eb28b release-1.7.3 +cbb146b120296852e781079d5138b04495bab6df release-1.7.4 From mdounin at mdounin.ru Tue Aug 5 13:52:36 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Tue, 05 Aug 2014 13:52:36 +0000 Subject: [nginx] Version bump. Message-ID: details: http://hg.nginx.org/nginx/rev/af00bfa9b701 branches: stable-1.6 changeset: 5790:af00bfa9b701 user: Maxim Dounin date: Tue Aug 05 12:20:22 2014 +0400 description: Version bump. diffstat: src/core/nginx.h | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) diffs (14 lines): diff --git a/src/core/nginx.h b/src/core/nginx.h --- a/src/core/nginx.h +++ b/src/core/nginx.h @@ -9,8 +9,8 @@ #define _NGINX_H_INCLUDED_ -#define nginx_version 1006000 -#define NGINX_VERSION "1.6.0" +#define nginx_version 1006001 +#define NGINX_VERSION "1.6.1" #define NGINX_VER "nginx/" NGINX_VERSION #define NGINX_VAR "NGINX" From mdounin at mdounin.ru Tue Aug 5 13:52:46 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Tue, 05 Aug 2014 13:52:46 +0000 Subject: [nginx] Mail: added a check for the number of arguments in MAIL/... Message-ID: details: http://hg.nginx.org/nginx/rev/d68cd5070f3d branches: stable-1.6 changeset: 5791:d68cd5070f3d user: Maxim Dounin date: Wed May 21 21:56:20 2014 +0400 description: Mail: added a check for the number of arguments in MAIL/RCPT. Missed during introduction of the SMTP pipelining support (04e43d03e153, 1.5.6). Previously, the check wasn't needed as s->buffer was used directly and the number of arguments didn't matter. Reported by Svyatoslav Nikolsky. diffstat: src/mail/ngx_mail_smtp_handler.c | 10 ++++++++++ 1 files changed, 10 insertions(+), 0 deletions(-) diffs (27 lines): diff --git a/src/mail/ngx_mail_smtp_handler.c b/src/mail/ngx_mail_smtp_handler.c --- a/src/mail/ngx_mail_smtp_handler.c +++ b/src/mail/ngx_mail_smtp_handler.c @@ -679,6 +679,11 @@ ngx_mail_smtp_mail(ngx_mail_session_t *s return NGX_OK; } + if (s->args.nelts == 0) { + ngx_str_set(&s->out, smtp_invalid_argument); + return NGX_OK; + } + arg = s->args.elts; arg += s->args.nelts - 1; @@ -713,6 +718,11 @@ ngx_mail_smtp_rcpt(ngx_mail_session_t *s return NGX_OK; } + if (s->args.nelts == 0) { + ngx_str_set(&s->out, smtp_invalid_argument); + return NGX_OK; + } + arg = s->args.elts; arg += s->args.nelts - 1; From mdounin at mdounin.ru Tue Aug 5 13:52:56 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Tue, 05 Aug 2014 13:52:56 +0000 Subject: [nginx] Updated OpenSSL used for win32 builds. Message-ID: details: http://hg.nginx.org/nginx/rev/82f7240614de branches: stable-1.6 changeset: 5792:82f7240614de user: Maxim Dounin date: Tue Jun 17 11:38:55 2014 +0400 description: Updated OpenSSL used for win32 builds. diffstat: misc/GNUmakefile | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diffs (12 lines): diff --git a/misc/GNUmakefile b/misc/GNUmakefile --- a/misc/GNUmakefile +++ b/misc/GNUmakefile @@ -5,7 +5,7 @@ NGINX = nginx-$(VER) TEMP = tmp OBJS = objs.msvc8 -OPENSSL = openssl-1.0.1g +OPENSSL = openssl-1.0.1h ZLIB = zlib-1.2.8 PCRE = pcre-8.34 From mdounin at mdounin.ru Tue Aug 5 13:53:06 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Tue, 05 Aug 2014 13:53:06 +0000 Subject: [nginx] Updated PCRE used for win32 builds. Message-ID: details: http://hg.nginx.org/nginx/rev/358186e79635 branches: stable-1.6 changeset: 5793:358186e79635 user: Maxim Dounin date: Fri Jul 04 22:34:03 2014 +0400 description: Updated PCRE used for win32 builds. diffstat: misc/GNUmakefile | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diffs (12 lines): diff --git a/misc/GNUmakefile b/misc/GNUmakefile --- a/misc/GNUmakefile +++ b/misc/GNUmakefile @@ -7,7 +7,7 @@ TEMP = tmp OBJS = objs.msvc8 OPENSSL = openssl-1.0.1h ZLIB = zlib-1.2.8 -PCRE = pcre-8.34 +PCRE = pcre-8.35 release: export From mdounin at mdounin.ru Tue Aug 5 13:53:17 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Tue, 05 Aug 2014 13:53:17 +0000 Subject: [nginx] Reset of r->uri.len on URI parsing errors. Message-ID: details: http://hg.nginx.org/nginx/rev/b8188afb3bbb branches: stable-1.6 changeset: 5794:b8188afb3bbb user: Maxim Dounin date: Fri Jul 18 20:02:11 2014 +0400 description: Reset of r->uri.len on URI parsing errors. This ensures that debug logging and the $uri variable (if used in 400 Bad Request processing) will not try to access uninitialized memory. Found by Sergey Bobrov. diffstat: src/http/ngx_http_request.c | 2 ++ 1 files changed, 2 insertions(+), 0 deletions(-) diffs (12 lines): diff --git a/src/http/ngx_http_request.c b/src/http/ngx_http_request.c --- a/src/http/ngx_http_request.c +++ b/src/http/ngx_http_request.c @@ -1071,6 +1071,8 @@ ngx_http_process_request_uri(ngx_http_re cscf = ngx_http_get_module_srv_conf(r, ngx_http_core_module); if (ngx_http_parse_complex_uri(r, cscf->merge_slashes) != NGX_OK) { + r->uri.len = 0; + ngx_log_error(NGX_LOG_INFO, r->connection->log, 0, "client sent invalid request"); ngx_http_finalize_request(r, NGX_HTTP_BAD_REQUEST); From mdounin at mdounin.ru Tue Aug 5 13:53:28 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Tue, 05 Aug 2014 13:53:28 +0000 Subject: [nginx] Mail: discard pipelined commands after SMTP STARTTLS. Message-ID: details: http://hg.nginx.org/nginx/rev/4b8a54df08de branches: stable-1.6 changeset: 5795:4b8a54df08de user: Maxim Dounin date: Tue Aug 05 12:22:07 2014 +0400 description: Mail: discard pipelined commands after SMTP STARTTLS. The bug had appeared in nginx 1.5.6 (04e43d03e153). Reported by Chris Boulton. diffstat: src/mail/ngx_mail_smtp_handler.c | 3 +++ 1 files changed, 3 insertions(+), 0 deletions(-) diffs (13 lines): diff --git a/src/mail/ngx_mail_smtp_handler.c b/src/mail/ngx_mail_smtp_handler.c --- a/src/mail/ngx_mail_smtp_handler.c +++ b/src/mail/ngx_mail_smtp_handler.c @@ -777,6 +777,9 @@ ngx_mail_smtp_starttls(ngx_mail_session_ ngx_str_null(&s->smtp_from); ngx_str_null(&s->smtp_to); + s->buffer->pos = s->buffer->start; + s->buffer->last = s->buffer->start; + c->read->handler = ngx_mail_starttls_handler; return NGX_OK; } From mdounin at mdounin.ru Tue Aug 5 13:53:39 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Tue, 05 Aug 2014 13:53:39 +0000 Subject: [nginx] nginx-1.6.1-RELEASE Message-ID: details: http://hg.nginx.org/nginx/rev/0cf84a39c1db branches: stable-1.6 changeset: 5796:0cf84a39c1db user: Maxim Dounin date: Tue Aug 05 15:18:34 2014 +0400 description: nginx-1.6.1-RELEASE diffstat: docs/xml/nginx/changes.xml | 46 ++++++++++++++++++++++++++++++++++++++++++++++ 1 files changed, 46 insertions(+), 0 deletions(-) diffs (56 lines): diff --git a/docs/xml/nginx/changes.xml b/docs/xml/nginx/changes.xml --- a/docs/xml/nginx/changes.xml +++ b/docs/xml/nginx/changes.xml @@ -5,6 +5,52 @@ + + + + +pipelined-??????? ?? ????????????? +????? ??????? STARTTLS ? SMTP ??????-??????? (CVE-2014-3556); +?????? ????????? ? 1.5.6.
+??????? Chris Boulton. +
+ +pipelined commands were not discarded +after STARTTLS command in SMTP proxy (CVE-2014-3556); +the bug had appeared in 1.5.6.
+Thanks to Chris Boulton. +
+
+ + + +?????????? $uri ????? ????????? ????? +??? ???????? ?????? ? ????? 400.
+??????? ?????? ???????. +
+ +the $uri variable might contain garbage +when returning errors with code 400.
+Thanks to Sergey Bobrov. +
+
+ + + +? ?????? ????????? none ????????? smtp_auth; +?????? ????????? ? 1.5.6.
+??????? ?????????? ???????????. +
+ +in the "none" parameter in the "smtp_auth" directive; +the bug had appeared in 1.5.6.
+Thanks to Svyatoslav Nikolsky. +
+
+ +
+ + From mdounin at mdounin.ru Tue Aug 5 13:53:49 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Tue, 05 Aug 2014 13:53:49 +0000 Subject: [nginx] release-1.6.1 tag Message-ID: details: http://hg.nginx.org/nginx/rev/0bf52ed7a36c branches: stable-1.6 changeset: 5797:0bf52ed7a36c user: Maxim Dounin date: Tue Aug 05 15:18:34 2014 +0400 description: release-1.6.1 tag diffstat: .hgtags | 1 + 1 files changed, 1 insertions(+), 0 deletions(-) diffs (8 lines): diff --git a/.hgtags b/.hgtags --- a/.hgtags +++ b/.hgtags @@ -370,3 +370,4 @@ f995a10d4c7e9a817157a6ce7b753297ad32897e 97b47d95e4449cbde976657cf8cbbc118351ffe0 release-1.5.12 fd722b890eabc600394349730a093f50dac31639 release-1.5.13 daa5384fd526a9c18fff4f5135646743628f6bc7 release-1.6.0 +0cf84a39c1db6d6477a1a5b68aee345289d4d87a release-1.6.1 From quanah at zimbra.com Tue Aug 5 17:10:45 2014 From: quanah at zimbra.com (Quanah Gibson-Mount) Date: Tue, 05 Aug 2014 10:10:45 -0700 Subject: [PATCH] The directives such as user, rlimit_core should also be effective on master process In-Reply-To: <20140805075819.GY1849@mdounin.ru> References: <181368080.86148.1407223619639.JavaMail.zimbra@mbs01.zimbra.com> <20140805075819.GY1849@mdounin.ru> Message-ID: <454FE0D76429B30AE437D2F1@[192.168.1.2]> --On Tuesday, August 05, 2014 12:58 PM +0400 Maxim Dounin wrote: > Hello! > > On Tue, Aug 05, 2014 at 02:26:59AM -0500, Kunal Pariani wrote: > >> # HG changeset patch >> # User Kunal Pariani >> # Date 1407194790 25200 >> # Mon Aug 04 16:26:30 2014 -0700 >> # Node ID f25ab24517bb5e45b1b7fa1a1502b907f2cff213 >> # Parent f8764e20fcd7f87d98fe97f82b2a8d0a77ed9097 >> The directives such as user, rlimit_core should also be effective on >> master process > > No, thanks. The master process should remain root to be able to > read priveleged configuration files during configuration reload, > open listening sockets on priveleged ports and so on. The drop to the user happens *after* the files & ports have been opened already. This is how many *nix server processes work, as an additional security measure. We've been using nginx with this patch since nginx 0.5.37, so it's quite heavily tested. See also: and -Quanah -- Quanah Gibson-Mount Server Architect Zimbra, Inc. -------------------- Zimbra :: the leader in open source messaging and collaboration From igor at sysoev.ru Tue Aug 5 17:22:38 2014 From: igor at sysoev.ru (Igor Sysoev) Date: Tue, 5 Aug 2014 21:22:38 +0400 Subject: [PATCH] The directives such as user, rlimit_core should also be effective on master process In-Reply-To: <454FE0D76429B30AE437D2F1@[192.168.1.2]> References: <181368080.86148.1407223619639.JavaMail.zimbra@mbs01.zimbra.com> <20140805075819.GY1849@mdounin.ru> <454FE0D76429B30AE437D2F1@[192.168.1.2]> Message-ID: <41AAA7B7-E434-4BA6-B9E2-4D6F9A7307A5@sysoev.ru> On 05 Aug 2014, at 21:10, Quanah Gibson-Mount wrote: > --On Tuesday, August 05, 2014 12:58 PM +0400 Maxim Dounin wrote: > >> Hello! >> >> On Tue, Aug 05, 2014 at 02:26:59AM -0500, Kunal Pariani wrote: >> >>> # HG changeset patch >>> # User Kunal Pariani >>> # Date 1407194790 25200 >>> # Mon Aug 04 16:26:30 2014 -0700 >>> # Node ID f25ab24517bb5e45b1b7fa1a1502b907f2cff213 >>> # Parent f8764e20fcd7f87d98fe97f82b2a8d0a77ed9097 >>> The directives such as user, rlimit_core should also be effective on >>> master process >> >> No, thanks. The master process should remain root to be able to >> read priveleged configuration files during configuration reload, >> open listening sockets on priveleged ports and so on. > > The drop to the user happens *after* the files & ports have been opened already. This is how many *nix server processes work, as an additional security measure. We've been using nginx with this patch since nginx 0.5.37, so it's quite heavily tested. > > See also: > and > nginx master process does not process incoming connections. It just keep listen ports open. If master process will drop root privileges it will be unable to open new listen ports, files, etc. after graceful reconfiguration on the HUP signal. -- Igor Sysoev http://nginx.com From mdounin at mdounin.ru Tue Aug 5 20:54:07 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Tue, 05 Aug 2014 20:54:07 +0000 Subject: [nginx] Version bump. Message-ID: details: http://hg.nginx.org/nginx/rev/3ef33f32dde8 branches: changeset: 5798:3ef33f32dde8 user: Maxim Dounin date: Tue Aug 05 22:50:25 2014 +0400 description: Version bump. diffstat: src/core/nginx.h | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) diffs (14 lines): diff --git a/src/core/nginx.h b/src/core/nginx.h --- a/src/core/nginx.h +++ b/src/core/nginx.h @@ -9,8 +9,8 @@ #define _NGINX_H_INCLUDED_ -#define nginx_version 1007004 -#define NGINX_VERSION "1.7.4" +#define nginx_version 1007005 +#define NGINX_VERSION "1.7.5" #define NGINX_VER "nginx/" NGINX_VERSION #ifdef NGX_BUILD From mdounin at mdounin.ru Tue Aug 5 20:54:19 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Tue, 05 Aug 2014 20:54:19 +0000 Subject: [nginx] Access log: allowed logs to syslog with "if=" (ticket #5... Message-ID: details: http://hg.nginx.org/nginx/rev/b1f8285297a7 branches: changeset: 5799:b1f8285297a7 user: Maxim Dounin date: Wed Aug 06 00:22:36 2014 +0400 description: Access log: allowed logs to syslog with "if=" (ticket #596). diffstat: src/http/modules/ngx_http_log_module.c | 17 ++++++----------- 1 files changed, 6 insertions(+), 11 deletions(-) diffs (34 lines): diff --git a/src/http/modules/ngx_http_log_module.c b/src/http/modules/ngx_http_log_module.c --- a/src/http/modules/ngx_http_log_module.c +++ b/src/http/modules/ngx_http_log_module.c @@ -1254,17 +1254,6 @@ process_formats: return NGX_CONF_ERROR; } - if (log->syslog_peer != NULL) { - if (cf->args->nelts > 3) { - ngx_conf_log_error(NGX_LOG_EMERG, cf, 0, - "parameter \"%V\" is not supported by syslog", - &value[3]); - return NGX_CONF_ERROR; - } - - return NGX_CONF_OK; - } - size = 0; flush = 0; gzip = 0; @@ -1361,6 +1350,12 @@ process_formats: return NGX_CONF_ERROR; } + if (log->syslog_peer) { + ngx_conf_log_error(NGX_LOG_EMERG, cf, 0, + "logs to syslog cannot be buffered"); + return NGX_CONF_ERROR; + } + if (log->file->data) { buffer = log->file->data; From mdounin at mdounin.ru Tue Aug 5 21:26:03 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Tue, 05 Aug 2014 21:26:03 +0000 Subject: [nginx] Perl: NULL-terminate argument list. Message-ID: details: http://hg.nginx.org/nginx/rev/0570e42ffeed branches: changeset: 5800:0570e42ffeed user: Piotr Sikora date: Thu Jun 19 04:16:36 2014 -0700 description: Perl: NULL-terminate argument list. perl_parse() function expects argv/argc-style argument list, which according to the C standard must be NULL-terminated, that is: argv[argc] == NULL. This change fixes a crash (SIGSEGV) that could happen because of the buffer overrun during perl module initialization. Signed-off-by: Piotr Sikora diffstat: src/http/modules/perl/ngx_http_perl_module.c | 3 ++- 1 files changed, 2 insertions(+), 1 deletions(-) diffs (20 lines): diff --git a/src/http/modules/perl/ngx_http_perl_module.c b/src/http/modules/perl/ngx_http_perl_module.c --- a/src/http/modules/perl/ngx_http_perl_module.c +++ b/src/http/modules/perl/ngx_http_perl_module.c @@ -577,7 +577,7 @@ ngx_http_perl_create_interpreter(ngx_con n = (pmcf->modules != NGX_CONF_UNSET_PTR) ? pmcf->modules->nelts * 2 : 0; - embedding = ngx_palloc(cf->pool, (4 + n) * sizeof(char *)); + embedding = ngx_palloc(cf->pool, (5 + n) * sizeof(char *)); if (embedding == NULL) { goto fail; } @@ -595,6 +595,7 @@ ngx_http_perl_create_interpreter(ngx_con embedding[n++] = "-Mnginx"; embedding[n++] = "-e"; embedding[n++] = "0"; + embedding[n] = NULL; n = perl_parse(perl, ngx_http_perl_xs_init, n, embedding, NULL); From mdounin at mdounin.ru Tue Aug 5 21:26:28 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Wed, 6 Aug 2014 01:26:28 +0400 Subject: [PATCH] Perl: NULL-terminate argument list In-Reply-To: References: <290f3fcb9cf552c235b9.1403176643@Piotrs-MacBook-Pro.local> <20140619132641.GA1849@mdounin.ru> <20140620194102.GQ1849@mdounin.ru> <20140805091142.GZ1849@mdounin.ru> Message-ID: <20140805212627.GZ1849@mdounin.ru> Hello! On Tue, Aug 05, 2014 at 03:33:18AM -0700, Piotr Sikora wrote: > Hey Maxim, > > > Somebody have to investigate these crashes to find out where the > > problem is. As previously suggested, nginx code matches perlembed > > samples, and it looks like as either perl itself or perlembed bug. > > I also suspect that the patch fixes things due to some > > side effect, not because it does something needed. > > > > If you have no plans to investigate it yourself, you may want to > > share a way to reproduce the problem. > > I did enough of investigation to be satisfied with the patch I > provided and I have no plans to dig into perl's code, especially > because I have no interest in the perl module itself. > > Like I said, this condition can be triggered when using malloc with > guard pages, when argv[argc] == pool->d.last == pool->d.end. > > For me, it's 100% reproducible when running nginx-tests under OSX's > guard malloc against fully built nginx built: [...] Thanks, I was able to reproduce it here. It also happily dies with the perlembed sample code modified to allocate embedding array dynamically. Looking into further into perl code shows this: Program received signal SIGSEGV, Segmentation fault. 0x000000010002b364 in S_parse_body (env=0x0, xsinit=0x0) at perl.c:2055 2055 scriptname = argv[0]; (gdb) list 2050 # endif 2051 } 2052 #endif 2053 2054 if (!scriptname) 2055 scriptname = argv[0]; 2056 if (PL_e_script) { 2057 argc++,argv--; 2058 scriptname = BIT_BUCKET; /* don't look for script or read stdin */ 2059 } (gdb) p argv[0] Cannot access memory at address 0x10057b000 I.e., while it generally uses argc to control number of arguments used, in some cases it unconditionally tries to use an argument. In additional to this place, I see at least one more code path where this happens, in "-e" parsing if no parameter follows. So this indeed looks like perlembed bug, and trailing NULL is required due to assumptions in the code. I've committed the patch, thanks. -- Maxim Dounin http://nginx.org/ From mdounin at mdounin.ru Tue Aug 5 21:39:20 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Wed, 6 Aug 2014 01:39:20 +0400 Subject: [PATCH] Style: use ngx_str_set() In-Reply-To: <42d455aafd7fc9c3ab20.1407228779@Piotrs-MacBook-Pro.local> References: <42d455aafd7fc9c3ab20.1407228779@Piotrs-MacBook-Pro.local> Message-ID: <20140805213920.GA1849@mdounin.ru> Hello! On Tue, Aug 05, 2014 at 01:52:59AM -0700, Piotr Sikora wrote: > # HG changeset patch > # User Piotr Sikora > # Date 1407228679 25200 > # Tue Aug 05 01:51:19 2014 -0700 > # Node ID 42d455aafd7fc9c3ab204c6bf7f60a8aafb5179c > # Parent f8764e20fcd7f87d98fe97f82b2a8d0a77ed9097 > Style: use ngx_str_set(). I don't like at least some of the changes, and I would rather say no. -- Maxim Dounin http://nginx.org/ From piotr at cloudflare.com Tue Aug 5 21:59:12 2014 From: piotr at cloudflare.com (Piotr Sikora) Date: Tue, 5 Aug 2014 14:59:12 -0700 Subject: [PATCH] Style: use ngx_str_set() In-Reply-To: <20140805213920.GA1849@mdounin.ru> References: <42d455aafd7fc9c3ab20.1407228779@Piotrs-MacBook-Pro.local> <20140805213920.GA1849@mdounin.ru> Message-ID: Hey Maxim, > I don't like at least some of the changes, and I would rather say > no. Which one and why? I suspected that you might not like some of the changes, but I decided to send them all together instead of one by one... It doesn't have to be all or nothing. I also think that it's a good practice to set the struct (data+len) whenever possible, instead of operating on data and len fields separately, in different parts of code, for the sake of micro-optimization (i.e. auth_basic), especially when those fields are exported and can be accessed and modified by 3rd-party code. Best regards, Piotr Sikora From dev.panky2014 at gmail.com Wed Aug 6 04:43:14 2014 From: dev.panky2014 at gmail.com (Pankaj Kaushik) Date: Wed, 6 Aug 2014 10:13:14 +0530 Subject: Regarding nginx control pannel Message-ID: Hi Guys, I want to see gui for nginx where i want to look at how ngnix is distributing load among set of given servers. Can anybody let me now from where i can get nginx gui or some kind of control panel for that. Thanks, Panky -------------- next part -------------- An HTML attachment was scrubbed... URL: From andrew at nginx.com Wed Aug 6 07:13:16 2014 From: andrew at nginx.com (Andrew Alexeev) Date: Wed, 6 Aug 2014 11:13:16 +0400 Subject: Regarding nginx control pannel In-Reply-To: References: Message-ID: <4D4BAE8F-F7A8-4F5C-A6BE-D43DA65CC6B9@nginx.com> On Aug 6, 2014, at 8:43 AM, Pankaj Kaushik wrote: > Hi Guys, > > I want to see gui for nginx where i want to look at how ngnix is distributing load among set of given servers. > > Can anybody let me now from where i can get nginx gui or some kind of control panel for that Apologies, but let's move these questions out of nginx-devel@ - this is not the list for either community support or documentation issues. You may want to take a look at the offerings we have on http://nginx.com. Cheers From mdounin at mdounin.ru Wed Aug 6 09:29:05 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Wed, 6 Aug 2014 13:29:05 +0400 Subject: [PATCH] Style: use ngx_str_set() In-Reply-To: References: <42d455aafd7fc9c3ab20.1407228779@Piotrs-MacBook-Pro.local> <20140805213920.GA1849@mdounin.ru> Message-ID: <20140806092905.GD1849@mdounin.ru> Hello! On Tue, Aug 05, 2014 at 02:59:12PM -0700, Piotr Sikora wrote: > Hey Maxim, > > > I don't like at least some of the changes, and I would rather say > > no. > > Which one and why? I suspected that you might not like some of the > changes, but I decided to send them all together instead of one by > one... It doesn't have to be all or nothing. Most notably I dislike changes in ngx_http_set_expires() and in ngx_http_auth_basic_user(). And some other changes as well, in particular the code in ngx_set_environment() looks strange when the var pointer (which is normally an ngx_str_t pointer used to iterate array) is used in ngx_str_set(). I don't think that changes of all the code to use ngx_str_set() is really something needed. I rather think that both forms are acceptable, and one or another may be used whichever fits better. > I also think that it's a good practice to set the struct (data+len) > whenever possible, instead of operating on data and len fields > separately, in different parts of code, for the sake of > micro-optimization (i.e. auth_basic), especially when those fields are > exported and can be accessed and modified by 3rd-party code. In nginx, 3rd party code can access all the data, and there is more than one way to break things. -- Maxim Dounin http://nginx.org/ From albertcasademont at gmail.com Wed Aug 6 09:46:22 2014 From: albertcasademont at gmail.com (Albert Casademont Filella) Date: Wed, 6 Aug 2014 11:46:22 +0200 Subject: [nginx] Entity tags: downgrade strong etags to weak ones as needed. In-Reply-To: References: Message-ID: yup, just tested this today and works like a charm, thanks! On Mon, Jul 21, 2014 at 6:48 PM, Aaron Peschel wrote: > Thank you for your help through this whole process, Maxim! > > On Wed, Jun 25, 2014 at 3:40 PM, Maxim Dounin wrote: > > details: http://hg.nginx.org/nginx/rev/e491b26fa5a1 > > branches: > > changeset: 5733:e491b26fa5a1 > > user: Maxim Dounin > > date: Thu Jun 26 02:21:01 2014 +0400 > > description: > > Entity tags: downgrade strong etags to weak ones as needed. > > > > See > http://mailman.nginx.org/pipermail/nginx-devel/2013-November/004523.html. > > > > diffstat: > > > > src/http/modules/ngx_http_addition_filter_module.c | 2 +- > > src/http/modules/ngx_http_gunzip_filter_module.c | 2 +- > > src/http/modules/ngx_http_gzip_filter_module.c | 2 +- > > src/http/modules/ngx_http_ssi_filter_module.c | 5 ++- > > src/http/modules/ngx_http_sub_filter_module.c | 5 ++- > > src/http/modules/ngx_http_xslt_filter_module.c | 6 ++- > > src/http/ngx_http_core_module.c | 40 > ++++++++++++++++++++++ > > src/http/ngx_http_core_module.h | 1 + > > 8 files changed, 56 insertions(+), 7 deletions(-) > > > > diffs (153 lines): > > > > diff --git a/src/http/modules/ngx_http_addition_filter_module.c > b/src/http/modules/ngx_http_addition_filter_module.c > > --- a/src/http/modules/ngx_http_addition_filter_module.c > > +++ b/src/http/modules/ngx_http_addition_filter_module.c > > @@ -121,7 +121,7 @@ ngx_http_addition_header_filter(ngx_http > > > > ngx_http_clear_content_length(r); > > ngx_http_clear_accept_ranges(r); > > - ngx_http_clear_etag(r); > > + ngx_http_weak_etag(r); > > > > return ngx_http_next_header_filter(r); > > } > > diff --git a/src/http/modules/ngx_http_gunzip_filter_module.c > b/src/http/modules/ngx_http_gunzip_filter_module.c > > --- a/src/http/modules/ngx_http_gunzip_filter_module.c > > +++ b/src/http/modules/ngx_http_gunzip_filter_module.c > > @@ -165,7 +165,7 @@ ngx_http_gunzip_header_filter(ngx_http_r > > > > ngx_http_clear_content_length(r); > > ngx_http_clear_accept_ranges(r); > > - ngx_http_clear_etag(r); > > + ngx_http_weak_etag(r); > > > > return ngx_http_next_header_filter(r); > > } > > diff --git a/src/http/modules/ngx_http_gzip_filter_module.c > b/src/http/modules/ngx_http_gzip_filter_module.c > > --- a/src/http/modules/ngx_http_gzip_filter_module.c > > +++ b/src/http/modules/ngx_http_gzip_filter_module.c > > @@ -306,7 +306,7 @@ ngx_http_gzip_header_filter(ngx_http_req > > > > ngx_http_clear_content_length(r); > > ngx_http_clear_accept_ranges(r); > > - ngx_http_clear_etag(r); > > + ngx_http_weak_etag(r); > > > > return ngx_http_next_header_filter(r); > > } > > diff --git a/src/http/modules/ngx_http_ssi_filter_module.c > b/src/http/modules/ngx_http_ssi_filter_module.c > > --- a/src/http/modules/ngx_http_ssi_filter_module.c > > +++ b/src/http/modules/ngx_http_ssi_filter_module.c > > @@ -369,10 +369,13 @@ ngx_http_ssi_header_filter(ngx_http_requ > > if (r == r->main) { > > ngx_http_clear_content_length(r); > > ngx_http_clear_accept_ranges(r); > > - ngx_http_clear_etag(r); > > > > if (!slcf->last_modified) { > > ngx_http_clear_last_modified(r); > > + ngx_http_clear_etag(r); > > + > > + } else { > > + ngx_http_weak_etag(r); > > } > > } > > > > diff --git a/src/http/modules/ngx_http_sub_filter_module.c > b/src/http/modules/ngx_http_sub_filter_module.c > > --- a/src/http/modules/ngx_http_sub_filter_module.c > > +++ b/src/http/modules/ngx_http_sub_filter_module.c > > @@ -175,10 +175,13 @@ ngx_http_sub_header_filter(ngx_http_requ > > > > if (r == r->main) { > > ngx_http_clear_content_length(r); > > - ngx_http_clear_etag(r); > > > > if (!slcf->last_modified) { > > ngx_http_clear_last_modified(r); > > + ngx_http_clear_etag(r); > > + > > + } else { > > + ngx_http_weak_etag(r); > > } > > } > > > > diff --git a/src/http/modules/ngx_http_xslt_filter_module.c > b/src/http/modules/ngx_http_xslt_filter_module.c > > --- a/src/http/modules/ngx_http_xslt_filter_module.c > > +++ b/src/http/modules/ngx_http_xslt_filter_module.c > > @@ -337,12 +337,14 @@ ngx_http_xslt_send(ngx_http_request_t *r > > r->headers_out.content_length = NULL; > > } > > > > - ngx_http_clear_etag(r); > > - > > conf = ngx_http_get_module_loc_conf(r, > ngx_http_xslt_filter_module); > > > > if (!conf->last_modified) { > > ngx_http_clear_last_modified(r); > > + ngx_http_clear_etag(r); > > + > > + } else { > > + ngx_http_weak_etag(r); > > } > > } > > > > diff --git a/src/http/ngx_http_core_module.c > b/src/http/ngx_http_core_module.c > > --- a/src/http/ngx_http_core_module.c > > +++ b/src/http/ngx_http_core_module.c > > @@ -1851,6 +1851,46 @@ ngx_http_set_etag(ngx_http_request_t *r) > > } > > > > > > +void > > +ngx_http_weak_etag(ngx_http_request_t *r) > > +{ > > + size_t len; > > + u_char *p; > > + ngx_table_elt_t *etag; > > + > > + etag = r->headers_out.etag; > > + > > + if (etag == NULL) { > > + return; > > + } > > + > > + if (etag->value.len > 2 > > + && etag->value.data[0] == 'W' > > + && etag->value.data[1] == '/') > > + { > > + return; > > + } > > + > > + if (etag->value.len < 1 || etag->value.data[0] != '"') { > > + r->headers_out.etag->hash = 0; > > + r->headers_out.etag = NULL; > > + return; > > + } > > + > > + p = ngx_pnalloc(r->pool, etag->value.len + 2); > > + if (p == NULL) { > > + r->headers_out.etag->hash = 0; > > + r->headers_out.etag = NULL; > > + return; > > + } > > + > > + len = ngx_sprintf(p, "W/%V", &etag->value) - p; > > + > > + etag->value.data = p; > > + etag->value.len = len; > > +} > > + > > + > > ngx_int_t > > ngx_http_send_response(ngx_http_request_t *r, ngx_uint_t status, > > ngx_str_t *ct, ngx_http_complex_value_t *cv) > > diff --git a/src/http/ngx_http_core_module.h > b/src/http/ngx_http_core_module.h > > --- a/src/http/ngx_http_core_module.h > > +++ b/src/http/ngx_http_core_module.h > > @@ -501,6 +501,7 @@ void *ngx_http_test_content_type(ngx_htt > > ngx_int_t ngx_http_set_content_type(ngx_http_request_t *r); > > void ngx_http_set_exten(ngx_http_request_t *r); > > ngx_int_t ngx_http_set_etag(ngx_http_request_t *r); > > +void ngx_http_weak_etag(ngx_http_request_t *r); > > ngx_int_t ngx_http_send_response(ngx_http_request_t *r, ngx_uint_t > status, > > ngx_str_t *ct, ngx_http_complex_value_t *cv); > > u_char *ngx_http_map_uri_to_path(ngx_http_request_t *r, ngx_str_t *name, > > > > _______________________________________________ > > nginx-devel mailing list > > nginx-devel at nginx.org > > http://mailman.nginx.org/mailman/listinfo/nginx-devel > > _______________________________________________ > nginx-devel mailing list > nginx-devel at nginx.org > http://mailman.nginx.org/mailman/listinfo/nginx-devel > -------------- next part -------------- An HTML attachment was scrubbed... URL: From cubicdaiya at gmail.com Wed Aug 6 15:02:38 2014 From: cubicdaiya at gmail.com (Tatsuhiko Kubo) Date: Thu, 07 Aug 2014 00:02:38 +0900 Subject: [PATCH] Style: use specified macro instead of magic-number Message-ID: <16acd71658e33e498910.1407337358@classic-club.local> # HG changeset patch # User Tatsuhiko Kubo # Date 1407337124 -32400 # Wed Aug 06 23:58:44 2014 +0900 # Node ID 16acd71658e33e4989104a3456bba73e64de5aaa # Parent 0570e42ffeed428338b7dd528f63b37d6d552a24 Style: use specified macro instead of magic-number. diff -r 0570e42ffeed -r 16acd71658e3 src/http/modules/ngx_http_memcached_module.c --- a/src/http/modules/ngx_http_memcached_module.c Thu Jun 19 04:16:36 2014 -0700 +++ b/src/http/modules/ngx_http_memcached_module.c Wed Aug 06 23:58:44 2014 +0900 @@ -391,7 +391,7 @@ found: p = line.data + line.len; u->headers_in.content_length_n = ngx_atoof(start, p - start); - if (u->headers_in.content_length_n == -1) { + if (u->headers_in.content_length_n == NGX_ERROR) { ngx_log_error(NGX_LOG_ERR, r->connection->log, 0, "memcached sent invalid length in response \"%V\" " "for key \"%V\"", From mdounin at mdounin.ru Wed Aug 6 16:20:31 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Wed, 06 Aug 2014 16:20:31 +0000 Subject: [nginx] Style: use specified macro instead of magic-number. Message-ID: details: http://hg.nginx.org/nginx/rev/ab48149b77a6 branches: changeset: 5801:ab48149b77a6 user: Tatsuhiko Kubo date: Wed Aug 06 23:58:44 2014 +0900 description: Style: use specified macro instead of magic-number. diffstat: src/http/modules/ngx_http_memcached_module.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diffs (12 lines): diff --git a/src/http/modules/ngx_http_memcached_module.c b/src/http/modules/ngx_http_memcached_module.c --- a/src/http/modules/ngx_http_memcached_module.c +++ b/src/http/modules/ngx_http_memcached_module.c @@ -391,7 +391,7 @@ found: p = line.data + line.len; u->headers_in.content_length_n = ngx_atoof(start, p - start); - if (u->headers_in.content_length_n == -1) { + if (u->headers_in.content_length_n == NGX_ERROR) { ngx_log_error(NGX_LOG_ERR, r->connection->log, 0, "memcached sent invalid length in response \"%V\" " "for key \"%V\"", From mdounin at mdounin.ru Wed Aug 6 16:20:58 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Wed, 6 Aug 2014 20:20:58 +0400 Subject: [PATCH] Style: use specified macro instead of magic-number In-Reply-To: <16acd71658e33e498910.1407337358@classic-club.local> References: <16acd71658e33e498910.1407337358@classic-club.local> Message-ID: <20140806162058.GI1849@mdounin.ru> Hello! On Thu, Aug 07, 2014 at 12:02:38AM +0900, Tatsuhiko Kubo wrote: > # HG changeset patch > # User Tatsuhiko Kubo > # Date 1407337124 -32400 > # Wed Aug 06 23:58:44 2014 +0900 > # Node ID 16acd71658e33e4989104a3456bba73e64de5aaa > # Parent 0570e42ffeed428338b7dd528f63b37d6d552a24 > Style: use specified macro instead of magic-number. > > diff -r 0570e42ffeed -r 16acd71658e3 src/http/modules/ngx_http_memcached_module.c > --- a/src/http/modules/ngx_http_memcached_module.c Thu Jun 19 04:16:36 2014 -0700 > +++ b/src/http/modules/ngx_http_memcached_module.c Wed Aug 06 23:58:44 2014 +0900 > @@ -391,7 +391,7 @@ found: > p = line.data + line.len; > > u->headers_in.content_length_n = ngx_atoof(start, p - start); > - if (u->headers_in.content_length_n == -1) { > + if (u->headers_in.content_length_n == NGX_ERROR) { > ngx_log_error(NGX_LOG_ERR, r->connection->log, 0, > "memcached sent invalid length in response \"%V\" " > "for key \"%V\"", Committed, thanks. -- Maxim Dounin http://nginx.org/ From jefftk at google.com Fri Aug 8 14:44:59 2014 From: jefftk at google.com (Jeff Kaufman) Date: Fri, 8 Aug 2014 10:44:59 -0400 Subject: processing request body in files Message-ID: In ngx_pagespeed we want to handle POSTs, processing them in our nginx module and performing an action. This works fine for small POSTs, but if they're too big then r->request_body->temp_file is set and the data isn't available in memory. Right now we just log an error, but I'd like to fix this. To read from that file, is it safe to just call ngx_open_tempfile(r->request_body->temp_file->file.name, ...) and then ngx_read_file() until that returns 0 for EOF (like standard unix) or an error? Or is there something more complex we need to do? (I've looked for existing modules that do this, but haven't found any. I'm not finding any modules that currently do that; just ones that issue a rename command to move r->request_body->temp_file into the final destination.) From mdounin at mdounin.ru Fri Aug 8 16:42:41 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Fri, 8 Aug 2014 20:42:41 +0400 Subject: processing request body in files In-Reply-To: References: Message-ID: <20140808164240.GW1849@mdounin.ru> Hello! On Fri, Aug 08, 2014 at 10:44:59AM -0400, Jeff Kaufman wrote: > In ngx_pagespeed we want to handle POSTs, processing them in our nginx > module and performing an action. This works fine for small POSTs, but > if they're too big then r->request_body->temp_file is set and the data > isn't available in memory. Right now we just log an error, but I'd > like to fix this. > > To read from that file, is it safe to just call > ngx_open_tempfile(r->request_body->temp_file->file.name, ...) and then > ngx_read_file() until that returns 0 for EOF (like standard unix) or > an error? Or is there something more complex we need to do? > > (I've looked for existing modules that do this, but haven't found any. > I'm not finding any modules that currently do that; just ones that > issue a rename command to move r->request_body->temp_file into the > final destination.) The modules which read request body from a file don't do this via a file name (in most cases, there is no name at all, as temporary files are unlink()'ed after they are open), but rather through already opened file descriptor. E.g., upstream does this (using output chain) to send a request body to a backend server. In either case, reading temporary files might be suboptimal - you may consider using request body filters instead, an experimental patch can be found, e.g., here: http://mailman.nginx.org/pipermail/nginx-devel/2013-March/003492.html -- Maxim Dounin http://nginx.org/ From dev.panky2014 at gmail.com Sat Aug 9 13:06:52 2014 From: dev.panky2014 at gmail.com (Pankaj Kaushik) Date: Sat, 9 Aug 2014 18:36:52 +0530 Subject: Regarding ERROR: XmlParseFailure Message-ID: Hi, When my uri contains "/", then i do not receive any error. But when my uri contains "/abc/" then i receive xml parsing error. Can any body please let me know what is the issue and why i am getting this error. Below is my nginx.conf. map $uri $key2 { ~^(?/[^/]+)/ $key; default ""; } upstream backend { hash $key2 consistent; server 10.0.0.22:8080; server 10.0.0.23:8080; server 10.0.0.24:8080; server 10.0.0.25:8080; server 10.0.0.26:8080; } upstream backend_2 { server 10.0.0.22:8080; server 10.0.0.23:8080; server 10.0.0.24:8080; server 10.0.0.25:8080; server 10.0.0.26:8080; } server { listen 90 default_server; location = / { proxy_pass http://backend_2; } location $uri { proxy_pass http://backend; } } Thanks, Panky -------------- next part -------------- An HTML attachment was scrubbed... URL: From dev.panky2014 at gmail.com Sat Aug 9 13:38:27 2014 From: dev.panky2014 at gmail.com (Pankaj Kaushik) Date: Sat, 9 Aug 2014 19:08:27 +0530 Subject: Regarding ERROR: XmlParseFailure In-Reply-To: References: Message-ID: Hi, Also when i checked the error log, the client request is not going to any upstream server. Instead it is routing to some default location. 2014/08/09 06:31:55 [info] 32681#0: *2 client 14.102.112.84 closed keepalive connection 2014/08/09 06:36:13 [notice] 329#0: signal process started 2014/08/09 06:36:13 [notice] 32669#0: signal 1 (SIGHUP) received, reconfiguring 2014/08/09 06:36:13 [notice] 32669#0: reconfiguring 2014/08/09 06:36:13 [notice] 32669#0: using the "epoll" event method 2014/08/09 06:36:13 [notice] 32669#0: start worker processes 2014/08/09 06:36:13 [notice] 32669#0: start worker process 330 2014/08/09 06:36:14 [notice] 32681#0: gracefully shutting down 2014/08/09 06:36:14 [notice] 32681#0: exiting 2014/08/09 06:36:14 [notice] 32681#0: exit 2014/08/09 06:36:14 [notice] 32669#0: signal 17 (SIGCHLD) received 2014/08/09 06:36:14 [notice] 32669#0: worker process 32681 exited with code 0 2014/08/09 06:36:14 [notice] 32669#0: signal 29 (SIGIO) received 2014/08/09 06:36:19 [error] 330#0: *3 "/etc/nginx/html/index.html" is not found (2: No such file or directory), client: 14.102.112.84, server: , request: "GET / HTTP/1.1", host: "65.60.72.19:90" 2014/08/09 06:36:19 [info] 330#0: *3 client 14.102.112.84 closed keepalive connection Why it is happening? Thanks, Panky On Sat, Aug 9, 2014 at 6:36 PM, Pankaj Kaushik wrote: > Hi, > > When my uri contains "/", then i do not receive any error. > > But when my uri contains "/abc/" then i receive xml parsing error. > > Can any body please let me know what is the issue and why i am getting > this error. > > Below is my nginx.conf. > > map $uri $key2 { > ~^(?/[^/]+)/ $key; > default ""; > } > > > upstream backend { > hash $key2 consistent; > server 10.0.0.22:8080; > server 10.0.0.23:8080; > server 10.0.0.24:8080; > server 10.0.0.25:8080; > server 10.0.0.26:8080; > } > upstream backend_2 { > server 10.0.0.22:8080; > server 10.0.0.23:8080; > server 10.0.0.24:8080; > server 10.0.0.25:8080; > server 10.0.0.26:8080; > } > > server { > listen 90 default_server; > > location = / { > proxy_pass http://backend_2; > } > location $uri { > proxy_pass http://backend; > } > > } > > Thanks, > Panky > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mdounin at mdounin.ru Sun Aug 10 00:56:01 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Sun, 10 Aug 2014 04:56:01 +0400 Subject: Regarding ERROR: XmlParseFailure In-Reply-To: References: Message-ID: <20140810005601.GC1849@mdounin.ru> Hello! On Sat, Aug 09, 2014 at 07:08:27PM +0530, Pankaj Kaushik wrote: > Hi, > > Also when i checked the error log, the client request is not going to any > upstream server. Instead it is routing to some default location. [...] This type of questions is certainly off-topic for the nginx-devel@ list. Please use nginx@ list for further questions. Thank you. -- Maxim Dounin http://nginx.org/ From mdounin at mdounin.ru Sun Aug 10 13:17:03 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Sun, 10 Aug 2014 13:17:03 +0000 Subject: [nginx] Events: changed nevents type to unsigned in poll module. Message-ID: details: http://hg.nginx.org/nginx/rev/a4e9c7e6e062 branches: changeset: 5802:a4e9c7e6e062 user: Yves Crespin date: Thu Aug 07 14:56:57 2014 +0200 description: Events: changed nevents type to unsigned in poll module. diffstat: src/event/modules/ngx_poll_module.c | 12 ++++++------ 1 files changed, 6 insertions(+), 6 deletions(-) diffs (46 lines): diff --git a/src/event/modules/ngx_poll_module.c b/src/event/modules/ngx_poll_module.c --- a/src/event/modules/ngx_poll_module.c +++ b/src/event/modules/ngx_poll_module.c @@ -22,7 +22,7 @@ static char *ngx_poll_init_conf(ngx_cycl static struct pollfd *event_list; -static ngx_int_t nevents; +static ngx_uint_t nevents; static ngx_str_t poll_name = ngx_string("poll"); @@ -198,7 +198,7 @@ ngx_poll_del_event(ngx_event_t *ev, ngx_ if (e == NULL || e->index == NGX_INVALID_INDEX) { nevents--; - if (ev->index < (ngx_uint_t) nevents) { + if (ev->index < nevents) { ngx_log_debug2(NGX_LOG_DEBUG_EVENT, ev->log, 0, "index: copy event %ui to %i", nevents, ev->index); @@ -212,11 +212,11 @@ ngx_poll_del_event(ngx_event_t *ev, ngx_ "unexpected last event"); } else { - if (c->read->index == (ngx_uint_t) nevents) { + if (c->read->index == nevents) { c->read->index = ev->index; } - if (c->write->index == (ngx_uint_t) nevents) { + if (c->write->index == nevents) { c->write->index = ev->index; } } @@ -240,8 +240,8 @@ ngx_poll_process_events(ngx_cycle_t *cyc { int ready, revents; ngx_err_t err; - ngx_int_t i, nready; - ngx_uint_t found, level; + ngx_int_t nready; + ngx_uint_t i, found, level; ngx_event_t *ev, **queue; ngx_connection_t *c; From mdounin at mdounin.ru Sun Aug 10 13:17:19 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Sun, 10 Aug 2014 13:17:19 +0000 Subject: [nginx] Events: format specifier fixes. Message-ID: details: http://hg.nginx.org/nginx/rev/ecfc601f1225 branches: changeset: 5803:ecfc601f1225 user: Yves Crespin date: Thu Aug 07 15:04:43 2014 +0200 description: Events: format specifier fixes. diffstat: src/event/modules/ngx_poll_module.c | 8 ++++---- src/event/modules/ngx_select_module.c | 4 ++-- 2 files changed, 6 insertions(+), 6 deletions(-) diffs (57 lines): diff --git a/src/event/modules/ngx_poll_module.c b/src/event/modules/ngx_poll_module.c --- a/src/event/modules/ngx_poll_module.c +++ b/src/event/modules/ngx_poll_module.c @@ -251,7 +251,7 @@ ngx_poll_process_events(ngx_cycle_t *cyc if (cycle->log->log_level & NGX_LOG_DEBUG_ALL) { for (i = 0; i < nevents; i++) { ngx_log_debug3(NGX_LOG_DEBUG_EVENT, cycle->log, 0, - "poll: %d: fd:%d ev:%04Xd", + "poll: %ui: fd:%d ev:%04Xd", i, event_list[i].fd, event_list[i].events); } } @@ -268,7 +268,7 @@ ngx_poll_process_events(ngx_cycle_t *cyc } ngx_log_debug2(NGX_LOG_DEBUG_EVENT, cycle->log, 0, - "poll ready %d of %d", ready, nevents); + "poll ready %d of %ui", ready, nevents); if (err) { if (err == NGX_EINTR) { @@ -308,12 +308,12 @@ ngx_poll_process_events(ngx_cycle_t *cyc #if 1 ngx_log_debug4(NGX_LOG_DEBUG_EVENT, cycle->log, 0, - "poll: %d: fd:%d ev:%04Xd rev:%04Xd", + "poll: %ui: fd:%d ev:%04Xd rev:%04Xd", i, event_list[i].fd, event_list[i].events, revents); #else if (revents) { ngx_log_debug4(NGX_LOG_DEBUG_EVENT, cycle->log, 0, - "poll: %d: fd:%d ev:%04Xd rev:%04Xd", + "poll: %ui: fd:%d ev:%04Xd rev:%04Xd", i, event_list[i].fd, event_list[i].events, revents); } #endif diff --git a/src/event/modules/ngx_select_module.c b/src/event/modules/ngx_select_module.c --- a/src/event/modules/ngx_select_module.c +++ b/src/event/modules/ngx_select_module.c @@ -227,7 +227,7 @@ ngx_select_process_events(ngx_cycle_t *c } ngx_log_debug1(NGX_LOG_DEBUG_EVENT, cycle->log, 0, - "change max_fd: %d", max_fd); + "change max_fd: %i", max_fd); } #if (NGX_DEBUG) @@ -240,7 +240,7 @@ ngx_select_process_events(ngx_cycle_t *c } ngx_log_debug1(NGX_LOG_DEBUG_EVENT, cycle->log, 0, - "max_fd: %d", max_fd); + "max_fd: %i", max_fd); } #endif From pdn at cryptopro.ru Mon Aug 11 04:36:12 2014 From: pdn at cryptopro.ru (Dmitrii Pichulin) Date: Mon, 11 Aug 2014 08:36:12 +0400 Subject: [PATCH] allow to use engine keyform for server private key In-Reply-To: References: <20140801165839.GT1849@mdounin.ru> Message-ID: <53E8483C.8020507@cryptopro.ru> What is the current statusof this patch? Let us know if it has any issues. From jefftk at google.com Mon Aug 11 13:35:11 2014 From: jefftk at google.com (Jeff Kaufman) Date: Mon, 11 Aug 2014 09:35:11 -0400 Subject: processing request body in files In-Reply-To: <20140808164240.GW1849@mdounin.ru> References: <20140808164240.GW1849@mdounin.ru> Message-ID: Thanks! On Fri, Aug 8, 2014 at 12:42 PM, Maxim Dounin wrote: > Hello! > > On Fri, Aug 08, 2014 at 10:44:59AM -0400, Jeff Kaufman wrote: > >> In ngx_pagespeed we want to handle POSTs, processing them in our nginx >> module and performing an action. This works fine for small POSTs, but >> if they're too big then r->request_body->temp_file is set and the data >> isn't available in memory. Right now we just log an error, but I'd >> like to fix this. >> >> To read from that file, is it safe to just call >> ngx_open_tempfile(r->request_body->temp_file->file.name, ...) and then >> ngx_read_file() until that returns 0 for EOF (like standard unix) or >> an error? Or is there something more complex we need to do? >> >> (I've looked for existing modules that do this, but haven't found any. >> I'm not finding any modules that currently do that; just ones that >> issue a rename command to move r->request_body->temp_file into the >> final destination.) > > The modules which read request body from a file don't do this via > a file name (in most cases, there is no name at all, as temporary > files are unlink()'ed after they are open), but rather through > already opened file descriptor. E.g., upstream does this (using > output chain) to send a request body to a backend server. > > In either case, reading temporary files might be suboptimal - you > may consider using request body filters instead, an experimental > patch can be found, e.g., here: > > http://mailman.nginx.org/pipermail/nginx-devel/2013-March/003492.html > > -- > Maxim Dounin > http://nginx.org/ > > _______________________________________________ > nginx-devel mailing list > nginx-devel at nginx.org > http://mailman.nginx.org/mailman/listinfo/nginx-devel From mdounin at mdounin.ru Tue Aug 12 00:43:34 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Tue, 12 Aug 2014 04:43:34 +0400 Subject: [PATCH] allow to use engine keyform for server private key In-Reply-To: <53E8483C.8020507@cryptopro.ru> References: <20140801165839.GT1849@mdounin.ru> <53E8483C.8020507@cryptopro.ru> Message-ID: <20140812004334.GN1849@mdounin.ru> Hello! On Mon, Aug 11, 2014 at 08:36:12AM +0400, Dmitrii Pichulin wrote: > What is the current statusof this patch? > > Let us know if it has any issues. As far as I see, there are no serious problems left. There are various minor style issues though (like incorrect patch subject, use of "char" instead of "u_char", and so on). I'm going to cleanup these as time permits. Please ping again if it won't happen in a week or two. (Meanwhile, you may try to clean things yourself and post an updated patch.) BTW, if you have a good example of an engine to test with, this may be also helpful. -- Maxim Dounin http://nginx.org/ From mdounin at mdounin.ru Tue Aug 12 00:45:03 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Tue, 12 Aug 2014 00:45:03 +0000 Subject: [nginx] Events: removed unused variable in ngx_poll_process_even... Message-ID: details: http://hg.nginx.org/nginx/rev/92490a725fef branches: changeset: 5804:92490a725fef user: Maxim Dounin date: Sun Aug 10 17:44:46 2014 +0400 description: Events: removed unused variable in ngx_poll_process_events(). diffstat: src/event/modules/ngx_poll_module.c | 5 +---- 1 files changed, 1 insertions(+), 4 deletions(-) diffs (29 lines): diff --git a/src/event/modules/ngx_poll_module.c b/src/event/modules/ngx_poll_module.c --- a/src/event/modules/ngx_poll_module.c +++ b/src/event/modules/ngx_poll_module.c @@ -240,7 +240,6 @@ ngx_poll_process_events(ngx_cycle_t *cyc { int ready, revents; ngx_err_t err; - ngx_int_t nready; ngx_uint_t i, found, level; ngx_event_t *ev, **queue; ngx_connection_t *c; @@ -300,8 +299,6 @@ ngx_poll_process_events(ngx_cycle_t *cyc ngx_mutex_lock(ngx_posted_events_mutex); - nready = 0; - for (i = 0; i < nevents && ready; i++) { revents = event_list[i].revents; @@ -414,7 +411,7 @@ ngx_poll_process_events(ngx_cycle_t *cyc ngx_log_error(NGX_LOG_ALERT, cycle->log, 0, "poll ready != events"); } - return nready; + return NGX_OK; } From s.martin49 at gmail.com Tue Aug 12 14:03:17 2014 From: s.martin49 at gmail.com (Samuel Martin) Date: Tue, 12 Aug 2014 16:03:17 +0200 Subject: [PATCH 0 of 4 v3] Build-system: Cross-compilation support improvement In-Reply-To: References: Message-ID: ping? On Sat, Aug 2, 2014 at 1:14 AM, Samuel Martin wrote: > Hi all, > > > Here is the 3rd round of this short series improving nginx build-system > support for cross-compilation. > > This series fixes a couple of issues found in the previous submission. > It still tries to be as less intrusive as possible, and intends to make > easier integration in cross-compilataion frameworks such as Buildroot [1]. > > These patches include most of the changes needed for porperly supporting > the cross-compilation, except the sys_nerr guessing part [2], which cannot > merged in nginx because it is too unix-oriented and based on fragile > assumptions > > > [1] http://buildroot.net/ > [2] https://github.com/tSed/buildroot/blob/nginx-integration/package/nginx/nginx-0005-auto-unix-make-sys_nerr-guessing-cross-friendly.patch > > Yours, > Samuel -- Samuel From yu.zhao at getcwd.com Tue Aug 12 21:33:00 2014 From: yu.zhao at getcwd.com (Yu Zhao) Date: Tue, 12 Aug 2014 14:33:00 -0700 Subject: upstream server max_conns Message-ID: Greetings, I'm looking for a solution to limit number of connections to upstream servers because my app engine running on upstream servers performs best when requests are serialized per CPU. The commercial version of Nginx provides upstream server max_conns parameter which "limits the maximum number of simultaneous connections to the proxied server". It sounds very promising and yet somewhat unclear to me. Let's say I have following configuration: worker_processes 4; upstream backend { server backend1.example.com max_conns=8; server backend2.example.com max_conns=8; keepalive 4; } My questions are: 1) From the parameter's description, I believe it's a global (not per worker) option. When it's set to a small value, would one of the workers use all 16 connections? If yes, would all queued requests on other workers just simply timeout? Or the worker who has all connections would releases some of the connections and wake up other workers so they will have a chance to process the queued requests? (AFAIK, the only non-block way to wake up a worker is to use the channel which currently only exists between master and worker.) 2) When total number of connections reaches max_conns, will new requests be queued per upstream server (within a worker) or in a single queue (within a worker)? In another word, will a upstream server be selected before a request is queued or not? If yes, what would happen when a upstream server becomes unavailable? If not, are round-robin and ip_hash load balance algorithms still effective or they just behave like least_conn when there are queued requests? 3) When keepalive is used, what would happen when total number of keepalive connections is equal to total number of max_conns? Will workers hold more connections to some upstream servers and less to other upstream servers (i.e. connections to upstream servers are not evenly distributed). Thanks, Yu -------------- next part -------------- An HTML attachment was scrubbed... URL: From vl at nginx.com Wed Aug 13 04:57:17 2014 From: vl at nginx.com (Homutov Vladimir) Date: Wed, 13 Aug 2014 08:57:17 +0400 Subject: upstream server max_conns In-Reply-To: References: Message-ID: <20140813045716.GA3157@vl> On Tue, Aug 12, 2014 at 02:33:00PM -0700, Yu Zhao wrote: > Greetings, > > I'm looking for a solution to limit number of connections to upstream > servers because my app engine running on upstream servers performs best > when requests are serialized per CPU. > > The commercial version of Nginx provides upstream server max_conns > parameter which "limits the maximum number of simultaneous connections to > the proxied server". It sounds very promising and yet somewhat unclear to > me. > > Let's say I have following configuration: > > worker_processes 4; > > upstream backend { > server backend1.example.com max_conns=8; > server backend2.example.com max_conns=8; > keepalive 4; > } > > My questions are: > > 1) From the parameter's description, I believe it's a global (not per > worker) option. When it's set to a small value, would one of the workers > use all 16 connections? If yes, would all queued requests on other workers > just simply timeout? Or the worker who has all connections would releases > some of the connections and wake up other workers so they will have a > chance to process the queued requests? (AFAIK, the only non-block way to > wake up a worker is to use the channel which currently only exists between > master and worker.) The parameter is global if the nginx.org/r/zone is specified in the upstream configuration. In this case, it means that no more than 8 (in your example) connections will be open to each backend server. If you have 8 open connections, the 9th will be rejected, unless the nginx.org/r/queue is specified. In the latter case, 9th connection will wait during specified timeout for available server. Any worker has chances to process requests in its queue. > 2) When total number of connections reaches max_conns, will new requests be > queued per upstream server (within a worker) or in a single queue (within a > worker)? The requests are queued in each worker. > In another word, will a upstream server be selected before a > request is queued or not? > If yes, what would happen when a upstream server > becomes unavailable? If not, are round-robin and ip_hash load balance > algorithms still effective or they just behave like least_conn when there > are queued requests? If a server cannot be selected (due to all available servers are busy), the request may be put into queue. The next attempt to process the request will pick first available server using the configured balancing algorithm (if there will be only 1 available server, it will be picked, if there are more than one, the balancing algorithm will be applied to choose). > 3) When keepalive is used, what would happen when total number of keepalive > connections is equal to total number of max_conns? Will workers hold more > connections to some upstream servers and less to other upstream servers > (i.e. connections to upstream servers are not evenly distributed). The max_conns parameter is independent from the keepalive and only controls that after accepting N connections server cannot be selected for processing requests. How keepalive connections are distributed among workers depends on your load pattern. > > Thanks, > Yu > _______________________________________________ > nginx-devel mailing list > nginx-devel at nginx.org > http://mailman.nginx.org/mailman/listinfo/nginx-devel From flygoast at 126.com Wed Aug 13 07:03:52 2014 From: flygoast at 126.com (flygoast) Date: Wed, 13 Aug 2014 15:03:52 +0800 (CST) Subject: [PATCH]Upstream: avoided directly terminating the connection. Message-ID: <4621925e.91c.147ce2e14d8.Coremail.flygoast@126.com> # HG changeset patch # User FengGu # Date 1407912835 -28800 # Wed Aug 13 14:53:55 2014 +0800 # Node ID 9dd4e38fcce05bcbba4747873a579bc2c9392dd2 # Parent 92490a725fef588939abfe0e1fe89cda0788d02d Upstream: avoided directly terminating the connection. When memory allocation failed in ngx_http_upstream_cache(), the connection would be terminated directly in ngx_http_upstream_init_request(). Return a INTERNAL_SERVER_ERROR response instead. diff -r 92490a725fef -r 9dd4e38fcce0 src/http/ngx_http_upstream.c --- a/src/http/ngx_http_upstream.c Sun Aug 10 17:44:46 2014 +0400 +++ b/src/http/ngx_http_upstream.c Wed Aug 13 14:53:55 2014 +0800 @@ -519,6 +519,11 @@ ngx_http_upstream_init_request(ngx_http_ return; } + if (rc == NGX_ERROR) { + ngx_http_finalize_request(r, NGX_HTTP_INTERNAL_SERVER_ERROR); + return; + } + if (rc != NGX_DECLINED) { ngx_http_finalize_request(r, rc); return; -------------- next part -------------- An HTML attachment was scrubbed... URL: From a.marinov at ucdn.com Wed Aug 13 09:24:14 2014 From: a.marinov at ucdn.com (Anatoli Marinov) Date: Wed, 13 Aug 2014 12:24:14 +0300 Subject: if-range issue Message-ID: Hello colleagues, Yesterday I found an issue with latest chrome and nginx 1.6.0. For the following request nginx returns 200 OK instead of 206 partial response: REQUEST GET /test.mp4 HTTP/1.1 Range: bytes=53037809-53037867 User-Agent: curl/7.33.0 Host: mydomain.com Accept: */* If-Range: 99f71760accc95bbdeadece7a87f507a RESPONSE HTTP/1.1 200 OK Server: nginx Date: Wed, 13 Aug 2014 09:09:18 GMT Content-Type: video/mp4 Content-Length: 53037868 Connection: keep-alive Keep-Alive: timeout=150 Last-Modified: Fri, 30 May 2014 00:54:47 GMT Etag: 99f71760accc95bbdeadece7a87f507a Expires: Fri, 15 Aug 2014 15:22:31 GMT Cache-Control: max-age=360000 Accept-Ranges: bytes For this request nginx should return partial response with just a few bytes (58). In general Etag and If-Range are equal and there and the range request should be satisfied. I thing in the file ngx_http_range_filter_module.c there is wrong comparation. This check: if (r->headers_in.if_range) { if_range = &r->headers_in.if_range->value; if (if_range->len >= 2 && if_range->data[if_range->len - 1] == '"') { if (r->headers_out.etag == NULL) { goto next_filter; } etag = &r->headers_out.etag->value; ngx_log_debug2(NGX_LOG_DEBUG_HTTP, r->connection->log, 0, "http ir:%V etag:%V", if_range, etag); if (if_range->len != etag->len || ngx_strncmp(if_range->data, etag->data, etag->len) != 0) { goto next_filter; } goto parse; } if (r->headers_out.last_modified_time == (time_t) -1) { goto next_filter; } if_range_time = ngx_http_parse_time(if_range->data, if_range->len); ngx_log_debug2(NGX_LOG_DEBUG_HTTP, r->connection->log, 0, "http ir:%d lm:%d", if_range_time, r->headers_out.last_modified_time); if (if_range_time != r->headers_out.last_modified_time) { goto next_filter; } } ------------------------------------------------------------------------------------------------------------------------ Should be replaced with this: if (r->headers_in.if_range) { if_range = &r->headers_in.if_range->value; if (r->headers_out.etag) { etag = &r->headers_out.etag->value; ngx_log_debug2(NGX_LOG_DEBUG_HTTP, r->connection->log, 0, "http ir:%V etag:%V", if_range, etag); if (if_range->len == etag->len && ngx_strncmp(if_range->data, etag->data, etag->len) == 0) { goto parse; } } if (r->headers_out.last_modified_time != (time_t) -1) { if_range_time = ngx_http_parse_time(if_range->data, if_range->len); ngx_log_debug2(NGX_LOG_DEBUG_HTTP, r->connection->log, 0, "http ir:%d lm:%d", if_range_time, r->headers_out.last_modified_time); if (if_range_time == r->headers_out.last_modified_time) { goto parse; } } goto next_filter; } The new code is simpler and it implements the standard instead the old one. Please comment my proposition. Anatoli Marinov -------------- next part -------------- An HTML attachment was scrubbed... URL: From arut at nginx.com Wed Aug 13 10:22:14 2014 From: arut at nginx.com (Roman Arutyunyan) Date: Wed, 13 Aug 2014 14:22:14 +0400 Subject: if-range issue In-Reply-To: References: Message-ID: <8ADFDC6D-B4C0-47CB-BFA6-124D66BBDA94@nginx.com> Hello, Etag value must be a quoted string. Please add double quotes to ETag and If-Range arguments and try again. RFC7232 / Section 2.3 ETag = entity-tag entity-tag = [ weak ] opaque-tag weak = %x57.2F ; "W/", case-sensitive opaque-tag = DQUOTE *etagc DQUOTE etagc = %x21 / %x23-7E / obs-text ; VCHAR except double quotes, plus obs-text On 13 Aug 2014, at 13:24, Anatoli Marinov wrote: > Hello colleagues, > Yesterday I found an issue with latest chrome and nginx 1.6.0. > For the following request nginx returns 200 OK instead of 206 partial response: > > REQUEST > GET /test.mp4 HTTP/1.1 > Range: bytes=53037809-53037867 > User-Agent: curl/7.33.0 > Host: mydomain.com > Accept: */* > If-Range: 99f71760accc95bbdeadece7a87f507a > > RESPONSE > HTTP/1.1 200 OK > Server: nginx > Date: Wed, 13 Aug 2014 09:09:18 GMT > Content-Type: video/mp4 > Content-Length: 53037868 > Connection: keep-alive > Keep-Alive: timeout=150 > Last-Modified: Fri, 30 May 2014 00:54:47 GMT > Etag: 99f71760accc95bbdeadece7a87f507a > Expires: Fri, 15 Aug 2014 15:22:31 GMT > Cache-Control: max-age=360000 > Accept-Ranges: bytes > > For this request nginx should return partial response with just a few bytes (58). > In general Etag and If-Range are equal and there and the range request should be satisfied. > I thing in the file ngx_http_range_filter_module.c there is wrong comparation. > > This check: > if (r->headers_in.if_range) { > > if_range = &r->headers_in.if_range->value; > > if (if_range->len >= 2 && if_range->data[if_range->len - 1] == '"') { > > if (r->headers_out.etag == NULL) { > goto next_filter; > } > > etag = &r->headers_out.etag->value; > > ngx_log_debug2(NGX_LOG_DEBUG_HTTP, r->connection->log, 0, > "http ir:%V etag:%V", if_range, etag); > > if (if_range->len != etag->len > || ngx_strncmp(if_range->data, etag->data, etag->len) != 0) > { > goto next_filter; > } > > goto parse; > } > > if (r->headers_out.last_modified_time == (time_t) -1) { > goto next_filter; > } > > if_range_time = ngx_http_parse_time(if_range->data, if_range->len); > > ngx_log_debug2(NGX_LOG_DEBUG_HTTP, r->connection->log, 0, > "http ir:%d lm:%d", > if_range_time, r->headers_out.last_modified_time); > > if (if_range_time != r->headers_out.last_modified_time) { > goto next_filter; > } > } > > > > ------------------------------------------------------------------------------------------------------------------------ > > > > Should be replaced with this: > if (r->headers_in.if_range) { > > if_range = &r->headers_in.if_range->value; > > if (r->headers_out.etag) { > etag = &r->headers_out.etag->value; > > ngx_log_debug2(NGX_LOG_DEBUG_HTTP, r->connection->log, 0, > "http ir:%V etag:%V", if_range, etag); > > if (if_range->len == etag->len > && ngx_strncmp(if_range->data, etag->data, etag->len) == 0) { > goto parse; > } > } > > if (r->headers_out.last_modified_time != (time_t) -1) { > if_range_time = ngx_http_parse_time(if_range->data, if_range->len); > > ngx_log_debug2(NGX_LOG_DEBUG_HTTP, r->connection->log, 0, > "http ir:%d lm:%d", > if_range_time, r->headers_out.last_modified_time); > > if (if_range_time == r->headers_out.last_modified_time) { > goto parse; > } > } > > goto next_filter; > } > > The new code is simpler and it implements the standard instead the old one. > Please comment my proposition. > > Anatoli Marinov > > _______________________________________________ > nginx-devel mailing list > nginx-devel at nginx.org > http://mailman.nginx.org/mailman/listinfo/nginx-devel From a.marinov at ucdn.com Wed Aug 13 11:08:10 2014 From: a.marinov at ucdn.com (Anatoli Marinov) Date: Wed, 13 Aug 2014 14:08:10 +0300 Subject: if-range issue In-Reply-To: <8ADFDC6D-B4C0-47CB-BFA6-124D66BBDA94@nginx.com> References: <8ADFDC6D-B4C0-47CB-BFA6-124D66BBDA94@nginx.com> Message-ID: I know this, but I used nginx as a reverse proxy and Etag comes from remote server through nginx upstream. In general I do not have any control on its value. Nginx uses this value as it is sent from the other server. The file is cached with it and also this value is send to all http clients. On Wed, Aug 13, 2014 at 1:22 PM, Roman Arutyunyan wrote: > Hello, > > Etag value must be a quoted string. Please add double quotes to > ETag and If-Range arguments and try again. > > RFC7232 / Section 2.3 > > ETag = entity-tag > > entity-tag = [ weak ] opaque-tag > weak = %x57.2F ; "W/", case-sensitive > opaque-tag = DQUOTE *etagc DQUOTE > etagc = %x21 / %x23-7E / obs-text > ; VCHAR except double quotes, plus obs-text > > On 13 Aug 2014, at 13:24, Anatoli Marinov wrote: > > > Hello colleagues, > > Yesterday I found an issue with latest chrome and nginx 1.6.0. > > For the following request nginx returns 200 OK instead of 206 partial > response: > > > > REQUEST > > GET /test.mp4 HTTP/1.1 > > Range: bytes=53037809-53037867 > > User-Agent: curl/7.33.0 > > Host: mydomain.com > > Accept: */* > > If-Range: 99f71760accc95bbdeadece7a87f507a > > > > RESPONSE > > HTTP/1.1 200 OK > > Server: nginx > > Date: Wed, 13 Aug 2014 09:09:18 GMT > > Content-Type: video/mp4 > > Content-Length: 53037868 > > Connection: keep-alive > > Keep-Alive: timeout=150 > > Last-Modified: Fri, 30 May 2014 00:54:47 GMT > > Etag: 99f71760accc95bbdeadece7a87f507a > > Expires: Fri, 15 Aug 2014 15:22:31 GMT > > Cache-Control: max-age=360000 > > Accept-Ranges: bytes > > > > For this request nginx should return partial response with just a few > bytes (58). > > In general Etag and If-Range are equal and there and the range request > should be satisfied. > > I thing in the file ngx_http_range_filter_module.c there is wrong > comparation. > > > > This check: > > if (r->headers_in.if_range) { > > > > if_range = &r->headers_in.if_range->value; > > > > if (if_range->len >= 2 && if_range->data[if_range->len - 1] == > '"') { > > > > if (r->headers_out.etag == NULL) { > > goto next_filter; > > } > > > > etag = &r->headers_out.etag->value; > > > > ngx_log_debug2(NGX_LOG_DEBUG_HTTP, r->connection->log, 0, > > "http ir:%V etag:%V", if_range, etag); > > > > if (if_range->len != etag->len > > || ngx_strncmp(if_range->data, etag->data, etag->len) != > 0) > > { > > goto next_filter; > > } > > > > goto parse; > > } > > > > if (r->headers_out.last_modified_time == (time_t) -1) { > > goto next_filter; > > } > > > > if_range_time = ngx_http_parse_time(if_range->data, > if_range->len); > > > > ngx_log_debug2(NGX_LOG_DEBUG_HTTP, r->connection->log, 0, > > "http ir:%d lm:%d", > > if_range_time, r->headers_out.last_modified_time); > > > > if (if_range_time != r->headers_out.last_modified_time) { > > goto next_filter; > > } > > } > > > > > > > > > ------------------------------------------------------------------------------------------------------------------------ > > > > > > > > Should be replaced with this: > > if (r->headers_in.if_range) { > > > > if_range = &r->headers_in.if_range->value; > > > > if (r->headers_out.etag) { > > etag = &r->headers_out.etag->value; > > > > ngx_log_debug2(NGX_LOG_DEBUG_HTTP, r->connection->log, 0, > > "http ir:%V etag:%V", if_range, etag); > > > > if (if_range->len == etag->len > > && ngx_strncmp(if_range->data, etag->data, etag->len) == > 0) { > > goto parse; > > } > > } > > > > if (r->headers_out.last_modified_time != (time_t) -1) { > > if_range_time = ngx_http_parse_time(if_range->data, > if_range->len); > > > > ngx_log_debug2(NGX_LOG_DEBUG_HTTP, r->connection->log, 0, > > "http ir:%d lm:%d", > > if_range_time, > r->headers_out.last_modified_time); > > > > if (if_range_time == r->headers_out.last_modified_time) { > > goto parse; > > } > > } > > > > goto next_filter; > > } > > > > The new code is simpler and it implements the standard instead the old > one. > > Please comment my proposition. > > > > Anatoli Marinov > > > > _______________________________________________ > > nginx-devel mailing list > > nginx-devel at nginx.org > > http://mailman.nginx.org/mailman/listinfo/nginx-devel > > _______________________________________________ > nginx-devel mailing list > nginx-devel at nginx.org > http://mailman.nginx.org/mailman/listinfo/nginx-devel > -------------- next part -------------- An HTML attachment was scrubbed... URL: From a.marinov at ucdn.com Wed Aug 13 11:17:04 2014 From: a.marinov at ucdn.com (Anatoli Marinov) Date: Wed, 13 Aug 2014 14:17:04 +0300 Subject: if-range issue In-Reply-To: References: <8ADFDC6D-B4C0-47CB-BFA6-124D66BBDA94@nginx.com> Message-ID: The patch change the behavior for more liberal direction :) On Wed, Aug 13, 2014 at 2:08 PM, Anatoli Marinov wrote: > I know this, but I used nginx as a reverse proxy and Etag comes from > remote server through nginx upstream. In general I do not have any control > on its value. Nginx uses this value as it is sent from the other server. > The file is cached with it and also this value is send to all http clients. > > > On Wed, Aug 13, 2014 at 1:22 PM, Roman Arutyunyan wrote: > >> Hello, >> >> Etag value must be a quoted string. Please add double quotes to >> ETag and If-Range arguments and try again. >> >> RFC7232 / Section 2.3 >> >> ETag = entity-tag >> >> entity-tag = [ weak ] opaque-tag >> weak = %x57.2F ; "W/", case-sensitive >> opaque-tag = DQUOTE *etagc DQUOTE >> etagc = %x21 / %x23-7E / obs-text >> ; VCHAR except double quotes, plus obs-text >> >> On 13 Aug 2014, at 13:24, Anatoli Marinov wrote: >> >> > Hello colleagues, >> > Yesterday I found an issue with latest chrome and nginx 1.6.0. >> > For the following request nginx returns 200 OK instead of 206 partial >> response: >> > >> > REQUEST >> > GET /test.mp4 HTTP/1.1 >> > Range: bytes=53037809-53037867 >> > User-Agent: curl/7.33.0 >> > Host: mydomain.com >> > Accept: */* >> > If-Range: 99f71760accc95bbdeadece7a87f507a >> > >> > RESPONSE >> > HTTP/1.1 200 OK >> > Server: nginx >> > Date: Wed, 13 Aug 2014 09:09:18 GMT >> > Content-Type: video/mp4 >> > Content-Length: 53037868 >> > Connection: keep-alive >> > Keep-Alive: timeout=150 >> > Last-Modified: Fri, 30 May 2014 00:54:47 GMT >> > Etag: 99f71760accc95bbdeadece7a87f507a >> > Expires: Fri, 15 Aug 2014 15:22:31 GMT >> > Cache-Control: max-age=360000 >> > Accept-Ranges: bytes >> > >> > For this request nginx should return partial response with just a few >> bytes (58). >> > In general Etag and If-Range are equal and there and the range request >> should be satisfied. >> > I thing in the file ngx_http_range_filter_module.c there is wrong >> comparation. >> > >> > This check: >> > if (r->headers_in.if_range) { >> > >> > if_range = &r->headers_in.if_range->value; >> > >> > if (if_range->len >= 2 && if_range->data[if_range->len - 1] == >> '"') { >> > >> > if (r->headers_out.etag == NULL) { >> > goto next_filter; >> > } >> > >> > etag = &r->headers_out.etag->value; >> > >> > ngx_log_debug2(NGX_LOG_DEBUG_HTTP, r->connection->log, 0, >> > "http ir:%V etag:%V", if_range, etag); >> > >> > if (if_range->len != etag->len >> > || ngx_strncmp(if_range->data, etag->data, etag->len) >> != 0) >> > { >> > goto next_filter; >> > } >> > >> > goto parse; >> > } >> > >> > if (r->headers_out.last_modified_time == (time_t) -1) { >> > goto next_filter; >> > } >> > >> > if_range_time = ngx_http_parse_time(if_range->data, >> if_range->len); >> > >> > ngx_log_debug2(NGX_LOG_DEBUG_HTTP, r->connection->log, 0, >> > "http ir:%d lm:%d", >> > if_range_time, >> r->headers_out.last_modified_time); >> > >> > if (if_range_time != r->headers_out.last_modified_time) { >> > goto next_filter; >> > } >> > } >> > >> > >> > >> > >> ------------------------------------------------------------------------------------------------------------------------ >> > >> > >> > >> > Should be replaced with this: >> > if (r->headers_in.if_range) { >> > >> > if_range = &r->headers_in.if_range->value; >> > >> > if (r->headers_out.etag) { >> > etag = &r->headers_out.etag->value; >> > >> > ngx_log_debug2(NGX_LOG_DEBUG_HTTP, r->connection->log, 0, >> > "http ir:%V etag:%V", if_range, etag); >> > >> > if (if_range->len == etag->len >> > && ngx_strncmp(if_range->data, etag->data, etag->len) >> == 0) { >> > goto parse; >> > } >> > } >> > >> > if (r->headers_out.last_modified_time != (time_t) -1) { >> > if_range_time = ngx_http_parse_time(if_range->data, >> if_range->len); >> > >> > ngx_log_debug2(NGX_LOG_DEBUG_HTTP, r->connection->log, 0, >> > "http ir:%d lm:%d", >> > if_range_time, >> r->headers_out.last_modified_time); >> > >> > if (if_range_time == r->headers_out.last_modified_time) { >> > goto parse; >> > } >> > } >> > >> > goto next_filter; >> > } >> > >> > The new code is simpler and it implements the standard instead the old >> one. >> > Please comment my proposition. >> > >> > Anatoli Marinov >> > >> > _______________________________________________ >> > nginx-devel mailing list >> > nginx-devel at nginx.org >> > http://mailman.nginx.org/mailman/listinfo/nginx-devel >> >> _______________________________________________ >> nginx-devel mailing list >> nginx-devel at nginx.org >> http://mailman.nginx.org/mailman/listinfo/nginx-devel >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From worley at berkeley.edu Thu Aug 14 03:06:10 2014 From: worley at berkeley.edu (Connor Worley) Date: Wed, 13 Aug 2014 23:06:10 -0400 Subject: [PATCH] chroot directive for unix Message-ID: <342f712f72ceaeed3a51.1407985570@desuplayer.moe> # HG changeset patch # User Connor Worley # Date 1407983583 14400 # Wed Aug 13 22:33:03 2014 -0400 # Node ID 342f712f72ceaeed3a5169fde96beb2a9950944a # Parent 92490a725fef588939abfe0e1fe89cda0788d02d chroot directive for unix Adds a chroot directive that calls chroot on worker processes for unix diff -r 92490a725fef -r 342f712f72ce src/core/nginx.c --- a/src/core/nginx.c Sun Aug 10 17:44:46 2014 +0400 +++ b/src/core/nginx.c Wed Aug 13 22:33:03 2014 -0400 @@ -125,6 +125,13 @@ offsetof(ngx_core_conf_t, rlimit_sigpending), NULL }, + { ngx_string("chroot"), + NGX_MAIN_CONF|NGX_DIRECT_CONF|NGX_CONF_TAKE1, + ngx_conf_set_str_slot, + 0, + offsetof(ngx_core_conf_t, chroot), + NULL }, + { ngx_string("working_directory"), NGX_MAIN_CONF|NGX_DIRECT_CONF|NGX_CONF_TAKE1, ngx_conf_set_str_slot, diff -r 92490a725fef -r 342f712f72ce src/core/ngx_cycle.h --- a/src/core/ngx_cycle.h Sun Aug 10 17:44:46 2014 +0400 +++ b/src/core/ngx_cycle.h Wed Aug 13 22:33:03 2014 -0400 @@ -94,6 +94,8 @@ ngx_uid_t user; ngx_gid_t group; + ngx_str_t chroot; + ngx_str_t working_directory; ngx_str_t lock_file; diff -r 92490a725fef -r 342f712f72ce src/os/unix/ngx_process_cycle.c --- a/src/os/unix/ngx_process_cycle.c Sun Aug 10 17:44:46 2014 +0400 +++ b/src/os/unix/ngx_process_cycle.c Wed Aug 13 22:33:03 2014 -0400 @@ -949,6 +949,15 @@ #endif + if(ccf->chroot.len) { + if (chroot((char *) ccf->chroot.data) == -1) { + ngx_log_error(NGX_LOG_EMERG, cycle->log, ngx_errno, + "chroot(\"%s\") failed", ccf->chroot.data); + /* fatal */ + exit(2); + } + } + if (ccf->working_directory.len) { if (chdir((char *) ccf->working_directory.data) == -1) { ngx_log_error(NGX_LOG_ALERT, cycle->log, ngx_errno, From pluknet at nginx.com Fri Aug 15 10:09:44 2014 From: pluknet at nginx.com (Sergey Kandaurov) Date: Fri, 15 Aug 2014 10:09:44 +0000 Subject: [nginx] Fixed typo. Message-ID: details: http://hg.nginx.org/nginx/rev/79553cb77747 branches: changeset: 5805:79553cb77747 user: Sergey Kandaurov date: Fri Aug 15 14:09:29 2014 +0400 description: Fixed typo. diffstat: src/http/ngx_http_core_module.h | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diffs (12 lines): diff -r 92490a725fef -r 79553cb77747 src/http/ngx_http_core_module.h --- a/src/http/ngx_http_core_module.h Sun Aug 10 17:44:46 2014 +0400 +++ b/src/http/ngx_http_core_module.h Fri Aug 15 14:09:29 2014 +0400 @@ -554,7 +554,7 @@ extern ngx_str_t ngx_http_core_get_meth r->headers_out.content_length->hash = 0; \ r->headers_out.content_length = NULL; \ } - \ + #define ngx_http_clear_accept_ranges(r) \ \ r->allow_ranges = 0; \ From portl4t.cn at gmail.com Sat Aug 16 14:00:48 2014 From: portl4t.cn at gmail.com (portl4t.cn at gmail.com) Date: Sat, 16 Aug 2014 22:00:48 +0800 Subject: [PATCH] Mp4: use trak->smhd_size in ngx_http_mp4_read_smhd_atom() Message-ID: <0FFABE7C908444FE84B55B2D91ABD275@gmail.com> # HG changeset patch # User Gang Li # Date 1408195574 -28800 # Sat Aug 16 21:26:14 2014 +0800 # Node ID 083b9aeb29bce7b18171cd6df5bcd5f34b9e2f1b # Parent 79553cb7774753f9c40a6fa6744b0e0611e41908 Mp4: use trak->smhd_size in ngx_http_mp4_read_smhd_atom() diff -r 79553cb77747 -r 083b9aeb29bc src/http/modules/ngx_http_mp4_module.c --- a/src/http/modules/ngx_http_mp4_module.c Fri Aug 15 14:09:29 2014 +0400 +++ b/src/http/modules/ngx_http_mp4_module.c Sat Aug 16 21:26:14 2014 +0800 @@ -1840,7 +1840,7 @@ atom->pos = atom_header; atom->last = atom_header + atom_size; - trak->vmhd_size += atom_size; + trak->smhd_size += atom_size; trak->out[NGX_HTTP_MP4_SMHD_ATOM].buf = atom; ngx_mp4_atom_next(mp4, atom_data_size); -------------- next part -------------- An HTML attachment was scrubbed... URL: From Markus.Linnala at cybercom.com Sun Aug 17 18:23:03 2014 From: Markus.Linnala at cybercom.com (Markus Linnala) Date: Sun, 17 Aug 2014 21:23:03 +0300 Subject: [PATCH 0 of 1] crash at src/os/unix/ngx_setproctitle.c:128 Message-ID: I got following crash, (gdb) where title=0x7f6d4bf6f750 "master process ../objs/nginx -p /tmp/nginx-test-T8yhrruZvl -c nginx.conf") at src/os/unix/ngx_setproctitle.c:128 at src/os/unix/ngx_process_cycle.c:144 (gdb) up title=0x7f6d4bf6f750 "master process ../objs/nginx -p /tmp/nginx-test-T8yhrruZvl -c nginx.conf") at src/os/unix/ngx_setproctitle.c:128 128 ngx_memset(p, NGX_SETPROCTITLE_PAD, ngx_os_argv_last - (char *) p); (gdb) print ngx_os_argv_last $3 = 0x0 If ngx_init_setproctitle fails for example when ngx_alloc fails, then ngx_setproctitle crashes because ngx_os_argv_last is NULL. From Markus.Linnala at cybercom.com Sun Aug 17 18:23:04 2014 From: Markus.Linnala at cybercom.com (Markus Linnala) Date: Sun, 17 Aug 2014 21:23:04 +0300 Subject: [PATCH 1 of 1] fix crash with ngx_setproctitle In-Reply-To: References: Message-ID: <5524fc592881550bc5ed.1408299784@maage-hp-eb.localdomain> # HG changeset patch # User Markus Linnala # Date 1408298513 -10800 # Sun Aug 17 21:01:53 2014 +0300 # Node ID 5524fc592881550bc5ed108993af148ebd09fa22 # Parent 79553cb7774753f9c40a6fa6744b0e0611e41908 fix crash with ngx_setproctitle Test ngx_init_setproctitle return value to prevent crash at ngx_setproctitle. diff -r 79553cb77747 -r 5524fc592881 src/os/unix/ngx_posix_init.c --- a/src/os/unix/ngx_posix_init.c Fri Aug 15 14:09:29 2014 +0400 +++ b/src/os/unix/ngx_posix_init.c Sun Aug 17 21:01:53 2014 +0300 @@ -40,7 +40,9 @@ } #endif - ngx_init_setproctitle(log); + if (ngx_init_setproctitle(log) != NGX_OK) { + return NGX_ERROR; + } ngx_pagesize = getpagesize(); ngx_cacheline_size = NGX_CPU_CACHE_LINE; From mdounin at mdounin.ru Sun Aug 17 19:03:59 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Sun, 17 Aug 2014 23:03:59 +0400 Subject: [PATCH 1 of 1] fix crash with ngx_setproctitle In-Reply-To: <5524fc592881550bc5ed.1408299784@maage-hp-eb.localdomain> References: <5524fc592881550bc5ed.1408299784@maage-hp-eb.localdomain> Message-ID: <20140817190359.GS1849@mdounin.ru> Hello! On Sun, Aug 17, 2014 at 09:23:04PM +0300, Markus Linnala wrote: > # HG changeset patch > # User Markus Linnala > # Date 1408298513 -10800 > # Sun Aug 17 21:01:53 2014 +0300 > # Node ID 5524fc592881550bc5ed108993af148ebd09fa22 > # Parent 79553cb7774753f9c40a6fa6744b0e0611e41908 > fix crash with ngx_setproctitle > > Test ngx_init_setproctitle return value to prevent crash at ngx_setproctitle. > > diff -r 79553cb77747 -r 5524fc592881 src/os/unix/ngx_posix_init.c > --- a/src/os/unix/ngx_posix_init.c Fri Aug 15 14:09:29 2014 +0400 > +++ b/src/os/unix/ngx_posix_init.c Sun Aug 17 21:01:53 2014 +0300 > @@ -40,7 +40,9 @@ > } > #endif > > - ngx_init_setproctitle(log); > + if (ngx_init_setproctitle(log) != NGX_OK) { > + return NGX_ERROR; > + } > > ngx_pagesize = getpagesize(); > ngx_cacheline_size = NGX_CPU_CACHE_LINE; This is at least incomplete, it will only work for platforms where ngx_init_setproctitle() actually does something. It break things for platforms with setproctitle() and platforms where setproctitle() can't be emulated. At least something like this is needed: --- a/src/os/unix/ngx_setproctitle.h +++ b/src/os/unix/ngx_setproctitle.h @@ -13,7 +13,7 @@ /* FreeBSD, NetBSD, OpenBSD */ -#define ngx_init_setproctitle(log) +#define ngx_init_setproctitle(log) NGX_OK #define ngx_setproctitle(title) setproctitle("%s", title) @@ -39,7 +39,7 @@ void ngx_setproctitle(char *title); #else -#define ngx_init_setproctitle(log) +#define ngx_init_setproctitle(log) NGX_OK #define ngx_setproctitle(title) #endif /* OSes */ -- Maxim Dounin http://nginx.org/ From Markus.Linnala at cybercom.com Sun Aug 17 19:02:20 2014 From: Markus.Linnala at cybercom.com (Markus Linnala) Date: Sun, 17 Aug 2014 22:02:20 +0300 Subject: [PATCH 0 of 1] Problem with create_*_conf return values Message-ID: Many third party modules return other than NULL from create_(main|srv|loc)_conf. Main problem is to return NGX_CONF_ERROR as it causes problems later when we do merge. src/mysql has this problem too. From Markus.Linnala at cybercom.com Sun Aug 17 19:02:21 2014 From: Markus.Linnala at cybercom.com (Markus Linnala) Date: Sun, 17 Aug 2014 22:02:21 +0300 Subject: [PATCH 1 of 1] fix ngx_http_mysql_test_create_loc_conf return value on error In-Reply-To: References: Message-ID: <0719145489f842b14765.1408302141@maage-hp-eb.localdomain> # HG changeset patch # User Markus Linnala # Date 1408301978 -10800 # Sun Aug 17 21:59:38 2014 +0300 # Node ID 0719145489f842b14765506f8856798c2203e3cc # Parent 5524fc592881550bc5ed108993af148ebd09fa22 fix ngx_http_mysql_test_create_loc_conf return value on error diff -r 5524fc592881 -r 0719145489f8 src/mysql/ngx_http_mysql_test.c --- a/src/mysql/ngx_http_mysql_test.c Sun Aug 17 21:01:53 2014 +0300 +++ b/src/mysql/ngx_http_mysql_test.c Sun Aug 17 21:59:38 2014 +0300 @@ -163,7 +163,7 @@ conf = ngx_pcalloc(cf->pool, sizeof(ngx_http_mysql_test_conf_t)); if (conf == NULL) { - return NGX_CONF_ERROR; + return NULL; } return conf; From mdounin at mdounin.ru Sun Aug 17 19:21:35 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Sun, 17 Aug 2014 23:21:35 +0400 Subject: [PATCH 0 of 1] Problem with create_*_conf return values In-Reply-To: References: Message-ID: <20140817192135.GT1849@mdounin.ru> Hello! On Sun, Aug 17, 2014 at 10:02:20PM +0300, Markus Linnala wrote: > Many third party modules return other than NULL from > create_(main|srv|loc)_conf. Main problem is to return NGX_CONF_ERROR as > it causes problems later when we do merge. In nginx itself, this was cleaned up in 0.8.0, see http://hg.nginx.org/nginx/rev/c7d57b539248. > src/mysql has this problem too. The mysql module is incomplete and not connected to the build. -- Maxim Dounin http://nginx.org/ From Markus.Linnala at cybercom.com Sun Aug 17 20:00:29 2014 From: Markus.Linnala at cybercom.com (Markus Linnala) Date: Sun, 17 Aug 2014 23:00:29 +0300 Subject: [PATCH 1 of 2] clear err when ngx_regex_compile has allocation failure In-Reply-To: References: Message-ID: # HG changeset patch # User Markus Linnala # Date 1408303316 -10800 # Sun Aug 17 22:21:56 2014 +0300 # Node ID d350d288cffef0e6395ae1f412842c3b55bc8d42 # Parent 0719145489f842b14765506f8856798c2203e3cc clear err when ngx_regex_compile has allocation failure diff -r 0719145489f8 -r d350d288cffe src/core/ngx_regex.c --- a/src/core/ngx_regex.c Sun Aug 17 21:59:38 2014 +0300 +++ b/src/core/ngx_regex.c Sun Aug 17 22:21:56 2014 +0300 @@ -149,6 +149,7 @@ rc->regex = ngx_pcalloc(rc->pool, sizeof(ngx_regex_t)); if (rc->regex == NULL) { + rc->err.len = 0; return NGX_ERROR; } @@ -159,6 +160,7 @@ if (ngx_pcre_studies != NULL) { elt = ngx_list_push(ngx_pcre_studies); if (elt == NULL) { + rc->err.len = 0; return NGX_ERROR; } From Markus.Linnala at cybercom.com Sun Aug 17 20:00:30 2014 From: Markus.Linnala at cybercom.com (Markus Linnala) Date: Sun, 17 Aug 2014 23:00:30 +0300 Subject: [PATCH 2 of 2] fix ETag allocation failure In-Reply-To: References: Message-ID: <6af8cc12f3c933eed752.1408305630@maage-hp-eb.localdomain> # HG changeset patch # User Markus Linnala # Date 1408305457 -10800 # Sun Aug 17 22:57:37 2014 +0300 # Node ID 6af8cc12f3c933eed752e9ca61622d27a909ca00 # Parent d350d288cffef0e6395ae1f412842c3b55bc8d42 fix ETag allocation failure diff -r d350d288cffe -r 6af8cc12f3c9 src/http/ngx_http_core_module.c --- a/src/http/ngx_http_core_module.c Sun Aug 17 22:21:56 2014 +0300 +++ b/src/http/ngx_http_core_module.c Sun Aug 17 22:57:37 2014 +0300 @@ -1832,14 +1832,15 @@ return NGX_ERROR; } + etag->value.data = ngx_pnalloc(r->pool, NGX_OFF_T_LEN + NGX_TIME_T_LEN + 3); + if (etag->value.data == NULL) { + etag->hash = 0; + return NGX_ERROR; + } + etag->hash = 1; ngx_str_set(&etag->key, "ETag"); - etag->value.data = ngx_pnalloc(r->pool, NGX_OFF_T_LEN + NGX_TIME_T_LEN + 3); - if (etag->value.data == NULL) { - return NGX_ERROR; - } - etag->value.len = ngx_sprintf(etag->value.data, "\"%xT-%xO\"", r->headers_out.last_modified_time, r->headers_out.content_length_n) From Markus.Linnala at cybercom.com Sun Aug 17 20:00:28 2014 From: Markus.Linnala at cybercom.com (Markus Linnala) Date: Sun, 17 Aug 2014 23:00:28 +0300 Subject: [PATCH 0 of 2] Two allocation failure path errors Message-ID: I've found two of allocation failure path problems with clear way to test them. ngx_regex_compile logs uninitialized memory if allocation fails. Mark error message as zero length to prevent errors. Example valgrind output of a case when usin nginx-test rewrite.t: src/core/ngx_regex.c 150 rc->regex = NULL; // ngx_pcalloc(rc->pool, sizeof(ngx_regex_t)); ==20818== Syscall param write(buf) points to uninitialised byte(s) ==20818== at 0x3EEB00E800: __write_nocancel (in /usr/lib64/libpthread-2.18.so) ==20818== by 0x404B1F: ngx_log_error_core (ngx_files.h:147) ==20818== by 0x412AD3: ngx_conf_log_error (ngx_conf_file.c:936) ==20818== by 0x43CF37: ngx_http_regex_compile (ngx_http_variables.c:2312) ==20818== by 0x45F980: ngx_http_rewrite (ngx_http_rewrite_module.c:337) ==20818== by 0x4135C0: ngx_conf_parse (ngx_conf_file.c:391) ==20818== by 0x42A080: ngx_http_core_location (ngx_http_core_module.c:3286) ==20818== by 0x4135C0: ngx_conf_parse (ngx_conf_file.c:391) ==20818== by 0x42A2BB: ngx_http_core_server (ngx_http_core_module.c:3058) ==20818== by 0x4135C0: ngx_conf_parse (ngx_conf_file.c:391) ==20818== by 0x425C52: ngx_http_block (ngx_http.c:239) ==20818== by 0x4135C0: ngx_conf_parse (ngx_conf_file.c:391) ==20818== Address 0xffeffd955 is on thread 1's stack ==20818== And there is emerg message with cleary uninitialized memory. Allocating etags can fail and we still print it when sending Internal Server Error message because we set hash to 1 before failing allocation. This can be tested with not_modified.t from nginx-tests. Exceprt error message when allocation fails: src/http/ngx_http_core_module.c 1838 etag->value.data = NULL; // ngx_pnalloc(r->pool, NGX_OFF_T_LEN + NGX_TIME_T_LEN + 3); # 'HTTP/1.1 500 Internal Server Error # Server: nginx/1.7.5 # Date: Sun, 17 Aug 2014 19:30:15 GMT # Content-Type: text/html # Content-Length: 192 # Connection: close # ETag: # # # 500 Internal Server Error # #

500 Internal Server Error

#
nginx/1.7.5
# # # ' From Markus.Linnala at cybercom.com Sun Aug 17 20:24:00 2014 From: Markus.Linnala at cybercom.com (Markus Linnala) Date: Sun, 17 Aug 2014 23:24:00 +0300 Subject: [PATCH] fix crash with ngx_setproctitle Message-ID: <69fdb8f8270e5297f155.1408307040@maage-hp-eb.localdomain> Test ngx_init_setproctitle return value to prevent crash at ngx_setproctitle. Added modifications not to break environments without setproctitle. Fix from Maxim Dounin. diff -r 79553cb77747 -r 69fdb8f8270e src/os/unix/ngx_posix_init.c --- a/src/os/unix/ngx_posix_init.c Fri Aug 15 14:09:29 2014 +0400 +++ b/src/os/unix/ngx_posix_init.c Sun Aug 17 21:01:53 2014 +0300 @@ -40,7 +40,9 @@ } #endif - ngx_init_setproctitle(log); + if (ngx_init_setproctitle(log) != NGX_OK) { + return NGX_ERROR; + } ngx_pagesize = getpagesize(); ngx_cacheline_size = NGX_CPU_CACHE_LINE; diff -r 79553cb77747 -r 69fdb8f8270e src/os/unix/ngx_setproctitle.h --- a/src/os/unix/ngx_setproctitle.h Fri Aug 15 14:09:29 2014 +0400 +++ b/src/os/unix/ngx_setproctitle.h Sun Aug 17 21:01:53 2014 +0300 @@ -13,7 +13,7 @@ /* FreeBSD, NetBSD, OpenBSD */ -#define ngx_init_setproctitle(log) +#define ngx_init_setproctitle(log) NGX_OK #define ngx_setproctitle(title) setproctitle("%s", title) @@ -39,7 +39,7 @@ #else -#define ngx_init_setproctitle(log) +#define ngx_init_setproctitle(log) NGX_OK #define ngx_setproctitle(title) #endif /* OSes */ From pluknet at nginx.com Mon Aug 18 08:05:42 2014 From: pluknet at nginx.com (Sergey Kandaurov) Date: Mon, 18 Aug 2014 08:05:42 +0000 Subject: [nginx] Image filter: downgrade strong etags to weak ones as nee... Message-ID: details: http://hg.nginx.org/nginx/rev/8cdec62a7751 branches: changeset: 5806:8cdec62a7751 user: Sergey Kandaurov date: Mon Aug 18 12:03:41 2014 +0400 description: Image filter: downgrade strong etags to weak ones as needed. diffstat: src/http/modules/ngx_http_image_filter_module.c | 1 + 1 files changed, 1 insertions(+), 0 deletions(-) diffs (11 lines): diff -r 79553cb77747 -r 8cdec62a7751 src/http/modules/ngx_http_image_filter_module.c --- a/src/http/modules/ngx_http_image_filter_module.c Fri Aug 15 14:09:29 2014 +0400 +++ b/src/http/modules/ngx_http_image_filter_module.c Mon Aug 18 12:03:41 2014 +0400 @@ -1012,6 +1012,7 @@ transparent: b->last_buf = 1; ngx_http_image_length(r, b); + ngx_http_weak_etag(r); return b; } From pauloasilva at gmail.com Mon Aug 18 17:04:26 2014 From: pauloasilva at gmail.com (Paulo Silva) Date: Mon, 18 Aug 2014 18:04:26 +0100 Subject: ngx_conf_set_str_array_slot() bug? Message-ID: Hi, For a module with the following configuration /** * do_nothing_tuple "val1" "val2"; */ static ngx_command_t ngx_http_do_nothing_commands[] = { { ngx_string("do_nothing_tuple"), NGX_HTTP_LOC_CONF | NGX_CONF_TAKE2, ngx_conf_set_str_array_slot, NGX_HTTP_LOC_CONF_OFFSET, offsetof(ngx_http_do_nothing_loc_conf_t, tuple), NULL }, ngx_null_command }; I was expecting that ngx_conf_set_str_array_slot() would create and initialize an ngx_array_t with two elements (nelts == 2), the given parameters, but it saves just the first element (see http://fossies.org/dox/nginx-1.7.4/ngx__conf__file_8c_source.html#l01034). Is it a bug or ngx_conf_set_str_array_slot() use case is different? Regards, Paulo A. Silva http://tech.pauloasilva.com http://linkedin.com/in/devpauloasilva/ From vbart at nginx.com Mon Aug 18 18:32:54 2014 From: vbart at nginx.com (Valentin V. Bartenev) Date: Mon, 18 Aug 2014 22:32:54 +0400 Subject: ngx_conf_set_str_array_slot() bug? In-Reply-To: References: Message-ID: <1772135.mJF0AnsFOn@vbart-laptop> On Monday 18 August 2014 18:04:26 Paulo Silva wrote: > Hi, > For a module with the following configuration > > /** > * do_nothing_tuple "val1" "val2"; > */ > static ngx_command_t ngx_http_do_nothing_commands[] = { > > { > ngx_string("do_nothing_tuple"), > NGX_HTTP_LOC_CONF | NGX_CONF_TAKE2, > ngx_conf_set_str_array_slot, > NGX_HTTP_LOC_CONF_OFFSET, > offsetof(ngx_http_do_nothing_loc_conf_t, tuple), > NULL > }, > > ngx_null_command > }; > > I was expecting that ngx_conf_set_str_array_slot() would create and > initialize an ngx_array_t with two elements (nelts == 2), the given > parameters, but it saves just the first element (see > http://fossies.org/dox/nginx-1.7.4/ngx__conf__file_8c_source.html#l01034). > > Is it a bug or ngx_conf_set_str_array_slot() use case is different? > [..] You may just look at the directives that use this function. For example: http://nginx.org/r/proxy_hide_header Typical configuration can be: proxy_hide_header X-Header-One; proxy_hide_header X-Header-Two; And it will result array with two elements: "X-Header-One" and "X-Header-Two". wbr, Valentin V. Bartenev From pauloasilva at gmail.com Mon Aug 18 23:39:29 2014 From: pauloasilva at gmail.com (Paulo Silva) Date: Tue, 19 Aug 2014 00:39:29 +0100 Subject: ngx_conf_set_str_array_slot() bug? In-Reply-To: <1772135.mJF0AnsFOn@vbart-laptop> References: <1772135.mJF0AnsFOn@vbart-laptop> Message-ID: Great. I was looking for some modules that use this handler (thanks to grep) but it is not an easy task when we do not have an example of the configuration. Thanks a bunch, On Mon, Aug 18, 2014 at 7:32 PM, Valentin V. Bartenev wrote: > On Monday 18 August 2014 18:04:26 Paulo Silva wrote: >> Hi, >> For a module with the following configuration >> >> /** >> * do_nothing_tuple "val1" "val2"; >> */ >> static ngx_command_t ngx_http_do_nothing_commands[] = { >> >> { >> ngx_string("do_nothing_tuple"), >> NGX_HTTP_LOC_CONF | NGX_CONF_TAKE2, >> ngx_conf_set_str_array_slot, >> NGX_HTTP_LOC_CONF_OFFSET, >> offsetof(ngx_http_do_nothing_loc_conf_t, tuple), >> NULL >> }, >> >> ngx_null_command >> }; >> >> I was expecting that ngx_conf_set_str_array_slot() would create and >> initialize an ngx_array_t with two elements (nelts == 2), the given >> parameters, but it saves just the first element (see >> http://fossies.org/dox/nginx-1.7.4/ngx__conf__file_8c_source.html#l01034). >> >> Is it a bug or ngx_conf_set_str_array_slot() use case is different? >> > [..] > > You may just look at the directives that use this function. > For example: http://nginx.org/r/proxy_hide_header > > Typical configuration can be: > > proxy_hide_header X-Header-One; > proxy_hide_header X-Header-Two; > > And it will result array with two elements: "X-Header-One" > and "X-Header-Two". > > wbr, Valentin V. Bartenev > > _______________________________________________ > nginx-devel mailing list > nginx-devel at nginx.org > http://mailman.nginx.org/mailman/listinfo/nginx-devel -- Paulo A. Silva http://tech.pauloasilva.com http://linkedin.com/in/devpauloasilva/ From nginx at philliptaylor.net Mon Aug 18 23:50:31 2014 From: nginx at philliptaylor.net (Phillip Taylor (nginx)) Date: Tue, 19 Aug 2014 00:50:31 +0100 Subject: Guard websites with a secret handshake [ngx_http_knock_module] Message-ID: <2ec7814a33f25172053d128a2a1e5223@master.philliptaylor.net> Hello everyone, I've written a module for nginx that takes the concept of "Port Knocking" and applies it to websites. When you visit an configured nginx website, it returns a 404 "page not found" error. However if you go to secret urls, even though they all, on the service appear to return 404, you are secretly handshaking with nginx. After you've hit the magic combination you ip is logged server side, and you're allowed to the visit the site (that is, it returns content instead of 404). The benefits include: * private websites * protect login pages against bots and scripts * protect against zero day exploits * protect against known exploits if you're slow to defend the site. The code, documentation and even a link to demonstration youtube video is available here: https://github.com/PhillipTaylor/ngx_http_knock_module I contact this mailing list: * to raise awareness that I have developed it. * for possibly inclusion on the 3rd Party Modules page for nginx (http://wiki.nginx.org/3rdPartyModules) * to ask if you would be so kind to provide some code review feedback and advice regarding its quality. * and any other thoughts. Thank you Phillip Taylor From rohit.c.joshi at gmail.com Tue Aug 19 00:24:25 2014 From: rohit.c.joshi at gmail.com (Rohit Joshi) Date: Mon, 18 Aug 2014 20:24:25 -0400 Subject: Patch: Support for two way/mutual SSL authentication for upstream http proxy Message-ID: Patch: Attached patch adds support for two way SSL authentication using client certificate and key for upstream in http proxy (ngx_http_proxy_module.c) Use Case: At my company, we are using two way SSL authentication for communication among all application servers. This is security (NPI/PCI) requirement due to being a financial firm. Currently we are using Oracle Service Bus (OSB) as a reverse proxy, client authentication and upstream routing which I am planning to replace using nginx. In my prototype, I found that nginx doesn't support two way SSL authentication for upstream proxy for which I have provided fix. Patch details: The logic is as below. if proxy_ssl_trusted_certificate is configured and (proxy_ssl_client_certificate or proxy_ssl_client_certificate_key) configured it logs warning for proxy_ssl_client_certificate or proxy_ssl_client_certificate_key will be ignored. if proxy_ssl_trusted_certificate is configured then it use ssl_trusted_certificate for authentication else if proxy_ssl_client_certificate and proxy_ssl_client_certificate_key configured, it uses both to do two way authentication else logs error as either proxy_ssl_trusted_certificate or (proxy_ssl_client_certificate and proxy_ssl_client_certificate_key) required. Added following two new config parameters: proxy_ssl_client_certificate cert.pem; proxy_ssl_client_certificate_key cert.key; Please let me know if you have any questions Thanks, Rohit Joshi -- This e-mail and any attachment is for authorized use by the intended recipient(s) only. It may contain proprietary material, confidential information and/or be subject to legal privilege. It should not be copied, disclosed to, retained or used by, any other party. If you are not an intended recipient then please promptly delete this e-mail and any attachment and all copies and inform the sender. Thank you. -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: ngx_http_proxy_module.c.patch Type: application/octet-stream Size: 4792 bytes Desc: not available URL: From rohit.c.joshi at gmail.com Tue Aug 19 02:13:10 2014 From: rohit.c.joshi at gmail.com (Rohit Joshi) Date: Mon, 18 Aug 2014 22:13:10 -0400 Subject: Patch: Support for two way/mutual SSL authentication for upstream http proxy In-Reply-To: References: Message-ID: Looks like attachment didn't go through. Here is a patch: # HG changeset patch # User Rohit Joshi # Date 1408406738 14400 # Mon Aug 18 20:05:38 2014 -0400 # Node ID 61724860610aee50d73a3a0515c17ee09e8eb349 # Parent 8cdec62a7751153117a46acdf46b50dcf8ac24de Mail:Support for two way SSL for upstream http proxy Added support for two way SSL using client certificate/key. diff -r 8cdec62a7751 -r 61724860610a src/http/modules/ngx_http_proxy_module.c --- a/src/http/modules/ngx_http_proxy_module.c Mon Aug 18 12:03:41 2014 +0400 +++ b/src/http/modules/ngx_http_proxy_module.c Mon Aug 18 20:05:38 2014 -0400 @@ -84,6 +84,8 @@ ngx_uint_t ssl_verify_depth; ngx_str_t ssl_trusted_certificate; ngx_str_t ssl_crl; + ngx_str_t ssl_client_certificate; + ngx_str_t ssl_client_certificate_key; #endif } ngx_http_proxy_loc_conf_t; @@ -598,6 +600,21 @@ offsetof(ngx_http_proxy_loc_conf_t, ssl_crl), NULL }, + { ngx_string("proxy_ssl_client_certificate"), + NGX_HTTP_MAIN_CONF|NGX_HTTP_SRV_CONF|NGX_HTTP_LOC_CONF|NGX_CONF_TAKE1, + ngx_conf_set_str_slot, + NGX_HTTP_LOC_CONF_OFFSET, + offsetof(ngx_http_proxy_loc_conf_t, ssl_client_certificate), + NULL }, + + { ngx_string("proxy_ssl_client_certificate_key"), + NGX_HTTP_MAIN_CONF|NGX_HTTP_SRV_CONF|NGX_HTTP_LOC_CONF|NGX_CONF_TAKE1, + ngx_conf_set_str_slot, + NGX_HTTP_LOC_CONF_OFFSET, + offsetof(ngx_http_proxy_loc_conf_t, ssl_client_certificate_key), + NULL }, + + #endif ngx_null_command @@ -2451,6 +2468,8 @@ * conf->ssl_ciphers = { 0, NULL }; * conf->ssl_trusted_certificate = { 0, NULL }; * conf->ssl_crl = { 0, NULL }; + * conf->ssl_client_certificate = { 0, NULL }; + * conf->ssl_client_certificate_key = { 0, NULL }; */ conf->upstream.store = NGX_CONF_UNSET; @@ -2795,6 +2814,19 @@ if (conf->ssl && ngx_http_proxy_set_ssl(cf, conf) != NGX_OK) { return NGX_CONF_ERROR; } + ngx_conf_merge_str_value(conf->ssl_client_certificate, + prev->ssl_client_certificate, ""); + ngx_conf_merge_str_value(conf->ssl_client_certificate_key, + prev->ssl_client_certificate_key, ""); + if( conf->ssl_trusted_certificate.len != 0 && + ( conf->ssl_client_certificate.len != 0 + || conf->ssl_client_certificate_key.len != 0) ) { + + ngx_log_error(NGX_LOG_WARN, cf->log, 0, + "proxy_ssl_trusted_certificate is configured " + "so proxy_ssl_client_certificate and " + "proxy_ssl_client_certificate_key will be ignored"); + } #endif @@ -3861,22 +3893,42 @@ } if (plcf->upstream.ssl_verify) { - if (plcf->ssl_trusted_certificate.len == 0) { - ngx_log_error(NGX_LOG_EMERG, cf->log, 0, - "no proxy_ssl_trusted_certificate for proxy_ssl_verify"); - return NGX_ERROR; - } - - if (ngx_ssl_trusted_certificate(cf, plcf->upstream.ssl, + + if (plcf->ssl_trusted_certificate.len != 0) { + + if (ngx_ssl_trusted_certificate(cf, plcf->upstream.ssl, &plcf->ssl_trusted_certificate, plcf->ssl_verify_depth) - != NGX_OK) - { + != NGX_OK) + { + return NGX_ERROR; + } + + if (ngx_ssl_crl(cf, plcf->upstream.ssl, &plcf->ssl_crl) != NGX_OK) { + return NGX_ERROR; + } + + }else if (plcf->ssl_client_certificate_key.len != 0 && + plcf->ssl_client_certificate.len != 0) { + + if (ngx_ssl_certificate(cf, plcf->upstream.ssl, + &plcf->ssl_client_certificate, + &plcf->ssl_client_certificate_key, + 0) + != NGX_OK) + { + ngx_ssl_error(NGX_LOG_EMERG, cf->log, 0, + "ngx_ssl_certificate failed."); + return NGX_ERROR; + } + }else { + ngx_log_error(NGX_LOG_EMERG, cf->log, 0, + "no proxy_ssl_trusted_certificate or " + "(proxy_ssl_client_certificate and " + "proxy_ssl_client_certificate_key for " + "mutual authentication) for proxy_ssl_verify"); return NGX_ERROR; - } - - if (ngx_ssl_crl(cf, plcf->upstream.ssl, &plcf->ssl_crl) != NGX_OK) { - return NGX_ERROR; + } } -------------- next part -------------- An HTML attachment was scrubbed... URL: From mat999 at gmail.com Tue Aug 19 02:54:16 2014 From: mat999 at gmail.com (SplitIce) Date: Tue, 19 Aug 2014 12:54:16 +1000 Subject: Guard websites with a secret handshake [ngx_http_knock_module] In-Reply-To: <2ec7814a33f25172053d128a2a1e5223@master.philliptaylor.net> References: <2ec7814a33f25172053d128a2a1e5223@master.philliptaylor.net> Message-ID: Is it just me or would access_records be per worker and hence this module not work with workers > 1 ? Additionally for (i = 0; i < NGX_HTTP_KNOCK__IP_DB_SIZE && i < ngx_http_knock_next_free_slot; i++) { if (access_records[i].ip_addr == ip_addr) Perhaps using the nginx Red-Black tree here would lead to better performance (in place of O(n)) and remove the limitation on number of IPs. On Tue, Aug 19, 2014 at 9:50 AM, Phillip Taylor (nginx) < nginx at philliptaylor.net> wrote: > Hello everyone, > > I've written a module for nginx that takes the concept of "Port Knocking" > and applies it to websites. When you visit an configured nginx website, it > returns a 404 "page not found" error. However if you go to secret urls, > even though they all, on the service appear to return 404, you are secretly > handshaking with nginx. After you've hit the magic combination you ip is > logged server side, and you're allowed to the visit the site (that is, it > returns content instead of 404). > > The benefits include: > * private websites > * protect login pages against bots and scripts > * protect against zero day exploits > * protect against known exploits if you're slow to defend the site. > > The code, documentation and even a link to demonstration youtube video is > available here: > > https://github.com/PhillipTaylor/ngx_http_knock_module > > I contact this mailing list: > > * to raise awareness that I have developed it. > * for possibly inclusion on the 3rd Party Modules page for nginx ( > http://wiki.nginx.org/3rdPartyModules) > * to ask if you would be so kind to provide some code review feedback and > advice regarding its quality. > * and any other thoughts. > > Thank you > > Phillip Taylor > > _______________________________________________ > nginx-devel mailing list > nginx-devel at nginx.org > http://mailman.nginx.org/mailman/listinfo/nginx-devel > -------------- next part -------------- An HTML attachment was scrubbed... URL: From arut at nginx.com Tue Aug 19 10:16:06 2014 From: arut at nginx.com (Roman Arutyunyan) Date: Tue, 19 Aug 2014 10:16:06 +0000 Subject: [nginx] Mp4: use trak->smhd_size in ngx_http_mp4_read_smhd_atom(). Message-ID: details: http://hg.nginx.org/nginx/rev/6d468b45f40e branches: changeset: 5807:6d468b45f40e user: Roman Arutyunyan date: Tue Aug 19 14:13:39 2014 +0400 description: Mp4: use trak->smhd_size in ngx_http_mp4_read_smhd_atom(). Reported by Gang Li. diffstat: src/http/modules/ngx_http_mp4_module.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diffs (12 lines): diff -r 8cdec62a7751 -r 6d468b45f40e src/http/modules/ngx_http_mp4_module.c --- a/src/http/modules/ngx_http_mp4_module.c Mon Aug 18 12:03:41 2014 +0400 +++ b/src/http/modules/ngx_http_mp4_module.c Tue Aug 19 14:13:39 2014 +0400 @@ -1840,7 +1840,7 @@ ngx_http_mp4_read_smhd_atom(ngx_http_mp4 atom->pos = atom_header; atom->last = atom_header + atom_size; - trak->vmhd_size += atom_size; + trak->smhd_size += atom_size; trak->out[NGX_HTTP_MP4_SMHD_ATOM].buf = atom; ngx_mp4_atom_next(mp4, atom_data_size); From arut at nginx.com Tue Aug 19 10:16:39 2014 From: arut at nginx.com (Roman Arutyunyan) Date: Tue, 19 Aug 2014 14:16:39 +0400 Subject: [PATCH] Mp4: use trak->smhd_size in ngx_http_mp4_read_smhd_atom() In-Reply-To: <0FFABE7C908444FE84B55B2D91ABD275@gmail.com> References: <0FFABE7C908444FE84B55B2D91ABD275@gmail.com> Message-ID: Committed, thanks. On 16 Aug 2014, at 18:00, portl4t.cn at gmail.com wrote: > # HG changeset patch > # User Gang Li > # Date 1408195574 -28800 > # Sat Aug 16 21:26:14 2014 +0800 > # Node ID 083b9aeb29bce7b18171cd6df5bcd5f34b9e2f1b > # Parent 79553cb7774753f9c40a6fa6744b0e0611e41908 > Mp4: use trak->smhd_size in ngx_http_mp4_read_smhd_atom() > > diff -r 79553cb77747 -r 083b9aeb29bc src/http/modules/ngx_http_mp4_module.c > --- a/src/http/modules/ngx_http_mp4_module.c Fri Aug 15 14:09:29 2014 +0400 > +++ b/src/http/modules/ngx_http_mp4_module.c Sat Aug 16 21:26:14 2014 +0800 > @@ -1840,7 +1840,7 @@ > atom->pos = atom_header; > atom->last = atom_header + atom_size; > > - trak->vmhd_size += atom_size; > + trak->smhd_size += atom_size; > trak->out[NGX_HTTP_MP4_SMHD_ATOM].buf = atom; > > ngx_mp4_atom_next(mp4, atom_data_size); > _______________________________________________ > nginx-devel mailing list > nginx-devel at nginx.org > http://mailman.nginx.org/mailman/listinfo/nginx-devel From kpariani at zimbra.com Tue Aug 19 22:58:15 2014 From: kpariani at zimbra.com (Kunal Pariani) Date: Tue, 19 Aug 2014 17:58:15 -0500 (CDT) Subject: [PATCH] SSL support for the mail proxy module Message-ID: <1399448073.850051.1408489095173.JavaMail.zimbra@zimbra.com> # HG changeset patch # User Kunal Pariani # Date 1408485440 25200 # Tue Aug 19 14:57:20 2014 -0700 # Node ID 7858c2a9ac0e83aa779197fc028b4d078612e7e8 # Parent f25ab24517bb5e45b1b7fa1a1502b907f2cff213 SSL support for the mail proxy diff -r f25ab24517bb -r 7858c2a9ac0e src/mail/ngx_mail_proxy_module.c --- a/src/mail/ngx_mail_proxy_module.c Mon Aug 04 16:26:30 2014 -0700 +++ b/src/mail/ngx_mail_proxy_module.c Tue Aug 19 14:57:20 2014 -0700 @@ -18,6 +18,8 @@ ngx_flag_t xclient; size_t buffer_size; ngx_msec_t timeout; + ngx_flag_t proxy_ssl; + ngx_ssl_t *ssl; } ngx_mail_proxy_conf_t; @@ -35,7 +37,13 @@ static void *ngx_mail_proxy_create_conf(ngx_conf_t *cf); static char *ngx_mail_proxy_merge_conf(ngx_conf_t *cf, void *parent, void *child); - +#if (NGX_MAIL_SSL) +static char *ngx_mail_proxy_ssl(ngx_conf_t *cf, ngx_command_t *cmd, + void *conf); +static void ngx_mail_proxy_ssl_init_connection(ngx_mail_session_t *s, + ngx_connection_t *c); +static void ngx_mail_proxy_ssl_handshake(ngx_connection_t *c); +#endif static ngx_command_t ngx_mail_proxy_commands[] = { @@ -74,6 +82,16 @@ offsetof(ngx_mail_proxy_conf_t, xclient), NULL }, +#if (NGX_MAIL_SSL) + { ngx_string("proxy_ssl"), + NGX_MAIL_MAIN_CONF|NGX_MAIL_SRV_CONF|NGX_CONF_FLAG, + ngx_mail_proxy_ssl, + NGX_MAIL_SRV_CONF_OFFSET, + offsetof(ngx_mail_proxy_conf_t, proxy_ssl), + NULL }, + +#endif + ngx_null_command }; @@ -174,6 +192,15 @@ s->out.len = 0; +#if (NGX_MAIL_SSL) + + if (pcf->proxy_ssl && p->upstream.connection->ssl == NULL) { + ngx_mail_proxy_ssl_init_connection(s, p->upstream.connection); + return; + } + +#endif + switch (s->protocol) { case NGX_MAIL_POP3_PROTOCOL: @@ -1092,6 +1119,13 @@ "close mail proxy connection: %d", s->proxy->upstream.connection->fd); +#if (NGX_MAIL_SSL) + if (s->proxy->upstream.connection->ssl) { + s->proxy->upstream.connection->ssl->no_wait_shutdown = 1; + ngx_ssl_shutdown(s->proxy->upstream.connection); + } +#endif + ngx_close_connection(s->proxy->upstream.connection); } @@ -1114,6 +1148,8 @@ pcf->xclient = NGX_CONF_UNSET; pcf->buffer_size = NGX_CONF_UNSET_SIZE; pcf->timeout = NGX_CONF_UNSET_MSEC; + pcf->proxy_ssl = NGX_CONF_UNSET; + pcf->ssl = NGX_CONF_UNSET_PTR; return pcf; } @@ -1131,6 +1167,118 @@ ngx_conf_merge_size_value(conf->buffer_size, prev->buffer_size, (size_t) ngx_pagesize); ngx_conf_merge_msec_value(conf->timeout, prev->timeout, 24 * 60 * 60000); + ngx_conf_merge_value(conf->proxy_ssl, prev->proxy_ssl, 0); + ngx_conf_merge_ptr_value(conf->ssl, prev->ssl, NULL); return NGX_CONF_OK; } + +#if (NGX_MAIL_SSL) + +static char * +ngx_mail_proxy_ssl(ngx_conf_t *cf, ngx_command_t *cmd, void *conf) { + + ngx_pool_cleanup_t *cln; + char *rc; + ngx_mail_proxy_conf_t *pcf; + + rc = ngx_conf_set_flag_slot(cf, cmd, conf); + if (rc != NGX_CONF_OK) { + return rc; + } + + pcf = (ngx_mail_proxy_conf_t *)conf; + + if (!pcf->proxy_ssl) { + return NGX_CONF_OK; + } + + pcf->ssl = ngx_pcalloc(cf->pool, sizeof(ngx_ssl_t)); + + if (pcf->ssl == NULL) { + return NGX_CONF_ERROR; + } + + pcf->ssl->log = cf->log; + + // don't support SSLv2 anymore + if (ngx_ssl_create(pcf->ssl, NGX_SSL_SSLv3|NGX_SSL_TLSv1, NULL) + != NGX_OK) { + return NGX_CONF_ERROR; + } + + cln = ngx_pool_cleanup_add(cf->pool, 0); + if (cln == NULL) { + return NGX_CONF_ERROR; + } + + cln->handler = ngx_ssl_cleanup_ctx; + cln->data = pcf->ssl; + + return NGX_CONF_OK; +} + +static void +ngx_mail_proxy_ssl_init_connection(ngx_mail_session_t *s, ngx_connection_t *c) +{ + ngx_int_t rc; + ngx_mail_proxy_conf_t *pcf; + + pcf = ngx_mail_get_module_srv_conf(s, ngx_mail_proxy_module); + + if (ngx_ssl_create_connection(pcf->ssl, c, + NGX_SSL_BUFFER|NGX_SSL_CLIENT) + != NGX_OK) + { + ngx_mail_proxy_internal_server_error(s); + return; + } + + s->connection->log->action = "SSL handshaking to upstream"; + + rc = ngx_ssl_handshake(c); + + if (rc == NGX_AGAIN) { + c->ssl->handler = ngx_mail_proxy_ssl_handshake; + return; + } + + ngx_mail_proxy_ssl_handshake(c); +} + + +static void +ngx_mail_proxy_ssl_handshake(ngx_connection_t *c) +{ + ngx_mail_session_t *s; + s = c->data; + + if (c->ssl->handshaked) { + c->write->handler = ngx_mail_proxy_dummy_handler; + switch (s->protocol) { + + case NGX_MAIL_POP3_PROTOCOL: + c->read->handler = ngx_mail_proxy_pop3_handler; + s->mail_state = ngx_pop3_start; + break; + + case NGX_MAIL_IMAP_PROTOCOL: + c->read->handler = ngx_mail_proxy_imap_handler; + s->mail_state = ngx_imap_start; + break; + + default: /* NGX_MAIL_SMTP_PROTOCOL */ + c->read->handler = ngx_mail_proxy_smtp_handler; + s->mail_state = ngx_smtp_start; + break; + } + + /* server might have sent the initial welcome msg */ + c->read->handler(c->read); + } else { + /* when handshake fails, we should close the session */ + ngx_mail_proxy_upstream_error(s); + } +} + +#endif Thanks -Kunal -------------- next part -------------- An HTML attachment was scrubbed... URL: From allegro at mychinamap.com Wed Aug 20 07:48:56 2014 From: allegro at mychinamap.com (XueHJ) Date: Wed, 20 Aug 2014 15:48:56 +0800 Subject: load large data in nginx Message-ID: Hello everyone, I'm a newborn, I want to load a large data structures in nginx, about 800MB, how to initialize and access? Better give me a rough framework code, thanks. best regards, From vbart at nginx.com Wed Aug 20 09:15:28 2014 From: vbart at nginx.com (Valentin Bartenev) Date: Wed, 20 Aug 2014 09:15:28 +0000 Subject: [nginx] Mail: fixed a comment. Message-ID: details: http://hg.nginx.org/nginx/rev/a305a4477af0 branches: changeset: 5808:a305a4477af0 user: Valentin Bartenev date: Wed Aug 20 13:13:27 2014 +0400 description: Mail: fixed a comment. There's no loc_conf in the mail module. diffstat: src/mail/ngx_mail.c | 3 +-- 1 files changed, 1 insertions(+), 2 deletions(-) diffs (13 lines): diff -r 6d468b45f40e -r a305a4477af0 src/mail/ngx_mail.c --- a/src/mail/ngx_mail.c Tue Aug 19 14:13:39 2014 +0400 +++ b/src/mail/ngx_mail.c Wed Aug 20 13:13:27 2014 +0400 @@ -131,8 +131,7 @@ ngx_mail_block(ngx_conf_t *cf, ngx_comma /* - * create the main_conf's, the null srv_conf's, and the null loc_conf's - * of the all mail modules + * create the main_conf's and the null srv_conf's of the all mail modules */ for (m = 0; ngx_modules[m]; m++) { From vbart at nginx.com Wed Aug 20 09:15:32 2014 From: vbart at nginx.com (Valentin Bartenev) Date: Wed, 20 Aug 2014 09:15:32 +0000 Subject: [nginx] Mail: fixed number of arguments allowed in the listen di... Message-ID: details: http://hg.nginx.org/nginx/rev/bb26f7ceaaf1 branches: changeset: 5809:bb26f7ceaaf1 user: Valentin Bartenev date: Wed Aug 20 13:13:27 2014 +0400 description: Mail: fixed number of arguments allowed in the listen directive. diffstat: src/mail/ngx_mail_core_module.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diffs (12 lines): diff -r a305a4477af0 -r bb26f7ceaaf1 src/mail/ngx_mail_core_module.c --- a/src/mail/ngx_mail_core_module.c Wed Aug 20 13:13:27 2014 +0400 +++ b/src/mail/ngx_mail_core_module.c Wed Aug 20 13:13:27 2014 +0400 @@ -41,7 +41,7 @@ static ngx_command_t ngx_mail_core_comm NULL }, { ngx_string("listen"), - NGX_MAIL_SRV_CONF|NGX_CONF_TAKE12, + NGX_MAIL_SRV_CONF|NGX_CONF_1MORE, ngx_mail_core_listen, NGX_MAIL_SRV_CONF_OFFSET, 0, From ml at kernelobjects.org Wed Aug 20 10:44:08 2014 From: ml at kernelobjects.org (Mario) Date: Wed, 20 Aug 2014 12:44:08 +0200 Subject: Compatibilitypatch for the module ngx_http_secure_link_module Message-ID: <0cf07a1a49482a2809970d9f8103fb3c@marioli.de> Hi nginx team, as I replaced some lighttpd service by nginx (without touching the web application) I had to apply some changes to the source of the ngx_http_secure_link_module. Some of the changes I found already as a patch in a forum but I had to extend this to have a 100% compatibility to the lighttpd mod_secdownload. Here are the differences in short: - lighttpd/secdownload expects the hash to be encoded as hexdecimal of the md5 instead of base64 - lighttpd/secdownload adds the timestamp hexdecimal encoded es well - instead of decimal - lighttpd/secdownload expects the timestamp to be the create time and calculates a delta itself My nginx config is basically like this (relevant part only): # 1st match: hash(hex), 2nd match: time(hex), 3rd match: real location location ~* "^/([a-fA-F0-9]{32,32})/([a-fA-f0-9]{8,8})/(.*)$" { ## this is the key compared to 'secdownload.secret' set $sec_key 'my-secret-key'; ## this is the location of hash and time; ## combinded with this context it equals ## secdownload.uri-prefix = '/' and secdownload.timeout = 600 ## my patch introduced a hex time when the timestamp starts with 0x secure_link $1,0x$2+600; ## this is default with lighttpd/secdownload (according to its documentation) secure_link_md5 $sec_key/$3$2; if ($secure_link = "") { # not valid return 403; } if ($secure_link = "0") { # expired return 410; } proxy_pass http://127.0.0.1:8080/$3; proxy_set_header Host mylocalservice.intra; } Does this patch make sense to your project? This would be great because otherwise I need to patch every version myself befor I can use it. And maybe there are other usecases where lighttpd needs to be replaced ;-) Here is the patch I actually use: ----- START PATCH ----- --- ngx_http_secure_link_module.c.old 2014-08-05 13:18:35.000000000 +0200 +++ ngx_http_secure_link_module.c 2014-08-17 12:19:00.770229341 +0200 @@ -101,13 +101,15 @@ static ngx_int_t ngx_http_secure_link_variable(ngx_http_request_t *r, ngx_http_variable_value_t *v, uintptr_t data) { - u_char *p, *last; + u_char *p, *q, *last; ngx_str_t val, hash; - time_t expires; + time_t expires, ttl; ngx_md5_t md5; ngx_http_secure_link_ctx_t *ctx; ngx_http_secure_link_conf_t *conf; u_char hash_buf[16], md5_buf[16]; + ngx_int_t n; + ngx_uint_t i; conf = ngx_http_get_module_loc_conf(r, ngx_http_secure_link_module); @@ -129,16 +131,38 @@ ngx_http_secure_link_variable(ngx_http_r last = val.data + val.len; p = ngx_strlchr(val.data, last, ','); + expires = 0; + ttl = 0; - if (p) { - val.len = p++ - val.data; + if (p) { // expires + val.len = p - val.data; - expires = ngx_atotm(p, last - p); - if (expires <= 0) { + if (last - ++p < 0) { goto not_found; } + q = ngx_strlchr(p, last, '+'); + + if (q) { // ttl + if (last - ++q < 0 || (ttl = ngx_atotm(q, last - q)) < 0 || --q - p <= 0) { + goto not_found; + } + } else { + q = last; + } + + if (p[0] == '0' && p[1] == 'x' && p[2]) { /* hexdecimal time */ + p += 2; /* skip '0x' */ + if((expires = ngx_hextoi(p, q - p)) == NGX_ERROR) { + goto not_found; + } + } else if ((expires = ngx_atotm(p, q - p)) <= 0) { /* decimal time */ + goto not_found; + } + + expires += ttl; + ctx = ngx_pcalloc(r->pool, sizeof(ngx_http_secure_link_ctx_t)); if (ctx == NULL) { return NGX_ERROR; @@ -146,22 +170,26 @@ ngx_http_secure_link_variable(ngx_http_r ngx_http_set_ctx(r, ctx, ngx_http_secure_link_module); - ctx->expires.len = last - p; + ctx->expires.len = q - p; ctx->expires.data = p; } - if (val.len > 24) { - goto not_found; - } - hash.len = 16; hash.data = hash_buf; - if (ngx_decode_base64url(&hash, &val) != NGX_OK) { - goto not_found; - } - - if (hash.len != 16) { + if (val.len == 32) { // hexadecimal md5 + for (i = 0; i < 16; i++) { + n = ngx_hextoi(&val.data[2 * i], 2); + if (n == NGX_ERROR) { + goto not_found; + } + hash.data[i] = n; + } + } else if (val.len <= 24) { // base64 md5 + if (ngx_decode_base64url(&hash, &val) != NGX_OK || hash.len != 16) { + goto not_found; + } + } else { goto not_found; } ----- END PATCH ----- Thank you for your feedback! /Mario From richard at fussenegger.info Wed Aug 20 17:48:34 2014 From: richard at fussenegger.info (Richard Fussenegger, BSc) Date: Wed, 20 Aug 2014 19:48:34 +0200 Subject: Development environment Message-ID: <53F4DF72.1070902@fussenegger.info> Hi! I'd like to start working on the nginx source. I know my way around in C (although I'm not a crack at all) but I'm totally new to nginx. Do you have some starting points that would help a new developer to understand how things work? - Coding standards (already noticed an 80 character hard limit)? - Preferred OS (FreeBSD, or is Debian good enough)? - Any other good tips or articles to help one get started? I searched the Internet but wasn't able to find anything that would help, so I decided to turn to the mailing list. I'm sorry if this question is redundant. Thanks in advance Richard From pauloasilva at gmail.com Wed Aug 20 20:13:19 2014 From: pauloasilva at gmail.com (Paulo Silva) Date: Wed, 20 Aug 2014 21:13:19 +0100 Subject: Development environment In-Reply-To: <53F4DF72.1070902@fussenegger.info> References: <53F4DF72.1070902@fussenegger.info> Message-ID: Hi Richard, I am also new to nginx development and the best resource, so far are: * http://www.evanmiller.org/nginx-modules-guide.html * http://antoine.bonavita.free.fr/nginx_mod_dev_en.html I met a guy, Wandenberg Peixoto ( https://github.com/wandenberg) who has been an invaluable help. I called my first module "do_nothing" and it was challenging enough. Although the blog post is written in Portuguese, the source code is documented in english: http://tech.pauloasilva.com/nginx-desenvolvimento-de-modulos/ I have a few months of nginx development experience but I had already fight big challenges with some nginx stuffs. If you have any questions and if you think I can help you, drop me a line by email. On Aug 20, 2014 6:53 PM, "Richard Fussenegger, BSc" < richard at fussenegger.info> wrote: > Hi! > > I'd like to start working on the nginx source. I know my way around in C > (although I'm not a crack at all) but I'm totally new to nginx. Do you have > some starting points that would help a new developer to understand how > things work? > > - Coding standards (already noticed an 80 character hard limit)? > - Preferred OS (FreeBSD, or is Debian good enough)? > - Any other good tips or articles to help one get started? > > I searched the Internet but wasn't able to find anything that would help, > so I decided to turn to the mailing list. I'm sorry if this question is > redundant. > > Thanks in advance > Richard > > _______________________________________________ > nginx-devel mailing list > nginx-devel at nginx.org > http://mailman.nginx.org/mailman/listinfo/nginx-devel > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ru at nginx.com Thu Aug 21 07:34:27 2014 From: ru at nginx.com (Ruslan Ermilov) Date: Thu, 21 Aug 2014 11:34:27 +0400 Subject: Development environment In-Reply-To: <53F4DF72.1070902@fussenegger.info> References: <53F4DF72.1070902@fussenegger.info> Message-ID: <20140821073427.GA55977@lo0.su> On Wed, Aug 20, 2014 at 07:48:34PM +0200, Richard Fussenegger, BSc wrote: > I'd like to start working on the nginx source. I know my way around in C > (although I'm not a crack at all) but I'm totally new to nginx. Do you > have some starting points that would help a new developer to understand > how things work? > > - Coding standards (already noticed an 80 character hard limit)? http://nginx.org/en/docs/contributing_changes.html > - Preferred OS (FreeBSD, or is Debian good enough)? Any supported OS of your choice: http://nginx.org/en/#tested_os_and_platforms > - Any other good tips or articles to help one get started? http://nginx.org/en/links.html http://nginx.org/en/books.html HTH From richard at fussenegger.info Thu Aug 21 11:14:34 2014 From: richard at fussenegger.info (Richard Fussenegger, BSc) Date: Thu, 21 Aug 2014 13:14:34 +0200 Subject: Development environment In-Reply-To: References: <53F4DF72.1070902@fussenegger.info> Message-ID: <53F5D49A.4070601@fussenegger.info> Many thanks to Paulo, Ruslan and oscaretu for all the answers. I'll dig deep and will most certainly open up new threads with questions regarding the features I wish to implement. If someone has anything else of interest to share, don't hesitate. Richard On 8/20/2014 10:13 PM, Paulo Silva wrote: > > Hi Richard, > I am also new to nginx development and the best resource, so far are: > * http://www.evanmiller.org/nginx-modules-guide.html > * http://antoine.bonavita.free.fr/nginx_mod_dev_en.html > > I met a guy, Wandenberg Peixoto ( https://github.com/wandenberg) who > has been an invaluable help. > > I called my first module "do_nothing" and it was challenging enough. > Although the blog post is written in Portuguese, the source code is > documented in english: > http://tech.pauloasilva.com/nginx-desenvolvimento-de-modulos/ > > I have a few months of nginx development experience but I had already > fight big challenges with some nginx stuffs. If you have any questions > and if you think I can help you, drop me a line by email. > > On Aug 20, 2014 6:53 PM, "Richard Fussenegger, BSc" > > wrote: > > Hi! > > I'd like to start working on the nginx source. I know my way > around in C (although I'm not a crack at all) but I'm totally new > to nginx. Do you have some starting points that would help a new > developer to understand how things work? > > - Coding standards (already noticed an 80 character hard limit)? > - Preferred OS (FreeBSD, or is Debian good enough)? > - Any other good tips or articles to help one get started? > > I searched the Internet but wasn't able to find anything that > would help, so I decided to turn to the mailing list. I'm sorry if > this question is redundant. > > Thanks in advance > Richard > > _______________________________________________ > nginx-devel mailing list > nginx-devel at nginx.org > http://mailman.nginx.org/mailman/listinfo/nginx-devel > > > > _______________________________________________ > nginx-devel mailing list > nginx-devel at nginx.org > http://mailman.nginx.org/mailman/listinfo/nginx-devel From maxim at nginx.com Thu Aug 21 11:36:44 2014 From: maxim at nginx.com (Maxim Konovalov) Date: Thu, 21 Aug 2014 15:36:44 +0400 Subject: Development environment In-Reply-To: <53F5D49A.4070601@fussenegger.info> References: <53F4DF72.1070902@fussenegger.info> <53F5D49A.4070601@fussenegger.info> Message-ID: <53F5D9CC.1040201@nginx.com> On 8/21/14 3:14 PM, Richard Fussenegger, BSc wrote: > Many thanks to Paulo, Ruslan and oscaretu for all the answers. I'll > dig deep and will most certainly open up new threads with questions > regarding the features I wish to implement. > > If someone has anything else of interest to share, don't hesitate. > If you are going to write nginx code I'd also suggest to use on a regular basis our test suit that you can find here: http://hg.nginx.org/ -- Maxim Konovalov http://nginx.com From steven.hartland at multiplay.co.uk Thu Aug 21 23:30:22 2014 From: steven.hartland at multiplay.co.uk (Steven Hartland) Date: Fri, 22 Aug 2014 00:30:22 +0100 Subject: header value null termination? Message-ID: <2AAB3683F40C4B74AAA6A15945ADE8BE@multiplay.co.uk> I'm creating a module in which I needed to set some of the standard headers e.g. Content-Range and Range. Looking through the core source code the standard way to do this seems to be something like this:- r->headers_in.range->value.len = ngx_sprintf(r->headers_in.range->value.data, "bytes %O-%O/%O", range->start, range->end - 1, r->headers_out.content_length_n) - r->headers_in.range->value.data; This appears to write a string to value.data which is not \0 terminated hence when interacting with functions such as ngx_http_range_parse unpredictable behaviour follows as it expects the range header to be null terminated. So the question is:- Should header values be null terminated or should functions such as ngx_http_range_parse be updated to deal with non-null terminated strings? Regards Steve From yingqi.lu at intel.com Fri Aug 22 16:55:26 2014 From: yingqi.lu at intel.com (Lu, Yingqi) Date: Fri, 22 Aug 2014 16:55:26 +0000 Subject: [Patch] SO_REUSEPORT support from master process Message-ID: <9ACD5B67AAC5594CB6268234CF29CF9AA37A2778@ORSMSX113.amr.corp.intel.com> Dear All, The "SO_REUSEPORT support for listen sockets support" patches submitted by Sepherosa Ziehau are posted and discussed in [1] and [2]. Last update on the threads was 09/05/2013 and the patch is not included in the current Nginx code. Reading from the discussion, my understanding is that his patch makes a dedicated listen socket for each of the child process. In order to make sure at any given time there is always a listen socket available, the patch makes the first worker process different/special than the rest. Here, I am proposing a simpler way to enable the SO_REUSEPORT support. It is just to create and configure certain number of listen sockets in the master process with SO_REUSEPORT enabled. All the children processes can inherit. In this case, we do not need to worry about ensuring 1 available listen socket at the run time. The number of the listen sockets to be created is calculated based on the number of active CPU threads. With big system that has more CPU threads (where we have the scalability issue), there are more duplicated listen sockets created to improve the throughput and scalability. With system that has only 8 or less CPU threads, there will be only 1 listen socket. This makes sure duplicated listen sockets only being created when necessary. In case that SO_REUSEPORT is not supported by the OS, it will fall back to the default/original behavior (this is tested on Linux kernel 3.8.8 where SO_REUSEPORT is not supported). This prototype patch has been tested on an Intel modern dual socket platform with a three tier open source web server workload (PHP+Nginx/memcached/MySQL). The web server has 2 IP network interfaces configured for testing. The Linux kernel used for testing is 3.13.9. Data show: Case 1: with single listen statement (Listen 80) specified in the configuration file, there is 46.3% throughout increase. Case 2: with dual listen statements (for example, Listen 192.168.1.1:80 and Listen 192.168.1.2:80), there is 10% throughput increase. Both testing cases keep everything the same except the patch itself to get above result. The reason that Case1 has bigger performance gains is that Case1 by default only has 1 listen socket while Case2 by default already has 2. Please review it and let me know your questions and comments. [1] http://forum.nginx.org/read.php?29,241283,241283 [2] http://forum.nginx.org/read.php?29,241470,241470 1. Software and workloads used in performance tests may have been optimized for performance only on Intel microprocessors. Performance tests, such as SYSmark and MobileMark, are measured using specific computer systems, components, software, operations and functions. Any change to any of those factors may cause the results to vary. You should consult other information and performance tests to assist you in fully evaluating your contemplated purchases, including the performance of that product when combined with other products. -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: so_reuseport.patch Type: application/octet-stream Size: 5257 bytes Desc: so_reuseport.patch URL: From yingqi.lu at intel.com Fri Aug 22 16:59:39 2014 From: yingqi.lu at intel.com (Lu, Yingqi) Date: Fri, 22 Aug 2014 16:59:39 +0000 Subject: [Patch] SO_REUSEPORT support from master process In-Reply-To: <9ACD5B67AAC5594CB6268234CF29CF9AA37A2778@ORSMSX113.amr.corp.intel.com> References: <9ACD5B67AAC5594CB6268234CF29CF9AA37A2778@ORSMSX113.amr.corp.intel.com> Message-ID: <9ACD5B67AAC5594CB6268234CF29CF9AA37A279C@ORSMSX113.amr.corp.intel.com> Dear All, The "SO_REUSEPORT support for listen sockets support" patches submitted by Sepherosa Ziehau are posted and discussed in [1] and [2]. Last update on the threads was 09/05/2013 and the patch is not included in the current Nginx code. Reading from the discussion, my understanding is that his patch makes a dedicated listen socket for each of the child process. In order to make sure at any given time there is always a listen socket available, the patch makes the first worker process different/special than the rest. Here, I am proposing a simpler way to enable the SO_REUSEPORT support. It is just to create and configure certain number of listen sockets in the master process with SO_REUSEPORT enabled. All the children processes can inherit. In this case, we do not need to worry about ensuring 1 available listen socket at the run time. The number of the listen sockets to be created is calculated based on the number of active CPU threads. With big system that has more CPU threads (where we have the scalability issue), there are more duplicated listen sockets created to improve the throughput and scalability. With system that has only 8 or less CPU threads, there will be only 1 listen socket. This makes sure duplicated listen sockets only being created when necessary. In case that SO_REUSEPORT is not supported by the OS, it will fall back to the default/original behavior (this is tested on Linux kernel 3.8.8 where SO_REUSEPORT is not supported). This prototype patch has been tested on an Intel modern dual socket platform with a three tier open source web server workload (PHP+Nginx/memcached/MySQL). The web server has 2 IP network interfaces configured for testing. The Linux kernel used for testing is 3.13.9. Data show: Case 1: with single listen statement (Listen 80) specified in the configuration file, there is 46.3% throughout increase. Case 2: with dual listen statements (for example, Listen 192.168.1.1:80 and Listen 192.168.1.2:80), there is 10% throughput increase. Both testing cases keep everything the same except the patch itself to get above result. The reason that Case1 has bigger performance gains is that Case1 by default only has 1 listen socket while Case2 by default already has 2. Please review it and let me know your questions and comments. Thanks very much for your time reviewing the patch. Thanks, Yingqi Lu [1] http://forum.nginx.org/read.php?29,241283,241283 [2] http://forum.nginx.org/read.php?29,241470,241470 1. Software and workloads used in performance tests may have been optimized for performance only on Intel microprocessors. Performance tests, such as SYSmark and MobileMark, are measured using specific computer systems, components, software, operations and functions. Any change to any of those factors may cause the results to vary. You should consult other information and performance tests to assist you in fully evaluating your contemplated purchases, including the performance of that product when combined with other products. -------------- next part -------------- An HTML attachment was scrubbed... URL: From yingqi.lu at intel.com Fri Aug 22 17:00:01 2014 From: yingqi.lu at intel.com (Lu, Yingqi) Date: Fri, 22 Aug 2014 17:00:01 +0000 Subject: Recall: [Patch] SO_REUSEPORT support from master process Message-ID: <9ACD5B67AAC5594CB6268234CF29CF9AA37A27A5@ORSMSX113.amr.corp.intel.com> Lu, Yingqi would like to recall the message, "[Patch] SO_REUSEPORT support from master process". From yingqi.lu at intel.com Fri Aug 22 17:00:11 2014 From: yingqi.lu at intel.com (Lu, Yingqi) Date: Fri, 22 Aug 2014 17:00:11 +0000 Subject: Recall: [Patch] SO_REUSEPORT support from master process Message-ID: <9ACD5B67AAC5594CB6268234CF29CF9AA37A27AF@ORSMSX113.amr.corp.intel.com> Lu, Yingqi would like to recall the message, "[Patch] SO_REUSEPORT support from master process". From yingqi.lu at intel.com Fri Aug 22 17:00:41 2014 From: yingqi.lu at intel.com (Lu, Yingqi) Date: Fri, 22 Aug 2014 17:00:41 +0000 Subject: [Patch] SO_REUSEPORT support from master process Message-ID: <9ACD5B67AAC5594CB6268234CF29CF9AA37A27D2@ORSMSX113.amr.corp.intel.com> Dear All, The "SO_REUSEPORT support for listen sockets support" patches submitted by Sepherosa Ziehau are posted and discussed in [1] and [2]. Last update on the threads was 09/05/2013 and the patch is not included in the current Nginx code. Reading from the discussion, my understanding is that his patch makes a dedicated listen socket for each of the child process. In order to make sure at any given time there is always a listen socket available, the patch makes the first worker process different/special than the rest. Here, I am proposing a simpler way to enable the SO_REUSEPORT support. It is just to create and configure certain number of listen sockets in the master process with SO_REUSEPORT enabled. All the children processes can inherit. In this case, we do not need to worry about ensuring 1 available listen socket at the run time. The number of the listen sockets to be created is calculated based on the number of active CPU threads. With big system that has more CPU threads (where we have the scalability issue), there are more duplicated listen sockets created to improve the throughput and scalability. With system that has only 8 or less CPU threads, there will be only 1 listen socket. This makes sure duplicated listen sockets only being created when necessary. In case that SO_REUSEPORT is not supported by the OS, it will fall back to the default/original behavior (this is tested on Linux kernel 3.8.8 where SO_REUSEPORT is not supported). This prototype patch has been tested on an Intel modern dual socket platform with a three tier open source web server workload (PHP+Nginx/memcached/MySQL). The web server has 2 IP network interfaces configured for testing. The Linux kernel used for testing is 3.13.9. Data show: Case 1: with single listen statement (Listen 80) specified in the configuration file, there is 46.3% throughout increase. Case 2: with dual listen statements (for example, Listen 192.168.1.1:80 and Listen 192.168.1.2:80), there is 10% throughput increase. Both testing cases keep everything the same except the patch itself to get above result. The reason that Case1 has bigger performance gains is that Case1 by default only has 1 listen socket while Case2 by default already has 2. Please review it and let me know your questions and comments. Thanks very much for your time reviewing the patch. Thanks, Yingqi Lu [1] http://forum.nginx.org/read.php?29,241283,241283 [2] http://forum.nginx.org/read.php?29,241470,241470 1. Software and workloads used in performance tests may have been optimized for performance only on Intel microprocessors. Performance tests, such as SYSmark and MobileMark, are measured using specific computer systems, components, software, operations and functions. Any change to any of those factors may cause the results to vary. You should consult other information and performance tests to assist you in fully evaluating your contemplated purchases, including the performance of that product when combined with other products. -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: so_reuseport.patch Type: application/octet-stream Size: 5257 bytes Desc: so_reuseport.patch URL: From kpariani at zimbra.com Fri Aug 22 21:13:27 2014 From: kpariani at zimbra.com (Kunal Pariani) Date: Fri, 22 Aug 2014 16:13:27 -0500 (CDT) Subject: [PATCH] SSL support for the mail proxy module In-Reply-To: <1209872235.1254538.1408741993218.JavaMail.zimbra@zimbra.com> References: <1399448073.850051.1408489095173.JavaMail.zimbra@zimbra.com> Message-ID: <1971574953.1254572.1408742007421.JavaMail.zimbra@zimbra.com> Any comments on this yet ? Thanks -Kunal From: "Kunal Pariani" To: "nginx-devel" Sent: Tuesday, August 19, 2014 3:58:15 PM Subject: [PATCH] SSL support for the mail proxy module # HG changeset patch # User Kunal Pariani # Date 1408485440 25200 # Tue Aug 19 14:57:20 2014 -0700 # Node ID 7858c2a9ac0e83aa779197fc028b4d078612e7e8 # Parent f25ab24517bb5e45b1b7fa1a1502b907f2cff213 SSL support for the mail proxy diff -r f25ab24517bb -r 7858c2a9ac0e src/mail/ngx_mail_proxy_module.c --- a/src/mail/ngx_mail_proxy_module.c Mon Aug 04 16:26:30 2014 -0700 +++ b/src/mail/ngx_mail_proxy_module.c Tue Aug 19 14:57:20 2014 -0700 @@ -18,6 +18,8 @@ ngx_flag_t xclient; size_t buffer_size; ngx_msec_t timeout; + ngx_flag_t proxy_ssl; + ngx_ssl_t *ssl; } ngx_mail_proxy_conf_t; @@ -35,7 +37,13 @@ static void *ngx_mail_proxy_create_conf(ngx_conf_t *cf); static char *ngx_mail_proxy_merge_conf(ngx_conf_t *cf, void *parent, void *child); - +#if (NGX_MAIL_SSL) +static char *ngx_mail_proxy_ssl(ngx_conf_t *cf, ngx_command_t *cmd, + void *conf); +static void ngx_mail_proxy_ssl_init_connection(ngx_mail_session_t *s, + ngx_connection_t *c); +static void ngx_mail_proxy_ssl_handshake(ngx_connection_t *c); +#endif static ngx_command_t ngx_mail_proxy_commands[] = { @@ -74,6 +82,16 @@ offsetof(ngx_mail_proxy_conf_t, xclient), NULL }, +#if (NGX_MAIL_SSL) + { ngx_string("proxy_ssl"), + NGX_MAIL_MAIN_CONF|NGX_MAIL_SRV_CONF|NGX_CONF_FLAG, + ngx_mail_proxy_ssl, + NGX_MAIL_SRV_CONF_OFFSET, + offsetof(ngx_mail_proxy_conf_t, proxy_ssl), + NULL }, + +#endif + ngx_null_command }; @@ -174,6 +192,15 @@ s->out.len = 0; +#if (NGX_MAIL_SSL) + + if (pcf->proxy_ssl && p->upstream.connection->ssl == NULL) { + ngx_mail_proxy_ssl_init_connection(s, p->upstream.connection); + return; + } + +#endif + switch (s->protocol) { case NGX_MAIL_POP3_PROTOCOL: @@ -1092,6 +1119,13 @@ "close mail proxy connection: %d", s->proxy->upstream.connection->fd); +#if (NGX_MAIL_SSL) + if (s->proxy->upstream.connection->ssl) { + s->proxy->upstream.connection->ssl->no_wait_shutdown = 1; + ngx_ssl_shutdown(s->proxy->upstream.connection); + } +#endif + ngx_close_connection(s->proxy->upstream.connection); } @@ -1114,6 +1148,8 @@ pcf->xclient = NGX_CONF_UNSET; pcf->buffer_size = NGX_CONF_UNSET_SIZE; pcf->timeout = NGX_CONF_UNSET_MSEC; + pcf->proxy_ssl = NGX_CONF_UNSET; + pcf->ssl = NGX_CONF_UNSET_PTR; return pcf; } @@ -1131,6 +1167,118 @@ ngx_conf_merge_size_value(conf->buffer_size, prev->buffer_size, (size_t) ngx_pagesize); ngx_conf_merge_msec_value(conf->timeout, prev->timeout, 24 * 60 * 60000); + ngx_conf_merge_value(conf->proxy_ssl, prev->proxy_ssl, 0); + ngx_conf_merge_ptr_value(conf->ssl, prev->ssl, NULL); return NGX_CONF_OK; } + +#if (NGX_MAIL_SSL) + +static char * +ngx_mail_proxy_ssl(ngx_conf_t *cf, ngx_command_t *cmd, void *conf) { + + ngx_pool_cleanup_t *cln; + char *rc; + ngx_mail_proxy_conf_t *pcf; + + rc = ngx_conf_set_flag_slot(cf, cmd, conf); + if (rc != NGX_CONF_OK) { + return rc; + } + + pcf = (ngx_mail_proxy_conf_t *)conf; + + if (!pcf->proxy_ssl) { + return NGX_CONF_OK; + } + + pcf->ssl = ngx_pcalloc(cf->pool, sizeof(ngx_ssl_t)); + + if (pcf->ssl == NULL) { + return NGX_CONF_ERROR; + } + + pcf->ssl->log = cf->log; + + // don't support SSLv2 anymore + if (ngx_ssl_create(pcf->ssl, NGX_SSL_SSLv3|NGX_SSL_TLSv1, NULL) + != NGX_OK) { + return NGX_CONF_ERROR; + } + + cln = ngx_pool_cleanup_add(cf->pool, 0); + if (cln == NULL) { + return NGX_CONF_ERROR; + } + + cln->handler = ngx_ssl_cleanup_ctx; + cln->data = pcf->ssl; + + return NGX_CONF_OK; +} + +static void +ngx_mail_proxy_ssl_init_connection(ngx_mail_session_t *s, ngx_connection_t *c) +{ + ngx_int_t rc; + ngx_mail_proxy_conf_t *pcf; + + pcf = ngx_mail_get_module_srv_conf(s, ngx_mail_proxy_module); + + if (ngx_ssl_create_connection(pcf->ssl, c, + NGX_SSL_BUFFER|NGX_SSL_CLIENT) + != NGX_OK) + { + ngx_mail_proxy_internal_server_error(s); + return; + } + + s->connection->log->action = "SSL handshaking to upstream"; + + rc = ngx_ssl_handshake(c); + + if (rc == NGX_AGAIN) { + c->ssl->handler = ngx_mail_proxy_ssl_handshake; + return; + } + + ngx_mail_proxy_ssl_handshake(c); +} + + +static void +ngx_mail_proxy_ssl_handshake(ngx_connection_t *c) +{ + ngx_mail_session_t *s; + s = c->data; + + if (c->ssl->handshaked) { + c->write->handler = ngx_mail_proxy_dummy_handler; + switch (s->protocol) { + + case NGX_MAIL_POP3_PROTOCOL: + c->read->handler = ngx_mail_proxy_pop3_handler; + s->mail_state = ngx_pop3_start; + break; + + case NGX_MAIL_IMAP_PROTOCOL: + c->read->handler = ngx_mail_proxy_imap_handler; + s->mail_state = ngx_imap_start; + break; + + default: /* NGX_MAIL_SMTP_PROTOCOL */ + c->read->handler = ngx_mail_proxy_smtp_handler; + s->mail_state = ngx_smtp_start; + break; + } + + /* server might have sent the initial welcome msg */ + c->read->handler(c->read); + } else { + /* when handshake fails, we should close the session */ + ngx_mail_proxy_upstream_error(s); + } +} + +#endif Thanks -Kunal _______________________________________________ nginx-devel mailing list nginx-devel at nginx.org http://mailman.nginx.org/mailman/listinfo/nginx-devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From kpariani at zimbra.com Fri Aug 22 21:50:23 2014 From: kpariani at zimbra.com (Kunal Pariani) Date: Fri, 22 Aug 2014 16:50:23 -0500 (CDT) Subject: SASL support for mail proxy in NGINX Message-ID: <639768787.1257597.1408744223634.JavaMail.zimbra@zimbra.com> Hello, Wanted to know if there's a plan to add SASL support to Nginx anytime soon ? Zimbra has patches that add SASL support for POP3 & IMAP. Would you be interested in that as a contribution from us ? We plan on adding SASL support to SMTP as well unless you guys have plan to do that already ? Thanks -Kunal -------------- next part -------------- An HTML attachment was scrubbed... URL: From vbart at nginx.com Mon Aug 25 12:09:28 2014 From: vbart at nginx.com (Valentin Bartenev) Date: Mon, 25 Aug 2014 12:09:28 +0000 Subject: [nginx] Sub filter: fixed matching for a single character. Message-ID: details: http://hg.nginx.org/nginx/rev/5322be87fc02 branches: changeset: 5810:5322be87fc02 user: Valentin Bartenev date: Mon Aug 25 16:08:55 2014 +0400 description: Sub filter: fixed matching for a single character. diffstat: src/http/modules/ngx_http_sub_filter_module.c | 8 ++++++++ 1 files changed, 8 insertions(+), 0 deletions(-) diffs (18 lines): diff -r bb26f7ceaaf1 -r 5322be87fc02 src/http/modules/ngx_http_sub_filter_module.c --- a/src/http/modules/ngx_http_sub_filter_module.c Wed Aug 20 13:13:27 2014 +0400 +++ b/src/http/modules/ngx_http_sub_filter_module.c Mon Aug 25 16:08:55 2014 +0400 @@ -546,6 +546,14 @@ ngx_http_sub_parse(ngx_http_request_t *r for ( ;; ) { if (ch == match) { + + if (ctx->match.len == 1) { + ctx->pos = p + 1; + ctx->copy_end = p; + + return NGX_OK; + } + copy_end = p; ctx->looked.data[0] = *p; looked = 1; From donatas.abraitis at gmail.com Mon Aug 25 14:07:12 2014 From: donatas.abraitis at gmail.com (Donatas Abraitis) Date: Mon, 25 Aug 2014 17:07:12 +0300 Subject: return 403 instead of next phase Message-ID: Hey, static ngx_int_t ngx_http_hostprotect_init(ngx_conf_t *cf) { ngx_http_handler_pt *h; ngx_http_core_main_conf_t *cscf; cscf = ngx_http_conf_get_module_main_conf(cf, ngx_http_core_module); h = ngx_array_push(&cscf->phases[NGX_HTTP_ACCESS_PHASE].handlers); if(h == NULL) return NGX_ERROR; *h = ngx_http_hostprotect_handler; return NGX_OK; } static ngx_int_t ngx_http_hostprotect_handler(ngx_http_request_t *r) { ... r->headers_out.status = NGX_HTTP_FORBIDDEN; r->headers_out.content_length_n = sizeof(err_msg); ngx_http_send_header(r); return ngx_http_output_filter(r, &out); } Everything looks fine, but backend (in this case Apache) still receives requests. Any options to drop the request without allowing it to reach backend? -- Donatas -------------- next part -------------- An HTML attachment was scrubbed... URL: From mdounin at mdounin.ru Mon Aug 25 14:32:50 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Mon, 25 Aug 2014 18:32:50 +0400 Subject: header value null termination? In-Reply-To: <2AAB3683F40C4B74AAA6A15945ADE8BE@multiplay.co.uk> References: <2AAB3683F40C4B74AAA6A15945ADE8BE@multiplay.co.uk> Message-ID: <20140825143250.GF1849@mdounin.ru> Hello! On Fri, Aug 22, 2014 at 12:30:22AM +0100, Steven Hartland wrote: > I'm creating a module in which I needed to set some > of the standard headers e.g. Content-Range and Range. > > Looking through the core source code the standard way > to do this seems to be something like this:- > > r->headers_in.range->value.len = > ngx_sprintf(r->headers_in.range->value.data, > "bytes %O-%O/%O", > range->start, range->end - 1, > r->headers_out.content_length_n) > - r->headers_in.range->value.data; > > This appears to write a string to value.data which > is not \0 terminated hence when interacting with > functions such as ngx_http_range_parse unpredictable > behaviour follows as it expects the range header to > be null terminated. > > So the question is:- > > Should header values be null terminated or should > functions such as ngx_http_range_parse be updated > to deal with non-null terminated strings? The answer is: no. In general, strings in nginx are not null terminated, and there is no need to null-terminated them. In some cases though strings are guaranteed to be null-terminated - notably, configuration directive arguments are always null-terminated, as well as input headers. The ngx_http_range_parse() uses an input header from r->headers_in, and it's guaranteed to be null-terminated. The problem is in the "sample" code you've provided though: it tries to modify input headers in r->headers_in. This is wrong thing to do. -- Maxim Dounin http://nginx.org/ From mdounin at mdounin.ru Mon Aug 25 16:09:35 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Mon, 25 Aug 2014 20:09:35 +0400 Subject: return 403 instead of next phase In-Reply-To: References: Message-ID: <20140825160935.GI1849@mdounin.ru> Hello! On Mon, Aug 25, 2014 at 05:07:12PM +0300, Donatas Abraitis wrote: > Hey, > > static ngx_int_t ngx_http_hostprotect_init(ngx_conf_t *cf) > { > ngx_http_handler_pt *h; > ngx_http_core_main_conf_t *cscf; > > cscf = ngx_http_conf_get_module_main_conf(cf, ngx_http_core_module); > h = ngx_array_push(&cscf->phases[NGX_HTTP_ACCESS_PHASE].handlers); > if(h == NULL) > return NGX_ERROR; > > *h = ngx_http_hostprotect_handler; > > return NGX_OK; > } > > static ngx_int_t ngx_http_hostprotect_handler(ngx_http_request_t *r) > { > ... > r->headers_out.status = NGX_HTTP_FORBIDDEN; > r->headers_out.content_length_n = sizeof(err_msg); > ngx_http_send_header(r); > return ngx_http_output_filter(r, &out); > } > > Everything looks fine, but backend (in this case Apache) still receives > requests. Any options to drop the request without allowing it to reach > backend? In your code you return NGX_OK from the access phase handler, and this means that access checks passed. This probably not what you mean to return. You have to return NGX_HTTP_FORBIDDEN instead, without sending anything back - nginx will send an error page for you (either compiled in, or set with error_page directive). -- Maxim Dounin http://nginx.org/ From steven.hartland at multiplay.co.uk Mon Aug 25 18:56:13 2014 From: steven.hartland at multiplay.co.uk (Steven Hartland) Date: Mon, 25 Aug 2014 19:56:13 +0100 Subject: header value null termination? References: <2AAB3683F40C4B74AAA6A15945ADE8BE@multiplay.co.uk> <20140825143250.GF1849@mdounin.ru> Message-ID: <36576796BF8E4676B045066D8BF20D21@multiplay.co.uk> ----- Original Message ----- From: "Maxim Dounin" To: Sent: Monday, August 25, 2014 3:32 PM Subject: Re: header value null termination? > Hello! > > On Fri, Aug 22, 2014 at 12:30:22AM +0100, Steven Hartland wrote: > >> I'm creating a module in which I needed to set some >> of the standard headers e.g. Content-Range and Range. >> >> Looking through the core source code the standard way >> to do this seems to be something like this:- >> >> r->headers_in.range->value.len = >> ngx_sprintf(r->headers_in.range->value.data, >> "bytes %O-%O/%O", >> range->start, range->end - 1, >> r->headers_out.content_length_n) >> - r->headers_in.range->value.data; >> >> This appears to write a string to value.data which >> is not \0 terminated hence when interacting with >> functions such as ngx_http_range_parse unpredictable >> behaviour follows as it expects the range header to >> be null terminated. >> >> So the question is:- >> >> Should header values be null terminated or should >> functions such as ngx_http_range_parse be updated >> to deal with non-null terminated strings? > > The answer is: no. > > In general, strings in nginx are not null terminated, and there is > no need to null-terminated them. In some cases though strings are > guaranteed to be null-terminated - notably, configuration > directive arguments are always null-terminated, as well as input > headers. The ngx_http_range_parse() uses an input header from > r->headers_in, and it's guaranteed to be null-terminated. > > The problem is in the "sample" code you've provided though: it > tries to modify input headers in r->headers_in. This is wrong > thing to do. Thanks for the confirming all input headers should be null terminated, thats good to hear as thats what I did :) The task I'm trying to achieve is quite a strange thing to be doing, essentially I'm sending different headers to subrequests than the original client sent. I seems the only way to achieve this is to alter sr->headers_in as in the sample; unless there's another way to do this? As part of this project I've got a patch which adds the ability to do range requests from 206 responses to ngx_http_range_filter_module, is this something that you would consider upstreaming? Regards Steve From s.martin49 at gmail.com Tue Aug 26 07:40:57 2014 From: s.martin49 at gmail.com (Samuel Martin) Date: Tue, 26 Aug 2014 09:40:57 +0200 Subject: [PATCH 0 of 4 v3] Build-system: Cross-compilation support improvement In-Reply-To: References: Message-ID: Is this a "444 No Response" answer? On Tue, Aug 12, 2014 at 4:03 PM, Samuel Martin wrote: > ping? > > On Sat, Aug 2, 2014 at 1:14 AM, Samuel Martin wrote: >> Hi all, >> >> >> Here is the 3rd round of this short series improving nginx build-system >> support for cross-compilation. >> >> This series fixes a couple of issues found in the previous submission. >> It still tries to be as less intrusive as possible, and intends to make >> easier integration in cross-compilataion frameworks such as Buildroot [1]. >> >> These patches include most of the changes needed for porperly supporting >> the cross-compilation, except the sys_nerr guessing part [2], which cannot >> merged in nginx because it is too unix-oriented and based on fragile >> assumptions >> >> >> [1] http://buildroot.net/ >> [2] https://github.com/tSed/buildroot/blob/nginx-integration/package/nginx/nginx-0005-auto-unix-make-sys_nerr-guessing-cross-friendly.patch >> >> Yours, >> Samuel > > > > -- > Samuel -- Samuel From donatas.abraitis at gmail.com Tue Aug 26 12:09:44 2014 From: donatas.abraitis at gmail.com (Donatas Abraitis) Date: Tue, 26 Aug 2014 15:09:44 +0300 Subject: nginx-devel Digest, Vol 58, Issue 34 In-Reply-To: References: Message-ID: Hey, Maxim, after trying to change to NGX_DONE or NGX_HTTP_FORBIDDEN in static ngx_int_t ngx_http_hostprotect_init(ngx_conf_t *cf), I got this when doing nginx restart: nginx: configuration file /opt/nginx/etc/nginx.conf test failed Thank you. On Tue, Aug 26, 2014 at 3:00 PM, wrote: > Send nginx-devel mailing list submissions to > nginx-devel at nginx.org > > To subscribe or unsubscribe via the World Wide Web, visit > http://mailman.nginx.org/mailman/listinfo/nginx-devel > or, via email, send a message with subject or body 'help' to > nginx-devel-request at nginx.org > > You can reach the person managing the list at > nginx-devel-owner at nginx.org > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of nginx-devel digest..." > > > Today's Topics: > > 1. [nginx] Sub filter: fixed matching for a single character. > (Valentin Bartenev) > 2. return 403 instead of next phase (Donatas Abraitis) > 3. Re: header value null termination? (Maxim Dounin) > 4. Re: return 403 instead of next phase (Maxim Dounin) > 5. Re: header value null termination? (Steven Hartland) > 6. Re: [PATCH 0 of 4 v3] Build-system: Cross-compilation support > improvement (Samuel Martin) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Mon, 25 Aug 2014 12:09:28 +0000 > From: Valentin Bartenev > To: nginx-devel at nginx.org > Subject: [nginx] Sub filter: fixed matching for a single character. > Message-ID: > > Content-Type: text/plain; charset="us-ascii" > > details: http://hg.nginx.org/nginx/rev/5322be87fc02 > branches: > changeset: 5810:5322be87fc02 > user: Valentin Bartenev > date: Mon Aug 25 16:08:55 2014 +0400 > description: > Sub filter: fixed matching for a single character. > > diffstat: > > src/http/modules/ngx_http_sub_filter_module.c | 8 ++++++++ > 1 files changed, 8 insertions(+), 0 deletions(-) > > diffs (18 lines): > > diff -r bb26f7ceaaf1 -r 5322be87fc02 > src/http/modules/ngx_http_sub_filter_module.c > --- a/src/http/modules/ngx_http_sub_filter_module.c Wed Aug 20 > 13:13:27 2014 +0400 > +++ b/src/http/modules/ngx_http_sub_filter_module.c Mon Aug 25 > 16:08:55 2014 +0400 > @@ -546,6 +546,14 @@ ngx_http_sub_parse(ngx_http_request_t *r > > for ( ;; ) { > if (ch == match) { > + > + if (ctx->match.len == 1) { > + ctx->pos = p + 1; > + ctx->copy_end = p; > + > + return NGX_OK; > + } > + > copy_end = p; > ctx->looked.data[0] = *p; > looked = 1; > > > > ------------------------------ > > Message: 2 > Date: Mon, 25 Aug 2014 17:07:12 +0300 > From: Donatas Abraitis > To: nginx-devel at nginx.org > Subject: return 403 instead of next phase > Message-ID: > JY-EweOBKUnjK2DpgFJwdn8dw at mail.gmail.com> > Content-Type: text/plain; charset="utf-8" > > Hey, > > static ngx_int_t ngx_http_hostprotect_init(ngx_conf_t *cf) > { > ngx_http_handler_pt *h; > ngx_http_core_main_conf_t *cscf; > > cscf = ngx_http_conf_get_module_main_conf(cf, ngx_http_core_module); > h = ngx_array_push(&cscf->phases[NGX_HTTP_ACCESS_PHASE].handlers); > if(h == NULL) > return NGX_ERROR; > > *h = ngx_http_hostprotect_handler; > > return NGX_OK; > } > > static ngx_int_t ngx_http_hostprotect_handler(ngx_http_request_t *r) > { > ... > r->headers_out.status = NGX_HTTP_FORBIDDEN; > r->headers_out.content_length_n = sizeof(err_msg); > ngx_http_send_header(r); > return ngx_http_output_filter(r, &out); > } > > Everything looks fine, but backend (in this case Apache) still receives > requests. Any options to drop the request without allowing it to reach > backend? > > -- > Donatas > -------------- next part -------------- > An HTML attachment was scrubbed... > URL: < > http://mailman.nginx.org/pipermail/nginx-devel/attachments/20140825/a88a69b4/attachment-0001.html > > > > ------------------------------ > > Message: 3 > Date: Mon, 25 Aug 2014 18:32:50 +0400 > From: Maxim Dounin > To: nginx-devel at nginx.org > Subject: Re: header value null termination? > Message-ID: <20140825143250.GF1849 at mdounin.ru> > Content-Type: text/plain; charset=us-ascii > > Hello! > > On Fri, Aug 22, 2014 at 12:30:22AM +0100, Steven Hartland wrote: > > > I'm creating a module in which I needed to set some > > of the standard headers e.g. Content-Range and Range. > > > > Looking through the core source code the standard way > > to do this seems to be something like this:- > > > > r->headers_in.range->value.len = > > ngx_sprintf(r->headers_in.range->value.data, > > "bytes %O-%O/%O", > > range->start, range->end - 1, > > r->headers_out.content_length_n) > > - r->headers_in.range->value.data; > > > > This appears to write a string to value.data which > > is not \0 terminated hence when interacting with > > functions such as ngx_http_range_parse unpredictable > > behaviour follows as it expects the range header to > > be null terminated. > > > > So the question is:- > > > > Should header values be null terminated or should > > functions such as ngx_http_range_parse be updated > > to deal with non-null terminated strings? > > The answer is: no. > > In general, strings in nginx are not null terminated, and there is > no need to null-terminated them. In some cases though strings are > guaranteed to be null-terminated - notably, configuration > directive arguments are always null-terminated, as well as input > headers. The ngx_http_range_parse() uses an input header from > r->headers_in, and it's guaranteed to be null-terminated. > > The problem is in the "sample" code you've provided though: it > tries to modify input headers in r->headers_in. This is wrong > thing to do. > > -- > Maxim Dounin > http://nginx.org/ > > > > ------------------------------ > > Message: 4 > Date: Mon, 25 Aug 2014 20:09:35 +0400 > From: Maxim Dounin > To: nginx-devel at nginx.org > Subject: Re: return 403 instead of next phase > Message-ID: <20140825160935.GI1849 at mdounin.ru> > Content-Type: text/plain; charset=us-ascii > > Hello! > > On Mon, Aug 25, 2014 at 05:07:12PM +0300, Donatas Abraitis wrote: > > > Hey, > > > > static ngx_int_t ngx_http_hostprotect_init(ngx_conf_t *cf) > > { > > ngx_http_handler_pt *h; > > ngx_http_core_main_conf_t *cscf; > > > > cscf = ngx_http_conf_get_module_main_conf(cf, ngx_http_core_module); > > h = ngx_array_push(&cscf->phases[NGX_HTTP_ACCESS_PHASE].handlers); > > if(h == NULL) > > return NGX_ERROR; > > > > *h = ngx_http_hostprotect_handler; > > > > return NGX_OK; > > } > > > > static ngx_int_t ngx_http_hostprotect_handler(ngx_http_request_t *r) > > { > > ... > > r->headers_out.status = NGX_HTTP_FORBIDDEN; > > r->headers_out.content_length_n = sizeof(err_msg); > > ngx_http_send_header(r); > > return ngx_http_output_filter(r, &out); > > } > > > > Everything looks fine, but backend (in this case Apache) still receives > > requests. Any options to drop the request without allowing it to reach > > backend? > > In your code you return NGX_OK from the access phase handler, and > this means that access checks passed. This probably not what you > mean to return. > > You have to return NGX_HTTP_FORBIDDEN instead, without sending > anything back - nginx will send an error page for you (either > compiled in, or set with error_page directive). > > -- > Maxim Dounin > http://nginx.org/ > > > > ------------------------------ > > Message: 5 > Date: Mon, 25 Aug 2014 19:56:13 +0100 > From: "Steven Hartland" > To: > Subject: Re: header value null termination? > Message-ID: <36576796BF8E4676B045066D8BF20D21 at multiplay.co.uk> > Content-Type: text/plain; format=flowed; charset="iso-8859-1"; > reply-type=original > > > ----- Original Message ----- > From: "Maxim Dounin" > To: > Sent: Monday, August 25, 2014 3:32 PM > Subject: Re: header value null termination? > > > > Hello! > > > > On Fri, Aug 22, 2014 at 12:30:22AM +0100, Steven Hartland wrote: > > > >> I'm creating a module in which I needed to set some > >> of the standard headers e.g. Content-Range and Range. > >> > >> Looking through the core source code the standard way > >> to do this seems to be something like this:- > >> > >> r->headers_in.range->value.len = > >> ngx_sprintf(r->headers_in.range->value.data, > >> "bytes %O-%O/%O", > >> range->start, range->end - 1, > >> r->headers_out.content_length_n) > >> - r->headers_in.range->value.data; > >> > >> This appears to write a string to value.data which > >> is not \0 terminated hence when interacting with > >> functions such as ngx_http_range_parse unpredictable > >> behaviour follows as it expects the range header to > >> be null terminated. > >> > >> So the question is:- > >> > >> Should header values be null terminated or should > >> functions such as ngx_http_range_parse be updated > >> to deal with non-null terminated strings? > > > > The answer is: no. > > > > In general, strings in nginx are not null terminated, and there is > > no need to null-terminated them. In some cases though strings are > > guaranteed to be null-terminated - notably, configuration > > directive arguments are always null-terminated, as well as input > > headers. The ngx_http_range_parse() uses an input header from > > r->headers_in, and it's guaranteed to be null-terminated. > > > > The problem is in the "sample" code you've provided though: it > > tries to modify input headers in r->headers_in. This is wrong > > thing to do. > > Thanks for the confirming all input headers should be null > terminated, thats good to hear as thats what I did :) > > The task I'm trying to achieve is quite a strange thing to be doing, > essentially I'm sending different headers to subrequests than the > original client sent. > > I seems the only way to achieve this is to alter sr->headers_in > as in the sample; unless there's another way to do this? > > As part of this project I've got a patch which adds the ability > to do range requests from 206 responses to > ngx_http_range_filter_module, is this something that you would > consider upstreaming? > > Regards > Steve > > > > ------------------------------ > > Message: 6 > Date: Tue, 26 Aug 2014 09:40:57 +0200 > From: Samuel Martin > To: nginx-devel at nginx.org > Subject: Re: [PATCH 0 of 4 v3] Build-system: Cross-compilation support > improvement > Message-ID: > OKWiSQ-4oFJ+dO7Pqv+scQ4MVqw_2hPw at mail.gmail.com> > Content-Type: text/plain; charset=ISO-8859-1 > > Is this a "444 No Response" answer? > > On Tue, Aug 12, 2014 at 4:03 PM, Samuel Martin > wrote: > > ping? > > > > On Sat, Aug 2, 2014 at 1:14 AM, Samuel Martin > wrote: > >> Hi all, > >> > >> > >> Here is the 3rd round of this short series improving nginx build-system > >> support for cross-compilation. > >> > >> This series fixes a couple of issues found in the previous submission. > >> It still tries to be as less intrusive as possible, and intends to make > >> easier integration in cross-compilataion frameworks such as Buildroot > [1]. > >> > >> These patches include most of the changes needed for porperly supporting > >> the cross-compilation, except the sys_nerr guessing part [2], which > cannot > >> merged in nginx because it is too unix-oriented and based on fragile > >> assumptions > >> > >> > >> [1] http://buildroot.net/ > >> [2] > https://github.com/tSed/buildroot/blob/nginx-integration/package/nginx/nginx-0005-auto-unix-make-sys_nerr-guessing-cross-friendly.patch > >> > >> Yours, > >> Samuel > > > > > > > > -- > > Samuel > > > > -- > Samuel > > > > ------------------------------ > > _______________________________________________ > nginx-devel mailing list > nginx-devel at nginx.org > http://mailman.nginx.org/mailman/listinfo/nginx-devel > > End of nginx-devel Digest, Vol 58, Issue 34 > ******************************************* > -- Donatas -------------- next part -------------- An HTML attachment was scrubbed... URL: From mdounin at mdounin.ru Tue Aug 26 12:20:10 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Tue, 26 Aug 2014 16:20:10 +0400 Subject: nginx-devel Digest, Vol 58, Issue 34 In-Reply-To: References: Message-ID: <20140826122010.GT1849@mdounin.ru> Hello! On Tue, Aug 26, 2014 at 03:09:44PM +0300, Donatas Abraitis wrote: > Hey, Maxim, > > after trying to change to NGX_DONE or NGX_HTTP_FORBIDDEN in static > ngx_int_t ngx_http_hostprotect_init(ngx_conf_t *cf), I got this when doing > nginx restart: You have to change the access phase handler code (that is, the ngx_http_hostprotect_handler() function), not init handler of your module. A side note: please make sure to never reply to mailing list digests. It breaks threads and looses context. Also, top-posting isn't a good idea. Thank you. -- Maxim Dounin http://nginx.org/ From ru at nginx.com Tue Aug 26 13:38:03 2014 From: ru at nginx.com (Ruslan Ermilov) Date: Tue, 26 Aug 2014 13:38:03 +0000 Subject: [nginx] Stub status: corrected the "stub_status" directive. Message-ID: details: http://hg.nginx.org/nginx/rev/f5b612019042 branches: changeset: 5811:f5b612019042 user: Ruslan Ermilov date: Tue Aug 26 17:35:23 2014 +0400 description: Stub status: corrected the "stub_status" directive. The "stub_status" directive does not require an argument. diffstat: src/http/modules/ngx_http_stub_status_module.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diffs (12 lines): diff -r 5322be87fc02 -r f5b612019042 src/http/modules/ngx_http_stub_status_module.c --- a/src/http/modules/ngx_http_stub_status_module.c Mon Aug 25 16:08:55 2014 +0400 +++ b/src/http/modules/ngx_http_stub_status_module.c Tue Aug 26 17:35:23 2014 +0400 @@ -21,7 +21,7 @@ static char *ngx_http_set_stub_status(ng static ngx_command_t ngx_http_status_commands[] = { { ngx_string("stub_status"), - NGX_HTTP_SRV_CONF|NGX_HTTP_LOC_CONF|NGX_CONF_FLAG, + NGX_HTTP_SRV_CONF|NGX_HTTP_LOC_CONF|NGX_CONF_NOARGS|NGX_CONF_TAKE1, ngx_http_set_stub_status, 0, 0, From donatas.abraitis at gmail.com Tue Aug 26 17:41:51 2014 From: donatas.abraitis at gmail.com (Donatas Abraitis) Date: Tue, 26 Aug 2014 20:41:51 +0300 Subject: nginx-devel Digest, Vol 58, Issue 35 In-Reply-To: References: Message-ID: Everything fine Maxim, thank you. Solved this by "return NGX_HTTP_FORBIDDEN", instead of "return ngx_http_output_filter()". On Tue, Aug 26, 2014 at 3:09 PM, wrote: > Send nginx-devel mailing list submissions to > nginx-devel at nginx.org > > To subscribe or unsubscribe via the World Wide Web, visit > http://mailman.nginx.org/mailman/listinfo/nginx-devel > or, via email, send a message with subject or body 'help' to > nginx-devel-request at nginx.org > > You can reach the person managing the list at > nginx-devel-owner at nginx.org > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of nginx-devel digest..." > > > Today's Topics: > > 1. Re: nginx-devel Digest, Vol 58, Issue 34 (Donatas Abraitis) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Tue, 26 Aug 2014 15:09:44 +0300 > From: Donatas Abraitis > To: nginx-devel at nginx.org > Subject: Re: nginx-devel Digest, Vol 58, Issue 34 > Message-ID: > BVDWCBA6mJFtN1JDcjzvA at mail.gmail.com> > Content-Type: text/plain; charset="utf-8" > > Hey, Maxim, > > after trying to change to NGX_DONE or NGX_HTTP_FORBIDDEN in static > ngx_int_t ngx_http_hostprotect_init(ngx_conf_t *cf), I got this when doing > nginx restart: > > nginx: configuration file /opt/nginx/etc/nginx.conf test failed > > Thank you. > > > On Tue, Aug 26, 2014 at 3:00 PM, wrote: > > > Send nginx-devel mailing list submissions to > > nginx-devel at nginx.org > > > > To subscribe or unsubscribe via the World Wide Web, visit > > http://mailman.nginx.org/mailman/listinfo/nginx-devel > > or, via email, send a message with subject or body 'help' to > > nginx-devel-request at nginx.org > > > > You can reach the person managing the list at > > nginx-devel-owner at nginx.org > > > > When replying, please edit your Subject line so it is more specific > > than "Re: Contents of nginx-devel digest..." > > > > > > Today's Topics: > > > > 1. [nginx] Sub filter: fixed matching for a single character. > > (Valentin Bartenev) > > 2. return 403 instead of next phase (Donatas Abraitis) > > 3. Re: header value null termination? (Maxim Dounin) > > 4. Re: return 403 instead of next phase (Maxim Dounin) > > 5. Re: header value null termination? (Steven Hartland) > > 6. Re: [PATCH 0 of 4 v3] Build-system: Cross-compilation support > > improvement (Samuel Martin) > > > > > > ---------------------------------------------------------------------- > > > > Message: 1 > > Date: Mon, 25 Aug 2014 12:09:28 +0000 > > From: Valentin Bartenev > > To: nginx-devel at nginx.org > > Subject: [nginx] Sub filter: fixed matching for a single character. > > Message-ID: > > > > Content-Type: text/plain; charset="us-ascii" > > > > details: http://hg.nginx.org/nginx/rev/5322be87fc02 > > branches: > > changeset: 5810:5322be87fc02 > > user: Valentin Bartenev > > date: Mon Aug 25 16:08:55 2014 +0400 > > description: > > Sub filter: fixed matching for a single character. > > > > diffstat: > > > > src/http/modules/ngx_http_sub_filter_module.c | 8 ++++++++ > > 1 files changed, 8 insertions(+), 0 deletions(-) > > > > diffs (18 lines): > > > > diff -r bb26f7ceaaf1 -r 5322be87fc02 > > src/http/modules/ngx_http_sub_filter_module.c > > --- a/src/http/modules/ngx_http_sub_filter_module.c Wed Aug 20 > > 13:13:27 2014 +0400 > > +++ b/src/http/modules/ngx_http_sub_filter_module.c Mon Aug 25 > > 16:08:55 2014 +0400 > > @@ -546,6 +546,14 @@ ngx_http_sub_parse(ngx_http_request_t *r > > > > for ( ;; ) { > > if (ch == match) { > > + > > + if (ctx->match.len == 1) { > > + ctx->pos = p + 1; > > + ctx->copy_end = p; > > + > > + return NGX_OK; > > + } > > + > > copy_end = p; > > ctx->looked.data[0] = *p; > > looked = 1; > > > > > > > > ------------------------------ > > > > Message: 2 > > Date: Mon, 25 Aug 2014 17:07:12 +0300 > > From: Donatas Abraitis > > To: nginx-devel at nginx.org > > Subject: return 403 instead of next phase > > Message-ID: > > > JY-EweOBKUnjK2DpgFJwdn8dw at mail.gmail.com> > > Content-Type: text/plain; charset="utf-8" > > > > Hey, > > > > static ngx_int_t ngx_http_hostprotect_init(ngx_conf_t *cf) > > { > > ngx_http_handler_pt *h; > > ngx_http_core_main_conf_t *cscf; > > > > cscf = ngx_http_conf_get_module_main_conf(cf, ngx_http_core_module); > > h = ngx_array_push(&cscf->phases[NGX_HTTP_ACCESS_PHASE].handlers); > > if(h == NULL) > > return NGX_ERROR; > > > > *h = ngx_http_hostprotect_handler; > > > > return NGX_OK; > > } > > > > static ngx_int_t ngx_http_hostprotect_handler(ngx_http_request_t *r) > > { > > ... > > r->headers_out.status = NGX_HTTP_FORBIDDEN; > > r->headers_out.content_length_n = sizeof(err_msg); > > ngx_http_send_header(r); > > return ngx_http_output_filter(r, &out); > > } > > > > Everything looks fine, but backend (in this case Apache) still receives > > requests. Any options to drop the request without allowing it to reach > > backend? > > > > -- > > Donatas > > -------------- next part -------------- > > An HTML attachment was scrubbed... > > URL: < > > > http://mailman.nginx.org/pipermail/nginx-devel/attachments/20140825/a88a69b4/attachment-0001.html > > > > > > > ------------------------------ > > > > Message: 3 > > Date: Mon, 25 Aug 2014 18:32:50 +0400 > > From: Maxim Dounin > > To: nginx-devel at nginx.org > > Subject: Re: header value null termination? > > Message-ID: <20140825143250.GF1849 at mdounin.ru> > > Content-Type: text/plain; charset=us-ascii > > > > Hello! > > > > On Fri, Aug 22, 2014 at 12:30:22AM +0100, Steven Hartland wrote: > > > > > I'm creating a module in which I needed to set some > > > of the standard headers e.g. Content-Range and Range. > > > > > > Looking through the core source code the standard way > > > to do this seems to be something like this:- > > > > > > r->headers_in.range->value.len = > > > ngx_sprintf(r->headers_in.range->value.data, > > > "bytes %O-%O/%O", > > > range->start, range->end - 1, > > > r->headers_out.content_length_n) > > > - r->headers_in.range->value.data; > > > > > > This appears to write a string to value.data which > > > is not \0 terminated hence when interacting with > > > functions such as ngx_http_range_parse unpredictable > > > behaviour follows as it expects the range header to > > > be null terminated. > > > > > > So the question is:- > > > > > > Should header values be null terminated or should > > > functions such as ngx_http_range_parse be updated > > > to deal with non-null terminated strings? > > > > The answer is: no. > > > > In general, strings in nginx are not null terminated, and there is > > no need to null-terminated them. In some cases though strings are > > guaranteed to be null-terminated - notably, configuration > > directive arguments are always null-terminated, as well as input > > headers. The ngx_http_range_parse() uses an input header from > > r->headers_in, and it's guaranteed to be null-terminated. > > > > The problem is in the "sample" code you've provided though: it > > tries to modify input headers in r->headers_in. This is wrong > > thing to do. > > > > -- > > Maxim Dounin > > http://nginx.org/ > > > > > > > > ------------------------------ > > > > Message: 4 > > Date: Mon, 25 Aug 2014 20:09:35 +0400 > > From: Maxim Dounin > > To: nginx-devel at nginx.org > > Subject: Re: return 403 instead of next phase > > Message-ID: <20140825160935.GI1849 at mdounin.ru> > > Content-Type: text/plain; charset=us-ascii > > > > Hello! > > > > On Mon, Aug 25, 2014 at 05:07:12PM +0300, Donatas Abraitis wrote: > > > > > Hey, > > > > > > static ngx_int_t ngx_http_hostprotect_init(ngx_conf_t *cf) > > > { > > > ngx_http_handler_pt *h; > > > ngx_http_core_main_conf_t *cscf; > > > > > > cscf = ngx_http_conf_get_module_main_conf(cf, ngx_http_core_module); > > > h = ngx_array_push(&cscf->phases[NGX_HTTP_ACCESS_PHASE].handlers); > > > if(h == NULL) > > > return NGX_ERROR; > > > > > > *h = ngx_http_hostprotect_handler; > > > > > > return NGX_OK; > > > } > > > > > > static ngx_int_t ngx_http_hostprotect_handler(ngx_http_request_t *r) > > > { > > > ... > > > r->headers_out.status = NGX_HTTP_FORBIDDEN; > > > r->headers_out.content_length_n = sizeof(err_msg); > > > ngx_http_send_header(r); > > > return ngx_http_output_filter(r, &out); > > > } > > > > > > Everything looks fine, but backend (in this case Apache) still receives > > > requests. Any options to drop the request without allowing it to reach > > > backend? > > > > In your code you return NGX_OK from the access phase handler, and > > this means that access checks passed. This probably not what you > > mean to return. > > > > You have to return NGX_HTTP_FORBIDDEN instead, without sending > > anything back - nginx will send an error page for you (either > > compiled in, or set with error_page directive). > > > > -- > > Maxim Dounin > > http://nginx.org/ > > > > > > > > ------------------------------ > > > > Message: 5 > > Date: Mon, 25 Aug 2014 19:56:13 +0100 > > From: "Steven Hartland" > > To: > > Subject: Re: header value null termination? > > Message-ID: <36576796BF8E4676B045066D8BF20D21 at multiplay.co.uk> > > Content-Type: text/plain; format=flowed; charset="iso-8859-1"; > > reply-type=original > > > > > > ----- Original Message ----- > > From: "Maxim Dounin" > > To: > > Sent: Monday, August 25, 2014 3:32 PM > > Subject: Re: header value null termination? > > > > > > > Hello! > > > > > > On Fri, Aug 22, 2014 at 12:30:22AM +0100, Steven Hartland wrote: > > > > > >> I'm creating a module in which I needed to set some > > >> of the standard headers e.g. Content-Range and Range. > > >> > > >> Looking through the core source code the standard way > > >> to do this seems to be something like this:- > > >> > > >> r->headers_in.range->value.len = > > >> ngx_sprintf(r->headers_in.range->value.data, > > >> "bytes %O-%O/%O", > > >> range->start, range->end - 1, > > >> r->headers_out.content_length_n) > > >> - r->headers_in.range->value.data; > > >> > > >> This appears to write a string to value.data which > > >> is not \0 terminated hence when interacting with > > >> functions such as ngx_http_range_parse unpredictable > > >> behaviour follows as it expects the range header to > > >> be null terminated. > > >> > > >> So the question is:- > > >> > > >> Should header values be null terminated or should > > >> functions such as ngx_http_range_parse be updated > > >> to deal with non-null terminated strings? > > > > > > The answer is: no. > > > > > > In general, strings in nginx are not null terminated, and there is > > > no need to null-terminated them. In some cases though strings are > > > guaranteed to be null-terminated - notably, configuration > > > directive arguments are always null-terminated, as well as input > > > headers. The ngx_http_range_parse() uses an input header from > > > r->headers_in, and it's guaranteed to be null-terminated. > > > > > > The problem is in the "sample" code you've provided though: it > > > tries to modify input headers in r->headers_in. This is wrong > > > thing to do. > > > > Thanks for the confirming all input headers should be null > > terminated, thats good to hear as thats what I did :) > > > > The task I'm trying to achieve is quite a strange thing to be doing, > > essentially I'm sending different headers to subrequests than the > > original client sent. > > > > I seems the only way to achieve this is to alter sr->headers_in > > as in the sample; unless there's another way to do this? > > > > As part of this project I've got a patch which adds the ability > > to do range requests from 206 responses to > > ngx_http_range_filter_module, is this something that you would > > consider upstreaming? > > > > Regards > > Steve > > > > > > > > ------------------------------ > > > > Message: 6 > > Date: Tue, 26 Aug 2014 09:40:57 +0200 > > From: Samuel Martin > > To: nginx-devel at nginx.org > > Subject: Re: [PATCH 0 of 4 v3] Build-system: Cross-compilation support > > improvement > > Message-ID: > > > OKWiSQ-4oFJ+dO7Pqv+scQ4MVqw_2hPw at mail.gmail.com> > > Content-Type: text/plain; charset=ISO-8859-1 > > > > Is this a "444 No Response" answer? > > > > On Tue, Aug 12, 2014 at 4:03 PM, Samuel Martin > > wrote: > > > ping? > > > > > > On Sat, Aug 2, 2014 at 1:14 AM, Samuel Martin > > wrote: > > >> Hi all, > > >> > > >> > > >> Here is the 3rd round of this short series improving nginx > build-system > > >> support for cross-compilation. > > >> > > >> This series fixes a couple of issues found in the previous submission. > > >> It still tries to be as less intrusive as possible, and intends to > make > > >> easier integration in cross-compilataion frameworks such as Buildroot > > [1]. > > >> > > >> These patches include most of the changes needed for porperly > supporting > > >> the cross-compilation, except the sys_nerr guessing part [2], which > > cannot > > >> merged in nginx because it is too unix-oriented and based on fragile > > >> assumptions > > >> > > >> > > >> [1] http://buildroot.net/ > > >> [2] > > > https://github.com/tSed/buildroot/blob/nginx-integration/package/nginx/nginx-0005-auto-unix-make-sys_nerr-guessing-cross-friendly.patch > > >> > > >> Yours, > > >> Samuel > > > > > > > > > > > > -- > > > Samuel > > > > > > > > -- > > Samuel > > > > > > > > ------------------------------ > > > > _______________________________________________ > > nginx-devel mailing list > > nginx-devel at nginx.org > > http://mailman.nginx.org/mailman/listinfo/nginx-devel > > > > End of nginx-devel Digest, Vol 58, Issue 34 > > ******************************************* > > > > > > -- > Donatas > -------------- next part -------------- > An HTML attachment was scrubbed... > URL: < > http://mailman.nginx.org/pipermail/nginx-devel/attachments/20140826/907e77bd/attachment.html > > > > ------------------------------ > > _______________________________________________ > nginx-devel mailing list > nginx-devel at nginx.org > http://mailman.nginx.org/mailman/listinfo/nginx-devel > > End of nginx-devel Digest, Vol 58, Issue 35 > ******************************************* > -- Donatas -------------- next part -------------- An HTML attachment was scrubbed... URL: From ru at nginx.com Tue Aug 26 18:18:35 2014 From: ru at nginx.com (Ruslan Ermilov) Date: Tue, 26 Aug 2014 18:18:35 +0000 Subject: [nginx] Resolver: notify all waiting requests on timeout. Message-ID: details: http://hg.nginx.org/nginx/rev/954867a2f0a6 branches: changeset: 5812:954867a2f0a6 user: Ruslan Ermilov date: Tue Aug 19 15:43:26 2014 +0400 description: Resolver: notify all waiting requests on timeout. If a "resolver_timeout" occurs, only the first waiting request was notified. Other requests may hang forever. diffstat: src/core/ngx_resolver.c | 30 +++++++++++++++++++----------- 1 files changed, 19 insertions(+), 11 deletions(-) diffs (68 lines): diff -r f5b612019042 -r 954867a2f0a6 src/core/ngx_resolver.c --- a/src/core/ngx_resolver.c Tue Aug 26 17:35:23 2014 +0400 +++ b/src/core/ngx_resolver.c Tue Aug 19 15:43:26 2014 +0400 @@ -417,7 +417,7 @@ ngx_resolve_name_done(ngx_resolver_ctx_t /* lock name mutex */ - if (ctx->state == NGX_AGAIN || ctx->state == NGX_RESOLVE_TIMEDOUT) { + if (ctx->state == NGX_AGAIN) { hash = ngx_crc32_short(ctx->name.data, ctx->name.len); @@ -664,7 +664,7 @@ ngx_resolve_name_locked(ngx_resolver_t * } ctx->event->handler = ngx_resolver_timeout_handler; - ctx->event->data = ctx; + ctx->event->data = rn; ctx->event->log = r->log; ctx->ident = -1; @@ -857,7 +857,7 @@ ngx_resolve_addr(ngx_resolver_ctx_t *ctx } ctx->event->handler = ngx_resolver_timeout_handler; - ctx->event->data = ctx; + ctx->event->data = rn; ctx->event->log = r->log; ctx->ident = -1; @@ -949,7 +949,7 @@ ngx_resolve_addr_done(ngx_resolver_ctx_t /* lock addr mutex */ - if (ctx->state == NGX_AGAIN || ctx->state == NGX_RESOLVE_TIMEDOUT) { + if (ctx->state == NGX_AGAIN) { switch (ctx->addr.sockaddr->sa_family) { @@ -2790,13 +2790,21 @@ done: static void ngx_resolver_timeout_handler(ngx_event_t *ev) { - ngx_resolver_ctx_t *ctx; - - ctx = ev->data; - - ctx->state = NGX_RESOLVE_TIMEDOUT; - - ctx->handler(ctx); + ngx_resolver_ctx_t *ctx, *next; + ngx_resolver_node_t *rn; + + rn = ev->data; + ctx = rn->waiting; + rn->waiting = NULL; + + do { + ctx->state = NGX_RESOLVE_TIMEDOUT; + next = ctx->next; + + ctx->handler(ctx); + + ctx = next; + } while (ctx); } From sepherosa at gmail.com Wed Aug 27 09:09:25 2014 From: sepherosa at gmail.com (Sepherosa Ziehau) Date: Wed, 27 Aug 2014 17:09:25 +0800 Subject: [Patch] SO_REUSEPORT support from master process In-Reply-To: <9ACD5B67AAC5594CB6268234CF29CF9AA37A2778@ORSMSX113.amr.corp.intel.com> References: <9ACD5B67AAC5594CB6268234CF29CF9AA37A2778@ORSMSX113.amr.corp.intel.com> Message-ID: Does Linux handle un'accepted(2)' sockets inheritance between SO_REUSEPORT sockets, if one of the SO_REUSEPORT socket is closed? It's one of the concern that nginx folks have about SO_REUSEPORT. I am not following Linux, so I am not sure about it at all. That's also why I wanted to keep one SO_REUSEPORT socket opened. BTW, DragonFly had my patch enabled by default in the dports system for a long time. And many systems have the symbol of SO_REUSEPORT and the setsockopt works too, e.g. FreeBSD and probably other BSDs, but on kernel side, SO_REUSEPORT does not work in the fashion like Linux or DragonFly. So even simple run-time test would not be able detect the "new" SO_REUSEPORT support. Best Regards, sephe On Sat, Aug 23, 2014 at 12:55 AM, Lu, Yingqi wrote: > Dear All, > > > > The ?SO_REUSEPORT support for listen sockets support? patches submitted by > Sepherosa Ziehau are posted and discussed in [1] and [2]. Last update on the > threads was 09/05/2013 and the patch is not included in the current Nginx > code. Reading from the discussion, my understanding is that his patch makes > a dedicated listen socket for each of the child process. In order to make > sure at any given time there is always a listen socket available, the patch > makes the first worker process different/special than the rest. > > > > Here, I am proposing a simpler way to enable the SO_REUSEPORT support. It is > just to create and configure certain number of listen sockets in the master > process with SO_REUSEPORT enabled. All the children processes can inherit. > In this case, we do not need to worry about ensuring 1 available listen > socket at the run time. The number of the listen sockets to be created is > calculated based on the number of active CPU threads. With big system that > has more CPU threads (where we have the scalability issue), there are more > duplicated listen sockets created to improve the throughput and scalability. > With system that has only 8 or less CPU threads, there will be only 1 listen > socket. This makes sure duplicated listen sockets only being created when > necessary. In case that SO_REUSEPORT is not supported by the OS, it will > fall back to the default/original behavior (this is tested on Linux kernel > 3.8.8 where SO_REUSEPORT is not supported). > > > > This prototype patch has been tested on an Intel modern dual socket platform > with a three tier open source web server workload > (PHP+Nginx/memcached/MySQL). The web server has 2 IP network interfaces > configured for testing. The Linux kernel used for testing is 3.13.9. Data > show: > > > > Case 1: with single listen statement (Listen 80) specified in the > configuration file, there is 46.3% throughout increase. > > Case 2: with dual listen statements (for example, Listen 192.168.1.1:80 and > Listen 192.168.1.2:80), there is 10% throughput increase. > > > > Both testing cases keep everything the same except the patch itself to get > above result. > > > > The reason that Case1 has bigger performance gains is that Case1 by default > only has 1 listen socket while Case2 by default already has 2. > > > > Please review it and let me know your questions and comments. > > > > [1] http://forum.nginx.org/read.php?29,241283,241283 > > [2] http://forum.nginx.org/read.php?29,241470,241470 > > > > 1. Software and workloads used in performance tests may have been optimized > for performance only on Intel microprocessors. Performance tests, such as > SYSmark and MobileMark, are measured using specific computer systems, > components, software, operations and functions. Any change to any of those > factors may cause the results to vary. You should consult other information > and performance tests to assist you in fully evaluating your contemplated > purchases, including the performance of that product when combined with > other products. > > > _______________________________________________ > nginx-devel mailing list > nginx-devel at nginx.org > http://mailman.nginx.org/mailman/listinfo/nginx-devel -- Tomorrow Will Never Die From yingqi.lu at intel.com Wed Aug 27 16:23:34 2014 From: yingqi.lu at intel.com (Lu, Yingqi) Date: Wed, 27 Aug 2014 16:23:34 +0000 Subject: [Patch] SO_REUSEPORT support from master process In-Reply-To: References: <9ACD5B67AAC5594CB6268234CF29CF9AA37A2778@ORSMSX113.amr.corp.intel.com> Message-ID: <9ACD5B67AAC5594CB6268234CF29CF9AA37A5188@ORSMSX113.amr.corp.intel.com> Hi Sephe, Thanks for your email. The main difference between your patch and this prototype is that we create certain number of duplicated listen sockets in the master process instead of child processes. All the child processes have access to the same set of the duplicated listen sockets. In this case, all the child processes are the same, there is no special child needed. I think this addresses the Nginx folks concern. Also, the number of the duplicated listen sockets is calculated as 1/8 of the number of active CPU threads (when the system has only 8 or less active threads, there is no duplication). It has nothing to do with the number of the child process. We use this way to extend the system CPU scalability (when needed) as well as make sure there will not be a lot of listen sockets open in the kernel. Our prototype also testes the availability of the SO_REUSEPORT feature. When the feature is not available, it will fall back to the default behavior. I test this on Linux Kernel 3.8.8 where the feature is not available. Thanks, Yingqi -----Original Message----- From: nginx-devel-bounces at nginx.org [mailto:nginx-devel-bounces at nginx.org] On Behalf Of Sepherosa Ziehau Sent: Wednesday, August 27, 2014 2:09 AM To: nginx-devel at nginx.org Subject: Re: [Patch] SO_REUSEPORT support from master process Does Linux handle un'accepted(2)' sockets inheritance between SO_REUSEPORT sockets, if one of the SO_REUSEPORT socket is closed? It's one of the concern that nginx folks have about SO_REUSEPORT. I am not following Linux, so I am not sure about it at all. That's also why I wanted to keep one SO_REUSEPORT socket opened. BTW, DragonFly had my patch enabled by default in the dports system for a long time. And many systems have the symbol of SO_REUSEPORT and the setsockopt works too, e.g. FreeBSD and probably other BSDs, but on kernel side, SO_REUSEPORT does not work in the fashion like Linux or DragonFly. So even simple run-time test would not be able detect the "new" SO_REUSEPORT support. Best Regards, sephe On Sat, Aug 23, 2014 at 12:55 AM, Lu, Yingqi wrote: > Dear All, > > > > The ?SO_REUSEPORT support for listen sockets support? patches > submitted by Sepherosa Ziehau are posted and discussed in [1] and [2]. > Last update on the threads was 09/05/2013 and the patch is not > included in the current Nginx code. Reading from the discussion, my > understanding is that his patch makes a dedicated listen socket for > each of the child process. In order to make sure at any given time > there is always a listen socket available, the patch makes the first worker process different/special than the rest. > > > > Here, I am proposing a simpler way to enable the SO_REUSEPORT support. > It is just to create and configure certain number of listen sockets in > the master process with SO_REUSEPORT enabled. All the children processes can inherit. > In this case, we do not need to worry about ensuring 1 available > listen socket at the run time. The number of the listen sockets to be > created is calculated based on the number of active CPU threads. With > big system that has more CPU threads (where we have the scalability > issue), there are more duplicated listen sockets created to improve the throughput and scalability. > With system that has only 8 or less CPU threads, there will be only 1 > listen socket. This makes sure duplicated listen sockets only being > created when necessary. In case that SO_REUSEPORT is not supported by > the OS, it will fall back to the default/original behavior (this is > tested on Linux kernel > 3.8.8 where SO_REUSEPORT is not supported). > > > > This prototype patch has been tested on an Intel modern dual socket > platform with a three tier open source web server workload > (PHP+Nginx/memcached/MySQL). The web server has 2 IP network > interfaces configured for testing. The Linux kernel used for testing > is 3.13.9. Data > show: > > > > Case 1: with single listen statement (Listen 80) specified in the > configuration file, there is 46.3% throughout increase. > > Case 2: with dual listen statements (for example, Listen > 192.168.1.1:80 and Listen 192.168.1.2:80), there is 10% throughput increase. > > > > Both testing cases keep everything the same except the patch itself to > get above result. > > > > The reason that Case1 has bigger performance gains is that Case1 by > default only has 1 listen socket while Case2 by default already has 2. > > > > Please review it and let me know your questions and comments. > > > > [1] http://forum.nginx.org/read.php?29,241283,241283 > > [2] http://forum.nginx.org/read.php?29,241470,241470 > > > > 1. Software and workloads used in performance tests may have been > optimized for performance only on Intel microprocessors. Performance > tests, such as SYSmark and MobileMark, are measured using specific > computer systems, components, software, operations and functions. Any > change to any of those factors may cause the results to vary. You > should consult other information and performance tests to assist you > in fully evaluating your contemplated purchases, including the > performance of that product when combined with other products. > > > _______________________________________________ > nginx-devel mailing list > nginx-devel at nginx.org > http://mailman.nginx.org/mailman/listinfo/nginx-devel -- Tomorrow Will Never Die _______________________________________________ nginx-devel mailing list nginx-devel at nginx.org http://mailman.nginx.org/mailman/listinfo/nginx-devel From yingqi.lu at intel.com Wed Aug 27 16:30:58 2014 From: yingqi.lu at intel.com (Lu, Yingqi) Date: Wed, 27 Aug 2014 16:30:58 +0000 Subject: submitting patch Message-ID: <9ACD5B67AAC5594CB6268234CF29CF9AA37A51E5@ORSMSX113.amr.corp.intel.com> Dear All, I have a general question regarding to patch submission. Last Friday, I submitted a patch to nginx-devel at nginx.org with the patch as an attachment ([Patch] SO_REUSEPORT support from master process). Later, I found the email itself is listed at http://forum.nginx.org/read.php?29,252762,252762#msg-252762. However, I do not see the patch attachment any more. I am not sure if this is normal or I missed some steps during the submission process? Please let me know. Thanks, Yingqi -------------- next part -------------- An HTML attachment was scrubbed... URL: From vbart at nginx.com Wed Aug 27 16:47:08 2014 From: vbart at nginx.com (Valentin Bartenev) Date: Wed, 27 Aug 2014 16:47:08 +0000 Subject: [nginx] SPDY: avoid setting timeout on stream events in ngx_http... Message-ID: details: http://hg.nginx.org/nginx/rev/11790b461a6f branches: changeset: 5813:11790b461a6f user: Valentin Bartenev date: Wed Aug 27 20:44:11 2014 +0400 description: SPDY: avoid setting timeout on stream events in ngx_http_writer(). The SPDY module doesn't expect timers can be set on stream events for reasons other than delaying output. But ngx_http_writer() could add timer on write event if the delayed flag wasn't set and nginx is waiting for AIO completion. That could cause delays in sending response over SPDY when file AIO was used. diffstat: src/http/ngx_http_request.c | 6 ++++++ 1 files changed, 6 insertions(+), 0 deletions(-) diffs (16 lines): diff -r 954867a2f0a6 -r 11790b461a6f src/http/ngx_http_request.c --- a/src/http/ngx_http_request.c Tue Aug 19 15:43:26 2014 +0400 +++ b/src/http/ngx_http_request.c Wed Aug 27 20:44:11 2014 +0400 @@ -2656,6 +2656,12 @@ ngx_http_writer(ngx_http_request_t *r) if (r->buffered || r->postponed || (r == r->main && c->buffered)) { +#if (NGX_HTTP_SPDY) + if (r->spdy_stream) { + return; + } +#endif + if (!wev->delayed) { ngx_add_timer(wev, clcf->send_timeout); } From vbart at nginx.com Wed Aug 27 16:47:13 2014 From: vbart at nginx.com (Valentin Bartenev) Date: Wed, 27 Aug 2014 16:47:13 +0000 Subject: [nginx] SPDY: added a comment about handling stream with the tim... Message-ID: details: http://hg.nginx.org/nginx/rev/a5886df87cbb branches: changeset: 5814:a5886df87cbb user: Valentin Bartenev date: Mon Aug 18 13:23:45 2014 +0400 description: SPDY: added a comment about handling stream with the timer set. diffstat: src/http/ngx_http_spdy_filter_module.c | 5 +++++ 1 files changed, 5 insertions(+), 0 deletions(-) diffs (15 lines): diff -r 11790b461a6f -r a5886df87cbb src/http/ngx_http_spdy_filter_module.c --- a/src/http/ngx_http_spdy_filter_module.c Wed Aug 27 20:44:11 2014 +0400 +++ b/src/http/ngx_http_spdy_filter_module.c Mon Aug 18 13:23:45 2014 +0400 @@ -1141,6 +1141,11 @@ ngx_http_spdy_handle_stream(ngx_http_spd wev = stream->request->connection->write; + /* + * This timer can only be set if the stream was delayed because of rate + * limit. In that case the event should be triggered by the timer. + */ + if (!wev->timer_set) { wev->delayed = 0; From mdounin at mdounin.ru Wed Aug 27 16:50:38 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Wed, 27 Aug 2014 20:50:38 +0400 Subject: submitting patch In-Reply-To: <9ACD5B67AAC5594CB6268234CF29CF9AA37A51E5@ORSMSX113.amr.corp.intel.com> References: <9ACD5B67AAC5594CB6268234CF29CF9AA37A51E5@ORSMSX113.amr.corp.intel.com> Message-ID: <20140827165038.GV1849@mdounin.ru> Hello! On Wed, Aug 27, 2014 at 04:30:58PM +0000, Lu, Yingqi wrote: > Dear All, > > I have a general question regarding to patch submission. > > Last Friday, I submitted a patch to > nginx-devel at nginx.org with the > patch as an attachment ([Patch] SO_REUSEPORT support from master > process). > > Later, I found the email itself is listed at > http://forum.nginx.org/read.php?29,252762,252762#msg-252762. > However, I do not see the patch attachment any more. > > I am not sure if this is normal or I missed some steps during > the submission process? Likely, your HTML postings confused forum interface. While it's not recommended to post HTML emails here, the forum interface can be more or less safely ignored - it's not used by the developers. Mailing list archives are available here: http://mailman.nginx.org/pipermail/nginx-devel/ As for patch submission, the recommended process is outlined here: http://nginx.org/en/docs/contributing_changes.html -- Maxim Dounin http://nginx.org/ From yingqi.lu at intel.com Wed Aug 27 17:20:45 2014 From: yingqi.lu at intel.com (Lu, Yingqi) Date: Wed, 27 Aug 2014 17:20:45 +0000 Subject: submitting patch In-Reply-To: <20140827165038.GV1849@mdounin.ru> References: <9ACD5B67AAC5594CB6268234CF29CF9AA37A51E5@ORSMSX113.amr.corp.intel.com> <20140827165038.GV1849@mdounin.ru> Message-ID: <9ACD5B67AAC5594CB6268234CF29CF9AA37A52C3@ORSMSX113.amr.corp.intel.com> Thanks very much for your suggestion. I will repost. Thanks, Yingqi -----Original Message----- From: nginx-devel-bounces at nginx.org [mailto:nginx-devel-bounces at nginx.org] On Behalf Of Maxim Dounin Sent: Wednesday, August 27, 2014 9:51 AM To: nginx-devel at nginx.org Subject: Re: submitting patch Hello! On Wed, Aug 27, 2014 at 04:30:58PM +0000, Lu, Yingqi wrote: > Dear All, > > I have a general question regarding to patch submission. > > Last Friday, I submitted a patch to > nginx-devel at nginx.org with the patch as > an attachment ([Patch] SO_REUSEPORT support from master process). > > Later, I found the email itself is listed at > http://forum.nginx.org/read.php?29,252762,252762#msg-252762. > However, I do not see the patch attachment any more. > > I am not sure if this is normal or I missed some steps during the > submission process? Likely, your HTML postings confused forum interface. While it's not recommended to post HTML emails here, the forum interface can be more or less safely ignored - it's not used by the developers. Mailing list archives are available here: http://mailman.nginx.org/pipermail/nginx-devel/ As for patch submission, the recommended process is outlined here: http://nginx.org/en/docs/contributing_changes.html -- Maxim Dounin http://nginx.org/ _______________________________________________ nginx-devel mailing list nginx-devel at nginx.org http://mailman.nginx.org/mailman/listinfo/nginx-devel From yingqi.lu at intel.com Wed Aug 27 17:32:41 2014 From: yingqi.lu at intel.com (Lu, Yingqi) Date: Wed, 27 Aug 2014 17:32:41 +0000 Subject: [Patch] SO_REUSEPORT support from master process Message-ID: <9ACD5B67AAC5594CB6268234CF29CF9AA37A52F5@ORSMSX113.amr.corp.intel.com> Dear All, I am resending this patch with plain text instead of HTML format. I will also post the patch at the end of this email. Hope this will be easier for all of you to review. Please let me know if you have trouble viewing the message or the patch itself. This is our first time submitting the patch here. Your feedback and suggestions are highly appreciated. The "SO_REUSEPORT support for listen sockets support" patches submitted by Sepherosa Ziehau are posted and discussed in [1] and [2]. Last update on the threads was 09/05/2013 and the patch is not included in the current Nginx code. Reading from the discussion, my understanding is that his patch makes a dedicated listen socket for each of the child process. In order to make sure at any given time there is always a listen socket available, the patch makes the first worker process different/special than the rest. Here, I am proposing a simpler way to enable the SO_REUSEPORT support. It is just to create and configure certain number of listen sockets in the master process with SO_REUSEPORT enabled. All the children processes can inherit. In this case, we do not need to worry about ensuring 1 available listen socket at the run time. The number of the listen sockets to be created is calculated based on the number of active CPU threads. With big system that has more CPU threads (where we have the scalability issue), there are more duplicated listen sockets created to improve the throughput and scalability. With system that has only 8 or less CPU threads, there will be only 1 listen socket. This makes sure duplicated listen sockets only being created when necessary. In case that SO_REUSEPORT is not supported by the OS, it will fall back to the default/original behavior (this is tested on Linux kernel 3.8.8 where SO_REUSEPORT is not supported). This prototype patch has been tested on an Intel modern dual socket platform with a three tier open source web server workload (PHP+Nginx/memcached/MySQL). The web server has 2 IP network interfaces configured for testing. The Linux kernel used for testing is 3.13.9. Data show: Case 1: with single listen statement (Listen 80) specified in the configuration file, there is 46.3% throughout increase. Case 2: with dual listen statements (for example, Listen 192.168.1.1:80 and Listen 192.168.1.2:80), there is 10% throughput increase. Both testing cases keep everything the same except the patch itself to get above result. The reason that Case1 has bigger performance gains is that Case1 by default only has 1 listen socket while Case2 by default already has 2. Please review it and let me know your questions and comments. Thanks very much for your time reviewing the patch. Thanks, Yingqi Lu [1] http://forum.nginx.org/read.php?29,241283,241283 [2] http://forum.nginx.org/read.php?29,241470,241470 # HG changeset patch # User Yingqi Lu # Date 1408145210 25200 # Fri Aug 15 16:26:50 2014 -0700 # Node ID d9c7259d275dbcae8a0d001ee9703b13312b3263 # Parent 6edcb183e62d610808addebbd18249abb7224a0a These are the patch files for SO_REUSEPORT support. diff -r 6edcb183e62d -r d9c7259d275d ngx_connection.c --- a/ngx_connection.c Fri Aug 15 16:25:32 2014 -0700 +++ b/ngx_connection.c Fri Aug 15 16:26:50 2014 -0700 @@ -304,7 +304,7 @@ ngx_int_t ngx_open_listening_sockets(ngx_cycle_t *cycle) { - int reuseaddr; + int reuseaddr, reuseport; ngx_uint_t i, tries, failed; ngx_err_t err; ngx_log_t *log; @@ -312,6 +312,7 @@ ngx_listening_t *ls; reuseaddr = 1; + reuseport = 1; #if (NGX_SUPPRESS_WARN) failed = 0; #endif @@ -370,6 +371,24 @@ return NGX_ERROR; } + if (so_reuseport_enabled) + { + if (setsockopt(s, SOL_SOCKET, SO_REUSEPORT, + (const void *) &reuseport, sizeof(int)) + == -1) { + ngx_log_error(NGX_LOG_EMERG, log, ngx_socket_errno, + "setsockopt(SO_REUSEPORT) %V failed", + &ls[i].addr_text); + if (ngx_close_socket(s) == -1) { + ngx_log_error(NGX_LOG_EMERG, log, ngx_socket_errno, + ngx_close_socket_n " %V failed", + &ls[i].addr_text); + } + + return NGX_ERROR; + } + } + #if (NGX_HAVE_INET6 && defined IPV6_V6ONLY) if (ls[i].sockaddr->sa_family == AF_INET6) { diff -r 6edcb183e62d -r d9c7259d275d ngx_cycle.c --- a/ngx_cycle.c Fri Aug 15 16:25:32 2014 -0700 +++ b/ngx_cycle.c Fri Aug 15 16:26:50 2014 -0700 @@ -25,7 +25,7 @@ ngx_uint_t ngx_test_config; ngx_uint_t ngx_quiet_mode; - +ngx_uint_t so_reuseport_enabled; #if (NGX_THREADS) ngx_tls_key_t ngx_core_tls_key; #endif @@ -55,6 +55,34 @@ ngx_core_module_t *module; char hostname[NGX_MAXHOSTNAMELEN]; + ngx_uint_t j, num_cores, num_dup_sockets, orig_nelts; + ngx_socket_t temp_s; + int one = 1; + so_reuseport_enabled = 0; + temp_s = ngx_socket(AF_INET, SOCK_STREAM, 0); +#ifndef SO_REUSEPORT +#define SO_REUSEPORT 15 +#endif + if (setsockopt(temp_s, SOL_SOCKET, SO_REUSEPORT, + (const void *) &one, sizeof(int)) == 0) { + so_reuseport_enabled = 1; + } + ngx_close_socket(temp_s); + + if (so_reuseport_enabled) { +#ifdef _SC_NPROCESSORS_ONLN + num_cores = sysconf(_SC_NPROCESSORS_ONLN); +#else + num_cores = 1; +#endif + if (num_cores > 8) { + num_dup_sockets = num_cores/8; + } else { + num_dup_sockets = 1; + } + } else { + num_dup_sockets = 1; + } ngx_timezone_update(); /* force localtime update with a new timezone */ @@ -114,7 +142,7 @@ } - n = old_cycle->paths.nelts ? old_cycle->paths.nelts : 10; + n = old_cycle->paths.nelts ? old_cycle->paths.nelts : 10 * num_dup_sockets; cycle->paths.elts = ngx_pcalloc(pool, n * sizeof(ngx_path_t *)); if (cycle->paths.elts == NULL) { @@ -164,7 +192,7 @@ return NULL; } - n = old_cycle->listening.nelts ? old_cycle->listening.nelts : 10; + n = old_cycle->listening.nelts ? old_cycle->listening.nelts : 10 * num_dup_sockets; cycle->listening.elts = ngx_pcalloc(pool, n * sizeof(ngx_listening_t)); if (cycle->listening.elts == NULL) { @@ -231,7 +259,7 @@ ngx_memzero(&conf, sizeof(ngx_conf_t)); /* STUB: init array ? */ - conf.args = ngx_array_create(pool, 10, sizeof(ngx_str_t)); + conf.args = ngx_array_create(pool, (10 * num_dup_sockets), sizeof(ngx_str_t)); if (conf.args == NULL) { ngx_destroy_pool(pool); return NULL; @@ -575,7 +603,15 @@ #endif } } + orig_nelts = cycle->listening.nelts; + cycle->listening.nelts = cycle->listening.nelts * num_dup_sockets; + ls = cycle->listening.elts; + for (i = 0; i < num_dup_sockets; i++) { + for(j = 0; j < orig_nelts; j++) { + ls[j + i * orig_nelts] = ls[j]; + } + } if (ngx_open_listening_sockets(cycle) != NGX_OK) { goto failed; } @@ -747,7 +783,7 @@ exit(1); } - n = 10; + n = 10 * num_dup_sockets; ngx_old_cycles.elts = ngx_pcalloc(ngx_temp_pool, n * sizeof(ngx_cycle_t *)); if (ngx_old_cycles.elts == NULL) { diff -r 6edcb183e62d -r d9c7259d275d ngx_cycle.h --- a/ngx_cycle.h Fri Aug 15 16:25:32 2014 -0700 +++ b/ngx_cycle.h Fri Aug 15 16:26:50 2014 -0700 @@ -136,6 +136,7 @@ extern ngx_module_t ngx_core_module; extern ngx_uint_t ngx_test_config; extern ngx_uint_t ngx_quiet_mode; +extern ngx_uint_t so_reuseport_enabled; #if (NGX_THREADS) extern ngx_tls_key_t ngx_core_tls_key; #endif 1. Software and workloads used in performance tests may have been optimized for performance only on Intel microprocessors. Performance tests, such as SYSmark and MobileMark, are measured using specific computer systems, components, software, operations and functions. Any change to any of those factors may cause the results to vary. You should consult other information and performance tests to assist you in fully evaluating your contemplated purchases, including the performance of that product when combined with other products. From mdounin at mdounin.ru Wed Aug 27 17:46:14 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Wed, 27 Aug 2014 17:46:14 +0000 Subject: [nginx] Variables: fixed non-indexed access of prefix vars (tick... Message-ID: details: http://hg.nginx.org/nginx/rev/6c99c5f00fc9 branches: changeset: 5815:6c99c5f00fc9 user: Maxim Dounin date: Wed Aug 27 21:38:04 2014 +0400 description: Variables: fixed non-indexed access of prefix vars (ticket #600). Previously, a configuration like location / { ssi on; ssi_types *; set $http_foo "bar"; return 200 '\n'; } resulted in NULL pointer dereference in ngx_http_get_variable() as the variable was explicitly added to the variables hash, but its get_handler wasn't properly set in the hash. Fix is to make sure that get_handler is properly set by ngx_http_variables_init_vars(). diffstat: src/http/ngx_http_variables.c | 7 +++++-- 1 files changed, 5 insertions(+), 2 deletions(-) diffs (24 lines): diff --git a/src/http/ngx_http_variables.c b/src/http/ngx_http_variables.c --- a/src/http/ngx_http_variables.c +++ b/src/http/ngx_http_variables.c @@ -2511,8 +2511,7 @@ ngx_http_variables_init_vars(ngx_conf_t av = key[n].value; - if (av->get_handler - && v[i].name.len == key[n].key.len + if (v[i].name.len == key[n].key.len && ngx_strncmp(v[i].name.data, key[n].key.data, v[i].name.len) == 0) { @@ -2524,6 +2523,10 @@ ngx_http_variables_init_vars(ngx_conf_t av->index = i; + if (av->get_handler == NULL) { + break; + } + goto next; } } From mdounin at mdounin.ru Wed Aug 27 17:46:16 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Wed, 27 Aug 2014 17:46:16 +0000 Subject: [nginx] Variables: updated list of prefixes in ngx_http_rewrite_... Message-ID: details: http://hg.nginx.org/nginx/rev/16a371063d20 branches: changeset: 5816:16a371063d20 user: Maxim Dounin date: Wed Aug 27 21:38:08 2014 +0400 description: Variables: updated list of prefixes in ngx_http_rewrite_set(). diffstat: src/http/modules/ngx_http_rewrite_module.c | 6 +++++- 1 files changed, 5 insertions(+), 1 deletions(-) diffs (16 lines): diff --git a/src/http/modules/ngx_http_rewrite_module.c b/src/http/modules/ngx_http_rewrite_module.c --- a/src/http/modules/ngx_http_rewrite_module.c +++ b/src/http/modules/ngx_http_rewrite_module.c @@ -930,7 +930,11 @@ ngx_http_rewrite_set(ngx_conf_t *cf, ngx if (v->get_handler == NULL && ngx_strncasecmp(value[1].data, (u_char *) "http_", 5) != 0 && ngx_strncasecmp(value[1].data, (u_char *) "sent_http_", 10) != 0 - && ngx_strncasecmp(value[1].data, (u_char *) "upstream_http_", 14) != 0) + && ngx_strncasecmp(value[1].data, (u_char *) "upstream_http_", 14) != 0 + && ngx_strncasecmp(value[1].data, (u_char *) "cookie_", 7) != 0 + && ngx_strncasecmp(value[1].data, (u_char *) "upstream_cookie_", 16) + != 0 + && ngx_strncasecmp(value[1].data, (u_char *) "arg_", 4) != 0) { v->get_handler = ngx_http_rewrite_var; v->data = index; From sepherosa at gmail.com Thu Aug 28 09:27:17 2014 From: sepherosa at gmail.com (Sepherosa Ziehau) Date: Thu, 28 Aug 2014 17:27:17 +0800 Subject: [Patch] SO_REUSEPORT support from master process In-Reply-To: <9ACD5B67AAC5594CB6268234CF29CF9AA37A5188@ORSMSX113.amr.corp.intel.com> References: <9ACD5B67AAC5594CB6268234CF29CF9AA37A2778@ORSMSX113.amr.corp.intel.com> <9ACD5B67AAC5594CB6268234CF29CF9AA37A5188@ORSMSX113.amr.corp.intel.com> Message-ID: On Thu, Aug 28, 2014 at 12:23 AM, Lu, Yingqi wrote: > Hi Sephe, > > Thanks for your email. > > The main difference between your patch and this prototype is that we create certain number of duplicated listen sockets in the master process instead of child processes. All the child processes have access to the same set of the duplicated listen sockets. In this case, all the child processes are the same, there is no special child needed. I think this addresses the Nginx folks concern. > As I have said, the special last listen socket is for inheriting other listen sockets '3-way hand shaken but not accepted' sockets. It is used to support non-disruptive binary upgrading. Kernel side also has work to do, that's why I asked about how linux handles non-disruptive binary upgrading in kernel. e.g. in DragonFly: S1: s1 s2 s3 S2: s4 s5 s6 If S2 is closed, then S1: s1 s2 s3 s4 s5 s6 S1 and S2 are SO_REUSEPORT listen sockets, s1~s6 are '3-way hand shaken but not accepted' sockets. Maybe you should check Linux's SO_REUSEPORT implementation first. > Also, the number of the duplicated listen sockets is calculated as 1/8 of the number of active CPU threads (when the system has only 8 or less active threads, there is no duplication). It has nothing to do with the number of the child process. We use this way to extend the system CPU scalability (when needed) as well as make sure there will not be a lot of listen sockets open in the kernel. > It's OS specific, in DragonFly, we get best performance if worker process uses its own listen socket due to the nature of its network stack design. > Our prototype also testes the availability of the SO_REUSEPORT feature. When the feature is not available, it will fall back to the default behavior. I test this on Linux Kernel 3.8.8 where the feature is not available. As I said, SO_REUSEPORT is defined even in BSD 4.4. And setsockopt(s, SOL_SOCKET, SO_REUSEPORT, &on, sizeof(on)); will return 0 on all BSDs. But FreeBSD and other BSDs SO_REUSEPORT obviously do _not_ work in the way like Linux and DragonFly. Best Regards, sephe From faskiri.devel at gmail.com Thu Aug 28 17:34:13 2014 From: faskiri.devel at gmail.com (Fasih) Date: Thu, 28 Aug 2014 23:04:13 +0530 Subject: Reading request body in PRE_ACCESS phase Message-ID: Hi Guys I am trying to read the request body in pre_access phase. This seems like a regular requirement but I dont seem to find a good way to do this. Since the request body is read asynchronously, I have to do phases++ and core_run_phases myself in the read_completion callback. I also have to set the r->write_event_handler to empty because otherwise a write event calls run_phases which I obviously dont want till I read the body. I am trying to understand what is the correct way to do this since I seem to be hacking my way around this. Best Regards! +Fasih From agentzh at gmail.com Thu Aug 28 20:46:37 2014 From: agentzh at gmail.com (Yichun Zhang (agentzh)) Date: Thu, 28 Aug 2014 13:46:37 -0700 Subject: Reading request body in PRE_ACCESS phase In-Reply-To: References: Message-ID: Hello! On Thu, Aug 28, 2014 at 10:34 AM, Fasih wrote: > I am trying to read the request body in pre_access phase. This seems > like a regular requirement but I dont seem to find a good way to do > this. Since the request body is read asynchronously, I have to do > phases++ and core_run_phases myself in the read_completion callback. I > also have to set the r->write_event_handler to empty because otherwise > a write event calls run_phases which I obviously dont want till I read > the body. I am trying to understand what is the correct way to do this > since I seem to be hacking my way around this. > Incrementing r->phase_handler manually in your "post_read" handler is a bad idea. You should just call ngx_http_core_run_phases() there if it is not a recursive call (when you read the body in a single run). And in your pre-access handler, just return appropriate return values according to the state in your "ctx". If the request body is done, you set a flag in your "ctx" in your "post_read" handler, and your pre-access phase handler checks this flag and decides whether to return NGX_DECLINED to proceed to the next phase handler, or just return NGX_DONE to yield the control back to the nginx event loop. The ngx_form_input module reads the request body in the rewrite phase, which should be similar: https://github.com/calio/form-input-nginx-module Regards, -agentzh From donatas.abraitis at gmail.com Thu Aug 28 21:37:24 2014 From: donatas.abraitis at gmail.com (Donatas Abraitis) Date: Fri, 29 Aug 2014 00:37:24 +0300 Subject: ngx_str_rbtree_lookup() always return NULL Message-ID: Hey, who can explain this function's behavior? Am I using this wrong? It always returns NULL, though I 100% ensured, there should be. Is here any way to list all nodes in the rbtree? static int purge_from_cache(ngx_slab_pool_t *shpool, ngx_rbtree_t *rbtree, ngx_str_t ip, ngx_http_request_t *r) { uint32_t hash; ngx_http_hostprotect_value_node_t *found; hash = ngx_crc32_long(ip.data, ip.len); ngx_log_error(NGX_LOG_ERR, r->connection->log, 0, "%s: Trying to purge %s (%lu)", MODULE_NAME, ip.data, (unsigned long)hash); ngx_shmtx_lock(&shpool->mutex); found = (ngx_http_hostprotect_value_node_t *) ngx_str_rbtree_lookup(rbtree, &ip, hash); ngx_shmtx_unlock(&shpool->mutex); if(found) { ngx_log_error(NGX_LOG_ERR, r->connection->log, 0, "%s: Purging from active cache %s", MODULE_NAME, ip.data); ngx_shmtx_lock(&shpool->mutex); ngx_rbtree_delete(rbtree, found); ngx_slab_free_locked(shpool, found); ngx_shmtx_unlock(&shpool->mutex); return 1; } return 0; } -- Donatas -------------- next part -------------- An HTML attachment was scrubbed... URL: From niq at apache.org Fri Aug 29 00:51:55 2014 From: niq at apache.org (Nick Kew) Date: Fri, 29 Aug 2014 01:51:55 +0100 Subject: nginx tempfiles Message-ID: <1409273515.5553.89.camel@mimir.webthing.com> I have a filter that may want to rewrite request bodies. Stream-editing an nginx chain is straightforward enough (the equivalent output filter works fine). But as I understand it, I need to cater for a case where a request body is in r->request_body->temp_file. Assuming it's there for a reason, I thought the least-bad approach would be to stream-edit it to another tempfile and switch r->request_body->temp_file to that. However, that approach appears to be pushing against nginx's tempfile handling, from the moment I look at ngx_create_temp_file and realise the amount of preparation required to set up a path and file before calling it. Will Bad Things happen if I simplify by using a system call like mkstemp and then just substitute file->fd in nginx's temp_file struct? Or is this approach completely barking up the wrong tree? -- Nick Kew From faskiri.devel at gmail.com Fri Aug 29 07:53:56 2014 From: faskiri.devel at gmail.com (Fasih) Date: Fri, 29 Aug 2014 13:23:56 +0530 Subject: Reading request body in PRE_ACCESS phase In-Reply-To: References: Message-ID: Thanks for the reply. I am also doing this. Basically, have a way to see if it the body was read synchronously or asynchronously. If synchronous, let nginx handle it else we have to do that ourselves [The way your module does it]. But I thought there is a better way to do this. Btw, I think you have to set write_event_handler to empty. Basically, if you dont set it, and there is a write_event (while the body is not read), nginx would call core_run_phases which you werent expecting. On Fri, Aug 29, 2014 at 2:16 AM, Yichun Zhang (agentzh) wrote: > Hello! > > On Thu, Aug 28, 2014 at 10:34 AM, Fasih wrote: >> I am trying to read the request body in pre_access phase. This seems >> like a regular requirement but I dont seem to find a good way to do >> this. Since the request body is read asynchronously, I have to do >> phases++ and core_run_phases myself in the read_completion callback. I >> also have to set the r->write_event_handler to empty because otherwise >> a write event calls run_phases which I obviously dont want till I read >> the body. I am trying to understand what is the correct way to do this >> since I seem to be hacking my way around this. >> > > Incrementing r->phase_handler manually in your "post_read" handler is > a bad idea. You should just call ngx_http_core_run_phases() there if > it is not a recursive call (when you read the body in a single run). > And in your pre-access handler, just return appropriate return values > according to the state in your "ctx". If the request body is done, you > set a flag in your "ctx" in your "post_read" handler, and your > pre-access phase handler checks this flag and decides whether to > return NGX_DECLINED to proceed to the next phase handler, or just > return NGX_DONE to yield the control back to the nginx event loop. > > The ngx_form_input module reads the request body in the rewrite phase, > which should be similar: > > https://github.com/calio/form-input-nginx-module > > Regards, > -agentzh > > _______________________________________________ > nginx-devel mailing list > nginx-devel at nginx.org > http://mailman.nginx.org/mailman/listinfo/nginx-devel From mdounin at mdounin.ru Fri Aug 29 13:26:59 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Fri, 29 Aug 2014 17:26:59 +0400 Subject: nginx tempfiles In-Reply-To: <1409273515.5553.89.camel@mimir.webthing.com> References: <1409273515.5553.89.camel@mimir.webthing.com> Message-ID: <20140829132659.GO1849@mdounin.ru> Hello! On Fri, Aug 29, 2014 at 01:51:55AM +0100, Nick Kew wrote: > I have a filter that may want to rewrite request bodies. > > Stream-editing an nginx chain is straightforward enough > (the equivalent output filter works fine). > But as I understand it, I need to cater for a case where > a request body is in r->request_body->temp_file. > Assuming it's there for a reason, I thought the least-bad > approach would be to stream-edit it to another tempfile > and switch r->request_body->temp_file to that. > > However, that approach appears to be pushing against > nginx's tempfile handling, from the moment I look at > ngx_create_temp_file and realise the amount of preparation > required to set up a path and file before calling it. > > Will Bad Things happen if I simplify by using a system > call like mkstemp and then just substitute file->fd > in nginx's temp_file struct? This will likely break client_body_in_file_only, see http://nginx.org/r/client_body_in_file_only. Not to mention that mkstemp() won't work on win32, and you'll have to unlink files yourself then. In either case, you may want to consider using request body filters instead, as recently discussed here: http://mailman.nginx.org/pipermail/nginx-devel/2014-August/005781.html Reading and then writing temorary files will be suboptimal. And replacing r->request_body->temp_file, even if you'll be able to do it properly, will likely result in your module being broken during further nginx development. -- Maxim Dounin http://nginx.org/ From pluknet at nginx.com Fri Aug 29 14:04:53 2014 From: pluknet at nginx.com (Sergey Kandaurov) Date: Fri, 29 Aug 2014 14:04:53 +0000 Subject: [nginx] Headers filter: "add_header" with "always" parameter (ti... Message-ID: details: http://hg.nginx.org/nginx/rev/74ffe03555d0 branches: changeset: 5817:74ffe03555d0 user: Sergey Kandaurov date: Fri Aug 29 18:00:10 2014 +0400 description: Headers filter: "add_header" with "always" parameter (ticket #98). If specified, the header field is set regardless of the status code. diffstat: src/http/modules/ngx_http_headers_filter_module.c | 54 +++++++++++++++++----- 1 files changed, 41 insertions(+), 13 deletions(-) diffs (108 lines): diff -r 16a371063d20 -r 74ffe03555d0 src/http/modules/ngx_http_headers_filter_module.c --- a/src/http/modules/ngx_http_headers_filter_module.c Wed Aug 27 21:38:08 2014 +0400 +++ b/src/http/modules/ngx_http_headers_filter_module.c Fri Aug 29 18:00:10 2014 +0400 @@ -28,6 +28,7 @@ struct ngx_http_header_val_s { ngx_str_t key; ngx_http_set_header_pt handler; ngx_uint_t offset; + ngx_uint_t always; /* unsigned always:1 */ }; @@ -98,7 +99,7 @@ static ngx_command_t ngx_http_headers_f { ngx_string("add_header"), NGX_HTTP_MAIN_CONF|NGX_HTTP_SRV_CONF|NGX_HTTP_LOC_CONF|NGX_HTTP_LIF_CONF - |NGX_CONF_TAKE2, + |NGX_CONF_TAKE23, ngx_http_headers_add, NGX_HTTP_LOC_CONF_OFFSET, 0, @@ -146,28 +147,38 @@ static ngx_int_t ngx_http_headers_filter(ngx_http_request_t *r) { ngx_str_t value; - ngx_uint_t i; + ngx_uint_t i, safe_status; ngx_http_header_val_t *h; ngx_http_headers_conf_t *conf; conf = ngx_http_get_module_loc_conf(r, ngx_http_headers_filter_module); if ((conf->expires == NGX_HTTP_EXPIRES_OFF && conf->headers == NULL) - || r != r->main - || (r->headers_out.status != NGX_HTTP_OK - && r->headers_out.status != NGX_HTTP_CREATED - && r->headers_out.status != NGX_HTTP_NO_CONTENT - && r->headers_out.status != NGX_HTTP_PARTIAL_CONTENT - && r->headers_out.status != NGX_HTTP_MOVED_PERMANENTLY - && r->headers_out.status != NGX_HTTP_MOVED_TEMPORARILY - && r->headers_out.status != NGX_HTTP_SEE_OTHER - && r->headers_out.status != NGX_HTTP_NOT_MODIFIED - && r->headers_out.status != NGX_HTTP_TEMPORARY_REDIRECT)) + || r != r->main) { return ngx_http_next_header_filter(r); } - if (conf->expires != NGX_HTTP_EXPIRES_OFF) { + switch (r->headers_out.status) { + + case NGX_HTTP_OK: + case NGX_HTTP_CREATED: + case NGX_HTTP_NO_CONTENT: + case NGX_HTTP_PARTIAL_CONTENT: + case NGX_HTTP_MOVED_PERMANENTLY: + case NGX_HTTP_MOVED_TEMPORARILY: + case NGX_HTTP_SEE_OTHER: + case NGX_HTTP_NOT_MODIFIED: + case NGX_HTTP_TEMPORARY_REDIRECT: + safe_status = 1; + break; + + default: + safe_status = 0; + break; + } + + if (conf->expires != NGX_HTTP_EXPIRES_OFF && safe_status) { if (ngx_http_set_expires(r, conf) != NGX_OK) { return NGX_ERROR; } @@ -177,6 +188,10 @@ ngx_http_headers_filter(ngx_http_request h = conf->headers->elts; for (i = 0; i < conf->headers->nelts; i++) { + if (!safe_status && !h[i].always) { + continue; + } + if (ngx_http_complex_value(r, &h[i].value, &value) != NGX_OK) { return NGX_ERROR; } @@ -603,6 +618,7 @@ ngx_http_headers_add(ngx_conf_t *cf, ngx hv->key = value[1]; hv->handler = ngx_http_add_header; hv->offset = 0; + hv->always = 0; set = ngx_http_set_headers; for (i = 0; set[i].name.len; i++) { @@ -631,5 +647,17 @@ ngx_http_headers_add(ngx_conf_t *cf, ngx return NGX_CONF_ERROR; } + if (cf->args->nelts == 3) { + return NGX_CONF_OK; + } + + if (ngx_strcmp(value[3].data, "always") != 0) { + ngx_conf_log_error(NGX_LOG_EMERG, cf, 0, + "invalid parameter \"%V\"", &value[3]); + return NGX_CONF_ERROR; + } + + hv->always = 1; + return NGX_CONF_OK; } From agentzh at gmail.com Fri Aug 29 20:35:28 2014 From: agentzh at gmail.com (Yichun Zhang (agentzh)) Date: Fri, 29 Aug 2014 13:35:28 -0700 Subject: Reading request body in PRE_ACCESS phase In-Reply-To: References: Message-ID: Hello! On Fri, Aug 29, 2014 at 12:53 AM, Fasih wrote: > Btw, I think you have to set write_event_handler to empty. Basically, > if you dont set it, and there is a write_event (while the body is not > read), nginx would call core_run_phases which you werent expecting. > The ngx_http_read_client_request_body function already sets that automatically for us :) Also, even if your phase handler is called prematurely, you can still guard that with the state recorded in your own "ctx". Remember that you need to set a flag in "ctx" in your "post_read" callback? Regards, -agentzh