From mdounin at mdounin.ru Tue Jul 1 00:10:27 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Tue, 1 Jul 2014 04:10:27 +0400 Subject: ngx resolver set custom name server In-Reply-To: References: Message-ID: <20140701001027.GP1849@mdounin.ru> Hello! On Mon, Jun 30, 2014 at 03:43:56PM +0300, Donatas Abraitis wrote: > Hello, > > is it possible to set resolver ( > http://nginx.org/en/docs/http/ngx_http_core_module.html#resolver) writing > own nginx module? > > I want to set custom name servers for querying by gethostbyaddr(). By > default gethostbyaddr() uses /etc/resolv.conf, /etc/hosts. I want to > override this by setting somehow. > > Anyone? Sorry, but I doubt anyone was able to understand your question. At least I wasn't. You may try to clarify what you want to do and what is your question. Just in case, parameters of the "resolver" directive specify DNS servers to use, and it's completely unrelated to system gethostbyaddr(), resolv.conf and /etc/hosts. For an example of how to use resolver from an nginx module take a look at ngx_http_upstream.c and/or ngx_mail_smtp_handler.c. -- Maxim Dounin http://nginx.org/ From mdounin at mdounin.ru Tue Jul 1 01:05:51 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Tue, 1 Jul 2014 05:05:51 +0400 Subject: [PATCH] Core: use uppercase hexadecimal digits for percent-encoding In-Reply-To: References: <177382006b7d7a421688.1403851227@Piotrs-MacBook-Pro.local> <20140627175754.GX1849@mdounin.ru> Message-ID: <20140701010551.GT1849@mdounin.ru> Hello! On Mon, Jun 30, 2014 at 02:01:30PM -0700, Piotr Sikora wrote: > Hey Maxim, > > > I can't say I like this change. I've considered this a while ago, > > and decided to keep it as is. This preserve compatibility with > > what nginx used to do for years. And it also looks like Apache > > does the same. > > > > Any other opinions? > > Compatibility for whom? This change is transparent for decoders > (upper- and lowercase hex digits are equivalent, per RFC) and, as far > as I can tell, it only affects people who try to encode URLs to match > what nginx produces and/or do case-sensitive matching (like > nginx-tests/autoindex.t). So, you've already found at least one place which will be broken by this change. And I would expect more. -- Maxim Dounin http://nginx.org/ From mdounin at mdounin.ru Tue Jul 1 01:13:18 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Tue, 1 Jul 2014 05:13:18 +0400 Subject: [PATCH] Core: use uppercase hexadecimal digits for percent-encoding In-Reply-To: References: <177382006b7d7a421688.1403851227@Piotrs-MacBook-Pro.local> <20140627175754.GX1849@mdounin.ru> Message-ID: <20140701011318.GU1849@mdounin.ru> Hello! On Mon, Jun 30, 2014 at 04:24:58PM -0700, Yichun Zhang (agentzh) wrote: > Hello! > > On Fri, Jun 27, 2014 at 10:57 AM, Maxim Dounin wrote: > > I can't say I like this change. I've considered this a while ago, > > and decided to keep it as is. This preserve compatibility with > > what nginx used to do for years. And it also looks like Apache > > does the same. > > > > Any other opinions? > > > > I agree with Piotr Sikora and hope that nginx uses %DD instead of %dd > for URI escaping. Right now ngx_lua also uses %dd for the consistency > with the nginx core, which has already yielded several complaints from > ngx_lua's user community (the users also pointed me to the RFC). Once > the nginx core switches over to %DD, I can make a similar change to > ngx_lua accordingly :) Thanks, counted. -- Maxim Dounin http://nginx.org/ From mdounin at mdounin.ru Tue Jul 1 01:15:06 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Tue, 1 Jul 2014 05:15:06 +0400 Subject: [PATCH] Core: use uppercase hexadecimal digits for percent-encoding In-Reply-To: <3338179.ENjqyOEFCX@vbart-workstation> References: <177382006b7d7a421688.1403851227@Piotrs-MacBook-Pro.local> <20140627175754.GX1849@mdounin.ru> <3338179.ENjqyOEFCX@vbart-workstation> Message-ID: <20140701011505.GV1849@mdounin.ru> Hello! On Mon, Jun 30, 2014 at 04:41:26PM +0400, Valentin V. Bartenev wrote: > On Friday 27 June 2014 21:57:54 Maxim Dounin wrote: > > Hello! > > > > On Thu, Jun 26, 2014 at 11:40:27PM -0700, Piotr Sikora wrote: > > > > > # HG changeset patch > > > # User Piotr Sikora > > > # Date 1403851163 25200 > > > # Thu Jun 26 23:39:23 2014 -0700 > > > # Node ID 177382006b7d7a421688831d5793b2e417074b48 > > > # Parent 42114bf12da0cf3d428d0e695139f5366cbd0513 > > > Core: use uppercase hexadecimal digits for percent-encoding. > > > > > > RFC3986 says that, for consistency, URI producers and normalizers > > > should use uppercase hexadecimal digits for all percent-encodings. > > > > > > This is also what modern web browsers and other tools use. > > > > > > Using lowercase hexadecimal digits makes it harder to interact with > > > those tools in case when use of the percent-encoded URI is required, > > > for example when $request_uri is part of the cache key. > > > > > > Signed-off-by: Piotr Sikora > > > > > > diff -r 42114bf12da0 -r 177382006b7d src/core/ngx_string.c > > > --- a/src/core/ngx_string.c Mon Jun 16 19:43:25 2014 +0400 > > > +++ b/src/core/ngx_string.c Thu Jun 26 23:39:23 2014 -0700 > > > @@ -1407,7 +1407,7 @@ ngx_escape_uri(u_char *dst, u_char *src, > > > { > > > ngx_uint_t n; > > > uint32_t *escape; > > > - static u_char hex[] = "0123456789abcdef"; > > > + static u_char hex[] = "0123456789ABCDEF"; > > > > > > /* " ", "#", "%", "?", %00-%1F, %7F-%FF */ > > > > I can't say I like this change. I've considered this a while ago, > > and decided to keep it as is. This preserve compatibility with > > what nginx used to do for years. And it also looks like Apache > > does the same. > > > > Any other opinions? > > > > I prefer to fix this instead of keeping it for another few years. > > Uppercase digits also look more distinctly, since all other parts > of path are usually in lowercase. Ruslan? Igor? -- Maxim Dounin http://nginx.org/ From ru at nginx.com Tue Jul 1 05:26:32 2014 From: ru at nginx.com (Ruslan Ermilov) Date: Tue, 1 Jul 2014 09:26:32 +0400 Subject: [PATCH] Core: use uppercase hexadecimal digits for percent-encoding In-Reply-To: <20140701011505.GV1849@mdounin.ru> References: <177382006b7d7a421688.1403851227@Piotrs-MacBook-Pro.local> <20140627175754.GX1849@mdounin.ru> <3338179.ENjqyOEFCX@vbart-workstation> <20140701011505.GV1849@mdounin.ru> Message-ID: <20140701052632.GA5800@lo0.su> On Tue, Jul 01, 2014 at 05:15:06AM +0400, Maxim Dounin wrote: > On Mon, Jun 30, 2014 at 04:41:26PM +0400, Valentin V. Bartenev wrote: > > On Friday 27 June 2014 21:57:54 Maxim Dounin wrote: > > > Hello! > > > > > > On Thu, Jun 26, 2014 at 11:40:27PM -0700, Piotr Sikora wrote: > > > > > > > # HG changeset patch > > > > # User Piotr Sikora > > > > # Date 1403851163 25200 > > > > # Thu Jun 26 23:39:23 2014 -0700 > > > > # Node ID 177382006b7d7a421688831d5793b2e417074b48 > > > > # Parent 42114bf12da0cf3d428d0e695139f5366cbd0513 > > > > Core: use uppercase hexadecimal digits for percent-encoding. > > > > > > > > RFC3986 says that, for consistency, URI producers and normalizers > > > > should use uppercase hexadecimal digits for all percent-encodings. > > > > > > > > This is also what modern web browsers and other tools use. > > > > > > > > Using lowercase hexadecimal digits makes it harder to interact with > > > > those tools in case when use of the percent-encoded URI is required, > > > > for example when $request_uri is part of the cache key. > > > > > > > > Signed-off-by: Piotr Sikora > > > > > > > > diff -r 42114bf12da0 -r 177382006b7d src/core/ngx_string.c > > > > --- a/src/core/ngx_string.c Mon Jun 16 19:43:25 2014 +0400 > > > > +++ b/src/core/ngx_string.c Thu Jun 26 23:39:23 2014 -0700 > > > > @@ -1407,7 +1407,7 @@ ngx_escape_uri(u_char *dst, u_char *src, > > > > { > > > > ngx_uint_t n; > > > > uint32_t *escape; > > > > - static u_char hex[] = "0123456789abcdef"; > > > > + static u_char hex[] = "0123456789ABCDEF"; > > > > > > > > /* " ", "#", "%", "?", %00-%1F, %7F-%FF */ > > > > > > I can't say I like this change. I've considered this a while ago, > > > and decided to keep it as is. This preserve compatibility with > > > what nginx used to do for years. And it also looks like Apache > > > does the same. > > > > > > Any other opinions? > > > > > > > I prefer to fix this instead of keeping it for another few years. > > > > Uppercase digits also look more distinctly, since all other parts > > of path are usually in lowercase. > > Ruslan? Igor? I don't really care, but just to be in line with RFC recommendations and our own comments (see above, it's in the patch context), I think we can switch to using upper case. From 315567599 at qq.com Tue Jul 1 05:30:55 2014 From: 315567599 at qq.com (=?utf-8?B?44CAZ29kemV5ZeOAgA==?=) Date: Tue, 1 Jul 2014 13:30:55 +0800 Subject: Fw:Re: [PATCH] Core: use uppercase hexadecimal digits forpercent-encoding Message-ID: I think lowercase hexadecimal is good for chinese.? ------------------ Original ------------------ From: "Ruslan Ermilov";; Date: Tue, Jul 1, 2014 01:26 PM To: "Maxim Dounin"; Cc: "nginx-devel"; Subject: Re: [PATCH] Core: use uppercase hexadecimal digits forpercent-encoding On Tue, Jul 01, 2014 at 05:15:06AM +0400, Maxim Dounin wrote: > On Mon, Jun 30, 2014 at 04:41:26PM +0400, Valentin V. Bartenev wrote: > > On Friday 27 June 2014 21:57:54 Maxim Dounin wrote: > > > Hello! > > > > > > On Thu, Jun 26, 2014 at 11:40:27PM -0700, Piotr Sikora wrote: > > > > > > > # HG changeset patch > > > > # User Piotr Sikora > > > > # Date 1403851163 25200 > > > > # Thu Jun 26 23:39:23 2014 -0700 > > > > # Node ID 177382006b7d7a421688831d5793b2e417074b48 > > > > # Parent 42114bf12da0cf3d428d0e695139f5366cbd0513 > > > > Core: use uppercase hexadecimal digits for percent-encoding. > > > > > > > > RFC3986 says that, for consistency, URI producers and normalizers > > > > should use uppercase hexadecimal digits for all percent-encodings. > > > > > > > > This is also what modern web browsers and other tools use. > > > > > > > > Using lowercase hexadecimal digits makes it harder to interact with > > > > those tools in case when use of the percent-encoded URI is required, > > > > for example when $request_uri is part of the cache key. > > > > > > > > Signed-off-by: Piotr Sikora > > > > > > > > diff -r 42114bf12da0 -r 177382006b7d src/core/ngx_string.c > > > > --- a/src/core/ngx_string.c Mon Jun 16 19:43:25 2014 +0400 > > > > +++ b/src/core/ngx_string.c Thu Jun 26 23:39:23 2014 -0700 > > > > @@ -1407,7 +1407,7 @@ ngx_escape_uri(u_char *dst, u_char *src, > > > > { > > > > ngx_uint_t n; > > > > uint32_t *escape; > > > > - static u_char hex[] = "0123456789abcdef"; > > > > + static u_char hex[] = "0123456789ABCDEF"; > > > > > > > > /* " ", "#", "%", "?", %00-%1F, %7F-%FF */ > > > > > > I can't say I like this change. I've considered this a while ago, > > > and decided to keep it as is. This preserve compatibility with > > > what nginx used to do for years. And it also looks like Apache > > > does the same. > > > > > > Any other opinions? > > > > > > > I prefer to fix this instead of keeping it for another few years. > > > > Uppercase digits also look more distinctly, since all other parts > > of path are usually in lowercase. > > Ruslan? Igor? I don't really care, but just to be in line with RFC recommendations and our own comments (see above, it's in the patch context), I think we can switch to using upper case. _______________________________________________ nginx-devel mailing list nginx-devel at nginx.org http://mailman.nginx.org/mailman/listinfo/nginx-devel . -------------- next part -------------- An HTML attachment was scrubbed... URL: From dcarlier at afilias.info Tue Jul 1 14:30:43 2014 From: dcarlier at afilias.info (David Carlier) Date: Tue, 1 Jul 2014 15:30:43 +0100 Subject: Development In-Reply-To: References: Message-ID: Hi Filipe, What if I look into this one http://trac.nginx.org/nginx/ticket/485 This kind of change in the ngx_http_request_t struct will be accepted (and furthermore auth basic module, upstream ...) ?? Thanks in advance. On 30 June 2014 09:05, Filipe Da Silva wrote: > Hi, > > In short : http://nginx.org/en/docs/contributing_changes.html > > And, patch must be made with this option set in your hgrc file : > > [diff] > showfunc = True > > Rgds, > Filipe > > 2014-06-28 8:22 GMT+02:00 David Carlier : > > HI All, > > I am working as C/C++ developer for a company which makes nginx modules > and > > would like to know if I can contribute a bit. > > > > Kind regards. > > David CARLIER > > > > dotMobi / Afilias Technologies DUBLIN > > > > _______________________________________________ > nginx-devel mailing list > nginx-devel at nginx.org > http://mailman.nginx.org/mailman/listinfo/nginx-devel > -------------- next part -------------- An HTML attachment was scrubbed... URL: From grrm77 at gmail.com Tue Jul 1 21:54:59 2014 From: grrm77 at gmail.com (grrm grrm) Date: Wed, 2 Jul 2014 00:54:59 +0300 Subject: Patch: Refactor ngx_http_write_request_body into a filter In-Reply-To: <20140627135419.GR1849@mdounin.ru> References: <20140620180926.GO1849@mdounin.ru> <20140627135419.GR1849@mdounin.ru> Message-ID: Hello! What do you think about this patch? It passes the tests and does the same number of writes to the disk as the old code (at least in the tests). >From a readability point of view is also no very bad. Thanks. 2014-06-27 16:54 GMT+03:00 Maxim Dounin : > Hello! > > On Thu, Jun 26, 2014 at 10:07:55PM +0300, grrm grrm wrote: > >> Hi! >> >> I managed to fix the write to disk issue, but as you said the code now >> looks quite convoluted. Those ifs are horrible. Understandably I >> guess, when you try to move logic from different places into one place >> but it still depends on external context (rb->buf). My patch was >> mostly a try to pave the way to non-buffered request body processing >> in way similar to the response processing pipeline where all the work >> is done by the filters. >> >> I saw this feature in the tengine fork of nginx, however there the >> work is still done by a handler similar to write_to_file. Also, all >> the body data need to be copied inside the memory at least one time, >> which is not good. >> >> I also looked at the repose pipeline and there are two main methods of >> reading from the client - the nonbuffered and ngx_event_pipe_t. Do you >> think the pipe could be used in reverse (client->upstream)? Or would >> it even make sense to do it that way? > > In theory it should be possible to use event pipe in any > direction. But I don't think that it would be easy to > integrate it with various request body requirements. > >> Also, do you have any work done into this direction (if you can >> comment on that)? Granted, my attempt was maybe too big a step. > > I've previously posted an experimental patch which introduces an > ability to insert filters into request body chain, which can be > considered as a "work in this direction". > > I think it should be implemented as another filter in > the request body filter chain. It will likely require various > modification to the request body reading code though. > > -- > Maxim Dounin > http://nginx.org/ > > _______________________________________________ > nginx-devel mailing list > nginx-devel at nginx.org > http://mailman.nginx.org/mailman/listinfo/nginx-devel -------------- next part -------------- A non-text attachment was scrubbed... Name: request_body2.patch Type: application/octet-stream Size: 11871 bytes Desc: not available URL: From mdounin at mdounin.ru Wed Jul 2 11:01:22 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Wed, 2 Jul 2014 15:01:22 +0400 Subject: Development In-Reply-To: References: Message-ID: <20140702110122.GV1849@mdounin.ru> Hello! On Tue, Jul 01, 2014 at 03:30:43PM +0100, David Carlier wrote: > What if I look into this one http://trac.nginx.org/nginx/ticket/485 > > This kind of change in the ngx_http_request_t struct will be accepted (and > furthermore auth basic module, upstream ...) ?? I've responded in the ticket. I don't think that the change suggested is right, see there. (And, actually, I don't think that ticket #485 is a good place to start with.) -- Maxim Dounin http://nginx.org/ From vbart at nginx.com Wed Jul 2 12:01:12 2014 From: vbart at nginx.com (Valentin Bartenev) Date: Wed, 02 Jul 2014 12:01:12 +0000 Subject: [nginx] Upstream: fixed handling of write event after sending re... Message-ID: details: http://hg.nginx.org/nginx/rev/9d3a9c45fc43 branches: changeset: 5745:9d3a9c45fc43 user: Valentin Bartenev date: Tue Jul 01 20:52:08 2014 +0400 description: Upstream: fixed handling of write event after sending request. The ngx_http_upstream_dummy_handler() must be set regardless of the read event state. This prevents possible additional call of ngx_http_upstream_send_request_handler(). diffstat: src/http/ngx_http_upstream.c | 14 +++++++------- 1 files changed, 7 insertions(+), 7 deletions(-) diffs (31 lines): diff -r 42114bf12da0 -r 9d3a9c45fc43 src/http/ngx_http_upstream.c --- a/src/http/ngx_http_upstream.c Mon Jun 16 19:43:25 2014 +0400 +++ b/src/http/ngx_http_upstream.c Tue Jul 01 20:52:08 2014 +0400 @@ -1674,13 +1674,6 @@ ngx_http_upstream_send_request(ngx_http_ c->tcp_nopush = NGX_TCP_NOPUSH_UNSET; } - ngx_add_timer(c->read, u->conf->read_timeout); - - if (c->read->ready) { - ngx_http_upstream_process_header(r, u); - return; - } - u->write_event_handler = ngx_http_upstream_dummy_handler; if (ngx_handle_write_event(c->write, 0) != NGX_OK) { @@ -1688,6 +1681,13 @@ ngx_http_upstream_send_request(ngx_http_ NGX_HTTP_INTERNAL_SERVER_ERROR); return; } + + ngx_add_timer(c->read, u->conf->read_timeout); + + if (c->read->ready) { + ngx_http_upstream_process_header(r, u); + return; + } } From cubicdaiya at gmail.com Thu Jul 3 03:16:25 2014 From: cubicdaiya at gmail.com (Tatsuhiko Kubo) Date: Thu, 3 Jul 2014 12:16:25 +0900 Subject: [PATCH]Added .hgignore Message-ID: # HG changeset patch # User Tatsuhiko Kubo # Date 1404357165 -32400 # Thu Jul 03 12:12:45 2014 +0900 # Node ID a173bd6fbccfa0a737ec8e84c56efcb434d3c185 # Parent 9d3a9c45fc434a059c1429c20d27933df36bb7fc Added .hgignore. These files are generated when nginx is built. diff -r 9d3a9c45fc43 -r a173bd6fbccf .hgignore --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/.hgignore Thu Jul 03 12:12:45 2014 +0900 @@ -0,0 +1,10 @@ +syntax: glob + +# files which are generated when nginx is built +Makefile +objs/autoconf.err +objs/nginx +objs/nginx.8 +objs/ngx_auto_config.h +objs/ngx_auto_headers.h +objs/ngx_modules.c -- Tatsuhiko Kubo E-Mail: cubicdaiya at gmail.com -------------- next part -------------- A non-text attachment was scrubbed... Name: add-hgignore.patch Type: application/octet-stream Size: 642 bytes Desc: not available URL: From mdounin at mdounin.ru Thu Jul 3 09:26:44 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Thu, 3 Jul 2014 13:26:44 +0400 Subject: [PATCH]Added .hgignore In-Reply-To: References: Message-ID: <20140703092644.GF1849@mdounin.ru> Hello! On Thu, Jul 03, 2014 at 12:16:25PM +0900, Tatsuhiko Kubo wrote: > # HG changeset patch > # User Tatsuhiko Kubo > # Date 1404357165 -32400 > # Thu Jul 03 12:12:45 2014 +0900 > # Node ID a173bd6fbccfa0a737ec8e84c56efcb434d3c185 > # Parent 9d3a9c45fc434a059c1429c20d27933df36bb7fc > Added .hgignore. > > These files are generated when nginx is built. > > diff -r 9d3a9c45fc43 -r a173bd6fbccf .hgignore > --- /dev/null Thu Jan 01 00:00:00 1970 +0000 > +++ b/.hgignore Thu Jul 03 12:12:45 2014 +0900 > @@ -0,0 +1,10 @@ > +syntax: glob > + > +# files which are generated when nginx is built > +Makefile > +objs/autoconf.err > +objs/nginx > +objs/nginx.8 > +objs/ngx_auto_config.h > +objs/ngx_auto_headers.h > +objs/ngx_modules.c I would rather no, thanks. These aren't the only files generated (e.g., obj's directory name depends on configure arguments), and the .hgignore file is usually placed locally with contents matching particular environment. Right now I have something like this in my main working directory: syntax:glob objs objs.msvc8 .hgignore Makefile autotest.core tmp old/ nginx-* -- Maxim Dounin http://nginx.org/ From cubicdaiya at gmail.com Fri Jul 4 01:27:47 2014 From: cubicdaiya at gmail.com (Tatsuhiko Kubo) Date: Fri, 4 Jul 2014 10:27:47 +0900 Subject: [PATCH]Added .hgignore In-Reply-To: <20140703092644.GF1849@mdounin.ru> References: <20140703092644.GF1849@mdounin.ru> Message-ID: Hello! Thanks for your reply and advice! 2014-07-03 18:26 GMT+09:00 Maxim Dounin : > Hello! > > On Thu, Jul 03, 2014 at 12:16:25PM +0900, Tatsuhiko Kubo wrote: > >> # HG changeset patch >> # User Tatsuhiko Kubo >> # Date 1404357165 -32400 >> # Thu Jul 03 12:12:45 2014 +0900 >> # Node ID a173bd6fbccfa0a737ec8e84c56efcb434d3c185 >> # Parent 9d3a9c45fc434a059c1429c20d27933df36bb7fc >> Added .hgignore. >> >> These files are generated when nginx is built. >> >> diff -r 9d3a9c45fc43 -r a173bd6fbccf .hgignore >> --- /dev/null Thu Jan 01 00:00:00 1970 +0000 >> +++ b/.hgignore Thu Jul 03 12:12:45 2014 +0900 >> @@ -0,0 +1,10 @@ >> +syntax: glob >> + >> +# files which are generated when nginx is built >> +Makefile >> +objs/autoconf.err >> +objs/nginx >> +objs/nginx.8 >> +objs/ngx_auto_config.h >> +objs/ngx_auto_headers.h >> +objs/ngx_modules.c > > I would rather no, thanks. > > These aren't the only files generated (e.g., obj's directory name > depends on configure arguments), and the .hgignore file is usually > placed locally with contents matching particular environment. > Right now I have something like this in my main working directory: > > syntax:glob > objs > objs.msvc8 > .hgignore > Makefile > autotest.core > tmp > old/ > nginx-* > > -- > Maxim Dounin > http://nginx.org/ > > _______________________________________________ > nginx-devel mailing list > nginx-devel at nginx.org > http://mailman.nginx.org/mailman/listinfo/nginx-devel -- Tatsuhiko Kubo E-Mail: cubicdaiya at gmail.com From pdn at cryptopro.ru Fri Jul 4 12:18:03 2014 From: pdn at cryptopro.ru (=?koi8-r?B?8Mne1czJziDkzcnU0snKIO7Jy8/MwcXXyd4=?=) Date: Fri, 4 Jul 2014 12:18:03 +0000 Subject: [PATCH 0 of 1] allow to use engine keyform for server private key Message-ID: We looked at STORE_METHOD but didn't find any good examples. We looked at "format=%s engine=% key=%" at single ssl_certificate_key directive but found this way more complex. Currently, we want to add 2 directives to ngx_http_ssl_module (as seen in openssl apps args): ssl_certificate_keyform type; ssl_certificate_engine device; "Type" can be PEM or ENGINE with default PEM. "Device" defines ssl engine when "type" is ENGINE in a current http, server context. Will this be enough? -------------- next part -------------- An HTML attachment was scrubbed... URL: From mdounin at mdounin.ru Fri Jul 4 12:31:52 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Fri, 4 Jul 2014 16:31:52 +0400 Subject: [PATCH 0 of 1] allow to use engine keyform for server private key In-Reply-To: References: Message-ID: <20140704123152.GQ1849@mdounin.ru> Hello! On Fri, Jul 04, 2014 at 12:18:03PM +0000, ??????? ??????? ?????????? wrote: > We looked at STORE_METHOD but didn't find any good examples. > We looked at "format=%s engine=% key=%" at single ssl_certificate_key directive but found this way more complex. > > Currently, we want to add 2 directives to ngx_http_ssl_module (as seen in openssl apps args): > ssl_certificate_keyform type; > ssl_certificate_engine device; > > "Type" can be PEM or ENGINE with default PEM. "Device" defines ssl engine when "type" is ENGINE in a current http, server context. > > Will this be enough? Doesn't looks like a good solution for me. BTW, posting in text/plain, with appropriate quoting and with In-Reply-To is a plus. Thank you. -- Maxim Dounin http://nginx.org/ From saprykin.dmitry at gmail.com Fri Jul 4 13:18:52 2014 From: saprykin.dmitry at gmail.com (Dmitry Saprykin) Date: Fri, 4 Jul 2014 17:18:52 +0400 Subject: Nginx patch for unexpected FASTCGI_END_REQUEST on php keep_conn + fastcgi_finish_request Message-ID: Hello, This changeset adds support for FastCGI FCGI_END_REQUEST record type. Now nginx does not process this type of FastCGI record. In case of usage php fastcgi upstream which finishes FastCGI requests before end of script using fastcgi_finish_request() call it leads to "upstream sent unexpected FastCGI record: 3 while reading response header from upstream" error messages and 502 for clients. Changeset parses FCGI_END_REQUEST FastCGI records and ignores it if keep_conn is enabled and record has type FCGI_REQUEST_COMPLETE. Kind regards, Dmitry Saprykin -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: FCGI_END_REQUEST.patch Type: text/x-patch Size: 3233 bytes Desc: not available URL: From mdounin at mdounin.ru Fri Jul 4 13:23:31 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Fri, 4 Jul 2014 17:23:31 +0400 Subject: Nginx patch for unexpected FASTCGI_END_REQUEST on php keep_conn + fastcgi_finish_request In-Reply-To: References: Message-ID: <20140704132331.GR1849@mdounin.ru> Hello! On Fri, Jul 04, 2014 at 05:18:52PM +0400, Dmitry Saprykin wrote: > Hello, > > This changeset adds support for FastCGI FCGI_END_REQUEST record type. > Now nginx does not process this type of FastCGI record. > In case of usage php fastcgi upstream which finishes FastCGI > requests before end of script using fastcgi_finish_request() call > it leads to "upstream sent unexpected FastCGI record: 3 while reading > response header from upstream" error messages and 502 for clients. > > Changeset parses FCGI_END_REQUEST FastCGI records and ignores it > if keep_conn is enabled and record has type FCGI_REQUEST_COMPLETE. What makes you think that this is something to be fixed in nginx? The FCGI_END_REQUEST record is clearly unexpected if there are no requests in flight. You may want to focus on fixing the problem in php instead. -- Maxim Dounin http://nginx.org/ From saprykin.dmitry at gmail.com Fri Jul 4 13:33:43 2014 From: saprykin.dmitry at gmail.com (Dmitry Saprykin) Date: Fri, 4 Jul 2014 17:33:43 +0400 Subject: Nginx patch for unexpected FASTCGI_END_REQUEST on php keep_conn + fastcgi_finish_request In-Reply-To: <20140704132331.GR1849@mdounin.ru> References: <20140704132331.GR1849@mdounin.ru> Message-ID: Ok, quite clear. ) Just thought that in case of keepalive connection to FastCGI upstream we can process more than 1 request using one connect. Unexpected FCGI_END_REQUEST is received in following case: 1) Request 1 sent to backend, results received and sent to client 2) Request 2 sent to backend by nginx and is in queue because php is busy after fastcgi_finish_request() call 3) php finishes extra work after fastcgi_finish_request() call and sends FCGI_END_REQUEST 4) Nginx expects STDERR or STDOUT of request 2 but receives FCGI_END_REQUEST of request 1. If this situation is normal by design I will just not use keep_conn. Kind regards, Dmitry Saprykin On 4 July 2014 17:23, Maxim Dounin wrote: > Hello! > > On Fri, Jul 04, 2014 at 05:18:52PM +0400, Dmitry Saprykin wrote: > > > Hello, > > > > This changeset adds support for FastCGI FCGI_END_REQUEST record type. > > Now nginx does not process this type of FastCGI record. > > In case of usage php fastcgi upstream which finishes FastCGI > > requests before end of script using fastcgi_finish_request() call > > it leads to "upstream sent unexpected FastCGI record: 3 while reading > > response header from upstream" error messages and 502 for clients. > > > > Changeset parses FCGI_END_REQUEST FastCGI records and ignores it > > if keep_conn is enabled and record has type FCGI_REQUEST_COMPLETE. > > What makes you think that this is something to be fixed in nginx? > The FCGI_END_REQUEST record is clearly unexpected if there are no > requests in flight. You may want to focus on fixing the problem > in php instead. > > -- > Maxim Dounin > http://nginx.org/ > > _______________________________________________ > nginx-devel mailing list > nginx-devel at nginx.org > http://mailman.nginx.org/mailman/listinfo/nginx-devel > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mdounin at mdounin.ru Fri Jul 4 16:13:06 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Fri, 4 Jul 2014 20:13:06 +0400 Subject: Nginx patch for unexpected FASTCGI_END_REQUEST on php keep_conn + fastcgi_finish_request In-Reply-To: References: <20140704132331.GR1849@mdounin.ru> Message-ID: <20140704161305.GS1849@mdounin.ru> Hello! On Fri, Jul 04, 2014 at 05:33:43PM +0400, Dmitry Saprykin wrote: > Ok, quite clear. ) Just thought that in case of keepalive connection to > FastCGI upstream we can process more than 1 request using one connect. > > Unexpected FCGI_END_REQUEST is received in following case: > 1) Request 1 sent to backend, results received and sent to client > 2) Request 2 sent to backend by nginx and is in queue because php is busy > after fastcgi_finish_request() call > 3) php finishes extra work after fastcgi_finish_request() call and sends > FCGI_END_REQUEST > 4) Nginx expects STDERR or STDOUT of request 2 but receives FCGI_END_REQUEST > of request 1. > > If this situation is normal by design I will just not use keep_conn. This situation isn't normal. It's a bug in php which sends two FCGI_END_REQUEST records for a request - first one after fastcgi_finish_request(), and another one once it finishes the work. And this probably should be fixed - in php. There is nothing to fix in nginx though. -- Maxim Dounin http://nginx.org/ From mdounin at mdounin.ru Fri Jul 4 17:10:04 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Fri, 04 Jul 2014 17:10:04 +0000 Subject: [nginx] Upstream: p->downstream_error instead of closing connect... Message-ID: details: http://hg.nginx.org/nginx/rev/35990c69b3ac branches: changeset: 5746:35990c69b3ac user: Maxim Dounin date: Fri Jul 04 20:47:16 2014 +0400 description: Upstream: p->downstream_error instead of closing connection. Previously, nginx closed client connection in cases when a response body from upstream was needed to be cached or stored but shouldn't be sent to the client. While this is normal for HTTP, it is unacceptable for SPDY. Fix is to use instead the p->downstream_error flag to prevent nginx from sending anything downstream. To make this work, the event pipe code was modified to properly cache empty responses with the flag set. diffstat: src/event/ngx_event_pipe.c | 15 +++++---------- src/http/ngx_http_upstream.c | 20 ++++++++------------ 2 files changed, 13 insertions(+), 22 deletions(-) diffs (65 lines): diff --git a/src/event/ngx_event_pipe.c b/src/event/ngx_event_pipe.c --- a/src/event/ngx_event_pipe.c +++ b/src/event/ngx_event_pipe.c @@ -439,7 +439,11 @@ ngx_event_pipe_read_upstream(ngx_event_p } } - if (p->cacheable && p->in) { + if (p->cacheable && (p->in || p->buf_to_file)) { + + ngx_log_debug0(NGX_LOG_DEBUG_EVENT, p->log, 0, + "pipe write chain"); + if (ngx_event_pipe_write_chain_to_temp_file(p) == NGX_ABORT) { return NGX_ABORT; } @@ -515,15 +519,6 @@ ngx_event_pipe_write_to_downstream(ngx_e p->in = NULL; } - if (p->cacheable && p->buf_to_file) { - ngx_log_debug0(NGX_LOG_DEBUG_EVENT, p->log, 0, - "pipe write chain"); - - if (ngx_event_pipe_write_chain_to_temp_file(p) == NGX_ABORT) { - return NGX_ABORT; - } - } - ngx_log_debug0(NGX_LOG_DEBUG_EVENT, p->log, 0, "pipe write downstream done"); diff --git a/src/http/ngx_http_upstream.c b/src/http/ngx_http_upstream.c --- a/src/http/ngx_http_upstream.c +++ b/src/http/ngx_http_upstream.c @@ -2357,21 +2357,17 @@ ngx_http_upstream_send_response(ngx_http if (r->header_only) { - if (u->cacheable || u->store) { - - if (ngx_shutdown_socket(c->fd, NGX_WRITE_SHUTDOWN) == -1) { - ngx_connection_error(c, ngx_socket_errno, - ngx_shutdown_socket_n " failed"); - } - - r->read_event_handler = ngx_http_request_empty_handler; - r->write_event_handler = ngx_http_request_empty_handler; - c->error = 1; - - } else { + if (!u->buffering) { ngx_http_upstream_finalize_request(r, u, rc); return; } + + if (!u->cacheable && !u->store) { + ngx_http_upstream_finalize_request(r, u, rc); + return; + } + + u->pipe->downstream_error = 1; } if (r->request_body && r->request_body->temp_file) { From mdounin at mdounin.ru Fri Jul 4 18:31:25 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Fri, 04 Jul 2014 18:31:25 +0000 Subject: [nginx] SSL: logging level of "peer closed connection in SSL han... Message-ID: details: http://hg.nginx.org/nginx/rev/57c05ff57980 branches: changeset: 5747:57c05ff57980 user: Maxim Dounin date: Fri Jul 04 22:14:36 2014 +0400 description: SSL: logging level of "peer closed connection in SSL handshake". Previously, the NGX_LOG_INFO level was used unconditionally. This is correct for client SSL connections, but too low for connections to upstream servers. To resolve this, ngx_connection_error() now used to log this error, it will select logging level appropriately. With this change, if an upstream connection is closed during SSL handshake, it is now properly logged at "error" level. diffstat: src/event/ngx_event_openssl.c | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) diffs (14 lines): diff --git a/src/event/ngx_event_openssl.c b/src/event/ngx_event_openssl.c --- a/src/event/ngx_event_openssl.c +++ b/src/event/ngx_event_openssl.c @@ -1112,8 +1112,8 @@ ngx_ssl_handshake(ngx_connection_t *c) c->read->eof = 1; if (sslerr == SSL_ERROR_ZERO_RETURN || ERR_peek_error() == 0) { - ngx_log_error(NGX_LOG_INFO, c->log, err, - "peer closed connection in SSL handshake"); + ngx_connection_error(c, err, + "peer closed connection in SSL handshake"); return NGX_ERROR; } From mdounin at mdounin.ru Sat Jul 5 16:47:54 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Sat, 05 Jul 2014 16:47:54 +0000 Subject: [nginx] Updated PCRE used for win32 builds. Message-ID: details: http://hg.nginx.org/nginx/rev/d2c286564fd4 branches: changeset: 5748:d2c286564fd4 user: Maxim Dounin date: Fri Jul 04 22:34:03 2014 +0400 description: Updated PCRE used for win32 builds. diffstat: misc/GNUmakefile | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diffs (12 lines): diff --git a/misc/GNUmakefile b/misc/GNUmakefile --- a/misc/GNUmakefile +++ b/misc/GNUmakefile @@ -7,7 +7,7 @@ TEMP = tmp OBJS = objs.msvc8 OPENSSL = openssl-1.0.1h ZLIB = zlib-1.2.8 -PCRE = pcre-8.34 +PCRE = pcre-8.35 release: export From mdounin at mdounin.ru Sun Jul 6 19:32:05 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Sun, 06 Jul 2014 19:32:05 +0000 Subject: [nginx] Win32: ngx_open_dir() and ngx_close_dir() fixes. Message-ID: details: http://hg.nginx.org/nginx/rev/a680bf4dddd5 branches: changeset: 5749:a680bf4dddd5 user: Maxim Dounin date: Sat Jul 05 23:29:47 2014 +0400 description: Win32: ngx_open_dir() and ngx_close_dir() fixes. The ngx_open_dir() function changed to restore name passed to it. This fixes removing destination directory in dav module, as caught by dav.t. The ngx_close_dir() function introduced to properly convert errors, as FindClose() returns 0 on error. diffstat: src/os/win32/ngx_files.c | 13 +++++++++++++ src/os/win32/ngx_files.h | 2 +- 2 files changed, 14 insertions(+), 1 deletions(-) diffs (42 lines): diff --git a/src/os/win32/ngx_files.c b/src/os/win32/ngx_files.c --- a/src/os/win32/ngx_files.c +++ b/src/os/win32/ngx_files.c @@ -430,6 +430,8 @@ ngx_open_dir(ngx_str_t *name, ngx_dir_t dir->dir = FindFirstFile((const char *) name->data, &dir->finddata); + name->data[name->len] = '\0'; + if (dir->dir == INVALID_HANDLE_VALUE) { return NGX_ERROR; } @@ -459,6 +461,17 @@ ngx_read_dir(ngx_dir_t *dir) ngx_int_t +ngx_close_dir(ngx_dir_t *dir) +{ + if (FindClose(dir->dir) == 0) { + return NGX_ERROR; + } + + return NGX_OK; +} + + +ngx_int_t ngx_open_glob(ngx_glob_t *gl) { u_char *p; diff --git a/src/os/win32/ngx_files.h b/src/os/win32/ngx_files.h --- a/src/os/win32/ngx_files.h +++ b/src/os/win32/ngx_files.h @@ -193,7 +193,7 @@ ngx_int_t ngx_read_dir(ngx_dir_t *dir); #define ngx_read_dir_n "FindNextFile()" -#define ngx_close_dir(d) FindClose((d)->dir) +ngx_int_t ngx_close_dir(ngx_dir_t *dir); #define ngx_close_dir_n "FindClose()" From piotr at cloudflare.com Sun Jul 6 23:50:48 2014 From: piotr at cloudflare.com (Piotr Sikora) Date: Sun, 06 Jul 2014 16:50:48 -0700 Subject: [PATCH 1 of 4] SSL: include correct OpenSSL headers Message-ID: <8c3e64571adf1ff45fa4.1404690648@Piotrs-MacBook-Pro.local> # HG changeset patch # User Piotr Sikora # Date 1404690073 25200 # Sun Jul 06 16:41:13 2014 -0700 # Node ID 8c3e64571adf1ff45fa40559f761ea317e29d474 # Parent a680bf4dddd5c4b106419e3dfb0264815c401275 SSL: include correct OpenSSL headers. Previously, , , and were pulled in by using OpenSSL's deprecated interface, which meant that nginx couldn't have been built with -DOPENSSL_NO_DEPRECATED. Both and are pulled in by , but we're calling X509 functions directly, so let's include those as well. is pulled in by virtually everything, but we're calling CRYPTO_add() directly, so let's include it as well. Signed-off-by: Piotr Sikora diff -r a680bf4dddd5 -r 8c3e64571adf src/event/ngx_event_openssl.h --- a/src/event/ngx_event_openssl.h Sat Jul 05 23:29:47 2014 +0400 +++ b/src/event/ngx_event_openssl.h Sun Jul 06 16:41:13 2014 -0700 @@ -14,10 +14,17 @@ #include #include +#include #include +#include +#include #include #include #include +#include +#include +#include +#include #define NGX_SSL_NAME "OpenSSL" From piotr at cloudflare.com Sun Jul 6 23:50:49 2014 From: piotr at cloudflare.com (Piotr Sikora) Date: Sun, 06 Jul 2014 16:50:49 -0700 Subject: [PATCH 2 of 4] SSL: return temporary RSA key only when the key length matches In-Reply-To: <8c3e64571adf1ff45fa4.1404690648@Piotrs-MacBook-Pro.local> References: <8c3e64571adf1ff45fa4.1404690648@Piotrs-MacBook-Pro.local> Message-ID: <2ca8a17cedfd35da799b.1404690649@Piotrs-MacBook-Pro.local> # HG changeset patch # User Piotr Sikora # Date 1404690073 25200 # Sun Jul 06 16:41:13 2014 -0700 # Node ID 2ca8a17cedfd35da799b258d5d17427d7bee62b2 # Parent 8c3e64571adf1ff45fa40559f761ea317e29d474 SSL: return temporary RSA key only when the key length matches. This change is mostly cosmetic, because in practice this callback is used only for 512-bit RSA keys. Signed-off-by: Piotr Sikora diff -r 8c3e64571adf -r 2ca8a17cedfd src/event/ngx_event_openssl.c --- a/src/event/ngx_event_openssl.c Sun Jul 06 16:41:13 2014 -0700 +++ b/src/event/ngx_event_openssl.c Sun Jul 06 16:41:13 2014 -0700 @@ -652,10 +652,12 @@ ngx_ssl_rsa512_key_callback(ngx_ssl_conn { static RSA *key; - if (key_length == 512) { - if (key == NULL) { - key = RSA_generate_key(512, RSA_F4, NULL, NULL); - } + if (key_length != 512) { + return NULL; + } + + if (key == NULL) { + key = RSA_generate_key(512, RSA_F4, NULL, NULL); } return key; From piotr at cloudflare.com Sun Jul 6 23:50:50 2014 From: piotr at cloudflare.com (Piotr Sikora) Date: Sun, 06 Jul 2014 16:50:50 -0700 Subject: [PATCH 3 of 4] SSL: stop using deprecated RSA_generate_key() function In-Reply-To: <8c3e64571adf1ff45fa4.1404690648@Piotrs-MacBook-Pro.local> References: <8c3e64571adf1ff45fa4.1404690648@Piotrs-MacBook-Pro.local> Message-ID: # HG changeset patch # User Piotr Sikora # Date 1404690074 25200 # Sun Jul 06 16:41:14 2014 -0700 # Node ID e015093a00f2d8ebdbcdd8adcb16d87b291765f8 # Parent 2ca8a17cedfd35da799b258d5d17427d7bee62b2 SSL: stop using deprecated RSA_generate_key() function. Fixes build with -DOPENSSL_NO_DEPRECATED. Signed-off-by: Piotr Sikora diff -r 2ca8a17cedfd -r e015093a00f2 src/event/ngx_event_openssl.c --- a/src/event/ngx_event_openssl.c Sun Jul 06 16:41:13 2014 -0700 +++ b/src/event/ngx_event_openssl.c Sun Jul 06 16:41:14 2014 -0700 @@ -650,6 +650,10 @@ RSA * ngx_ssl_rsa512_key_callback(ngx_ssl_conn_t *ssl_conn, int is_export, int key_length) { +#if OPENSSL_VERSION_NUMBER >= 0x0090800fL + RSA *rsa; + BIGNUM *e; +#endif static RSA *key; if (key_length != 512) { @@ -657,7 +661,42 @@ ngx_ssl_rsa512_key_callback(ngx_ssl_conn } if (key == NULL) { + +#if OPENSSL_VERSION_NUMBER >= 0x0090800fL + + rsa = RSA_new(); + if (rsa == NULL) { + return NULL; + } + + e = BN_new(); + if (e == NULL) { + RSA_free(rsa); + return NULL; + } + + if (BN_set_word(e, RSA_F4) == 0) { + BN_free(e); + RSA_free(rsa); + return NULL; + } + + if (RSA_generate_key_ex(rsa, 512, e, NULL) == 0) { + BN_free(e); + RSA_free(rsa); + return NULL; + } + + BN_free(e); + + key = rsa; + +#else + key = RSA_generate_key(512, RSA_F4, NULL, NULL); + +#endif + } return key; From piotr at cloudflare.com Sun Jul 6 23:50:51 2014 From: piotr at cloudflare.com (Piotr Sikora) Date: Sun, 06 Jul 2014 16:50:51 -0700 Subject: [PATCH 4 of 4] SSL: stop accessing SSL_SESSION's fields directly In-Reply-To: <8c3e64571adf1ff45fa4.1404690648@Piotrs-MacBook-Pro.local> References: <8c3e64571adf1ff45fa4.1404690648@Piotrs-MacBook-Pro.local> Message-ID: <75eb8fc4d403e9624122.1404690651@Piotrs-MacBook-Pro.local> # HG changeset patch # User Piotr Sikora # Date 1404690074 25200 # Sun Jul 06 16:41:14 2014 -0700 # Node ID 75eb8fc4d403e9624122fe6cc56beec89dfec33e # Parent e015093a00f2d8ebdbcdd8adcb16d87b291765f8 SSL: stop accessing SSL_SESSION's fields directly. SSL_SESSION struct is internal part of the OpenSSL library and it's fields should be accessed via API (when exposed), not directly. The unfortunate side-effect of this change is that we're losing reference count that used to be printed at the debug log level, but this seems to be an acceptable trade-off. Almost fixes build with -DOPENSSL_NO_SSL_INTERN. Signed-off-by: Piotr Sikora diff -r e015093a00f2 -r 75eb8fc4d403 src/event/ngx_event_openssl.c --- a/src/event/ngx_event_openssl.c Sun Jul 06 16:41:14 2014 -0700 +++ b/src/event/ngx_event_openssl.c Sun Jul 06 16:41:14 2014 -0700 @@ -2113,9 +2113,10 @@ static int ngx_ssl_new_session(ngx_ssl_conn_t *ssl_conn, ngx_ssl_session_t *sess) { int len; - u_char *p, *id, *cached_sess; + u_char *p, *id, *cached_sess, *session_id; uint32_t hash; SSL_CTX *ssl_ctx; + unsigned int session_id_length; ngx_shm_zone_t *shm_zone; ngx_connection_t *c; ngx_slab_pool_t *shpool; @@ -2178,13 +2179,24 @@ ngx_ssl_new_session(ngx_ssl_conn_t *ssl_ } } +#if OPENSSL_VERSION_NUMBER >= 0x0090800fL + + session_id = (u_char *) SSL_SESSION_get_id(sess, &session_id_length); + +#else + + session_id = sess->session_id; + session_id_length = sess->session_id_length; + +#endif + #if (NGX_PTR_SIZE == 8) id = sess_id->sess_id; #else - id = ngx_slab_alloc_locked(shpool, sess->session_id_length); + id = ngx_slab_alloc_locked(shpool, session_id_length); if (id == NULL) { @@ -2192,7 +2204,7 @@ ngx_ssl_new_session(ngx_ssl_conn_t *ssl_ ngx_ssl_expire_sessions(cache, shpool, 0); - id = ngx_slab_alloc_locked(shpool, sess->session_id_length); + id = ngx_slab_alloc_locked(shpool, session_id_length); if (id == NULL) { goto failed; @@ -2203,16 +2215,16 @@ ngx_ssl_new_session(ngx_ssl_conn_t *ssl_ ngx_memcpy(cached_sess, buf, len); - ngx_memcpy(id, sess->session_id, sess->session_id_length); - - hash = ngx_crc32_short(sess->session_id, sess->session_id_length); + ngx_memcpy(id, session_id, session_id_length); + + hash = ngx_crc32_short(session_id, session_id_length); ngx_log_debug3(NGX_LOG_DEBUG_EVENT, c->log, 0, - "ssl new session: %08XD:%d:%d", - hash, sess->session_id_length, len); + "ssl new session: %08XD:%ud:%d", + hash, session_id_length, len); sess_id->node.key = hash; - sess_id->node.data = (u_char) sess->session_id_length; + sess_id->node.data = (u_char) session_id_length; sess_id->id = id; sess_id->len = len; sess_id->session = cached_sess; @@ -2360,10 +2372,10 @@ ngx_ssl_remove_cached_session(SSL_CTX *s static void ngx_ssl_remove_session(SSL_CTX *ssl, ngx_ssl_session_t *sess) { - size_t len; u_char *id; uint32_t hash; ngx_int_t rc; + unsigned int len; ngx_shm_zone_t *shm_zone; ngx_slab_pool_t *shpool; ngx_rbtree_node_t *node, *sentinel; @@ -2378,13 +2390,21 @@ ngx_ssl_remove_session(SSL_CTX *ssl, ngx cache = shm_zone->data; +#if OPENSSL_VERSION_NUMBER >= 0x0090800fL + + id = (u_char *) SSL_SESSION_get_id(sess, &len); + +#else + id = sess->session_id; - len = (size_t) sess->session_id_length; + len = sess->session_id_length; + +#endif hash = ngx_crc32_short(id, len); ngx_log_debug2(NGX_LOG_DEBUG_EVENT, ngx_cycle->log, 0, - "ssl remove session: %08XD:%uz", hash, len); + "ssl remove session: %08XD:%ud", hash, len); shpool = (ngx_slab_pool_t *) shm_zone->shm.addr; @@ -2926,9 +2946,9 @@ ngx_ssl_get_cipher_name(ngx_connection_t ngx_int_t ngx_ssl_get_session_id(ngx_connection_t *c, ngx_pool_t *pool, ngx_str_t *s) { - int len; - u_char *buf; - SSL_SESSION *sess; + u_char *buf; + SSL_SESSION *sess; + unsigned int len; sess = SSL_get0_session(c->ssl->connection); if (sess == NULL) { @@ -2936,9 +2956,17 @@ ngx_ssl_get_session_id(ngx_connection_t return NGX_OK; } +#if OPENSSL_VERSION_NUMBER >= 0x0090800fL + + buf = (u_char *) SSL_SESSION_get_id(sess, &len); + +#else + buf = sess->session_id; len = sess->session_id_length; +#endif + s->len = 2 * len; s->data = ngx_pnalloc(pool, 2 * len); if (s->data == NULL) { diff -r e015093a00f2 -r 75eb8fc4d403 src/http/ngx_http_upstream_round_robin.c --- a/src/http/ngx_http_upstream_round_robin.c Sun Jul 06 16:41:14 2014 -0700 +++ b/src/http/ngx_http_upstream_round_robin.c Sun Jul 06 16:41:14 2014 -0700 @@ -632,9 +632,8 @@ ngx_http_upstream_set_round_robin_peer_s rc = ngx_ssl_set_session(pc->connection, ssl_session); - ngx_log_debug2(NGX_LOG_DEBUG_HTTP, pc->log, 0, - "set session: %p:%d", - ssl_session, ssl_session ? ssl_session->references : 0); + ngx_log_debug1(NGX_LOG_DEBUG_HTTP, pc->log, 0, + "set session: %p", ssl_session); /* ngx_unlock_mutex(rrp->peers->mutex); */ @@ -657,8 +656,8 @@ ngx_http_upstream_save_round_robin_peer_ return; } - ngx_log_debug2(NGX_LOG_DEBUG_HTTP, pc->log, 0, - "save session: %p:%d", ssl_session, ssl_session->references); + ngx_log_debug1(NGX_LOG_DEBUG_HTTP, pc->log, 0, + "save session: %p", ssl_session); peer = &rrp->peers->peer[rrp->current]; @@ -672,9 +671,8 @@ ngx_http_upstream_save_round_robin_peer_ if (old_ssl_session) { - ngx_log_debug2(NGX_LOG_DEBUG_HTTP, pc->log, 0, - "old session: %p:%d", - old_ssl_session, old_ssl_session->references); + ngx_log_debug1(NGX_LOG_DEBUG_HTTP, pc->log, 0, + "old session: %p", old_ssl_session); /* TODO: may block */ From mdounin at mdounin.ru Mon Jul 7 01:13:15 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Mon, 7 Jul 2014 05:13:15 +0400 Subject: [PATCH 3 of 4] SSL: stop using deprecated RSA_generate_key() function In-Reply-To: References: <8c3e64571adf1ff45fa4.1404690648@Piotrs-MacBook-Pro.local> Message-ID: <20140707011315.GZ1849@mdounin.ru> Hello! On Sun, Jul 06, 2014 at 04:50:50PM -0700, Piotr Sikora wrote: > # HG changeset patch > # User Piotr Sikora > # Date 1404690074 25200 > # Sun Jul 06 16:41:14 2014 -0700 > # Node ID e015093a00f2d8ebdbcdd8adcb16d87b291765f8 > # Parent 2ca8a17cedfd35da799b258d5d17427d7bee62b2 > SSL: stop using deprecated RSA_generate_key() function. > > Fixes build with -DOPENSSL_NO_DEPRECATED. > > Signed-off-by: Piotr Sikora > > diff -r 2ca8a17cedfd -r e015093a00f2 src/event/ngx_event_openssl.c > --- a/src/event/ngx_event_openssl.c Sun Jul 06 16:41:13 2014 -0700 > +++ b/src/event/ngx_event_openssl.c Sun Jul 06 16:41:14 2014 -0700 > @@ -650,6 +650,10 @@ RSA * > ngx_ssl_rsa512_key_callback(ngx_ssl_conn_t *ssl_conn, int is_export, > int key_length) > { > +#if OPENSSL_VERSION_NUMBER >= 0x0090800fL > + RSA *rsa; > + BIGNUM *e; > +#endif > static RSA *key; > > if (key_length != 512) { > @@ -657,7 +661,42 @@ ngx_ssl_rsa512_key_callback(ngx_ssl_conn > } > > if (key == NULL) { > + > +#if OPENSSL_VERSION_NUMBER >= 0x0090800fL > + > + rsa = RSA_new(); > + if (rsa == NULL) { > + return NULL; > + } > + > + e = BN_new(); > + if (e == NULL) { > + RSA_free(rsa); > + return NULL; > + } > + > + if (BN_set_word(e, RSA_F4) == 0) { > + BN_free(e); > + RSA_free(rsa); > + return NULL; > + } > + > + if (RSA_generate_key_ex(rsa, 512, e, NULL) == 0) { > + BN_free(e); > + RSA_free(rsa); > + return NULL; > + } > + > + BN_free(e); > + > + key = rsa; > + > +#else > + > key = RSA_generate_key(512, RSA_F4, NULL, NULL); > + > +#endif > + > } > > return key; I can't say I like this change - it introduces lots of code for no real reason. And I don't think we should follow some arbitrarily set "deprecated" flag introduced for an unknown reasons years ago and still undocumented in the latest release (much like the replacement function). Moreover, the RSA_generate_key() is still used in OpenSSL's own codebase, as well as in multiple demos and man pages. -- Maxim Dounin http://nginx.org/ From piotr at cloudflare.com Mon Jul 7 02:16:44 2014 From: piotr at cloudflare.com (Piotr Sikora) Date: Sun, 6 Jul 2014 19:16:44 -0700 Subject: [PATCH 3 of 4] SSL: stop using deprecated RSA_generate_key() function In-Reply-To: <20140707011315.GZ1849@mdounin.ru> References: <8c3e64571adf1ff45fa4.1404690648@Piotrs-MacBook-Pro.local> <20140707011315.GZ1849@mdounin.ru> Message-ID: Hey Maxim, > I can't say I like this change - it introduces lots of code for no > real reason. > > And I don't think we should follow some arbitrarily set > "deprecated" flag introduced for an unknown reasons years ago and > still undocumented in the latest release (much like the > replacement function). Moreover, the RSA_generate_key() is still > used in OpenSSL's own codebase, as well as in multiple demos and > man pages. RSA_generate_key() is clearly marked as deprecated in the OpenSSL's documentation [1] and RSA_generate_key_ex() is documented on the same page. I don't think we should blindly follow -DOPENSSL_NO_DEPRECATED and -DOPENSSL_NO_SSL_INTERN, but it's useful to find potential issues with existing code. [1] https://www.openssl.org/docs/crypto/RSA_generate_key.html Best regards, Piotr Sikora From flygoast at 126.com Mon Jul 7 09:19:45 2014 From: flygoast at 126.com (flygoast) Date: Mon, 7 Jul 2014 17:19:45 +0800 (CST) Subject: [PATCH]Upstream: delete temp file when error in ngx_http_upstream_store(). Message-ID: <73504be7.fe2.147101f310c.Coremail.flygoast@126.com> # HG changeset patch # User FengGu # Date 1404723967 -28800 # Mon Jul 07 17:06:07 2014 +0800 # Node ID d1d597fbf6d8e08059a308d8900e5f90def5377a # Parent a680bf4dddd5c4b106419e3dfb0264815c401275 Upstream: delete temp file when error in ngx_http_upstream_store(). Previously, didn't process the case when ngx_http_map_uri_to_path() returned NULL. And when error occured, didn't delete the temp file. diff -r a680bf4dddd5 -r d1d597fbf6d8 src/http/ngx_http_upstream.c --- a/src/http/ngx_http_upstream.cSat Jul 05 23:29:47 2014 +0400 +++ b/src/http/ngx_http_upstream.cMon Jul 07 17:06:07 2014 +0800 @@ -3375,14 +3375,16 @@ if (u->conf->store_lengths == NULL) { - ngx_http_map_uri_to_path(r, &path, &root, 0); + if (ngx_http_map_uri_to_path(r, &path, &root, 0) == NULL) { + goto error; + } } else { if (ngx_http_script_run(r, &path, u->conf->store_lengths->elts, 0, u->conf->store_values->elts) == NULL) { - return; + goto error; } } @@ -3393,6 +3395,16 @@ tf->file.name.data, path.data); (void) ngx_ext_rename_file(&tf->file.name, &path, &ext); + + return; + +error: + + if (ngx_delete_file(tf->file.name.data) == NGX_FILE_ERROR) { + ngx_log_error(NGX_LOG_CRIT, r->connection->log, ngx_errno, + ngx_delete_file_n " \"%s\" failed", + tf->file.name.data); + } } -------------- next part -------------- An HTML attachment was scrubbed... URL: From mdounin at mdounin.ru Mon Jul 7 13:04:49 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Mon, 7 Jul 2014 17:04:49 +0400 Subject: [PATCH 3 of 4] SSL: stop using deprecated RSA_generate_key() function In-Reply-To: References: <8c3e64571adf1ff45fa4.1404690648@Piotrs-MacBook-Pro.local> <20140707011315.GZ1849@mdounin.ru> Message-ID: <20140707130449.GC1849@mdounin.ru> Hello! On Sun, Jul 06, 2014 at 07:16:44PM -0700, Piotr Sikora wrote: > Hey Maxim, > > > I can't say I like this change - it introduces lots of code for no > > real reason. > > > > And I don't think we should follow some arbitrarily set > > "deprecated" flag introduced for an unknown reasons years ago and > > still undocumented in the latest release (much like the > > replacement function). Moreover, the RSA_generate_key() is still > > used in OpenSSL's own codebase, as well as in multiple demos and > > man pages. > > RSA_generate_key() is clearly marked as deprecated in the OpenSSL's > documentation [1] and RSA_generate_key_ex() is documented on the same > page. It's marked as deprecated in master branch, but not in the latest release. Try looking into the latest release docs, 1.0.1h - doc/crypto/RSA_generate_key.pod doesn't even mention RSA_generate_key_ex. > I don't think we should blindly follow -DOPENSSL_NO_DEPRECATED and > -DOPENSSL_NO_SSL_INTERN, but it's useful to find potential issues with > existing code. Sure, it can and likely will be helpful. In this particular case the replacement code seems to be too long though. For development needs, it will probably be enough to just return NULL if OPENSSL_NO_DEPRECATED is defined. -- Maxim Dounin http://nginx.org/ From mdounin at mdounin.ru Mon Jul 7 16:22:23 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Mon, 7 Jul 2014 20:22:23 +0400 Subject: [PATCH]Upstream: delete temp file when error in ngx_http_upstream_store(). In-Reply-To: <73504be7.fe2.147101f310c.Coremail.flygoast@126.com> References: <73504be7.fe2.147101f310c.Coremail.flygoast@126.com> Message-ID: <20140707162223.GG1849@mdounin.ru> Hello! On Mon, Jul 07, 2014 at 05:19:45PM +0800, flygoast wrote: > # HG changeset patch > # User FengGu > # Date 1404723967 -28800 > # Mon Jul 07 17:06:07 2014 +0800 > # Node ID d1d597fbf6d8e08059a308d8900e5f90def5377a > # Parent a680bf4dddd5c4b106419e3dfb0264815c401275 > Upstream: delete temp file when error in ngx_http_upstream_store(). > > > Previously, didn't process the case when ngx_http_map_uri_to_path() returned > NULL. And when error occured, didn't delete the temp file. > > > diff -r a680bf4dddd5 -r d1d597fbf6d8 src/http/ngx_http_upstream.c > --- a/src/http/ngx_http_upstream.cSat Jul 05 23:29:47 2014 +0400 > +++ b/src/http/ngx_http_upstream.cMon Jul 07 17:06:07 2014 +0800 > @@ -3375,14 +3375,16 @@ > > if (u->conf->store_lengths == NULL) { > > - ngx_http_map_uri_to_path(r, &path, &root, 0); > + if (ngx_http_map_uri_to_path(r, &path, &root, 0) == NULL) { > + goto error; > + } > > } else { > if (ngx_http_script_run(r, &path, u->conf->store_lengths->elts, 0, > u->conf->store_values->elts) > == NULL) > { > - return; > + goto error; > } > } > > @@ -3393,6 +3395,16 @@ > tf->file.name.data, path.data); > > (void) ngx_ext_rename_file(&tf->file.name, &path, &ext); > + > + return; > + > +error: > + > + if (ngx_delete_file(tf->file.name.data) == NGX_FILE_ERROR) { > + ngx_log_error(NGX_LOG_CRIT, r->connection->log, ngx_errno, > + ngx_delete_file_n " \"%s\" failed", > + tf->file.name.data); > + } > } > Thanks for noting this. What do you think about the following patch instead? There is a code in ngx_http_upstream_finalize_request() which normally removes temporary files after errors with proxy_store, but in this particular case it won't be triggered due to u->store set to 0 even in case of errors. With this patch, u->store is set to 0 only in if there are no errors, and thus temporary files are properly removed in case of errors in ngx_http_upstream_store(). diff --git a/src/http/ngx_http_upstream.c b/src/http/ngx_http_upstream.c --- a/src/http/ngx_http_upstream.c +++ b/src/http/ngx_http_upstream.c @@ -3253,7 +3253,6 @@ ngx_http_upstream_process_request(ngx_ht || u->headers_in.content_length_n == tf->offset)) { ngx_http_upstream_store(r, u); - u->store = 0; } } } @@ -3375,7 +3374,9 @@ ngx_http_upstream_store(ngx_http_request if (u->conf->store_lengths == NULL) { - ngx_http_map_uri_to_path(r, &path, &root, 0); + if (ngx_http_map_uri_to_path(r, &path, &root, 0) == NULL) { + return; + } } else { if (ngx_http_script_run(r, &path, u->conf->store_lengths->elts, 0, @@ -3392,7 +3393,11 @@ ngx_http_upstream_store(ngx_http_request "upstream stores \"%s\" to \"%s\"", tf->file.name.data, path.data); - (void) ngx_ext_rename_file(&tf->file.name, &path, &ext); + if (ngx_ext_rename_file(&tf->file.name, &path, &ext) != NGX_OK) { + return; + } + + u->store = 0; } -- Maxim Dounin http://nginx.org/ From grrm77 at gmail.com Mon Jul 7 17:23:20 2014 From: grrm77 at gmail.com (grrm grrm) Date: Mon, 7 Jul 2014 20:23:20 +0300 Subject: Patch: Refactor ngx_http_write_request_body into a filter In-Reply-To: References: <20140620180926.GO1849@mdounin.ru> <20140627135419.GR1849@mdounin.ru> Message-ID: Hi! Did you have a chance to look at the code? Any answer is appreciated. Thank you. 2014-07-02 0:54 GMT+03:00 grrm grrm : > Hello! > > What do you think about this patch? It passes the tests and does the > same number of writes to the disk as the old code (at least in the > tests). > From a readability point of view is also no very bad. > > Thanks. > > 2014-06-27 16:54 GMT+03:00 Maxim Dounin : >> Hello! >> >> On Thu, Jun 26, 2014 at 10:07:55PM +0300, grrm grrm wrote: >> >>> Hi! >>> >>> I managed to fix the write to disk issue, but as you said the code now >>> looks quite convoluted. Those ifs are horrible. Understandably I >>> guess, when you try to move logic from different places into one place >>> but it still depends on external context (rb->buf). My patch was >>> mostly a try to pave the way to non-buffered request body processing >>> in way similar to the response processing pipeline where all the work >>> is done by the filters. >>> >>> I saw this feature in the tengine fork of nginx, however there the >>> work is still done by a handler similar to write_to_file. Also, all >>> the body data need to be copied inside the memory at least one time, >>> which is not good. >>> >>> I also looked at the repose pipeline and there are two main methods of >>> reading from the client - the nonbuffered and ngx_event_pipe_t. Do you >>> think the pipe could be used in reverse (client->upstream)? Or would >>> it even make sense to do it that way? >> >> In theory it should be possible to use event pipe in any >> direction. But I don't think that it would be easy to >> integrate it with various request body requirements. >> >>> Also, do you have any work done into this direction (if you can >>> comment on that)? Granted, my attempt was maybe too big a step. >> >> I've previously posted an experimental patch which introduces an >> ability to insert filters into request body chain, which can be >> considered as a "work in this direction". >> >> I think it should be implemented as another filter in >> the request body filter chain. It will likely require various >> modification to the request body reading code though. >> >> -- >> Maxim Dounin >> http://nginx.org/ >> >> _______________________________________________ >> nginx-devel mailing list >> nginx-devel at nginx.org >> http://mailman.nginx.org/mailman/listinfo/nginx-devel From mdounin at mdounin.ru Mon Jul 7 18:04:49 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Mon, 7 Jul 2014 22:04:49 +0400 Subject: Patch: Refactor ngx_http_write_request_body into a filter In-Reply-To: References: <20140620180926.GO1849@mdounin.ru> <20140627135419.GR1849@mdounin.ru> Message-ID: <20140707180449.GH1849@mdounin.ru> Hello! On Mon, Jul 07, 2014 at 08:23:20PM +0300, grrm grrm wrote: > Did you have a chance to look at the code? I still don't think that suggested change improves the code. -- Maxim Dounin http://nginx.org/ From bsfranks at gmail.com Mon Jul 7 20:49:17 2014 From: bsfranks at gmail.com (bsfranks at gmail.com) Date: Mon, 7 Jul 2014 13:49:17 -0700 Subject: couple questions regarding log phase handlers Message-ID: In the log phase, what is the best way to determine if the client had prematurely closed or aborted the request connection? To check r->connection->error and look for HTTP_BAD_REQUEST someplace? Or possibly check one of the members of the connection read or write structs? Also, is the log phase resumable? It looks like all the log phase handlers are looped through in ngx_http_log_request() called from ngx_http_free_request(). However, return codes are not checked. Was hoping for a way to return NGX_AGAIN from within the log phase so that it could be resumable rather than blocking. Possible? If not possible from within log phase, is there another phase or output filter location that takes place after content phase or content handlers have completed that is non-blocking and resumable? Thank you! -------------- next part -------------- An HTML attachment was scrubbed... URL: From piotr at cloudflare.com Mon Jul 7 22:02:14 2014 From: piotr at cloudflare.com (Piotr Sikora) Date: Mon, 7 Jul 2014 15:02:14 -0700 Subject: [PATCH 3 of 4] SSL: stop using deprecated RSA_generate_key() function In-Reply-To: <20140707130449.GC1849@mdounin.ru> References: <8c3e64571adf1ff45fa4.1404690648@Piotrs-MacBook-Pro.local> <20140707011315.GZ1849@mdounin.ru> <20140707130449.GC1849@mdounin.ru> Message-ID: Hey Maxim, > It's marked as deprecated in master branch, but not in the latest > release. Try looking into the latest release docs, 1.0.1h - > doc/crypto/RSA_generate_key.pod doesn't even mention > RSA_generate_key_ex. It's been deprecated before OpenSSL-0.9.8 release, see git history [1]. OpenSSL's documentation is terrible source of information and that's not news - nginx itself is using a lot of undocumented functions, especially in the OCSP stapling code. > Sure, it can and likely will be helpful. In this particular case > the replacement code seems to be too long though. For > development needs, it will probably be enough to just return NULL > if OPENSSL_NO_DEPRECATED is defined. Sigh, I really don't think that the amount of code is really a problem here... But if you're really unhappy with it, maybe let's just remove the callback altogether? It's not like it's used with nginx's default ciphers list and I'm not aware of anything since Windows NT 4.0 that would require it. [1] https://git.openssl.org/gitweb/?p=openssl.git;a=commitdiff;h=9d473aa2e4076beb959bc9701786a0860877ee12 Best regards, Piotr Sikora From mdounin at mdounin.ru Mon Jul 7 22:28:30 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Tue, 8 Jul 2014 02:28:30 +0400 Subject: couple questions regarding log phase handlers In-Reply-To: References: Message-ID: <20140707222830.GM1849@mdounin.ru> Hello! On Mon, Jul 07, 2014 at 01:49:17PM -0700, bsfranks at gmail.com wrote: > In the log phase, what is the best way to determine if the client had > prematurely closed or aborted the request connection? To check > r->connection->error and look for HTTP_BAD_REQUEST someplace? Or possibly > check one of the members of the connection read or write structs? Probably r->request_complete, as used for the $request_completion variable, see http://nginx.org/r/$request_completion. > Also, is the log phase resumable? It looks like all the log phase handlers > are looped through in > ngx_http_log_request() called from ngx_http_free_request(). However, > return codes are not checked. Was hoping for a way to return NGX_AGAIN > from within the log phase so that it could be resumable rather than > blocking. Possible? No. > If not possible from within log phase, is there > another phase or output filter location that takes place after content > phase or content handlers have completed that is non-blocking and resumable? I don't think there are good ways to do it. Most relevant functionality seems to be post_action, see http://nginx.org/r/post_action for docs and the code for details. -- Maxim Dounin http://nginx.org/ From agentzh at gmail.com Mon Jul 7 22:49:52 2014 From: agentzh at gmail.com (Yichun Zhang (agentzh)) Date: Mon, 7 Jul 2014 15:49:52 -0700 Subject: couple questions regarding log phase handlers In-Reply-To: References: Message-ID: Hello! On Mon, Jul 7, 2014 at 1:49 PM, bsfranks wrote: > Also, is the log phase resumable? It looks like all the log phase handlers > are looped through in > ngx_http_log_request() called from ngx_http_free_request(). However, return > codes are not checked. Was hoping for a way to return NGX_AGAIN from within > the log phase so that it could be resumable rather than blocking. Possible? In the context of ngx_lua, we use nginx timers as a work around. For instance, the lua-resty-logger-socket library uses this work-around and we've been using it to *reliably* collect nginx logs over sockets in production: https://github.com/cloudflare/lua-resty-logger-socket#readme EAGAIN and etc are properly handled there. Regards, -agentzh From mdounin at mdounin.ru Mon Jul 7 23:32:24 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Tue, 8 Jul 2014 03:32:24 +0400 Subject: [PATCH 3 of 4] SSL: stop using deprecated RSA_generate_key() function In-Reply-To: References: <8c3e64571adf1ff45fa4.1404690648@Piotrs-MacBook-Pro.local> <20140707011315.GZ1849@mdounin.ru> <20140707130449.GC1849@mdounin.ru> Message-ID: <20140707233224.GN1849@mdounin.ru> Hello! On Mon, Jul 07, 2014 at 03:02:14PM -0700, Piotr Sikora wrote: > Hey Maxim, > > > It's marked as deprecated in master branch, but not in the latest > > release. Try looking into the latest release docs, 1.0.1h - > > doc/crypto/RSA_generate_key.pod doesn't even mention > > RSA_generate_key_ex. > > It's been deprecated before OpenSSL-0.9.8 release, see git history [1]. Correct link is: https://git.openssl.org/gitweb/?p=openssl.git;a=commitdiff;h=e9224c717711eefb30038c9b37c69795dda93c9a And it looks like the only rationale for deprecation is a new interface introduced. > OpenSSL's documentation is terrible source of information and that's > not news - nginx itself is using a lot of undocumented functions, > especially in the OCSP stapling code. It's not about quality of OpenSSL's documentation, it's about the fact that OpenSSL itself doesn't care about said deprecation. > > Sure, it can and likely will be helpful. In this particular case > > the replacement code seems to be too long though. For > > development needs, it will probably be enough to just return NULL > > if OPENSSL_NO_DEPRECATED is defined. > > Sigh, I really don't think that the amount of code is really a problem > here... But if you're really unhappy with it, maybe let's just remove The amount of code is certainly a problem, and it indicates that the "new" interface, uhm, have problems. > the callback altogether? It's not like it's used with nginx's default > ciphers list and I'm not aware of anything since Windows NT 4.0 that > would require it. I don't think it's time to remove it, but as I previously said, I will be fine with something like this: diff --git a/src/event/ngx_event_openssl.c b/src/event/ngx_event_openssl.c --- a/src/event/ngx_event_openssl.c +++ b/src/event/ngx_event_openssl.c @@ -652,12 +652,16 @@ ngx_ssl_rsa512_key_callback(ngx_ssl_conn { static RSA *key; +#ifndef OPENSSL_NO_DEPRECATED + if (key_length == 512) { if (key == NULL) { key = RSA_generate_key(512, RSA_F4, NULL, NULL); } } +#endif + return key; } This won't change anything for normal builds, but will allow test builds with OPENSSL_NO_DEPRECATED defined. -- Maxim Dounin http://nginx.org/ From flygoast at 126.com Tue Jul 8 02:28:44 2014 From: flygoast at 126.com (flygoast) Date: Tue, 8 Jul 2014 10:28:44 +0800 (CST) Subject: [PATCH]Upstream: delete temp file when error in ngx_http_upstream_store(). In-Reply-To: <20140707162223.GG1849@mdounin.ru> References: <73504be7.fe2.147101f310c.Coremail.flygoast@126.com> <20140707162223.GG1849@mdounin.ru> Message-ID: <35616283.1abe.14713cd4061.Coremail.flygoast@126.com> Thanks for reply. When ngx_ext_rename_file() return NGX_ERROR, it should has called ngx_delete_file(). So I think maybe also should change "ext.delete_file = 1;" to "ext.delete_file = 0;". At 2014-07-08 00:22:23,"Maxim Dounin" wrote: >Hello! > >On Mon, Jul 07, 2014 at 05:19:45PM +0800, flygoast wrote: > >> # HG changeset patch >> # User FengGu >> # Date 1404723967 -28800 >> # Mon Jul 07 17:06:07 2014 +0800 >> # Node ID d1d597fbf6d8e08059a308d8900e5f90def5377a >> # Parent a680bf4dddd5c4b106419e3dfb0264815c401275 >> Upstream: delete temp file when error in ngx_http_upstream_store(). >> >> >> Previously, didn't process the case when ngx_http_map_uri_to_path() returned >> NULL. And when error occured, didn't delete the temp file. >> >> >> diff -r a680bf4dddd5 -r d1d597fbf6d8 src/http/ngx_http_upstream.c >> --- a/src/http/ngx_http_upstream.cSat Jul 05 23:29:47 2014 +0400 >> +++ b/src/http/ngx_http_upstream.cMon Jul 07 17:06:07 2014 +0800 >> @@ -3375,14 +3375,16 @@ >> >> if (u->conf->store_lengths == NULL) { >> >> - ngx_http_map_uri_to_path(r, &path, &root, 0); >> + if (ngx_http_map_uri_to_path(r, &path, &root, 0) == NULL) { >> + goto error; >> + } >> >> } else { >> if (ngx_http_script_run(r, &path, u->conf->store_lengths->elts, 0, >> u->conf->store_values->elts) >> == NULL) >> { >> - return; >> + goto error; >> } >> } >> >> @@ -3393,6 +3395,16 @@ >> tf->file.name.data, path.data); >> >> (void) ngx_ext_rename_file(&tf->file.name, &path, &ext); >> + >> + return; >> + >> +error: >> + >> + if (ngx_delete_file(tf->file.name.data) == NGX_FILE_ERROR) { >> + ngx_log_error(NGX_LOG_CRIT, r->connection->log, ngx_errno, >> + ngx_delete_file_n " \"%s\" failed", >> + tf->file.name.data); >> + } >> } >> > >Thanks for noting this. What do you think about the following patch >instead? > >There is a code in ngx_http_upstream_finalize_request() which >normally removes temporary files after errors with proxy_store, >but in this particular case it won't be triggered due to u->store >set to 0 even in case of errors. With this patch, u->store is set >to 0 only in if there are no errors, and thus temporary files are >properly removed in case of errors in ngx_http_upstream_store(). > >diff --git a/src/http/ngx_http_upstream.c b/src/http/ngx_http_upstream.c >--- a/src/http/ngx_http_upstream.c >+++ b/src/http/ngx_http_upstream.c >@@ -3253,7 +3253,6 @@ ngx_http_upstream_process_request(ngx_ht > || u->headers_in.content_length_n == tf->offset)) > { > ngx_http_upstream_store(r, u); >- u->store = 0; > } > } > } >@@ -3375,7 +3374,9 @@ ngx_http_upstream_store(ngx_http_request > > if (u->conf->store_lengths == NULL) { > >- ngx_http_map_uri_to_path(r, &path, &root, 0); >+ if (ngx_http_map_uri_to_path(r, &path, &root, 0) == NULL) { >+ return; >+ } > > } else { > if (ngx_http_script_run(r, &path, u->conf->store_lengths->elts, 0, >@@ -3392,7 +3393,11 @@ ngx_http_upstream_store(ngx_http_request > "upstream stores \"%s\" to \"%s\"", > tf->file.name.data, path.data); > >- (void) ngx_ext_rename_file(&tf->file.name, &path, &ext); >+ if (ngx_ext_rename_file(&tf->file.name, &path, &ext) != NGX_OK) { >+ return; >+ } >+ >+ u->store = 0; > } > > >-- >Maxim Dounin >http://nginx.org/ > >_______________________________________________ >nginx-devel mailing list >nginx-devel at nginx.org >http://mailman.nginx.org/mailman/listinfo/nginx-devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From piotr at cloudflare.com Tue Jul 8 09:29:21 2014 From: piotr at cloudflare.com (Piotr Sikora) Date: Tue, 08 Jul 2014 02:29:21 -0700 Subject: [PATCH] SPDY: fix support for headers with multiple values Message-ID: <0d4b16900e55bc3495fb.1404811761@Piotrs-MacBook-Pro.local> # HG changeset patch # User Piotr Sikora # Date 1404811064 25200 # Tue Jul 08 02:17:44 2014 -0700 # Node ID 0d4b16900e55bc3495fb63b3e93f06cac690497b # Parent a680bf4dddd5c4b106419e3dfb0264815c401275 SPDY: fix support for headers with multiple values. Split SPDY header with multiple, NULL-separated values: cookie: foo\0bar into two separate HTTP headers with the same name: cookie: foo cookie: bar Even though the logic for this behavior already existed in the source code, it doesn't look that it ever worked and SPDY streams with such headers were simply rejected. Signed-off-by: Piotr Sikora diff -r a680bf4dddd5 -r 0d4b16900e55 src/http/ngx_http_spdy.c --- a/src/http/ngx_http_spdy.c Sat Jul 05 23:29:47 2014 +0400 +++ b/src/http/ngx_http_spdy.c Tue Jul 08 02:17:44 2014 -0700 @@ -2584,6 +2584,8 @@ ngx_http_spdy_parse_header(ngx_http_requ r->header_end = p; r->header_in->pos = p + 1; + r->state = state; + return NGX_OK; } From vbart at nginx.com Tue Jul 8 09:44:30 2014 From: vbart at nginx.com (Valentin V. Bartenev) Date: Tue, 08 Jul 2014 13:44:30 +0400 Subject: [PATCH] SPDY: fix support for headers with multiple values In-Reply-To: <0d4b16900e55bc3495fb.1404811761@Piotrs-MacBook-Pro.local> References: <0d4b16900e55bc3495fb.1404811761@Piotrs-MacBook-Pro.local> Message-ID: <702409271.IPOsu0nb15@vbart-workstation> On Tuesday 08 July 2014 02:29:21 Piotr Sikora wrote: > # HG changeset patch > # User Piotr Sikora > # Date 1404811064 25200 > # Tue Jul 08 02:17:44 2014 -0700 > # Node ID 0d4b16900e55bc3495fb63b3e93f06cac690497b > # Parent a680bf4dddd5c4b106419e3dfb0264815c401275 > SPDY: fix support for headers with multiple values. > > Split SPDY header with multiple, NULL-separated values: > > cookie: foo\0bar > > into two separate HTTP headers with the same name: > > cookie: foo > cookie: bar > > Even though the logic for this behavior already existed > in the source code, it doesn't look that it ever worked > and SPDY streams with such headers were simply rejected. > > Signed-off-by: Piotr Sikora > > diff -r a680bf4dddd5 -r 0d4b16900e55 src/http/ngx_http_spdy.c > --- a/src/http/ngx_http_spdy.c Sat Jul 05 23:29:47 2014 +0400 > +++ b/src/http/ngx_http_spdy.c Tue Jul 08 02:17:44 2014 -0700 > @@ -2584,6 +2584,8 @@ ngx_http_spdy_parse_header(ngx_http_requ > r->header_end = p; > r->header_in->pos = p + 1; > > + r->state = state; > + > return NGX_OK; > } > Nice catch, thank you. Maxim, do you approve this for push? wbr, Valentin V. Bartenev From piotr at cloudflare.com Tue Jul 8 10:06:18 2014 From: piotr at cloudflare.com (Piotr Sikora) Date: Tue, 08 Jul 2014 03:06:18 -0700 Subject: [PATCH 2 of 5] Style: remove whitespace between function name and parentheses In-Reply-To: References: Message-ID: # HG changeset patch # User Piotr Sikora # Date 1404813794 25200 # Tue Jul 08 03:03:14 2014 -0700 # Node ID b478fc52fb96a20d72dba1b46ebcaaf24c7356d2 # Parent e00ca08c44e1e410e9a0ff3492b9419a2f32d80a Style: remove whitespace between function name and parentheses. Signed-off-by: Piotr Sikora diff -r e00ca08c44e1 -r b478fc52fb96 src/http/modules/ngx_http_geoip_module.c --- a/src/http/modules/ngx_http_geoip_module.c Tue Jul 08 03:03:13 2014 -0700 +++ b/src/http/modules/ngx_http_geoip_module.c Tue Jul 08 03:03:14 2014 -0700 @@ -691,7 +691,7 @@ ngx_http_geoip_country(ngx_conf_t *cf, n if (cf->args->nelts == 3) { if (ngx_strcmp(value[2].data, "utf8") == 0) { - GeoIP_set_charset (gcf->country, GEOIP_CHARSET_UTF8); + GeoIP_set_charset(gcf->country, GEOIP_CHARSET_UTF8); } else { ngx_conf_log_error(NGX_LOG_EMERG, cf, 0, @@ -746,7 +746,7 @@ ngx_http_geoip_org(ngx_conf_t *cf, ngx_c if (cf->args->nelts == 3) { if (ngx_strcmp(value[2].data, "utf8") == 0) { - GeoIP_set_charset (gcf->org, GEOIP_CHARSET_UTF8); + GeoIP_set_charset(gcf->org, GEOIP_CHARSET_UTF8); } else { ngx_conf_log_error(NGX_LOG_EMERG, cf, 0, @@ -807,7 +807,7 @@ ngx_http_geoip_city(ngx_conf_t *cf, ngx_ if (cf->args->nelts == 3) { if (ngx_strcmp(value[2].data, "utf8") == 0) { - GeoIP_set_charset (gcf->city, GEOIP_CHARSET_UTF8); + GeoIP_set_charset(gcf->city, GEOIP_CHARSET_UTF8); } else { ngx_conf_log_error(NGX_LOG_EMERG, cf, 0, diff -r e00ca08c44e1 -r b478fc52fb96 src/http/modules/ngx_http_scgi_module.c --- a/src/http/modules/ngx_http_scgi_module.c Tue Jul 08 03:03:13 2014 -0700 +++ b/src/http/modules/ngx_http_scgi_module.c Tue Jul 08 03:03:14 2014 -0700 @@ -1645,7 +1645,7 @@ ngx_http_scgi_pass(ngx_conf_t *cf, ngx_c return "is duplicate"; } - clcf = ngx_http_conf_get_module_loc_conf (cf, ngx_http_core_module); + clcf = ngx_http_conf_get_module_loc_conf(cf, ngx_http_core_module); clcf->handler = ngx_http_scgi_handler; value = cf->args->elts; diff -r e00ca08c44e1 -r b478fc52fb96 src/http/modules/ngx_http_uwsgi_module.c --- a/src/http/modules/ngx_http_uwsgi_module.c Tue Jul 08 03:03:13 2014 -0700 +++ b/src/http/modules/ngx_http_uwsgi_module.c Tue Jul 08 03:03:14 2014 -0700 @@ -1868,7 +1868,7 @@ ngx_http_uwsgi_pass(ngx_conf_t *cf, ngx_ return "is duplicate"; } - clcf = ngx_http_conf_get_module_loc_conf (cf, ngx_http_core_module); + clcf = ngx_http_conf_get_module_loc_conf(cf, ngx_http_core_module); clcf->handler = ngx_http_uwsgi_handler; value = cf->args->elts; From piotr at cloudflare.com Tue Jul 8 10:06:17 2014 From: piotr at cloudflare.com (Piotr Sikora) Date: Tue, 08 Jul 2014 03:06:17 -0700 Subject: [PATCH 1 of 5] Style: add whitespace between control statement and parentheses Message-ID: # HG changeset patch # User Piotr Sikora # Date 1404813793 25200 # Tue Jul 08 03:03:13 2014 -0700 # Node ID e00ca08c44e1e410e9a0ff3492b9419a2f32d80a # Parent a680bf4dddd5c4b106419e3dfb0264815c401275 Style: add whitespace between control statement and parentheses. Signed-off-by: Piotr Sikora diff -r a680bf4dddd5 -r e00ca08c44e1 src/http/modules/ngx_http_geo_module.c --- a/src/http/modules/ngx_http_geo_module.c Sat Jul 05 23:29:47 2014 +0400 +++ b/src/http/modules/ngx_http_geo_module.c Tue Jul 08 03:03:13 2014 -0700 @@ -1470,7 +1470,7 @@ ngx_http_geo_include_binary_base(ngx_con vv = (ngx_http_variable_value_t *) (base + sizeof(ngx_http_geo_header_t)); - while(vv->data) { + while (vv->data) { len = ngx_align(sizeof(ngx_http_variable_value_t) + vv->len, sizeof(void *)); ngx_crc32_update(&crc32, (u_char *) vv, len); diff -r a680bf4dddd5 -r e00ca08c44e1 src/http/ngx_http_core_module.c --- a/src/http/ngx_http_core_module.c Sat Jul 05 23:29:47 2014 +0400 +++ b/src/http/ngx_http_core_module.c Tue Jul 08 03:03:13 2014 -0700 @@ -2355,7 +2355,7 @@ ngx_http_gzip_accept_encoding(ngx_str_t p += 4; while (p < last) { - switch(*p++) { + switch (*p++) { case ',': return NGX_OK; case ';': @@ -2372,7 +2372,7 @@ ngx_http_gzip_accept_encoding(ngx_str_t quantity: while (p < last) { - switch(*p++) { + switch (*p++) { case 'q': case 'Q': goto equal; diff -r a680bf4dddd5 -r e00ca08c44e1 src/http/ngx_http_parse.c --- a/src/http/ngx_http_parse.c Sat Jul 05 23:29:47 2014 +0400 +++ b/src/http/ngx_http_parse.c Tue Jul 08 03:03:13 2014 -0700 @@ -1287,7 +1287,7 @@ ngx_http_parse_complex_uri(ngx_http_requ break; } - switch(ch) { + switch (ch) { #if (NGX_WIN32) case '\\': if (u - 2 >= r->uri.data @@ -1357,7 +1357,7 @@ ngx_http_parse_complex_uri(ngx_http_requ break; } - switch(ch) { + switch (ch) { #if (NGX_WIN32) case '\\': break; @@ -1400,7 +1400,7 @@ ngx_http_parse_complex_uri(ngx_http_requ break; } - switch(ch) { + switch (ch) { #if (NGX_WIN32) case '\\': #endif @@ -1441,7 +1441,7 @@ ngx_http_parse_complex_uri(ngx_http_requ break; } - switch(ch) { + switch (ch) { #if (NGX_WIN32) case '\\': #endif diff -r a680bf4dddd5 -r e00ca08c44e1 src/http/ngx_http_upstream.c --- a/src/http/ngx_http_upstream.c Sat Jul 05 23:29:47 2014 +0400 +++ b/src/http/ngx_http_upstream.c Tue Jul 08 03:03:13 2014 -0700 @@ -3441,7 +3441,7 @@ ngx_http_upstream_next(ngx_http_request_ u->peer.tries++; } else { - switch(ft_type) { + switch (ft_type) { case NGX_HTTP_UPSTREAM_FT_TIMEOUT: status = NGX_HTTP_GATEWAY_TIME_OUT; diff -r a680bf4dddd5 -r e00ca08c44e1 src/os/win32/ngx_service.c --- a/src/os/win32/ngx_service.c Sat Jul 05 23:29:47 2014 +0400 +++ b/src/os/win32/ngx_service.c Tue Jul 08 03:03:13 2014 -0700 @@ -89,7 +89,7 @@ u_int service_handler(u_int control, u_i { /* primary thread */ - switch(control) { + switch (control) { case SERVICE_CONTROL_INTERROGATE: status = NGX_IOCP_INTERROGATE; From piotr at cloudflare.com Tue Jul 8 10:06:20 2014 From: piotr at cloudflare.com (Piotr Sikora) Date: Tue, 08 Jul 2014 03:06:20 -0700 Subject: [PATCH 4 of 5] Style: use ngx_memcpy() instead of memcpy() In-Reply-To: References: Message-ID: <0265cb73cd77b551541a.1404813980@Piotrs-MacBook-Pro.local> # HG changeset patch # User Piotr Sikora # Date 1404813794 25200 # Tue Jul 08 03:03:14 2014 -0700 # Node ID 0265cb73cd77b551541a05db6894f3e03cbe5083 # Parent 255c428fcd007d6aea07d51d5d010c2a4cefe966 Style: use ngx_memcpy() instead of memcpy(). Signed-off-by: Piotr Sikora diff -r 255c428fcd00 -r 0265cb73cd77 src/event/ngx_event_openssl.c --- a/src/event/ngx_event_openssl.c Tue Jul 08 03:03:14 2014 -0700 +++ b/src/event/ngx_event_openssl.c Tue Jul 08 03:03:14 2014 -0700 @@ -2638,7 +2638,7 @@ ngx_ssl_session_ticket_key_callback(ngx_ EVP_EncryptInit_ex(ectx, EVP_aes_128_cbc(), NULL, key[0].aes_key, iv); HMAC_Init_ex(hctx, key[0].hmac_key, 16, ngx_ssl_session_ticket_md(), NULL); - memcpy(name, key[0].name, 16); + ngx_memcpy(name, key[0].name, 16); return 0; From piotr at cloudflare.com Tue Jul 8 10:06:19 2014 From: piotr at cloudflare.com (Piotr Sikora) Date: Tue, 08 Jul 2014 03:06:19 -0700 Subject: [PATCH 3 of 5] Style: keep return type and function name on different lines In-Reply-To: References: Message-ID: <255c428fcd007d6aea07.1404813979@Piotrs-MacBook-Pro.local> # HG changeset patch # User Piotr Sikora # Date 1404813794 25200 # Tue Jul 08 03:03:14 2014 -0700 # Node ID 255c428fcd007d6aea07d51d5d010c2a4cefe966 # Parent b478fc52fb96a20d72dba1b46ebcaaf24c7356d2 Style: keep return type and function name on different lines. Signed-off-by: Piotr Sikora diff -r b478fc52fb96 -r 255c428fcd00 src/os/win32/ngx_service.c --- a/src/os/win32/ngx_service.c Tue Jul 08 03:03:14 2014 -0700 +++ b/src/os/win32/ngx_service.c Tue Jul 08 03:03:14 2014 -0700 @@ -16,7 +16,8 @@ SERVICE_TABLE_ENTRY st[] = { }; -ngx_int_t ngx_service(ngx_log_t *log) +ngx_int_t +ngx_service(ngx_log_t *log) { /* primary thread */ @@ -32,7 +33,8 @@ ngx_int_t ngx_service(ngx_log_t *log) } -void service_main(u_int argc, char **argv) +void +service_main(u_int argc, char **argv) { SERVICE_STATUS status; SERVICE_STATUS_HANDLE service; @@ -85,7 +87,8 @@ void service_main(u_int argc, char **arg } -u_int service_handler(u_int control, u_int type, void *data, void *ctx) +u_int +service_handler(u_int control, u_int type, void *data, void *ctx) { /* primary thread */ From piotr at cloudflare.com Tue Jul 8 10:06:21 2014 From: piotr at cloudflare.com (Piotr Sikora) Date: Tue, 08 Jul 2014 03:06:21 -0700 Subject: [PATCH 5 of 5] Style: fix typo In-Reply-To: References: Message-ID: <35f7eb512945e2b7d39d.1404813981@Piotrs-MacBook-Pro.local> # HG changeset patch # User Piotr Sikora # Date 1404813794 25200 # Tue Jul 08 03:03:14 2014 -0700 # Node ID 35f7eb512945e2b7d39ddbedde57dc40dfac1abe # Parent 0265cb73cd77b551541a05db6894f3e03cbe5083 Style: fix typo. Signed-off-by: Piotr Sikora diff -r 0265cb73cd77 -r 35f7eb512945 src/misc/ngx_cpp_test_module.cpp --- a/src/misc/ngx_cpp_test_module.cpp Tue Jul 08 03:03:14 2014 -0700 +++ b/src/misc/ngx_cpp_test_module.cpp Tue Jul 08 03:03:14 2014 -0700 @@ -1,5 +1,5 @@ -// stub module to test header files' C++ compatibilty +// stub module to test header files' C++ compatibility extern "C" { #include From vbart at nginx.com Tue Jul 8 10:07:27 2014 From: vbart at nginx.com (Valentin V. Bartenev) Date: Tue, 08 Jul 2014 14:07:27 +0400 Subject: [PATCH] SPDY: fix support for headers with multiple values In-Reply-To: <702409271.IPOsu0nb15@vbart-workstation> References: <0d4b16900e55bc3495fb.1404811761@Piotrs-MacBook-Pro.local> <702409271.IPOsu0nb15@vbart-workstation> Message-ID: <1516744.9SXB61dCXs@vbart-workstation> On Tuesday 08 July 2014 13:44:30 Valentin V. Bartenev wrote: > On Tuesday 08 July 2014 02:29:21 Piotr Sikora wrote: > > # HG changeset patch > > # User Piotr Sikora > > # Date 1404811064 25200 > > # Tue Jul 08 02:17:44 2014 -0700 > > # Node ID 0d4b16900e55bc3495fb63b3e93f06cac690497b > > # Parent a680bf4dddd5c4b106419e3dfb0264815c401275 > > SPDY: fix support for headers with multiple values. > > > > Split SPDY header with multiple, NULL-separated values: > > > > cookie: foo\0bar > > > > into two separate HTTP headers with the same name: > > > > cookie: foo > > cookie: bar > > > > Even though the logic for this behavior already existed > > in the source code, it doesn't look that it ever worked > > and SPDY streams with such headers were simply rejected. > > > > Signed-off-by: Piotr Sikora > > > > diff -r a680bf4dddd5 -r 0d4b16900e55 src/http/ngx_http_spdy.c > > --- a/src/http/ngx_http_spdy.c Sat Jul 05 23:29:47 2014 +0400 > > +++ b/src/http/ngx_http_spdy.c Tue Jul 08 02:17:44 2014 -0700 > > @@ -2584,6 +2584,8 @@ ngx_http_spdy_parse_header(ngx_http_requ > > r->header_end = p; > > r->header_in->pos = p + 1; > > > > + r->state = state; > > + > > return NGX_OK; > > } > > > > Nice catch, thank you. > > Maxim, do you approve this for push? > [..] Probably, it's better to use constant here (like r->state = 0 in the same function): diff -r 9d3a9c45fc43 src/http/ngx_http_spdy.c --- a/src/http/ngx_http_spdy.c Tue Jul 01 20:52:08 2014 +0400 +++ b/src/http/ngx_http_spdy.c Tue Jul 08 13:55:52 2014 +0400 @@ -2584,6 +2584,8 @@ ngx_http_spdy_parse_header(ngx_http_requ r->header_end = p; r->header_in->pos = p + 1; + r->state = sw_value; + return NGX_OK; } From piotr at cloudflare.com Tue Jul 8 10:12:51 2014 From: piotr at cloudflare.com (Piotr Sikora) Date: Tue, 8 Jul 2014 03:12:51 -0700 Subject: [PATCH] SPDY: fix support for headers with multiple values In-Reply-To: <1516744.9SXB61dCXs@vbart-workstation> References: <0d4b16900e55bc3495fb.1404811761@Piotrs-MacBook-Pro.local> <702409271.IPOsu0nb15@vbart-workstation> <1516744.9SXB61dCXs@vbart-workstation> Message-ID: Hey Valentin, > Probably, it's better to use constant here (like r->state = 0 in > the same function): > > diff -r 9d3a9c45fc43 src/http/ngx_http_spdy.c > --- a/src/http/ngx_http_spdy.c Tue Jul 01 20:52:08 2014 +0400 > +++ b/src/http/ngx_http_spdy.c Tue Jul 08 13:55:52 2014 +0400 > @@ -2584,6 +2584,8 @@ ngx_http_spdy_parse_header(ngx_http_requ > r->header_end = p; > r->header_in->pos = p + 1; > > + r->state = sw_value; > + > return NGX_OK; > } > > That's fine with me. Best regards, Piotr Sikora From piotr at cloudflare.com Tue Jul 8 10:22:21 2014 From: piotr at cloudflare.com (Piotr Sikora) Date: Tue, 8 Jul 2014 03:22:21 -0700 Subject: [PATCH 3 of 4] SSL: stop using deprecated RSA_generate_key() function In-Reply-To: <20140707233224.GN1849@mdounin.ru> References: <8c3e64571adf1ff45fa4.1404690648@Piotrs-MacBook-Pro.local> <20140707011315.GZ1849@mdounin.ru> <20140707130449.GC1849@mdounin.ru> <20140707233224.GN1849@mdounin.ru> Message-ID: Hey Maxim, > I don't think it's time to remove it, but as I previously said, I > will be fine with something like this: > > diff --git a/src/event/ngx_event_openssl.c b/src/event/ngx_event_openssl.c > --- a/src/event/ngx_event_openssl.c > +++ b/src/event/ngx_event_openssl.c > @@ -652,12 +652,16 @@ ngx_ssl_rsa512_key_callback(ngx_ssl_conn > { > static RSA *key; > > +#ifndef OPENSSL_NO_DEPRECATED > + > if (key_length == 512) { > if (key == NULL) { > key = RSA_generate_key(512, RSA_F4, NULL, NULL); > } > } > > +#endif > + > return key; > } > > > This won't change anything for normal builds, but will allow test > builds with OPENSSL_NO_DEPRECATED defined. I'd prefer to see the RSA_generate_key_ex() being used (even though it's more complex interface), but I don't care enough to fight over this, so I guess your patch is "good enough", however I'd like to see my patch #2 applied first (i.e. if key_length != 512 return NULL), so that only RSA_generate_key() is being guarded. Best regards, Piotr Sikora From mdounin at mdounin.ru Tue Jul 8 11:47:34 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Tue, 8 Jul 2014 15:47:34 +0400 Subject: [PATCH]Upstream: delete temp file when error in ngx_http_upstream_store(). In-Reply-To: <35616283.1abe.14713cd4061.Coremail.flygoast@126.com> References: <73504be7.fe2.147101f310c.Coremail.flygoast@126.com> <20140707162223.GG1849@mdounin.ru> <35616283.1abe.14713cd4061.Coremail.flygoast@126.com> Message-ID: <20140708114734.GP1849@mdounin.ru> Hello! On Tue, Jul 08, 2014 at 10:28:44AM +0800, flygoast wrote: > Thanks for reply. > When ngx_ext_rename_file() return NGX_ERROR, it should has called ngx_delete_file(). > So I think maybe also should change "ext.delete_file = 1;" to "ext.delete_file = 0;". Thanks, missed this. Probably it would be easier to just leave the "(void) ngx_ext_rename_file" as is then. -- Maxim Dounin http://nginx.org/ From mdounin at mdounin.ru Tue Jul 8 13:32:32 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Tue, 08 Jul 2014 13:32:32 +0000 Subject: [nginx] nginx-1.7.3-RELEASE Message-ID: details: http://hg.nginx.org/nginx/rev/fe7cd01828d5 branches: changeset: 5750:fe7cd01828d5 user: Maxim Dounin date: Tue Jul 08 17:22:38 2014 +0400 description: nginx-1.7.3-RELEASE diffstat: docs/xml/nginx/changes.xml | 80 ++++++++++++++++++++++++++++++++++++++++++++++ 1 files changed, 80 insertions(+), 0 deletions(-) diffs (90 lines): diff --git a/docs/xml/nginx/changes.xml b/docs/xml/nginx/changes.xml --- a/docs/xml/nginx/changes.xml +++ b/docs/xml/nginx/changes.xml @@ -5,6 +5,86 @@ + + + + +weak entity tags ?????? ?? ????????? ??? ?????????? ??????, +? strong entity tags ????????????? ? weak. + + +weak entity tags are now preserved on response modifications, +and strong ones are changed to weak. + + + + + +??????????? ????????? ???? ??????, ???? ??? ????????, +?????????? ????????? If-None-Match. + + +cache revalidation now uses If-None-Match header +if possible. + + + + + +????????? ssl_password_file. + + +the "ssl_password_file" directive. + + + + + +??? ???????? ?????? ?? ???? +????????? ??????? If-None-Match ?????????????, +???? ? ?????? ?? ???? ????????? Last-Modified. + + +the If-None-Match request header line was ignored +if there was no Last-Modified header +in a response returned from cache. + + + + + +????????? "peer closed connection in SSL handshake" +??? ?????????? ? ????????? ????????????? ?? ?????? info ?????? error. + + +"peer closed connection in SSL handshake" messages +were logged at "info" level instead of "error" while connecting to backends. + + + + + +? ?????? ngx_http_dav_module ? nginx/Windows. + + +in the ngx_http_dav_module module in nginx/Windows. + + + + + +SPDY-?????????? ????? ?????????? ???????????, +???? ?????????????? ???????????. + + +SPDY connections might be closed prematurely +if caching was used. + + + + + + From mdounin at mdounin.ru Tue Jul 8 13:32:35 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Tue, 08 Jul 2014 13:32:35 +0000 Subject: [nginx] release-1.7.3 tag Message-ID: details: http://hg.nginx.org/nginx/rev/24833eba29c2 branches: changeset: 5751:24833eba29c2 user: Maxim Dounin date: Tue Jul 08 17:22:38 2014 +0400 description: release-1.7.3 tag diffstat: .hgtags | 1 + 1 files changed, 1 insertions(+), 0 deletions(-) diffs (8 lines): diff --git a/.hgtags b/.hgtags --- a/.hgtags +++ b/.hgtags @@ -372,3 +372,4 @@ fd722b890eabc600394349730a093f50dac31639 d161d68df8be32e5cbf72b07db1a707714827803 release-1.7.0 0351a6d89c3dbcc7a76295024ba6b70e27b9a497 release-1.7.1 0bd223a546192fdf2e862f33938f4ec2a3b5b283 release-1.7.2 +fe7cd01828d5ca7491059f0690bb4453645eb28b release-1.7.3 From grrm77 at gmail.com Tue Jul 8 20:16:47 2014 From: grrm77 at gmail.com (grrm grrm) Date: Tue, 8 Jul 2014 23:16:47 +0300 Subject: Patch: Refactor ngx_http_write_request_body into a filter In-Reply-To: <20140707180449.GH1849@mdounin.ru> References: <20140620180926.GO1849@mdounin.ru> <20140627135419.GR1849@mdounin.ru> <20140707180449.GH1849@mdounin.ru> Message-ID: Hello, Thanks for the reply. I understand the hesitation to include new code with changes more than cosmetic, but without any real benefit. What if this was part of a greater initiative to implement non-buffered request body processing. Would that be an acceptable modification to start with? (Because that was tentatively my intention). Thank you. 2014-07-07 21:04 GMT+03:00 Maxim Dounin : > Hello! > > On Mon, Jul 07, 2014 at 08:23:20PM +0300, grrm grrm wrote: > >> Did you have a chance to look at the code? > > I still don't think that suggested change improves the code. > > -- > Maxim Dounin > http://nginx.org/ > > _______________________________________________ > nginx-devel mailing list > nginx-devel at nginx.org > http://mailman.nginx.org/mailman/listinfo/nginx-devel From mdounin at mdounin.ru Wed Jul 9 01:49:36 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Wed, 09 Jul 2014 01:49:36 +0000 Subject: [nginx] Version bump. Message-ID: details: http://hg.nginx.org/nginx/rev/ebcd01d080c9 branches: changeset: 5752:ebcd01d080c9 user: Maxim Dounin date: Wed Jul 09 03:15:00 2014 +0400 description: Version bump. diffstat: src/core/nginx.h | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) diffs (14 lines): diff --git a/src/core/nginx.h b/src/core/nginx.h --- a/src/core/nginx.h +++ b/src/core/nginx.h @@ -9,8 +9,8 @@ #define _NGINX_H_INCLUDED_ -#define nginx_version 1007003 -#define NGINX_VERSION "1.7.3" +#define nginx_version 1007004 +#define NGINX_VERSION "1.7.4" #define NGINX_VER "nginx/" NGINX_VERSION #ifdef NGX_BUILD From mdounin at mdounin.ru Wed Jul 9 01:49:39 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Wed, 09 Jul 2014 01:49:39 +0000 Subject: [nginx] SSL: include correct OpenSSL headers. Message-ID: details: http://hg.nginx.org/nginx/rev/febce92c82f6 branches: changeset: 5753:febce92c82f6 user: Piotr Sikora date: Sun Jul 06 16:41:13 2014 -0700 description: SSL: include correct OpenSSL headers. Previously, , , and were pulled in by using OpenSSL's deprecated interface, which meant that nginx couldn't have been built with -DOPENSSL_NO_DEPRECATED. Both and are pulled in by , but we're calling X509 functions directly, so let's include those as well. is pulled in by virtually everything, but we're calling CRYPTO_add() directly, so let's include it as well. Signed-off-by: Piotr Sikora diffstat: src/event/ngx_event_openssl.h | 7 +++++++ 1 files changed, 7 insertions(+), 0 deletions(-) diffs (21 lines): diff --git a/src/event/ngx_event_openssl.h b/src/event/ngx_event_openssl.h --- a/src/event/ngx_event_openssl.h +++ b/src/event/ngx_event_openssl.h @@ -14,10 +14,17 @@ #include #include +#include #include +#include +#include #include #include #include +#include +#include +#include +#include #define NGX_SSL_NAME "OpenSSL" From mdounin at mdounin.ru Wed Jul 9 01:49:42 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Wed, 09 Jul 2014 01:49:42 +0000 Subject: [nginx] SSL: return temporary RSA key only when the key length m... Message-ID: details: http://hg.nginx.org/nginx/rev/c7ecd097b883 branches: changeset: 5754:c7ecd097b883 user: Piotr Sikora date: Sun Jul 06 16:41:13 2014 -0700 description: SSL: return temporary RSA key only when the key length matches. This change is mostly cosmetic, because in practice this callback is used only for 512-bit RSA keys. Signed-off-by: Piotr Sikora diffstat: src/event/ngx_event_openssl.c | 10 ++++++---- 1 files changed, 6 insertions(+), 4 deletions(-) diffs (20 lines): diff --git a/src/event/ngx_event_openssl.c b/src/event/ngx_event_openssl.c --- a/src/event/ngx_event_openssl.c +++ b/src/event/ngx_event_openssl.c @@ -652,10 +652,12 @@ ngx_ssl_rsa512_key_callback(ngx_ssl_conn { static RSA *key; - if (key_length == 512) { - if (key == NULL) { - key = RSA_generate_key(512, RSA_F4, NULL, NULL); - } + if (key_length != 512) { + return NULL; + } + + if (key == NULL) { + key = RSA_generate_key(512, RSA_F4, NULL, NULL); } return key; From mdounin at mdounin.ru Wed Jul 9 01:49:45 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Wed, 09 Jul 2014 01:49:45 +0000 Subject: [nginx] SSL: fixed build with OPENSSL_NO_DEPRECATED defined. Message-ID: details: http://hg.nginx.org/nginx/rev/8df08465fcfd branches: changeset: 5755:8df08465fcfd user: Maxim Dounin date: Wed Jul 09 04:08:13 2014 +0400 description: SSL: fixed build with OPENSSL_NO_DEPRECATED defined. The RSA_generate_key() is marked as deprecated and causes build to fail. On the other hand, replacement function, RSA_generate_key_ex(), requires much more code. Since RSA_generate_key() is only needed for barely usable EXP ciphers, the #ifdef was added instead. Prodded by Piotr Sikora. diffstat: src/event/ngx_event_openssl.c | 4 ++++ 1 files changed, 4 insertions(+), 0 deletions(-) diffs (18 lines): diff --git a/src/event/ngx_event_openssl.c b/src/event/ngx_event_openssl.c --- a/src/event/ngx_event_openssl.c +++ b/src/event/ngx_event_openssl.c @@ -656,10 +656,14 @@ ngx_ssl_rsa512_key_callback(ngx_ssl_conn return NULL; } +#ifndef OPENSSL_NO_DEPRECATED + if (key == NULL) { key = RSA_generate_key(512, RSA_F4, NULL, NULL); } +#endif + return key; } From mdounin at mdounin.ru Wed Jul 9 01:49:49 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Wed, 09 Jul 2014 01:49:49 +0000 Subject: [nginx] SSL: stop accessing SSL_SESSION's fields directly. Message-ID: details: http://hg.nginx.org/nginx/rev/5b7276408565 branches: changeset: 5756:5b7276408565 user: Piotr Sikora date: Sun Jul 06 16:41:14 2014 -0700 description: SSL: stop accessing SSL_SESSION's fields directly. SSL_SESSION struct is internal part of the OpenSSL library and it's fields should be accessed via API (when exposed), not directly. The unfortunate side-effect of this change is that we're losing reference count that used to be printed at the debug log level, but this seems to be an acceptable trade-off. Almost fixes build with -DOPENSSL_NO_SSL_INTERN. Signed-off-by: Piotr Sikora diffstat: src/event/ngx_event_openssl.c | 58 +++++++++++++++++++++++-------- src/http/ngx_http_upstream_round_robin.c | 14 +++---- 2 files changed, 49 insertions(+), 23 deletions(-) diffs (178 lines): diff --git a/src/event/ngx_event_openssl.c b/src/event/ngx_event_openssl.c --- a/src/event/ngx_event_openssl.c +++ b/src/event/ngx_event_openssl.c @@ -2078,9 +2078,10 @@ static int ngx_ssl_new_session(ngx_ssl_conn_t *ssl_conn, ngx_ssl_session_t *sess) { int len; - u_char *p, *id, *cached_sess; + u_char *p, *id, *cached_sess, *session_id; uint32_t hash; SSL_CTX *ssl_ctx; + unsigned int session_id_length; ngx_shm_zone_t *shm_zone; ngx_connection_t *c; ngx_slab_pool_t *shpool; @@ -2143,13 +2144,24 @@ ngx_ssl_new_session(ngx_ssl_conn_t *ssl_ } } +#if OPENSSL_VERSION_NUMBER >= 0x0090800fL + + session_id = (u_char *) SSL_SESSION_get_id(sess, &session_id_length); + +#else + + session_id = sess->session_id; + session_id_length = sess->session_id_length; + +#endif + #if (NGX_PTR_SIZE == 8) id = sess_id->sess_id; #else - id = ngx_slab_alloc_locked(shpool, sess->session_id_length); + id = ngx_slab_alloc_locked(shpool, session_id_length); if (id == NULL) { @@ -2157,7 +2169,7 @@ ngx_ssl_new_session(ngx_ssl_conn_t *ssl_ ngx_ssl_expire_sessions(cache, shpool, 0); - id = ngx_slab_alloc_locked(shpool, sess->session_id_length); + id = ngx_slab_alloc_locked(shpool, session_id_length); if (id == NULL) { goto failed; @@ -2168,16 +2180,16 @@ ngx_ssl_new_session(ngx_ssl_conn_t *ssl_ ngx_memcpy(cached_sess, buf, len); - ngx_memcpy(id, sess->session_id, sess->session_id_length); - - hash = ngx_crc32_short(sess->session_id, sess->session_id_length); + ngx_memcpy(id, session_id, session_id_length); + + hash = ngx_crc32_short(session_id, session_id_length); ngx_log_debug3(NGX_LOG_DEBUG_EVENT, c->log, 0, - "ssl new session: %08XD:%d:%d", - hash, sess->session_id_length, len); + "ssl new session: %08XD:%ud:%d", + hash, session_id_length, len); sess_id->node.key = hash; - sess_id->node.data = (u_char) sess->session_id_length; + sess_id->node.data = (u_char) session_id_length; sess_id->id = id; sess_id->len = len; sess_id->session = cached_sess; @@ -2325,10 +2337,10 @@ ngx_ssl_remove_cached_session(SSL_CTX *s static void ngx_ssl_remove_session(SSL_CTX *ssl, ngx_ssl_session_t *sess) { - size_t len; u_char *id; uint32_t hash; ngx_int_t rc; + unsigned int len; ngx_shm_zone_t *shm_zone; ngx_slab_pool_t *shpool; ngx_rbtree_node_t *node, *sentinel; @@ -2343,13 +2355,21 @@ ngx_ssl_remove_session(SSL_CTX *ssl, ngx cache = shm_zone->data; +#if OPENSSL_VERSION_NUMBER >= 0x0090800fL + + id = (u_char *) SSL_SESSION_get_id(sess, &len); + +#else + id = sess->session_id; - len = (size_t) sess->session_id_length; + len = sess->session_id_length; + +#endif hash = ngx_crc32_short(id, len); ngx_log_debug2(NGX_LOG_DEBUG_EVENT, ngx_cycle->log, 0, - "ssl remove session: %08XD:%uz", hash, len); + "ssl remove session: %08XD:%ud", hash, len); shpool = (ngx_slab_pool_t *) shm_zone->shm.addr; @@ -2891,9 +2911,9 @@ ngx_ssl_get_cipher_name(ngx_connection_t ngx_int_t ngx_ssl_get_session_id(ngx_connection_t *c, ngx_pool_t *pool, ngx_str_t *s) { - int len; - u_char *buf; - SSL_SESSION *sess; + u_char *buf; + SSL_SESSION *sess; + unsigned int len; sess = SSL_get0_session(c->ssl->connection); if (sess == NULL) { @@ -2901,9 +2921,17 @@ ngx_ssl_get_session_id(ngx_connection_t return NGX_OK; } +#if OPENSSL_VERSION_NUMBER >= 0x0090800fL + + buf = (u_char *) SSL_SESSION_get_id(sess, &len); + +#else + buf = sess->session_id; len = sess->session_id_length; +#endif + s->len = 2 * len; s->data = ngx_pnalloc(pool, 2 * len); if (s->data == NULL) { diff --git a/src/http/ngx_http_upstream_round_robin.c b/src/http/ngx_http_upstream_round_robin.c --- a/src/http/ngx_http_upstream_round_robin.c +++ b/src/http/ngx_http_upstream_round_robin.c @@ -632,9 +632,8 @@ ngx_http_upstream_set_round_robin_peer_s rc = ngx_ssl_set_session(pc->connection, ssl_session); - ngx_log_debug2(NGX_LOG_DEBUG_HTTP, pc->log, 0, - "set session: %p:%d", - ssl_session, ssl_session ? ssl_session->references : 0); + ngx_log_debug1(NGX_LOG_DEBUG_HTTP, pc->log, 0, + "set session: %p", ssl_session); /* ngx_unlock_mutex(rrp->peers->mutex); */ @@ -657,8 +656,8 @@ ngx_http_upstream_save_round_robin_peer_ return; } - ngx_log_debug2(NGX_LOG_DEBUG_HTTP, pc->log, 0, - "save session: %p:%d", ssl_session, ssl_session->references); + ngx_log_debug1(NGX_LOG_DEBUG_HTTP, pc->log, 0, + "save session: %p", ssl_session); peer = &rrp->peers->peer[rrp->current]; @@ -672,9 +671,8 @@ ngx_http_upstream_save_round_robin_peer_ if (old_ssl_session) { - ngx_log_debug2(NGX_LOG_DEBUG_HTTP, pc->log, 0, - "old session: %p:%d", - old_ssl_session, old_ssl_session->references); + ngx_log_debug1(NGX_LOG_DEBUG_HTTP, pc->log, 0, + "old session: %p", old_ssl_session); /* TODO: may block */ From mdounin at mdounin.ru Wed Jul 9 01:49:52 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Wed, 09 Jul 2014 01:49:52 +0000 Subject: [nginx] Style: add whitespace between control statement and pare... Message-ID: details: http://hg.nginx.org/nginx/rev/c5ec6944de98 branches: changeset: 5757:c5ec6944de98 user: Piotr Sikora date: Tue Jul 08 03:03:13 2014 -0700 description: Style: add whitespace between control statement and parentheses. Signed-off-by: Piotr Sikora diffstat: src/http/modules/ngx_http_geo_module.c | 2 +- src/http/ngx_http_core_module.c | 4 ++-- src/http/ngx_http_parse.c | 8 ++++---- src/http/ngx_http_upstream.c | 2 +- src/os/win32/ngx_service.c | 2 +- 5 files changed, 9 insertions(+), 9 deletions(-) diffs (96 lines): diff --git a/src/http/modules/ngx_http_geo_module.c b/src/http/modules/ngx_http_geo_module.c --- a/src/http/modules/ngx_http_geo_module.c +++ b/src/http/modules/ngx_http_geo_module.c @@ -1470,7 +1470,7 @@ ngx_http_geo_include_binary_base(ngx_con vv = (ngx_http_variable_value_t *) (base + sizeof(ngx_http_geo_header_t)); - while(vv->data) { + while (vv->data) { len = ngx_align(sizeof(ngx_http_variable_value_t) + vv->len, sizeof(void *)); ngx_crc32_update(&crc32, (u_char *) vv, len); diff --git a/src/http/ngx_http_core_module.c b/src/http/ngx_http_core_module.c --- a/src/http/ngx_http_core_module.c +++ b/src/http/ngx_http_core_module.c @@ -2355,7 +2355,7 @@ ngx_http_gzip_accept_encoding(ngx_str_t p += 4; while (p < last) { - switch(*p++) { + switch (*p++) { case ',': return NGX_OK; case ';': @@ -2372,7 +2372,7 @@ ngx_http_gzip_accept_encoding(ngx_str_t quantity: while (p < last) { - switch(*p++) { + switch (*p++) { case 'q': case 'Q': goto equal; diff --git a/src/http/ngx_http_parse.c b/src/http/ngx_http_parse.c --- a/src/http/ngx_http_parse.c +++ b/src/http/ngx_http_parse.c @@ -1287,7 +1287,7 @@ ngx_http_parse_complex_uri(ngx_http_requ break; } - switch(ch) { + switch (ch) { #if (NGX_WIN32) case '\\': if (u - 2 >= r->uri.data @@ -1357,7 +1357,7 @@ ngx_http_parse_complex_uri(ngx_http_requ break; } - switch(ch) { + switch (ch) { #if (NGX_WIN32) case '\\': break; @@ -1400,7 +1400,7 @@ ngx_http_parse_complex_uri(ngx_http_requ break; } - switch(ch) { + switch (ch) { #if (NGX_WIN32) case '\\': #endif @@ -1441,7 +1441,7 @@ ngx_http_parse_complex_uri(ngx_http_requ break; } - switch(ch) { + switch (ch) { #if (NGX_WIN32) case '\\': #endif diff --git a/src/http/ngx_http_upstream.c b/src/http/ngx_http_upstream.c --- a/src/http/ngx_http_upstream.c +++ b/src/http/ngx_http_upstream.c @@ -3441,7 +3441,7 @@ ngx_http_upstream_next(ngx_http_request_ u->peer.tries++; } else { - switch(ft_type) { + switch (ft_type) { case NGX_HTTP_UPSTREAM_FT_TIMEOUT: status = NGX_HTTP_GATEWAY_TIME_OUT; diff --git a/src/os/win32/ngx_service.c b/src/os/win32/ngx_service.c --- a/src/os/win32/ngx_service.c +++ b/src/os/win32/ngx_service.c @@ -89,7 +89,7 @@ u_int service_handler(u_int control, u_i { /* primary thread */ - switch(control) { + switch (control) { case SERVICE_CONTROL_INTERROGATE: status = NGX_IOCP_INTERROGATE; From mdounin at mdounin.ru Wed Jul 9 01:49:56 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Wed, 09 Jul 2014 01:49:56 +0000 Subject: [nginx] Style: remove whitespace between function name and paren... Message-ID: details: http://hg.nginx.org/nginx/rev/f3df4e420ae7 branches: changeset: 5758:f3df4e420ae7 user: Piotr Sikora date: Tue Jul 08 03:03:14 2014 -0700 description: Style: remove whitespace between function name and parentheses. Signed-off-by: Piotr Sikora diffstat: src/http/modules/ngx_http_geoip_module.c | 6 +++--- src/http/modules/ngx_http_scgi_module.c | 2 +- src/http/modules/ngx_http_uwsgi_module.c | 2 +- 3 files changed, 5 insertions(+), 5 deletions(-) diffs (54 lines): diff --git a/src/http/modules/ngx_http_geoip_module.c b/src/http/modules/ngx_http_geoip_module.c --- a/src/http/modules/ngx_http_geoip_module.c +++ b/src/http/modules/ngx_http_geoip_module.c @@ -691,7 +691,7 @@ ngx_http_geoip_country(ngx_conf_t *cf, n if (cf->args->nelts == 3) { if (ngx_strcmp(value[2].data, "utf8") == 0) { - GeoIP_set_charset (gcf->country, GEOIP_CHARSET_UTF8); + GeoIP_set_charset(gcf->country, GEOIP_CHARSET_UTF8); } else { ngx_conf_log_error(NGX_LOG_EMERG, cf, 0, @@ -746,7 +746,7 @@ ngx_http_geoip_org(ngx_conf_t *cf, ngx_c if (cf->args->nelts == 3) { if (ngx_strcmp(value[2].data, "utf8") == 0) { - GeoIP_set_charset (gcf->org, GEOIP_CHARSET_UTF8); + GeoIP_set_charset(gcf->org, GEOIP_CHARSET_UTF8); } else { ngx_conf_log_error(NGX_LOG_EMERG, cf, 0, @@ -807,7 +807,7 @@ ngx_http_geoip_city(ngx_conf_t *cf, ngx_ if (cf->args->nelts == 3) { if (ngx_strcmp(value[2].data, "utf8") == 0) { - GeoIP_set_charset (gcf->city, GEOIP_CHARSET_UTF8); + GeoIP_set_charset(gcf->city, GEOIP_CHARSET_UTF8); } else { ngx_conf_log_error(NGX_LOG_EMERG, cf, 0, diff --git a/src/http/modules/ngx_http_scgi_module.c b/src/http/modules/ngx_http_scgi_module.c --- a/src/http/modules/ngx_http_scgi_module.c +++ b/src/http/modules/ngx_http_scgi_module.c @@ -1645,7 +1645,7 @@ ngx_http_scgi_pass(ngx_conf_t *cf, ngx_c return "is duplicate"; } - clcf = ngx_http_conf_get_module_loc_conf (cf, ngx_http_core_module); + clcf = ngx_http_conf_get_module_loc_conf(cf, ngx_http_core_module); clcf->handler = ngx_http_scgi_handler; value = cf->args->elts; diff --git a/src/http/modules/ngx_http_uwsgi_module.c b/src/http/modules/ngx_http_uwsgi_module.c --- a/src/http/modules/ngx_http_uwsgi_module.c +++ b/src/http/modules/ngx_http_uwsgi_module.c @@ -1868,7 +1868,7 @@ ngx_http_uwsgi_pass(ngx_conf_t *cf, ngx_ return "is duplicate"; } - clcf = ngx_http_conf_get_module_loc_conf (cf, ngx_http_core_module); + clcf = ngx_http_conf_get_module_loc_conf(cf, ngx_http_core_module); clcf->handler = ngx_http_uwsgi_handler; value = cf->args->elts; From mdounin at mdounin.ru Wed Jul 9 01:49:59 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Wed, 09 Jul 2014 01:49:59 +0000 Subject: [nginx] Style: keep return type and function name on different l... Message-ID: details: http://hg.nginx.org/nginx/rev/a1caf3989b49 branches: changeset: 5759:a1caf3989b49 user: Piotr Sikora date: Tue Jul 08 03:03:14 2014 -0700 description: Style: keep return type and function name on different lines. Signed-off-by: Piotr Sikora diffstat: src/os/win32/ngx_service.c | 9 ++++++--- 1 files changed, 6 insertions(+), 3 deletions(-) diffs (33 lines): diff --git a/src/os/win32/ngx_service.c b/src/os/win32/ngx_service.c --- a/src/os/win32/ngx_service.c +++ b/src/os/win32/ngx_service.c @@ -16,7 +16,8 @@ SERVICE_TABLE_ENTRY st[] = { }; -ngx_int_t ngx_service(ngx_log_t *log) +ngx_int_t +ngx_service(ngx_log_t *log) { /* primary thread */ @@ -32,7 +33,8 @@ ngx_int_t ngx_service(ngx_log_t *log) } -void service_main(u_int argc, char **argv) +void +service_main(u_int argc, char **argv) { SERVICE_STATUS status; SERVICE_STATUS_HANDLE service; @@ -85,7 +87,8 @@ void service_main(u_int argc, char **arg } -u_int service_handler(u_int control, u_int type, void *data, void *ctx) +u_int +service_handler(u_int control, u_int type, void *data, void *ctx) { /* primary thread */ From mdounin at mdounin.ru Wed Jul 9 01:50:02 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Wed, 09 Jul 2014 01:50:02 +0000 Subject: [nginx] Style: use ngx_memcpy() instead of memcpy(). Message-ID: details: http://hg.nginx.org/nginx/rev/4b668378ad8b branches: changeset: 5760:4b668378ad8b user: Piotr Sikora date: Tue Jul 08 03:03:14 2014 -0700 description: Style: use ngx_memcpy() instead of memcpy(). Signed-off-by: Piotr Sikora diffstat: src/event/ngx_event_openssl.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diffs (12 lines): diff --git a/src/event/ngx_event_openssl.c b/src/event/ngx_event_openssl.c --- a/src/event/ngx_event_openssl.c +++ b/src/event/ngx_event_openssl.c @@ -2664,7 +2664,7 @@ ngx_ssl_session_ticket_key_callback(ngx_ EVP_EncryptInit_ex(ectx, EVP_aes_128_cbc(), NULL, key[0].aes_key, iv); HMAC_Init_ex(hctx, key[0].hmac_key, 16, ngx_ssl_session_ticket_md(), NULL); - memcpy(name, key[0].name, 16); + ngx_memcpy(name, key[0].name, 16); return 0; From mdounin at mdounin.ru Wed Jul 9 01:50:04 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Wed, 09 Jul 2014 01:50:04 +0000 Subject: [nginx] Style: fix typo. Message-ID: details: http://hg.nginx.org/nginx/rev/1d693deab8ae branches: changeset: 5761:1d693deab8ae user: Piotr Sikora date: Tue Jul 08 03:03:14 2014 -0700 description: Style: fix typo. Signed-off-by: Piotr Sikora diffstat: src/misc/ngx_cpp_test_module.cpp | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diffs (10 lines): diff --git a/src/misc/ngx_cpp_test_module.cpp b/src/misc/ngx_cpp_test_module.cpp --- a/src/misc/ngx_cpp_test_module.cpp +++ b/src/misc/ngx_cpp_test_module.cpp @@ -1,5 +1,5 @@ -// stub module to test header files' C++ compatibilty +// stub module to test header files' C++ compatibility extern "C" { #include From mdounin at mdounin.ru Wed Jul 9 01:50:53 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Wed, 9 Jul 2014 05:50:53 +0400 Subject: [PATCH 3 of 4] SSL: stop using deprecated RSA_generate_key() function In-Reply-To: References: <8c3e64571adf1ff45fa4.1404690648@Piotrs-MacBook-Pro.local> <20140707011315.GZ1849@mdounin.ru> <20140707130449.GC1849@mdounin.ru> <20140707233224.GN1849@mdounin.ru> Message-ID: <20140709015053.GG1849@mdounin.ru> Hello! On Tue, Jul 08, 2014 at 03:22:21AM -0700, Piotr Sikora wrote: > Hey Maxim, > > > I don't think it's time to remove it, but as I previously said, I > > will be fine with something like this: > > > > diff --git a/src/event/ngx_event_openssl.c b/src/event/ngx_event_openssl.c > > --- a/src/event/ngx_event_openssl.c > > +++ b/src/event/ngx_event_openssl.c > > @@ -652,12 +652,16 @@ ngx_ssl_rsa512_key_callback(ngx_ssl_conn > > { > > static RSA *key; > > > > +#ifndef OPENSSL_NO_DEPRECATED > > + > > if (key_length == 512) { > > if (key == NULL) { > > key = RSA_generate_key(512, RSA_F4, NULL, NULL); > > } > > } > > > > +#endif > > + > > return key; > > } > > > > > > This won't change anything for normal builds, but will allow test > > builds with OPENSSL_NO_DEPRECATED defined. > > I'd prefer to see the RSA_generate_key_ex() being used (even though > it's more complex interface), but I don't care enough to fight over > this, so I guess your patch is "good enough", however I'd like to see > my patch #2 applied first (i.e. if key_length != 512 return NULL), so > that only RSA_generate_key() is being guarded. Committed with other patches of this series, thanks. -- Maxim Dounin http://nginx.org/ From mdounin at mdounin.ru Wed Jul 9 01:55:17 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Wed, 9 Jul 2014 05:55:17 +0400 Subject: [PATCH] SPDY: fix support for headers with multiple values In-Reply-To: <1516744.9SXB61dCXs@vbart-workstation> References: <0d4b16900e55bc3495fb.1404811761@Piotrs-MacBook-Pro.local> <702409271.IPOsu0nb15@vbart-workstation> <1516744.9SXB61dCXs@vbart-workstation> Message-ID: <20140709015517.GH1849@mdounin.ru> Hello! On Tue, Jul 08, 2014 at 02:07:27PM +0400, Valentin V. Bartenev wrote: > On Tuesday 08 July 2014 13:44:30 Valentin V. Bartenev wrote: > > On Tuesday 08 July 2014 02:29:21 Piotr Sikora wrote: > > > # HG changeset patch > > > # User Piotr Sikora > > > # Date 1404811064 25200 > > > # Tue Jul 08 02:17:44 2014 -0700 > > > # Node ID 0d4b16900e55bc3495fb63b3e93f06cac690497b > > > # Parent a680bf4dddd5c4b106419e3dfb0264815c401275 > > > SPDY: fix support for headers with multiple values. > > > > > > Split SPDY header with multiple, NULL-separated values: > > > > > > cookie: foo\0bar > > > > > > into two separate HTTP headers with the same name: > > > > > > cookie: foo > > > cookie: bar > > > > > > Even though the logic for this behavior already existed > > > in the source code, it doesn't look that it ever worked > > > and SPDY streams with such headers were simply rejected. > > > > > > Signed-off-by: Piotr Sikora > > > > > > diff -r a680bf4dddd5 -r 0d4b16900e55 src/http/ngx_http_spdy.c > > > --- a/src/http/ngx_http_spdy.c Sat Jul 05 23:29:47 2014 +0400 > > > +++ b/src/http/ngx_http_spdy.c Tue Jul 08 02:17:44 2014 -0700 > > > @@ -2584,6 +2584,8 @@ ngx_http_spdy_parse_header(ngx_http_requ > > > r->header_end = p; > > > r->header_in->pos = p + 1; > > > > > > + r->state = state; > > > + > > > return NGX_OK; > > > } > > > > > > > Nice catch, thank you. > > > > Maxim, do you approve this for push? > > > [..] > > Probably, it's better to use constant here (like r->state = 0 in > the same function): > > diff -r 9d3a9c45fc43 src/http/ngx_http_spdy.c > --- a/src/http/ngx_http_spdy.c Tue Jul 01 20:52:08 2014 +0400 > +++ b/src/http/ngx_http_spdy.c Tue Jul 08 13:55:52 2014 +0400 > @@ -2584,6 +2584,8 @@ ngx_http_spdy_parse_header(ngx_http_requ > r->header_end = p; > r->header_in->pos = p + 1; > > + r->state = sw_value; > + > return NGX_OK; > } Feel free to commit whichever version you prefer, I'm fine with both of them. -- Maxim Dounin http://nginx.org/ From mdounin at mdounin.ru Wed Jul 9 01:56:01 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Wed, 9 Jul 2014 05:56:01 +0400 Subject: [PATCH 5 of 5] Style: fix typo In-Reply-To: <35f7eb512945e2b7d39d.1404813981@Piotrs-MacBook-Pro.local> References: <35f7eb512945e2b7d39d.1404813981@Piotrs-MacBook-Pro.local> Message-ID: <20140709015601.GI1849@mdounin.ru> Hello! On Tue, Jul 08, 2014 at 03:06:21AM -0700, Piotr Sikora wrote: [...] All patches committed, thanks. -- Maxim Dounin http://nginx.org/ From mdounin at mdounin.ru Wed Jul 9 01:58:00 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Wed, 9 Jul 2014 05:58:00 +0400 Subject: Patch: Refactor ngx_http_write_request_body into a filter In-Reply-To: References: <20140620180926.GO1849@mdounin.ru> <20140627135419.GR1849@mdounin.ru> <20140707180449.GH1849@mdounin.ru> Message-ID: <20140709015800.GJ1849@mdounin.ru> Hello! On Tue, Jul 08, 2014 at 11:16:47PM +0300, grrm grrm wrote: > Hello, > > Thanks for the reply. I understand the hesitation to include new code > with changes more than cosmetic, but without any real benefit. > What if this was part of a greater initiative to implement > non-buffered request body processing. Would that be an acceptable > modification to start with? (Because that was tentatively my > intention). No. If you want to move writing a request body into the filter chain, it should be done better. -- Maxim Dounin http://nginx.org/ From vbart at nginx.com Wed Jul 9 07:59:42 2014 From: vbart at nginx.com (Valentin Bartenev) Date: Wed, 09 Jul 2014 07:59:42 +0000 Subject: [nginx] SPDY: fix support for headers with multiple values. Message-ID: details: http://hg.nginx.org/nginx/rev/b522d5b137fd branches: changeset: 5762:b522d5b137fd user: Piotr Sikora date: Tue Jul 08 02:17:44 2014 -0700 description: SPDY: fix support for headers with multiple values. Split SPDY header with multiple, NULL-separated values: cookie: foo\0bar into two separate HTTP headers with the same name: cookie: foo cookie: bar Even though the logic for this behavior already existed in the source code, it doesn't look that it ever worked and SPDY streams with such headers were simply rejected. Signed-off-by: Piotr Sikora diffstat: src/http/ngx_http_spdy.c | 2 ++ 1 files changed, 2 insertions(+), 0 deletions(-) diffs (12 lines): diff -r 1d693deab8ae -r b522d5b137fd src/http/ngx_http_spdy.c --- a/src/http/ngx_http_spdy.c Tue Jul 08 03:03:14 2014 -0700 +++ b/src/http/ngx_http_spdy.c Tue Jul 08 02:17:44 2014 -0700 @@ -2584,6 +2584,8 @@ ngx_http_spdy_parse_header(ngx_http_requ r->header_end = p; r->header_in->pos = p + 1; + r->state = sw_value; + return NGX_OK; } From cubicdaiya at gmail.com Wed Jul 9 14:31:09 2014 From: cubicdaiya at gmail.com (cubicdaiya at gmail.com) Date: Wed, 09 Jul 2014 23:31:09 +0900 Subject: [PATCH 2 of 3] Style: use ngx_str_null() In-Reply-To: <09e734739e462555dda0.1404916268@classic-club.local> References: <09e734739e462555dda0.1404916268@classic-club.local> Message-ID: # HG changeset patch # User Tatsuhiko Kubo # Date 1404915734 -32400 # Wed Jul 09 23:22:14 2014 +0900 # Node ID f166c521b619dab231b15e60cec47dd81f52833e # Parent 09e734739e462555dda00fb4cd420bdb07853cb7 Style: use ngx_str_null(). diff -r 09e734739e46 -r f166c521b619 src/core/ngx_resolver.c --- a/src/core/ngx_resolver.c Wed Jul 09 23:20:40 2014 +0900 +++ b/src/core/ngx_resolver.c Wed Jul 09 23:22:14 2014 +0900 @@ -2722,8 +2722,7 @@ } if (len == -1) { - name->len = 0; - name->data = NULL; + ngx_str_null(name); return NGX_OK; } From cubicdaiya at gmail.com Wed Jul 9 14:31:08 2014 From: cubicdaiya at gmail.com (cubicdaiya at gmail.com) Date: Wed, 09 Jul 2014 23:31:08 +0900 Subject: [PATCH 1 of 3] Style: use ngx_strlen() instead of strlen() Message-ID: <09e734739e462555dda0.1404916268@classic-club.local> # HG changeset patch # User Tatsuhiko Kubo # Date 1404915640 -32400 # Wed Jul 09 23:20:40 2014 +0900 # Node ID 09e734739e462555dda00fb4cd420bdb07853cb7 # Parent b522d5b137fdf64163ebed3e3da6b11a1b1c6561 Style: use ngx_strlen() instead of strlen(). diff -r b522d5b137fd -r 09e734739e46 src/core/ngx_log.h --- a/src/core/ngx_log.h Tue Jul 08 02:17:44 2014 -0700 +++ b/src/core/ngx_log.h Wed Jul 09 23:20:40 2014 +0900 @@ -248,7 +248,7 @@ static ngx_inline void ngx_write_stderr(char *text) { - (void) ngx_write_fd(ngx_stderr, text, strlen(text)); + (void) ngx_write_fd(ngx_stderr, text, ngx_strlen(text)); } From cubicdaiya at gmail.com Wed Jul 9 14:31:10 2014 From: cubicdaiya at gmail.com (cubicdaiya at gmail.com) Date: Wed, 09 Jul 2014 23:31:10 +0900 Subject: [PATCH 3 of 3] Style: use ngx_str_set() In-Reply-To: <09e734739e462555dda0.1404916268@classic-club.local> References: <09e734739e462555dda0.1404916268@classic-club.local> Message-ID: # HG changeset patch # User Tatsuhiko Kubo # Date 1404915839 -32400 # Wed Jul 09 23:23:59 2014 +0900 # Node ID d80543940f9a33b262d05864a30ab8b22e906455 # Parent f166c521b619dab231b15e60cec47dd81f52833e Style: use ngx_str_set(). diff -r f166c521b619 -r d80543940f9a src/http/modules/ngx_http_memcached_module.c --- a/src/http/modules/ngx_http_memcached_module.c Wed Jul 09 23:22:14 2014 +0900 +++ b/src/http/modules/ngx_http_memcached_module.c Wed Jul 09 23:23:59 2014 +0900 @@ -380,11 +380,8 @@ } h->hash = 1; - h->key.len = sizeof("Content-Encoding") - 1; - h->key.data = (u_char *) "Content-Encoding"; - h->value.len = sizeof("gzip") - 1; - h->value.data = (u_char *) "gzip"; - + ngx_str_set(&h->key, "Content-Encoding"); + ngx_str_set(&h->value, "gzip"); r->headers_out.content_encoding = h; } From mdounin at mdounin.ru Wed Jul 9 15:30:37 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Wed, 09 Jul 2014 15:30:37 +0000 Subject: [nginx] Style: use ngx_strlen() instead of strlen(). Message-ID: details: http://hg.nginx.org/nginx/rev/09e734739e46 branches: changeset: 5763:09e734739e46 user: Tatsuhiko Kubo date: Wed Jul 09 23:20:40 2014 +0900 description: Style: use ngx_strlen() instead of strlen(). diffstat: src/core/ngx_log.h | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diffs (12 lines): diff --git a/src/core/ngx_log.h b/src/core/ngx_log.h --- a/src/core/ngx_log.h +++ b/src/core/ngx_log.h @@ -248,7 +248,7 @@ char *ngx_log_set_log(ngx_conf_t *cf, ng static ngx_inline void ngx_write_stderr(char *text) { - (void) ngx_write_fd(ngx_stderr, text, strlen(text)); + (void) ngx_write_fd(ngx_stderr, text, ngx_strlen(text)); } From mdounin at mdounin.ru Wed Jul 9 15:30:40 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Wed, 09 Jul 2014 15:30:40 +0000 Subject: [nginx] Style: use ngx_str_null(). Message-ID: details: http://hg.nginx.org/nginx/rev/f166c521b619 branches: changeset: 5764:f166c521b619 user: Tatsuhiko Kubo date: Wed Jul 09 23:22:14 2014 +0900 description: Style: use ngx_str_null(). diffstat: src/core/ngx_resolver.c | 3 +-- 1 files changed, 1 insertions(+), 2 deletions(-) diffs (13 lines): diff --git a/src/core/ngx_resolver.c b/src/core/ngx_resolver.c --- a/src/core/ngx_resolver.c +++ b/src/core/ngx_resolver.c @@ -2722,8 +2722,7 @@ done: } if (len == -1) { - name->len = 0; - name->data = NULL; + ngx_str_null(name); return NGX_OK; } From mdounin at mdounin.ru Wed Jul 9 15:30:43 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Wed, 09 Jul 2014 15:30:43 +0000 Subject: [nginx] Style: use ngx_str_set(). Message-ID: details: http://hg.nginx.org/nginx/rev/d80543940f9a branches: changeset: 5765:d80543940f9a user: Tatsuhiko Kubo date: Wed Jul 09 23:23:59 2014 +0900 description: Style: use ngx_str_set(). diffstat: src/http/modules/ngx_http_memcached_module.c | 7 ++----- 1 files changed, 2 insertions(+), 5 deletions(-) diffs (17 lines): diff --git a/src/http/modules/ngx_http_memcached_module.c b/src/http/modules/ngx_http_memcached_module.c --- a/src/http/modules/ngx_http_memcached_module.c +++ b/src/http/modules/ngx_http_memcached_module.c @@ -380,11 +380,8 @@ found: } h->hash = 1; - h->key.len = sizeof("Content-Encoding") - 1; - h->key.data = (u_char *) "Content-Encoding"; - h->value.len = sizeof("gzip") - 1; - h->value.data = (u_char *) "gzip"; - + ngx_str_set(&h->key, "Content-Encoding"); + ngx_str_set(&h->value, "gzip"); r->headers_out.content_encoding = h; } From mdounin at mdounin.ru Wed Jul 9 15:31:09 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Wed, 9 Jul 2014 19:31:09 +0400 Subject: [PATCH 3 of 3] Style: use ngx_str_set() In-Reply-To: References: <09e734739e462555dda0.1404916268@classic-club.local> Message-ID: <20140709153108.GS1849@mdounin.ru> Hello! On Wed, Jul 09, 2014 at 11:31:10PM +0900, cubicdaiya at gmail.com wrote: > # HG changeset patch > # User Tatsuhiko Kubo > # Date 1404915839 -32400 > # Wed Jul 09 23:23:59 2014 +0900 > # Node ID d80543940f9a33b262d05864a30ab8b22e906455 > # Parent f166c521b619dab231b15e60cec47dd81f52833e > Style: use ngx_str_set(). > > diff -r f166c521b619 -r d80543940f9a src/http/modules/ngx_http_memcached_module.c > --- a/src/http/modules/ngx_http_memcached_module.c Wed Jul 09 23:22:14 2014 +0900 > +++ b/src/http/modules/ngx_http_memcached_module.c Wed Jul 09 23:23:59 2014 +0900 > @@ -380,11 +380,8 @@ > } > > h->hash = 1; > - h->key.len = sizeof("Content-Encoding") - 1; > - h->key.data = (u_char *) "Content-Encoding"; > - h->value.len = sizeof("gzip") - 1; > - h->value.data = (u_char *) "gzip"; > - > + ngx_str_set(&h->key, "Content-Encoding"); > + ngx_str_set(&h->value, "gzip"); > r->headers_out.content_encoding = h; > } All patches committed, thanks. -- Maxim Dounin http://nginx.org/ From mdounin at mdounin.ru Wed Jul 9 18:40:14 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Wed, 09 Jul 2014 18:40:14 +0000 Subject: [nginx] Core: bugfix for the ngx_slab_max_size case. Message-ID: details: http://hg.nginx.org/nginx/rev/abdb027be9d5 branches: changeset: 5766:abdb027be9d5 user: Jianjun Zheng date: Wed Jun 18 16:39:59 2014 +0800 description: Core: bugfix for the ngx_slab_max_size case. diffstat: src/core/ngx_slab.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diffs (12 lines): diff --git a/src/core/ngx_slab.c b/src/core/ngx_slab.c --- a/src/core/ngx_slab.c +++ b/src/core/ngx_slab.c @@ -160,7 +160,7 @@ ngx_slab_alloc_locked(ngx_slab_pool_t *p ngx_uint_t i, slot, shift, map; ngx_slab_page_t *page, *prev, *slots; - if (size >= ngx_slab_max_size) { + if (size > ngx_slab_max_size) { ngx_log_debug1(NGX_LOG_DEBUG_ALLOC, ngx_cycle->log, 0, "slab alloc: %uz", size); From mdounin at mdounin.ru Wed Jul 9 18:40:34 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Wed, 9 Jul 2014 22:40:34 +0400 Subject: [PATCH] Core: bugfix for the ngx_slab_max_size case In-Reply-To: References: Message-ID: <20140709184034.GX1849@mdounin.ru> Hello! On Wed, Jun 18, 2014 at 05:00:14PM +0800, Jianjun Zheng wrote: > At present, alloting memory with size of ngx_slab_max_size causes > > 1) an internal fragmentation, size of ngx_slab_max_size, comes into being > > 2) the slot with index of (ngx_pagesize_shift - pool->min_shift - 1) > is the right slot for this size. > > > # HG changeset patch > # User Jianjun Zheng > # Date 1403080799 -28800 > # Wed Jun 18 16:39:59 2014 +0800 > # Node ID 1704335dd810e2e2abb2b393b4f7b7c9004c6012 > # Parent ec919574cc14f7781c0ca212cffec586f88eec40 > Core: bugfix for the ngx_slab_max_size case > > diff -r ec919574cc14 -r 1704335dd810 src/core/ngx_slab.c > --- a/src/core/ngx_slab.c Tue Jun 17 16:51:25 2014 +0400 > +++ b/src/core/ngx_slab.c Wed Jun 18 16:39:59 2014 +0800 > @@ -160,7 +160,7 @@ > ngx_uint_t i, slot, shift, map; > ngx_slab_page_t *page, *prev, *slots; > > - if (size >= ngx_slab_max_size) { > + if (size > ngx_slab_max_size) { > > ngx_log_debug1(NGX_LOG_DEBUG_ALLOC, ngx_cycle->log, 0, > "slab alloc: %uz", size); Sorry for late reply. Yes, the patch is right, allocation of ngx_slab_max_size exactly (aka half page, 2048 bytes on platforms with 4k pages) can be satisfied from the largest slot and don't need the whole page to be allocated. While it's a corner case, it's certainly can be important in some cases and worth fixing. Committed, thanks. -- Maxim Dounin http://nginx.org/ From piotr at cloudflare.com Wed Jul 9 19:28:30 2014 From: piotr at cloudflare.com (Piotr Sikora) Date: Wed, 09 Jul 2014 12:28:30 -0700 Subject: [PATCH] SSL: fix build with recent OpenSSL Message-ID: <1087b3beb12068ca3641.1404934110@piotrs-macbook-pro.local> # HG changeset patch # User Piotr Sikora # Date 1404934035 25200 # Wed Jul 09 12:27:15 2014 -0700 # Node ID 1087b3beb12068ca3641240d10c0e18a2dae6013 # Parent d80543940f9a33b262d05864a30ab8b22e906455 SSL: fix build with recent OpenSSL. X509_check_host() prototype changed recently: - http://git.openssl.org/gitweb/?p=openssl.git;a=commitdiff;h=ced3d91 - http://git.openssl.org/gitweb/?p=openssl.git;a=commitdiff;h=297c67f Bump version requirement, so that OpenSSL-1.0.2-beta1 uses fallback code. Signed-off-by: Piotr Sikora diff -r d80543940f9a -r 1087b3beb120 src/event/ngx_event_openssl.c --- a/src/event/ngx_event_openssl.c Wed Jul 09 23:23:59 2014 +0900 +++ b/src/event/ngx_event_openssl.c Wed Jul 09 12:27:15 2014 -0700 @@ -50,7 +50,7 @@ static int ngx_ssl_session_ticket_key_ca HMAC_CTX *hctx, int enc); #endif -#if OPENSSL_VERSION_NUMBER < 0x10002001L +#if OPENSSL_VERSION_NUMBER < 0x10002002L static ngx_int_t ngx_ssl_check_name(ngx_str_t *name, ASN1_STRING *str); #endif @@ -2733,7 +2733,7 @@ ngx_ssl_check_host(ngx_connection_t *c, return NGX_ERROR; } -#if OPENSSL_VERSION_NUMBER >= 0x10002001L +#if OPENSSL_VERSION_NUMBER >= 0x10002002L /* X509_check_host() is only available in OpenSSL 1.0.2+ */ @@ -2741,7 +2741,9 @@ ngx_ssl_check_host(ngx_connection_t *c, goto failed; } - if (X509_check_host(cert, name->data, name->len, 0) != 1) { + if (X509_check_host(cert, (const char *) name->data, name->len, 0, NULL) + != 1) + { ngx_log_debug0(NGX_LOG_DEBUG_EVENT, c->log, 0, "X509_check_host(): no match"); goto failed; @@ -2850,7 +2852,7 @@ found: } -#if OPENSSL_VERSION_NUMBER < 0x10002001L +#if OPENSSL_VERSION_NUMBER < 0x10002002L static ngx_int_t ngx_ssl_check_name(ngx_str_t *name, ASN1_STRING *pattern) From pdn at cryptopro.ru Fri Jul 11 12:24:51 2014 From: pdn at cryptopro.ru (Dmitrii Pichulin) Date: Fri, 11 Jul 2014 16:24:51 +0400 Subject: [PATCH 0 of 1] allow to use engine keyform for server private key In-Reply-To: <20140704123152.GQ1849@mdounin.ru> References: <20140704123152.GQ1849@mdounin.ru> Message-ID: <53BFD793.8010308@cryptopro.ru> This solution adds support for two openssl engines at least, which are currently unsupported ? our gost_capi and pkcs11 one: http://www.freebsd.org/cgi/ports.cgi?query=engine_pkcs11&stype=all&sektion=all https://github.com/OpenSC/OpenSC/wiki/OpenSSL-engine-for-PKCS%2311-modules If you have a solution how to support them (and others) in a better way for nginx, please, describe it more clearly. Currently we do not understand why our solution is not good enough for you. On 04.07.2014 16:31, Maxim Dounin wrote: > Hello! > > On Fri, Jul 04, 2014 at 12:18:03PM +0000, ??????? ??????? ?????????? wrote: > >> We looked at STORE_METHOD but didn't find any good examples. >> We looked at "format=%s engine=% key=%" at single ssl_certificate_key directive but found this way more complex. >> >> Currently, we want to add 2 directives to ngx_http_ssl_module (as seen in openssl apps args): >> ssl_certificate_keyform type; >> ssl_certificate_engine device; >> >> "Type" can be PEM or ENGINE with default PEM. "Device" defines ssl engine when "type" is ENGINE in a current http, server context. >> >> Will this be enough? > Doesn't looks like a good solution for me. > > BTW, posting in text/plain, with appropriate quoting and with > In-Reply-To is a plus. Thank you. > From mdounin at mdounin.ru Fri Jul 11 15:58:04 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Fri, 11 Jul 2014 15:58:04 +0000 Subject: [nginx] SSL: fix build with recent OpenSSL. Message-ID: details: http://hg.nginx.org/nginx/rev/abd460ece11e branches: changeset: 5767:abd460ece11e user: Piotr Sikora date: Wed Jul 09 12:27:15 2014 -0700 description: SSL: fix build with recent OpenSSL. X509_check_host() prototype changed recently: - http://git.openssl.org/gitweb/?p=openssl.git;a=commitdiff;h=ced3d91 - http://git.openssl.org/gitweb/?p=openssl.git;a=commitdiff;h=297c67f Bump version requirement, so that OpenSSL-1.0.2-beta1 uses fallback code. Signed-off-by: Piotr Sikora diffstat: src/event/ngx_event_openssl.c | 8 ++++---- 1 files changed, 4 insertions(+), 4 deletions(-) diffs (39 lines): diff --git a/src/event/ngx_event_openssl.c b/src/event/ngx_event_openssl.c --- a/src/event/ngx_event_openssl.c +++ b/src/event/ngx_event_openssl.c @@ -50,7 +50,7 @@ static int ngx_ssl_session_ticket_key_ca HMAC_CTX *hctx, int enc); #endif -#if OPENSSL_VERSION_NUMBER < 0x10002001L +#if OPENSSL_VERSION_NUMBER < 0x10002002L static ngx_int_t ngx_ssl_check_name(ngx_str_t *name, ASN1_STRING *str); #endif @@ -2733,7 +2733,7 @@ ngx_ssl_check_host(ngx_connection_t *c, return NGX_ERROR; } -#if OPENSSL_VERSION_NUMBER >= 0x10002001L +#if OPENSSL_VERSION_NUMBER >= 0x10002002L /* X509_check_host() is only available in OpenSSL 1.0.2+ */ @@ -2741,7 +2741,7 @@ ngx_ssl_check_host(ngx_connection_t *c, goto failed; } - if (X509_check_host(cert, name->data, name->len, 0) != 1) { + if (X509_check_host(cert, (char *) name->data, name->len, 0, NULL) != 1) { ngx_log_debug0(NGX_LOG_DEBUG_EVENT, c->log, 0, "X509_check_host(): no match"); goto failed; @@ -2850,7 +2850,7 @@ found: } -#if OPENSSL_VERSION_NUMBER < 0x10002001L +#if OPENSSL_VERSION_NUMBER < 0x10002002L static ngx_int_t ngx_ssl_check_name(ngx_str_t *name, ASN1_STRING *pattern) From mdounin at mdounin.ru Fri Jul 11 15:58:14 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Fri, 11 Jul 2014 19:58:14 +0400 Subject: [PATCH] SSL: fix build with recent OpenSSL In-Reply-To: <1087b3beb12068ca3641.1404934110@piotrs-macbook-pro.local> References: <1087b3beb12068ca3641.1404934110@piotrs-macbook-pro.local> Message-ID: <20140711155814.GN1849@mdounin.ru> Hello! On Wed, Jul 09, 2014 at 12:28:30PM -0700, Piotr Sikora wrote: > # HG changeset patch > # User Piotr Sikora > # Date 1404934035 25200 > # Wed Jul 09 12:27:15 2014 -0700 > # Node ID 1087b3beb12068ca3641240d10c0e18a2dae6013 > # Parent d80543940f9a33b262d05864a30ab8b22e906455 > SSL: fix build with recent OpenSSL. > > X509_check_host() prototype changed recently: > - http://git.openssl.org/gitweb/?p=openssl.git;a=commitdiff;h=ced3d91 > - http://git.openssl.org/gitweb/?p=openssl.git;a=commitdiff;h=297c67f > > Bump version requirement, so that OpenSSL-1.0.2-beta1 uses fallback code. [...] > @@ -2741,7 +2741,9 @@ ngx_ssl_check_host(ngx_connection_t *c, > goto failed; > } > > - if (X509_check_host(cert, name->data, name->len, 0) != 1) { > + if (X509_check_host(cert, (const char *) name->data, name->len, 0, NULL) > + != 1) > + { > ngx_log_debug0(NGX_LOG_DEBUG_EVENT, c->log, 0, > "X509_check_host(): no match"); > goto failed; The "const" qualifier isn't needed in the cast here, otherwise looks good. Pushed with this instead: @@ -2741,7 +2741,7 @@ ngx_ssl_check_host(ngx_connection_t *c, goto failed; } - if (X509_check_host(cert, name->data, name->len, 0) != 1) { + if (X509_check_host(cert, (char *) name->data, name->len, 0, NULL) != 1) { ngx_log_debug0(NGX_LOG_DEBUG_EVENT, c->log, 0, "X509_check_host(): no match"); goto failed; Thanks! -- Maxim Dounin http://nginx.org/ From jags.gediya at gmail.com Sat Jul 12 13:00:30 2014 From: jags.gediya at gmail.com (jags gediya) Date: Sat, 12 Jul 2014 18:30:30 +0530 Subject: nginx for ARM Message-ID: I want to use nginx web server for arm based developement board. It's end application is board will work as IOTG for home automation. For this purpose, I want to cross compile nginx for ARM and port it on linux runnig on my board. Is it possible to cross compile nginx for ARM? From juremenart at gmail.com Sat Jul 12 15:30:53 2014 From: juremenart at gmail.com (Jure Menart) Date: Sat, 12 Jul 2014 17:30:53 +0200 Subject: nginx for ARM In-Reply-To: References: Message-ID: Dear Jags, it is easy to compile nginx for ARM, it works almost out of the box. I've compiled and run it without any problems on Red Pitaya project ( www.redpitaya.com) which is based on Xilinx Zynq (with embedded ARM cores) SoC and is running GNU/Linux. I've used gcc cross-compiler and to make it work I've written small script for configuration and to patch Makefile, you can see it at: https://github.com/RedPitaya/RedPitaya/blob/master/Bazaar/nginx/nginx-1.5.3/rp_configure.sh Last part od the script changes the obj/Makefile to compile it using cross-compiler. Hope it helps. Regards, Jure Menart On Jul 12, 2014 3:00 PM, "jags gediya" wrote: > I want to use nginx web server for arm based developement board. It's > end application is board will work as IOTG for home automation. For > this purpose, I want to cross compile nginx for ARM and port it on > linux runnig on my board. Is it possible to cross compile nginx for > ARM? > > _______________________________________________ > nginx-devel mailing list > nginx-devel at nginx.org > http://mailman.nginx.org/mailman/listinfo/nginx-devel > -------------- next part -------------- An HTML attachment was scrubbed... URL: From gunnlaugur at gmail.com Sat Jul 12 20:03:12 2014 From: gunnlaugur at gmail.com (Gunnlaugur Thor Briem) Date: Sat, 12 Jul 2014 20:03:12 +0000 Subject: [PATCH] Fix minor flaws in gzip_proxied documentation Message-ID: <6fa2b98f62a731520795.1405195392@toodee.local> # HG changeset patch # User Gunnlaugur Thor Briem # Date 1405194241 0 # Sat Jul 12 19:44:01 2014 +0000 # Node ID 6fa2b98f62a73152079587c5e166aac3d2bc54cf # Parent 7020b8262dc0339c56b2d80c911b112b6f5b1a7a Fix minor flaws in gzip_proxied documentation * "that the request is proxied", not "the response" * "A directive" -> "The directive" diff -r 7020b8262dc0 -r 6fa2b98f62a7 xml/en/docs/http/ngx_http_gzip_module.xml --- a/xml/en/docs/http/ngx_http_gzip_module.xml Wed Jul 09 13:39:09 2014 +0400 +++ b/xml/en/docs/http/ngx_http_gzip_module.xml Sat Jul 12 19:44:01 2014 +0000 @@ -169,9 +169,9 @@ Enables or disables gzipping of responses for proxied requests depending on the request and response. -The fact that the response is proxied is determined by +The fact that the request is proxied is determined by the presence of the
Via
request header field. -A directive accepts multiple parameters: +The directive accepts multiple parameters: off From yar at nginx.com Mon Jul 14 10:47:49 2014 From: yar at nginx.com (Yaroslav Zhuravlev) Date: Mon, 14 Jul 2014 14:47:49 +0400 Subject: [PATCH] Fix minor flaws in gzip_proxied documentation In-Reply-To: <6fa2b98f62a731520795.1405195392@toodee.local> References: <6fa2b98f62a731520795.1405195392@toodee.local> Message-ID: <6E2A2631-A42F-4221-A4B5-504CF78C0223@nginx.com> On 13 Jul 2014, at 00:03, Gunnlaugur Thor Briem wrote: > # HG changeset patch > # User Gunnlaugur Thor Briem > # Date 1405194241 0 > # Sat Jul 12 19:44:01 2014 +0000 > # Node ID 6fa2b98f62a73152079587c5e166aac3d2bc54cf > # Parent 7020b8262dc0339c56b2d80c911b112b6f5b1a7a > Fix minor flaws in gzip_proxied documentation [...] Hi Gunnlaugur, Thank you, the patch was committed with few minor changes (version bump, commit log): http://nginx.org/en/docs/http/ngx_http_gzip_module.html#gzip_proxied Best regards, Yaroslav Zhuravlev From agentzh at gmail.com Mon Jul 14 22:58:10 2014 From: agentzh at gmail.com (Yichun Zhang (agentzh)) Date: Mon, 14 Jul 2014 15:58:10 -0700 Subject: [nginx] Entity tags: explicit flag to skip not modified filter. In-Reply-To: References: Message-ID: Hello! On Wed, Jun 25, 2014 at 3:40 PM, Maxim Dounin wrote: > > + r->disable_not_modified = !u->cacheable; > + Hmm, this line of change is giving me problems to fix the new regression in my ngx_memc module's "memc_flags_to_last_modified" feature (https://github.com/openresty/memc-nginx-module#memc_flags_to_last_modified ) caused by nginx 1.7.3. It seems that there is no hooks for 3rd-party upstream C modules to override this r->disable_not_modified field? The nearest hook, u->process_header, is called *before* the ngx_http_upstream_process_headers function setting this flag. And setting u->cacheable leads to undesired side effects in ngx_http_upstream_send_response(), which is also just too hacky. Any hints on this? Thanks! -agentzh From ru at nginx.com Wed Jul 16 06:21:57 2014 From: ru at nginx.com (Ruslan Ermilov) Date: Wed, 16 Jul 2014 06:21:57 +0000 Subject: [nginx] Resolver: fixed resend on malformed responses. Message-ID: details: http://hg.nginx.org/nginx/rev/70a0408cca7b branches: changeset: 5768:70a0408cca7b user: Ruslan Ermilov date: Wed Jul 16 10:21:28 2014 +0400 description: Resolver: fixed resend on malformed responses. DNS request resend on malformed responses was broken in 98876ce2a7fd (1.5.8). Reported by Pramod Korathota. diffstat: src/core/ngx_resolver.c | 29 +++++++++++++++++++++++++++-- 1 files changed, 27 insertions(+), 2 deletions(-) diffs (81 lines): diff -r abd460ece11e -r 70a0408cca7b src/core/ngx_resolver.c --- a/src/core/ngx_resolver.c Wed Jul 09 12:27:15 2014 -0700 +++ b/src/core/ngx_resolver.c Wed Jul 16 10:21:28 2014 +0400 @@ -1467,7 +1467,6 @@ ngx_resolver_process_a(ngx_resolver_t *r goto failed; } - rn->naddrs6 = 0; qident = (rn->query6[0] << 8) + rn->query6[1]; break; @@ -1482,7 +1481,6 @@ ngx_resolver_process_a(ngx_resolver_t *r goto failed; } - rn->naddrs = 0; qident = (rn->query[0] << 8) + rn->query[1]; } @@ -1507,6 +1505,8 @@ ngx_resolver_process_a(ngx_resolver_t *r case NGX_RESOLVE_AAAA: + rn->naddrs6 = 0; + if (rn->naddrs == (u_short) -1) { goto next; } @@ -1519,6 +1519,8 @@ ngx_resolver_process_a(ngx_resolver_t *r default: /* NGX_RESOLVE_A */ + rn->naddrs = 0; + if (rn->naddrs6 == (u_short) -1) { goto next; } @@ -1539,6 +1541,8 @@ ngx_resolver_process_a(ngx_resolver_t *r case NGX_RESOLVE_AAAA: + rn->naddrs6 = 0; + if (rn->naddrs == (u_short) -1) { rn->code = (u_char) code; goto next; @@ -1548,6 +1552,8 @@ ngx_resolver_process_a(ngx_resolver_t *r default: /* NGX_RESOLVE_A */ + rn->naddrs = 0; + if (rn->naddrs6 == (u_short) -1) { rn->code = (u_char) code; goto next; @@ -1817,6 +1823,25 @@ ngx_resolver_process_a(ngx_resolver_t *r } } + switch (qtype) { + +#if (NGX_HAVE_INET6) + case NGX_RESOLVE_AAAA: + + if (rn->naddrs6 == (u_short) -1) { + rn->naddrs6 = 0; + } + + break; +#endif + + default: /* NGX_RESOLVE_A */ + + if (rn->naddrs == (u_short) -1) { + rn->naddrs = 0; + } + } + if (rn->naddrs != (u_short) -1 #if (NGX_HAVE_INET6) && rn->naddrs6 != (u_short) -1 From mdounin at mdounin.ru Wed Jul 16 15:54:03 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Wed, 16 Jul 2014 19:54:03 +0400 Subject: [nginx] Entity tags: explicit flag to skip not modified filter. In-Reply-To: References: Message-ID: <20140716155403.GR1849@mdounin.ru> Hello! On Mon, Jul 14, 2014 at 03:58:10PM -0700, Yichun Zhang (agentzh) wrote: > Hello! > > On Wed, Jun 25, 2014 at 3:40 PM, Maxim Dounin wrote: > > > > + r->disable_not_modified = !u->cacheable; > > + > > Hmm, this line of change is giving me problems to fix the new > regression in my ngx_memc module's "memc_flags_to_last_modified" > feature (https://github.com/openresty/memc-nginx-module#memc_flags_to_last_modified > ) caused by nginx 1.7.3. > > It seems that there is no hooks for 3rd-party upstream C modules to > override this r->disable_not_modified field? The nearest hook, > u->process_header, is called *before* the > ngx_http_upstream_process_headers function setting this flag. And > setting u->cacheable leads to undesired side effects in > ngx_http_upstream_send_response(), which is also just too hacky. > > Any hints on this? As the r->disable_not_modified is a flag that affects response returned, it's something to be set during the copy headers process. Protocol modules are not expected to be able to (and even try to) influence this except by providing appropriate upstream response headers. The same applies to r->headers_out.last_modified_time modifications in your module - what previously worked in your module was a hack, and there is no surprise it no longer works after changes. I see two possible ways to make it work again: - Implement appropriate flag in upstream (or upstram configuration) to make it possible to activate not modified filter for responses which doesn't use cache. - Emulate 304 responses by the module itself. The latter is obviously easier from nginx core point of view. :) Also, it should be compatible with all nginx versions. -- Maxim Dounin http://nginx.org/ From agentzh at gmail.com Wed Jul 16 19:48:20 2014 From: agentzh at gmail.com (Yichun Zhang (agentzh)) Date: Wed, 16 Jul 2014 12:48:20 -0700 Subject: [nginx] Entity tags: explicit flag to skip not modified filter. In-Reply-To: <20140716155403.GR1849@mdounin.ru> References: <20140716155403.GR1849@mdounin.ru> Message-ID: Hello! On Wed, Jul 16, 2014 at 8:54 AM, Maxim Dounin wrote: > The same applies to r->headers_out.last_modified_time > modifications in your module - what previously worked in your > module was a hack, and there is no surprise it no longer works > after changes. IMHO, the ngx_http_upstream_module mechanism should be flexible enough to do something like this. This is just a requirement that is so simple. > - Implement appropriate flag in upstream (or upstram > configuration) to make it possible to activate not modified > filter for responses which doesn't use cache. > > - Emulate 304 responses by the module itself. > > The latter is obviously easier from nginx core point of view. :) > Also, it should be compatible with all nginx versions. > I'd vote for the first option and I hope you guys will find some time to implement that in the nginx core. The second approach means duplicating most of the logic in ngx_http_not_modified_filter_module.c, which is not satisfying at all from 3rd-party module developers' perspective (that is, my perspective). It might be satisfying to the nginx core developers though ;) One strength of the nginx core is that most of the time the 3rd-party module developers can find our way when doing something unplanned by the original authors of the core. You may call it a hack or something as you like but I consider it as a strength. Otherwise the nginx world is just so boring and limited and no fun. Anyway the nginx core uses so-called "hacks" here and there over the years so it is shameless (and yeah, whether a thing should be called a "hack" is totally subjective). Just my 2 cents. Best regards, -agentzh From phoydahl at cisco.com Fri Jul 18 12:05:53 2014 From: phoydahl at cisco.com (=?iso-8859-1?q?Per_Olav_H=F8ydahl_Ohme?=) Date: Fri, 18 Jul 2014 14:05:53 +0200 Subject: [PATCH 7 of 8] Created crypt-function based on OpenSSL encryption In-Reply-To: References: Message-ID: <4b15d9a30a661d82130b.1405685153@per-w530> # HG changeset patch # User Per Olav Hoydahl Ohme # Date 1405683324 -7200 # Fri Jul 18 13:35:24 2014 +0200 # Node ID 4b15d9a30a661d82130b8661ee0f97e1a9192ef6 # Parent 0d4f62e2c8115cb0d8b372fcabe7e8582a54d299 Created crypt-function based on OpenSSL encryption. The encryption algorithm used depends on the provided salt: $1$.. - MD5, $5$.. - SHA256, $6$.. - SHA512, other - DES diff -r 0d4f62e2c811 -r 4b15d9a30a66 src/os/unix/ngx_user.c --- a/src/os/unix/ngx_user.c Fri Jul 18 13:35:24 2014 +0200 +++ b/src/os/unix/ngx_user.c Fri Jul 18 13:35:24 2014 +0200 @@ -8,6 +8,11 @@ #include #include +#if (NGX_ANDROID) +#include +#include +#include +#endif /* * Solaris has thread-safe crypt() @@ -53,6 +58,219 @@ return NGX_ERROR; } +#elif (NGX_ANDROID) + +static ngx_int_t +ngx_DES_crypt(ngx_pool_t *pool, u_char *key, u_char *salt, u_char **encrypted) +{ + char *value; + size_t len; + ngx_err_t err; + +#if (NGX_THREADS && NGX_NONREENTRANT_CRYPT) + + /* crypt() is a time consuming function, so we only try to lock */ + + if (ngx_mutex_trylock(ngx_crypt_mutex) != NGX_OK) { + return NGX_AGAIN; + } + +#endif + + value = DES_crypt((char *) key, (char *) salt); + + if (value) { + len = ngx_strlen(value) + 1; + + *encrypted = ngx_pnalloc(pool, len); + if (*encrypted == NULL) { +#if (NGX_THREADS && NGX_NONREENTRANT_CRYPT) + ngx_mutex_unlock(ngx_crypt_mutex); +#endif + return NGX_ERROR; + } + + ngx_memcpy(*encrypted, value, len); +#if (NGX_THREADS && NGX_NONREENTRANT_CRYPT) + ngx_mutex_unlock(ngx_crypt_mutex); +#endif + return NGX_OK; + } + + err = ngx_errno; + +#if (NGX_THREADS && NGX_NONREENTRANT_CRYPT) + ngx_mutex_unlock(ngx_crypt_mutex); +#endif + + ngx_log_error(NGX_LOG_CRIT, pool->log, err, "crypt() failed"); + + return NGX_ERROR; +} + +static ngx_int_t +ngx_MD5_crypt(ngx_pool_t *pool, u_char *key, u_char *salt, u_char **encrypted) +{ + u_char *value; + size_t len; + ngx_err_t err; + +#if (NGX_THREADS && NGX_NONREENTRANT_CRYPT) + + /* MD5() is a time consuming function, so we only try to lock */ + + if (ngx_mutex_trylock(ngx_crypt_mutex) != NGX_OK) { + return NGX_AGAIN; + } + +#endif + + value = MD5(key, ngx_strlen(key), NULL); + + if (value) { + len = ngx_strlen(value) + 1; + + *encrypted = ngx_pnalloc(pool, len); + if (*encrypted == NULL) { +#if (NGX_THREADS && NGX_NONREENTRANT_CRYPT) + ngx_mutex_unlock(ngx_crypt_mutex); +#endif + return NGX_ERROR; + } + + ngx_memcpy(*encrypted, value, len); +#if (NGX_THREADS && NGX_NONREENTRANT_CRYPT) + ngx_mutex_unlock(ngx_crypt_mutex); +#endif + return NGX_OK; + } + + err = ngx_errno; + +#if (NGX_THREADS && NGX_NONREENTRANT_CRYPT) + ngx_mutex_unlock(ngx_crypt_mutex); +#endif + + ngx_log_error(NGX_LOG_CRIT, pool->log, err, "MD5() failed"); + + return NGX_ERROR; +} + +static ngx_int_t +ngx_SHA256_crypt(ngx_pool_t *pool, u_char *key, u_char *salt, + u_char **encrypted) +{ + char *value; + size_t len; + ngx_err_t err; + +#if (NGX_THREADS && NGX_NONREENTRANT_CRYPT) + + /* SHA256 is a time consuming function, so we only try to lock */ + + if (ngx_mutex_trylock(ngx_crypt_mutex) != NGX_OK) { + return NGX_AGAIN; + } + +#endif + + value = SHA256(key, ngx_strlen(key), NULL); + + if (value) { + len = ngx_strlen(value) + 1; + + *encrypted = ngx_pnalloc(pool, len); + if (*encrypted == NULL) { +#if (NGX_THREADS && NGX_NONREENTRANT_CRYPT) + ngx_mutex_unlock(ngx_crypt_mutex); +#endif + return NGX_ERROR; + } + + ngx_memcpy(*encrypted, value, len); +#if (NGX_THREADS && NGX_NONREENTRANT_CRYPT) + ngx_mutex_unlock(ngx_crypt_mutex); +#endif + return NGX_OK; + } + + err = ngx_errno; + +#if (NGX_THREADS && NGX_NONREENTRANT_CRYPT) + ngx_mutex_unlock(ngx_crypt_mutex); +#endif + + ngx_log_error(NGX_LOG_CRIT, pool->log, err, "SHA256() failed"); + + return NGX_ERROR; +} + +static ngx_int_t +ngx_SHA512_crypt(ngx_pool_t *pool, u_char *key, u_char *salt, + u_char **encrypted) +{ + char *value; + size_t len; + ngx_err_t err; + +#if (NGX_THREADS && NGX_NONREENTRANT_CRYPT) + + /* SHA512() is a time consuming function, so we only try to lock */ + + if (ngx_mutex_trylock(ngx_crypt_mutex) != NGX_OK) { + return NGX_AGAIN; + } + +#endif + + value = SHA512(key, ngx_strlen(key), NULL); + + if (value) { + len = ngx_strlen(value) + 1; + + *encrypted = ngx_pnalloc(pool, len); + if (*encrypted == NULL) { +#if (NGX_THREADS && NGX_NONREENTRANT_CRYPT) + ngx_mutex_unlock(ngx_crypt_mutex); +#endif + return NGX_ERROR; + } + + ngx_memcpy(*encrypted, value, len); +#if (NGX_THREADS && NGX_NONREENTRANT_CRYPT) + ngx_mutex_unlock(ngx_crypt_mutex); +#endif + return NGX_OK; + } + + err = ngx_errno; + +#if (NGX_THREADS && NGX_NONREENTRANT_CRYPT) + ngx_mutex_unlock(ngx_crypt_mutex); +#endif + + ngx_log_error(NGX_LOG_CRIT, pool->log, err, "SHA512() failed"); + + return NGX_ERROR; +} + +ngx_int_t +ngx_libc_crypt(ngx_pool_t *pool, u_char *key, u_char *salt, u_char **encrypted) +{ + if (ngx_strlen(salt) > 3 + && salt[0] == '$' && salt[1] == '1' && salt[2] == '$') { + return ngx_MD5_crypt(pool, key, salt, encrypted); + } else if (ngx_strlen(salt) > 3 + && salt[0] == '$' && salt[1] == '5' && salt[2] == '$') { + return ngx_SHA256_crypt(pool, key, salt, encrypted); + } else if (ngx_strlen(salt) > 3 + && salt[0] == '$' && salt[1] == '6' && salt[2] == '$') { + return ngx_SHA512_crypt(pool, key, salt, encrypted); + } else { + return ngx_DES_crypt(pool, key, salt, encrypted); + } +} + #else ngx_int_t From phoydahl at cisco.com Fri Jul 18 12:05:47 2014 From: phoydahl at cisco.com (=?iso-8859-1?q?Per_Olav_H=F8ydahl_Ohme?=) Date: Fri, 18 Jul 2014 14:05:47 +0200 Subject: [PATCH 1 of 8] Set configure vars for crossbuild and avoid some platform tests In-Reply-To: References: Message-ID: <6368ecc92be2d8e85168.1405685147@per-w530> # HG changeset patch # User Per Olav Hoydahl Ohme # Date 1405683324 -7200 # Fri Jul 18 13:35:24 2014 +0200 # Node ID 6368ecc92be2d8e851686d7fb02163c600b5b843 # Parent 70a0408cca7b733fc408223e13b1c0c79cf55e32 Set configure vars for crossbuild and avoid some platform tests. Platform info from avoided tests are instead set via cmd options. diff -r 70a0408cca7b -r 6368ecc92be2 auto/options --- a/auto/options Wed Jul 16 10:21:28 2014 +0400 +++ b/auto/options Fri Jul 18 13:35:24 2014 +0200 @@ -33,6 +33,7 @@ NGX_TEST_BUILD_RTSIG=NO NGX_TEST_BUILD_SOLARIS_SENDFILEV=NO +NGX_CROSSBUILD=NO NGX_PLATFORM= NGX_WINE= @@ -154,6 +155,15 @@ NGX_POST_CONF_MSG= +# MUST BE SPECIFIED FOR CROSSBUILD +NGX_PTR_SIZE= +NGX_MAX_SIZE_T_VALUE= +NGX_SIZE_T_LEN= +NGX_MAX_OFF_T_VALUE= +NGX_OFF_T_LEN= +NGX_TIME_T_LEN= +NGX_SYS_NERR= + opt= for option @@ -178,7 +188,16 @@ --user=*) NGX_USER="$value" ;; --group=*) NGX_GROUP="$value" ;; - --crossbuild=*) NGX_PLATFORM="$value" ;; + --crossbuild=*) NGX_CROSSBUILD=YES + NGX_PLATFORM="$value" ;; + --ptr-size=*) NGX_PTR_SIZE="$value" ;; + --max-size-t-value=*) NGX_MAX_SIZE_T_VALUE="$value" ;; + --size-t-len=*) NGX_SIZE_T_LEN="$value" ;; + --max-off-t-value=*) NGX_MAX_OFF_T_VALUE="$value" ;; + --off-t-len=*) NGX_OFF_T_LEN="$value" ;; + --time-t-len=*) NGX_TIME_T_LEN="$value" ;; + --sys-nerr=*) NGX_SYS_NERR="$value" ;; + --with-root-prefixes=*) NGX_ROOT_PREFIXES="$value" ;; --build=*) NGX_BUILD="$value" ;; --builddir=*) NGX_OBJS="$value" ;; @@ -345,6 +364,8 @@ --group=GROUP set non-privileged group for worker processes + --crossbuild=PLATFORM set target platform for crossbuild + --build=NAME set build name --builddir=DIR set build directory diff -r 70a0408cca7b -r 6368ecc92be2 auto/unix --- a/auto/unix Wed Jul 16 10:21:28 2014 +0400 +++ b/auto/unix Fri Jul 18 13:35:24 2014 +0200 @@ -460,6 +460,8 @@ # C types +if [ $NGX_CROSSBUILD = NO ]; then + ngx_type="int"; . auto/types/sizeof ngx_type="long"; . auto/types/sizeof @@ -469,9 +471,16 @@ ngx_type="void *"; . auto/types/sizeof; ngx_ptr_size=$ngx_size ngx_param=NGX_PTR_SIZE; ngx_value=$ngx_size; . auto/types/value +else + +ngx_param=NGX_PTR_SIZE; ngx_value=$NGX_PTR_SIZE; . auto/types/value + +fi # POSIX types +if [ $NGX_CROSSBUILD = NO ]; then + case "$NGX_AUTO_CONFIG_H" in /*) NGX_INCLUDE_AUTO_CONFIG_H="#include \"$NGX_AUTO_CONFIG_H\"" @@ -511,6 +520,22 @@ ngx_param=NGX_TIME_T_SIZE; ngx_value=$ngx_size; . auto/types/value ngx_param=NGX_TIME_T_LEN; ngx_value=$ngx_max_len; . auto/types/value +else + +ngx_type="in_port_t"; ngx_types="u_short"; +echo "typedef $ngx_types $ngx_type;" >> $NGX_AUTO_CONFIG_H + +ngx_param=NGX_MAX_SIZE_T_VALUE; ngx_value=$NGX_MAX_SIZE_T_VALUE +. auto/types/value +ngx_param=NGX_SIZE_T_LEN; ngx_value=$NGX_SIZE_T_LEN; . auto/types/value + +ngx_param=NGX_MAX_OFF_T_VALUE; ngx_value=$NGX_MAX_OFF_T_VALUE +. auto/types/value +ngx_param=NGX_OFF_T_LEN; ngx_value=$NGX_OFF_T_LEN; . auto/types/value + +ngx_param=NGX_TIME_T_LEN; ngx_value=$NGX_TIME_T_LEN; . auto/types/value + +fi # syscalls, libc calls and some features @@ -825,3 +850,10 @@ if (getaddrinfo("localhost", NULL, NULL, &res) != 0) return 1; freeaddrinfo(res)' . auto/feature + + +if [ $NGX_CROSSBUILD = YES ]; then + +ngx_param=NGX_SYS_NERR; ngx_value=$NGX_SYS_NERR; . auto/types/value + +fi From phoydahl at cisco.com Fri Jul 18 12:05:50 2014 From: phoydahl at cisco.com (=?iso-8859-1?q?Per_Olav_H=F8ydahl_Ohme?=) Date: Fri, 18 Jul 2014 14:05:50 +0200 Subject: [PATCH 4 of 8] Specify config and init files for Android In-Reply-To: References: Message-ID: <3c5bed66c767bccd84a7.1405685150@per-w530> # HG changeset patch # User Per Olav Hoydahl Ohme # Date 1405683324 -7200 # Fri Jul 18 13:35:24 2014 +0200 # Node ID 3c5bed66c767bccd84a7b550873721f79279dfc1 # Parent 59f889ac3d3d6a5845fb5cbf7aa57ac2bc74d382 Specify config and init files for Android. - Added auto/os/android, which is similar to auto/os/linux, but setting NGX_ANDROID and using Android srcs and deps - Added src/os/unix/ngx_android_config.h, which is similar to src/os/unix/ngx_linux_config.h, but defining IOV_MAX 16 and later used for including ngx_android_glob.h diff -r 59f889ac3d3d -r 3c5bed66c767 auto/os/android --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/auto/os/android Fri Jul 18 13:35:24 2014 +0200 @@ -0,0 +1,185 @@ + +# Copyright (C) Igor Sysoev +# Copyright (C) Nginx, Inc. + + +have=NGX_ANDROID . auto/have_headers + +CORE_INCS="$UNIX_INCS" +CORE_DEPS="$UNIX_DEPS $ANDROID_DEPS" +CORE_SRCS="$UNIX_SRCS $ANDROID_SRCS" + +ngx_spacer=' +' + +cc_aux_flags="$CC_AUX_FLAGS" +CC_AUX_FLAGS="$cc_aux_flags -D_GNU_SOURCE -D_FILE_OFFSET_BITS=64" + + +# Linux kernel version + +version=$((`uname -r \ + | sed -n -e 's/^\([0-9][0-9]*\)\.\([0-9][0-9]*\)\.\([0-9][0-9]*\).*/ \ + \1*256*256+\2*256+\3/p' \ + -e 's/^\([0-9][0-9]*\)\.\([0-9][0-9]*\).*/\1*256*256+\2*256/p'`)) + +version=${version:-0} + + +# enable the rt signals on Linux between 2.2.19 and 2.6.17 + +if [ \( $version -ge 131603 -a $version -lt 132626 \) -o $EVENT_RTSIG = YES ] +then + echo " + rt signals found" + have=NGX_HAVE_RTSIG . auto/have + EVENT_MODULES="$EVENT_MODULES $RTSIG_MODULE" + CORE_SRCS="$CORE_SRCS $RTSIG_SRCS" + EVENT_FOUND=YES +fi + + +# posix_fadvise64() had been implemented in 2.5.60 + +if [ $version -lt 132412 ]; then + have=NGX_HAVE_POSIX_FADVISE . auto/nohave +fi + +# epoll, EPOLLET version + +ngx_feature="epoll" +ngx_feature_name="NGX_HAVE_EPOLL" +ngx_feature_run=yes +ngx_feature_incs="#include " +ngx_feature_path= +ngx_feature_libs= +ngx_feature_test="int efd = 0; + struct epoll_event ee; + ee.events = EPOLLIN|EPOLLOUT|EPOLLET; + ee.data.ptr = NULL; + efd = epoll_create(100); + if (efd == -1) return 1;" +. auto/feature + +if [ $ngx_found = yes ]; then + have=NGX_HAVE_CLEAR_EVENT . auto/have + CORE_SRCS="$CORE_SRCS $EPOLL_SRCS" + EVENT_MODULES="$EVENT_MODULES $EPOLL_MODULE" + EVENT_FOUND=YES + + + # EPOLLRDHUP appeared in Linux 2.6.17, glibc 2.8 + + ngx_feature="EPOLLRDHUP" + ngx_feature_name="NGX_HAVE_EPOLLRDHUP" + ngx_feature_run=no + ngx_feature_incs="#include " + ngx_feature_path= + ngx_feature_libs= + ngx_feature_test="int efd = 0, fd = 0; + struct epoll_event ee; + ee.events = EPOLLIN|EPOLLRDHUP|EPOLLET; + ee.data.ptr = NULL; + epoll_ctl(efd, EPOLL_CTL_ADD, fd, &ee)" + . auto/feature +fi + + +# O_PATH and AT_EMPTY_PATH were introduced in 2.6.39, glibc 2.14 + +ngx_feature="O_PATH" +ngx_feature_name="NGX_HAVE_O_PATH" +ngx_feature_run=no +ngx_feature_incs="#include + #include + #include " +ngx_feature_path= +ngx_feature_libs= +ngx_feature_test="int fd; struct stat sb; + fd = openat(AT_FDCWD, \".\", O_PATH|O_DIRECTORY|O_NOFOLLOW); + if (fstatat(fd, \"\", &sb, AT_EMPTY_PATH) != 0) return 1" +. auto/feature + + +# sendfile() + +CC_AUX_FLAGS="$cc_aux_flags -D_GNU_SOURCE" +ngx_feature="sendfile()" +ngx_feature_name="NGX_HAVE_SENDFILE" +ngx_feature_run=yes +ngx_feature_incs="#include + #include " +ngx_feature_path= +ngx_feature_libs= +ngx_feature_test="int s = 0, fd = 1; + ssize_t n; off_t off = 0; + n = sendfile(s, fd, &off, 1); + if (n == -1 && errno == ENOSYS) return 1" +. auto/feature + +if [ $ngx_found = yes ]; then + CORE_SRCS="$CORE_SRCS $LINUX_SENDFILE_SRCS" +fi + + +# sendfile64() + +CC_AUX_FLAGS="$cc_aux_flags -D_GNU_SOURCE -D_FILE_OFFSET_BITS=64" +ngx_feature="sendfile64()" +ngx_feature_name="NGX_HAVE_SENDFILE64" +ngx_feature_run=yes +ngx_feature_incs="#include + #include " +ngx_feature_path= +ngx_feature_libs= +ngx_feature_test="int s = 0, fd = 1; + ssize_t n; off_t off = 0; + n = sendfile(s, fd, &off, 1); + if (n == -1 && errno == ENOSYS) return 1" +. auto/feature + + +ngx_include="sys/prctl.h"; . auto/include + +# prctl(PR_SET_DUMPABLE) + +ngx_feature="prctl(PR_SET_DUMPABLE)" +ngx_feature_name="NGX_HAVE_PR_SET_DUMPABLE" +ngx_feature_run=yes +ngx_feature_incs="#include " +ngx_feature_path= +ngx_feature_libs= +ngx_feature_test="if (prctl(PR_SET_DUMPABLE, 1, 0, 0, 0) == -1) return 1" +. auto/feature + + +# sched_setaffinity() + +ngx_feature="sched_setaffinity()" +ngx_feature_name="NGX_HAVE_SCHED_SETAFFINITY" +ngx_feature_run=no +ngx_feature_incs="#include " +ngx_feature_path= +ngx_feature_libs= +ngx_feature_test="cpu_set_t mask; + CPU_ZERO(&mask); + sched_setaffinity(0, sizeof(cpu_set_t), &mask)" +. auto/feature + + +# crypt_r() + +ngx_feature="crypt_r()" +ngx_feature_name="NGX_HAVE_GNU_CRYPT_R" +ngx_feature_run=no +ngx_feature_incs="#include " +ngx_feature_path= +ngx_feature_libs=-lcrypt +ngx_feature_test="struct crypt_data cd; + crypt_r(\"key\", \"salt\", &cd);" +. auto/feature + + +ngx_include="sys/vfs.h"; . auto/include + + +CC_AUX_FLAGS="$cc_aux_flags -D_GNU_SOURCE -D_FILE_OFFSET_BITS=64" diff -r 59f889ac3d3d -r 3c5bed66c767 auto/os/conf --- a/auto/os/conf Fri Jul 18 13:35:24 2014 +0200 +++ b/auto/os/conf Fri Jul 18 13:35:24 2014 +0200 @@ -7,6 +7,10 @@ case "$NGX_PLATFORM" in + Android:*) + . auto/os/android + ;; + FreeBSD:*) . auto/os/freebsd ;; diff -r 59f889ac3d3d -r 3c5bed66c767 auto/sources --- a/auto/sources Fri Jul 18 13:35:24 2014 +0200 +++ b/auto/sources Fri Jul 18 13:35:24 2014 +0200 @@ -193,6 +193,9 @@ POSIX_DEPS=src/os/unix/ngx_posix_config.h +ANDROID_DEPS="src/os/unix/ngx_android_config.h src/os/unix/ngx_linux.h" +ANDROID_SRCS="src/os/unix/ngx_linux_init.c" + FREEBSD_DEPS="src/os/unix/ngx_freebsd_config.h src/os/unix/ngx_freebsd.h" FREEBSD_SRCS=src/os/unix/ngx_freebsd_init.c FREEBSD_SENDFILE_SRCS=src/os/unix/ngx_freebsd_sendfile_chain.c diff -r 59f889ac3d3d -r 3c5bed66c767 src/core/ngx_config.h --- a/src/core/ngx_config.h Fri Jul 18 13:35:24 2014 +0200 +++ b/src/core/ngx_config.h Fri Jul 18 13:35:24 2014 +0200 @@ -18,7 +18,10 @@ #endif -#if (NGX_FREEBSD) +#if (NGX_ANDROID) +#include + +#elif (NGX_FREEBSD) #include diff -r 59f889ac3d3d -r 3c5bed66c767 src/os/unix/ngx_android_config.h --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/os/unix/ngx_android_config.h Fri Jul 18 13:35:24 2014 +0200 @@ -0,0 +1,128 @@ + +/* + * Copyright (C) Igor Sysoev + * Copyright (C) Nginx, Inc. + */ + + +#ifndef _NGX_ANDROID_CONFIG_H_INCLUDED_ +#define _NGX_ANDROID_CONFIG_H_INCLUDED_ + + +#ifndef _GNU_SOURCE +#define _GNU_SOURCE /* pread(), pwrite(), gethostname() */ +#endif + +#define _FILE_OFFSET_BITS 64 + +#include +#include +#include +#include +#include /* offsetof() */ +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include /* statfs() */ + +#include +#include +#include + +#include +#include +#include +#include + +#include +#include +#include /* TCP_NODELAY, TCP_CORK */ +#include +#include +#include + +#include /* tzset() */ +#include /* memalign() */ +#include /* IOV_MAX */ +#include +#include /* uname() */ + +#ifndef IOV_MAX +#define IOV_MAX 16 +#endif + +#include + + +#if (NGX_HAVE_POSIX_SEM) +#include +#endif + + +#if (NGX_HAVE_SYS_PRCTL_H) +#include +#endif + + +#if (NGX_HAVE_SENDFILE64) +#include +#else +extern ssize_t sendfile(int s, int fd, int32_t *offset, size_t size); +#define NGX_SENDFILE_LIMIT 0x80000000 +#endif + + +#if (NGX_HAVE_POLL) +#include +#endif + + +#if (NGX_HAVE_RTSIG) +#include +#include +#endif + + +#if (NGX_HAVE_EPOLL) +#include +#endif + + +#if (NGX_HAVE_FILE_AIO) +#if (NGX_HAVE_SYS_EVENTFD_H) +#include +#endif +#include +#include +typedef struct iocb ngx_aiocb_t; +#endif + + +#define NGX_LISTEN_BACKLOG 511 + + +#ifndef NGX_HAVE_SO_SNDLOWAT +/* setsockopt(SO_SNDLOWAT) returns ENOPROTOOPT */ +#define NGX_HAVE_SO_SNDLOWAT 0 +#endif + + +#ifndef NGX_HAVE_INHERITED_NONBLOCK +#define NGX_HAVE_INHERITED_NONBLOCK 0 +#endif + + +#define NGX_HAVE_OS_SPECIFIC_INIT 1 +#define ngx_debug_init() + + +extern char **environ; + + +#endif /* _NGX_ANDROID_CONFIG_H_INCLUDED_ */ diff -r 59f889ac3d3d -r 3c5bed66c767 src/os/unix/ngx_os.h --- a/src/os/unix/ngx_os.h Fri Jul 18 13:35:24 2014 +0200 +++ b/src/os/unix/ngx_os.h Fri Jul 18 13:35:24 2014 +0200 @@ -67,6 +67,10 @@ #include +#elif (NGX_ANDROID) +#include + + #elif (NGX_LINUX) #include From phoydahl at cisco.com Fri Jul 18 12:05:51 2014 From: phoydahl at cisco.com (=?iso-8859-1?q?Per_Olav_H=F8ydahl_Ohme?=) Date: Fri, 18 Jul 2014 14:05:51 +0200 Subject: [PATCH 5 of 8] Include code to allow globbing on Android In-Reply-To: References: Message-ID: # HG changeset patch # User Per Olav Hoydahl Ohme # Date 1405683324 -7200 # Fri Jul 18 13:35:24 2014 +0200 # Node ID fae8ad3d60a2e4ffeac836eb4c05cf29f39c0824 # Parent 3c5bed66c767bccd84a7b550873721f79279dfc1 Include code to allow globbing on Android. glob.h/c are lifted from https://github.com/white-gecko/TokyoCabinet/raw/master (based on ones from FreeBSD, but adopted to compile for Android) diff -r 3c5bed66c767 -r fae8ad3d60a2 auto/sources --- a/auto/sources Fri Jul 18 13:35:24 2014 +0200 +++ b/auto/sources Fri Jul 18 13:35:24 2014 +0200 @@ -194,7 +194,7 @@ POSIX_DEPS=src/os/unix/ngx_posix_config.h ANDROID_DEPS="src/os/unix/ngx_android_config.h src/os/unix/ngx_linux.h" -ANDROID_SRCS="src/os/unix/ngx_linux_init.c" +ANDROID_SRCS="src/os/unix/ngx_linux_init.c src/os/unix/ngx_android_glob.c" FREEBSD_DEPS="src/os/unix/ngx_freebsd_config.h src/os/unix/ngx_freebsd.h" FREEBSD_SRCS=src/os/unix/ngx_freebsd_init.c diff -r 3c5bed66c767 -r fae8ad3d60a2 src/os/unix/ngx_android_config.h --- a/src/os/unix/ngx_android_config.h Fri Jul 18 13:35:24 2014 +0200 +++ b/src/os/unix/ngx_android_config.h Fri Jul 18 13:35:24 2014 +0200 @@ -29,6 +29,7 @@ #include #include #include +#include #include /* statfs() */ #include diff -r 3c5bed66c767 -r fae8ad3d60a2 src/os/unix/ngx_android_glob.c --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/os/unix/ngx_android_glob.c Fri Jul 18 13:35:24 2014 +0200 @@ -0,0 +1,906 @@ +/* + * Natanael Arndt, 2011: removed collate.h dependencies + * (my changes are trivial) + * + * Copyright (c) 1989, 1993 + * The Regents of the University of California. All rights reserved. + * + * This code is derived from software contributed to Berkeley by + * Guido van Rossum. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * 4. Neither the name of the University nor the names of its contributors + * may be used to endorse or promote products derived from this software + * without specific prior written permission. + * + * THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND + * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE + * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE + * ARE DISCLAIMED. IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE + * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL + * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS + * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) + * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT + * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY + * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF + * SUCH DAMAGE. + */ + +#if defined(LIBC_SCCS) && !defined(lint) +static char sccsid[] = "@(#)glob.c 8.3 (Berkeley) 10/13/93"; +#endif /* LIBC_SCCS and not lint */ +#include +__FBSDID("$FreeBSD$"); + +/* + * glob(3) -- a superset of the one defined in POSIX 1003.2. + * + * The [!...] convention to negate a range is supported (SysV, Posix, ksh). + * + * Optional extra services, controlled by flags not defined by POSIX: + * + * GLOB_QUOTE: + * Escaping convention: \ inhibits any special meaning the following + * character might have (except \ at end of string is retained). + * GLOB_MAGCHAR: + * Set in gl_flags if pattern contained a globbing character. + * GLOB_NOMAGIC: + * Same as GLOB_NOCHECK, but it will only append pattern if it did + * not contain any magic characters. [Used in csh style globbing] + * GLOB_ALTDIRFUNC: + * Use alternately specified directory access functions. + * GLOB_TILDE: + * expand ~user/foo to the /home/dir/of/user/foo + * GLOB_BRACE: + * expand {1,2}{a,b} to 1a 1b 2a 2b + * gl_matchc: + * Number of matches in the current invocation of glob. + */ + +/* + * Some notes on multibyte character support: + * 1. Patterns with illegal byte sequences match nothing - even if + * GLOB_NOCHECK is specified. + * 2. Illegal byte sequences in filenames are handled by treating them as + * single-byte characters with a value of the first byte of the sequence + * cast to wchar_t. + * 3. State-dependent encodings are not currently supported. + */ + +#include +#include + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#define DOLLAR '$' +#define DOT '.' +#define EOS '\0' +#define LBRACKET '[' +#define NOT '!' +#define QUESTION '?' +#define QUOTE '\\' +#define RANGE '-' +#define RBRACKET ']' +#define SEP '/' +#define STAR '*' +#define TILDE '~' +#define UNDERSCORE '_' +#define LBRACE '{' +#define RBRACE '}' +#define SLASH '/' +#define COMMA ',' + +#ifndef DEBUG + +#define M_QUOTE 0x8000000000ULL +#define M_PROTECT 0x4000000000ULL +#define M_MASK 0xffffffffffULL +#define M_CHAR 0x00ffffffffULL + +typedef uint_fast64_t Char; + +#else + +#define M_QUOTE 0x80 +#define M_PROTECT 0x40 +#define M_MASK 0xff +#define M_CHAR 0x7f + +typedef char Char; + +#endif + + +#define CHAR(c) ((Char)((c)&M_CHAR)) +#define META(c) ((Char)((c)|M_QUOTE)) +#define M_ALL META('*') +#define M_END META(']') +#define M_NOT META('!') +#define M_ONE META('?') +#define M_RNG META('-') +#define M_SET META('[') +#define ismeta(c) (((c)&M_QUOTE) != 0) + + +static int compare(const void *, const void *); +static int g_Ctoc(const Char *, char *, size_t); +static int g_lstat(Char *, struct stat *, glob_t *); +static DIR *g_opendir(Char *, glob_t *); +static const Char *g_strchr(const Char *, wchar_t); +#ifdef notdef +static Char *g_strcat(Char *, const Char *); +#endif +static int g_stat(Char *, struct stat *, glob_t *); +static int glob0(const Char *, glob_t *, size_t *); +static int glob1(Char *, glob_t *, size_t *); +static int glob2(Char *, Char *, Char *, Char *, glob_t *, size_t *); +static int glob3(Char *, Char *, Char *, Char *, Char *, glob_t *, size_t *); +static int globextend(const Char *, glob_t *, size_t *); +static const Char * + globtilde(const Char *, Char *, size_t, glob_t *); +static int globexp1(const Char *, glob_t *, size_t *); +static int globexp2(const Char *, const Char *, glob_t *, int *, size_t *); +static int match(Char *, Char *, Char *); +#ifdef DEBUG +static void qprintf(const char *, Char *); +#endif + +int +glob(const char *pattern, int flags, int (*errfunc)(const char *, int), glob_t *pglob) +{ + const char *patnext; + size_t limit; + Char *bufnext, *bufend, patbuf[MAXPATHLEN], prot; + mbstate_t mbs; + wchar_t wc; + size_t clen; + + patnext = pattern; + if (!(flags & GLOB_APPEND)) { + pglob->gl_pathc = 0; + pglob->gl_pathv = NULL; + if (!(flags & GLOB_DOOFFS)) + pglob->gl_offs = 0; + } + if (flags & GLOB_LIMIT) { + limit = pglob->gl_matchc; + if (limit == 0) + limit = ARG_MAX; + } else + limit = 0; + pglob->gl_flags = flags & ~GLOB_MAGCHAR; + pglob->gl_errfunc = errfunc; + pglob->gl_matchc = 0; + + bufnext = patbuf; + bufend = bufnext + MAXPATHLEN - 1; + if (flags & GLOB_NOESCAPE) { + memset(&mbs, 0, sizeof(mbs)); + while (bufend - bufnext >= MB_CUR_MAX) { + clen = mbrtowc(&wc, patnext, MB_LEN_MAX, &mbs); + if (clen == (size_t)-1 || clen == (size_t)-2) + return (GLOB_NOMATCH); + else if (clen == 0) + break; + *bufnext++ = wc; + patnext += clen; + } + } else { + /* Protect the quoted characters. */ + memset(&mbs, 0, sizeof(mbs)); + while (bufend - bufnext >= MB_CUR_MAX) { + if (*patnext == QUOTE) { + if (*++patnext == EOS) { + *bufnext++ = QUOTE | M_PROTECT; + continue; + } + prot = M_PROTECT; + } else + prot = 0; + clen = mbrtowc(&wc, patnext, MB_LEN_MAX, &mbs); + if (clen == (size_t)-1 || clen == (size_t)-2) + return (GLOB_NOMATCH); + else if (clen == 0) + break; + *bufnext++ = wc | prot; + patnext += clen; + } + } + *bufnext = EOS; + + if (flags & GLOB_BRACE) + return globexp1(patbuf, pglob, &limit); + else + return glob0(patbuf, pglob, &limit); +} + +/* + * Expand recursively a glob {} pattern. When there is no more expansion + * invoke the standard globbing routine to glob the rest of the magic + * characters + */ +static int +globexp1(const Char *pattern, glob_t *pglob, size_t *limit) +{ + const Char* ptr = pattern; + int rv; + + /* Protect a single {}, for find(1), like csh */ + if (pattern[0] == LBRACE && pattern[1] == RBRACE && pattern[2] == EOS) + return glob0(pattern, pglob, limit); + + while ((ptr = g_strchr(ptr, LBRACE)) != NULL) + if (!globexp2(ptr, pattern, pglob, &rv, limit)) + return rv; + + return glob0(pattern, pglob, limit); +} + + +/* + * Recursive brace globbing helper. Tries to expand a single brace. + * If it succeeds then it invokes globexp1 with the new pattern. + * If it fails then it tries to glob the rest of the pattern and returns. + */ +static int +globexp2(const Char *ptr, const Char *pattern, glob_t *pglob, int *rv, size_t *limit) +{ + int i; + Char *lm, *ls; + const Char *pe, *pm, *pm1, *pl; + Char patbuf[MAXPATHLEN]; + + /* copy part up to the brace */ + for (lm = patbuf, pm = pattern; pm != ptr; *lm++ = *pm++) + continue; + *lm = EOS; + ls = lm; + + /* Find the balanced brace */ + for (i = 0, pe = ++ptr; *pe; pe++) + if (*pe == LBRACKET) { + /* Ignore everything between [] */ + for (pm = pe++; *pe != RBRACKET && *pe != EOS; pe++) + continue; + if (*pe == EOS) { + /* + * We could not find a matching RBRACKET. + * Ignore and just look for RBRACE + */ + pe = pm; + } + } + else if (*pe == LBRACE) + i++; + else if (*pe == RBRACE) { + if (i == 0) + break; + i--; + } + + /* Non matching braces; just glob the pattern */ + if (i != 0 || *pe == EOS) { + *rv = glob0(patbuf, pglob, limit); + return 0; + } + + for (i = 0, pl = pm = ptr; pm <= pe; pm++) + switch (*pm) { + case LBRACKET: + /* Ignore everything between [] */ + for (pm1 = pm++; *pm != RBRACKET && *pm != EOS; pm++) + continue; + if (*pm == EOS) { + /* + * We could not find a matching RBRACKET. + * Ignore and just look for RBRACE + */ + pm = pm1; + } + break; + + case LBRACE: + i++; + break; + + case RBRACE: + if (i) { + i--; + break; + } + /* FALLTHROUGH */ + case COMMA: + if (i && *pm == COMMA) + break; + else { + /* Append the current string */ + for (lm = ls; (pl < pm); *lm++ = *pl++) + continue; + /* + * Append the rest of the pattern after the + * closing brace + */ + for (pl = pe + 1; (*lm++ = *pl++) != EOS;) + continue; + + /* Expand the current pattern */ +#ifdef DEBUG + qprintf("globexp2:", patbuf); +#endif + *rv = globexp1(patbuf, pglob, limit); + + /* move after the comma, to the next string */ + pl = pm + 1; + } + break; + + default: + break; + } + *rv = 0; + return 0; +} + + + +/* + * expand tilde from the passwd file. + */ +static const Char * +globtilde(const Char *pattern, Char *patbuf, size_t patbuf_len, glob_t *pglob) +{ + struct passwd *pwd; + char *h; + const Char *p; + Char *b, *eb; + + if (*pattern != TILDE || !(pglob->gl_flags & GLOB_TILDE)) + return pattern; + + /* + * Copy up to the end of the string or / + */ + eb = &patbuf[patbuf_len - 1]; + for (p = pattern + 1, h = (char *) patbuf; + h < (char *)eb && *p && *p != SLASH; *h++ = *p++) + continue; + + *h = EOS; + + if (((char *) patbuf)[0] == EOS) { + /* + * handle a plain ~ or ~/ by expanding $HOME first (iff + * we're not running setuid or setgid) and then trying + * the password file + */ + if (issetugid() != 0 || + (h = getenv("HOME")) == NULL) { + if (((h = getlogin()) != NULL && + (pwd = getpwnam(h)) != NULL) || + (pwd = getpwuid(getuid())) != NULL) + h = pwd->pw_dir; + else + return pattern; + } + } + else { + /* + * Expand a ~user + */ + if ((pwd = getpwnam((char*) patbuf)) == NULL) + return pattern; + else + h = pwd->pw_dir; + } + + /* Copy the home directory */ + for (b = patbuf; b < eb && *h; *b++ = *h++) + continue; + + /* Append the rest of the pattern */ + while (b < eb && (*b++ = *p++) != EOS) + continue; + *b = EOS; + + return patbuf; +} + + +/* + * The main glob() routine: compiles the pattern (optionally processing + * quotes), calls glob1() to do the real pattern matching, and finally + * sorts the list (unless unsorted operation is requested). Returns 0 + * if things went well, nonzero if errors occurred. + */ +static int +glob0(const Char *pattern, glob_t *pglob, size_t *limit) +{ + const Char *qpatnext; + int err; + size_t oldpathc; + Char *bufnext, c, patbuf[MAXPATHLEN]; + + qpatnext = globtilde(pattern, patbuf, MAXPATHLEN, pglob); + oldpathc = pglob->gl_pathc; + bufnext = patbuf; + + /* We don't need to check for buffer overflow any more. */ + while ((c = *qpatnext++) != EOS) { + switch (c) { + case LBRACKET: + c = *qpatnext; + if (c == NOT) + ++qpatnext; + if (*qpatnext == EOS || + g_strchr(qpatnext+1, RBRACKET) == NULL) { + *bufnext++ = LBRACKET; + if (c == NOT) + --qpatnext; + break; + } + *bufnext++ = M_SET; + if (c == NOT) + *bufnext++ = M_NOT; + c = *qpatnext++; + do { + *bufnext++ = CHAR(c); + if (*qpatnext == RANGE && + (c = qpatnext[1]) != RBRACKET) { + *bufnext++ = M_RNG; + *bufnext++ = CHAR(c); + qpatnext += 2; + } + } while ((c = *qpatnext++) != RBRACKET); + pglob->gl_flags |= GLOB_MAGCHAR; + *bufnext++ = M_END; + break; + case QUESTION: + pglob->gl_flags |= GLOB_MAGCHAR; + *bufnext++ = M_ONE; + break; + case STAR: + pglob->gl_flags |= GLOB_MAGCHAR; + /* collapse adjacent stars to one, + * to avoid exponential behavior + */ + if (bufnext == patbuf || bufnext[-1] != M_ALL) + *bufnext++ = M_ALL; + break; + default: + *bufnext++ = CHAR(c); + break; + } + } + *bufnext = EOS; +#ifdef DEBUG + qprintf("glob0:", patbuf); +#endif + + if ((err = glob1(patbuf, pglob, limit)) != 0) + return(err); + + /* + * If there was no match we are going to append the pattern + * if GLOB_NOCHECK was specified or if GLOB_NOMAGIC was specified + * and the pattern did not contain any magic characters + * GLOB_NOMAGIC is there just for compatibility with csh. + */ + if (pglob->gl_pathc == oldpathc) { + if (((pglob->gl_flags & GLOB_NOCHECK) || + ((pglob->gl_flags & GLOB_NOMAGIC) && + !(pglob->gl_flags & GLOB_MAGCHAR)))) + return(globextend(pattern, pglob, limit)); + else + return(GLOB_NOMATCH); + } + if (!(pglob->gl_flags & GLOB_NOSORT)) + qsort(pglob->gl_pathv + pglob->gl_offs + oldpathc, + pglob->gl_pathc - oldpathc, sizeof(char *), compare); + return(0); +} + +static int +compare(const void *p, const void *q) +{ + return(strcmp(*(char **)p, *(char **)q)); +} + +static int +glob1(Char *pattern, glob_t *pglob, size_t *limit) +{ + Char pathbuf[MAXPATHLEN]; + + /* A null pathname is invalid -- POSIX 1003.1 sect. 2.4. */ + if (*pattern == EOS) + return(0); + return(glob2(pathbuf, pathbuf, pathbuf + MAXPATHLEN - 1, + pattern, pglob, limit)); +} + +/* + * The functions glob2 and glob3 are mutually recursive; there is one level + * of recursion for each segment in the pattern that contains one or more + * meta characters. + */ +static int +glob2(Char *pathbuf, Char *pathend, Char *pathend_last, Char *pattern, + glob_t *pglob, size_t *limit) +{ + struct stat sb; + Char *p, *q; + int anymeta; + + /* + * Loop over pattern segments until end of pattern or until + * segment with meta character found. + */ + for (anymeta = 0;;) { + if (*pattern == EOS) { /* End of pattern? */ + *pathend = EOS; + if (g_lstat(pathbuf, &sb, pglob)) + return(0); + + if (((pglob->gl_flags & GLOB_MARK) && + pathend[-1] != SEP) && (S_ISDIR(sb.st_mode) + || (S_ISLNK(sb.st_mode) && + (g_stat(pathbuf, &sb, pglob) == 0) && + S_ISDIR(sb.st_mode)))) { + if (pathend + 1 > pathend_last) + return (GLOB_ABORTED); + *pathend++ = SEP; + *pathend = EOS; + } + ++pglob->gl_matchc; + return(globextend(pathbuf, pglob, limit)); + } + + /* Find end of next segment, copy tentatively to pathend. */ + q = pathend; + p = pattern; + while (*p != EOS && *p != SEP) { + if (ismeta(*p)) + anymeta = 1; + if (q + 1 > pathend_last) + return (GLOB_ABORTED); + *q++ = *p++; + } + + if (!anymeta) { /* No expansion, do next segment. */ + pathend = q; + pattern = p; + while (*pattern == SEP) { + if (pathend + 1 > pathend_last) + return (GLOB_ABORTED); + *pathend++ = *pattern++; + } + } else /* Need expansion, recurse. */ + return(glob3(pathbuf, pathend, pathend_last, pattern, p, + pglob, limit)); + } + /* NOTREACHED */ +} + +static int +glob3(Char *pathbuf, Char *pathend, Char *pathend_last, + Char *pattern, Char *restpattern, + glob_t *pglob, size_t *limit) +{ + struct dirent *dp; + DIR *dirp; + int err; + char buf[MAXPATHLEN]; + + /* + * The readdirfunc declaration can't be prototyped, because it is + * assigned, below, to two functions which are prototyped in glob.h + * and dirent.h as taking pointers to differently typed opaque + * structures. + */ + struct dirent *(*readdirfunc)(); + + if (pathend > pathend_last) + return (GLOB_ABORTED); + *pathend = EOS; + errno = 0; + + if ((dirp = g_opendir(pathbuf, pglob)) == NULL) { + /* TODO: don't call for ENOENT or ENOTDIR? */ + if (pglob->gl_errfunc) { + if (g_Ctoc(pathbuf, buf, sizeof(buf))) + return (GLOB_ABORTED); + if (pglob->gl_errfunc(buf, errno) || + pglob->gl_flags & GLOB_ERR) + return (GLOB_ABORTED); + } + return(0); + } + + err = 0; + + /* Search directory for matching names. */ + if (pglob->gl_flags & GLOB_ALTDIRFUNC) + readdirfunc = pglob->gl_readdir; + else + readdirfunc = readdir; + while ((dp = (*readdirfunc)(dirp))) { + char *sc; + Char *dc; + wchar_t wc; + size_t clen; + mbstate_t mbs; + + /* Initial DOT must be matched literally. */ + if (dp->d_name[0] == DOT && *pattern != DOT) + continue; + memset(&mbs, 0, sizeof(mbs)); + dc = pathend; + sc = dp->d_name; + while (dc < pathend_last) { + clen = mbrtowc(&wc, sc, MB_LEN_MAX, &mbs); + if (clen == (size_t)-1 || clen == (size_t)-2) { + wc = *sc; + clen = 1; + memset(&mbs, 0, sizeof(mbs)); + } + if ((*dc++ = wc) == EOS) + break; + sc += clen; + } + if (!match(pathend, pattern, restpattern)) { + *pathend = EOS; + continue; + } + err = glob2(pathbuf, --dc, pathend_last, restpattern, + pglob, limit); + if (err) + break; + } + + if (pglob->gl_flags & GLOB_ALTDIRFUNC) + (*pglob->gl_closedir)(dirp); + else + closedir(dirp); + return(err); +} + + +/* + * Extend the gl_pathv member of a glob_t structure to accomodate a new item, + * add the new item, and update gl_pathc. + * + * This assumes the BSD realloc, which only copies the block when its size + * crosses a power-of-two boundary; for v7 realloc, this would cause quadratic + * behavior. + * + * Return 0 if new item added, error code if memory couldn't be allocated. + * + * Invariant of the glob_t structure: + * Either gl_pathc is zero and gl_pathv is NULL; or gl_pathc > 0 and + * gl_pathv points to (gl_offs + gl_pathc + 1) items. + */ +static int +globextend(const Char *path, glob_t *pglob, size_t *limit) +{ + char **pathv; + size_t i, newsize, len; + char *copy; + const Char *p; + + if (*limit && pglob->gl_pathc > *limit) { + errno = 0; + return (GLOB_NOSPACE); + } + + newsize = sizeof(*pathv) * (2 + pglob->gl_pathc + pglob->gl_offs); + pathv = pglob->gl_pathv ? + realloc((char *)pglob->gl_pathv, newsize) : + malloc(newsize); + if (pathv == NULL) { + if (pglob->gl_pathv) { + free(pglob->gl_pathv); + pglob->gl_pathv = NULL; + } + return(GLOB_NOSPACE); + } + + if (pglob->gl_pathv == NULL && pglob->gl_offs > 0) { + /* first time around -- clear initial gl_offs items */ + pathv += pglob->gl_offs; + for (i = pglob->gl_offs + 1; --i > 0; ) + *--pathv = NULL; + } + pglob->gl_pathv = pathv; + + for (p = path; *p++;) + continue; + len = MB_CUR_MAX * (size_t)(p - path); /* XXX overallocation */ + if ((copy = malloc(len)) != NULL) { + if (g_Ctoc(path, copy, len)) { + free(copy); + return (GLOB_NOSPACE); + } + pathv[pglob->gl_offs + pglob->gl_pathc++] = copy; + } + pathv[pglob->gl_offs + pglob->gl_pathc] = NULL; + return(copy == NULL ? GLOB_NOSPACE : 0); +} + +/* + * pattern matching function for filenames. Each occurrence of the * + * pattern causes a recursion level. + */ +static int +match(Char *name, Char *pat, Char *patend) +{ + int ok, negate_range; + Char c, k; + + while (pat < patend) { + c = *pat++; + switch (c & M_MASK) { + case M_ALL: + if (pat == patend) + return(1); + do + if (match(name, pat, patend)) + return(1); + while (*name++ != EOS); + return(0); + case M_ONE: + if (*name++ == EOS) + return(0); + break; + case M_SET: + ok = 0; + if ((k = *name++) == EOS) + return(0); + if ((negate_range = ((*pat & M_MASK) == M_NOT)) != EOS) + ++pat; + while (((c = *pat++) & M_MASK) != M_END) + if ((*pat & M_MASK) == M_RNG) { + if (CHAR(c) <= CHAR(k) && CHAR(k) <= CHAR(pat[1])) ok = 1; + pat += 2; + } else if (c == k) + ok = 1; + if (ok == negate_range) + return(0); + break; + default: + if (*name++ != c) + return(0); + break; + } + } + return(*name == EOS); +} + +/* Free allocated data belonging to a glob_t structure. */ +void +globfree(glob_t *pglob) +{ + size_t i; + char **pp; + + if (pglob->gl_pathv != NULL) { + pp = pglob->gl_pathv + pglob->gl_offs; + for (i = pglob->gl_pathc; i--; ++pp) + if (*pp) + free(*pp); + free(pglob->gl_pathv); + pglob->gl_pathv = NULL; + } +} + +static DIR * +g_opendir(Char *str, glob_t *pglob) +{ + char buf[MAXPATHLEN]; + + if (!*str) + strcpy(buf, "."); + else { + if (g_Ctoc(str, buf, sizeof(buf))) + return (NULL); + } + + if (pglob->gl_flags & GLOB_ALTDIRFUNC) + return((*pglob->gl_opendir)(buf)); + + return(opendir(buf)); +} + +static int +g_lstat(Char *fn, struct stat *sb, glob_t *pglob) +{ + char buf[MAXPATHLEN]; + + if (g_Ctoc(fn, buf, sizeof(buf))) { + errno = ENAMETOOLONG; + return (-1); + } + if (pglob->gl_flags & GLOB_ALTDIRFUNC) + return((*pglob->gl_lstat)(buf, sb)); + return(lstat(buf, sb)); +} + +static int +g_stat(Char *fn, struct stat *sb, glob_t *pglob) +{ + char buf[MAXPATHLEN]; + + if (g_Ctoc(fn, buf, sizeof(buf))) { + errno = ENAMETOOLONG; + return (-1); + } + if (pglob->gl_flags & GLOB_ALTDIRFUNC) + return((*pglob->gl_stat)(buf, sb)); + return(stat(buf, sb)); +} + +static const Char * +g_strchr(const Char *str, wchar_t ch) +{ + + do { + if (*str == ch) + return (str); + } while (*str++); + return (NULL); +} + +static int +g_Ctoc(const Char *str, char *buf, size_t len) +{ + mbstate_t mbs; + size_t clen; + + memset(&mbs, 0, sizeof(mbs)); + while (len >= MB_CUR_MAX) { + clen = wcrtomb(buf, *str, &mbs); + if (clen == (size_t)-1) + return (1); + if (*str == L'\0') + return (0); + str++; + buf += clen; + len -= clen; + } + return (1); +} + +#ifdef DEBUG +static void +qprintf(const char *str, Char *s) +{ + Char *p; + + (void)printf("%s:\n", str); + for (p = s; *p; p++) + (void)printf("%c", CHAR(*p)); + (void)printf("\n"); + for (p = s; *p; p++) + (void)printf("%c", *p & M_PROTECT ? '"' : ' '); + (void)printf("\n"); + for (p = s; *p; p++) + (void)printf("%c", ismeta(*p) ? '_' : ' '); + (void)printf("\n"); +} +#endif diff -r 3c5bed66c767 -r fae8ad3d60a2 src/os/unix/ngx_android_glob.h --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/os/unix/ngx_android_glob.h Fri Jul 18 13:35:24 2014 +0200 @@ -0,0 +1,105 @@ +/* + * Copyright (c) 1989, 1993 + * The Regents of the University of California. All rights reserved. + * + * This code is derived from software contributed to Berkeley by + * Guido van Rossum. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * 3. Neither the name of the University nor the names of its contributors + * may be used to endorse or promote products derived from this software + * without specific prior written permission. + * + * THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND + * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE + * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE + * ARE DISCLAIMED. IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE + * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL + * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS + * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) + * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT + * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY + * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF + * SUCH DAMAGE. + * + * @(#)glob.h 8.1 (Berkeley) 6/2/93 + * $FreeBSD$ + */ + +#ifndef _GLOB_H_ +#define _GLOB_H_ + +#include +#include + +#ifndef _SIZE_T_DECLARED +typedef __size_t size_t; +#define _SIZE_T_DECLARED +#endif + +struct stat; +typedef struct { + size_t gl_pathc; /* Count of total paths so far. */ + size_t gl_matchc; /* Count of paths matching pattern. */ + size_t gl_offs; /* Reserved at beginning of gl_pathv. */ + int gl_flags; /* Copy of flags parameter to glob. */ + char **gl_pathv; /* List of paths matching pattern. */ + /* Copy of errfunc parameter to glob. */ + int (*gl_errfunc)(const char *, int); + + /* + * Alternate filesystem access methods for glob; replacement + * versions of closedir(3), readdir(3), opendir(3), stat(2) + * and lstat(2). + */ + void (*gl_closedir)(void *); + struct dirent *(*gl_readdir)(void *); + void *(*gl_opendir)(const char *); + int (*gl_lstat)(const char *, struct stat *); + int (*gl_stat)(const char *, struct stat *); +} glob_t; + +#if __POSIX_VISIBLE >= 199209 +/* Believed to have been introduced in 1003.2-1992 */ +#define GLOB_APPEND 0x0001 /* Append to output from previous call. */ +#define GLOB_DOOFFS 0x0002 /* Use gl_offs. */ +#define GLOB_ERR 0x0004 /* Return on error. */ +#define GLOB_MARK 0x0008 /* Append / to matching directories. */ +#define GLOB_NOCHECK 0x0010 /* Return pattern itself if nothing matches. */ +#define GLOB_NOSORT 0x0020 /* Don't sort. */ +#define GLOB_NOESCAPE 0x2000 /* Disable backslash escaping. */ + +/* Error values returned by glob(3) */ +#define GLOB_NOSPACE (-1) /* Malloc call failed. */ +#define GLOB_ABORTED (-2) /* Unignored error. */ +#define GLOB_NOMATCH (-3) /* No match and GLOB_NOCHECK was not set. */ +#define GLOB_NOSYS (-4) /* Obsolete: source comptability only. */ +#endif /* __POSIX_VISIBLE >= 199209 */ + +#if __BSD_VISIBLE +#define GLOB_ALTDIRFUNC 0x0040 /* Use alternately specified directory funcs. */ +#define GLOB_BRACE 0x0080 /* Expand braces ala csh. */ +#define GLOB_MAGCHAR 0x0100 /* Pattern had globbing characters. */ +#define GLOB_NOMAGIC 0x0200 /* GLOB_NOCHECK without magic chars (csh). */ +#define GLOB_QUOTE 0x0400 /* Quote special chars with \. */ +#define GLOB_TILDE 0x0800 /* Expand tilde names from the passwd file. */ +#define GLOB_LIMIT 0x1000 /* limit number of returned paths */ + +/* source compatibility, these are the old names */ +#define GLOB_MAXPATH GLOB_LIMIT +#define GLOB_ABEND GLOB_ABORTED +#endif /* __BSD_VISIBLE */ + +__BEGIN_DECLS +int glob(const char *, int, int (*)(const char *, int), glob_t *); +void globfree(glob_t *); +__END_DECLS + +#endif /* !_GLOB_H_ */ From phoydahl at cisco.com Fri Jul 18 12:05:54 2014 From: phoydahl at cisco.com (=?iso-8859-1?q?Per_Olav_H=F8ydahl_Ohme?=) Date: Fri, 18 Jul 2014 14:05:54 +0200 Subject: [PATCH 8 of 8] Do not set O_DIRECT in Android builds In-Reply-To: References: Message-ID: # HG changeset patch # User Per Olav Hoydahl Ohme # Date 1405683324 -7200 # Fri Jul 18 13:35:24 2014 +0200 # Node ID e0bfe62e3ccac0660d832cb6ed880527ac159801 # Parent 4b15d9a30a661d82130b8661ee0f97e1a9192ef6 Do not set O_DIRECT in Android builds. Use of O_DIRECT flag for open() is not supported by Bionic/Android. diff -r 4b15d9a30a66 -r e0bfe62e3cca auto/unix --- a/auto/unix Fri Jul 18 13:35:24 2014 +0200 +++ b/auto/unix Fri Jul 18 13:35:24 2014 +0200 @@ -193,6 +193,7 @@ ngx_feature_test="posix_fadvise(0, 0, 0, POSIX_FADV_SEQUENTIAL);" . auto/feature +if [[ $NGX_PLATFORM != "Android:"* ]]; then ngx_feature="O_DIRECT" ngx_feature_name="NGX_HAVE_O_DIRECT" @@ -203,6 +204,7 @@ ngx_feature_test="fcntl(0, F_SETFL, O_DIRECT);" . auto/feature +fi if [ $ngx_found = yes -a "$NGX_SYSTEM" = "Linux" ]; then have=NGX_HAVE_ALIGNED_DIRECTIO . auto/have @@ -217,6 +219,7 @@ ngx_feature_test="fcntl(0, F_NOCACHE, 1);" . auto/feature +if [[ $NGX_PLATFORM != "Android:"* ]]; then ngx_feature="directio()" ngx_feature_name="NGX_HAVE_DIRECTIO" @@ -228,6 +231,7 @@ ngx_feature_test="directio(0, DIRECTIO_ON);" . auto/feature +fi ngx_feature="statfs()" ngx_feature_name="NGX_HAVE_STATFS" From phoydahl at cisco.com Fri Jul 18 12:05:46 2014 From: phoydahl at cisco.com (=?iso-8859-1?q?Per_Olav_H=F8ydahl_Ohme?=) Date: Fri, 18 Jul 2014 14:05:46 +0200 Subject: [PATCH 0 of 8] Allow crossbuilding and Android support Message-ID: To allow crossbuilding: - Added configure options informing about target platform. - Avoiding executions of test programs for target platform features on host, including features with successfully compiled test programs. Android support: - Created Android config and init files, quite similar to existing Linux ones. - Added crypt-function and code for globbing, and avoid O_DIRECT flag on Android From phoydahl at cisco.com Fri Jul 18 12:05:48 2014 From: phoydahl at cisco.com (=?iso-8859-1?q?Per_Olav_H=F8ydahl_Ohme?=) Date: Fri, 18 Jul 2014 14:05:48 +0200 Subject: [PATCH 2 of 8] Added new root prefixes option In-Reply-To: References: Message-ID: # HG changeset patch # User Per Olav Hoydahl Ohme # Date 1405683324 -7200 # Fri Jul 18 13:35:24 2014 +0200 # Node ID afa61241e2957de3858aba2eb16f8c1b65842484 # Parent 6368ecc92be2d8e851686d7fb02163c600b5b843 Added new root prefixes option. Root prefixes are used to set feature include and lib paths. diff -r 6368ecc92be2 -r afa61241e295 auto/feature --- a/auto/feature Fri Jul 18 13:35:24 2014 +0200 +++ b/auto/feature Fri Jul 18 13:35:24 2014 +0200 @@ -14,6 +14,13 @@ ngx_found=no +if test -n "$NGX_ROOT_PREFIXES"; then + for ngx_temp in $NGX_ROOT_PREFIXES; do + ngx_feature_inc_path="$ngx_feature_inc_path -I ${ngx_temp}/include" + ngx_feature_lib_path="$ngx_feature_lib_path -L ${ngx_temp}/lib" + done +fi + if test -n "$ngx_feature_name"; then ngx_have_feature=`echo $ngx_feature_name \ | tr abcdefghijklmnopqrstuvwxyz ABCDEFGHIJKLMNOPQRSTUVWXYZ` @@ -40,9 +47,11 @@ ngx_test="$CC $CC_TEST_FLAGS $CC_AUX_FLAGS $ngx_feature_inc_path \ - -o $NGX_AUTOTEST $NGX_AUTOTEST.c $NGX_TEST_LD_OPT $ngx_feature_libs" + $ngx_feature_lib_path -o $NGX_AUTOTEST $NGX_AUTOTEST.c \ + $NGX_TEST_LD_OPT $ngx_feature_libs" ngx_feature_inc_path= +ngx_feature_lib_path= eval "/bin/sh -c \"$ngx_test\" >> $NGX_AUTOCONF_ERR 2>&1" From phoydahl at cisco.com Fri Jul 18 12:05:52 2014 From: phoydahl at cisco.com (=?iso-8859-1?q?Per_Olav_H=F8ydahl_Ohme?=) Date: Fri, 18 Jul 2014 14:05:52 +0200 Subject: [PATCH 6 of 8] Added new run-time prefix option for configure files In-Reply-To: References: Message-ID: <0d4f62e2c8115cb0d8b3.1405685152@per-w530> # HG changeset patch # User Per Olav Hoydahl Ohme # Date 1405683324 -7200 # Fri Jul 18 13:35:24 2014 +0200 # Node ID 0d4f62e2c8115cb0d8b372fcabe7e8582a54d299 # Parent fae8ad3d60a2e4ffeac836eb4c05cf29f39c0824 Added new run-time prefix option for configure files. Needed in case configure files are relocated after build. diff -r fae8ad3d60a2 -r 0d4f62e2c811 auto/configure --- a/auto/configure Fri Jul 18 13:35:24 2014 +0200 +++ b/auto/configure Fri Jul 18 13:35:24 2014 +0200 @@ -79,6 +79,9 @@ if [ ".$NGX_CONF_PREFIX" != "." ]; then have=NGX_CONF_PREFIX value="\"$NGX_CONF_PREFIX/\"" . auto/define fi +if [ "$NGX_RT_CONF_PREFIX" ]; then + have=NGX_RT_CONF_PREFIX value="\"$NGX_RT_CONF_PREFIX\"" . auto/define +fi have=NGX_SBIN_PATH value="\"$NGX_SBIN_PATH\"" . auto/define have=NGX_CONF_PATH value="\"$NGX_CONF_PATH\"" . auto/define diff -r fae8ad3d60a2 -r 0d4f62e2c811 auto/options --- a/auto/options Fri Jul 18 13:35:24 2014 +0200 +++ b/auto/options Fri Jul 18 13:35:24 2014 +0200 @@ -9,6 +9,7 @@ NGX_SBIN_PATH= NGX_CONF_PREFIX= NGX_CONF_PATH= +NGX_RT_CONF_PREFIX= NGX_ERROR_LOG_PATH= NGX_PID_PATH= NGX_LOCK_PATH= @@ -182,6 +183,7 @@ --prefix=*) NGX_PREFIX="$value" ;; --sbin-path=*) NGX_SBIN_PATH="$value" ;; --conf-path=*) NGX_CONF_PATH="$value" ;; + --rt-conf-prefix=*) NGX_RT_CONF_PREFIX="$value";; --error-log-path=*) NGX_ERROR_LOG_PATH="$value";; --pid-path=*) NGX_PID_PATH="$value" ;; --lock-path=*) NGX_LOCK_PATH="$value" ;; @@ -365,6 +367,7 @@ worker processes --crossbuild=PLATFORM set target platform for crossbuild + --rt-conf-prefix=PATH set run-time prefix for configure files --build=NAME set build name --builddir=DIR set build directory diff -r fae8ad3d60a2 -r 0d4f62e2c811 src/core/ngx_conf_file.c --- a/src/core/ngx_conf_file.c Fri Jul 18 13:35:24 2014 +0200 +++ b/src/core/ngx_conf_file.c Fri Jul 18 13:35:24 2014 +0200 @@ -120,6 +120,18 @@ /* open configuration file */ +#ifdef NGX_RT_CONF_PREFIX + char rt_prefixed_data[512]; + snprintf(rt_prefixed_data, 512, "%s%s", NGX_RT_CONF_PREFIX, + filename->data); + fd = ngx_open_file(rt_prefixed_data, NGX_FILE_RDONLY, NGX_FILE_OPEN, 0); + if (fd == NGX_INVALID_FILE) { + ngx_conf_log_error(NGX_LOG_EMERG, cf, ngx_errno, + ngx_open_file_n " \"%s\" failed", + rt_prefixed_data); + return NGX_CONF_ERROR; + } +#else fd = ngx_open_file(filename->data, NGX_FILE_RDONLY, NGX_FILE_OPEN, 0); if (fd == NGX_INVALID_FILE) { ngx_conf_log_error(NGX_LOG_EMERG, cf, ngx_errno, @@ -127,6 +139,7 @@ filename->data); return NGX_CONF_ERROR; } +#endif prev = cf->conf_file; From phoydahl at cisco.com Fri Jul 18 12:05:49 2014 From: phoydahl at cisco.com (=?iso-8859-1?q?Per_Olav_H=F8ydahl_Ohme?=) Date: Fri, 18 Jul 2014 14:05:49 +0200 Subject: [PATCH 3 of 8] Include features which can be compiled when crossbuilding In-Reply-To: References: Message-ID: <59f889ac3d3d6a5845fb.1405685149@per-w530> # HG changeset patch # User Per Olav Hoydahl Ohme # Date 1405683324 -7200 # Fri Jul 18 13:35:24 2014 +0200 # Node ID 59f889ac3d3d6a5845fb5cbf7aa57ac2bc74d382 # Parent afa61241e2957de3858aba2eb16f8c1b65842484 Include features which can be compiled when crossbuilding. Normally, features are only included if their tests execute successfully. diff -r afa61241e295 -r 59f889ac3d3d auto/feature --- a/auto/feature Fri Jul 18 13:35:24 2014 +0200 +++ b/auto/feature Fri Jul 18 13:35:24 2014 +0200 @@ -55,6 +55,7 @@ eval "/bin/sh -c \"$ngx_test\" >> $NGX_AUTOCONF_ERR 2>&1" +[ $NGX_CROSSBUILD = YES ] && ngx_feature_run=no if [ -x $NGX_AUTOTEST ]; then From mdounin at mdounin.ru Fri Jul 18 16:04:27 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Fri, 18 Jul 2014 16:04:27 +0000 Subject: [nginx] Reset of r->uri.len on URI parsing errors. Message-ID: details: http://hg.nginx.org/nginx/rev/e0aa54a4357e branches: changeset: 5769:e0aa54a4357e user: Maxim Dounin date: Fri Jul 18 20:02:11 2014 +0400 description: Reset of r->uri.len on URI parsing errors. This ensures that debug logging and the $uri variable (if used in 400 Bad Request processing) will not try to access uninitialized memory. Found by Sergey Bobrov. diffstat: src/http/ngx_http_request.c | 2 ++ 1 files changed, 2 insertions(+), 0 deletions(-) diffs (12 lines): diff --git a/src/http/ngx_http_request.c b/src/http/ngx_http_request.c --- a/src/http/ngx_http_request.c +++ b/src/http/ngx_http_request.c @@ -1071,6 +1071,8 @@ ngx_http_process_request_uri(ngx_http_re cscf = ngx_http_get_module_srv_conf(r, ngx_http_core_module); if (ngx_http_parse_complex_uri(r, cscf->merge_slashes) != NGX_OK) { + r->uri.len = 0; + ngx_log_error(NGX_LOG_INFO, r->connection->log, 0, "client sent invalid request"); ngx_http_finalize_request(r, NGX_HTTP_BAD_REQUEST); From mdounin at mdounin.ru Fri Jul 18 16:13:37 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Fri, 18 Jul 2014 16:13:37 +0000 Subject: [nginx] Upstream: ngx_http_upstream_store() error handling fixes. Message-ID: details: http://hg.nginx.org/nginx/rev/9de5820bb3e0 branches: changeset: 5770:9de5820bb3e0 user: Maxim Dounin date: Fri Jul 18 20:11:40 2014 +0400 description: Upstream: ngx_http_upstream_store() error handling fixes. Previously, ngx_http_map_uri_to_path() errors were not checked in ngx_http_upstream_store(). Moreover, in case of errors temporary files were not deleted, as u->store was set to 0, preventing cleanup code in ngx_http_upstream_finalize_request() from removing them. With this patch, u->store is set to 0 only if there were no errors. Reported by Feng Gu. diffstat: src/http/ngx_http_upstream.c | 7 +++++-- 1 files changed, 5 insertions(+), 2 deletions(-) diffs (31 lines): diff --git a/src/http/ngx_http_upstream.c b/src/http/ngx_http_upstream.c --- a/src/http/ngx_http_upstream.c +++ b/src/http/ngx_http_upstream.c @@ -3253,7 +3253,6 @@ ngx_http_upstream_process_request(ngx_ht || u->headers_in.content_length_n == tf->offset)) { ngx_http_upstream_store(r, u); - u->store = 0; } } } @@ -3375,7 +3374,9 @@ ngx_http_upstream_store(ngx_http_request if (u->conf->store_lengths == NULL) { - ngx_http_map_uri_to_path(r, &path, &root, 0); + if (ngx_http_map_uri_to_path(r, &path, &root, 0) == NULL) { + return; + } } else { if (ngx_http_script_run(r, &path, u->conf->store_lengths->elts, 0, @@ -3393,6 +3394,8 @@ ngx_http_upstream_store(ngx_http_request tf->file.name.data, path.data); (void) ngx_ext_rename_file(&tf->file.name, &path, &ext); + + u->store = 0; } From jiakai1000 at gmail.com Sun Jul 20 03:06:56 2014 From: jiakai1000 at gmail.com (=?GB2312?B?vNa/rQ==?=) Date: Sun, 20 Jul 2014 11:06:56 +0800 Subject: Why ngx_trylock do extra judgement? In-Reply-To: <53CB318E.2010208@gmail.com> References: <53CB318E.2010208@gmail.com> Message-ID: <53CB3250.4050802@gmail.com> Hi there, function ngx_shmtx_lock: if (*mtx->lock == 0 && ngx_atomic_cmp_set(mtx->lock, 0, ngx_pid)) and ngx_trylock (ngx_atomic.h): (*(lock) == 0 && ngx_atomic_cmp_set(lock, 0, 1)) I think ngx_atomic_cmp_set is enough, why Nginx do extra judgement ahead of it ? From jammy.linux at gmail.com Sun Jul 20 12:38:25 2014 From: jammy.linux at gmail.com (Meng Zhang) Date: Sun, 20 Jul 2014 20:38:25 +0800 Subject: Why ngx_trylock do extra judgement? In-Reply-To: <53CB3250.4050802@gmail.com> References: <53CB318E.2010208@gmail.com> <53CB3250.4050802@gmail.com> Message-ID: I believe it tries to avoid unnecessary memory barrier or bus locking operations, e.g. on X86, usually, the CAS is implemented by "lock cmpxchg X,Y". Regards, Jammy On 20 Jul, 2014, at 11:06 am, ?? wrote: > > Hi there, > > function ngx_shmtx_lock: if (*mtx->lock == 0 && ngx_atomic_cmp_set(mtx->lock, 0, ngx_pid)) > and ngx_trylock (ngx_atomic.h): (*(lock) == 0 && ngx_atomic_cmp_set(lock, 0, 1)) > > I think ngx_atomic_cmp_set is enough, why Nginx do extra judgement ahead of it ? > > > > _______________________________________________ > nginx-devel mailing list > nginx-devel at nginx.org > http://mailman.nginx.org/mailman/listinfo/nginx-devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From aaron.peschel at gmail.com Mon Jul 21 16:48:02 2014 From: aaron.peschel at gmail.com (Aaron Peschel) Date: Mon, 21 Jul 2014 09:48:02 -0700 Subject: [nginx] Entity tags: downgrade strong etags to weak ones as needed. In-Reply-To: References: Message-ID: Thank you for your help through this whole process, Maxim! On Wed, Jun 25, 2014 at 3:40 PM, Maxim Dounin wrote: > details: http://hg.nginx.org/nginx/rev/e491b26fa5a1 > branches: > changeset: 5733:e491b26fa5a1 > user: Maxim Dounin > date: Thu Jun 26 02:21:01 2014 +0400 > description: > Entity tags: downgrade strong etags to weak ones as needed. > > See http://mailman.nginx.org/pipermail/nginx-devel/2013-November/004523.html. > > diffstat: > > src/http/modules/ngx_http_addition_filter_module.c | 2 +- > src/http/modules/ngx_http_gunzip_filter_module.c | 2 +- > src/http/modules/ngx_http_gzip_filter_module.c | 2 +- > src/http/modules/ngx_http_ssi_filter_module.c | 5 ++- > src/http/modules/ngx_http_sub_filter_module.c | 5 ++- > src/http/modules/ngx_http_xslt_filter_module.c | 6 ++- > src/http/ngx_http_core_module.c | 40 ++++++++++++++++++++++ > src/http/ngx_http_core_module.h | 1 + > 8 files changed, 56 insertions(+), 7 deletions(-) > > diffs (153 lines): > > diff --git a/src/http/modules/ngx_http_addition_filter_module.c b/src/http/modules/ngx_http_addition_filter_module.c > --- a/src/http/modules/ngx_http_addition_filter_module.c > +++ b/src/http/modules/ngx_http_addition_filter_module.c > @@ -121,7 +121,7 @@ ngx_http_addition_header_filter(ngx_http > > ngx_http_clear_content_length(r); > ngx_http_clear_accept_ranges(r); > - ngx_http_clear_etag(r); > + ngx_http_weak_etag(r); > > return ngx_http_next_header_filter(r); > } > diff --git a/src/http/modules/ngx_http_gunzip_filter_module.c b/src/http/modules/ngx_http_gunzip_filter_module.c > --- a/src/http/modules/ngx_http_gunzip_filter_module.c > +++ b/src/http/modules/ngx_http_gunzip_filter_module.c > @@ -165,7 +165,7 @@ ngx_http_gunzip_header_filter(ngx_http_r > > ngx_http_clear_content_length(r); > ngx_http_clear_accept_ranges(r); > - ngx_http_clear_etag(r); > + ngx_http_weak_etag(r); > > return ngx_http_next_header_filter(r); > } > diff --git a/src/http/modules/ngx_http_gzip_filter_module.c b/src/http/modules/ngx_http_gzip_filter_module.c > --- a/src/http/modules/ngx_http_gzip_filter_module.c > +++ b/src/http/modules/ngx_http_gzip_filter_module.c > @@ -306,7 +306,7 @@ ngx_http_gzip_header_filter(ngx_http_req > > ngx_http_clear_content_length(r); > ngx_http_clear_accept_ranges(r); > - ngx_http_clear_etag(r); > + ngx_http_weak_etag(r); > > return ngx_http_next_header_filter(r); > } > diff --git a/src/http/modules/ngx_http_ssi_filter_module.c b/src/http/modules/ngx_http_ssi_filter_module.c > --- a/src/http/modules/ngx_http_ssi_filter_module.c > +++ b/src/http/modules/ngx_http_ssi_filter_module.c > @@ -369,10 +369,13 @@ ngx_http_ssi_header_filter(ngx_http_requ > if (r == r->main) { > ngx_http_clear_content_length(r); > ngx_http_clear_accept_ranges(r); > - ngx_http_clear_etag(r); > > if (!slcf->last_modified) { > ngx_http_clear_last_modified(r); > + ngx_http_clear_etag(r); > + > + } else { > + ngx_http_weak_etag(r); > } > } > > diff --git a/src/http/modules/ngx_http_sub_filter_module.c b/src/http/modules/ngx_http_sub_filter_module.c > --- a/src/http/modules/ngx_http_sub_filter_module.c > +++ b/src/http/modules/ngx_http_sub_filter_module.c > @@ -175,10 +175,13 @@ ngx_http_sub_header_filter(ngx_http_requ > > if (r == r->main) { > ngx_http_clear_content_length(r); > - ngx_http_clear_etag(r); > > if (!slcf->last_modified) { > ngx_http_clear_last_modified(r); > + ngx_http_clear_etag(r); > + > + } else { > + ngx_http_weak_etag(r); > } > } > > diff --git a/src/http/modules/ngx_http_xslt_filter_module.c b/src/http/modules/ngx_http_xslt_filter_module.c > --- a/src/http/modules/ngx_http_xslt_filter_module.c > +++ b/src/http/modules/ngx_http_xslt_filter_module.c > @@ -337,12 +337,14 @@ ngx_http_xslt_send(ngx_http_request_t *r > r->headers_out.content_length = NULL; > } > > - ngx_http_clear_etag(r); > - > conf = ngx_http_get_module_loc_conf(r, ngx_http_xslt_filter_module); > > if (!conf->last_modified) { > ngx_http_clear_last_modified(r); > + ngx_http_clear_etag(r); > + > + } else { > + ngx_http_weak_etag(r); > } > } > > diff --git a/src/http/ngx_http_core_module.c b/src/http/ngx_http_core_module.c > --- a/src/http/ngx_http_core_module.c > +++ b/src/http/ngx_http_core_module.c > @@ -1851,6 +1851,46 @@ ngx_http_set_etag(ngx_http_request_t *r) > } > > > +void > +ngx_http_weak_etag(ngx_http_request_t *r) > +{ > + size_t len; > + u_char *p; > + ngx_table_elt_t *etag; > + > + etag = r->headers_out.etag; > + > + if (etag == NULL) { > + return; > + } > + > + if (etag->value.len > 2 > + && etag->value.data[0] == 'W' > + && etag->value.data[1] == '/') > + { > + return; > + } > + > + if (etag->value.len < 1 || etag->value.data[0] != '"') { > + r->headers_out.etag->hash = 0; > + r->headers_out.etag = NULL; > + return; > + } > + > + p = ngx_pnalloc(r->pool, etag->value.len + 2); > + if (p == NULL) { > + r->headers_out.etag->hash = 0; > + r->headers_out.etag = NULL; > + return; > + } > + > + len = ngx_sprintf(p, "W/%V", &etag->value) - p; > + > + etag->value.data = p; > + etag->value.len = len; > +} > + > + > ngx_int_t > ngx_http_send_response(ngx_http_request_t *r, ngx_uint_t status, > ngx_str_t *ct, ngx_http_complex_value_t *cv) > diff --git a/src/http/ngx_http_core_module.h b/src/http/ngx_http_core_module.h > --- a/src/http/ngx_http_core_module.h > +++ b/src/http/ngx_http_core_module.h > @@ -501,6 +501,7 @@ void *ngx_http_test_content_type(ngx_htt > ngx_int_t ngx_http_set_content_type(ngx_http_request_t *r); > void ngx_http_set_exten(ngx_http_request_t *r); > ngx_int_t ngx_http_set_etag(ngx_http_request_t *r); > +void ngx_http_weak_etag(ngx_http_request_t *r); > ngx_int_t ngx_http_send_response(ngx_http_request_t *r, ngx_uint_t status, > ngx_str_t *ct, ngx_http_complex_value_t *cv); > u_char *ngx_http_map_uri_to_path(ngx_http_request_t *r, ngx_str_t *name, > > _______________________________________________ > nginx-devel mailing list > nginx-devel at nginx.org > http://mailman.nginx.org/mailman/listinfo/nginx-devel From pdn at cryptopro.ru Tue Jul 22 11:16:35 2014 From: pdn at cryptopro.ru (Dmitrii Pichulin) Date: Tue, 22 Jul 2014 15:16:35 +0400 Subject: [PATCH] allow to use engine keyform for server private key Message-ID: <638389b21e0e1522ed8b.1406027795@pichulin7.cp.ru> # HG changeset patch # User Dmitrii Pichulin # Date 1406021876 -14400 # Tue Jul 22 13:37:56 2014 +0400 # Node ID 638389b21e0e1522ed8b8205012f5af562dc50c7 # Parent 63d7d69d0fe48e030ff9fc520c7036dbd1ebc13f allow to use engine keyform for server private key diff -r 63d7d69d0fe4 -r 638389b21e0e src/event/ngx_event_openssl.c --- a/src/event/ngx_event_openssl.c Fri Jun 20 12:55:41 2014 +0400 +++ b/src/event/ngx_event_openssl.c Tue Jul 22 13:37:56 2014 +0400 @@ -257,11 +257,31 @@ ngx_int_t ngx_ssl_certificate(ngx_conf_t *cf, ngx_ssl_t *ssl, ngx_str_t *cert, - ngx_str_t *key) + ngx_str_t *key, ngx_str_t *keyform, ngx_str_t *engine) { - BIO *bio; - X509 *x509; - u_long n; + BIO *bio; + X509 *x509; + u_long n; + ngx_uint_t ssl_use_engine_keyform = 0; + + if (keyform->len) { + + if (ngx_strcmp(keyform->data, "ENGINE") == 0) { + ssl_use_engine_keyform = 1; + + } else if (ngx_strcmp(keyform->data, "PEM") != 0) { + ngx_conf_log_error(NGX_LOG_EMERG, cf, 0, + "invalid parameter: %V", keyform); + return NGX_ERROR; + } + } + + if (ssl_use_engine_keyform && engine->len == 0) { + ngx_conf_log_error(NGX_LOG_EMERG, cf, 0, + "no \"ssl_certificate_engine\" is defined " + "while \"ssl_certificate_keyform\" is \"ENGINE\""); + return NGX_ERROR; + } if (ngx_conf_full_name(cf->cycle, cert, 1) != NGX_OK) { return NGX_ERROR; @@ -344,17 +364,51 @@ BIO_free(bio); - if (ngx_conf_full_name(cf->cycle, key, 1) != NGX_OK) { - return NGX_ERROR; - } - - if (SSL_CTX_use_PrivateKey_file(ssl->ctx, (char *) key->data, - SSL_FILETYPE_PEM) - == 0) - { - ngx_ssl_error(NGX_LOG_EMERG, ssl->log, 0, - "SSL_CTX_use_PrivateKey_file(\"%s\") failed", key->data); - return NGX_ERROR; + if (ssl_use_engine_keyform) { + EVP_PKEY *pkey; + ENGINE *e; + + e = ENGINE_by_id((const char *) engine->data); + + if (e == NULL) { + ngx_ssl_error(NGX_LOG_EMERG, ssl->log, 0, + "ENGINE_by_id(\"%s\") failed", engine->data); + return NGX_ERROR; + } + + pkey = ENGINE_load_private_key(e, (const char *)key->data, 0, 0); + + ENGINE_free(e); + + if (!pkey) { + ngx_ssl_error(NGX_LOG_EMERG, ssl->log, 0, + "ENGINE_load_private_key(\"%s\") failed", key->data); + return NGX_ERROR; + } + + if (SSL_CTX_use_PrivateKey(ssl->ctx, pkey) == 0) { + ngx_ssl_error(NGX_LOG_EMERG, ssl->log, 0, + "SSL_CTX_use_PrivateKey_file(\"%s\") failed", key->data); + EVP_PKEY_free(pkey); + return NGX_ERROR; + } + + EVP_PKEY_free(pkey); + + } else { + + if (ngx_conf_full_name(cf->cycle, key, 1) != NGX_OK) { + return NGX_ERROR; + } + + if (SSL_CTX_use_PrivateKey_file(ssl->ctx, + (char *) key->data, SSL_FILETYPE_PEM) + == 0) + { + ngx_ssl_error(NGX_LOG_EMERG, ssl->log, 0, + "SSL_CTX_use_PrivateKey_file(\"%s\") failed", key->data); + return NGX_ERROR; + } } return NGX_OK; diff -r 63d7d69d0fe4 -r 638389b21e0e src/event/ngx_event_openssl.h --- a/src/event/ngx_event_openssl.h Fri Jun 20 12:55:41 2014 +0400 +++ b/src/event/ngx_event_openssl.h Tue Jul 22 13:37:56 2014 +0400 @@ -112,7 +112,7 @@ ngx_int_t ngx_ssl_init(ngx_log_t *log); ngx_int_t ngx_ssl_create(ngx_ssl_t *ssl, ngx_uint_t protocols, void *data); ngx_int_t ngx_ssl_certificate(ngx_conf_t *cf, ngx_ssl_t *ssl, - ngx_str_t *cert, ngx_str_t *key); + ngx_str_t *cert, ngx_str_t *key, ngx_str_t *keyform, ngx_str_t *engine); ngx_int_t ngx_ssl_client_certificate(ngx_conf_t *cf, ngx_ssl_t *ssl, ngx_str_t *cert, ngx_int_t depth); ngx_int_t ngx_ssl_trusted_certificate(ngx_conf_t *cf, ngx_ssl_t *ssl, diff -r 63d7d69d0fe4 -r 638389b21e0e src/http/modules/ngx_http_ssl_module.c --- a/src/http/modules/ngx_http_ssl_module.c Fri Jun 20 12:55:41 2014 +0400 +++ b/src/http/modules/ngx_http_ssl_module.c Tue Jul 22 13:37:56 2014 +0400 @@ -91,6 +91,20 @@ offsetof(ngx_http_ssl_srv_conf_t, certificate_key), NULL }, + { ngx_string("ssl_certificate_keyform"), + NGX_HTTP_MAIN_CONF|NGX_HTTP_SRV_CONF|NGX_CONF_TAKE1, + ngx_conf_set_str_slot, + NGX_HTTP_SRV_CONF_OFFSET, + offsetof(ngx_http_ssl_srv_conf_t, certificate_keyform), + NULL }, + + { ngx_string("ssl_certificate_engine"), + NGX_HTTP_MAIN_CONF|NGX_HTTP_SRV_CONF|NGX_CONF_TAKE1, + ngx_conf_set_str_slot, + NGX_HTTP_SRV_CONF_OFFSET, + offsetof(ngx_http_ssl_srv_conf_t, certificate_engine), + NULL }, + { ngx_string("ssl_dhparam"), NGX_HTTP_MAIN_CONF|NGX_HTTP_SRV_CONF|NGX_CONF_TAKE1, ngx_conf_set_str_slot, @@ -562,6 +576,10 @@ ngx_conf_merge_str_value(conf->certificate, prev->certificate, ""); ngx_conf_merge_str_value(conf->certificate_key, prev->certificate_key, ""); + ngx_conf_merge_str_value(conf->certificate_keyform, + prev->certificate_keyform, ""); + ngx_conf_merge_str_value(conf->certificate_engine, + prev->certificate_engine, ""); ngx_conf_merge_str_value(conf->dhparam, prev->dhparam, ""); @@ -652,7 +670,8 @@ cln->data = &conf->ssl; if (ngx_ssl_certificate(cf, &conf->ssl, &conf->certificate, - &conf->certificate_key) + &conf->certificate_key, &conf->certificate_keyform, + &conf->certificate_engine) != NGX_OK) { return NGX_CONF_ERROR; diff -r 63d7d69d0fe4 -r 638389b21e0e src/http/modules/ngx_http_ssl_module.h --- a/src/http/modules/ngx_http_ssl_module.h Fri Jun 20 12:55:41 2014 +0400 +++ b/src/http/modules/ngx_http_ssl_module.h Tue Jul 22 13:37:56 2014 +0400 @@ -34,6 +34,9 @@ ngx_str_t certificate; ngx_str_t certificate_key; + ngx_str_t certificate_keyform; + ngx_str_t certificate_engine; + ngx_str_t dhparam; ngx_str_t ecdh_curve; ngx_str_t client_certificate; diff -r 63d7d69d0fe4 -r 638389b21e0e src/mail/ngx_mail_ssl_module.c --- a/src/mail/ngx_mail_ssl_module.c Fri Jun 20 12:55:41 2014 +0400 +++ b/src/mail/ngx_mail_ssl_module.c Tue Jul 22 13:37:56 2014 +0400 @@ -230,6 +230,10 @@ ngx_conf_merge_str_value(conf->certificate, prev->certificate, ""); ngx_conf_merge_str_value(conf->certificate_key, prev->certificate_key, ""); + ngx_conf_merge_str_value(conf->certificate_keyform, + prev->certificate_keyform, ""); + ngx_conf_merge_str_value(conf->certificate_engine, + prev->certificate_engine, ""); ngx_conf_merge_str_value(conf->dhparam, prev->dhparam, ""); @@ -302,7 +306,8 @@ cln->data = &conf->ssl; if (ngx_ssl_certificate(cf, &conf->ssl, &conf->certificate, - &conf->certificate_key) + &conf->certificate_key, &conf->certificate_keyform, + &conf->certificate_engine) != NGX_OK) { return NGX_CONF_ERROR; diff -r 63d7d69d0fe4 -r 638389b21e0e src/mail/ngx_mail_ssl_module.h --- a/src/mail/ngx_mail_ssl_module.h Fri Jun 20 12:55:41 2014 +0400 +++ b/src/mail/ngx_mail_ssl_module.h Tue Jul 22 13:37:56 2014 +0400 @@ -34,6 +34,9 @@ ngx_str_t certificate; ngx_str_t certificate_key; + ngx_str_t certificate_keyform; + ngx_str_t certificate_engine; + ngx_str_t dhparam; ngx_str_t ecdh_curve; From mdounin at mdounin.ru Tue Jul 22 14:52:52 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Tue, 22 Jul 2014 18:52:52 +0400 Subject: [PATCH] allow to use engine keyform for server private key In-Reply-To: <638389b21e0e1522ed8b.1406027795@pichulin7.cp.ru> References: <638389b21e0e1522ed8b.1406027795@pichulin7.cp.ru> Message-ID: <20140722145252.GX1849@mdounin.ru> Hello! On Tue, Jul 22, 2014 at 03:16:35PM +0400, Dmitrii Pichulin wrote: > # HG changeset patch > # User Dmitrii Pichulin > # Date 1406021876 -14400 > # Tue Jul 22 13:37:56 2014 +0400 > # Node ID 638389b21e0e1522ed8b8205012f5af562dc50c7 > # Parent 63d7d69d0fe48e030ff9fc520c7036dbd1ebc13f > allow to use engine keyform for server private key The patch is built against an old version, and there are conflicting changes in nginx 1.7.3. > > diff -r 63d7d69d0fe4 -r 638389b21e0e src/event/ngx_event_openssl.c > --- a/src/event/ngx_event_openssl.c Fri Jun 20 12:55:41 2014 +0400 > +++ b/src/event/ngx_event_openssl.c Tue Jul 22 13:37:56 2014 +0400 > @@ -257,11 +257,31 @@ > > ngx_int_t > ngx_ssl_certificate(ngx_conf_t *cf, ngx_ssl_t *ssl, ngx_str_t *cert, > - ngx_str_t *key) > + ngx_str_t *key, ngx_str_t *keyform, ngx_str_t *engine) > { > - BIO *bio; > - X509 *x509; > - u_long n; > + BIO *bio; > + X509 *x509; > + u_long n; > + ngx_uint_t ssl_use_engine_keyform = 0; > + > + if (keyform->len) { > + > + if (ngx_strcmp(keyform->data, "ENGINE") == 0) { > + ssl_use_engine_keyform = 1; > + > + } else if (ngx_strcmp(keyform->data, "PEM") != 0) { > + ngx_conf_log_error(NGX_LOG_EMERG, cf, 0, > + "invalid parameter: %V", keyform); > + return NGX_ERROR; > + } > + } > + > + if (ssl_use_engine_keyform && engine->len == 0) { > + ngx_conf_log_error(NGX_LOG_EMERG, cf, 0, > + "no \"ssl_certificate_engine\" is defined " > + "while \"ssl_certificate_keyform\" is \"ENGINE\""); > + return NGX_ERROR; > + } > > if (ngx_conf_full_name(cf->cycle, cert, 1) != NGX_OK) { > return NGX_ERROR; As previously suggested, it should be "engine=" parameter of the ssl_certificate_key directive and/or some specific path prefix to load a key from an engine (like "engine::" instead of a file name), see the thread here: http://mailman.nginx.org/pipermail/nginx-devel/2014-March/005114.html There is no need to introduce such a number of directives - there is no real need for this, and it's confusing for users. > @@ -344,17 +364,51 @@ > > BIO_free(bio); > > - if (ngx_conf_full_name(cf->cycle, key, 1) != NGX_OK) { > - return NGX_ERROR; > - } > - > - if (SSL_CTX_use_PrivateKey_file(ssl->ctx, (char *) key->data, > - SSL_FILETYPE_PEM) > - == 0) > - { > - ngx_ssl_error(NGX_LOG_EMERG, ssl->log, 0, > - "SSL_CTX_use_PrivateKey_file(\"%s\") failed", key->data); > - return NGX_ERROR; > + if (ssl_use_engine_keyform) { > + EVP_PKEY *pkey; > + ENGINE *e; > + Please define all variables at function start. > + e = ENGINE_by_id((const char *) engine->data); There is no need to use "const" in the cast. > + > + if (e == NULL) { > + ngx_ssl_error(NGX_LOG_EMERG, ssl->log, 0, > + "ENGINE_by_id(\"%s\") failed", engine->data); > + return NGX_ERROR; > + } > + > + pkey = ENGINE_load_private_key(e, (const char *)key->data, 0, 0); Style: there should be a space between "(const char *)" and "key->data". (Also, there is no need to use "const" in the cast.) > + > + ENGINE_free(e); > + > + if (!pkey) { > + ngx_ssl_error(NGX_LOG_EMERG, ssl->log, 0, > + "ENGINE_load_private_key(\"%s\") failed", key->data); > + return NGX_ERROR; > + } Error stack may be changed by the ENGINE_free(). Additinally, it looks like ENGINE_free() can return errors, and it may be a good idea to check them. > + > + if (SSL_CTX_use_PrivateKey(ssl->ctx, pkey) == 0) { > + ngx_ssl_error(NGX_LOG_EMERG, ssl->log, 0, > + "SSL_CTX_use_PrivateKey_file(\"%s\") failed", key->data); Style: it may be a good idea to break the line into two here. > + EVP_PKEY_free(pkey); > + return NGX_ERROR; > + } > + > + EVP_PKEY_free(pkey); > + > + } else { It should be better to just return here and hence avoid indentation changes on the default code path. > + > + if (ngx_conf_full_name(cf->cycle, key, 1) != NGX_OK) { > + return NGX_ERROR; > + } > + > + if (SSL_CTX_use_PrivateKey_file(ssl->ctx, > + (char *) key->data, SSL_FILETYPE_PEM) > + == 0) > + { > + ngx_ssl_error(NGX_LOG_EMERG, ssl->log, 0, > + "SSL_CTX_use_PrivateKey_file(\"%s\") failed", key->data); > + return NGX_ERROR; > + } > } > > return NGX_OK; [...] -- Maxim Dounin http://nginx.org/ From agentzh at gmail.com Tue Jul 22 23:02:49 2014 From: agentzh at gmail.com (Yichun Zhang (agentzh)) Date: Tue, 22 Jul 2014 16:02:49 -0700 Subject: [PATCH] Proxy: added timeout protection to SSL handshake. Message-ID: # HG changeset patch # User Yichun Zhang # Date 1406068295 25200 # Tue Jul 22 15:31:35 2014 -0700 # Node ID 1db962fc3522ce61313b684ca8251a6462992d40 # Parent 93614769dd4b6df8844c3c43c6a0b3f83bfa6746 Proxy: added timeout protection to SSL handshake. Previously, proxy relied on the write event timer created when connect() could not complete immediately to protect SSL handshake timeouts. But when connect() can complete in a single run, there is no timer protection at all. diff -r 93614769dd4b -r 1db962fc3522 src/http/ngx_http_upstream.c --- a/src/http/ngx_http_upstream.c Sun May 11 21:56:07 2014 -0700 +++ b/src/http/ngx_http_upstream.c Tue Jul 22 15:31:35 2014 -0700 @@ -1387,6 +1387,7 @@ ngx_http_upstream_ssl_init_connection(ng rc = ngx_ssl_handshake(c); if (rc == NGX_AGAIN) { + ngx_add_timer(c->write, u->conf->connect_timeout); c->ssl->handler = ngx_http_upstream_ssl_handshake; return; } -------------- next part -------------- A non-text attachment was scrubbed... Name: proxy-ssl-handshake-timer.patch Type: text/x-patch Size: 969 bytes Desc: not available URL: From mdounin at mdounin.ru Wed Jul 23 14:10:14 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Wed, 23 Jul 2014 18:10:14 +0400 Subject: [PATCH] Proxy: added timeout protection to SSL handshake. In-Reply-To: References: Message-ID: <20140723141014.GI1849@mdounin.ru> Hello! On Tue, Jul 22, 2014 at 04:02:49PM -0700, Yichun Zhang (agentzh) wrote: > # HG changeset patch > # User Yichun Zhang > # Date 1406068295 25200 > # Tue Jul 22 15:31:35 2014 -0700 > # Node ID 1db962fc3522ce61313b684ca8251a6462992d40 > # Parent 93614769dd4b6df8844c3c43c6a0b3f83bfa6746 > Proxy: added timeout protection to SSL handshake. > > Previously, proxy relied on the write event timer created when connect() > could not complete immediately to protect SSL handshake timeouts. But when > connect() can complete in a single run, there is no timer protection at all. > > diff -r 93614769dd4b -r 1db962fc3522 src/http/ngx_http_upstream.c > --- a/src/http/ngx_http_upstream.c Sun May 11 21:56:07 2014 -0700 > +++ b/src/http/ngx_http_upstream.c Tue Jul 22 15:31:35 2014 -0700 > @@ -1387,6 +1387,7 @@ ngx_http_upstream_ssl_init_connection(ng > rc = ngx_ssl_handshake(c); > > if (rc == NGX_AGAIN) { > + ngx_add_timer(c->write, u->conf->connect_timeout); > c->ssl->handler = ngx_http_upstream_ssl_handshake; > return; > } Thanks for noting this. I think that it would be better to use slightly different code, similar to what to we use in case of client SSL handshakes: --- a/src/http/ngx_http_upstream.c +++ b/src/http/ngx_http_upstream.c @@ -1400,6 +1400,11 @@ ngx_http_upstream_ssl_init_connection(ng rc = ngx_ssl_handshake(c); if (rc == NGX_AGAIN) { + + if (!c->write->timer_set) { + ngx_add_timer(c->write, u->conf->connect_timeout); + } + c->ssl->handler = ngx_http_upstream_ssl_handshake; return; } This will consistently limit total connect and ssl handshake time to connect_timeout in all cases. Is it looks good? -- Maxim Dounin http://nginx.org/ From pdn at cryptopro.ru Wed Jul 23 14:58:23 2014 From: pdn at cryptopro.ru (Dmitrii Pichulin) Date: Wed, 23 Jul 2014 18:58:23 +0400 Subject: [PATCH] allow to use engine keyform for server private key In-Reply-To: <20140722145252.GX1849@mdounin.ru> References: <20140722145252.GX1849@mdounin.ru> Message-ID: # HG changeset patch # User Dmitrii Pichulin # Date 1406127158 -14400 # Wed Jul 23 18:52:38 2014 +0400 # Node ID fec1d814c8f363976a1217c81faec3d80e6c718f # Parent 9de5820bb3e04d7e21727b472a15831ec0b2be1d allow to use engine keyform for server private key diff -r 9de5820bb3e0 -r fec1d814c8f3 src/event/ngx_event_openssl.c --- a/src/event/ngx_event_openssl.c Fri Jul 18 20:11:40 2014 +0400 +++ b/src/event/ngx_event_openssl.c Wed Jul 23 18:52:38 2014 +0400 @@ -11,6 +11,7 @@ #define NGX_SSL_PASSWORD_BUFFER_SIZE 4096 +#define NGX_SSL_MAX_ENGINE_NAME_LEN 260 typedef struct { @@ -270,6 +271,10 @@ u_long n; ngx_str_t *pwd; ngx_uint_t tries; + EVP_PKEY *pkey; + ENGINE *e; + char *p, *last; + char e_name[NGX_SSL_MAX_ENGINE_NAME_LEN + 1]; if (ngx_conf_full_name(cf->cycle, cert, 1) != NGX_OK) { return NGX_ERROR; @@ -352,6 +357,61 @@ BIO_free(bio); + if (ngx_strncmp(key->data, "engine:", sizeof("engine:") - 1) == 0) { + + p = (char *) key->data + sizeof("engine:") - 1; + last = ngx_strchr(p, ':'); + + if (last == NULL || ngx_strchr(last + 1, ':') != NULL) { + ngx_conf_log_error(NGX_LOG_EMERG, cf, 0, "invalid syntax: %V", key); + return NGX_ERROR; + } + + if (last - p > NGX_SSL_MAX_ENGINE_NAME_LEN) { + ngx_ssl_error(NGX_LOG_EMERG, ssl->log, 0, + "too long engine name in \"ssl_certificate_key\""); + return NGX_ERROR; + } + + ngx_memcpy(e_name, p, last - p); + e_name[last - p] = 0; + + e = ENGINE_by_id((char *) e_name); + + if (e == NULL) { + ngx_ssl_error(NGX_LOG_EMERG, ssl->log, 0, + "ENGINE_by_id(\"%s\") failed", e_name); + return NGX_ERROR; + } + + pkey = ENGINE_load_private_key(e, (char *) last + 1, 0, 0); + + if (!pkey) { + ngx_ssl_error(NGX_LOG_EMERG, ssl->log, 0, + "ENGINE_load_private_key(\"%s\") failed", last + 1); + ENGINE_free(e); + return NGX_ERROR; + } + + if (SSL_CTX_use_PrivateKey(ssl->ctx, pkey) == 0) { + + ngx_ssl_error(NGX_LOG_EMERG, ssl->log, 0, + "SSL_CTX_use_PrivateKey_file(\"%s\") failed", last + 1); + EVP_PKEY_free(pkey); + ENGINE_free(e); + return NGX_ERROR; + } + + EVP_PKEY_free(pkey); + + if (ENGINE_free(e) == 0) { + ngx_ssl_error(NGX_LOG_EMERG, ssl->log, 0, "ENGINE_free() failed"); + return NGX_ERROR; + } + + return NGX_OK; + } + if (ngx_conf_full_name(cf->cycle, key, 1) != NGX_OK) { return NGX_ERROR; } From agentzh at gmail.com Wed Jul 23 19:00:14 2014 From: agentzh at gmail.com (Yichun Zhang (agentzh)) Date: Wed, 23 Jul 2014 12:00:14 -0700 Subject: [PATCH] Proxy: added timeout protection to SSL handshake. In-Reply-To: <20140723141014.GI1849@mdounin.ru> References: <20140723141014.GI1849@mdounin.ru> Message-ID: Hi Maxim! On Wed, Jul 23, 2014 at 7:10 AM, Maxim Dounin wrote: > > Thanks for noting this. I think that it would be better to use > slightly different code, similar to what to we use in case of > client SSL handshakes: [...] > > This will consistently limit total connect and ssl handshake time > to connect_timeout in all cases. Is it looks good? > Yes, it looks good. I'm fine with the change :) Thanks! -agentzh From grrm77 at gmail.com Wed Jul 23 19:57:41 2014 From: grrm77 at gmail.com (grrm grrm) Date: Wed, 23 Jul 2014 22:57:41 +0300 Subject: Patch: Refactor ngx_http_write_request_body into a filter In-Reply-To: <20140709015800.GJ1849@mdounin.ru> References: <20140620180926.GO1849@mdounin.ru> <20140627135419.GR1849@mdounin.ru> <20140707180449.GH1849@mdounin.ru> <20140709015800.GJ1849@mdounin.ru> Message-ID: Hello, Maxim. I would like to invest some time into this. Do you point out any specific mistakes I made, or where code just doesn't look right? Or it is just that a move to the request filters would imply some design changes that would have to be considered more carefully? Although I didn't like setting flush in the request body read function, I saw that flush is set for all buffers in the proxy input filter for example in the non-buffered logic. Thanks. 2014-07-09 4:58 GMT+03:00 Maxim Dounin : > Hello! > > On Tue, Jul 08, 2014 at 11:16:47PM +0300, grrm grrm wrote: > >> Hello, >> >> Thanks for the reply. I understand the hesitation to include new code >> with changes more than cosmetic, but without any real benefit. >> What if this was part of a greater initiative to implement >> non-buffered request body processing. Would that be an acceptable >> modification to start with? (Because that was tentatively my >> intention). > > No. If you want to move writing a request body into the filter > chain, it should be done better. > > -- > Maxim Dounin > http://nginx.org/ > > _______________________________________________ > nginx-devel mailing list > nginx-devel at nginx.org > http://mailman.nginx.org/mailman/listinfo/nginx-devel From vl at nginx.com Thu Jul 24 12:27:22 2014 From: vl at nginx.com (Homutov Vladimir) Date: Thu, 24 Jul 2014 12:27:22 +0000 Subject: [nginx] Core: fixed default log initialization. Message-ID: details: http://hg.nginx.org/nginx/rev/c3b08217f2a2 branches: changeset: 5771:c3b08217f2a2 user: Vladimir Homutov date: Thu Jul 24 16:25:07 2014 +0400 description: Core: fixed default log initialization. The ngx_log_insert() function may invalidate pointer passed to it, so make sure to don't use it after the ngx_log_insert() call. diffstat: src/core/ngx_log.c | 10 ++++++---- 1 files changed, 6 insertions(+), 4 deletions(-) diffs (30 lines): diff -r 9de5820bb3e0 -r c3b08217f2a2 src/core/ngx_log.c --- a/src/core/ngx_log.c Fri Jul 18 20:11:40 2014 +0400 +++ b/src/core/ngx_log.c Thu Jul 24 16:25:07 2014 +0400 @@ -387,20 +387,22 @@ ngx_log_open_default(ngx_cycle_t *cycle) return NGX_ERROR; } - log->log_level = NGX_LOG_ERR; - ngx_log_insert(&cycle->new_log, log); - } else { /* no error logs at all */ log = &cycle->new_log; - log->log_level = NGX_LOG_ERR; } + log->log_level = NGX_LOG_ERR; + log->file = ngx_conf_open_file(cycle, &error_log); if (log->file == NULL) { return NGX_ERROR; } + if (log != &cycle->new_log) { + ngx_log_insert(&cycle->new_log, log); + } + return NGX_OK; } From mdounin at mdounin.ru Thu Jul 24 20:04:57 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Fri, 25 Jul 2014 00:04:57 +0400 Subject: Patch: Refactor ngx_http_write_request_body into a filter In-Reply-To: References: <20140620180926.GO1849@mdounin.ru> <20140627135419.GR1849@mdounin.ru> <20140707180449.GH1849@mdounin.ru> <20140709015800.GJ1849@mdounin.ru> Message-ID: <20140724200457.GR1849@mdounin.ru> Hello! On Wed, Jul 23, 2014 at 10:57:41PM +0300, grrm grrm wrote: > Hello, Maxim. > I would like to invest some time into this. Do you point out any > specific mistakes I made, or where code just doesn't look right? Or it > is just that a move to the request filters would imply some design > changes that would have to be considered more carefully? In this particular patch you are trying to make a filter from the ngx_http_write_request_body() function, and you are moving existing save filter into it - instead of using already existing save filter, and using the ngx_http_write_request_body() in it. > Although I didn't like setting flush in the request body read > function, I saw that flush is set for all buffers in the proxy input > filter for example in the non-buffered logic. Unconditionally setting the flush flag looks wrong - in particular, this will cause unneeded flushes when sending large bodies over SSL connections. -- Maxim Dounin http://nginx.org/ From grrm77 at gmail.com Thu Jul 24 20:33:24 2014 From: grrm77 at gmail.com (grrm grrm) Date: Thu, 24 Jul 2014 23:33:24 +0300 Subject: Patch: Refactor ngx_http_write_request_body into a filter In-Reply-To: <20140724200457.GR1849@mdounin.ru> References: <20140620180926.GO1849@mdounin.ru> <20140627135419.GR1849@mdounin.ru> <20140707180449.GH1849@mdounin.ru> <20140709015800.GJ1849@mdounin.ru> <20140724200457.GR1849@mdounin.ru> Message-ID: Hi! Thanks for the response. Quick question, I set the flush flag only when the request body buffer is full. Is it still a problem? 2014-07-24 23:04 GMT+03:00 Maxim Dounin : > Hello! > > On Wed, Jul 23, 2014 at 10:57:41PM +0300, grrm grrm wrote: > >> Hello, Maxim. >> I would like to invest some time into this. Do you point out any >> specific mistakes I made, or where code just doesn't look right? Or it >> is just that a move to the request filters would imply some design >> changes that would have to be considered more carefully? > > In this particular patch you are trying to make a filter from the > ngx_http_write_request_body() function, and you are moving > existing save filter into it - instead of using already existing > save filter, and using the ngx_http_write_request_body() in it. > >> Although I didn't like setting flush in the request body read >> function, I saw that flush is set for all buffers in the proxy input >> filter for example in the non-buffered logic. > > Unconditionally setting the flush flag looks wrong - in > particular, this will cause unneeded flushes when sending large > bodies over SSL connections. > > -- > Maxim Dounin > http://nginx.org/ > > _______________________________________________ > nginx-devel mailing list > nginx-devel at nginx.org > http://mailman.nginx.org/mailman/listinfo/nginx-devel From mdounin at mdounin.ru Thu Jul 24 21:00:05 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Fri, 25 Jul 2014 01:00:05 +0400 Subject: Patch: Refactor ngx_http_write_request_body into a filter In-Reply-To: References: <20140627135419.GR1849@mdounin.ru> <20140707180449.GH1849@mdounin.ru> <20140709015800.GJ1849@mdounin.ru> <20140724200457.GR1849@mdounin.ru> Message-ID: <20140724210005.GT1849@mdounin.ru> Hello! On Thu, Jul 24, 2014 at 11:33:24PM +0300, grrm grrm wrote: > Hi! Thanks for the response. Quick question, I set the flush flag only > when the request body buffer is full. Is it still a problem? This may be ok if it's then handled and removed, and won't stay in the resulting body buffers chain. > > 2014-07-24 23:04 GMT+03:00 Maxim Dounin : > > Hello! > > > > On Wed, Jul 23, 2014 at 10:57:41PM +0300, grrm grrm wrote: > > > >> Hello, Maxim. > >> I would like to invest some time into this. Do you point out any > >> specific mistakes I made, or where code just doesn't look right? Or it > >> is just that a move to the request filters would imply some design > >> changes that would have to be considered more carefully? > > > > In this particular patch you are trying to make a filter from the > > ngx_http_write_request_body() function, and you are moving > > existing save filter into it - instead of using already existing > > save filter, and using the ngx_http_write_request_body() in it. > > > >> Although I didn't like setting flush in the request body read > >> function, I saw that flush is set for all buffers in the proxy input > >> filter for example in the non-buffered logic. > > > > Unconditionally setting the flush flag looks wrong - in > > particular, this will cause unneeded flushes when sending large > > bodies over SSL connections. > > > > -- > > Maxim Dounin > > http://nginx.org/ > > > > _______________________________________________ > > nginx-devel mailing list > > nginx-devel at nginx.org > > http://mailman.nginx.org/mailman/listinfo/nginx-devel > > _______________________________________________ > nginx-devel mailing list > nginx-devel at nginx.org > http://mailman.nginx.org/mailman/listinfo/nginx-devel -- Maxim Dounin http://nginx.org/ From piotr at cloudflare.com Fri Jul 25 11:48:51 2014 From: piotr at cloudflare.com (Piotr Sikora) Date: Fri, 25 Jul 2014 04:48:51 -0700 Subject: [PATCH] Proxy: add "proxy_ssl_padding" directive Message-ID: # HG changeset patch # User Piotr Sikora # Date 1406288796 25200 # Fri Jul 25 04:46:36 2014 -0700 # Node ID fa9bca0cb2876eb57048644aa4af15d1e6c85d26 # Parent c3b08217f2a24f4531e578082dff498d85818cf0 Proxy: add "proxy_ssl_padding" directive. This change adds support for the TLS padding extension (the workaround for the "TLS hang bug"), which might be necessary in order to establish SSL connection with upstream servers with and/or behind broken SSL stack. Previously, it was possible to connect to such servers only by reducing size of the ClientHello message to below 256 bytes (by reducing number of advertised cipher suites, removing support for newer SSL protocols and/or removing the Server Name Indication extension). Requires OpenSSL-1.0.1h+. Signed-off-by: Piotr Sikora diff -r c3b08217f2a2 -r fa9bca0cb287 src/http/modules/ngx_http_proxy_module.c --- a/src/http/modules/ngx_http_proxy_module.c Thu Jul 24 16:25:07 2014 +0400 +++ b/src/http/modules/ngx_http_proxy_module.c Fri Jul 25 04:46:36 2014 -0700 @@ -84,6 +84,7 @@ typedef struct { ngx_uint_t ssl_verify_depth; ngx_str_t ssl_trusted_certificate; ngx_str_t ssl_crl; + ngx_flag_t ssl_padding; #endif } ngx_http_proxy_loc_conf_t; @@ -164,6 +165,10 @@ static char *ngx_http_proxy_cache_key(ng #endif static char *ngx_http_proxy_lowat_check(ngx_conf_t *cf, void *post, void *data); +#if (NGX_HTTP_SSL) +static char *ngx_http_proxy_ssl_padding_check(ngx_conf_t *cf, void *post, + void *data); +#endif static ngx_int_t ngx_http_proxy_rewrite_regex(ngx_conf_t *cf, ngx_http_proxy_rewrite_t *pr, ngx_str_t *regex, ngx_uint_t caseless); @@ -177,6 +182,10 @@ static void ngx_http_proxy_set_vars(ngx_ static ngx_conf_post_t ngx_http_proxy_lowat_post = { ngx_http_proxy_lowat_check }; +#if (NGX_HTTP_SSL) +static ngx_conf_post_t ngx_http_proxy_ssl_padding_post = + { ngx_http_proxy_ssl_padding_check }; +#endif static ngx_conf_bitmask_t ngx_http_proxy_next_upstream_masks[] = { @@ -598,6 +607,13 @@ static ngx_command_t ngx_http_proxy_com offsetof(ngx_http_proxy_loc_conf_t, ssl_crl), NULL }, + { ngx_string("proxy_ssl_padding"), + NGX_HTTP_MAIN_CONF|NGX_HTTP_SRV_CONF|NGX_HTTP_LOC_CONF|NGX_CONF_FLAG, + ngx_conf_set_flag_slot, + NGX_HTTP_LOC_CONF_OFFSET, + offsetof(ngx_http_proxy_loc_conf_t, ssl_padding), + &ngx_http_proxy_ssl_padding_post }, + #endif ngx_null_command @@ -2495,6 +2511,7 @@ ngx_http_proxy_create_loc_conf(ngx_conf_ conf->upstream.ssl_server_name = NGX_CONF_UNSET; conf->upstream.ssl_verify = NGX_CONF_UNSET; conf->ssl_verify_depth = NGX_CONF_UNSET_UINT; + conf->ssl_padding = NGX_CONF_UNSET; #endif /* "proxy_cyclic_temp_file" is disabled */ @@ -2791,6 +2808,7 @@ ngx_http_proxy_merge_loc_conf(ngx_conf_t ngx_conf_merge_str_value(conf->ssl_trusted_certificate, prev->ssl_trusted_certificate, ""); ngx_conf_merge_str_value(conf->ssl_crl, prev->ssl_crl, ""); + ngx_conf_merge_value(conf->ssl_padding, prev->ssl_padding, 0); if (conf->ssl && ngx_http_proxy_set_ssl(cf, conf) != NGX_OK) { return NGX_CONF_ERROR; @@ -3824,6 +3842,18 @@ ngx_http_proxy_lowat_check(ngx_conf_t *c #if (NGX_HTTP_SSL) +static char * +ngx_http_proxy_ssl_padding_check(ngx_conf_t *cf, void *post, void *data) +{ +#ifndef SSL_OP_TLSEXT_PADDING + ngx_conf_log_error(NGX_LOG_WARN, cf, 0, + "\"proxy_ssl_padding\" is not supported, ignored"); +#endif + + return NGX_CONF_OK; +} + + static ngx_int_t ngx_http_proxy_set_ssl(ngx_conf_t *cf, ngx_http_proxy_loc_conf_t *plcf) { @@ -3880,6 +3910,12 @@ ngx_http_proxy_set_ssl(ngx_conf_t *cf, n } } +#ifdef SSL_OP_TLSEXT_PADDING + if (plcf->ssl_padding) { + SSL_CTX_set_options(plcf->upstream.ssl->ctx, SSL_OP_TLSEXT_PADDING); + } +#endif + return NGX_OK; } From mdounin at mdounin.ru Fri Jul 25 15:56:27 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Fri, 25 Jul 2014 19:56:27 +0400 Subject: [PATCH] Proxy: add "proxy_ssl_padding" directive In-Reply-To: References: Message-ID: <20140725155627.GW1849@mdounin.ru> Hello! On Fri, Jul 25, 2014 at 04:48:51AM -0700, Piotr Sikora wrote: > # HG changeset patch > # User Piotr Sikora > # Date 1406288796 25200 > # Fri Jul 25 04:46:36 2014 -0700 > # Node ID fa9bca0cb2876eb57048644aa4af15d1e6c85d26 > # Parent c3b08217f2a24f4531e578082dff498d85818cf0 > Proxy: add "proxy_ssl_padding" directive. > > This change adds support for the TLS padding extension (the workaround > for the "TLS hang bug"), which might be necessary in order to establish > SSL connection with upstream servers with and/or behind broken SSL stack. > > Previously, it was possible to connect to such servers only by reducing > size of the ClientHello message to below 256 bytes (by reducing number > of advertised cipher suites, removing support for newer SSL protocols > and/or removing the Server Name Indication extension). > > Requires OpenSSL-1.0.1h+. And it is also known to cause problems with some other broken SSL stacks: https://bugzilla.mozilla.org/show_bug.cgi?id=989062 https://rt.openssl.org/Ticket/Display.html?id=3336 So it doesn't looks like a good candidate for enabling unconditionally, like we do with other workaround options. On the other hand, I don't think it worth adding a configuration directive to control it. We've recently introduced proxy_ssl_protocols and proxy_ssl_ciphers mostly to mitigate issues with such broken servers, and it should be enough. -- Maxim Dounin http://nginx.org/ From piotr at cloudflare.com Fri Jul 25 19:06:16 2014 From: piotr at cloudflare.com (Piotr Sikora) Date: Fri, 25 Jul 2014 12:06:16 -0700 Subject: [PATCH] Proxy: add "proxy_ssl_padding" directive In-Reply-To: <20140725155627.GW1849@mdounin.ru> References: <20140725155627.GW1849@mdounin.ru> Message-ID: Hey, > And it is also known to cause problems with some other broken > SSL stacks: > > https://bugzilla.mozilla.org/show_bug.cgi?id=989062 > https://rt.openssl.org/Ticket/Display.html?id=3336 > > So it doesn't looks like a good candidate for enabling > unconditionally, like we do with other workaround options. Agreed, that's why I added it as an option. > On the > other hand, I don't think it worth adding a configuration > directive to control it. We've recently introduced > proxy_ssl_protocols and proxy_ssl_ciphers mostly to mitigate > issues with such broken servers, and it should be enough. Except that with "proxy_ssl_server_name" the ClientHello message can be >256 even with only a single SSL protocol and cipher suite enabled. Best regards, Piotr Sikora From mdounin at mdounin.ru Fri Jul 25 19:20:16 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Fri, 25 Jul 2014 23:20:16 +0400 Subject: [PATCH] Proxy: add "proxy_ssl_padding" directive In-Reply-To: References: <20140725155627.GW1849@mdounin.ru> Message-ID: <20140725192016.GZ1849@mdounin.ru> Hello! On Fri, Jul 25, 2014 at 12:06:16PM -0700, Piotr Sikora wrote: > > On the > > other hand, I don't think it worth adding a configuration > > directive to control it. We've recently introduced > > proxy_ssl_protocols and proxy_ssl_ciphers mostly to mitigate > > issues with such broken servers, and it should be enough. > > Except that with "proxy_ssl_server_name" the ClientHello message can > be >256 even with only a single SSL protocol and cipher suite enabled. This means that SNI can't be used with such backends (it never worked before as it can't work without TLS padding extension), and trivial solution is to don't switch it on. -- Maxim Dounin http://nginx.org/ From piotr at cloudflare.com Fri Jul 25 20:44:49 2014 From: piotr at cloudflare.com (Piotr Sikora) Date: Fri, 25 Jul 2014 13:44:49 -0700 Subject: [PATCH] Proxy: add "proxy_ssl_padding" directive In-Reply-To: <20140725192016.GZ1849@mdounin.ru> References: <20140725155627.GW1849@mdounin.ru> <20140725192016.GZ1849@mdounin.ru> Message-ID: Hey Maxim, > This means that SNI can't be used with such backends (it never > worked before as it can't work without TLS padding extension), and > trivial solution is to don't switch it on. I don't think that's valid solution, SNI should be used whenever possible. Also, I think that "on/off" switch is much nicer workaround for the problem than requiring users to play around with SSL protocols and/or cipher suites. Best regards, Piotr Sikora From piotr at cloudflare.com Fri Jul 25 20:48:56 2014 From: piotr at cloudflare.com (Piotr Sikora) Date: Fri, 25 Jul 2014 13:48:56 -0700 Subject: Fw:Re: [PATCH] Core: use uppercase hexadecimal digits forpercent-encoding In-Reply-To: References: Message-ID: Hey guys, did we reach a consensus here or is it going to stay lowercase? For what it's worth, ngx_lua already switched to uppercase [1]. [1] https://github.com/openresty/lua-nginx-module/commit/34ecd2f Best regards, Piotr Sikora From agentzh at gmail.com Fri Jul 25 22:00:31 2014 From: agentzh at gmail.com (Yichun Zhang (agentzh)) Date: Fri, 25 Jul 2014 15:00:31 -0700 Subject: [PATCH] GeoIP: not all variable fields were initialized. Message-ID: # HG changeset patch # User Yichun Zhang # Date 1406324609 25200 # Fri Jul 25 14:43:29 2014 -0700 # Node ID c038cc33739bbfab2ed50819191298471f22d233 # Parent 93614769dd4b6df8844c3c43c6a0b3f83bfa6746 GeoIP: not all variable fields were initialized. The ngx_http_geoip_city_float_variable and ngx_http_geoip_city_int_variable functions did not always initialize all variable fields like "not_found", which could lead to empty values for those corresponding nginx variales randomly. diff -r 93614769dd4b -r c038cc33739b src/http/modules/ngx_http_geoip_module.c --- a/src/http/modules/ngx_http_geoip_module.c Sun May 11 21:56:07 2014 -0700 +++ b/src/http/modules/ngx_http_geoip_module.c Fri Jul 25 14:43:29 2014 -0700 @@ -553,6 +553,9 @@ ngx_http_geoip_city_float_variable(ngx_h val = *(float *) ((char *) gr + data); v->len = ngx_sprintf(v->data, "%.4f", val) - v->data; + v->valid = 1; + v->no_cacheable = 0; + v->not_found = 0; GeoIPRecord_delete(gr); @@ -582,6 +585,9 @@ ngx_http_geoip_city_int_variable(ngx_htt val = *(int *) ((char *) gr + data); v->len = ngx_sprintf(v->data, "%d", val) - v->data; + v->valid = 1; + v->no_cacheable = 0; + v->not_found = 0; GeoIPRecord_delete(gr); -------------- next part -------------- A non-text attachment was scrubbed... Name: geoip-init-var-fields.patch Type: text/x-patch Size: 1272 bytes Desc: not available URL: From mdounin at mdounin.ru Sun Jul 27 17:01:22 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Sun, 27 Jul 2014 17:01:22 +0000 Subject: [nginx] Core: use uppercase hexadecimal digits for percent-encod... Message-ID: details: http://hg.nginx.org/nginx/rev/d421be0d842a branches: changeset: 5772:d421be0d842a user: Piotr Sikora date: Thu Jun 26 23:39:23 2014 -0700 description: Core: use uppercase hexadecimal digits for percent-encoding. RFC3986 says that, for consistency, URI producers and normalizers should use uppercase hexadecimal digits for all percent-encodings. This is also what modern web browsers and other tools use. Using lowercase hexadecimal digits makes it harder to interact with those tools in case when use of the percent-encoded URI is required, for example when $request_uri is part of the cache key. Signed-off-by: Piotr Sikora diffstat: src/core/ngx_string.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diffs (12 lines): diff --git a/src/core/ngx_string.c b/src/core/ngx_string.c --- a/src/core/ngx_string.c +++ b/src/core/ngx_string.c @@ -1407,7 +1407,7 @@ ngx_escape_uri(u_char *dst, u_char *src, { ngx_uint_t n; uint32_t *escape; - static u_char hex[] = "0123456789abcdef"; + static u_char hex[] = "0123456789ABCDEF"; /* " ", "#", "%", "?", %00-%1F, %7F-%FF */ From mdounin at mdounin.ru Sun Jul 27 17:02:19 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Sun, 27 Jul 2014 21:02:19 +0400 Subject: Fw:Re: [PATCH] Core: use uppercase hexadecimal digits forpercent-encoding In-Reply-To: References: Message-ID: <20140727170219.GB1849@mdounin.ru> Hello! On Fri, Jul 25, 2014 at 01:48:56PM -0700, Piotr Sikora wrote: > Hey guys, > did we reach a consensus here or is it going to stay lowercase? General consensus seems to be to do the change. I've pushed the patch, thanks. -- Maxim Dounin http://nginx.org/ From grrm77 at gmail.com Sun Jul 27 17:40:01 2014 From: grrm77 at gmail.com (grrm grrm) Date: Sun, 27 Jul 2014 20:40:01 +0300 Subject: Patch: Refactor ngx_http_write_request_body into a filter In-Reply-To: <20140724210005.GT1849@mdounin.ru> References: <20140627135419.GR1849@mdounin.ru> <20140707180449.GH1849@mdounin.ru> <20140709015800.GJ1849@mdounin.ru> <20140724200457.GR1849@mdounin.ru> <20140724210005.GT1849@mdounin.ru> Message-ID: Hello, As I said before, "refactoring" change's purpose was to be a first step to implement steamed request body sending to backend. Looking at the implementation of the response processing, I saw that all the events are handled by the upstream module. By that I mean that the read and write event handlers on the client and upstream socket are functions in the upstream module. Wouldn't it be appropriate that the request body reading and sending part be implemented in the upstream module too? I say that because the request_body_filter looks more like the upstream input_filter rather than the ngx_http_output_filter. Taking also into consideration the patch for inserting custom request body filters. If this is true, now I understand what you meant by "not clear it should work this way" :) Thank you. 2014-07-25 0:00 GMT+03:00 Maxim Dounin : > Hello! > > On Thu, Jul 24, 2014 at 11:33:24PM +0300, grrm grrm wrote: > >> Hi! Thanks for the response. Quick question, I set the flush flag only >> when the request body buffer is full. Is it still a problem? > > This may be ok if it's then handled and removed, and won't stay in > the resulting body buffers chain. > >> >> 2014-07-24 23:04 GMT+03:00 Maxim Dounin : >> > Hello! >> > >> > On Wed, Jul 23, 2014 at 10:57:41PM +0300, grrm grrm wrote: >> > >> >> Hello, Maxim. >> >> I would like to invest some time into this. Do you point out any >> >> specific mistakes I made, or where code just doesn't look right? Or it >> >> is just that a move to the request filters would imply some design >> >> changes that would have to be considered more carefully? >> > >> > In this particular patch you are trying to make a filter from the >> > ngx_http_write_request_body() function, and you are moving >> > existing save filter into it - instead of using already existing >> > save filter, and using the ngx_http_write_request_body() in it. >> > >> >> Although I didn't like setting flush in the request body read >> >> function, I saw that flush is set for all buffers in the proxy input >> >> filter for example in the non-buffered logic. >> > >> > Unconditionally setting the flush flag looks wrong - in >> > particular, this will cause unneeded flushes when sending large >> > bodies over SSL connections. >> > >> > -- >> > Maxim Dounin >> > http://nginx.org/ >> > >> > _______________________________________________ >> > nginx-devel mailing list >> > nginx-devel at nginx.org >> > http://mailman.nginx.org/mailman/listinfo/nginx-devel >> >> _______________________________________________ >> nginx-devel mailing list >> nginx-devel at nginx.org >> http://mailman.nginx.org/mailman/listinfo/nginx-devel > > -- > Maxim Dounin > http://nginx.org/ > > _______________________________________________ > nginx-devel mailing list > nginx-devel at nginx.org > http://mailman.nginx.org/mailman/listinfo/nginx-devel From mdounin at mdounin.ru Mon Jul 28 02:16:30 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Mon, 28 Jul 2014 02:16:30 +0000 Subject: [nginx] GeoIP: not all variable fields were initialized. Message-ID: details: http://hg.nginx.org/nginx/rev/494c2c2a0247 branches: changeset: 5773:494c2c2a0247 user: Yichun Zhang date: Fri Jul 25 14:43:29 2014 -0700 description: GeoIP: not all variable fields were initialized. The ngx_http_geoip_city_float_variable and ngx_http_geoip_city_int_variable functions did not always initialize all variable fields like "not_found", which could lead to empty values for those corresponding nginx variables randomly. diffstat: src/http/modules/ngx_http_geoip_module.c | 6 ++++++ 1 files changed, 6 insertions(+), 0 deletions(-) diffs (23 lines): diff --git a/src/http/modules/ngx_http_geoip_module.c b/src/http/modules/ngx_http_geoip_module.c --- a/src/http/modules/ngx_http_geoip_module.c +++ b/src/http/modules/ngx_http_geoip_module.c @@ -553,6 +553,9 @@ ngx_http_geoip_city_float_variable(ngx_h val = *(float *) ((char *) gr + data); v->len = ngx_sprintf(v->data, "%.4f", val) - v->data; + v->valid = 1; + v->no_cacheable = 0; + v->not_found = 0; GeoIPRecord_delete(gr); @@ -582,6 +585,9 @@ ngx_http_geoip_city_int_variable(ngx_htt val = *(int *) ((char *) gr + data); v->len = ngx_sprintf(v->data, "%d", val) - v->data; + v->valid = 1; + v->no_cacheable = 0; + v->not_found = 0; GeoIPRecord_delete(gr); From mdounin at mdounin.ru Mon Jul 28 02:17:24 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Mon, 28 Jul 2014 06:17:24 +0400 Subject: [PATCH] GeoIP: not all variable fields were initialized. In-Reply-To: References: Message-ID: <20140728021724.GD1849@mdounin.ru> Hello! On Fri, Jul 25, 2014 at 03:00:31PM -0700, Yichun Zhang (agentzh) wrote: > # HG changeset patch > # User Yichun Zhang > # Date 1406324609 25200 > # Fri Jul 25 14:43:29 2014 -0700 > # Node ID c038cc33739bbfab2ed50819191298471f22d233 > # Parent 93614769dd4b6df8844c3c43c6a0b3f83bfa6746 > GeoIP: not all variable fields were initialized. > > The ngx_http_geoip_city_float_variable and > ngx_http_geoip_city_int_variable functions did not always initialize > all variable fields like "not_found", which could lead to empty values > for those corresponding nginx variales randomly. Committed, thanks. -- Maxim Dounin http://nginx.org/ From mdounin at mdounin.ru Mon Jul 28 02:43:38 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Mon, 28 Jul 2014 06:43:38 +0400 Subject: [PATCH] allow to use engine keyform for server private key In-Reply-To: References: <20140722145252.GX1849@mdounin.ru> Message-ID: <20140728024338.GE1849@mdounin.ru> Hello! On Wed, Jul 23, 2014 at 06:58:23PM +0400, Dmitrii Pichulin wrote: > # HG changeset patch > # User Dmitrii Pichulin > # Date 1406127158 -14400 > # Wed Jul 23 18:52:38 2014 +0400 > # Node ID fec1d814c8f363976a1217c81faec3d80e6c718f > # Parent 9de5820bb3e04d7e21727b472a15831ec0b2be1d > allow to use engine keyform for server private key > > diff -r 9de5820bb3e0 -r fec1d814c8f3 src/event/ngx_event_openssl.c > --- a/src/event/ngx_event_openssl.c Fri Jul 18 20:11:40 2014 +0400 > +++ b/src/event/ngx_event_openssl.c Wed Jul 23 18:52:38 2014 +0400 > @@ -11,6 +11,7 @@ > > > #define NGX_SSL_PASSWORD_BUFFER_SIZE 4096 > +#define NGX_SSL_MAX_ENGINE_NAME_LEN 260 > > > typedef struct { > @@ -270,6 +271,10 @@ > u_long n; > ngx_str_t *pwd; > ngx_uint_t tries; > + EVP_PKEY *pkey; > + ENGINE *e; > + char *p, *last; > + char e_name[NGX_SSL_MAX_ENGINE_NAME_LEN + 1]; This doesn't match style used in nginx code. Both declaration order and variable names should be different. > > if (ngx_conf_full_name(cf->cycle, cert, 1) != NGX_OK) { > return NGX_ERROR; > @@ -352,6 +357,61 @@ > > BIO_free(bio); > > + if (ngx_strncmp(key->data, "engine:", sizeof("engine:") - 1) == 0) { > + > + p = (char *) key->data + sizeof("engine:") - 1; > + last = ngx_strchr(p, ':'); > + > + if (last == NULL || ngx_strchr(last + 1, ':') != NULL) { Why key id shouldn't contain colons? This restriction on key id looks strange. > + ngx_conf_log_error(NGX_LOG_EMERG, cf, 0, "invalid syntax: %V", key); > + return NGX_ERROR; > + } > + > + if (last - p > NGX_SSL_MAX_ENGINE_NAME_LEN) { > + ngx_ssl_error(NGX_LOG_EMERG, ssl->log, 0, > + "too long engine name in \"ssl_certificate_key\""); > + return NGX_ERROR; > + } > + > + ngx_memcpy(e_name, p, last - p); > + e_name[last - p] = 0; It should be fine to assume we can modify the key name passed, and just set ':' to '\0' instead of using a fixed-size buffer. > + > + e = ENGINE_by_id((char *) e_name); > + > + if (e == NULL) { > + ngx_ssl_error(NGX_LOG_EMERG, ssl->log, 0, > + "ENGINE_by_id(\"%s\") failed", e_name); > + return NGX_ERROR; > + } > + > + pkey = ENGINE_load_private_key(e, (char *) last + 1, 0, 0); It looks like ENGINE_load_private_key() may need to ask user for a password. The ssl_password_file should be respected here as well. > + > + if (!pkey) { > + ngx_ssl_error(NGX_LOG_EMERG, ssl->log, 0, > + "ENGINE_load_private_key(\"%s\") failed", last + 1); > + ENGINE_free(e); > + return NGX_ERROR; > + } > + > + if (SSL_CTX_use_PrivateKey(ssl->ctx, pkey) == 0) { > + > + ngx_ssl_error(NGX_LOG_EMERG, ssl->log, 0, > + "SSL_CTX_use_PrivateKey_file(\"%s\") failed", last + 1); I believe I've already pointed out in the previous review that it's better to split the line instead of shifting it to the left. > + EVP_PKEY_free(pkey); > + ENGINE_free(e); > + return NGX_ERROR; > + } > + > + EVP_PKEY_free(pkey); > + > + if (ENGINE_free(e) == 0) { > + ngx_ssl_error(NGX_LOG_EMERG, ssl->log, 0, "ENGINE_free() failed"); > + return NGX_ERROR; > + } Checking ENGINE_free(e) errors in one place but not in others is really bad idea. > + > + return NGX_OK; > + } > + > if (ngx_conf_full_name(cf->cycle, key, 1) != NGX_OK) { > return NGX_ERROR; > } > > _______________________________________________ > nginx-devel mailing list > nginx-devel at nginx.org > http://mailman.nginx.org/mailman/listinfo/nginx-devel -- Maxim Dounin http://nginx.org/ From hewanxiang at gmail.com Mon Jul 28 13:21:07 2014 From: hewanxiang at gmail.com (Andy) Date: Mon, 28 Jul 2014 21:21:07 +0800 Subject: Is there an efficient method to get request header by value? Message-ID: Hi dev, My code need to parse some private request headers, is there any method already in the core to do this job, get the request header by header value? I don't want to iterate the array one by one to compare the key. I see only some standard headers are set in the request_headers_in hash table. Thanks, Andy -------------- next part -------------- An HTML attachment was scrubbed... URL: From mdounin at mdounin.ru Mon Jul 28 13:44:10 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Mon, 28 Jul 2014 17:44:10 +0400 Subject: Is there an efficient method to get request header by value? In-Reply-To: References: Message-ID: <20140728134410.GH1849@mdounin.ru> Hello! On Mon, Jul 28, 2014 at 09:21:07PM +0800, Andy wrote: > Hi dev, > > My code need to parse some private request headers, is there any method > already in the core to do this job, get the request header by header value? > I don't want to iterate the array one by one to compare the key. I see only > some standard headers are set in the request_headers_in hash table. You mean "by header name"? There are two possible ways: - iterate through the r->headers_in list; - provide a way to configure variable or a complex value instead of a hardcoded private header name, and use $http_
by default. -- Maxim Dounin http://nginx.org/ From hewanxiang at gmail.com Mon Jul 28 14:29:06 2014 From: hewanxiang at gmail.com (Andy) Date: Mon, 28 Jul 2014 22:29:06 +0800 Subject: Is there an efficient method to get request header by value? In-Reply-To: <20140728134410.GH1849@mdounin.ru> References: <20140728134410.GH1849@mdounin.ru> Message-ID: Thanks Maxim, then I may have to iterate the list. Regards, Andy On Mon, Jul 28, 2014 at 9:44 PM, Maxim Dounin wrote: > Hello! > > On Mon, Jul 28, 2014 at 09:21:07PM +0800, Andy wrote: > > > Hi dev, > > > > My code need to parse some private request headers, is there any method > > already in the core to do this job, get the request header by header > value? > > I don't want to iterate the array one by one to compare the key. I see > only > > some standard headers are set in the request_headers_in hash table. > > You mean "by header name"? > There are two possible ways: > > - iterate through the r->headers_in list; > > - provide a way to configure variable or a complex value instead > of a hardcoded private header name, and use $http_
by > default. > > -- > Maxim Dounin > http://nginx.org/ > > _______________________________________________ > nginx-devel mailing list > nginx-devel at nginx.org > http://mailman.nginx.org/mailman/listinfo/nginx-devel > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mdounin at mdounin.ru Mon Jul 28 14:31:36 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Mon, 28 Jul 2014 14:31:36 +0000 Subject: [nginx] Upstream: SSL handshake timeouts. Message-ID: details: http://hg.nginx.org/nginx/rev/9bf58a3da25b branches: changeset: 5774:9bf58a3da25b user: Maxim Dounin date: Mon Jul 28 18:30:14 2014 +0400 description: Upstream: SSL handshake timeouts. Timeout may not be set on an upstream connection when we call ngx_ssl_handshake() in ngx_http_upstream_ssl_init_connection(), so make sure to arm it if it's not set. Based on a patch by Yichun Zhang. diffstat: src/http/ngx_http_upstream.c | 5 +++++ 1 files changed, 5 insertions(+), 0 deletions(-) diffs (15 lines): diff --git a/src/http/ngx_http_upstream.c b/src/http/ngx_http_upstream.c --- a/src/http/ngx_http_upstream.c +++ b/src/http/ngx_http_upstream.c @@ -1393,6 +1393,11 @@ ngx_http_upstream_ssl_init_connection(ng rc = ngx_ssl_handshake(c); if (rc == NGX_AGAIN) { + + if (!c->write->timer_set) { + ngx_add_timer(c->write, u->conf->connect_timeout); + } + c->ssl->handler = ngx_http_upstream_ssl_handshake; return; } From mdounin at mdounin.ru Mon Jul 28 14:31:46 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Mon, 28 Jul 2014 14:31:46 +0000 Subject: [nginx] SSL: misplaced space in debug message. Message-ID: details: http://hg.nginx.org/nginx/rev/294d020bbcfe branches: changeset: 5775:294d020bbcfe user: Maxim Dounin date: Mon Jul 28 18:30:18 2014 +0400 description: SSL: misplaced space in debug message. diffstat: src/event/ngx_event_openssl.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diffs (12 lines): diff --git a/src/event/ngx_event_openssl.c b/src/event/ngx_event_openssl.c --- a/src/event/ngx_event_openssl.c +++ b/src/event/ngx_event_openssl.c @@ -588,7 +588,7 @@ ngx_ssl_verify_callback(int ok, X509_STO ngx_log_debug5(NGX_LOG_DEBUG_EVENT, c->log, 0, "verify:%d, error:%d, depth:%d, " - "subject:\"%s\",issuer: \"%s\"", + "subject:\"%s\", issuer:\"%s\"", ok, err, depth, subject, issuer); if (sname) { From mdounin at mdounin.ru Mon Jul 28 14:31:56 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Mon, 28 Jul 2014 14:31:56 +0000 Subject: [nginx] Bigger iovec buffer in ngx_readv_chain(). Message-ID: details: http://hg.nginx.org/nginx/rev/d1bde5c3c5d2 branches: changeset: 5776:d1bde5c3c5d2 user: Maxim Dounin date: Mon Jul 28 18:30:19 2014 +0400 description: Bigger iovec buffer in ngx_readv_chain(). This helps to reduce likelyhood of memory allocations in ngx_readv_chain(), which are known to lead to noticeable effects in some cases, see http://mailman.nginx.org/pipermail/nginx/2014-July/044512.html. diffstat: src/os/unix/ngx_readv_chain.c | 6 +++++- 1 files changed, 5 insertions(+), 1 deletions(-) diffs (16 lines): diff --git a/src/os/unix/ngx_readv_chain.c b/src/os/unix/ngx_readv_chain.c --- a/src/os/unix/ngx_readv_chain.c +++ b/src/os/unix/ngx_readv_chain.c @@ -10,7 +10,11 @@ #include -#define NGX_IOVS 16 +#if (IOV_MAX > 64) +#define NGX_IOVS 64 +#else +#define NGX_IOVS IOV_MAX +#endif #if (NGX_HAVE_KQUEUE) From piotr at cloudflare.com Mon Jul 28 23:03:35 2014 From: piotr at cloudflare.com (Piotr Sikora) Date: Mon, 28 Jul 2014 16:03:35 -0700 Subject: [PATCH 1 of 2] SSL: fix build with OPENSSL_NO_ENGINE and/or OPENSSL_NO_OCSP Message-ID: # HG changeset patch # User Piotr Sikora # Date 1406575677 25200 # Mon Jul 28 12:27:57 2014 -0700 # Node ID bb74dfefeec04aae5a3a86ace2df45d03f691ded # Parent d1bde5c3c5d21368de04a59506a06c1174353a19 SSL: fix build with OPENSSL_NO_ENGINE and/or OPENSSL_NO_OCSP. This is really just a prerequisite for building against BoringSSL, which doesn't provide either of those features. Signed-off-by: Piotr Sikora diff -r d1bde5c3c5d2 -r bb74dfefeec0 src/event/ngx_event_openssl.c --- a/src/event/ngx_event_openssl.c Mon Jul 28 18:30:19 2014 +0400 +++ b/src/event/ngx_event_openssl.c Mon Jul 28 12:27:57 2014 -0700 @@ -3279,6 +3279,8 @@ ngx_openssl_create_conf(ngx_cycle_t *cyc static char * ngx_openssl_engine(ngx_conf_t *cf, ngx_command_t *cmd, void *conf) { +#ifndef OPENSSL_NO_ENGINE + ngx_openssl_conf_t *oscf = conf; ENGINE *engine; @@ -3313,6 +3315,12 @@ ngx_openssl_engine(ngx_conf_t *cf, ngx_c ENGINE_free(engine); return NGX_CONF_OK; + +#else + + return "is not supported"; + +#endif } @@ -3320,5 +3328,8 @@ static void ngx_openssl_exit(ngx_cycle_t *cycle) { EVP_cleanup(); + +#ifndef OPENSSL_NO_ENGINE ENGINE_cleanup(); +#endif } diff -r d1bde5c3c5d2 -r bb74dfefeec0 src/event/ngx_event_openssl.h --- a/src/event/ngx_event_openssl.h Mon Jul 28 18:30:19 2014 +0400 +++ b/src/event/ngx_event_openssl.h Mon Jul 28 12:27:57 2014 -0700 @@ -18,9 +18,13 @@ #include #include #include +#ifndef OPENSSL_NO_ENGINE #include +#endif #include +#ifndef OPENSSL_NO_OCSP #include +#endif #include #include #include diff -r d1bde5c3c5d2 -r bb74dfefeec0 src/event/ngx_event_openssl_stapling.c --- a/src/event/ngx_event_openssl_stapling.c Mon Jul 28 18:30:19 2014 +0400 +++ b/src/event/ngx_event_openssl_stapling.c Mon Jul 28 12:27:57 2014 -0700 @@ -11,7 +11,7 @@ #include -#ifdef SSL_CTRL_SET_TLSEXT_STATUS_REQ_CB +#if (!defined OPENSSL_NO_OCSP && defined SSL_CTRL_SET_TLSEXT_STATUS_REQ_CB) typedef struct { From piotr at cloudflare.com Mon Jul 28 23:03:36 2014 From: piotr at cloudflare.com (Piotr Sikora) Date: Mon, 28 Jul 2014 16:03:36 -0700 Subject: [PATCH 2 of 2] SSL: let it build against BoringSSL In-Reply-To: References: Message-ID: # HG changeset patch # User Piotr Sikora # Date 1406575677 25200 # Mon Jul 28 12:27:57 2014 -0700 # Node ID e3086fd5e59335f4f3f165ee74c094a7aca2aeb3 # Parent bb74dfefeec04aae5a3a86ace2df45d03f691ded SSL: let it build against BoringSSL. This change adds support for using BoringSSL as a drop-in replacement for OpenSSL without adding support for any of the BoringSSL-specific features. The #ifndefs around SSL_CTX_set_tmp_rsa_callback() aren't strictly necessary, since that function still exists in BoringSSL as a no-op, but they clearly mark the unsupported feature. Signed-off-by: Piotr Sikora diff -r bb74dfefeec0 -r e3086fd5e593 src/event/ngx_event_openssl.c --- a/src/event/ngx_event_openssl.c Mon Jul 28 12:27:57 2014 -0700 +++ b/src/event/ngx_event_openssl.c Mon Jul 28 12:27:57 2014 -0700 @@ -106,7 +106,9 @@ int ngx_ssl_stapling_index; ngx_int_t ngx_ssl_init(ngx_log_t *log) { +#ifndef OPENSSL_IS_BORINGSSL OPENSSL_config(NULL); +#endif SSL_library_init(); SSL_load_error_strings(); @@ -217,7 +219,10 @@ ngx_ssl_create(ngx_ssl_t *ssl, ngx_uint_ SSL_CTX_set_options(ssl->ctx, SSL_OP_MSIE_SSLV2_RSA_PADDING); #endif +#ifdef SSL_OP_SSLEAY_080_CLIENT_DH_BUG SSL_CTX_set_options(ssl->ctx, SSL_OP_SSLEAY_080_CLIENT_DH_BUG); +#endif + SSL_CTX_set_options(ssl->ctx, SSL_OP_TLS_D5_BUG); SSL_CTX_set_options(ssl->ctx, SSL_OP_TLS_BLOCK_PADDING_BUG); @@ -382,8 +387,13 @@ ngx_ssl_certificate(ngx_conf_t *cf, ngx_ if (--tries) { n = ERR_peek_error(); +#ifdef OPENSSL_IS_BORINGSSL + if (ERR_GET_LIB(n) == ERR_LIB_CIPHER + && ERR_GET_REASON(n) == CIPHER_R_BAD_DECRYPT) +#else if (ERR_GET_LIB(n) == ERR_LIB_EVP && ERR_GET_REASON(n) == EVP_R_BAD_DECRYPT) +#endif { ERR_clear_error(); SSL_CTX_set_default_passwd_cb_userdata(ssl->ctx, ++pwd); @@ -646,6 +656,8 @@ ngx_ssl_info_callback(const ngx_ssl_conn } +#ifndef OPENSSL_IS_BORINGSSL + RSA * ngx_ssl_rsa512_key_callback(ngx_ssl_conn_t *ssl_conn, int is_export, int key_length) @@ -667,6 +679,8 @@ ngx_ssl_rsa512_key_callback(ngx_ssl_conn return key; } +#endif + ngx_array_t * ngx_ssl_read_password_file(ngx_conf_t *cf, ngx_str_t *file) diff -r bb74dfefeec0 -r e3086fd5e593 src/event/ngx_event_openssl.h --- a/src/event/ngx_event_openssl.h Mon Jul 28 12:27:57 2014 -0700 +++ b/src/event/ngx_event_openssl.h Mon Jul 28 12:27:57 2014 -0700 @@ -133,8 +133,10 @@ ngx_int_t ngx_ssl_stapling(ngx_conf_t *c ngx_str_t *file, ngx_str_t *responder, ngx_uint_t verify); ngx_int_t ngx_ssl_stapling_resolver(ngx_conf_t *cf, ngx_ssl_t *ssl, ngx_resolver_t *resolver, ngx_msec_t resolver_timeout); +#ifndef OPENSSL_IS_BORINGSSL RSA *ngx_ssl_rsa512_key_callback(ngx_ssl_conn_t *ssl_conn, int is_export, int key_length); +#endif ngx_array_t *ngx_ssl_read_password_file(ngx_conf_t *cf, ngx_str_t *file); ngx_int_t ngx_ssl_dhparam(ngx_conf_t *cf, ngx_ssl_t *ssl, ngx_str_t *file); ngx_int_t ngx_ssl_ecdh_curve(ngx_conf_t *cf, ngx_ssl_t *ssl, ngx_str_t *name); diff -r bb74dfefeec0 -r e3086fd5e593 src/http/modules/ngx_http_ssl_module.c --- a/src/http/modules/ngx_http_ssl_module.c Mon Jul 28 12:27:57 2014 -0700 +++ b/src/http/modules/ngx_http_ssl_module.c Mon Jul 28 12:27:57 2014 -0700 @@ -715,8 +715,10 @@ ngx_http_ssl_merge_srv_conf(ngx_conf_t * SSL_CTX_set_options(conf->ssl.ctx, SSL_OP_CIPHER_SERVER_PREFERENCE); } +#ifndef OPENSSL_IS_BORINGSSL /* a temporary 512-bit RSA key is required for export versions of MSIE */ SSL_CTX_set_tmp_rsa_callback(conf->ssl.ctx, ngx_ssl_rsa512_key_callback); +#endif if (ngx_ssl_dhparam(cf, &conf->ssl, &conf->dhparam) != NGX_OK) { return NGX_CONF_ERROR; diff -r bb74dfefeec0 -r e3086fd5e593 src/mail/ngx_mail_ssl_module.c --- a/src/mail/ngx_mail_ssl_module.c Mon Jul 28 12:27:57 2014 -0700 +++ b/src/mail/ngx_mail_ssl_module.c Mon Jul 28 12:27:57 2014 -0700 @@ -334,7 +334,9 @@ ngx_mail_ssl_merge_conf(ngx_conf_t *cf, SSL_CTX_set_options(conf->ssl.ctx, SSL_OP_CIPHER_SERVER_PREFERENCE); } +#ifndef OPENSSL_IS_BORINGSSL SSL_CTX_set_tmp_rsa_callback(conf->ssl.ctx, ngx_ssl_rsa512_key_callback); +#endif if (ngx_ssl_dhparam(cf, &conf->ssl, &conf->dhparam) != NGX_OK) { return NGX_CONF_ERROR; From mertsas at cisco.com Tue Jul 29 06:24:21 2014 From: mertsas at cisco.com (Martin Ertsaas) Date: Tue, 29 Jul 2014 08:24:21 +0200 Subject: [PATCH 0 of 8] Allow crossbuilding and Android support In-Reply-To: References: Message-ID: <53D73E15.2090801@cisco.com> I haven't the nginx know how to evaluate this, but have anyone had the time to look at these patches? Regards Martin On 07/18/14 14:05, Per Olav H?ydahl Ohme wrote: > To allow crossbuilding: > - Added configure options informing about target platform. > - Avoiding executions of test programs for target platform features on host, > including features with successfully compiled test programs. > > Android support: > - Created Android config and init files, quite similar to existing Linux ones. > - Added crypt-function and code for globbing, and avoid O_DIRECT flag on Android > > _______________________________________________ > nginx-devel mailing list > nginx-devel at nginx.org > http://mailman.nginx.org/mailman/listinfo/nginx-devel From mdounin at mdounin.ru Tue Jul 29 12:57:56 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Tue, 29 Jul 2014 16:57:56 +0400 Subject: [PATCH 0 of 8] Allow crossbuilding and Android support In-Reply-To: <53D73E15.2090801@cisco.com> References: <53D73E15.2090801@cisco.com> Message-ID: <20140729125756.GM1849@mdounin.ru> Hello! On Tue, Jul 29, 2014 at 08:24:21AM +0200, Martin Ertsaas wrote: > I haven't the nginx know how to evaluate this, but have anyone had the > time to look at these patches? I've looked into these patches, and I don't think they worth considering. Looks like a set of dirty hacks. -- Maxim Dounin http://nginx.org/ From mertsas at cisco.com Tue Jul 29 13:02:21 2014 From: mertsas at cisco.com (Martin Ertsaas) Date: Tue, 29 Jul 2014 15:02:21 +0200 Subject: [PATCH 0 of 8] Allow crossbuilding and Android support In-Reply-To: <20140729125756.GM1849@mdounin.ru> References: <53D73E15.2090801@cisco.com> <20140729125756.GM1849@mdounin.ru> Message-ID: <53D79B5D.5050608@cisco.com> On 07/29/14 14:57, Maxim Dounin wrote: > Hello! > > On Tue, Jul 29, 2014 at 08:24:21AM +0200, Martin Ertsaas wrote: > >> I haven't the nginx know how to evaluate this, but have anyone had the >> time to look at these patches? > I've looked into these patches, and I don't think they worth > considering. Looks like a set of dirty hacks. > Do you have any specifics to what you don't like so we could look into changing them? Or are you just not interested in crossbuild support for nginx? - Martin From mdounin at mdounin.ru Tue Jul 29 13:34:11 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Tue, 29 Jul 2014 17:34:11 +0400 Subject: [PATCH 0 of 8] Allow crossbuilding and Android support In-Reply-To: <53D79B5D.5050608@cisco.com> References: <53D73E15.2090801@cisco.com> <20140729125756.GM1849@mdounin.ru> <53D79B5D.5050608@cisco.com> Message-ID: <20140729133411.GN1849@mdounin.ru> Hello! On Tue, Jul 29, 2014 at 03:02:21PM +0200, Martin Ertsaas wrote: > On 07/29/14 14:57, Maxim Dounin wrote: > > Hello! > > > > On Tue, Jul 29, 2014 at 08:24:21AM +0200, Martin Ertsaas wrote: > > > >> I haven't the nginx know how to evaluate this, but have anyone had the > >> time to look at these patches? > > I've looked into these patches, and I don't think they worth > > considering. Looks like a set of dirty hacks. > > > > Do you have any specifics to what you don't like so we could look into > changing them? Or are you just not interested in crossbuild support for > nginx? Crossbuilding was discussed more than once here. In short: while it's something nice to have, it's not something required from our point of view; therefore patches to add it should be clean, simple, and extandable; no hacks will be considered. -- Maxim Dounin http://nginx.org/ From mertsas at cisco.com Tue Jul 29 13:41:56 2014 From: mertsas at cisco.com (Martin Ertsaas) Date: Tue, 29 Jul 2014 15:41:56 +0200 Subject: [PATCH 0 of 8] Allow crossbuilding and Android support In-Reply-To: <20140729133411.GN1849@mdounin.ru> References: <53D73E15.2090801@cisco.com> <20140729125756.GM1849@mdounin.ru> <53D79B5D.5050608@cisco.com> <20140729133411.GN1849@mdounin.ru> Message-ID: <53D7A4A4.7050501@cisco.com> On 07/29/14 15:34, Maxim Dounin wrote: > Hello! > > On Tue, Jul 29, 2014 at 03:02:21PM +0200, Martin Ertsaas wrote: > >> On 07/29/14 14:57, Maxim Dounin wrote: >>> Hello! >>> >>> On Tue, Jul 29, 2014 at 08:24:21AM +0200, Martin Ertsaas wrote: >>> >>>> I haven't the nginx know how to evaluate this, but have anyone had the >>>> time to look at these patches? >>> I've looked into these patches, and I don't think they worth >>> considering. Looks like a set of dirty hacks. >>> >> Do you have any specifics to what you don't like so we could look into >> changing them? Or are you just not interested in crossbuild support for >> nginx? > Crossbuilding was discussed more than once here. In short: while it's > something nice to have, it's not something required from our point > of view; therefore patches to add it should be clean, simple, and > extandable; no hacks will be considered. > I get that, and we do want to do that. We therefor also would appreciate some critisism, and also pointers to how we else could solve things in your build system. For us cross building is important, and we are willing to spend time both adding it for ourselves and getting it upstreamed. We did not expect that these patches would be accepted as they were, but were hoping for some response as to which parts needs to be fixed from you guys that knows the nginx build system a lot better than we do. From pdn at cryptopro.ru Tue Jul 29 15:11:26 2014 From: pdn at cryptopro.ru (Dmitrii Pichulin) Date: Tue, 29 Jul 2014 19:11:26 +0400 Subject: [PATCH 0 of 1 ] Questions about ENGINE_load_private_key In-Reply-To: <20140728024338.GE1849@mdounin.ru> References: <20140728024338.GE1849@mdounin.ru> Message-ID: 1) How can UI_METHOD provided by nginx look like? 2) Trying passwords from ssl_password_file one by one is not a good idea for private keys (too many pin attempts) From pdn at cryptopro.ru Tue Jul 29 15:11:27 2014 From: pdn at cryptopro.ru (Dmitrii Pichulin) Date: Tue, 29 Jul 2014 19:11:27 +0400 Subject: [PATCH 1 of 1] allow to use engine keyform for server private key In-Reply-To: References: Message-ID: # HG changeset patch # User Dmitrii Pichulin # Date 1406644835 -14400 # Tue Jul 29 18:40:35 2014 +0400 # Node ID b5f409eef2ed6832eead4c53855f91fb90ee099b # Parent d1bde5c3c5d21368de04a59506a06c1174353a19 allow to use engine keyform for server private key diff -r d1bde5c3c5d2 -r b5f409eef2ed src/event/ngx_event_openssl.c --- a/src/event/ngx_event_openssl.c Mon Jul 28 18:30:19 2014 +0400 +++ b/src/event/ngx_event_openssl.c Tue Jul 29 18:40:35 2014 +0400 @@ -265,8 +265,11 @@ ngx_ssl_certificate(ngx_conf_t *cf, ngx_ssl_t *ssl, ngx_str_t *cert, ngx_str_t *key, ngx_array_t *passwords) { + char *p, *last; BIO *bio; X509 *x509; + ENGINE *engine; + EVP_PKEY *private_key; u_long n; ngx_str_t *pwd; ngx_uint_t tries; @@ -352,6 +355,54 @@ BIO_free(bio); + if (ngx_strncmp(key->data, "engine:", sizeof("engine:") - 1) == 0) { + + p = (char *) key->data + sizeof("engine:") - 1; + last = ngx_strchr(p, ':'); + + if (last == NULL) { + ngx_conf_log_error(NGX_LOG_EMERG, cf, 0, "invalid syntax: %V", key); + return NGX_ERROR; + } + + p[last - p] = '\0'; + last++; + + engine = ENGINE_by_id(p); + + if (engine == NULL) { + ngx_ssl_error(NGX_LOG_EMERG, ssl->log, 0, + "ENGINE_by_id(\"%s\") failed", p); + return NGX_ERROR; + } + + private_key = ENGINE_load_private_key(engine, last, 0, 0); + + if (ENGINE_free(engine) == 0) { + ngx_ssl_error(NGX_LOG_EMERG, ssl->log, 0, "ENGINE_free() failed"); + if (private_key) + EVP_PKEY_free(private_key); + return NGX_ERROR; + } + + if (private_key == NULL) { + ngx_ssl_error(NGX_LOG_EMERG, ssl->log, 0, + "ENGINE_load_private_key(\"%s\") failed", last); + return NGX_ERROR; + } + + if (SSL_CTX_use_PrivateKey(ssl->ctx, private_key) == 0) { + ngx_ssl_error(NGX_LOG_EMERG, ssl->log, 0, + "SSL_CTX_use_PrivateKey_file(\"%s\") failed", last); + EVP_PKEY_free(private_key); + return NGX_ERROR; + } + + EVP_PKEY_free(private_key); + + return NGX_OK; + } + if (ngx_conf_full_name(cf->cycle, key, 1) != NGX_OK) { return NGX_ERROR; } From mdounin at mdounin.ru Tue Jul 29 15:31:59 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Tue, 29 Jul 2014 19:31:59 +0400 Subject: [PATCH 0 of 1 ] Questions about ENGINE_load_private_key In-Reply-To: References: <20140728024338.GE1849@mdounin.ru> Message-ID: <20140729153159.GO1849@mdounin.ru> Hello! On Tue, Jul 29, 2014 at 07:11:26PM +0400, Dmitrii Pichulin wrote: > 1) How can UI_METHOD provided by nginx look like? The code is in the same file you are modifying. > 2) Trying passwords from ssl_password_file one by one is not a good idea for private keys (too many pin attempts) The ssl_password_file is an interface we have. How it will be used (multiple passwords, only one password for each certificate, not used at all...) isn't really relevant. -- Maxim Dounin http://nginx.org/ From mdounin at mdounin.ru Tue Jul 29 15:40:24 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Tue, 29 Jul 2014 19:40:24 +0400 Subject: [PATCH 1 of 1] allow to use engine keyform for server private key In-Reply-To: References: Message-ID: <20140729154024.GP1849@mdounin.ru> Hello! On Tue, Jul 29, 2014 at 07:11:27PM +0400, Dmitrii Pichulin wrote: > # HG changeset patch > # User Dmitrii Pichulin > # Date 1406644835 -14400 > # Tue Jul 29 18:40:35 2014 +0400 > # Node ID b5f409eef2ed6832eead4c53855f91fb90ee099b > # Parent d1bde5c3c5d21368de04a59506a06c1174353a19 > allow to use engine keyform for server private key > > diff -r d1bde5c3c5d2 -r b5f409eef2ed src/event/ngx_event_openssl.c > --- a/src/event/ngx_event_openssl.c Mon Jul 28 18:30:19 2014 +0400 > +++ b/src/event/ngx_event_openssl.c Tue Jul 29 18:40:35 2014 +0400 > @@ -265,8 +265,11 @@ > ngx_ssl_certificate(ngx_conf_t *cf, ngx_ssl_t *ssl, ngx_str_t *cert, > ngx_str_t *key, ngx_array_t *passwords) > { > + char *p, *last; > BIO *bio; > X509 *x509; > + ENGINE *engine; > + EVP_PKEY *private_key; > u_long n; > ngx_str_t *pwd; > ngx_uint_t tries; > @@ -352,6 +355,54 @@ > > BIO_free(bio); > > + if (ngx_strncmp(key->data, "engine:", sizeof("engine:") - 1) == 0) { > + > + p = (char *) key->data + sizeof("engine:") - 1; > + last = ngx_strchr(p, ':'); > + > + if (last == NULL) { > + ngx_conf_log_error(NGX_LOG_EMERG, cf, 0, "invalid syntax: %V", key); > + return NGX_ERROR; > + } > + > + p[last - p] = '\0'; > + last++; > + > + engine = ENGINE_by_id(p); > + > + if (engine == NULL) { > + ngx_ssl_error(NGX_LOG_EMERG, ssl->log, 0, > + "ENGINE_by_id(\"%s\") failed", p); > + return NGX_ERROR; > + } > + > + private_key = ENGINE_load_private_key(engine, last, 0, 0); > + > + if (ENGINE_free(engine) == 0) { > + ngx_ssl_error(NGX_LOG_EMERG, ssl->log, 0, "ENGINE_free() failed"); > + if (private_key) > + EVP_PKEY_free(private_key); Style: curly brackets are always used with if's. > + return NGX_ERROR; > + } > + > + if (private_key == NULL) { > + ngx_ssl_error(NGX_LOG_EMERG, ssl->log, 0, > + "ENGINE_load_private_key(\"%s\") failed", last); > + return NGX_ERROR; > + } As previously suggested, it's bad idea to check/log errors after calling other functions which may modify error stack. > + > + if (SSL_CTX_use_PrivateKey(ssl->ctx, private_key) == 0) { > + ngx_ssl_error(NGX_LOG_EMERG, ssl->log, 0, > + "SSL_CTX_use_PrivateKey_file(\"%s\") failed", last); Wrong function name logged. > + EVP_PKEY_free(private_key); > + return NGX_ERROR; > + } > + > + EVP_PKEY_free(private_key); > + > + return NGX_OK; > + } > + > if (ngx_conf_full_name(cf->cycle, key, 1) != NGX_OK) { > return NGX_ERROR; > } > > _______________________________________________ > nginx-devel mailing list > nginx-devel at nginx.org > http://mailman.nginx.org/mailman/listinfo/nginx-devel -- Maxim Dounin http://nginx.org/ From fdasilvayy at gmail.com Tue Jul 29 20:20:33 2014 From: fdasilvayy at gmail.com (Filipe Da Silva) Date: Tue, 29 Jul 2014 22:20:33 +0200 Subject: [PATCH ]Mail: add guards around imap, pop3, smtp dedicated source code. Message-ID: Hi, It's a patch to cut some code that becomes useless, when you build your binary with a ./configure option like --without-mail_smtp_module and so on. I re-compile the code using all combination of the 3 --without-mail_*_module options to test it. It is just a first shot, in order to to reduce a bit the amount of auth-http code . Any comments are welcome. Regards, Filipe da Silva -------------- next part -------------- # HG changeset patch # User Filipe da Silva # Date 1389727400 -3600 # Tue Jan 14 20:23:20 2014 +0100 # Node ID 9d0d0b0d085c85590c4469a4feaa8a6eab9604dc # Parent 675bda8dcfdbf66e4a17017839f39ed6c8cbb9f5 Mail: add guards around imap, pop3, smtp dedicated source code. No functionnal change. Disable some code that becomes unnecessary when adding configure directives like --without-mail_(smtp|imap|pop3)_module . diff -r 675bda8dcfdb auto/modules --- a/auto/modules Thu Jun 19 13:55:59 2014 +0400 +++ b/auto/modules Thu Jun 19 22:55:55 2014 +0200 @@ -466,18 +466,21 @@ if [ $MAIL = YES ]; then fi if [ $MAIL_POP3 = YES ]; then + have=NGX_MAIL_POP3 . auto/have modules="$modules $MAIL_POP3_MODULE" MAIL_DEPS="$MAIL_DEPS $MAIL_POP3_DEPS" MAIL_SRCS="$MAIL_SRCS $MAIL_POP3_SRCS" fi if [ $MAIL_IMAP = YES ]; then + have=NGX_MAIL_IMAP . auto/have modules="$modules $MAIL_IMAP_MODULE" MAIL_DEPS="$MAIL_DEPS $MAIL_IMAP_DEPS" MAIL_SRCS="$MAIL_SRCS $MAIL_IMAP_SRCS" fi if [ $MAIL_SMTP = YES ]; then + have=NGX_MAIL_SMTP . auto/have modules="$modules $MAIL_SMTP_MODULE" MAIL_DEPS="$MAIL_DEPS $MAIL_SMTP_DEPS" MAIL_SRCS="$MAIL_SRCS $MAIL_SMTP_SRCS" diff -r 675bda8dcfdb src/mail/ngx_mail_auth_http_module.c --- a/src/mail/ngx_mail_auth_http_module.c Thu Jun 19 13:55:59 2014 +0400 +++ b/src/mail/ngx_mail_auth_http_module.c Thu Jun 19 22:55:55 2014 +0200 @@ -146,7 +146,9 @@ static ngx_str_t ngx_mail_auth_http_me ngx_string("none") }; +#if (NGX_MAIL_SMTP) static ngx_str_t ngx_mail_smtp_errcode = ngx_string("535 5.7.0"); +#endif void @@ -519,14 +521,18 @@ ngx_mail_auth_http_process_headers(ngx_m switch (s->protocol) { +#if (NGX_MAIL_POP3) case NGX_MAIL_POP3_PROTOCOL: size = sizeof("-ERR ") - 1 + len + sizeof(CRLF) - 1; break; +#endif +#if (NGX_MAIL_IMAP) case NGX_MAIL_IMAP_PROTOCOL: size = s->tag.len + sizeof("NO ") - 1 + len + sizeof(CRLF) - 1; break; +#endif default: /* NGX_MAIL_SMTP_PROTOCOL */ ctx->err = ctx->errmsg; @@ -545,14 +551,18 @@ ngx_mail_auth_http_process_headers(ngx_m switch (s->protocol) { +#if (NGX_MAIL_POP3) case NGX_MAIL_POP3_PROTOCOL: *p++ = '-'; *p++ = 'E'; *p++ = 'R'; *p++ = 'R'; *p++ = ' '; break; +#endif +#if (NGX_MAIL_IMAP) case NGX_MAIL_IMAP_PROTOCOL: p = ngx_cpymem(p, s->tag.data, s->tag.len); *p++ = 'N'; *p++ = 'O'; *p++ = ' '; break; +#endif default: /* NGX_MAIL_SMTP_PROTOCOL */ break; @@ -688,6 +698,7 @@ ngx_mail_auth_http_process_headers(ngx_m ngx_log_error(NGX_LOG_INFO, s->connection->log, 0, "client login failed: \"%V\"", &ctx->errmsg); +#if (NGX_MAIL_SMTP) if (s->protocol == NGX_MAIL_SMTP_PROTOCOL) { if (ctx->errcode.len == 0) { @@ -711,6 +722,7 @@ ngx_mail_auth_http_process_headers(ngx_m p = ngx_cpymem(p, ctx->errmsg.data, ctx->errmsg.len); *p++ = CR; *p = LF; } +#endif s->out = ctx->err; timer = ctx->sleep; @@ -1170,9 +1182,11 @@ ngx_mail_auth_http_create_request(ngx_ma + sizeof("Client-IP: ") - 1 + s->connection->addr_text.len + sizeof(CRLF) - 1 + sizeof("Client-Host: ") - 1 + s->host.len + sizeof(CRLF) - 1 +#if (NGX_MAIL_SMTP) + sizeof("Auth-SMTP-Helo: ") - 1 + s->smtp_helo.len + sizeof("Auth-SMTP-From: ") - 1 + s->smtp_from.len + sizeof("Auth-SMTP-To: ") - 1 + s->smtp_to.len +#endif + ahcf->header.len + sizeof(CRLF) - 1; @@ -1234,6 +1248,7 @@ ngx_mail_auth_http_create_request(ngx_ma *b->last++ = CR; *b->last++ = LF; } +#if (NGX_MAIL_SMTP) if (s->auth_method == NGX_MAIL_AUTH_NONE) { /* HELO, MAIL FROM, and RCPT TO can't contain CRLF, no need to escape */ @@ -1254,6 +1269,7 @@ ngx_mail_auth_http_create_request(ngx_ma *b->last++ = CR; *b->last++ = LF; } +#endif if (ahcf->header.len) { b->last = ngx_copy(b->last, ahcf->header.data, ahcf->header.len); diff -r 675bda8dcfdb src/mail/ngx_mail_proxy_module.c --- a/src/mail/ngx_mail_proxy_module.c Thu Jun 19 13:55:59 2014 +0400 +++ b/src/mail/ngx_mail_proxy_module.c Thu Jun 19 22:55:55 2014 +0200 @@ -22,9 +22,15 @@ typedef struct { static void ngx_mail_proxy_block_read(ngx_event_t *rev); +#if (NGX_MAIL_POP3) static void ngx_mail_proxy_pop3_handler(ngx_event_t *rev); +#endif +#if (NGX_MAIL_IMAP) static void ngx_mail_proxy_imap_handler(ngx_event_t *rev); +#endif +#if (NGX_MAIL_SMTP) static void ngx_mail_proxy_smtp_handler(ngx_event_t *rev); +#endif static void ngx_mail_proxy_dummy_handler(ngx_event_t *ev); static ngx_int_t ngx_mail_proxy_read_response(ngx_mail_session_t *s, ngx_uint_t state); @@ -105,7 +111,9 @@ ngx_module_t ngx_mail_proxy_module = { }; +#if (NGX_MAIL_SMTP) static u_char smtp_auth_ok[] = "235 2.0.0 OK" CRLF; +#endif void @@ -176,20 +184,31 @@ ngx_mail_proxy_init(ngx_mail_session_t * switch (s->protocol) { +#if (NGX_MAIL_POP3) case NGX_MAIL_POP3_PROTOCOL: p->upstream.connection->read->handler = ngx_mail_proxy_pop3_handler; s->mail_state = ngx_pop3_start; break; +#endif +#if (NGX_MAIL_IMAP) case NGX_MAIL_IMAP_PROTOCOL: p->upstream.connection->read->handler = ngx_mail_proxy_imap_handler; s->mail_state = ngx_imap_start; break; +#endif - default: /* NGX_MAIL_SMTP_PROTOCOL */ +#if (NGX_MAIL_SMTP) + case NGX_MAIL_SMTP_PROTOCOL: p->upstream.connection->read->handler = ngx_mail_proxy_smtp_handler; s->mail_state = ngx_smtp_start; break; +#endif + + default: + p->upstream.connection->read->handler = ngx_mail_proxy_dummy_handler; + s->mail_state = 0; + break; } } @@ -211,6 +230,7 @@ ngx_mail_proxy_block_read(ngx_event_t *r } +#if (NGX_MAIL_POP3) static void ngx_mail_proxy_pop3_handler(ngx_event_t *rev) { @@ -322,8 +342,10 @@ ngx_mail_proxy_pop3_handler(ngx_event_t s->proxy->buffer->pos = s->proxy->buffer->start; s->proxy->buffer->last = s->proxy->buffer->start; } +#endif +#if (NGX_MAIL_IMAP) static void ngx_mail_proxy_imap_handler(ngx_event_t *rev) { @@ -456,8 +478,10 @@ ngx_mail_proxy_imap_handler(ngx_event_t s->proxy->buffer->pos = s->proxy->buffer->start; s->proxy->buffer->last = s->proxy->buffer->start; } +#endif +#if (NGX_MAIL_SMTP) static void ngx_mail_proxy_smtp_handler(ngx_event_t *rev) { @@ -708,6 +732,7 @@ ngx_mail_proxy_smtp_handler(ngx_event_t s->proxy->buffer->pos = s->proxy->buffer->start; s->proxy->buffer->last = s->proxy->buffer->start; } +#endif static void @@ -778,6 +803,7 @@ ngx_mail_proxy_read_response(ngx_mail_se } break; +#if (NGX_MAIL_IMAP) case NGX_MAIL_IMAP_PROTOCOL: switch (state) { @@ -805,6 +831,7 @@ ngx_mail_proxy_read_response(ngx_mail_se } break; +#endif default: /* NGX_MAIL_SMTP_PROTOCOL */ From mdounin at mdounin.ru Tue Jul 29 21:53:20 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Wed, 30 Jul 2014 01:53:20 +0400 Subject: [PATCH ]Mail: add guards around imap, pop3, smtp dedicated source code. In-Reply-To: References: Message-ID: <20140729215320.GQ1849@mdounin.ru> Hello! On Tue, Jul 29, 2014 at 10:20:33PM +0200, Filipe Da Silva wrote: > Hi, > > It's a patch to cut some code that becomes useless, when you build > your binary with a ./configure option like --without-mail_smtp_module > and so on. > > I re-compile the code using all combination of the 3 > --without-mail_*_module options to test it. > > It is just a first shot, in order to to reduce a bit the amount of > auth-http code . > Any comments are welcome. I don't think that it's something needed. -- Maxim Dounin http://nginx.org/ From mdounin at mdounin.ru Tue Jul 29 22:07:05 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Tue, 29 Jul 2014 22:07:05 +0000 Subject: [nginx] SSL: fix build with OPENSSL_NO_ENGINE and/or OPENSSL_NO_... Message-ID: details: http://hg.nginx.org/nginx/rev/4d092aa2f463 branches: changeset: 5777:4d092aa2f463 user: Piotr Sikora date: Mon Jul 28 12:27:57 2014 -0700 description: SSL: fix build with OPENSSL_NO_ENGINE and/or OPENSSL_NO_OCSP. This is really just a prerequisite for building against BoringSSL, which doesn't provide either of those features. Signed-off-by: Piotr Sikora diffstat: src/event/ngx_event_openssl.c | 10 ++++++++++ src/event/ngx_event_openssl.h | 4 ++++ src/event/ngx_event_openssl_stapling.c | 2 +- 3 files changed, 15 insertions(+), 1 deletions(-) diffs (62 lines): diff --git a/src/event/ngx_event_openssl.c b/src/event/ngx_event_openssl.c --- a/src/event/ngx_event_openssl.c +++ b/src/event/ngx_event_openssl.c @@ -3279,6 +3279,8 @@ ngx_openssl_create_conf(ngx_cycle_t *cyc static char * ngx_openssl_engine(ngx_conf_t *cf, ngx_command_t *cmd, void *conf) { +#ifndef OPENSSL_NO_ENGINE + ngx_openssl_conf_t *oscf = conf; ENGINE *engine; @@ -3313,6 +3315,12 @@ ngx_openssl_engine(ngx_conf_t *cf, ngx_c ENGINE_free(engine); return NGX_CONF_OK; + +#else + + return "is not supported"; + +#endif } @@ -3320,5 +3328,7 @@ static void ngx_openssl_exit(ngx_cycle_t *cycle) { EVP_cleanup(); +#ifndef OPENSSL_NO_ENGINE ENGINE_cleanup(); +#endif } diff --git a/src/event/ngx_event_openssl.h b/src/event/ngx_event_openssl.h --- a/src/event/ngx_event_openssl.h +++ b/src/event/ngx_event_openssl.h @@ -18,9 +18,13 @@ #include #include #include +#ifndef OPENSSL_NO_ENGINE #include +#endif #include +#ifndef OPENSSL_NO_OCSP #include +#endif #include #include #include diff --git a/src/event/ngx_event_openssl_stapling.c b/src/event/ngx_event_openssl_stapling.c --- a/src/event/ngx_event_openssl_stapling.c +++ b/src/event/ngx_event_openssl_stapling.c @@ -11,7 +11,7 @@ #include -#ifdef SSL_CTRL_SET_TLSEXT_STATUS_REQ_CB +#if (!defined OPENSSL_NO_OCSP && defined SSL_CTRL_SET_TLSEXT_STATUS_REQ_CB) typedef struct { From mdounin at mdounin.ru Tue Jul 29 22:08:01 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Wed, 30 Jul 2014 02:08:01 +0400 Subject: [PATCH 1 of 2] SSL: fix build with OPENSSL_NO_ENGINE and/or OPENSSL_NO_OCSP In-Reply-To: References: Message-ID: <20140729220801.GR1849@mdounin.ru> Hello! On Mon, Jul 28, 2014 at 04:03:35PM -0700, Piotr Sikora wrote: > # HG changeset patch > # User Piotr Sikora > # Date 1406575677 25200 > # Mon Jul 28 12:27:57 2014 -0700 > # Node ID bb74dfefeec04aae5a3a86ace2df45d03f691ded > # Parent d1bde5c3c5d21368de04a59506a06c1174353a19 > SSL: fix build with OPENSSL_NO_ENGINE and/or OPENSSL_NO_OCSP. > > This is really just a prerequisite for building against BoringSSL, > which doesn't provide either of those features. This one committed (with minor changes), thanks. -- Maxim Dounin http://nginx.org/ From mdounin at mdounin.ru Tue Jul 29 22:15:34 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Wed, 30 Jul 2014 02:15:34 +0400 Subject: [PATCH 2 of 2] SSL: let it build against BoringSSL In-Reply-To: References: Message-ID: <20140729221534.GS1849@mdounin.ru> Hello! On Mon, Jul 28, 2014 at 04:03:36PM -0700, Piotr Sikora wrote: > # HG changeset patch > # User Piotr Sikora > # Date 1406575677 25200 > # Mon Jul 28 12:27:57 2014 -0700 > # Node ID e3086fd5e59335f4f3f165ee74c094a7aca2aeb3 > # Parent bb74dfefeec04aae5a3a86ace2df45d03f691ded > SSL: let it build against BoringSSL. > > This change adds support for using BoringSSL as a drop-in replacement > for OpenSSL without adding support for any of the BoringSSL-specific > features. > > The #ifndefs around SSL_CTX_set_tmp_rsa_callback() aren't strictly > necessary, since that function still exists in BoringSSL as a no-op, > but they clearly mark the unsupported feature. I think that it's better idea to preserve the common code rather than to add unneded #ifndef's. > > Signed-off-by: Piotr Sikora > > diff -r bb74dfefeec0 -r e3086fd5e593 src/event/ngx_event_openssl.c > --- a/src/event/ngx_event_openssl.c Mon Jul 28 12:27:57 2014 -0700 > +++ b/src/event/ngx_event_openssl.c Mon Jul 28 12:27:57 2014 -0700 > @@ -106,7 +106,9 @@ int ngx_ssl_stapling_index; > ngx_int_t > ngx_ssl_init(ngx_log_t *log) > { > +#ifndef OPENSSL_IS_BORINGSSL > OPENSSL_config(NULL); > +#endif > > SSL_library_init(); > SSL_load_error_strings(); > @@ -217,7 +219,10 @@ ngx_ssl_create(ngx_ssl_t *ssl, ngx_uint_ > SSL_CTX_set_options(ssl->ctx, SSL_OP_MSIE_SSLV2_RSA_PADDING); > #endif > > +#ifdef SSL_OP_SSLEAY_080_CLIENT_DH_BUG > SSL_CTX_set_options(ssl->ctx, SSL_OP_SSLEAY_080_CLIENT_DH_BUG); > +#endif > + > SSL_CTX_set_options(ssl->ctx, SSL_OP_TLS_D5_BUG); > SSL_CTX_set_options(ssl->ctx, SSL_OP_TLS_BLOCK_PADDING_BUG); > No objections to these two changes. > @@ -382,8 +387,13 @@ ngx_ssl_certificate(ngx_conf_t *cf, ngx_ > if (--tries) { > n = ERR_peek_error(); > > +#ifdef OPENSSL_IS_BORINGSSL > + if (ERR_GET_LIB(n) == ERR_LIB_CIPHER > + && ERR_GET_REASON(n) == CIPHER_R_BAD_DECRYPT) > +#else > if (ERR_GET_LIB(n) == ERR_LIB_EVP > && ERR_GET_REASON(n) == EVP_R_BAD_DECRYPT) > +#endif > { > ERR_clear_error(); > SSL_CTX_set_default_passwd_cb_userdata(ssl->ctx, ++pwd); This one scares me though. In particular, because BoringSSL managed to move various EVP_* functions to CIPHER library, and this looks strange. I also wonder how many similar changes are unnoticed because they don't break build... -- Maxim Dounin http://nginx.org/ From piotr at cloudflare.com Tue Jul 29 23:19:39 2014 From: piotr at cloudflare.com (Piotr Sikora) Date: Tue, 29 Jul 2014 16:19:39 -0700 Subject: [PATCH 2 of 2] SSL: let it build against BoringSSL In-Reply-To: <20140729221534.GS1849@mdounin.ru> References: <20140729221534.GS1849@mdounin.ru> Message-ID: Hey Maxim, > I think that it's better idea to preserve the common code rather > than to add unneded #ifndef's. Well, my argument for #ifndefs is that both BoringSSL and LibreSSL (for which I have patch in my queue) removed support for export cipher suites, so I don't see a point in calling a function that we know doesn't do anything and that might be removed sooner rather than later. BoringSSL made SSL_CTX_set_tmp_rsa_callback() a no-op (at least for now) and just ignores the callback. LibreSSL sets and calls the callback (because of the SSL_OP_EPHEMERAL_RSA, which is still supported, even though it violates TLS standard), but I expect it might get removed soon... and then, depending on their implementation, it might put an error on the error queue, which will just mess things for us. I'm going to send the LibreSSL patch in a moment and let's see if the #ifndefs are still bothering you, but I feel rather strongly about keeping them. > This one scares me though. In particular, because BoringSSL > managed to move various EVP_* functions to CIPHER library, and > this looks strange. I also wonder how many similar changes are > unnoticed because they don't break build... Rest seems to work fine :) I'm rather committed to switching to BoringSSL myself in the near future, so this is more than just "it compiles" change. Best regards, Piotr Sikora From piotr at cloudflare.com Tue Jul 29 23:20:20 2014 From: piotr at cloudflare.com (Piotr Sikora) Date: Tue, 29 Jul 2014 16:20:20 -0700 Subject: [PATCH] SSL: let it build against LibreSSL Message-ID: # HG changeset patch # User Piotr Sikora # Date 1406575677 25200 # Mon Jul 28 12:27:57 2014 -0700 # Node ID c1abbfee85b3185c28a279c7935d0bb871933ed8 # Parent e3086fd5e59335f4f3f165ee74c094a7aca2aeb3 SSL: let it build against LibreSSL. LibreSSL developers decided that LibreSSL is OpenSSL-2.0.0, so tests for OpenSSL-1.0.2+ are now passing, even though the library doesn't provide functions that are expected from that version of OpenSSL. The #ifndefs around SSL_CTX_set_tmp_rsa_callback() aren't strictly necessary, but support for the export cipher suites has been removed from LibreSSL, so they clearly mark the unsupported feature. Signed-off-by: Piotr Sikora diff -r e3086fd5e593 -r c1abbfee85b3 src/event/ngx_event_openssl.c --- a/src/event/ngx_event_openssl.c Mon Jul 28 12:27:57 2014 -0700 +++ b/src/event/ngx_event_openssl.c Mon Jul 28 12:27:57 2014 -0700 @@ -50,7 +50,7 @@ static int ngx_ssl_session_ticket_key_ca HMAC_CTX *hctx, int enc); #endif -#if OPENSSL_VERSION_NUMBER < 0x10002002L +#if (OPENSSL_VERSION_NUMBER < 0x10002002L || defined LIBRESSL_VERSION_NUMBER) static ngx_int_t ngx_ssl_check_name(ngx_str_t *name, ASN1_STRING *str); #endif @@ -656,7 +656,7 @@ ngx_ssl_info_callback(const ngx_ssl_conn } -#ifndef OPENSSL_IS_BORINGSSL +#if (!defined OPENSSL_IS_BORINGSSL && !defined LIBRESSL_VERSION_NUMBER) RSA * ngx_ssl_rsa512_key_callback(ngx_ssl_conn_t *ssl_conn, int is_export, @@ -2747,7 +2747,7 @@ ngx_ssl_check_host(ngx_connection_t *c, return NGX_ERROR; } -#if OPENSSL_VERSION_NUMBER >= 0x10002002L +#if (OPENSSL_VERSION_NUMBER >= 0x10002002L && !defined LIBRESSL_VERSION_NUMBER) /* X509_check_host() is only available in OpenSSL 1.0.2+ */ @@ -2864,7 +2864,7 @@ found: } -#if OPENSSL_VERSION_NUMBER < 0x10002002L +#if (OPENSSL_VERSION_NUMBER < 0x10002002L || defined LIBRESSL_VERSION_NUMBER) static ngx_int_t ngx_ssl_check_name(ngx_str_t *name, ASN1_STRING *pattern) diff -r e3086fd5e593 -r c1abbfee85b3 src/event/ngx_event_openssl.h --- a/src/event/ngx_event_openssl.h Mon Jul 28 12:27:57 2014 -0700 +++ b/src/event/ngx_event_openssl.h Mon Jul 28 12:27:57 2014 -0700 @@ -133,7 +133,7 @@ ngx_int_t ngx_ssl_stapling(ngx_conf_t *c ngx_str_t *file, ngx_str_t *responder, ngx_uint_t verify); ngx_int_t ngx_ssl_stapling_resolver(ngx_conf_t *cf, ngx_ssl_t *ssl, ngx_resolver_t *resolver, ngx_msec_t resolver_timeout); -#ifndef OPENSSL_IS_BORINGSSL +#if (!defined OPENSSL_IS_BORINGSSL && !defined LIBRESSL_VERSION_NUMBER) RSA *ngx_ssl_rsa512_key_callback(ngx_ssl_conn_t *ssl_conn, int is_export, int key_length); #endif diff -r e3086fd5e593 -r c1abbfee85b3 src/http/modules/ngx_http_ssl_module.c --- a/src/http/modules/ngx_http_ssl_module.c Mon Jul 28 12:27:57 2014 -0700 +++ b/src/http/modules/ngx_http_ssl_module.c Mon Jul 28 12:27:57 2014 -0700 @@ -715,7 +715,7 @@ ngx_http_ssl_merge_srv_conf(ngx_conf_t * SSL_CTX_set_options(conf->ssl.ctx, SSL_OP_CIPHER_SERVER_PREFERENCE); } -#ifndef OPENSSL_IS_BORINGSSL +#if (!defined OPENSSL_IS_BORINGSSL && !defined LIBRESSL_VERSION_NUMBER) /* a temporary 512-bit RSA key is required for export versions of MSIE */ SSL_CTX_set_tmp_rsa_callback(conf->ssl.ctx, ngx_ssl_rsa512_key_callback); #endif diff -r e3086fd5e593 -r c1abbfee85b3 src/mail/ngx_mail_ssl_module.c --- a/src/mail/ngx_mail_ssl_module.c Mon Jul 28 12:27:57 2014 -0700 +++ b/src/mail/ngx_mail_ssl_module.c Mon Jul 28 12:27:57 2014 -0700 @@ -334,7 +334,7 @@ ngx_mail_ssl_merge_conf(ngx_conf_t *cf, SSL_CTX_set_options(conf->ssl.ctx, SSL_OP_CIPHER_SERVER_PREFERENCE); } -#ifndef OPENSSL_IS_BORINGSSL +#if (!defined OPENSSL_IS_BORINGSSL && !defined LIBRESSL_VERSION_NUMBER) SSL_CTX_set_tmp_rsa_callback(conf->ssl.ctx, ngx_ssl_rsa512_key_callback); #endif From piotr at cloudflare.com Tue Jul 29 23:22:57 2014 From: piotr at cloudflare.com (Piotr Sikora) Date: Tue, 29 Jul 2014 16:22:57 -0700 Subject: [PATCH] SSL: let it build against LibreSSL In-Reply-To: References: Message-ID: Hey, this change is based on BoringSSL change [1], but I can rebase it if we decide that the #ifndefs are not wanted after all. [1] http://mailman.nginx.org/pipermail/nginx-devel/2014-July/005656.html Best regards, Piotr Sikora From piotr at cloudflare.com Tue Jul 29 23:39:06 2014 From: piotr at cloudflare.com (Piotr Sikora) Date: Tue, 29 Jul 2014 16:39:06 -0700 Subject: [PATCH 2 of 2] SSL: let it build against BoringSSL In-Reply-To: References: <20140729221534.GS1849@mdounin.ru> Message-ID: Hey Maxim, > I'm rather committed to switching to BoringSSL myself in the near > future, so this is more than just "it compiles" change. btw: BoringSSL is using CMake so "--with-openssl" obviously doesn't work, but if you're interested in adding "--with-boringssl" then I'd be more than happy to provide it. Best regards, Piotr Sikora From mdounin at mdounin.ru Wed Jul 30 00:55:53 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Wed, 30 Jul 2014 04:55:53 +0400 Subject: [PATCH] SSL: let it build against LibreSSL In-Reply-To: References: Message-ID: <20140730005553.GV1849@mdounin.ru> Hello! On Tue, Jul 29, 2014 at 04:20:20PM -0700, Piotr Sikora wrote: > # HG changeset patch > # User Piotr Sikora > # Date 1406575677 25200 > # Mon Jul 28 12:27:57 2014 -0700 > # Node ID c1abbfee85b3185c28a279c7935d0bb871933ed8 > # Parent e3086fd5e59335f4f3f165ee74c094a7aca2aeb3 > SSL: let it build against LibreSSL. > > LibreSSL developers decided that LibreSSL is OpenSSL-2.0.0, so tests > for OpenSSL-1.0.2+ are now passing, even though the library doesn't > provide functions that are expected from that version of OpenSSL. As previously suggested, this doesn't looks like a right way to go. If LibreSSL folks continue to insist this is OpenSSL-2.0.0, then we'll probably have redefine OPENSSL_VERSION_NUMBER ourselves. > The #ifndefs around SSL_CTX_set_tmp_rsa_callback() aren't strictly > necessary, but support for the export cipher suites has been removed > from LibreSSL, so they clearly mark the unsupported feature. Same as for BoringSSL patch - I don't think we should add #if's here. -- Maxim Dounin http://nginx.org/ From piotr at cloudflare.com Wed Jul 30 02:15:06 2014 From: piotr at cloudflare.com (Piotr Sikora) Date: Tue, 29 Jul 2014 19:15:06 -0700 Subject: [PATCH] SSL: let it build against LibreSSL In-Reply-To: <20140730005553.GV1849@mdounin.ru> References: <20140730005553.GV1849@mdounin.ru> Message-ID: Hey Maxim, > As previously suggested, this doesn't looks like a right way to > go. If LibreSSL folks continue to insist this is OpenSSL-2.0.0, > then we'll probably have redefine OPENSSL_VERSION_NUMBER > ourselves. Unfortunately, it looks like they've made their mind :( The discussion on it stopped, they've made 4 releases with that version already and OpenBSD is in the release mode right now, so I don't expect that they're going to change it. Redefining OPENSSL_VERSION_NUMBER doesn't seem like the prettiest solution, though. > Same as for BoringSSL patch - I don't think we should add #if's > here. See my reply in the BoringSSL thread... I don't think that it makes sense to keep setting the callback if we know that it doesn't do anything. Effectively, we're masking the issue and pretending that the feature works with BoringSSL and LibreSSL, simply because it compiles (i.e. the thing you were worried about). Best regards, Piotr Sikora From mdounin at mdounin.ru Wed Jul 30 04:15:46 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Wed, 30 Jul 2014 08:15:46 +0400 Subject: [PATCH 2 of 2] SSL: let it build against BoringSSL In-Reply-To: References: <20140729221534.GS1849@mdounin.ru> Message-ID: <20140730041545.GX1849@mdounin.ru> Hello! On Tue, Jul 29, 2014 at 04:19:39PM -0700, Piotr Sikora wrote: > Hey Maxim, > > > I think that it's better idea to preserve the common code rather > > than to add unneded #ifndef's. > > Well, my argument for #ifndefs is that both BoringSSL and LibreSSL > (for which I have patch in my queue) removed support for export cipher > suites, so I don't see a point in calling a function that we know > doesn't do anything and that might be removed sooner rather than > later. The problem is lots of added lines required to avoid calling it. > BoringSSL made SSL_CTX_set_tmp_rsa_callback() a no-op (at least for > now) and just ignores the callback. > > LibreSSL sets and calls the callback (because of the > SSL_OP_EPHEMERAL_RSA, which is still supported, even though it > violates TLS standard), but I expect it might get removed soon... and > then, depending on their implementation, it might put an error on the > error queue, which will just mess things for us. > > I'm going to send the LibreSSL patch in a moment and let's see if the > #ifndefs are still bothering you, but I feel rather strongly about > keeping them. If expected further API changes in BoringSSL/LibreSSL is a concern, we may want to wait longer before the API settles a bit. I believe I've already suggested this during previous discussion about LibreSSL. > > This one scares me though. In particular, because BoringSSL > > managed to move various EVP_* functions to CIPHER library, and > > this looks strange. I also wonder how many similar changes are > > unnoticed because they don't break build... > > Rest seems to work fine :) > > I'm rather committed to switching to BoringSSL myself in the near > future, so this is more than just "it compiles" change. Even if it works, it would be non-trivial to support code with such #if's. We may want to add some additional level of abstraction here. Not sure if it worth the effort in this particular case though. -- Maxim Dounin http://nginx.org/ From mdounin at mdounin.ru Wed Jul 30 04:21:49 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Wed, 30 Jul 2014 08:21:49 +0400 Subject: [PATCH 2 of 2] SSL: let it build against BoringSSL In-Reply-To: References: <20140729221534.GS1849@mdounin.ru> Message-ID: <20140730042149.GY1849@mdounin.ru> Hello! On Tue, Jul 29, 2014 at 04:39:06PM -0700, Piotr Sikora wrote: > Hey Maxim, > > > I'm rather committed to switching to BoringSSL myself in the near > > future, so this is more than just "it compiles" change. > > btw: BoringSSL is using CMake so "--with-openssl" obviously doesn't > work, but if you're interested in adding "--with-boringssl" then I'd > be more than happy to provide it. No, I'm not. I would rather object - the "--with-..." options are to simplify builds, while the library seems to require careful handling and manual help during building (read: it fails to compile on FreeBSD). -- Maxim Dounin http://nginx.org/ From piotr at cloudflare.com Wed Jul 30 04:57:27 2014 From: piotr at cloudflare.com (Piotr Sikora) Date: Tue, 29 Jul 2014 21:57:27 -0700 Subject: [PATCH 2 of 2] SSL: let it build against BoringSSL In-Reply-To: <20140730041545.GX1849@mdounin.ru> References: <20140729221534.GS1849@mdounin.ru> <20140730041545.GX1849@mdounin.ru> Message-ID: Hey Maxim, > The problem is lots of added lines required to avoid calling it. And why exactly is that a problem? > If expected further API changes in BoringSSL/LibreSSL is a > concern, we may want to wait longer before the API settles a bit. > I believe I've already suggested this during previous discussion > about LibreSSL. LibreSSL already shipped their initial release and BoringSSL won't have stable API nor formal releases, so there is no point in delaying that in my opinion. > Even if it works, it would be non-trivial to support code with > such #if's. We may want to add some additional level of > abstraction here. Not sure if it worth the effort in this > particular case though. Yeah, this particular case looks a bit odd, but it's the best I can think of without introducing too much complexity. Best regards, Piotr Sikora From piotr at cloudflare.com Wed Jul 30 05:32:29 2014 From: piotr at cloudflare.com (Piotr Sikora) Date: Tue, 29 Jul 2014 22:32:29 -0700 Subject: [PATCH 2 of 2] SSL: let it build against BoringSSL In-Reply-To: <20140730041545.GX1849@mdounin.ru> References: <20140729221534.GS1849@mdounin.ru> <20140730041545.GX1849@mdounin.ru> Message-ID: Hey Maxim, > The problem is lots of added lines required to avoid calling it. Alternatively, would you be OK with a change that adds #ifndefs around SSL_CTX_set_tmp_rsa_callback() in ngx_http_ssl_module and ngx_mail_ssl_module, but keeps ngx_ssl_rsa512_key_callback without guards? It's exported, so compilers won't complain about it being unused. Best regards, Piotr Sikora From flygoast at 126.com Wed Jul 30 06:58:04 2014 From: flygoast at 126.com (flygoast) Date: Wed, 30 Jul 2014 14:58:04 +0800 (CST) Subject: [PATCH]Dav: ngx_http_map_uri_to_path() errors were not checked. Message-ID: <336f6c81.f17.147860fbff6.Coremail.flygoast@126.com> # HG changeset patch # User FengGu # Date 1406702708 -28800 # Wed Jul 30 14:45:08 2014 +0800 # Node ID 9a9f7c2783b1afe0e5d374450daa5df6acbfd332 # Parent 4d092aa2f4637ce50284d2accd99a8e91aae2b4c Dav: ngx_http_map_uri_to_path() errors were not checked. Once error occured, it could lead to use uninitialized variables to log, even more segmentation fault. diff -r 4d092aa2f463 -r 9a9f7c2783b1 src/http/modules/ngx_http_dav_module.c --- a/src/http/modules/ngx_http_dav_module.cMon Jul 28 12:27:57 2014 -0700 +++ b/src/http/modules/ngx_http_dav_module.cWed Jul 30 14:45:08 2014 +0800 @@ -212,7 +212,10 @@ return; } - ngx_http_map_uri_to_path(r, &path, &root, 0); + if (ngx_http_map_uri_to_path(r, &path, &root, 0) == NULL) { + ngx_http_finalize_request(r, NGX_HTTP_INTERNAL_SERVER_ERROR); + return; + } path.len--; @@ -320,7 +323,9 @@ ok: - ngx_http_map_uri_to_path(r, &path, &root, 0); + if (ngx_http_map_uri_to_path(r, &path, &root, 0) == NULL) { + return NGX_HTTP_INTERNAL_SERVER_ERROR; + } ngx_log_debug1(NGX_LOG_DEBUG_HTTP, r->connection->log, 0, "http delete filename: \"%s\"", path.data); @@ -488,6 +493,9 @@ } p = ngx_http_map_uri_to_path(r, &path, &root, 0); + if (p == NULL) { + return NGX_HTTP_INTERNAL_SERVER_ERROR; + } *(p - 1) = '\0'; r->uri.len--; @@ -666,7 +674,9 @@ overwrite_done: - ngx_http_map_uri_to_path(r, &path, &root, 0); + if (ngx_http_map_uri_to_path(r, &path, &root, 0) == NULL) { + return NGX_HTTP_INTERNAL_SERVER_ERROR; + } ngx_log_debug1(NGX_LOG_DEBUG_HTTP, r->connection->log, 0, "http copy from: \"%s\"", path.data); @@ -674,7 +684,9 @@ uri = r->uri; r->uri = duri; - ngx_http_map_uri_to_path(r, ©.path, &root, 0); + if (ngx_http_map_uri_to_path(r, ©.path, &root, 0) == NULL) { + return NGX_HTTP_INTERNAL_SERVER_ERROR; + } r->uri = uri; -------------- next part -------------- An HTML attachment was scrubbed... URL: From gmstima at gmail.com Wed Jul 30 09:32:05 2014 From: gmstima at gmail.com (Roman Savchenko) Date: Wed, 30 Jul 2014 12:32:05 +0300 Subject: Upstream module question. Message-ID: Hi all, I'm starting develop my own upstream module. And I have question: How can i disable server directive in upstream block. http { upstream my_upstream { my_endpoint addr port; } server { location / { proxy_pass http://my_upstream; } } } As can you see I didn't use server but i used my_endpoint. After some googling i found ZerroMQ module: static char * ngx_http_upstream_zeromq_endpoint(ngx_conf_t *cf, ngx_command_t *cmd, void *conf) { ngx_http_upstream_srv_conf_t *uscf; uscf = ngx_http_conf_get_module_srv_conf(cf, ngx_http_upstream_module); .... THEY USE THIS uscf->servers = (ngx_array_t *) -1; ... return NGX_CONF_OK; } But it doesn't work now (nginx closed without any errors/warnings). Thanks for any help. Regards, R. Savchenko. -------------- next part -------------- An HTML attachment was scrubbed... URL: From piotr at cloudflare.com Wed Jul 30 11:07:07 2014 From: piotr at cloudflare.com (Piotr Sikora) Date: Wed, 30 Jul 2014 04:07:07 -0700 Subject: Upstream module question. In-Reply-To: References: Message-ID: Hey, > THEY USE THIS > > uscf->servers = (ngx_array_t *) -1; Because it's unused. > But it doesn't work now (nginx closed without any errors/warnings). Thanks > for any help. It doesn't work because API of both: nginx and ZeroMQ changed since the ngx_zeromq module (PoC) was written and the code wasn't updated accordingly, not because of the "uscf->servers" value. I actually have the updated (working) version in my local tree, but I didn't have time to clean it up properly. Best regards, Piotr Sikora From gmstima at gmail.com Wed Jul 30 11:38:59 2014 From: gmstima at gmail.com (Roman Savchenko) Date: Wed, 30 Jul 2014 14:38:59 +0300 Subject: Upstream module question. In-Reply-To: References: Message-ID: Hi Piotr, It will be nice if you share updated version (no matter that is not clean). I'll use it as example, because many of documentation/samples/public modules out- of-date. Regards, Roman. 2014-07-30 14:07 GMT+03:00 Piotr Sikora : > Hey, > > > THEY USE THIS > > > > uscf->servers = (ngx_array_t *) -1; > > Because it's unused. > > > But it doesn't work now (nginx closed without any errors/warnings). > Thanks > > for any help. > > It doesn't work because API of both: nginx and ZeroMQ changed since > the ngx_zeromq module (PoC) was written and the code wasn't updated > accordingly, not because of the "uscf->servers" value. > > I actually have the updated (working) version in my local tree, but I > didn't have time to clean it up properly. > > Best regards, > Piotr Sikora > > _______________________________________________ > nginx-devel mailing list > nginx-devel at nginx.org > http://mailman.nginx.org/mailman/listinfo/nginx-devel > -------------- next part -------------- An HTML attachment was scrubbed... URL: From piotr at cloudflare.com Wed Jul 30 11:42:08 2014 From: piotr at cloudflare.com (Piotr Sikora) Date: Wed, 30 Jul 2014 04:42:08 -0700 Subject: [PATCH 0 of 2] SSL: let it build against BorginSSL and LibreSSL (take #2) Message-ID: Hey Maxim, just to speed this up, this is the version with #ifndefs added only around SSL_CTX_set_tmp_rsa_callback(). Feel free to commit whichever version you prefer. Best regards, Piotr Sikora From piotr at cloudflare.com Wed Jul 30 11:42:10 2014 From: piotr at cloudflare.com (Piotr Sikora) Date: Wed, 30 Jul 2014 04:42:10 -0700 Subject: [PATCH 2 of 2] SSL: let it build against LibreSSL In-Reply-To: References: Message-ID: <2c5ac97b08f64bd50c5e.1406720530@Piotrs-MacBook-Pro.local> # HG changeset patch # User Piotr Sikora # Date 1406719936 25200 # Wed Jul 30 04:32:16 2014 -0700 # Node ID 2c5ac97b08f64bd50c5ec52e2f2e3ca8d787fc10 # Parent 3a647f0d5104612c7fa5c9cc1245057a4c0a3dc2 SSL: let it build against LibreSSL. LibreSSL developers decided that LibreSSL is OpenSSL-2.0.0, so tests for OpenSSL-1.0.2+ are now passing, even though the library doesn't provide functions that are expected from that version of OpenSSL. The #ifndefs around SSL_CTX_set_tmp_rsa_callback() aren't strictly necessary, but support for the export cipher suites has been removed from LibreSSL, so they clearly mark the unsupported feature. Signed-off-by: Piotr Sikora diff -r 3a647f0d5104 -r 2c5ac97b08f6 src/event/ngx_event_openssl.c --- a/src/event/ngx_event_openssl.c Wed Jul 30 04:32:15 2014 -0700 +++ b/src/event/ngx_event_openssl.c Wed Jul 30 04:32:16 2014 -0700 @@ -50,7 +50,7 @@ static int ngx_ssl_session_ticket_key_ca HMAC_CTX *hctx, int enc); #endif -#if OPENSSL_VERSION_NUMBER < 0x10002002L +#if (OPENSSL_VERSION_NUMBER < 0x10002002L || defined LIBRESSL_VERSION_NUMBER) static ngx_int_t ngx_ssl_check_name(ngx_str_t *name, ASN1_STRING *str); #endif @@ -2743,7 +2743,7 @@ ngx_ssl_check_host(ngx_connection_t *c, return NGX_ERROR; } -#if OPENSSL_VERSION_NUMBER >= 0x10002002L +#if (OPENSSL_VERSION_NUMBER >= 0x10002002L && !defined LIBRESSL_VERSION_NUMBER) /* X509_check_host() is only available in OpenSSL 1.0.2+ */ @@ -2860,7 +2860,7 @@ found: } -#if OPENSSL_VERSION_NUMBER < 0x10002002L +#if (OPENSSL_VERSION_NUMBER < 0x10002002L || defined LIBRESSL_VERSION_NUMBER) static ngx_int_t ngx_ssl_check_name(ngx_str_t *name, ASN1_STRING *pattern) diff -r 3a647f0d5104 -r 2c5ac97b08f6 src/http/modules/ngx_http_ssl_module.c --- a/src/http/modules/ngx_http_ssl_module.c Wed Jul 30 04:32:15 2014 -0700 +++ b/src/http/modules/ngx_http_ssl_module.c Wed Jul 30 04:32:16 2014 -0700 @@ -715,7 +715,7 @@ ngx_http_ssl_merge_srv_conf(ngx_conf_t * SSL_CTX_set_options(conf->ssl.ctx, SSL_OP_CIPHER_SERVER_PREFERENCE); } -#ifndef OPENSSL_IS_BORINGSSL +#if (!defined OPENSSL_IS_BORINGSSL && !defined LIBRESSL_VERSION_NUMBER) /* a temporary 512-bit RSA key is required for export versions of MSIE */ SSL_CTX_set_tmp_rsa_callback(conf->ssl.ctx, ngx_ssl_rsa512_key_callback); #endif diff -r 3a647f0d5104 -r 2c5ac97b08f6 src/mail/ngx_mail_ssl_module.c --- a/src/mail/ngx_mail_ssl_module.c Wed Jul 30 04:32:15 2014 -0700 +++ b/src/mail/ngx_mail_ssl_module.c Wed Jul 30 04:32:16 2014 -0700 @@ -334,7 +334,7 @@ ngx_mail_ssl_merge_conf(ngx_conf_t *cf, SSL_CTX_set_options(conf->ssl.ctx, SSL_OP_CIPHER_SERVER_PREFERENCE); } -#ifndef OPENSSL_IS_BORINGSSL +#if (!defined OPENSSL_IS_BORINGSSL && !defined LIBRESSL_VERSION_NUMBER) SSL_CTX_set_tmp_rsa_callback(conf->ssl.ctx, ngx_ssl_rsa512_key_callback); #endif From piotr at cloudflare.com Wed Jul 30 11:42:09 2014 From: piotr at cloudflare.com (Piotr Sikora) Date: Wed, 30 Jul 2014 04:42:09 -0700 Subject: [PATCH 1 of 2] SSL: let it build against BoringSSL In-Reply-To: References: Message-ID: <3a647f0d5104612c7fa5.1406720529@Piotrs-MacBook-Pro.local> # HG changeset patch # User Piotr Sikora # Date 1406719935 25200 # Wed Jul 30 04:32:15 2014 -0700 # Node ID 3a647f0d5104612c7fa5c9cc1245057a4c0a3dc2 # Parent 4d092aa2f4637ce50284d2accd99a8e91aae2b4c SSL: let it build against BoringSSL. This change adds support for using BoringSSL as a drop-in replacement for OpenSSL without adding support for any of the BoringSSL-specific features. The #ifndefs around SSL_CTX_set_tmp_rsa_callback() aren't strictly necessary, since that function still exists in BoringSSL as a no-op, but they clearly mark the unsupported feature. Signed-off-by: Piotr Sikora diff -r 4d092aa2f463 -r 3a647f0d5104 src/event/ngx_event_openssl.c --- a/src/event/ngx_event_openssl.c Mon Jul 28 12:27:57 2014 -0700 +++ b/src/event/ngx_event_openssl.c Wed Jul 30 04:32:15 2014 -0700 @@ -106,7 +106,9 @@ int ngx_ssl_stapling_index; ngx_int_t ngx_ssl_init(ngx_log_t *log) { +#ifndef OPENSSL_IS_BORINGSSL OPENSSL_config(NULL); +#endif SSL_library_init(); SSL_load_error_strings(); @@ -217,7 +219,10 @@ ngx_ssl_create(ngx_ssl_t *ssl, ngx_uint_ SSL_CTX_set_options(ssl->ctx, SSL_OP_MSIE_SSLV2_RSA_PADDING); #endif +#ifdef SSL_OP_SSLEAY_080_CLIENT_DH_BUG SSL_CTX_set_options(ssl->ctx, SSL_OP_SSLEAY_080_CLIENT_DH_BUG); +#endif + SSL_CTX_set_options(ssl->ctx, SSL_OP_TLS_D5_BUG); SSL_CTX_set_options(ssl->ctx, SSL_OP_TLS_BLOCK_PADDING_BUG); @@ -382,8 +387,13 @@ ngx_ssl_certificate(ngx_conf_t *cf, ngx_ if (--tries) { n = ERR_peek_error(); +#ifdef OPENSSL_IS_BORINGSSL + if (ERR_GET_LIB(n) == ERR_LIB_CIPHER + && ERR_GET_REASON(n) == CIPHER_R_BAD_DECRYPT) +#else if (ERR_GET_LIB(n) == ERR_LIB_EVP && ERR_GET_REASON(n) == EVP_R_BAD_DECRYPT) +#endif { ERR_clear_error(); SSL_CTX_set_default_passwd_cb_userdata(ssl->ctx, ++pwd); diff -r 4d092aa2f463 -r 3a647f0d5104 src/http/modules/ngx_http_ssl_module.c --- a/src/http/modules/ngx_http_ssl_module.c Mon Jul 28 12:27:57 2014 -0700 +++ b/src/http/modules/ngx_http_ssl_module.c Wed Jul 30 04:32:15 2014 -0700 @@ -715,8 +715,10 @@ ngx_http_ssl_merge_srv_conf(ngx_conf_t * SSL_CTX_set_options(conf->ssl.ctx, SSL_OP_CIPHER_SERVER_PREFERENCE); } +#ifndef OPENSSL_IS_BORINGSSL /* a temporary 512-bit RSA key is required for export versions of MSIE */ SSL_CTX_set_tmp_rsa_callback(conf->ssl.ctx, ngx_ssl_rsa512_key_callback); +#endif if (ngx_ssl_dhparam(cf, &conf->ssl, &conf->dhparam) != NGX_OK) { return NGX_CONF_ERROR; diff -r 4d092aa2f463 -r 3a647f0d5104 src/mail/ngx_mail_ssl_module.c --- a/src/mail/ngx_mail_ssl_module.c Mon Jul 28 12:27:57 2014 -0700 +++ b/src/mail/ngx_mail_ssl_module.c Wed Jul 30 04:32:15 2014 -0700 @@ -334,7 +334,9 @@ ngx_mail_ssl_merge_conf(ngx_conf_t *cf, SSL_CTX_set_options(conf->ssl.ctx, SSL_OP_CIPHER_SERVER_PREFERENCE); } +#ifndef OPENSSL_IS_BORINGSSL SSL_CTX_set_tmp_rsa_callback(conf->ssl.ctx, ngx_ssl_rsa512_key_callback); +#endif if (ngx_ssl_dhparam(cf, &conf->ssl, &conf->dhparam) != NGX_OK) { return NGX_CONF_ERROR; From piotr at cloudflare.com Wed Jul 30 11:42:11 2014 From: piotr at cloudflare.com (Piotr Sikora) Date: Wed, 30 Jul 2014 04:42:11 -0700 Subject: [PATCH 0 of 2] SSL: let it build against BorginSSL and LibreSSL (take #3) Message-ID: Hey Maxim, just to speed this up, this is the version without any #ifndefs around SSL_CTX_set_tmp_rsa_callback(). Feel free to commit whichever version you prefer. Best regards, Piotr Sikora From piotr at cloudflare.com Wed Jul 30 11:42:12 2014 From: piotr at cloudflare.com (Piotr Sikora) Date: Wed, 30 Jul 2014 04:42:12 -0700 Subject: [PATCH 1 of 2] SSL: let it build against BoringSSL In-Reply-To: References: Message-ID: <45ed2f1f0a6a568b6e9b.1406720532@Piotrs-MacBook-Pro.local> # HG changeset patch # User Piotr Sikora # Date 1406719935 25200 # Wed Jul 30 04:32:15 2014 -0700 # Node ID 45ed2f1f0a6a568b6e9bfe31a28172c1ebcf5d8c # Parent 4d092aa2f4637ce50284d2accd99a8e91aae2b4c SSL: let it build against BoringSSL. This change adds support for using BoringSSL as a drop-in replacement for OpenSSL without adding support for any of the BoringSSL-specific features. Signed-off-by: Piotr Sikora diff -r 4d092aa2f463 -r 45ed2f1f0a6a src/event/ngx_event_openssl.c --- a/src/event/ngx_event_openssl.c Mon Jul 28 12:27:57 2014 -0700 +++ b/src/event/ngx_event_openssl.c Wed Jul 30 04:32:15 2014 -0700 @@ -106,7 +106,9 @@ int ngx_ssl_stapling_index; ngx_int_t ngx_ssl_init(ngx_log_t *log) { +#ifndef OPENSSL_IS_BORINGSSL OPENSSL_config(NULL); +#endif SSL_library_init(); SSL_load_error_strings(); @@ -217,7 +219,10 @@ ngx_ssl_create(ngx_ssl_t *ssl, ngx_uint_ SSL_CTX_set_options(ssl->ctx, SSL_OP_MSIE_SSLV2_RSA_PADDING); #endif +#ifdef SSL_OP_SSLEAY_080_CLIENT_DH_BUG SSL_CTX_set_options(ssl->ctx, SSL_OP_SSLEAY_080_CLIENT_DH_BUG); +#endif + SSL_CTX_set_options(ssl->ctx, SSL_OP_TLS_D5_BUG); SSL_CTX_set_options(ssl->ctx, SSL_OP_TLS_BLOCK_PADDING_BUG); @@ -382,8 +387,13 @@ ngx_ssl_certificate(ngx_conf_t *cf, ngx_ if (--tries) { n = ERR_peek_error(); +#ifdef OPENSSL_IS_BORINGSSL + if (ERR_GET_LIB(n) == ERR_LIB_CIPHER + && ERR_GET_REASON(n) == CIPHER_R_BAD_DECRYPT) +#else if (ERR_GET_LIB(n) == ERR_LIB_EVP && ERR_GET_REASON(n) == EVP_R_BAD_DECRYPT) +#endif { ERR_clear_error(); SSL_CTX_set_default_passwd_cb_userdata(ssl->ctx, ++pwd); From piotr at cloudflare.com Wed Jul 30 11:42:13 2014 From: piotr at cloudflare.com (Piotr Sikora) Date: Wed, 30 Jul 2014 04:42:13 -0700 Subject: [PATCH 2 of 2] SSL: let it build against LibreSSL In-Reply-To: References: Message-ID: # HG changeset patch # User Piotr Sikora # Date 1406719936 25200 # Wed Jul 30 04:32:16 2014 -0700 # Node ID e0eaf2d92a8cee90abe592d7ac01d3118cb0853a # Parent 45ed2f1f0a6a568b6e9bfe31a28172c1ebcf5d8c SSL: let it build against LibreSSL. LibreSSL developers decided that LibreSSL is OpenSSL-2.0.0, so tests for OpenSSL-1.0.2+ are now passing, even though the library doesn't provide functions that are expected from that version of OpenSSL. Signed-off-by: Piotr Sikora diff -r 45ed2f1f0a6a -r e0eaf2d92a8c src/event/ngx_event_openssl.c --- a/src/event/ngx_event_openssl.c Wed Jul 30 04:32:15 2014 -0700 +++ b/src/event/ngx_event_openssl.c Wed Jul 30 04:32:16 2014 -0700 @@ -50,7 +50,7 @@ static int ngx_ssl_session_ticket_key_ca HMAC_CTX *hctx, int enc); #endif -#if OPENSSL_VERSION_NUMBER < 0x10002002L +#if (OPENSSL_VERSION_NUMBER < 0x10002002L || defined LIBRESSL_VERSION_NUMBER) static ngx_int_t ngx_ssl_check_name(ngx_str_t *name, ASN1_STRING *str); #endif @@ -2743,7 +2743,7 @@ ngx_ssl_check_host(ngx_connection_t *c, return NGX_ERROR; } -#if OPENSSL_VERSION_NUMBER >= 0x10002002L +#if (OPENSSL_VERSION_NUMBER >= 0x10002002L && !defined LIBRESSL_VERSION_NUMBER) /* X509_check_host() is only available in OpenSSL 1.0.2+ */ @@ -2860,7 +2860,7 @@ found: } -#if OPENSSL_VERSION_NUMBER < 0x10002002L +#if (OPENSSL_VERSION_NUMBER < 0x10002002L || defined LIBRESSL_VERSION_NUMBER) static ngx_int_t ngx_ssl_check_name(ngx_str_t *name, ASN1_STRING *pattern) From mdounin at mdounin.ru Wed Jul 30 12:22:43 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Wed, 30 Jul 2014 16:22:43 +0400 Subject: [PATCH 2 of 2] SSL: let it build against BoringSSL In-Reply-To: References: <20140729221534.GS1849@mdounin.ru> <20140730041545.GX1849@mdounin.ru> Message-ID: <20140730122243.GZ1849@mdounin.ru> Hello! On Tue, Jul 29, 2014 at 10:32:29PM -0700, Piotr Sikora wrote: > Hey Maxim, > > > The problem is lots of added lines required to avoid calling it. > > Alternatively, would you be OK with a change that adds #ifndefs around > SSL_CTX_set_tmp_rsa_callback() in ngx_http_ssl_module and > ngx_mail_ssl_module, but keeps ngx_ssl_rsa512_key_callback without > guards? It's exported, so compilers won't complain about it being > unused. I don't really think we at all need these ifdefs. Additionally, I tend to think that we don't need SSL_CTX_set_tmp_rsa_callback() in mail/http either, it should be hidden in openssl glue code. -- Maxim Dounin http://nginx.org/ From mdounin at mdounin.ru Wed Jul 30 12:23:28 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Wed, 30 Jul 2014 16:23:28 +0400 Subject: [PATCH] SSL: let it build against LibreSSL In-Reply-To: References: <20140730005553.GV1849@mdounin.ru> Message-ID: <20140730122328.GA1849@mdounin.ru> Hello! On Tue, Jul 29, 2014 at 07:15:06PM -0700, Piotr Sikora wrote: > Hey Maxim, > > > As previously suggested, this doesn't looks like a right way to > > go. If LibreSSL folks continue to insist this is OpenSSL-2.0.0, > > then we'll probably have redefine OPENSSL_VERSION_NUMBER > > ourselves. > > Unfortunately, it looks like they've made their mind :( The discussion > on it stopped, they've made 4 releases with that version already and > OpenBSD is in the release mode right now, so I don't expect that > they're going to change it. > > Redefining OPENSSL_VERSION_NUMBER doesn't seem like the prettiest > solution, though. Well, that's anyway how it'll end up in multiple places, so I don't think it's a big problem. And it looks bettern than adding #ifs in all places. Alternatively, we may consider rewriting tests to avoid checking OPENSSL_VERSION_NUMBER and/or do configure time tests instead. -- Maxim Dounin http://nginx.org/ From pdn at cryptopro.ru Wed Jul 30 15:29:10 2014 From: pdn at cryptopro.ru (Dmitrii Pichulin) Date: Wed, 30 Jul 2014 19:29:10 +0400 Subject: [PATCH] allow to use engine keyform for server private key In-Reply-To: <20140729154024.GP1849@mdounin.ru> References: <20140729154024.GP1849@mdounin.ru> Message-ID: # HG changeset patch # User Dmitrii Pichulin # Date 1406733892 -14400 # Wed Jul 30 19:24:52 2014 +0400 # Node ID a4c89ae85f45153760637058a75f4338b3974219 # Parent 4d092aa2f4637ce50284d2accd99a8e91aae2b4c allow to use engine keyform for server private key diff -r 4d092aa2f463 -r a4c89ae85f45 src/event/ngx_event_openssl.c --- a/src/event/ngx_event_openssl.c Mon Jul 28 12:27:57 2014 -0700 +++ b/src/event/ngx_event_openssl.c Wed Jul 30 19:24:52 2014 +0400 @@ -17,6 +17,11 @@ ngx_uint_t engine; /* unsigned engine:1; */ } ngx_openssl_conf_t; +typedef struct { + const void *password; + const char *prompt_info; +} PW_CB_DATA; + static int ngx_ssl_password_callback(char *buf, int size, int rwflag, void *userdata); @@ -265,11 +270,16 @@ ngx_ssl_certificate(ngx_conf_t *cf, ngx_ssl_t *ssl, ngx_str_t *cert, ngx_str_t *key, ngx_array_t *passwords) { + char *p, *last; BIO *bio; X509 *x509; + ENGINE *engine; + EVP_PKEY *private_key; + PW_CB_DATA pwd_data; u_long n; ngx_str_t *pwd; ngx_uint_t tries; + u_char pwd_buf[NGX_SSL_PASSWORD_BUFFER_SIZE]; if (ngx_conf_full_name(cf->cycle, cert, 1) != NGX_OK) { return NGX_ERROR; @@ -352,6 +362,75 @@ BIO_free(bio); + if (ngx_strncmp(key->data, "engine:", sizeof("engine:") - 1) == 0) { + + p = (char *) key->data + sizeof("engine:") - 1; + last = ngx_strchr(p, ':'); + + if (last == NULL) { + ngx_conf_log_error(NGX_LOG_EMERG, cf, 0, "invalid syntax: %V", key); + return NGX_ERROR; + } + + p[last - p] = '\0'; + + engine = ENGINE_by_id(p); + + if (engine == NULL) { + ngx_ssl_error(NGX_LOG_EMERG, ssl->log, 0, + "ENGINE_by_id(\"%s\") failed", p); + return NGX_ERROR; + } + + p[last - p] = ':'; + + if (passwords) { + pwd = passwords->elts; + + ngx_cpystrn(pwd_buf, pwd->data, pwd->len + 1); + + pwd_data.password = pwd_buf; + } else { + pwd_data.password = NULL; + } + pwd_data.prompt_info = NULL; + + last++; + + private_key = ENGINE_load_private_key(engine, last, 0, + (void *) &pwd_data); + + ngx_memzero(pwd_buf, NGX_SSL_PASSWORD_BUFFER_SIZE); + + if (private_key == NULL) { + ngx_ssl_error(NGX_LOG_EMERG, ssl->log, 0, + "ENGINE_load_private_key(\"%s\") failed", last); + + if (ENGINE_free(engine) == 0) { + ngx_ssl_error(NGX_LOG_EMERG, ssl->log, 0, + "ENGINE_free() failed"); + } + return NGX_ERROR; + } + + if (ENGINE_free(engine) == 0) { + ngx_ssl_error(NGX_LOG_EMERG, ssl->log, 0, "ENGINE_free() failed"); + EVP_PKEY_free(private_key); + return NGX_ERROR; + } + + if (SSL_CTX_use_PrivateKey(ssl->ctx, private_key) == 0) { + ngx_ssl_error(NGX_LOG_EMERG, ssl->log, 0, + "SSL_CTX_use_PrivateKey(\"%s\") failed", last); + EVP_PKEY_free(private_key); + return NGX_ERROR; + } + + EVP_PKEY_free(private_key); + + return NGX_OK; + } + if (ngx_conf_full_name(cf->cycle, key, 1) != NGX_OK) { return NGX_ERROR; } From mdounin at mdounin.ru Wed Jul 30 16:42:59 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Wed, 30 Jul 2014 16:42:59 +0000 Subject: [nginx] SSL: let it build against BoringSSL. Message-ID: details: http://hg.nginx.org/nginx/rev/45ed2f1f0a6a branches: changeset: 5778:45ed2f1f0a6a user: Piotr Sikora date: Wed Jul 30 04:32:15 2014 -0700 description: SSL: let it build against BoringSSL. This change adds support for using BoringSSL as a drop-in replacement for OpenSSL without adding support for any of the BoringSSL-specific features. Signed-off-by: Piotr Sikora diffstat: src/event/ngx_event_openssl.c | 10 ++++++++++ 1 files changed, 10 insertions(+), 0 deletions(-) diffs (38 lines): diff --git a/src/event/ngx_event_openssl.c b/src/event/ngx_event_openssl.c --- a/src/event/ngx_event_openssl.c +++ b/src/event/ngx_event_openssl.c @@ -106,7 +106,9 @@ int ngx_ssl_stapling_index; ngx_int_t ngx_ssl_init(ngx_log_t *log) { +#ifndef OPENSSL_IS_BORINGSSL OPENSSL_config(NULL); +#endif SSL_library_init(); SSL_load_error_strings(); @@ -217,7 +219,10 @@ ngx_ssl_create(ngx_ssl_t *ssl, ngx_uint_ SSL_CTX_set_options(ssl->ctx, SSL_OP_MSIE_SSLV2_RSA_PADDING); #endif +#ifdef SSL_OP_SSLEAY_080_CLIENT_DH_BUG SSL_CTX_set_options(ssl->ctx, SSL_OP_SSLEAY_080_CLIENT_DH_BUG); +#endif + SSL_CTX_set_options(ssl->ctx, SSL_OP_TLS_D5_BUG); SSL_CTX_set_options(ssl->ctx, SSL_OP_TLS_BLOCK_PADDING_BUG); @@ -382,8 +387,13 @@ ngx_ssl_certificate(ngx_conf_t *cf, ngx_ if (--tries) { n = ERR_peek_error(); +#ifdef OPENSSL_IS_BORINGSSL + if (ERR_GET_LIB(n) == ERR_LIB_CIPHER + && ERR_GET_REASON(n) == CIPHER_R_BAD_DECRYPT) +#else if (ERR_GET_LIB(n) == ERR_LIB_EVP && ERR_GET_REASON(n) == EVP_R_BAD_DECRYPT) +#endif { ERR_clear_error(); SSL_CTX_set_default_passwd_cb_userdata(ssl->ctx, ++pwd); From mdounin at mdounin.ru Wed Jul 30 16:43:11 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Wed, 30 Jul 2014 16:43:11 +0000 Subject: [nginx] SSL: let it build against LibreSSL. Message-ID: details: http://hg.nginx.org/nginx/rev/e0eaf2d92a8c branches: changeset: 5779:e0eaf2d92a8c user: Piotr Sikora date: Wed Jul 30 04:32:16 2014 -0700 description: SSL: let it build against LibreSSL. LibreSSL developers decided that LibreSSL is OpenSSL-2.0.0, so tests for OpenSSL-1.0.2+ are now passing, even though the library doesn't provide functions that are expected from that version of OpenSSL. Signed-off-by: Piotr Sikora diffstat: src/event/ngx_event_openssl.c | 6 +++--- 1 files changed, 3 insertions(+), 3 deletions(-) diffs (30 lines): diff --git a/src/event/ngx_event_openssl.c b/src/event/ngx_event_openssl.c --- a/src/event/ngx_event_openssl.c +++ b/src/event/ngx_event_openssl.c @@ -50,7 +50,7 @@ static int ngx_ssl_session_ticket_key_ca HMAC_CTX *hctx, int enc); #endif -#if OPENSSL_VERSION_NUMBER < 0x10002002L +#if (OPENSSL_VERSION_NUMBER < 0x10002002L || defined LIBRESSL_VERSION_NUMBER) static ngx_int_t ngx_ssl_check_name(ngx_str_t *name, ASN1_STRING *str); #endif @@ -2743,7 +2743,7 @@ ngx_ssl_check_host(ngx_connection_t *c, return NGX_ERROR; } -#if OPENSSL_VERSION_NUMBER >= 0x10002002L +#if (OPENSSL_VERSION_NUMBER >= 0x10002002L && !defined LIBRESSL_VERSION_NUMBER) /* X509_check_host() is only available in OpenSSL 1.0.2+ */ @@ -2860,7 +2860,7 @@ found: } -#if OPENSSL_VERSION_NUMBER < 0x10002002L +#if (OPENSSL_VERSION_NUMBER < 0x10002002L || defined LIBRESSL_VERSION_NUMBER) static ngx_int_t ngx_ssl_check_name(ngx_str_t *name, ASN1_STRING *pattern) From mdounin at mdounin.ru Wed Jul 30 16:43:25 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Wed, 30 Jul 2014 20:43:25 +0400 Subject: [PATCH 0 of 2] SSL: let it build against BorginSSL and LibreSSL (take #3) In-Reply-To: References: Message-ID: <20140730164325.GE1849@mdounin.ru> Hello! On Wed, Jul 30, 2014 at 04:42:11AM -0700, Piotr Sikora wrote: > Hey Maxim, > just to speed this up, this is the version without any #ifndefs > around SSL_CTX_set_tmp_rsa_callback(). > > Feel free to commit whichever version you prefer. These two committed, thanks. -- Maxim Dounin http://nginx.org/ From piotr at cloudflare.com Wed Jul 30 22:13:12 2014 From: piotr at cloudflare.com (Piotr Sikora) Date: Wed, 30 Jul 2014 15:13:12 -0700 Subject: [PATCH] SSL: let it build against LibreSSL In-Reply-To: <20140730122328.GA1849@mdounin.ru> References: <20140730005553.GV1849@mdounin.ru> <20140730122328.GA1849@mdounin.ru> Message-ID: Hey Maxim, > Alternatively, we may consider rewriting tests to avoid checking > OPENSSL_VERSION_NUMBER and/or do configure time tests instead. Yeah, that's much better approach, although I don't think we can avoid it completely. Also, most of the existing checks can be removed once support for OpenSSL-0.9.7 is dropped... Best regards, Piotr Sikora From codeeply at gmail.com Thu Jul 31 02:07:37 2014 From: codeeply at gmail.com (Jianjun Zheng) Date: Thu, 31 Jul 2014 10:07:37 +0800 Subject: [PATCH] Core: bugfix for the ngx_slab_max_size case In-Reply-To: <20140709184034.GX1849@mdounin.ru> References: <20140709184034.GX1849@mdounin.ru> Message-ID: :) 2014-07-10 2:40 GMT+08:00 Maxim Dounin : > Hello! > > On Wed, Jun 18, 2014 at 05:00:14PM +0800, Jianjun Zheng wrote: > > > At present, alloting memory with size of ngx_slab_max_size causes > > > > 1) an internal fragmentation, size of ngx_slab_max_size, comes into being > > > > 2) the slot with index of (ngx_pagesize_shift - pool->min_shift - 1) > > is the right slot for this size. > > > > > > # HG changeset patch > > # User Jianjun Zheng > > # Date 1403080799 -28800 > > # Wed Jun 18 16:39:59 2014 +0800 > > # Node ID 1704335dd810e2e2abb2b393b4f7b7c9004c6012 > > # Parent ec919574cc14f7781c0ca212cffec586f88eec40 > > Core: bugfix for the ngx_slab_max_size case > > > > diff -r ec919574cc14 -r 1704335dd810 src/core/ngx_slab.c > > --- a/src/core/ngx_slab.c Tue Jun 17 16:51:25 2014 +0400 > > +++ b/src/core/ngx_slab.c Wed Jun 18 16:39:59 2014 +0800 > > @@ -160,7 +160,7 @@ > > ngx_uint_t i, slot, shift, map; > > ngx_slab_page_t *page, *prev, *slots; > > > > - if (size >= ngx_slab_max_size) { > > + if (size > ngx_slab_max_size) { > > > > ngx_log_debug1(NGX_LOG_DEBUG_ALLOC, ngx_cycle->log, 0, > > "slab alloc: %uz", size); > > Sorry for late reply. > > Yes, the patch is right, allocation of ngx_slab_max_size exactly > (aka half page, 2048 bytes on platforms with 4k pages) can be > satisfied from the largest slot and don't need the whole page to > be allocated. While it's a corner case, it's certainly can be > important in some cases and worth fixing. > > Committed, thanks. > > -- > Maxim Dounin > http://nginx.org/ > > _______________________________________________ > nginx-devel mailing list > nginx-devel at nginx.org > http://mailman.nginx.org/mailman/listinfo/nginx-devel > -------------- next part -------------- An HTML attachment was scrubbed... URL: From info at phpgangsta.de Thu Jul 31 02:32:16 2014 From: info at phpgangsta.de (Michael Kliewe) Date: Thu, 31 Jul 2014 04:32:16 +0200 Subject: [PATCH] Mail: send starttls flag value to auth script In-Reply-To: References: <51fd90f96449c23af007.1394099969@HPC> <20140306162718.GL34696@mdounin.ru> <877FD2F6-57CD-4C14-9F2B-4C9E909C3488@phpgangsta.de> Message-ID: <53D9AAB0.5060501@phpgangsta.de> Hi Maxim, this is very interesting to have in nginx, and it's a very easy patch. Any chance this gets into nginx? It helps a lot to migrate users to encrypted mail connections, allowing some users to still use unencrypted connections, and log in the auth script which user already uses encrypted connections. It would be great if you could add it to nginx. Thanks! Michael Am 19.06.2014 um 22:10 schrieb Filipe Da Silva: > Hi, > > I forget to post the reworked version. > > Here is it. > > Regards, > Filipe DA SILVA > > # HG changeset patch > # Parent b2b5b1b741290adf60220f44f6e37cd8bd9d3885 > Mail: send a secure connection flag to auth script. > > Allow to do logging (if logging takes place in the auth script) and or force > some users to use STARTTLS while others can use unencrypted connection. > > diff -r b2b5b1b74129 src/mail/ngx_mail_auth_http_module.c > --- a/src/mail/ngx_mail_auth_http_module.c Fri Mar 07 15:17:38 2014 +0400 > +++ b/src/mail/ngx_mail_auth_http_module.c Wed Mar 12 15:49:21 2014 +0100 > @@ -1165,6 +1165,9 @@ ngx_mail_auth_http_create_request(ngx_ma > + sizeof("Auth-Salt: ") - 1 + s->salt.len > + sizeof("Auth-Protocol: ") - 1 + cscf->protocol->name.len > + sizeof(CRLF) - 1 > +#if (NGX_MAIL_SSL) > + + sizeof("Auth-Secured: ") - 1 + 1 + sizeof(CRLF) - 1 > +#endif > + sizeof("Auth-Login-Attempt: ") - 1 + NGX_INT_T_LEN > + sizeof(CRLF) - 1 > + sizeof("Client-IP: ") - 1 + s->connection->addr_text.len > @@ -1219,6 +1222,13 @@ ngx_mail_auth_http_create_request(ngx_ma > cscf->protocol->name.len); > *b->last++ = CR; *b->last++ = LF; > > +#if (NGX_MAIL_SSL) > + b->last = ngx_cpymem(b->last, "Auth-Secured: ", > + sizeof("Auth-Secured: ") - 1); > + *b->last++ = s->connection->ssl ? '1' : '0' ; > + *b->last++ = CR; *b->last++ = LF; > +#endif > + > b->last = ngx_sprintf(b->last, "Auth-Login-Attempt: %ui" CRLF, > s->login_attempt); > > > > > 2014-03-06 18:03 GMT+01:00 Michael Kliewe : >> Hi Maxim, >> >> On Mar 6, 2014, at 5:27 PM, Maxim Dounin wrote: >> >>> Hello! >>> >>> On Thu, Mar 06, 2014 at 10:59:29AM +0100, Filipe da Silva wrote: >>> >>>> # HG changeset patch >>>> # User Filipe da Silva >>>> # Date 1394099468 -3600 >>>> # Thu Mar 06 10:51:08 2014 +0100 >>>> # Node ID 51fd90f96449c23af0076a19efbfdb1f88702125 >>>> # Parent 24df9fa5868957c1fb9a2d1569271e0958327dad >>>> Mail: send starttls flag value to auth script. >>>> >>>> Allow to do logging (if logging takes place in the auth script) and or force >>>> some users to use STARTTLS while others can use unencrypted connection. >>>> >>> I don't think that it's a good idea to pass STARTTLS into auth >>> script. If at all needed, it should be something like a flag "if >>> SSL is used", not an explicit STARTTLS status. From auth script >>> point of view there is no difference if a connection uses SSL on a >>> dedicated port or encryption was negotiated using STARTLS. >> yes, it is needed ;-) >> >> You are right, that would also be possible, the auth script then can check which port has been used, and then has the information if it has been STARTTLS or SSL. In our case we want to distinguish between STARTTLS and SSL in the auth script. >> >> Both solutions are fine I think, so let's take Maxims ;-) (Sorry Filipe for the extra work) >> >> Hope this easy patch gets into nginx then, we need it ;-) >> >> Thanks! >> Michael >> _______________________________________________ >> nginx-devel mailing list >> nginx-devel at nginx.org >> http://mailman.nginx.org/mailman/listinfo/nginx-devel >> >> >> _______________________________________________ >> nginx-devel mailing list >> nginx-devel at nginx.org >> http://mailman.nginx.org/mailman/listinfo/nginx-devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From pluknet at nginx.com Thu Jul 31 10:00:46 2014 From: pluknet at nginx.com (Sergey Kandaurov) Date: Thu, 31 Jul 2014 10:00:46 +0000 Subject: [nginx] Typos fixed. Message-ID: details: http://hg.nginx.org/nginx/rev/f1e05e533c8b branches: changeset: 5780:f1e05e533c8b user: Sergey Kandaurov date: Thu Jul 31 13:59:37 2014 +0400 description: Typos fixed. diffstat: docs/xml/nginx/changes.xml | 8 ++++---- 1 files changed, 4 insertions(+), 4 deletions(-) diffs (39 lines): diff -r e0eaf2d92a8c -r f1e05e533c8b docs/xml/nginx/changes.xml --- a/docs/xml/nginx/changes.xml Wed Jul 30 04:32:16 2014 -0700 +++ b/docs/xml/nginx/changes.xml Thu Jul 31 13:59:37 2014 +0400 @@ -5329,7 +5329,7 @@ the bug had appeared in 0.9.0. -????????? ?????? "If-Unmodified-Since" ? ????????? ??????? ???????. +????????? ?????? "If-Unmodified-Since" ? ????????? ??????? ???????. the "If-Unmodified-Since" client request header line support. @@ -11560,7 +11560,7 @@ if keepalive was enabled. -nginx ?? ??????????? ????? FastCGI-???????, ???? ?????? ????????? ????? ???? +nginx ?? ??????????? ????? FastCGI-???????, ???? ?????? ????????? ?????? ???? ? ????? ?????? FastCGI; ?????? ????????? ? 0.6.2.
??????? ?????? ??????. @@ -14974,7 +14974,7 @@ the "ip_hash" directive inside the "upst -?????? WAIT ? ?????? "Auth-Status" ? ????????? ?????? ??????? ?????????????? +?????? WAIT ? ?????? "Auth-Status" ? ????????? ?????? ??????? ?????????????? IMAP/POP3 ??????. @@ -19701,7 +19701,7 @@ then nginx started to request all backen -???? ? ????????? ??????? ???? ????????????? ?????? "Host", "Connection", +???? ? ????????? ??????? ???? ????????????? ?????? "Host", "Connection", "Content-Length" ? "Authorization", ?? nginx ?????? ?????? ?????? 400. From piotr at cloudflare.com Thu Jul 31 10:41:30 2014 From: piotr at cloudflare.com (Piotr Sikora) Date: Thu, 31 Jul 2014 03:41:30 -0700 Subject: [PATCH] Core: add support for expressing size in gigabytes Message-ID: # HG changeset patch # User Piotr Sikora # Date 1406803246 25200 # Thu Jul 31 03:40:46 2014 -0700 # Node ID c1aeec0f33fe6a42fde0a1851228a130f5ab12a1 # Parent e0eaf2d92a8cee90abe592d7ac01d3118cb0853a Core: add support for expressing size in gigabytes. Signed-off-by: Piotr Sikora diff -r e0eaf2d92a8c -r c1aeec0f33fe src/core/ngx_parse.c --- a/src/core/ngx_parse.c Wed Jul 30 04:32:16 2014 -0700 +++ b/src/core/ngx_parse.c Thu Jul 31 03:40:46 2014 -0700 @@ -33,6 +33,12 @@ ngx_parse_size(ngx_str_t *line) scale = 1024 * 1024; break; + case 'G': + case 'g': + len--; + scale = 1024 * 1024 * 1024; + break; + default: scale = 1; } From piotr at cloudflare.com Thu Jul 31 10:53:20 2014 From: piotr at cloudflare.com (Piotr Sikora) Date: Thu, 31 Jul 2014 03:53:20 -0700 Subject: [PATCH] Configure: remove outdated and unused patch.zlib.h Message-ID: <5be611309d6f79838791.1406804000@Piotrs-MacBook-Pro.local> # HG changeset patch # User Piotr Sikora # Date 1406803948 25200 # Thu Jul 31 03:52:28 2014 -0700 # Node ID 5be611309d6f7983879104054d4a68feece64142 # Parent f1e05e533c8b7028121104740f2ab76e49d9212f Configure: remove outdated and unused patch.zlib.h. Signed-off-by: Piotr Sikora diff -r f1e05e533c8b -r 5be611309d6f auto/lib/zlib/patch.zlib.h --- a/auto/lib/zlib/patch.zlib.h Thu Jul 31 13:59:37 2014 +0400 +++ /dev/null Thu Jan 01 00:00:00 1970 +0000 @@ -1,10 +0,0 @@ ---- zlib.h Thu Jul 9 20:06:56 1998 -+++ zlib-1.1.3/zlib.h Tue Mar 22 13:41:04 2005 -@@ -709,7 +709,6 @@ - (0 in case of error). - */ - --ZEXTERN int ZEXPORTVA gzprintf OF((gzFile file, const char *format, ...)); - /* - Converts, formats, and writes the args to the compressed file under - control of the format string, as in fprintf. gzprintf returns the number of From kristian.hermansen at gmail.com Thu Jul 31 10:56:59 2014 From: kristian.hermansen at gmail.com (Kristian Erik Hermansen) Date: Thu, 31 Jul 2014 03:56:59 -0700 Subject: [PATCH] Config: enhancing nginx default config file with added security options Message-ID: # HG changeset patch # User Kristian Erik Hermansen # Date 1406803911 25200 # Thu Jul 31 03:51:51 2014 -0700 # Node ID 8966ff589f5de5e9155335373247de4485451304 # Parent e0eaf2d92a8cee90abe592d7ac01d3118cb0853a Config: enhancing nginx default config file with added security options. diff -r e0eaf2d92a8c -r 8966ff589f5d conf/nginx.conf --- a/conf/nginx.conf Wed Jul 30 04:32:16 2014 -0700 +++ b/conf/nginx.conf Thu Jul 31 03:51:51 2014 -0700 @@ -105,9 +105,34 @@ # ssl_session_cache shared:SSL:1m; # ssl_session_timeout 5m; + # recommended protocols that provide better security and compatibility + # + # ssl_protocols SSLv3 TLSv1 TLSv1.1 TLSv1.2; + # ssl_ciphers HIGH:!aNULL:!MD5; # ssl_prefer_server_ciphers on; + # security headers recommended by OWASP to block common attacks + # + # add_header X-Frame-Options 'DENY'; + # add_header X-Content-Type-Options 'nosniff'; + # add_header X-XSS-Protection '1; mode=block'; + # add_header Cache-Control 'no-cache, no-store, must-revalidate'; + # add_header Pragma 'no-cache'; + # add_header Expires '-1'; + # + # security headers that require additional configuration + # + # #add_header X-Permitted-Cross-Domain-Policies 'master-only'; + # #add_header Content-Security-Policy-Report-Only "default-src 'self'; report-uri /csp_report_parser"; + # #add_header Content-Security-Policy "default-src 'self'; report-uri /csp_report_parser"; + # + # security header required to attain an 'A+' rating via Qualys SSL Labs + # + # #add_header Strict-Transport-Security 'max-age=31536000'; + + # charset utf-8; + # location / { # root html; # index index.html index.htm; -- Regards, Kristian Erik Hermansen https://www.linkedin.com/in/kristianhermansen https://google.com/+KristianHermansen From mdounin at mdounin.ru Thu Jul 31 12:25:43 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Thu, 31 Jul 2014 16:25:43 +0400 Subject: [PATCH] Config: enhancing nginx default config file with added security options In-Reply-To: References: Message-ID: <20140731122543.GI1849@mdounin.ru> Hello! On Thu, Jul 31, 2014 at 03:56:59AM -0700, Kristian Erik Hermansen wrote: > # HG changeset patch > # User Kristian Erik Hermansen > # Date 1406803911 25200 > # Thu Jul 31 03:51:51 2014 -0700 > # Node ID 8966ff589f5de5e9155335373247de4485451304 > # Parent e0eaf2d92a8cee90abe592d7ac01d3118cb0853a > Config: enhancing nginx default config file with added security options. No, thanks. We intentionally avoid various "security recommendations" except via providing appropriate defaults. People tend to have different ideas of what security is, and how it should be achieved. Additionally, all such recommendations tend to become stale in a very short period of time. Goal of the sample configuration file is to show how to configure things, not to give any recommendations. Some additional comments below. > diff -r e0eaf2d92a8c -r 8966ff589f5d conf/nginx.conf > --- a/conf/nginx.conf Wed Jul 30 04:32:16 2014 -0700 > +++ b/conf/nginx.conf Thu Jul 31 03:51:51 2014 -0700 > @@ -105,9 +105,34 @@ > # ssl_session_cache shared:SSL:1m; > # ssl_session_timeout 5m; > > + # recommended protocols that provide better security and compatibility > + # > + # ssl_protocols SSLv3 TLSv1 TLSv1.1 TLSv1.2; > + This is the default and usually there is no need to set it explicitly. > # ssl_ciphers HIGH:!aNULL:!MD5; > # ssl_prefer_server_ciphers on; > > + # security headers recommended by OWASP to block common attacks > + # > + # add_header X-Frame-Options 'DENY'; > + # add_header X-Content-Type-Options 'nosniff'; > + # add_header X-XSS-Protection '1; mode=block'; > + # add_header Cache-Control 'no-cache, no-store, must-revalidate'; > + # add_header Pragma 'no-cache'; > + # add_header Expires '-1'; Cache-related headers are either invalid (Expires syntax doesn't allow "-1" as a valid value, and "Pragma: no-cache" behaviour is unspecified when used in a response) or just silly (Cache-Control in question disables caching, which is irrelevant for security in most cases, but will make things much slower). Moreover, there is the "expires" directive to control cache-related headers, and it should be used in a proper nginx configuration instead, see http://nginx.org/r/expires. -- Maxim Dounin http://nginx.org/ From mdounin at mdounin.ru Thu Jul 31 13:49:18 2014 From: mdounin at mdounin.ru (Maxim Dounin) Date: Thu, 31 Jul 2014 17:49:18 +0400 Subject: [PATCH] allow to use engine keyform for server private key In-Reply-To: References: <20140729154024.GP1849@mdounin.ru> Message-ID: <20140731134918.GJ1849@mdounin.ru> Hello! On Wed, Jul 30, 2014 at 07:29:10PM +0400, Dmitrii Pichulin wrote: > # HG changeset patch > # User Dmitrii Pichulin > # Date 1406733892 -14400 > # Wed Jul 30 19:24:52 2014 +0400 > # Node ID a4c89ae85f45153760637058a75f4338b3974219 > # Parent 4d092aa2f4637ce50284d2accd99a8e91aae2b4c > allow to use engine keyform for server private key > > diff -r 4d092aa2f463 -r a4c89ae85f45 src/event/ngx_event_openssl.c > --- a/src/event/ngx_event_openssl.c Mon Jul 28 12:27:57 2014 -0700 > +++ b/src/event/ngx_event_openssl.c Wed Jul 30 19:24:52 2014 +0400 > @@ -17,6 +17,11 @@ > ngx_uint_t engine; /* unsigned engine:1; */ > } ngx_openssl_conf_t; > > +typedef struct { > + const void *password; > + const char *prompt_info; > +} PW_CB_DATA; > + > > static int ngx_ssl_password_callback(char *buf, int size, int rwflag, > void *userdata); > @@ -265,11 +270,16 @@ > ngx_ssl_certificate(ngx_conf_t *cf, ngx_ssl_t *ssl, ngx_str_t *cert, > ngx_str_t *key, ngx_array_t *passwords) > { > + char *p, *last; > BIO *bio; > X509 *x509; > + ENGINE *engine; > + EVP_PKEY *private_key; > + PW_CB_DATA pwd_data; > u_long n; > ngx_str_t *pwd; > ngx_uint_t tries; > + u_char pwd_buf[NGX_SSL_PASSWORD_BUFFER_SIZE]; > > if (ngx_conf_full_name(cf->cycle, cert, 1) != NGX_OK) { > return NGX_ERROR; > @@ -352,6 +362,75 @@ > > BIO_free(bio); > > + if (ngx_strncmp(key->data, "engine:", sizeof("engine:") - 1) == 0) { > + > + p = (char *) key->data + sizeof("engine:") - 1; > + last = ngx_strchr(p, ':'); > + > + if (last == NULL) { > + ngx_conf_log_error(NGX_LOG_EMERG, cf, 0, "invalid syntax: %V", key); > + return NGX_ERROR; > + } > + > + p[last - p] = '\0'; > + > + engine = ENGINE_by_id(p); After Piotr's patch (http://hg.nginx.org/nginx/rev/4d092aa2f463) we are able to work with OpenSSL compiled with OPENSSL_NO_ENGINE. Breaking this wouldn't be a good idea. > + > + if (engine == NULL) { > + ngx_ssl_error(NGX_LOG_EMERG, ssl->log, 0, > + "ENGINE_by_id(\"%s\") failed", p); > + return NGX_ERROR; > + } > + > + p[last - p] = ':'; > + > + if (passwords) { > + pwd = passwords->elts; > + > + ngx_cpystrn(pwd_buf, pwd->data, pwd->len + 1); > + > + pwd_data.password = pwd_buf; > + } else { > + pwd_data.password = NULL; > + } > + pwd_data.prompt_info = NULL; > + > + last++; > + > + private_key = ENGINE_load_private_key(engine, last, 0, > + (void *) &pwd_data); I don't see how it's expected to work. You only pass private data for UI callbacks, but not callbacks itself. Anyway, proper implementation of passing key passwords into an engine seems to be rather big, and as per my reading of the code under crypto/engine won't work with most of the engines anyway. It might be better idea to don't try to do this for now. > + > + ngx_memzero(pwd_buf, NGX_SSL_PASSWORD_BUFFER_SIZE); > + > + if (private_key == NULL) { > + ngx_ssl_error(NGX_LOG_EMERG, ssl->log, 0, > + "ENGINE_load_private_key(\"%s\") failed", last); > + > + if (ENGINE_free(engine) == 0) { > + ngx_ssl_error(NGX_LOG_EMERG, ssl->log, 0, > + "ENGINE_free() failed"); > + } > + return NGX_ERROR; > + } The above referenced commit shows that we don't check ENGINE_free() return codes, so probably we shouldn't try this here as well. > + > + if (ENGINE_free(engine) == 0) { > + ngx_ssl_error(NGX_LOG_EMERG, ssl->log, 0, "ENGINE_free() failed"); > + EVP_PKEY_free(private_key); > + return NGX_ERROR; > + } > + > + if (SSL_CTX_use_PrivateKey(ssl->ctx, private_key) == 0) { > + ngx_ssl_error(NGX_LOG_EMERG, ssl->log, 0, > + "SSL_CTX_use_PrivateKey(\"%s\") failed", last); > + EVP_PKEY_free(private_key); > + return NGX_ERROR; > + } > + > + EVP_PKEY_free(private_key); > + > + return NGX_OK; > + } > + > if (ngx_conf_full_name(cf->cycle, key, 1) != NGX_OK) { > return NGX_ERROR; > } > > _______________________________________________ > nginx-devel mailing list > nginx-devel at nginx.org > http://mailman.nginx.org/mailman/listinfo/nginx-devel -- Maxim Dounin http://nginx.org/ From kristian.hermansen at gmail.com Thu Jul 31 19:06:59 2014 From: kristian.hermansen at gmail.com (Kristian Erik Hermansen) Date: Thu, 31 Jul 2014 12:06:59 -0700 Subject: [PATCH] Config: enhancing nginx default config file with added security options In-Reply-To: <20140731122543.GI1849@mdounin.ru> References: <20140731122543.GI1849@mdounin.ru> Message-ID: ??????! On Thu, Jul 31, 2014 at 5:25 AM, Maxim Dounin wrote: > We intentionally avoid various "security recommendations" except > via providing appropriate defaults. > > People tend to have different ideas of what security is, and how > it should be achieved. Additionally, all such recommendations > tend to become stale in a very short period of time. How do you define "very short period of time"? These are standards that will remain effectively indefinitely. > Goal of the sample configuration file is to show how to configure > things, not to give any recommendations. And I thought that it was useful to be secure by default, rather than insecure by default. If nginx would like to take the stance that security should be avoided while preferring ease of use, well OK then, but state that publicly here and take ownership of that stance so that I can reference your lack of commitment. > Cache-related headers are either invalid (Expires syntax doesn't > allow "-1" as a valid value, and "Pragma: no-cache" behaviour is > unspecified when used in a response) or just silly (Cache-Control > in question disables caching, which is irrelevant for security in > most cases, but will make things much slower). If you don't agree that "Expires '-1'" is valid, then maybe you should update your own internal documentation and stop recommending it, but I think your stance is incorrect. It is not only valid, but recommended. http://nginx.org/en/docs/http/ngx_http_headers_module.html The Pragma / Cache-Control options are actually very relevant, especially in corporate environments. For instance, most corporations force outbound connections via an internal web proxy. By caching content served over HTTPS, an internal attacker can infer content via the proxy cache, which is a security issue. Sensitive content should not be cached, I hope we agree. And I request you consult RFC2616 if you think the behavior is "unspecified" as you surely aren't considering the same RFCs I am referencing. http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html > Moreover, there is the "expires" directive to control > cache-related headers, and it should be used in a proper nginx > configuration instead, see http://nginx.org/r/expires. Great. Again, see my comments above regarding using it. You contradict yourself... -- Regards, Kristian Erik Hermansen https://www.linkedin.com/in/kristianhermansen https://google.com/+KristianHermansen