From sscotti at sscotti.org Fri May 1 21:31:27 2020 From: sscotti at sscotti.org (sscotti at sscotti.org) Date: Fri, 1 May 2020 16:31:27 -0500 Subject: Setting Up a Nginx Server or OpenResty on OS X or UBUNTU with http_auth_request_module Message-ID: <7489FE86-0F4D-4866-A96E-8A459E1AC742@sscotti.org> I need to set up an NGINX server on OS X or UBUNTU that has the http_auth_request_module installed and enabled. I am kind of a novice when it comes to NGINX, but familiar with Apache, LINUX, etc. There is an NGINX server packaged with my MAMP Pro server on OS X, but I don't think that comes with the http_auth_request_module: I actually do (I think) want to try using Nginx as a reverse proxy, but I will have to do some research about that also. First, I need to just get a server up and running with that module enabled. I would prefer using OS X because I do most of my development on a Mac, but installing on an UBUNTU VM would also probably work. I did try installing OpenResty using the brew package manager on OS X, and if install with "verbose" I get the log for ./configure, which looks like it is compiled with that included. So that might work ? How do I tell if that module is actually enabled ? It does say --with that module (bold). brew reinstall openresty/brew/openresty --verbose (or just install) ____________________ Brew compile output: ./configure -j8 --prefix=/usr/local/Cellar/openresty/1.15.8.3_1 --pid-path=/usr/local/var/run/openresty.pid --lock-path=/usr/local/var/run/openresty.lock --conf-path=/usr/local/etc/openresty/nginx.conf --http-log-path=/usr/local/var/log/nginx/access.log --error-log-path=/usr/local/var/log/nginx/error.log --with-cc-opt=-I/usr/local/include -I/usr/local/opt/pcre/include -I/usr/local/opt/openresty-openssl/include --with-ld-opt=-L/usr/local/lib -L/usr/local/opt/pcre/lib -L/usr/local/opt/openresty-openssl/lib --with-pcre-jit --without-http_rds_json_module --without-http_rds_csv_module --without-lua_rds_parser --with-ipv6 --with-stream --with-stream_ssl_module --with-stream_ssl_preread_module --with-http_v2_module --without-mail_pop3_module --without-mail_imap_module --without-mail_smtp_module --with-http_stub_status_module --with-http_realip_module --with-http_addition_module --with-http_auth_request_module --with-http_secure_link_module --with-http_random_index_module --with-http_geoip_module --with-http_gzip_static_module --with-http_sub_module --with-http_dav_module --with-http_flv_module --with-http_mp4_module --with-http_gunzip_module --with-threads --with-luajit-xcflags=-DLUAJIT_NUMMODE=2 -DLUAJIT_ENABLE_LUA52COMPAT -fno-stack-check --with-dtrace-probes platform: macosx (darwin) . . . . To have launchd start openresty/brew/openresty now and restart at login: brew services start openresty/brew/openresty Or, if you don't want/need a background service you can just run: openresty ___________________ The config file for openresty is here: /usr/local/etc/openresty/nginx.conf, set to listen on 81, http, and that works, I get the welcome page. Welcome to OpenResty! So I guess my question is how to I know that the http_auth_request_module is actually there, and then how to go about setting up as a reverse proxy. I know that part is probably dependent on my setup. If someone can help with that it would be greatly appreciated, although it seems like it is installed on OS X. E-mails: sscotti at sscotti.org, sscotti at icloud.com -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: PastedGraphic-1.png Type: image/png Size: 64093 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 3873 bytes Desc: not available URL: From sscotti at sscotti.org Fri May 1 21:52:58 2020 From: sscotti at sscotti.org (sscotti at sscotti.org) Date: Fri, 1 May 2020 16:52:58 -0500 Subject: nginx-devel Digest, Vol 127, Issue 1 In-Reply-To: References: Message-ID: Seems that I can just execute: openresty -V 2>&1 | grep -- 'http_auth_request_module' From the terminal, which gives this, so it must be installed ? configure arguments: --prefix=/usr/local/Cellar/openresty/1.15.8.3_1/nginx --with-cc-opt='-O2 -I/usr/local/include -I/usr/local/opt/pcre/include -I/usr/local/opt/openresty-openssl/include' --add-module=../ngx_devel_kit-0.3.1rc1 --add-module=../echo-nginx-module-0.61 --add-module=../xss-nginx-module-0.06 --add-module=../ngx_coolkit-0.2 --add-module=../set-misc-nginx-module-0.32 --add-module=../form-input-nginx-module-0.12 --add-module=../encrypted-session-nginx-module-0.08 --add-module=../srcache-nginx-module-0.31 --add-module=../ngx_lua-0.10.15 --add-module=../ngx_lua_upstream-0.07 --add-module=../headers-more-nginx-module-0.33 --add-module=../array-var-nginx-module-0.05 --add-module=../memc-nginx-module-0.19 --add-module=../redis2-nginx-module-0.15 --add-module=../redis-nginx-module-0.3.7 --add-module=../ngx_stream_lua-0.0.7 --with-ld-opt='-Wl,-rpath,/usr/local/Cellar/openresty/1.15.8.3_1/luajit/lib -L/usr/local/lib -L/usr/local/opt/pcre/lib -L/usr/local/opt/openresty-openssl/lib' --pid-path=/usr/local/var/run/openresty.pid --lock-path=/usr/local/var/run/openresty.lock --conf-path=/usr/local/etc/openresty/nginx.conf --http-log-path=/usr/local/var/log/nginx/access.log --error-log-path=/usr/local/var/log/nginx/error.log --with-pcre-jit --with-ipv6 --with-stream --with-stream_ssl_module --with-stream_ssl_preread_module --with-http_v2_module --without-mail_pop3_module --without-mail_imap_module --without-mail_smtp_module --with-http_stub_status_module --with-http_realip_module --with-http_addition_module --with-http_auth_request_module --with-http_secure_link_module --with-http_random_index_module --with-http_geoip_module --with-http_gzip_static_module --with-http_sub_module --with-http_dav_module --with-http_flv_module --with-http_mp4_module --with-http_gunzip_module --with-threads --with-dtrace-probes --with-stream --with-stream_ssl_preread_module --with-http_ssl_module ___________________ The content of this email is confidential and intended for the recipient specified in message only. It is strictly forbidden to share any part of this message with any third party, without a written consent of the sender. If you received this message by mistake, please reply to this message and follow with its deletion, so that we can ensure such a mistake does not occur in the future. > > Today's Topics: > > 1. Setting Up a Nginx Server or OpenResty on OS X or UBUNTU with > http_auth_request_module (sscotti at sscotti.org ) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Fri, 1 May 2020 16:31:27 -0500 > From: "sscotti at sscotti.org " > > To: nginx-devel at nginx.org > Subject: Setting Up a Nginx Server or OpenResty on OS X or UBUNTU with > http_auth_request_module > Message-ID: <7489FE86-0F4D-4866-A96E-8A459E1AC742 at sscotti.org > > Content-Type: text/plain; charset="us-ascii" > > I need to set up an NGINX server on OS X or UBUNTU that has the http_auth_request_module installed and enabled. I am kind of a novice when it comes to NGINX, but familiar with Apache, LINUX, etc. There is an NGINX server packaged with my MAMP Pro server on OS X, but I don't think that comes with the http_auth_request_module: > > > > I actually do (I think) want to try using Nginx as a reverse proxy, but I will have to do some research about that also. First, I need to just get a server up and running with that module enabled. I would prefer using OS X because I do most of my development on a Mac, but installing on an UBUNTU VM would also probably work. > > I did try installing OpenResty using the brew package manager on OS X, and if install with "verbose" I get the log for ./configure, which looks like it is compiled with that included. So that might work ? How do I tell if that module is actually enabled ? It does say --with that module (bold). > > brew reinstall openresty/brew/openresty --verbose (or just install) > ____________________ > > Brew compile output: > > > ./configure -j8 --prefix=/usr/local/Cellar/openresty/1.15.8.3_1 --pid-path=/usr/local/var/run/openresty.pid --lock-path=/usr/local/var/run/openresty.lock --conf-path=/usr/local/etc/openresty/nginx.conf --http-log-path=/usr/local/var/log/nginx/access.log --error-log-path=/usr/local/var/log/nginx/error.log --with-cc-opt=-I/usr/local/include -I/usr/local/opt/pcre/include -I/usr/local/opt/openresty-openssl/include --with-ld-opt=-L/usr/local/lib -L/usr/local/opt/pcre/lib -L/usr/local/opt/openresty-openssl/lib --with-pcre-jit --without-http_rds_json_module --without-http_rds_csv_module --without-lua_rds_parser --with-ipv6 --with-stream --with-stream_ssl_module --with-stream_ssl_preread_module --with-http_v2_module --without-mail_pop3_module --without-mail_imap_module --without-mail_smtp_module --with-http_stub_status_module --with-http_realip_module --with-http_addition_module --with-http_auth_request_module --with-http_secure_link_module --with-http_random_index_module --with-http_geoip_ > module --with-http_gzip_static_module --with-http_sub_module --with-http_dav_module --with-http_flv_module --with-http_mp4_module --with-http_gunzip_module --with-threads --with-luajit-xcflags=-DLUAJIT_NUMMODE=2 -DLUAJIT_ENABLE_LUA52COMPAT -fno-stack-check --with-dtrace-probes > platform: macosx (darwin) > > . . . . > > To have launchd start openresty/brew/openresty now and restart at login: > brew services start openresty/brew/openresty > Or, if you don't want/need a background service you can just run: > openresty > > ___________________ > > The config file for openresty is here: /usr/local/etc/openresty/nginx.conf, set to listen on 81, http, and that works, I get the welcome page. > > Welcome to OpenResty! > > > So I guess my question is how to I know that the http_auth_request_module is actually there, and then how to go about setting up as a reverse proxy. > > I know that part is probably dependent on my setup. If someone can help with that it would be greatly appreciated, although it seems like it is installed on OS X. > > > E-mails: sscotti at sscotti.org , sscotti at icloud.com > > > > -------------- next part -------------- > An HTML attachment was scrubbed... > URL: > > -------------- next part -------------- > A non-text attachment was scrubbed... > Name: PastedGraphic-1.png > Type: image/png > Size: 64093 bytes > Desc: not available > URL: > > -------------- next part -------------- > A non-text attachment was scrubbed... > Name: smime.p7s > Type: application/pkcs7-signature > Size: 3873 bytes > Desc: not available > URL: > > > ------------------------------ > > Subject: Digest Footer > > _______________________________________________ > nginx-devel mailing list > nginx-devel at nginx.org > http://mailman.nginx.org/mailman/listinfo/nginx-devel > > ------------------------------ > > End of nginx-devel Digest, Vol 127, Issue 1 > ******************************************* -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 3873 bytes Desc: not available URL: From karthik.uthamann at gmail.com Fri May 1 22:05:05 2020 From: karthik.uthamann at gmail.com (Karthik Uthaman) Date: Fri, 1 May 2020 15:05:05 -0700 Subject: [PATCH] Follow RFC-7232 while evaluating the precedence of conditional headers Message-ID: Nginx evaluates the conditional headers in the following order if_unmodified_since if_match if_modified_since if_none_match which ignores some of the subtleties mentioned in RFC-7232, which gives more precedence to etag based validation over time. https://tools.ietf.org/html/rfc7232#section-3.4 A recipient MUST ignore If-Unmodified-Since if the request contains an If-Match header field; the condition in If-Match is considered to be a more accurate replacement for the condition in If-Unmodified-Since, and the two are only combined for the sake of interoperating with older intermediaries that might not implement If-Match. https://tools.ietf.org/html/rfc7232#section-3.3 and A recipient MUST ignore If-Modified-Since if the request contains an If-None-Match header field; the condition in If-None-Match is considered to be a more accurate replacement for the condition in If-Modified-Since, and the two are only combined for the sake of interoperating with older intermediaries that might not implement If-None-Match. Along with those I have followed the precedence guidance provided in https://tools.ietf.org/html/rfc7232#section-6 to change the order of conditional headers being evaluated in nginx. Please let me know if the changes in the patch file looks good. Happy to address any concerns you may have. - Karthik Uthaman -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: nginx.patch Type: application/octet-stream Size: 3387 bytes Desc: not available URL: From pdn at cryptopro.ru Sat May 2 19:53:58 2020 From: pdn at cryptopro.ru (=?koi8-r?B?8Mne1czJziDkzcnU0snKIO7Jy8/MwcXXyd4=?=) Date: Sat, 2 May 2020 19:53:58 +0000 Subject: [PATCH] Support loading server certificate from HW token In-Reply-To: <170259234.25242098.1588247754550.JavaMail.zimbra@redhat.com> References: <1045205432.24749495.1587988397159.JavaMail.zimbra@redhat.com>, <170259234.25242098.1588247754550.JavaMail.zimbra@redhat.com> Message-ID: <8058a696e8a946549c7516b66fbf9bc2@cryptopro.ru> I support the base idea to load certificates from engines but CMD_LOAD_CERT_CTRL ('LOAD_CERT_CTRL') seems not defined in openssl, it is an engine specific functionality. Is that the only way? And secondly, i can not imagine that you can not get a certificate from your hardware prior nginx run, because the certificate information is always open. So this new functionality is not a must, but a useful one. From poczta at krzysztofgrzadziel.pl Mon May 4 00:14:26 2020 From: poczta at krzysztofgrzadziel.pl (Krzysztof Grzadziel) Date: Mon, 04 May 2020 02:14:26 +0200 Subject: [PATCH] Cache: new options for set path and file permissions (ticket: #698) Message-ID: # HG changeset patch # User Krzysztof Grz?dziel # Date 1588474577 -7200 # Sun May 03 04:56:17 2020 +0200 # Node ID ae1a050faa5c718dc81928dced93337f1e64a4e6 # Parent 716eddd74bc2831537f5b3f7ecd16ad3e516d043 Cache: new options for set path and file permissions (ticket: #698) Patch introduces two optional parameters for *_cache_path: - path_access_rights which sets chmod for directories under cache path - file_access_rights which sets chmod for files under cache path diff -r 716eddd74bc2 -r ae1a050faa5c src/core/ngx_string.c --- a/src/core/ngx_string.c Thu Apr 23 15:10:26 2020 +0300 +++ b/src/core/ngx_string.c Sun May 03 04:56:17 2020 +0200 @@ -1120,6 +1120,32 @@ } +/* parse octal number string representation to integer */ +ngx_int_t +ngx_octtoi(u_char *line, size_t n) +{ + ngx_int_t value; + + if (n == 0) { + return NGX_ERROR; + } + + for (value = 0; n--; line++) { + if (*line < '0' || *line > '7') { + return NGX_ERROR; + } + + value = value * 8 + (*line - '0'); + } + + if (value < 0) { + return NGX_ERROR; + } + + return value; +} + + u_char * ngx_hex_dump(u_char *dst, u_char *src, size_t len) { diff -r 716eddd74bc2 -r ae1a050faa5c src/core/ngx_string.h --- a/src/core/ngx_string.h Thu Apr 23 15:10:26 2020 +0300 +++ b/src/core/ngx_string.h Sun May 03 04:56:17 2020 +0200 @@ -179,6 +179,7 @@ off_t ngx_atoof(u_char *line, size_t n); time_t ngx_atotm(u_char *line, size_t n); ngx_int_t ngx_hextoi(u_char *line, size_t n); +ngx_int_t ngx_octtoi(u_char *line, size_t n); u_char *ngx_hex_dump(u_char *dst, u_char *src, size_t len); diff -r 716eddd74bc2 -r ae1a050faa5c src/http/ngx_http_cache.h --- a/src/http/ngx_http_cache.h Thu Apr 23 15:10:26 2020 +0300 +++ b/src/http/ngx_http_cache.h Sun May 03 04:56:17 2020 +0200 @@ -177,6 +177,9 @@ ngx_msec_t manager_sleep; ngx_msec_t manager_threshold; + ngx_int_t file_access_rights; + ngx_int_t path_access_rights; + ngx_shm_zone_t *shm_zone; ngx_uint_t use_temp_path; diff -r 716eddd74bc2 -r ae1a050faa5c src/http/ngx_http_file_cache.c --- a/src/http/ngx_http_file_cache.c Thu Apr 23 15:10:26 2020 +0300 +++ b/src/http/ngx_http_file_cache.c Sun May 03 04:56:17 2020 +0200 @@ -1384,8 +1384,8 @@ "http file cache rename: \"%s\" to \"%s\"", tf->file.name.data, c->file.name.data); - ext.access = NGX_FILE_OWNER_ACCESS; - ext.path_access = NGX_FILE_OWNER_ACCESS; + ext.access = cache->file_access_rights; + ext.path_access = cache->path_access_rights; ext.time = -1; ext.create_path = 1; ext.delete_file = 1; @@ -2313,6 +2313,7 @@ ngx_msec_t loader_sleep, manager_sleep, loader_threshold, manager_threshold; ngx_uint_t i, n, use_temp_path; + ngx_int_t file_access_rights, path_access_rights; ngx_array_t *caches; ngx_http_file_cache_t *cache, **ce; @@ -2333,6 +2334,8 @@ loader_files = 100; loader_sleep = 50; loader_threshold = 200; + file_access_rights = NGX_FILE_OWNER_ACCESS; + path_access_rights = NGX_FILE_OWNER_ACCESS; manager_files = 100; manager_sleep = 50; @@ -2488,6 +2491,34 @@ continue; } + if (ngx_strncmp(value[i].data, "file_access_rights=", 19) == 0) { + + file_access_rights = ngx_octtoi(value[i].data + 19, + value[i].len - 19); + if (file_access_rights == NGX_ERROR) { + ngx_conf_log_error(NGX_LOG_EMERG, cf, 0, + "invalid file_access_rights value \"%V\"", + &value[i]); + return NGX_CONF_ERROR; + } + + continue; + } + + if (ngx_strncmp(value[i].data, "path_access_rights=", 19) == 0) { + + path_access_rights = ngx_octtoi(value[i].data + 19, + value[i].len - 19); + if (path_access_rights == NGX_ERROR) { + ngx_conf_log_error(NGX_LOG_EMERG, cf, 0, + "invalid path_access_rights value \"%V\"", + &value[i]); + return NGX_CONF_ERROR; + } + + continue; + } + if (ngx_strncmp(value[i].data, "loader_sleep=", 13) == 0) { s.len = value[i].len - 13; @@ -2580,6 +2611,8 @@ cache->loader_files = loader_files; cache->loader_sleep = loader_sleep; cache->loader_threshold = loader_threshold; + cache->file_access_rights = file_access_rights; + cache->path_access_rights = path_access_rights; cache->manager_files = manager_files; cache->manager_sleep = manager_sleep; cache->manager_threshold = manager_threshold; From luhliari at redhat.com Mon May 4 12:42:36 2020 From: luhliari at redhat.com (Lubos Uhliarik) Date: Mon, 4 May 2020 08:42:36 -0400 (EDT) Subject: [PATCH] Support loading server certificate from HW token In-Reply-To: <8058a696e8a946549c7516b66fbf9bc2@cryptopro.ru> References: <1045205432.24749495.1587988397159.JavaMail.zimbra@redhat.com> <170259234.25242098.1588247754550.JavaMail.zimbra@redhat.com> <8058a696e8a946549c7516b66fbf9bc2@cryptopro.ru> Message-ID: <1942859015.25708027.1588596156868.JavaMail.zimbra@redhat.com> Hi ??????????, you are right, this is not defined in openssl, it is an engine specific functionality as you wrote. p11-kit engine supports this command (https://github.com/OpenSC/libp11). I'm not aware of any other method, how to load it. As you stated, it is not a must, but since nginx is already able to load private key from engine, it would be nice to have opportunity to load public key (certificate) as well. In case engine doesn't support that command CMD_LOAD_CERT_CTRL, we can just return error. It would be good to check if the engine supports that command in following way: const char *cmd_name = "LOAD_CERT_CTRL"; if (!ENGINE_ctrl(conn->engine, ENGINE_CTRL_GET_CMD_FROM_NAME, 0, (void *)cmd_name, NULL)) I can alter the patch if it is the only problem you can see here. Best, ----- Original Message ----- > From: "??????? ??????? ??????????" > To: nginx-devel at nginx.org > Sent: Saturday, May 2, 2020 9:53:58 PM > Subject: Re: [PATCH] Support loading server certificate from HW token > > I support the base idea to load certificates from engines but > CMD_LOAD_CERT_CTRL ('LOAD_CERT_CTRL') seems not defined in openssl, it is an > engine specific functionality. Is that the only way? > > And secondly, i can not imagine that you can not get a certificate from your > hardware prior nginx run, because the certificate information is always > open. So this new functionality is not a must, but a useful one. > _______________________________________________ > nginx-devel mailing list > nginx-devel at nginx.org > http://mailman.nginx.org/mailman/listinfo/nginx-devel > > -- Lubos Uhliarik Software Engineer - EMEA ENG Developer Experience RH - Brno - TPB-C - 1D221 IRC: zero_byte at irc.freenode.net RED HAT | TRIED. TESTED. TRUSTED. Every airline in the Fortune 500 relies on Red Hat. Find out why at http://www.redhat.com/en/about/trusted Red Hat Inc. http://cz.redhat.com From aladjev.andrew at gmail.com Wed May 6 11:00:28 2020 From: aladjev.andrew at gmail.com (=?UTF-8?B?0JDQvdC00YDQtdC5INCQ0LvQsNC00YzQtdCy?=) Date: Wed, 6 May 2020 14:00:28 +0300 Subject: HTTP 1.1 parser whitespace in header value Message-ID: Hello. I've read nginx HTTP parser and found the following line: /* header value */ case sw_value: switch (ch) { case ' ': r->header_end = p; state = sw_space_after_value; break; It means that nginx HTTP parser will finish header value immediately after whitespace. But RFC 7230 declares the following: header-field = field-name ":" OWS field-value OWS field-value = *( field-content / obs-fold ) field-content = field-vchar [ 1*( SP / HTAB ) field-vchar ] field-vchar = VCHAR / obs-text "field-vchar [ 1*( SP / HTAB ) field-vchar ]" I think this ABNF means header value can have SP or HTAB between visible chars. So it looks like nginx HTTP parser is broken. What do you think about it? Thank you. -------------- next part -------------- An HTML attachment was scrubbed... URL: From mstavrev at gmail.com Wed May 6 12:45:01 2020 From: mstavrev at gmail.com (Marin Stavrev) Date: Wed, 6 May 2020 15:45:01 +0300 Subject: HTTP 1.1 parser whitespace in header value In-Reply-To: References: Message-ID: I had raised similar problem on December, 2019 - http://mailman.nginx.org/pipermail/nginx-devel/2020-January/012942.html but it had been put down as no other complains had been filed about it. On Wed, May 6, 2020, 14:00 ?????? ??????? wrote: > Hello. I've read nginx HTTP parser and found the following line: > > /* header value */ > case sw_value: > switch (ch) { > case ' ': > r->header_end = p; > state = sw_space_after_value; > break; > > It means that nginx HTTP parser will finish header value immediately after > whitespace. But RFC 7230 declares the following: > > header-field = field-name ":" OWS field-value OWS > field-value = *( field-content / obs-fold ) > field-content = field-vchar [ 1*( SP / HTAB ) field-vchar ] > field-vchar = VCHAR / obs-text > > "field-vchar [ 1*( SP / HTAB ) field-vchar ]" I think this ABNF means > header value can have SP or HTAB between visible chars. > > So it looks like nginx HTTP parser is broken. What do you think about it? > Thank you. > _______________________________________________ > nginx-devel mailing list > nginx-devel at nginx.org > http://mailman.nginx.org/mailman/listinfo/nginx-devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From adam.bambuch2 at gmail.com Wed May 6 23:40:54 2020 From: adam.bambuch2 at gmail.com (Adam Bambuch) Date: Thu, 7 May 2020 01:40:54 +0200 Subject: [PATCH] introduce new variable fs_watermark for proxy_cache_path Message-ID: # HG changeset patch # User Adam Bambuch # Date 1588808163 -7200 # Thu May 07 01:36:03 2020 +0200 # Node ID 8d054b64f07457cad2b74376d5f88162c887ba35 # Parent 716eddd74bc2831537f5b3f7ecd16ad3e516d043 introduce new variable fs_watermark for proxy_cache_path This configuration parameter should help with better disk usage, especially in environments, where nginx doesn't have a separate partition for caching and disk space is shared between multiple apps. It could also help in environments, where cache contains many small files, and cache loader can't load all the files in a reasonable time and therefore max_size is exceeded for many hours after nginx restart. diff -r 716eddd74bc2 -r 8d054b64f074 src/http/ngx_http_cache.h --- a/src/http/ngx_http_cache.h Thu Apr 23 15:10:26 2020 +0300 +++ b/src/http/ngx_http_cache.h Thu May 07 01:36:03 2020 +0200 @@ -160,6 +160,7 @@ ngx_path_t *path; + off_t fs_watermark; off_t max_size; size_t bsize; diff -r 716eddd74bc2 -r 8d054b64f074 src/http/ngx_http_file_cache.c --- a/src/http/ngx_http_file_cache.c Thu Apr 23 15:10:26 2020 +0300 +++ b/src/http/ngx_http_file_cache.c Thu May 07 01:36:03 2020 +0200 @@ -1959,7 +1959,7 @@ { ngx_http_file_cache_t *cache = data; - off_t size; + off_t fs_available, size; time_t wait; ngx_msec_t elapsed, next; ngx_uint_t count, watermark; @@ -1983,11 +1983,22 @@ ngx_shmtx_unlock(&cache->shpool->mutex); - ngx_log_debug3(NGX_LOG_DEBUG_HTTP, ngx_cycle->log, 0, - "http file cache size: %O c:%ui w:%i", - size, count, (ngx_int_t) watermark); - - if (size < cache->max_size && count < watermark) { + fs_available = NGX_MAX_OFF_T_VALUE; + if (cache->fs_watermark) { + fs_available = ngx_fs_available(cache->path->name.data); + if (fs_available == NGX_ERROR) { + ngx_log_error(NGX_LOG_CRIT, ngx_cycle->log, ngx_errno, + ngx_fs_available_n " \"%s\" failed", cache->path->name.data); + } + } + + ngx_log_debug4(NGX_LOG_DEBUG_HTTP, ngx_cycle->log, 0, + "http file cache size: %O c:%ui w:%i fa:%O", + size, count, (ngx_int_t) watermark, fs_available); + + if (size < cache->max_size && + count < watermark && + fs_available >= cache->fs_watermark) { break; } @@ -2304,7 +2315,7 @@ { char *confp = conf; - off_t max_size; + off_t fs_watermark, max_size; u_char *last, *p; time_t inactive; ssize_t size; @@ -2340,6 +2351,7 @@ name.len = 0; size = 0; + fs_watermark = 0; max_size = NGX_MAX_OFF_T_VALUE; value = cf->args->elts; @@ -2461,6 +2473,26 @@ continue; } + #if (NGX_WIN32 || NGX_HAVE_STATFS || NGX_HAVE_STATVFS) + + if (ngx_strncmp(value[i].data, "fs_watermark=", 13) == 0) { + + s.len = value[i].len - 13; + s.data = value[i].data + 13; + + fs_watermark = ngx_parse_offset(&s); + if (fs_watermark < 0) { + ngx_conf_log_error(NGX_LOG_EMERG, cf, 0, + "invalid fs_watermark value \"%V\"", + &value[i]); + return NGX_CONF_ERROR; + } + + continue; + } + + #endif + if (ngx_strncmp(value[i].data, "max_size=", 9) == 0) { s.len = value[i].len - 9; @@ -2606,6 +2638,7 @@ cache->use_temp_path = use_temp_path; cache->inactive = inactive; + cache->fs_watermark = fs_watermark; cache->max_size = max_size; caches = (ngx_array_t *) (confp + cmd->offset); diff -r 716eddd74bc2 -r 8d054b64f074 src/os/unix/ngx_files.c --- a/src/os/unix/ngx_files.c Thu Apr 23 15:10:26 2020 +0300 +++ b/src/os/unix/ngx_files.c Thu May 07 01:36:03 2020 +0200 @@ -878,6 +878,18 @@ return (size_t) fs.f_bsize; } +off_t +ngx_fs_available(u_char *name) +{ + struct statfs fs; + + if (statfs((char *) name, &fs) == -1) { + return NGX_ERROR; + } + + return (ssize_t) (fs.f_bavail * fs.f_bsize); +} + #elif (NGX_HAVE_STATVFS) size_t @@ -896,6 +908,18 @@ return (size_t) fs.f_frsize; } +off_t +ngx_fs_available(u_char *name) +{ + struct statvfs fs; + + if (statvfs((char *) name, &fs) == -1) { + return NGX_ERROR; + } + + return (ssize_t) (fs.f_bavail * fs.f_bsize); +} + #else size_t @@ -904,4 +928,10 @@ return 512; } +off_t +ngx_fs_available(u_char *name) +{ + return NGX_ERROR; +} + #endif diff -r 716eddd74bc2 -r 8d054b64f074 src/os/unix/ngx_files.h --- a/src/os/unix/ngx_files.h Thu Apr 23 15:10:26 2020 +0300 +++ b/src/os/unix/ngx_files.h Thu May 07 01:36:03 2020 +0200 @@ -347,6 +347,23 @@ size_t ngx_fs_bsize(u_char *name); +off_t ngx_fs_available(u_char *name); + + +#if (NGX_HAVE_STATFS) + +#define ngx_fs_available_n "statfs()" + +#elif (NGX_HAVE_STATVFS) + +#define ngx_fs_available_n "statvfs()" + +#else + +#define ngx_fs_available_n "ngx_fs_available()" + +#endif + #if (NGX_HAVE_OPENAT) diff -r 716eddd74bc2 -r 8d054b64f074 src/os/win32/ngx_files.c --- a/src/os/win32/ngx_files.c Thu Apr 23 15:10:26 2020 +0300 +++ b/src/os/win32/ngx_files.c Thu May 07 01:36:03 2020 +0200 @@ -657,6 +657,18 @@ return sc * bs; } +off_t +ngx_fs_available(u_char *name) +{ + ULARGE_INTEGER navail, ntotal, nfree; + + if (GetDiskFreeSpaceEx((const char *) name, &navail, &ntotal, &nfree) == 0) { + return NGX_ERROR; + } + + return (off_t) navail.QuadPart; +} + static ngx_int_t ngx_win32_check_filename(u_char *name, u_short *u, size_t len) diff -r 716eddd74bc2 -r 8d054b64f074 src/os/win32/ngx_files.h --- a/src/os/win32/ngx_files.h Thu Apr 23 15:10:26 2020 +0300 +++ b/src/os/win32/ngx_files.h Thu May 07 01:36:03 2020 +0200 @@ -260,6 +260,9 @@ size_t ngx_fs_bsize(u_char *name); +off_t ngx_fs_available(u_char *name); +#define ngx_fs_available_n "GetDiskFreeSpaceEx" + #define ngx_stdout GetStdHandle(STD_OUTPUT_HANDLE) #define ngx_stderr GetStdHandle(STD_ERROR_HANDLE) From xeioex at nginx.com Thu May 7 13:52:12 2020 From: xeioex at nginx.com (Dmitry Volyntsev) Date: Thu, 07 May 2020 13:52:12 +0000 Subject: [njs] Modules: added typescript API description. Message-ID: details: https://hg.nginx.org/njs/rev/8f3ef384f69e branches: changeset: 1386:8f3ef384f69e user: Dmitry Volyntsev date: Thu May 07 13:49:55 2020 +0000 description: Modules: added typescript API description. This closes #177 issue on Github. diffstat: auto/make | 7 +- nginx/ts/ngx_http_js_module.d.ts | 280 +++++++++++++++++++++++++++++++++++++ nginx/ts/ngx_stream_js_module.d.ts | 102 +++++++++++++ src/ts/njs_core.d.ts | 63 ++++++++ src/ts/njs_shell.d.ts | 9 + 5 files changed, 460 insertions(+), 1 deletions(-) diffs (494 lines): diff -r d47d99d15d2b -r 8f3ef384f69e auto/make --- a/auto/make Thu Apr 30 10:11:33 2020 +0000 +++ b/auto/make Thu May 07 13:49:55 2020 +0000 @@ -231,7 +231,7 @@ cat << END >> $NJS_MAKEFILE @exit 1 all: $NJS_BUILD_DIR/njs_auto_config.h \\ - $NJS_DEFAULT_TARGET test lib_test benchmark + $NJS_DEFAULT_TARGET ts test lib_test benchmark njs: $NJS_BUILD_DIR/njs_auto_config.h $NJS_BUILD_DIR/njs njs_fuzzer: $NJS_BUILD_DIR/njs_auto_config.h \\ @@ -262,6 +262,11 @@ benchmark: $NJS_BUILD_DIR/njs_auto_confi $NJS_BUILD_DIR/njs_benchmark +ts: + mkdir -p $NJS_BUILD_DIR/ts + cp nginx/ts/*.ts $NJS_BUILD_DIR/ts/ + cp src/ts/*.ts $NJS_BUILD_DIR/ts/ + dist: NJS_VER=`grep NJS_VERSION src/njs.h | sed -e 's/.*"\(.*\)".*/\1/'`; \\ rm -rf njs-\$\${NJS_VER} \\ diff -r d47d99d15d2b -r 8f3ef384f69e nginx/ts/ngx_http_js_module.d.ts --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/nginx/ts/ngx_http_js_module.d.ts Thu May 07 13:49:55 2020 +0000 @@ -0,0 +1,280 @@ +/// + +interface NginxHTTPArgs { + readonly [prop: string]: NjsByteString; +} + +interface NginxHeadersIn { + // common request headers + readonly 'accept'?: NjsByteString; + readonly 'accept-charset'?: NjsByteString; + readonly 'accept-encoding'?: NjsByteString; + readonly 'accept-language'?: NjsByteString; + readonly 'authorization'?: NjsByteString; + readonly 'cache-control'?: NjsByteString; + readonly 'connection'?: NjsByteString; + readonly 'content-length'?: NjsByteString; + readonly 'content-type'?: NjsByteString; + readonly 'cookie'?: NjsByteString; + readonly 'date'?: NjsByteString; + readonly 'expect'?: NjsByteString; + readonly 'forwarded'?: NjsByteString; + readonly 'from'?: NjsByteString; + readonly 'host'?: NjsByteString; + readonly 'if-match'?: NjsByteString; + readonly 'if-modified-since'?: NjsByteString; + readonly 'if-none-match'?: NjsByteString; + readonly 'if-range'?: NjsByteString; + readonly 'if-unmodified-since'?: NjsByteString; + readonly 'max-forwards'?: NjsByteString; + readonly 'origin'?: NjsByteString; + readonly 'pragma'?: NjsByteString; + readonly 'proxy-authorization'?: NjsByteString; + readonly 'range'?: NjsByteString; + readonly 'referer'?: NjsByteString; + readonly 'te'?: NjsByteString; + readonly 'user-agent'?: NjsByteString; + readonly 'upgrade'?: NjsByteString; + readonly 'via'?: NjsByteString; + readonly 'warning'?: NjsByteString; + readonly 'x-forwarded-for'?: NjsByteString; + + readonly [prop: string]: NjsByteString; +} + +interface NginxHeadersOut { + // common response headers + 'age'?: NjsStringLike; + 'allow'?: NjsStringLike; + 'alt-svc'?: NjsStringLike; + 'cache-control'?: NjsStringLike; + 'connection'?: NjsStringLike; + 'content-disposition'?: NjsStringLike; + 'content-encoding'?: NjsStringLike; + 'content-language'?: NjsStringLike; + 'content-length'?: NjsStringLike; + 'content-location'?: NjsStringLike; + 'content-range'?: NjsStringLike; + 'content-type'?: NjsStringLike; + 'date'?: NjsStringLike; + 'etag'?: NjsStringLike; + 'expires'?: NjsStringLike; + 'last-modified'?: NjsStringLike; + 'link'?: NjsStringLike; + 'location'?: NjsStringLike; + 'pragma'?: NjsStringLike; + 'proxy-authenticate'?: NjsStringLike; + 'retry-after'?: NjsStringLike; + 'server'?: NjsStringLike; + 'trailer'?: NjsStringLike; + 'transfer-encoding'?: NjsStringLike; + 'upgrade'?: NjsStringLike; + 'vary'?: NjsStringLike; + 'via'?: NjsStringLike; + 'warning'?: NjsStringLike; + 'www-authenticate'?: NjsStringLike; + + 'set-cookie'?: NjsStringLike[]; + + [prop: string]: NjsStringLike | NjsStringLike[]; +} + +interface NginxVariables { + readonly 'ancient_browser'?: NjsByteString; + readonly 'arg_'?: NjsByteString; + readonly 'args'?: NjsByteString; + readonly 'binary_remote_addr'?: NjsByteString; + readonly 'body_bytes_sent'?: NjsByteString; + readonly 'bytes_received'?: NjsByteString; + readonly 'bytes_sent'?: NjsByteString; + readonly 'connection'?: NjsByteString; + readonly 'connection_requests'?: NjsByteString; + readonly 'connections_active'?: NjsByteString; + readonly 'connections_reading'?: NjsByteString; + readonly 'connections_waiting'?: NjsByteString; + readonly 'connections_writing'?: NjsByteString; + readonly 'content_length'?: NjsByteString; + readonly 'content_type'?: NjsByteString; + readonly 'cookie_'?: NjsByteString; + readonly 'date_gmt'?: NjsByteString; + readonly 'date_local'?: NjsByteString; + readonly 'document_root'?: NjsByteString; + readonly 'document_uri'?: NjsByteString; + readonly 'fastcgi_path_info'?: NjsByteString; + readonly 'fastcgi_script_name'?: NjsByteString; + readonly 'geoip_area_code'?: NjsByteString; + readonly 'geoip_city'?: NjsByteString; + readonly 'geoip_city_continent_code'?: NjsByteString; + readonly 'geoip_city_country_code'?: NjsByteString; + readonly 'geoip_city_country_code3'?: NjsByteString; + readonly 'geoip_city_country_name'?: NjsByteString; + readonly 'geoip_country_code'?: NjsByteString; + readonly 'geoip_country_code3'?: NjsByteString; + readonly 'geoip_country_name'?: NjsByteString; + readonly 'geoip_dma_code'?: NjsByteString; + readonly 'geoip_latitude'?: NjsByteString; + readonly 'geoip_longitude'?: NjsByteString; + readonly 'geoip_org'?: NjsByteString; + readonly 'geoip_postal_code'?: NjsByteString; + readonly 'geoip_region'?: NjsByteString; + readonly 'geoip_region_name'?: NjsByteString; + readonly 'gzip_ratio'?: NjsByteString; + readonly 'host'?: NjsByteString; + readonly 'hostname'?: NjsByteString; + readonly 'http2'?: NjsByteString; + readonly 'http_'?: NjsByteString; + readonly 'https'?: NjsByteString; + readonly 'invalid_referer'?: NjsByteString; + readonly 'is_args'?: NjsByteString; + readonly 'jwt_claim_'?: NjsByteString; + readonly 'jwt_header_'?: NjsByteString; + readonly 'limit_conn_status'?: NjsByteString; + readonly 'limit_rate'?: NjsByteString; + readonly 'limit_req_status'?: NjsByteString; + readonly 'memcached_key'?: NjsByteString; + readonly 'modern_browser'?: NjsByteString; + readonly 'msec'?: NjsByteString; + readonly 'msie'?: NjsByteString; + readonly 'nginx_version'?: NjsByteString; + readonly 'pid'?: NjsByteString; + readonly 'pipe'?: NjsByteString; + readonly 'protocol'?: NjsByteString; + readonly 'proxy_add_x_forwarded_for'?: NjsByteString; + readonly 'proxy_host'?: NjsByteString; + readonly 'proxy_port'?: NjsByteString; + readonly 'proxy_protocol_addr'?: NjsByteString; + readonly 'proxy_protocol_port'?: NjsByteString; + readonly 'proxy_protocol_server_addr'?: NjsByteString; + readonly 'proxy_protocol_server_port'?: NjsByteString; + readonly 'query_string'?: NjsByteString; + readonly 'realip_remote_addr'?: NjsByteString; + readonly 'realip_remote_port'?: NjsByteString; + readonly 'realpath_root'?: NjsByteString; + readonly 'remote_addr'?: NjsByteString; + readonly 'remote_port'?: NjsByteString; + readonly 'remote_user'?: NjsByteString; + readonly 'request'?: NjsByteString; + readonly 'request_body'?: NjsByteString; + readonly 'request_body_file'?: NjsByteString; + readonly 'request_completion'?: NjsByteString; + readonly 'request_filename'?: NjsByteString; + readonly 'request_id'?: NjsByteString; + readonly 'request_length'?: NjsByteString; + readonly 'request_method'?: NjsByteString; + readonly 'request_time'?: NjsByteString; + readonly 'request_uri'?: NjsByteString; + readonly 'scheme'?: NjsByteString; + readonly 'secure_link'?: NjsByteString; + readonly 'secure_link_expires'?: NjsByteString; + readonly 'sent_http_'?: NjsByteString; + readonly 'sent_trailer_'?: NjsByteString; + readonly 'server_addr'?: NjsByteString; + readonly 'server_name'?: NjsByteString; + readonly 'server_port'?: NjsByteString; + readonly 'server_protocol'?: NjsByteString; + readonly 'session_log_binary_id'?: NjsByteString; + readonly 'session_log_id'?: NjsByteString; + readonly 'session_time'?: NjsByteString; + readonly 'slice_range'?: NjsByteString; + readonly 'spdy'?: NjsByteString; + readonly 'spdy_request_priority'?: NjsByteString; + readonly 'ssl_cipher'?: NjsByteString; + readonly 'ssl_ciphers'?: NjsByteString; + readonly 'ssl_client_cert'?: NjsByteString; + readonly 'ssl_client_escaped_cert'?: NjsByteString; + readonly 'ssl_client_fingerprint'?: NjsByteString; + readonly 'ssl_client_i_dn'?: NjsByteString; + readonly 'ssl_client_i_dn_legacy'?: NjsByteString; + readonly 'ssl_client_raw_cert'?: NjsByteString; + readonly 'ssl_client_s_dn'?: NjsByteString; + readonly 'ssl_client_s_dn_legacy'?: NjsByteString; + readonly 'ssl_client_serial'?: NjsByteString; + readonly 'ssl_client_v_end'?: NjsByteString; + readonly 'ssl_client_v_remain'?: NjsByteString; + readonly 'ssl_client_v_start'?: NjsByteString; + readonly 'ssl_client_verify'?: NjsByteString; + readonly 'ssl_curves'?: NjsByteString; + readonly 'ssl_early_data'?: NjsByteString; + readonly 'ssl_preread_alpn_protocols'?: NjsByteString; + readonly 'ssl_preread_protocol'?: NjsByteString; + readonly 'ssl_preread_server_name'?: NjsByteString; + readonly 'ssl_protocol'?: NjsByteString; + readonly 'ssl_server_name'?: NjsByteString; + readonly 'ssl_session_id'?: NjsByteString; + readonly 'ssl_session_reused'?: NjsByteString; + readonly 'status'?: NjsByteString; + readonly 'tcpinfo_rtt'?: NjsByteString; + readonly 'tcpinfo_rttvar'?: NjsByteString; + readonly 'tcpinfo_snd_cwnd'?: NjsByteString; + readonly 'tcpinfo_rcv_space'?: NjsByteString; + readonly 'time_iso8601'?: NjsByteString; + readonly 'time_local'?: NjsByteString; + readonly 'uid_got'?: NjsByteString; + readonly 'uid_reset'?: NjsByteString; + readonly 'uid_set'?: NjsByteString; + readonly 'upstream_addr'?: NjsByteString; + readonly 'upstream_bytes_received'?: NjsByteString; + readonly 'upstream_bytes_sent'?: NjsByteString; + readonly 'upstream_cache_status'?: NjsByteString; + readonly 'upstream_connect_time'?: NjsByteString; + readonly 'upstream_cookie_'?: NjsByteString; + readonly 'upstream_first_byte_time'?: NjsByteString; + readonly 'upstream_header_time'?: NjsByteString; + readonly 'upstream_http_'?: NjsByteString; + readonly 'upstream_queue_time'?: NjsByteString; + readonly 'upstream_response_length'?: NjsByteString; + readonly 'upstream_response_time'?: NjsByteString; + readonly 'upstream_session_time'?: NjsByteString; + readonly 'upstream_status'?: NjsByteString; + readonly 'upstream_trailer_'?: NjsByteString; + readonly 'uri'?: NjsByteString; + + [prop: string]: NjsStringLike; +} + +interface NginxSubrequestOptions { + method?: "GET" | "POST" | "OPTIONS" | "HEAD" | "PROPFIND" | "PUT" + | "MKCOL" | "DELETE" | "COPY" | "MOVE" | "PROPPATCH" + | "LOCK" | "PATCH" | "TRACE", + args?: NjsStringLike, + body?: NjsStringLike, + detached?: boolean +} + +interface NginxHTTPRequest { + // properties + readonly args: NginxHTTPArgs; + readonly headersIn: NginxHeadersIn; + readonly headersOut: NginxHeadersOut; + readonly httpVersion: NjsByteString; + readonly method: NjsByteString; + readonly parent?: NginxHTTPRequest; + readonly remoteAddress: NjsByteString; + readonly requestBody?: NjsByteString; + readonly responseBody?: NjsByteString; + readonly uri: NjsByteString; + readonly variables: NginxVariables; + + // control + status: number; + sendHeader(): void; + send(part: NjsStringLike): void; + return(status: number, body?: NjsStringLike): void; + internalRedirect(location: NjsStringLike): void; + finish(): void; + + // Promise version + subrequest(uri: NjsStringLike, options?: NginxSubrequestOptions | string): Promise; + // Long callback version + subrequest(uri: NjsStringLike, options: NginxSubrequestOptions | string, + callback:(reply:NginxHTTPRequest) => void): void; + // Short callback version + subrequest(uri: NjsStringLike, callback:(reply:NginxHTTPRequest) => void): void; + // Detached version + subrequest(uri: NjsStringLike, options: NginxSubrequestOptions): void; + + // logging + error(message: NjsStringLike): void; + warn(message: NjsStringLike): void; + log(message: NjsStringLike): void; +} diff -r d47d99d15d2b -r 8f3ef384f69e nginx/ts/ngx_stream_js_module.d.ts --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/nginx/ts/ngx_stream_js_module.d.ts Thu May 07 13:49:55 2020 +0000 @@ -0,0 +1,102 @@ +/// + +interface NginxStreamVariables { + readonly 'binary_remote_addr'?: NjsByteString; + readonly 'bytes_received'?: NjsByteString; + readonly 'bytes_sent'?: NjsByteString; + readonly 'connection'?: NjsByteString; + readonly 'geoip_area_code'?: NjsByteString; + readonly 'geoip_city'?: NjsByteString; + readonly 'geoip_city_continent_code'?: NjsByteString; + readonly 'geoip_city_country_code'?: NjsByteString; + readonly 'geoip_city_country_code3'?: NjsByteString; + readonly 'geoip_city_country_name'?: NjsByteString; + readonly 'geoip_country_code'?: NjsByteString; + readonly 'geoip_country_code3'?: NjsByteString; + readonly 'geoip_country_name'?: NjsByteString; + readonly 'geoip_dma_code'?: NjsByteString; + readonly 'geoip_latitude'?: NjsByteString; + readonly 'geoip_longitude'?: NjsByteString; + readonly 'geoip_org'?: NjsByteString; + readonly 'geoip_postal_code'?: NjsByteString; + readonly 'geoip_region'?: NjsByteString; + readonly 'geoip_region_name'?: NjsByteString; + readonly 'hostname'?: NjsByteString; + readonly 'limit_conn_status'?: NjsByteString; + readonly 'msec'?: NjsByteString; + readonly 'nginx_version'?: NjsByteString; + readonly 'pid'?: NjsByteString; + readonly 'proxy_add_x_forwarded_for'?: NjsByteString; + readonly 'proxy_host'?: NjsByteString; + readonly 'proxy_port'?: NjsByteString; + readonly 'proxy_protocol_addr'?: NjsByteString; + readonly 'proxy_protocol_port'?: NjsByteString; + readonly 'proxy_protocol_server_addr'?: NjsByteString; + readonly 'proxy_protocol_server_port'?: NjsByteString; + readonly 'realip_remote_addr'?: NjsByteString; + readonly 'realip_remote_port'?: NjsByteString; + readonly 'remote_addr'?: NjsByteString; + readonly 'remote_port'?: NjsByteString; + readonly 'server_addr'?: NjsByteString; + readonly 'server_port'?: NjsByteString; + readonly 'ssl_cipher'?: NjsByteString; + readonly 'ssl_ciphers'?: NjsByteString; + readonly 'ssl_client_cert'?: NjsByteString; + readonly 'ssl_client_escaped_cert'?: NjsByteString; + readonly 'ssl_client_fingerprint'?: NjsByteString; + readonly 'ssl_client_i_dn'?: NjsByteString; + readonly 'ssl_client_raw_cert'?: NjsByteString; + readonly 'ssl_client_s_dn'?: NjsByteString; + readonly 'ssl_client_s_dn_legacy'?: NjsByteString; + readonly 'ssl_client_serial'?: NjsByteString; + readonly 'ssl_client_v_end'?: NjsByteString; + readonly 'ssl_client_v_remain'?: NjsByteString; + readonly 'ssl_client_v_start'?: NjsByteString; + readonly 'ssl_client_verify'?: NjsByteString; + readonly 'ssl_curves'?: NjsByteString; + readonly 'ssl_early_data'?: NjsByteString; + readonly 'ssl_preread_alpn_protocols'?: NjsByteString; + readonly 'ssl_preread_protocol'?: NjsByteString; + readonly 'ssl_preread_server_name'?: NjsByteString; + readonly 'ssl_protocol'?: NjsByteString; + readonly 'ssl_server_name'?: NjsByteString; + readonly 'ssl_session_id'?: NjsByteString; + readonly 'ssl_session_reused'?: NjsByteString; + readonly 'status'?: NjsByteString; + readonly 'time_iso8601'?: NjsByteString; + readonly 'time_local'?: NjsByteString; + + [prop: string]: NjsByteString; +} + +interface NginxStreamCallbackFlags { + last: boolean +} + +interface NginxStreamSendOptions { + last?: boolean + flush?: boolean +} + +interface NginxStreamRequest { + // properties + readonly remoteAddress: NjsByteString; + readonly variables: NginxStreamVariables; + + // control + allow(): void; + decline(): void; + deny(): void; + done(code?: number): void; + + on(event: "upload" | "download", + callback:(data:NjsByteString, flags: NginxStreamCallbackFlags) => void): void; + off(event: "upload" | "download"): void; + + send(data: NjsStringLike, options?: NginxStreamSendOptions): void; + + // logging + error(message: NjsStringLike): void; + warn(message: NjsStringLike): void; + log(message: NjsStringLike): void; +} diff -r d47d99d15d2b -r 8f3ef384f69e src/ts/njs_core.d.ts --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/ts/njs_core.d.ts Thu May 07 13:49:55 2020 +0000 @@ -0,0 +1,63 @@ +interface StringConstructor { + /** + * Creates a byte string from an encoded string. + */ + bytesFrom(bytes: string, encoding: "hex" | "base64" | "base64url"): NjsByteString; + /** + * Creates a byte string from an array that contains octets. + */ + bytesFrom(bytes: Array): NjsByteString; +} + +interface String { + /** + * Serializes a Unicode string with code points up to 255 + * into a byte string, otherwise, null is returned. + */ + toBytes(start?: number, end?: number): NjsByteString; + /** + * Serializes a Unicode string to a byte string using UTF8 encoding. + */ + toUTF8(start?: number, end?: number): NjsByteString; +} + +interface NjsByteString extends String { + /** + * Returns a new Unicode string from a byte string where each byte is replaced + * with a corresponding Unicode code point. + */ + fromBytes(start?: number, end?: number): string; + /** + * Converts a byte string containing a valid UTF8 string into a Unicode string, + * otherwise null is returned. + */ + fromUTF8(start?: number, end?: number): string; + /** + * Encodes a byte string to hex, base64, or base64url. + */ + toString(encoding?: "hex" | "base64" | "base64url"): string; +} + +type NjsStringLike = string | NjsByteString; + +// Global objects + +interface NjsGlobal { + readonly version: string; + dump(value: any, indent?: number): string; +} + +declare const njs: NjsGlobal; + +interface NjsEnv { + readonly [prop: string]: NjsByteString; +} + +interface NjsProcess { + readonly pid: number; + readonly ppid: number; + readonly argv: string[]; + readonly env: NjsEnv; +} + +declare const process: NjsProcess; diff -r d47d99d15d2b -r 8f3ef384f69e src/ts/njs_shell.d.ts --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/ts/njs_shell.d.ts Thu May 07 13:49:55 2020 +0000 @@ -0,0 +1,9 @@ +/// + +interface Console { + log(...args: any[]): void; + dump(...args: any[]): void; + + time(label?: NjsStringLike): void; + timeEnd(label?: NjsStringLike): void; +} From pluknet at nginx.com Fri May 8 16:22:47 2020 From: pluknet at nginx.com (Sergey Kandaurov) Date: Fri, 08 May 2020 16:22:47 +0000 Subject: [nginx] Variables: fixed buffer over-read when evaluating "$arg_". Message-ID: details: https://hg.nginx.org/nginx/rev/028b16e2798f branches: changeset: 7648:028b16e2798f user: Sergey Kandaurov date: Fri May 08 19:19:16 2020 +0300 description: Variables: fixed buffer over-read when evaluating "$arg_". diffstat: src/http/ngx_http_variables.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diffs (12 lines): diff -r 716eddd74bc2 -r 028b16e2798f src/http/ngx_http_variables.c --- a/src/http/ngx_http_variables.c Thu Apr 23 15:10:26 2020 +0300 +++ b/src/http/ngx_http_variables.c Fri May 08 19:19:16 2020 +0300 @@ -1075,7 +1075,7 @@ ngx_http_variable_argument(ngx_http_requ len = name->len - (sizeof("arg_") - 1); arg = name->data + sizeof("arg_") - 1; - if (ngx_http_arg(r, arg, len, &value) != NGX_OK) { + if (len == 0 || ngx_http_arg(r, arg, len, &value) != NGX_OK) { v->not_found = 1; return NGX_OK; } From xeioex at nginx.com Fri May 8 16:36:43 2020 From: xeioex at nginx.com (Dmitry Volyntsev) Date: Fri, 08 May 2020 16:36:43 +0000 Subject: [njs] Added description for API methods. Message-ID: details: https://hg.nginx.org/njs/rev/1268384be910 branches: changeset: 1387:1268384be910 user: Dmitry Volyntsev date: Fri May 08 12:02:32 2020 +0000 description: Added description for API methods. diffstat: nginx/ts/ngx_http_js_module.d.ts | 129 ++++++++++++++++++++++++++++++------ nginx/ts/ngx_stream_js_module.d.ts | 70 ++++++++++++++++--- 2 files changed, 166 insertions(+), 33 deletions(-) diffs (255 lines): diff -r 8f3ef384f69e -r 1268384be910 nginx/ts/ngx_http_js_module.d.ts --- a/nginx/ts/ngx_http_js_module.d.ts Thu May 07 13:49:55 2020 +0000 +++ b/nginx/ts/ngx_http_js_module.d.ts Fri May 08 12:02:32 2020 +0000 @@ -233,48 +233,135 @@ interface NginxVariables { } interface NginxSubrequestOptions { + /** + * Arguments string, by default an empty string is used. + */ + args?: NjsStringLike, + /** + * Request body, by default the request body of the parent request object is used. + */ + body?: NjsStringLike, + /** + * HTTP method, by default the GET method is used. + */ method?: "GET" | "POST" | "OPTIONS" | "HEAD" | "PROPFIND" | "PUT" | "MKCOL" | "DELETE" | "COPY" | "MOVE" | "PROPPATCH" | "LOCK" | "PATCH" | "TRACE", - args?: NjsStringLike, - body?: NjsStringLike, + /** + * if true, the created subrequest is a detached subrequest. + * Responses to detached subrequests are ignored. + */ detached?: boolean } interface NginxHTTPRequest { - // properties + /** + * Request arguments object. + */ readonly args: NginxHTTPArgs; + /** + * Writes a string to the error log on the error level of logging. + * @param message Message to log. + */ + error(message: NjsStringLike): void; + /** + * Finishes sending a response to the client. + */ + finish(): void; + /** + * Incoming headers object. + */ readonly headersIn: NginxHeadersIn; + /** + * Outgoing headers object. + */ readonly headersOut: NginxHeadersOut; + /** + * HTTP protocol version. + */ readonly httpVersion: NjsByteString; + /** + * Performs an internal redirect to the specified uri. + * If the uri starts with the ?@? prefix, it is considered a named location. + * The actual redirect happens after the handler execution is completed. + * @param uri Location to redirect to. + */ + internalRedirect(uri: NjsStringLike): void; + /** + * Writes a string to the error log on the info level of logging. + * @param message Message to log. + */ + log(message: NjsStringLike): void; + /** + * HTTP method. + */ readonly method: NjsByteString; + /** + * Parent for subrequest object. + */ readonly parent?: NginxHTTPRequest; + /** + * Client address. + */ readonly remoteAddress: NjsByteString; + /** + * Client request body if it has not been written to a temporary file. + * To ensure that the client request body is in memory, its size should be + * limited by client_max_body_size, and a sufficient buffer size should be set + * using client_body_buffer_size. The property is available only in the js_content directive. + */ readonly requestBody?: NjsByteString; + /** + * Subrequest response body. The size of response body is limited by + * the subrequest_output_buffer_size directive. + */ readonly responseBody?: NjsByteString; - readonly uri: NjsByteString; - readonly variables: NginxVariables; - - // control - status: number; - sendHeader(): void; + /** + * Sends the entire response with the specified status to the client. + * It is possible to specify either a redirect URL (for codes 301, 302, 303, 307, and 308) + * or the response body text (for other codes) as the second argument. + * @param status Respose status code. + * @param body Respose body. + */ + return(status: number, body?: NjsStringLike): void; + /** + * Sends the HTTP headers to the client. + */ send(part: NjsStringLike): void; - return(status: number, body?: NjsStringLike): void; - internalRedirect(location: NjsStringLike): void; - finish(): void; - - // Promise version + /** + * Sends the HTTP headers to the client. + */ + sendHeader(): void; + /** + * Respose status code. + */ + status: number; + /** + * Creates a subrequest with the given uri and options. + * A subrequest shares its input headers with the client request. + * To send headers different from original headers to a proxied server, + * the proxy_set_header directive can be used. To send a completely new + * set of headers to a proxied server, the proxy_pass_request_headers directive can be used. + * @param uri Subrequest location. + * @param options Subrequest options. + * @param callback Completion callback. + */ subrequest(uri: NjsStringLike, options?: NginxSubrequestOptions | string): Promise; - // Long callback version subrequest(uri: NjsStringLike, options: NginxSubrequestOptions | string, callback:(reply:NginxHTTPRequest) => void): void; - // Short callback version subrequest(uri: NjsStringLike, callback:(reply:NginxHTTPRequest) => void): void; - // Detached version subrequest(uri: NjsStringLike, options: NginxSubrequestOptions): void; - - // logging - error(message: NjsStringLike): void; + /** + * Current URI in request, normalized. + */ + readonly uri: NjsByteString; + /** + * nginx variables object. + */ + readonly variables: NginxVariables; + /** + * Writes a string to the error log on the warn level of logging. + * @param message Message to log. + */ warn(message: NjsStringLike): void; - log(message: NjsStringLike): void; } diff -r 8f3ef384f69e -r 1268384be910 nginx/ts/ngx_stream_js_module.d.ts --- a/nginx/ts/ngx_stream_js_module.d.ts Thu May 07 13:49:55 2020 +0000 +++ b/nginx/ts/ngx_stream_js_module.d.ts Fri May 08 12:02:32 2020 +0000 @@ -70,33 +70,79 @@ interface NginxStreamVariables { } interface NginxStreamCallbackFlags { + /** + * True if data is a last buffer. + */ last: boolean } interface NginxStreamSendOptions { + /** + * True if data is a last buffer. + */ last?: boolean + /** + * True if the buffer should have the flush flag. + */ flush?: boolean } interface NginxStreamRequest { - // properties - readonly remoteAddress: NjsByteString; - readonly variables: NginxStreamVariables; - - // control + /** + * Successfully finalizes the phase handler. + */ allow(): void; + /** + * Finalizes the phase handler and passes control to the next handler. + */ decline(): void; + /** + * Finalizes the phase handler with the access error code. + */ deny(): void; + /** + * Successfully finalizes the current phase handler + * or finalizes it with the specified numeric code. + * @param code Finalization code. + */ done(code?: number): void; - + /** + * Writes a string to the error log on the error level of logging. + * @param message Message to log. + */ + error(message: NjsStringLike): void; + /** + * Writes a string to the error log on the info level of logging. + * @param message Message to log. + */ + log(message: NjsStringLike): void; + /** + * Unregisters the callback set by on() method. + */ + off(event: "upload" | "download"): void; + /** + * Registers a callback for the specified event. + */ on(event: "upload" | "download", callback:(data:NjsByteString, flags: NginxStreamCallbackFlags) => void): void; - off(event: "upload" | "download"): void; - + /** + * Client address. + */ + readonly remoteAddress: NjsByteString; + /** + * Sends the data to the client. + * @param data Data to send. + * @param options Object used to override nginx buffer flags derived from + * an incoming data chunk buffer. + */ send(data: NjsStringLike, options?: NginxStreamSendOptions): void; - - // logging - error(message: NjsStringLike): void; + /** + * nginx variables object. + */ + readonly variables: NginxStreamVariables; + /** + * Writes a string to the error log on the warn level of logging. + * @param message Message to log. + */ warn(message: NjsStringLike): void; - log(message: NjsStringLike): void; } From xeioex at nginx.com Fri May 8 16:36:45 2020 From: xeioex at nginx.com (Dmitry Volyntsev) Date: Fri, 08 May 2020 16:36:45 +0000 Subject: [njs] HTTP: introduced raw headers API. Message-ID: details: https://hg.nginx.org/njs/rev/92838d964b19 branches: changeset: 1388:92838d964b19 user: Dmitry Volyntsev date: Fri May 08 15:52:40 2020 +0000 description: HTTP: introduced raw headers API. 1) r.rawHeadersIn returns an array of key-value pairs exactly as they were received from the client. 2) r.rawHeadersOut returns an array of key-value pairs of respose headers. Header names are not lowercased, and duplicates are not merged. diffstat: nginx/ngx_http_js_module.c | 103 +++++++++++++++++++++++++++++++++++++++++++++ 1 files changed, 103 insertions(+), 0 deletions(-) diffs (134 lines): diff -r 1268384be910 -r 92838d964b19 nginx/ngx_http_js_module.c --- a/nginx/ngx_http_js_module.c Fri May 08 12:02:32 2020 +0000 +++ b/nginx/ngx_http_js_module.c Fri May 08 15:52:40 2020 +0000 @@ -84,6 +84,9 @@ static njs_int_t ngx_http_js_ext_keys_he njs_value_t *keys, ngx_list_t *headers); static ngx_table_elt_t *ngx_http_js_get_header(ngx_list_part_t *part, u_char *data, size_t len); +static njs_int_t ngx_http_js_ext_raw_header(njs_vm_t *vm, + njs_object_prop_t *prop, njs_value_t *value, njs_value_t *setval, + njs_value_t *retval); static njs_int_t ngx_http_js_ext_header_out(njs_vm_t *vm, njs_object_prop_t *prop, njs_value_t *value, njs_value_t *setval, njs_value_t *retval); @@ -354,6 +357,15 @@ static njs_external_t ngx_http_js_ext_r }, { + .flags = NJS_EXTERN_PROPERTY, + .name.string = njs_str("rawHeadersIn"), + .u.property = { + .handler = ngx_http_js_ext_raw_header, + .magic32 = 0, + } + }, + + { .flags = NJS_EXTERN_OBJECT, .name.string = njs_str("args"), .enumerable = 1, @@ -397,6 +409,15 @@ static njs_external_t ngx_http_js_ext_r }, { + .flags = NJS_EXTERN_PROPERTY, + .name.string = njs_str("rawHeadersOut"), + .u.property = { + .handler = ngx_http_js_ext_raw_header, + .magic32 = 1, + } + }, + + { .flags = NJS_EXTERN_METHOD, .name.string = njs_str("subrequest"), .writable = 1, @@ -934,6 +955,88 @@ ngx_http_js_get_header(ngx_list_part_t * static njs_int_t +ngx_http_js_ext_raw_header(njs_vm_t *vm, njs_object_prop_t *prop, + njs_value_t *value, njs_value_t *setval, njs_value_t *retval) +{ + njs_int_t rc; + ngx_uint_t i; + njs_value_t *array, *elem; + ngx_list_part_t *part; + ngx_list_t *headers; + ngx_table_elt_t *header, *h; + ngx_http_request_t *r; + + r = njs_vm_external(vm, value); + if (r == NULL) { + njs_value_undefined_set(retval); + return NJS_DECLINED; + } + + headers = (njs_vm_prop_magic32(prop) == 1) ? &r->headers_out.headers + : &r->headers_in.headers; + + rc = njs_vm_array_alloc(vm, retval, 8); + if (rc != NJS_OK) { + return NJS_ERROR; + } + + part = &headers->part; + header = part->elts; + + for (i = 0; /* void */ ; i++) { + + if (i >= part->nelts) { + if (part->next == NULL) { + break; + } + + part = part->next; + header = part->elts; + i = 0; + } + + h = &header[i]; + + if (h->hash == 0) { + continue; + } + + array = njs_vm_array_push(vm, retval); + if (array == NULL) { + return NJS_ERROR; + } + + rc = njs_vm_array_alloc(vm, array, 2); + if (rc != NJS_OK) { + return NJS_ERROR; + } + + elem = njs_vm_array_push(vm, array); + if (elem == NULL) { + return NJS_ERROR; + } + + rc = njs_vm_value_string_set(vm, elem, h->key.data, h->key.len); + if (rc != NJS_OK) { + return NJS_ERROR; + } + + elem = njs_vm_array_push(vm, array); + if (elem == NULL) { + return NJS_ERROR; + } + + rc = njs_vm_value_string_set(vm, elem, h->value.data, h->value.len); + if (rc != NJS_OK) { + return NJS_ERROR; + } + } + + return NJS_OK; +} + + +static njs_int_t ngx_http_js_ext_header_out(njs_vm_t *vm, njs_object_prop_t *prop, njs_value_t *value, njs_value_t *setval, njs_value_t *retval) { From pdn at cryptopro.ru Fri May 8 19:53:18 2020 From: pdn at cryptopro.ru (=?koi8-r?B?8Mne1czJziDkzcnU0snKIO7Jy8/MwcXXyd4=?=) Date: Fri, 8 May 2020 19:53:18 +0000 Subject: [PATCH] Support loading server certificate from HW token In-Reply-To: <1942859015.25708027.1588596156868.JavaMail.zimbra@redhat.com> References: <1045205432.24749495.1587988397159.JavaMail.zimbra@redhat.com> <170259234.25242098.1588247754550.JavaMail.zimbra@redhat.com> <8058a696e8a946549c7516b66fbf9bc2@cryptopro.ru>, <1942859015.25708027.1588596156868.JavaMail.zimbra@redhat.com> Message-ID: I dipped into the problem and came to the conclusion that this proposal cannot be used as a general one. First, although the ctrl number could be passed in the directive itself, for example "engine:pkcs11:205:slot_0-id_00", where 205 corresponds to CMD_LOAD_CERT_CTRL (ENGINE_CMD_BASE + 5 = 200 + 5), the argument "params" is too specific for this command, in fact, it is a binding to a specific non-extensible interface of a particular ENGINE command. Secondly, this binding to a bad interface actually, which is not able to return the certificate chain, CMD_LOAD_CERT_CTRL returns only the leaf certificate. Therefore, I do not see how this can be used outside of pkcs11 ENGINE and I do not see how this can be used in a production without a certificate chain. From thibaultcha at fastmail.com Fri May 8 20:41:08 2020 From: thibaultcha at fastmail.com (Thibault Charbonnier) Date: Fri, 8 May 2020 13:41:08 -0700 Subject: [PATCH] Ensured SIGQUIT deletes listening UNIX socket files. In-Reply-To: <6d8edc45-6133-de52-8220-05c962a4a48c@fastmail.com> References: <4b6bc48b-6c7a-7c76-51ae-1038421ed36c@fastmail.com> <20200227152429.GI12894@mdounin.ru> <2c06ad76-c309-d387-2c40-f6e60aacf4f8@fastmail.com> <20200303142811.GO12894@mdounin.ru> <22c4b64a-eb4e-a8a8-7428-34ab3225fcf5@fastmail.com> <20200426011240.GL20357@mdounin.ru> <6d8edc45-6133-de52-8220-05c962a4a48c@fastmail.com> Message-ID: <7112c3e4-01c7-8fa7-ad8d-aa3934190817@fastmail.com> On 4/27/20 4:26 PM, Thibault Charbonnier wrote: > Great! Thanks for the suggestion. Below is a revised approach for the > patch (also attached to this email) which passes all of the test cases > listed in my previous test file at the start of this thread: Hello, Are there plans for a follow-up on this patch? Thanks. From peter.stadler at student.uibk.ac.at Tue May 12 10:04:58 2020 From: peter.stadler at student.uibk.ac.at (Peter Stadler) Date: Tue, 12 May 2020 12:04:58 +0200 Subject: [RFC] Core: worker_processes <= NGX_MAX_PROCESSES Message-ID: <36bd8093-749f-d9c9-cccf-b85d74f9c683@student.uibk.ac.at> Would it be advisable to limit the number of worker_processes to NGX_MAX_PROCESSES at least when we have `worker_processes auto;` in the config? A corresponding patch could be: diff a/src/core/nginx.c b/src/core/nginx.c --- a/src/core/nginx.c +++ b/src/core/nginx.c @@ -1478,7 +1478,7 @@ ngx_set_worker_processes(ngx_conf_t *cf, ngx_command_t *cmd, void *conf) ???? value = cf->args->elts; ???? if (ngx_strcmp(value[1].data, "auto") == 0) { -??????? ccf->worker_processes = ngx_ncpu; +??????? ccf->worker_processes = ngx_min(ngx_ncpu, NGX_MAX_PROCESSES); ???????? return NGX_CONF_OK; ???? } From mdounin at mdounin.ru Tue May 12 13:43:29 2020 From: mdounin at mdounin.ru (Maxim Dounin) Date: Tue, 12 May 2020 16:43:29 +0300 Subject: [RFC] Core: worker_processes <= NGX_MAX_PROCESSES In-Reply-To: <36bd8093-749f-d9c9-cccf-b85d74f9c683@student.uibk.ac.at> References: <36bd8093-749f-d9c9-cccf-b85d74f9c683@student.uibk.ac.at> Message-ID: <20200512134329.GK20357@mdounin.ru> Hello! On Tue, May 12, 2020 at 12:04:58PM +0200, Peter Stadler wrote: > Would it be advisable to limit the number of worker_processes to > NGX_MAX_PROCESSES at least when we have `worker_processes auto;` in the > config? We've yet to see systems which need this, and if these systems are actually available - we may want to bump NGX_MAX_PROCESSES instead. -- Maxim Dounin http://mdounin.ru/ From peter.stadler at student.uibk.ac.at Tue May 12 16:58:45 2020 From: peter.stadler at student.uibk.ac.at (Peter Stadler) Date: Tue, 12 May 2020 18:58:45 +0200 Subject: [RFC] Core: worker_processes <= NGX_MAX_PROCESSES In-Reply-To: <20200512134329.GK20357@mdounin.ru> References: <36bd8093-749f-d9c9-cccf-b85d74f9c683@student.uibk.ac.at> <20200512134329.GK20357@mdounin.ru> Message-ID: <305ce0e1-f42a-1772-a9df-f51154c4e76d@student.uibk.ac.at> Hello Maxim Dounin, thank you very much for the fast response. Adopting the value to available systems is clearly a solution. I saw the problem, where NGX_MAX_PROCESSES is patched to 8 when using `worker_processes auto;`, and thought it may be good to limit it in all cases. All the best, Peter On 12.05.20 15:43, Maxim Dounin wrote: > Hello! > > On Tue, May 12, 2020 at 12:04:58PM +0200, Peter Stadler wrote: > >> Would it be advisable to limit the number of worker_processes to >> NGX_MAX_PROCESSES at least when we have `worker_processes auto;` in the >> config? > We've yet to see systems which need this, and if these systems are > actually available - we may want to bump NGX_MAX_PROCESSES > instead. > From mdounin at mdounin.ru Tue May 12 19:06:33 2020 From: mdounin at mdounin.ru (Maxim Dounin) Date: Tue, 12 May 2020 22:06:33 +0300 Subject: [PATCH] Support loading server certificate from HW token In-Reply-To: References: <1045205432.24749495.1587988397159.JavaMail.zimbra@redhat.com> <170259234.25242098.1588247754550.JavaMail.zimbra@redhat.com> <8058a696e8a946549c7516b66fbf9bc2@cryptopro.ru> <1942859015.25708027.1588596156868.JavaMail.zimbra@redhat.com> Message-ID: <20200512190633.GN20357@mdounin.ru> Hello! On Fri, May 08, 2020 at 07:53:18PM +0000, ??????? ??????? ?????????? wrote: > I dipped into the problem and came to the conclusion that this > proposal cannot be used as a general one. > > First, although the ctrl number could be passed in the directive > itself, for example "engine:pkcs11:205:slot_0-id_00", where 205 > corresponds to CMD_LOAD_CERT_CTRL (ENGINE_CMD_BASE + 5 = 200 + > 5), the argument "params" is too specific for this command, in > fact, it is a binding to a specific non-extensible interface of > a particular ENGINE command. > > Secondly, this binding to a bad interface actually, which is not > able to return the certificate chain, CMD_LOAD_CERT_CTRL returns > only the leaf certificate. > > Therefore, I do not see how this can be used outside of pkcs11 > ENGINE and I do not see how this can be used in a production > without a certificate chain. Thanks for the review, appreciated. A possible use case might be to use it for proxy_ssl_certificate, but I agree that this looks very limited and at most optional. -- Maxim Dounin http://mdounin.ru/ From mdounin at mdounin.ru Tue May 12 19:18:03 2020 From: mdounin at mdounin.ru (Maxim Dounin) Date: Tue, 12 May 2020 22:18:03 +0300 Subject: Feature suggestion: Additional check for SSL misconfiguration in stream proxy. In-Reply-To: References: Message-ID: <20200512191803.GO20357@mdounin.ru> Hello! On Thu, Apr 30, 2020 at 07:59:41AM +0300, Andrey Kulikov wrote: > Hello, > > Consider following configuration: > > stream { > server { > listen 5443; > proxy_pass my-tls-upstream:443; > proxy_ssl_verify on; > proxy_ssl_server_name on; > proxy_ssl_trusted_certificate trusted_root_CAs.cer; > } > } # end stream > > It is perfectly Ok for nginx, though it doesn't do what one would > expect it to - data being send to upstream server in plain text. > This is due to the fact that proxy_ssl if off by default. > So all proxy_ssl_* directives being ignored. > > This looks kind of error-prone, as unlike in HTTP-proxy module, we > can't specify schema for upstream connections. > > Thus, one could expect nginx to complain about misconfiguration (using > proxy_ssl_* without specifying proxy_ssl on; first), rather than > silently send data in cleartext. > > If patch with additional checks implementation for stream-proxy module > will be submitted, are there any chances it could be considered for > merging into upstream? Unlikely. The problem is that things like "proxy_ssl_verify" and "proxy_ssl_server_name" are optional configuration directives, and they, for example, might be set globally and inherited into the server in question, and/or set in an include file used for all servers. Or simply used in a configuration where there are multiple endpoints switched manually, some with "proxy_ssl on;" and some in plain text. Instead, consider checking that you have "proxy_ssl on;" in the configuration if you want nginx to use SSL with upstream servers in a stream server block. Alternatively, we may want to consider something like "ssl://" pseudo scheme instead of "proxy_ssl on;", similarly to how it works in "proxy_pass https://..." in the http module, but this might not be a good idea either. -- Maxim Dounin http://mdounin.ru/ From jmheisz at gmail.com Wed May 13 04:18:37 2020 From: jmheisz at gmail.com (Jeff Heisz) Date: Wed, 13 May 2020 00:18:37 -0400 Subject: Streaming responses from long running requests without descriptors? Message-ID: Let's start with some background. I'm developing a module to run inside the NGINX engine to process requests (standard content phase) against an API (not an upstream-compatible socket-based protocol). These requests could be long-running so the module is using thread tasks to execute and generate responses. I'm well aware of how to safely manage content in multi-threaded environments and the threading/pooling/data management constraints of the NGINX internals. For the most part, I've resolved all of the nuances of handling "small" requests and responses. However, the current code handles responses by accumulating a buffer chain of content to be issued with output_filter and finalize_request - which is less than optimal for really large responses which can result from certain parts of the API. Now for the question - I'm looking for some sample code or basic direction on how to best implement a streaming consumption model to both avoid memory overload and provide write-based throttling from the threaded API processing to the HTTP response instance. Of course, there are the obvious threading boundaries where the task instance cannot call response methods on the request and I know how to handle inter-thread processing/messaging. But what I can't figure out is how to pool/interact with such a transfer mechanism from the main event thread. Three thoughts/possible solutions: - overload the write event handler in the long-running circumstance to check/consume available chain buffers from the thread processor, with some kind of signalling/hold mechanism to throttle the thread and avoid memory overload. Two issues with this (so far) - how to capture write backlog to throttle the thread but more importantly, how to handle the edge-transition model of the underlying event loop to cause the write handler to actually fire to check for pending records. This (to me) is the 'cleanest' model that I've tried to implement but have been unable to properly get the write handler to fire consistently - use a continuous/never-ending 1ms timer to simulate an every-event-loop-callback to process backlog, which is even less elegant, it does fire consistently but creates a delay and still has the blocking measurement issues of the previous solution, not to mention having a single callback manage a collection of responses - the third option, which I originally didn't really like for several reasons but now that I've typed all this out think might be the best solution, is to bind a kernel pipe and use that to shuttle the content between the thread and the event loop, as if it were an external upstream. Was originally thinking of using the pipe to 'wake' the event thread to consume the records but now realizing that I could just use the pipe to shuttle all of the content. The only thing I don't like about this (aside from it seeming to be a much-too-complex solution to the problem) is the possible overload of pipes for a lot of concurrent requests. Any thoughts? Not looking for someone to provide the code (although if someone says 'hey, check out this module, it does exactly what you need' I won't be upset). Just looking for some guidance from anyone who might have some ideas on how to best approach this problem... jmh - From h312841925 at gmail.com Wed May 13 14:17:18 2020 From: h312841925 at gmail.com (Jim T) Date: Wed, 13 May 2020 22:17:18 +0800 Subject: Upstream: jump out of unnecessary loop after matching the status code. Message-ID: Hi: In ngx_http_upstream_test_next, the loop will continue when the status is matched, and the ngx_http_upstream_next_errors is non-repeating. I think we can jump out of for the unnecessary loop, and here is my patch: # HG changeset patch # User Jinhua Tan <312841925 at qq.com> # Date 1589378567 -28800 # Wed May 13 22:02:47 2020 +0800 # Node ID c79fcc4cf3acb68ca0e5c7767820c23308c40f6e # Parent 028b16e2798f8ec4c6a3d042a6ac8a07f7262690 Upstream: jump out of unnecessary loop after matching the status code. diff -r 028b16e2798f -r c79fcc4cf3ac src/http/ngx_http_upstream.c --- a/src/http/ngx_http_upstream.c Fri May 08 19:19:16 2020 +0300 +++ b/src/http/ngx_http_upstream.c Wed May 13 22:02:47 2020 +0800 @@ -2502,6 +2502,8 @@ } #endif + + break; } #if (NGX_HTTP_CACHE) Thanks! -------------- next part -------------- An HTML attachment was scrubbed... URL: From mdounin at mdounin.ru Wed May 13 21:02:08 2020 From: mdounin at mdounin.ru (Maxim Dounin) Date: Wed, 13 May 2020 21:02:08 +0000 Subject: [nginx] Upstream: jump out of loop after matching the status code. Message-ID: details: https://hg.nginx.org/nginx/rev/3c8082c3f98a branches: changeset: 7649:3c8082c3f98a user: Jinhua Tan <312841925 at qq.com> date: Wed May 13 22:02:47 2020 +0800 description: Upstream: jump out of loop after matching the status code. diffstat: src/http/ngx_http_upstream.c | 2 ++ 1 files changed, 2 insertions(+), 0 deletions(-) diffs (12 lines): diff -r 028b16e2798f -r 3c8082c3f98a src/http/ngx_http_upstream.c --- a/src/http/ngx_http_upstream.c Fri May 08 19:19:16 2020 +0300 +++ b/src/http/ngx_http_upstream.c Wed May 13 22:02:47 2020 +0800 @@ -2502,6 +2502,8 @@ ngx_http_upstream_test_next(ngx_http_req } #endif + + break; } #if (NGX_HTTP_CACHE) From mdounin at mdounin.ru Wed May 13 21:03:02 2020 From: mdounin at mdounin.ru (Maxim Dounin) Date: Thu, 14 May 2020 00:03:02 +0300 Subject: Upstream: jump out of unnecessary loop after matching the status code. In-Reply-To: References: Message-ID: <20200513210302.GA12747@mdounin.ru> Hello! On Wed, May 13, 2020 at 10:17:18PM +0800, Jim T wrote: > In ngx_http_upstream_test_next, the loop will continue when the status is > matched, and the ngx_http_upstream_next_errors is non-repeating. I think we > can jump out of for the unnecessary loop, and here is my patch: > > # HG changeset patch > # User Jinhua Tan <312841925 at qq.com> > # Date 1589378567 -28800 > # Wed May 13 22:02:47 2020 +0800 > # Node ID c79fcc4cf3acb68ca0e5c7767820c23308c40f6e > # Parent 028b16e2798f8ec4c6a3d042a6ac8a07f7262690 > Upstream: jump out of unnecessary loop after matching the status code. > > diff -r 028b16e2798f -r c79fcc4cf3ac src/http/ngx_http_upstream.c > --- a/src/http/ngx_http_upstream.c Fri May 08 19:19:16 2020 +0300 > +++ b/src/http/ngx_http_upstream.c Wed May 13 22:02:47 2020 +0800 > @@ -2502,6 +2502,8 @@ > } > > #endif > + > + break; > } > > #if (NGX_HTTP_CACHE) > > Thanks! Committed (with minor changes to commit log), thanks. -- Maxim Dounin http://mdounin.ru/ From mdounin at mdounin.ru Wed May 13 21:11:27 2020 From: mdounin at mdounin.ru (Maxim Dounin) Date: Thu, 14 May 2020 00:11:27 +0300 Subject: [PATCH] Cache: new options for set path and file permissions (ticket: #698) In-Reply-To: References: Message-ID: <20200513211127.GB12747@mdounin.ru> Hello! On Mon, May 04, 2020 at 02:14:26AM +0200, Krzysztof Grzadziel wrote: > # HG changeset patch > # User Krzysztof Grz?dziel > # Date 1588474577 -7200 > # Sun May 03 04:56:17 2020 +0200 > # Node ID ae1a050faa5c718dc81928dced93337f1e64a4e6 > # Parent 716eddd74bc2831537f5b3f7ecd16ad3e516d043 > Cache: new options for set path and file permissions (ticket: #698) > > Patch introduces two optional parameters for *_cache_path: > - path_access_rights which sets chmod for directories under cache path > - file_access_rights which sets chmod for files under cache path Thank you for the patch. This feature, however, seems to be very specific for your particular setup. And this isn't what we can recommend as a generic approach to managing cache, as general assumption is that the cache directory is managed by a single nginx instance. As such, this patch won't be merged. -- Maxim Dounin http://mdounin.ru/ From mdounin at mdounin.ru Thu May 14 13:07:09 2020 From: mdounin at mdounin.ru (Maxim Dounin) Date: Thu, 14 May 2020 16:07:09 +0300 Subject: [PATCH] Ensured SIGQUIT deletes listening UNIX socket files. In-Reply-To: <7112c3e4-01c7-8fa7-ad8d-aa3934190817@fastmail.com> References: <4b6bc48b-6c7a-7c76-51ae-1038421ed36c@fastmail.com> <20200227152429.GI12894@mdounin.ru> <2c06ad76-c309-d387-2c40-f6e60aacf4f8@fastmail.com> <20200303142811.GO12894@mdounin.ru> <22c4b64a-eb4e-a8a8-7428-34ab3225fcf5@fastmail.com> <20200426011240.GL20357@mdounin.ru> <6d8edc45-6133-de52-8220-05c962a4a48c@fastmail.com> <7112c3e4-01c7-8fa7-ad8d-aa3934190817@fastmail.com> Message-ID: <20200514130709.GD12747@mdounin.ru> Hello! On Fri, May 08, 2020 at 01:41:08PM -0700, Thibault Charbonnier wrote: > On 4/27/20 4:26 PM, Thibault Charbonnier wrote: > > Great! Thanks for the suggestion. Below is a revised approach for the > > patch (also attached to this email) which passes all of the test cases > > listed in my previous test file at the start of this thread: > > Hello, > > Are there plans for a follow-up on this patch? Yes, as long I'll have some time and resources (or will be able to convince someone else to look at it). Sorry it takes so long. -- Maxim Dounin http://mdounin.ru/ From mdounin at mdounin.ru Thu May 14 22:25:26 2020 From: mdounin at mdounin.ru (Maxim Dounin) Date: Fri, 15 May 2020 01:25:26 +0300 Subject: [PATCH] Follow RFC-7232 while evaluating the precedence of conditional headers In-Reply-To: References: Message-ID: <20200514222526.GJ12747@mdounin.ru> Hello! On Fri, May 01, 2020 at 03:05:05PM -0700, Karthik Uthaman wrote: > Nginx evaluates the conditional headers in the following order > if_unmodified_since > if_match > if_modified_since > if_none_match > > which ignores some of the subtleties mentioned in RFC-7232, which > gives more precedence to etag based validation over time. > > https://tools.ietf.org/html/rfc7232#section-3.4 > A recipient MUST ignore If-Unmodified-Since if the request contains > an If-Match header field; the condition in If-Match is considered to > be a more accurate replacement for the condition in > If-Unmodified-Since, and the two are only combined for the sake of > interoperating with older intermediaries that might not implement > If-Match. > > https://tools.ietf.org/html/rfc7232#section-3.3 and > A recipient MUST ignore If-Modified-Since if the request contains an > If-None-Match header field; the condition in If-None-Match is > considered to be a more accurate replacement for the condition in > If-Modified-Since, and the two are only combined for the sake of > interoperating with older intermediaries that might not implement > If-None-Match. > > Along with those I have followed the precedence guidance provided in > https://tools.ietf.org/html/rfc7232#section-6 to change the order of > conditional headers being evaluated in nginx. > > Please let me know if the changes in the patch file looks good. Happy to > address any concerns you may have. This was previously discussed in the thread here: http://mailman.nginx.org/pipermail/nginx-devel/2014-November/006245.html TL;DR: the required behaviour was changed in RFC 7232 compared to RFC 2616, and no explanation is known why the change in question was introduced in RFC 7232. Currently nginx follows RFC 2616 and checks both validators. -- Maxim Dounin http://mdounin.ru/ From naidilepn at gmail.com Fri May 15 08:44:34 2020 From: naidilepn at gmail.com (naidile.pn) Date: Fri, 15 May 2020 14:14:34 +0530 Subject: Error - Using Nginx Proxy Cache Message-ID: Hi All, We are using proxy cache to cache the response from upstream for a day. But, when the cache expires, API stops responding with 500. Getting the error: open() "proxycachepath" failed (13: Permission denied) On reloading the nginx, server is able to connect to upstream and cache response for the first time and works for next one day. Below is the nginx rule : location /somelocation { add_header X-Proxy-Cache $upstream_cache_status; proxy_cache my_zone; proxy_ignore_headers X-Accel-Expires Expires Cache-Control; proxy_cache_valid 200 1440m; proxy_cache_use_stale error timeout http_500 http_502 http_503 http_504; proxy_pass someupstream; proxy_pass_request_headers on; proxy_connect_timeout 120; proxy_send_timeout 120; proxy_read_timeout 120; } Can anyone please help here. Thanks! -------------- next part -------------- An HTML attachment was scrubbed... URL: From xeioex at nginx.com Mon May 18 13:23:06 2020 From: xeioex at nginx.com (Dmitry Volyntsev) Date: Mon, 18 May 2020 13:23:06 +0000 Subject: [njs] Fixed Array.prototype.slice() for sparse arrays. Message-ID: details: https://hg.nginx.org/njs/rev/80d95b2881f6 branches: changeset: 1389:80d95b2881f6 user: Dmitry Volyntsev date: Mon May 18 13:22:34 2020 +0000 description: Fixed Array.prototype.slice() for sparse arrays. diffstat: src/njs_array.c | 8 +++++--- src/test/njs_unit_test.c | 7 +++++-- 2 files changed, 10 insertions(+), 5 deletions(-) diffs (55 lines): diff -r 92838d964b19 -r 80d95b2881f6 src/njs_array.c --- a/src/njs_array.c Fri May 08 15:52:40 2020 +0000 +++ b/src/njs_array.c Mon May 18 13:22:34 2020 +0000 @@ -686,15 +686,17 @@ njs_array_prototype_slice_copy(njs_vm_t /* src value may be in Array.prototype object. */ - value = &array->start[n++]; - ret = njs_value_property_i64(vm, this, start++, value); + ret = njs_value_property_i64(vm, this, start++, + &array->start[n]); if (njs_slow_path(ret == NJS_ERROR)) { return NJS_ERROR; } if (ret != NJS_OK) { - njs_set_invalid(value); + njs_set_invalid(&array->start[n]); } + + n++; } length--; diff -r 92838d964b19 -r 80d95b2881f6 src/test/njs_unit_test.c --- a/src/test/njs_unit_test.c Fri May 08 15:52:40 2020 +0000 +++ b/src/test/njs_unit_test.c Mon May 18 13:22:34 2020 +0000 @@ -4313,6 +4313,9 @@ static njs_unit_test_t njs_test[] = { njs_str("[0,1].slice()"), njs_str("0,1") }, + { njs_str("[1,2,3,,,4].slice()"), + njs_str("1,2,3,,,4") }, + { njs_str("[0,1].slice(undefined)"), njs_str("0,1") }, @@ -4326,7 +4329,7 @@ static njs_unit_test_t njs_test[] = njs_str("") }, { njs_str("var a = [1,2,3,4,5], b = a.slice(3);" - "b[0] +' '+ b[1] +' '+ b[2]"), + "b[0] +' '+ b[1] +' '+ b[2]"), njs_str("4 5 undefined") }, { njs_str("var a = [1,2]; a.pop() +' '+ a.length +' '+ a"), @@ -4336,7 +4339,7 @@ static njs_unit_test_t njs_test[] = njs_str("3 3 1,2") }, { njs_str("var a = [1,2], len = a.push(3,4,5);" - "len +' '+ a.pop() +' '+ a"), + "len +' '+ a.pop() +' '+ a"), njs_str("5 5 1,2,3,4") }, { njs_str("var x = {'0': 'a', '1': 'b', '2': 'c', 'length': 3};" From xeioex at nginx.com Mon May 18 14:43:10 2020 From: xeioex at nginx.com (Dmitry Volyntsev) Date: Mon, 18 May 2020 14:43:10 +0000 Subject: [njs] Fixed benchmark test missed in 78b8cfd5b1e9. Message-ID: details: https://hg.nginx.org/njs/rev/cd1a7a0c3fe9 branches: changeset: 1390:cd1a7a0c3fe9 user: Dmitry Volyntsev date: Mon May 18 14:43:00 2020 +0000 description: Fixed benchmark test missed in 78b8cfd5b1e9. diffstat: src/test/njs_benchmark.c | 8 ++++---- 1 files changed, 4 insertions(+), 4 deletions(-) diffs (39 lines): diff -r 80d95b2881f6 -r cd1a7a0c3fe9 src/test/njs_benchmark.c --- a/src/test/njs_benchmark.c Mon May 18 13:22:34 2020 +0000 +++ b/src/test/njs_benchmark.c Mon May 18 14:43:00 2020 +0000 @@ -76,6 +76,7 @@ njs_benchmark_test(njs_vm_t *parent, njs n = test->repeat; expected = &test->result; + ret = NJS_ERROR; us = njs_time() / 1000; for (i = 0; i < n; i++) { @@ -96,7 +97,8 @@ njs_benchmark_test(njs_vm_t *parent, njs success = njs_strstr_eq(expected, &s); if (!success) { - njs_printf("failed: \"%V\" vs \"%V\"\n", expected, &s); + njs_printf("%s failed: \"%V\" vs \"%V\"\n", test->name, expected, + &s); goto done; } @@ -266,7 +268,7 @@ static njs_benchmark_test_t njs_test[] { "external object property ($shared.props.a)", njs_str("$shared.props.a"), - njs_str("4294967295"), + njs_str("11"), 1000 }, { "external dump (JSON.stringify($shared.header))", @@ -395,8 +397,6 @@ main(int argc, char **argv) njs_vm_start(vm); - ret = EXIT_FAILURE; - ret = njs_vm_array_alloc(vm, &report, 8); if (ret != NJS_OK) { njs_printf("njs_vm_array_alloc() failed\n"); From naidilepn at gmail.com Mon May 18 17:26:20 2020 From: naidilepn at gmail.com (naidile.pn) Date: Mon, 18 May 2020 22:56:20 +0530 Subject: Error - Using Nginx Proxy Cache In-Reply-To: References: Message-ID: Hi, Can you please help here - basically after the cache expires, the upstream call is failing as not accessible and we are serving stale data to avoid failures. After we reload the nginx- first call to upstream is successful and data is cached. Thanks! On Fri, May 15, 2020, 2:14 PM naidile.pn wrote: > Hi All, > > We are using proxy cache to cache the response from upstream for a day. > But, when the cache expires, API stops responding with 500. > > Getting the error: open() "proxycachepath" failed (13: Permission denied) > > On reloading the nginx, server is able to connect to upstream and cache > response for the first time and works for next one day. > > Below is the nginx rule : > > location /somelocation { > add_header X-Proxy-Cache $upstream_cache_status; > proxy_cache my_zone; > proxy_ignore_headers X-Accel-Expires Expires Cache-Control; > proxy_cache_valid 200 1440m; > proxy_cache_use_stale error timeout http_500 http_502 > http_503 http_504; > proxy_pass someupstream; > proxy_pass_request_headers on; > proxy_connect_timeout 120; > proxy_send_timeout 120; > proxy_read_timeout 120; > } > > Can anyone please help here. > > Thanks! > -------------- next part -------------- An HTML attachment was scrubbed... URL: From xeioex at nginx.com Tue May 19 11:41:33 2020 From: xeioex at nginx.com (Dmitry Volyntsev) Date: Tue, 19 May 2020 11:41:33 +0000 Subject: [njs] Version 0.4.1. Message-ID: details: https://hg.nginx.org/njs/rev/9400790bf538 branches: changeset: 1391:9400790bf538 user: Dmitry Volyntsev date: Tue May 19 11:41:05 2020 +0000 description: Version 0.4.1. diffstat: CHANGES | 12 ++++++++++++ 1 files changed, 12 insertions(+), 0 deletions(-) diffs (19 lines): diff -r cd1a7a0c3fe9 -r 9400790bf538 CHANGES --- a/CHANGES Mon May 18 14:43:00 2020 +0000 +++ b/CHANGES Tue May 19 11:41:05 2020 +0000 @@ -1,3 +1,15 @@ + +Changes with njs 0.4.1 19 May 2020 + + *) Feature: added support for multi-value headers in r.headersIn. + + *) Feature: introduced raw headers API. + + *) Feature: added TypedScript API description. + + Core: + + *) Bugfix: fixed Array.prototype.slice() for sparse arrays. Changes with njs 0.4.0 23 Apr 2020 From xeioex at nginx.com Tue May 19 11:42:34 2020 From: xeioex at nginx.com (Dmitry Volyntsev) Date: Tue, 19 May 2020 11:42:34 +0000 Subject: [njs] Added tag 0.4.1 for changeset 9400790bf538 Message-ID: details: https://hg.nginx.org/njs/rev/46ec2411fb7c branches: changeset: 1392:46ec2411fb7c user: Dmitry Volyntsev date: Tue May 19 11:42:11 2020 +0000 description: Added tag 0.4.1 for changeset 9400790bf538 diffstat: .hgtags | 1 + 1 files changed, 1 insertions(+), 0 deletions(-) diffs (8 lines): diff -r 9400790bf538 -r 46ec2411fb7c .hgtags --- a/.hgtags Tue May 19 11:41:05 2020 +0000 +++ b/.hgtags Tue May 19 11:42:11 2020 +0000 @@ -34,3 +34,4 @@ 9e5ef927c7eaed003de3e5e4a16fa3eab08de7f7 1abb97e9d9dc07dcff2616d4c75132e6d189a6aa 0.3.8 fc4eeaaf0dfe7dc7d41232f1b643bd364f1efe82 0.3.9 6144aafa1472fbdf79bc9d1f858555938ee08452 0.4.0 +9400790bf53843001f94c77b47bc99b05518af78 0.4.1 From h312841925 at gmail.com Wed May 20 04:29:45 2020 From: h312841925 at gmail.com (Jim T) Date: Wed, 20 May 2020 12:29:45 +0800 Subject: Style(core): fix the incorrect front spaces to unite style. Message-ID: Hello! Nginx is famous for code readability and neat code format. I use the tool( https://github.com/openresty/openresty-devel-utils) to try to unify the incorrect front spaces, Here is my attempt at core. # HG changeset patch # User Jinhua Tan <312841925 at qq.com> # Date 1589948492 -28800 # Wed May 20 12:21:32 2020 +0800 # Node ID 10678810de74edc4c02d5cd7303f0099eee9fac3 # Parent 3c8082c3f98ab7fdec8a598b55609701452b5254 Style(core): fix the incorrect front spaces to unite style. diff -r 3c8082c3f98a -r 10678810de74 src/core/ngx_conf_file.c --- a/src/core/ngx_conf_file.c Wed May 13 22:02:47 2020 +0800 +++ b/src/core/ngx_conf_file.c Wed May 20 12:21:32 2020 +0800 @@ -544,8 +544,8 @@ } ngx_conf_log_error(NGX_LOG_EMERG, cf, 0, - "unexpected end of file, " - "expecting \";\" or \"}\""); + "unexpected end of file, " + "expecting \";\" or \"}\""); return NGX_ERROR; } @@ -1047,9 +1047,9 @@ } else { ngx_conf_log_error(NGX_LOG_EMERG, cf, 0, - "invalid value \"%s\" in \"%s\" directive, " - "it must be \"on\" or \"off\"", - value[1].data, cmd->name.data); + "invalid value \"%s\" in \"%s\" directive, " + "it must be \"on\" or \"off\"", + value[1].data, cmd->name.data); return NGX_CONF_ERROR; } diff -r 3c8082c3f98a -r 10678810de74 src/core/ngx_cycle.c --- a/src/core/ngx_cycle.c Wed May 13 22:02:47 2020 +0800 +++ b/src/core/ngx_cycle.c Wed May 20 12:21:32 2020 +0800 @@ -1308,9 +1308,9 @@ if (tag != shm_zone[i].tag) { ngx_conf_log_error(NGX_LOG_EMERG, cf, 0, - "the shared memory zone \"%V\" is " - "already declared for a different use", - &shm_zone[i].shm.name); + "the shared memory zone \"%V\" is " + "already declared for a different use", + &shm_zone[i].shm.name); return NULL; } diff -r 3c8082c3f98a -r 10678810de74 src/core/ngx_file.c --- a/src/core/ngx_file.c Wed May 13 22:02:47 2020 +0800 +++ b/src/core/ngx_file.c Wed May 20 12:21:32 2020 +0800 @@ -1005,7 +1005,7 @@ name = ngx_de_name(&dir); ngx_log_debug2(NGX_LOG_DEBUG_CORE, ctx->log, 0, - "tree name %uz:\"%s\"", len, name); + "tree name %uz:\"%s\"", len, name); if (len == 1 && name[0] == '.') { continue; diff -r 3c8082c3f98a -r 10678810de74 src/core/ngx_log.c --- a/src/core/ngx_log.c Wed May 13 22:02:47 2020 +0800 +++ b/src/core/ngx_log.c Wed May 20 12:21:32 2020 +0800 @@ -121,7 +121,7 @@ /* pid#tid */ p = ngx_slprintf(p, last, "%P#" NGX_TID_T_FMT ": ", - ngx_log_pid, ngx_log_tid); + ngx_log_pid, ngx_log_tid); if (log->connection) { p = ngx_slprintf(p, last, "*%uA ", log->connection); @@ -385,8 +385,8 @@ ngx_open_file_n " \"%s\" failed", name); #if (NGX_WIN32) ngx_event_log(ngx_errno, - "could not open error log file: " - ngx_open_file_n " \"%s\" failed", name); + "could not open error log file: " + ngx_open_file_n " \"%s\" failed", name); #endif ngx_log_file.fd = ngx_stderr; diff -r 3c8082c3f98a -r 10678810de74 src/core/ngx_regex.c --- a/src/core/ngx_regex.c Wed May 13 22:02:47 2020 +0800 +++ b/src/core/ngx_regex.c Wed May 20 12:21:32 2020 +0800 @@ -111,8 +111,8 @@ if (re == NULL) { if ((size_t) erroff == rc->pattern.len) { rc->err.len = ngx_snprintf(rc->err.data, rc->err.len, - "pcre_compile() failed: %s in \"%V\"", - errstr, &rc->pattern) + "pcre_compile() failed: %s in \"%V\"", + errstr, &rc->pattern) - rc->err.data; } else { diff -r 3c8082c3f98a -r 10678810de74 src/core/ngx_resolver.c --- a/src/core/ngx_resolver.c Wed May 13 22:02:47 2020 +0800 +++ b/src/core/ngx_resolver.c Wed May 20 12:21:32 2020 +0800 @@ -3265,8 +3265,8 @@ } ngx_log_debug3(NGX_LOG_DEBUG_CORE, r->log, 0, - "resolver qt:%ui cl:%ui len:%uz", - type, class, len); + "resolver qt:%ui cl:%ui len:%uz", + type, class, len); i += sizeof(ngx_resolver_an_t); Thanks! -------------- next part -------------- An HTML attachment was scrubbed... URL: From h312841925 at gmail.com Wed May 20 04:31:11 2020 From: h312841925 at gmail.com (Jim T) Date: Wed, 20 May 2020 12:31:11 +0800 Subject: Style(http/modules): fix the incorrect front spaces to unite style. Message-ID: Hello! Nginx is famous for code readability and neat code format. I use the tool( https://github.com/openresty/openresty-devel-utils) to try to unify the incorrect front spaces, Here is my attempt at http/modules. # HG changeset patch # User Jinhua Tan <312841925 at qq.com> # Date 1589948645 -28800 # Wed May 20 12:24:05 2020 +0800 # Node ID 3242f98298975e556a7e87130611ce84799fe935 # Parent 10678810de74edc4c02d5cd7303f0099eee9fac3 Style(http/modules): fix the incorrect front spaces to unite style. diff -r 10678810de74 -r 3242f9829897 src/http/modules/ngx_http_access_module.c --- a/src/http/modules/ngx_http_access_module.c Wed May 20 12:21:32 2020 +0800 +++ b/src/http/modules/ngx_http_access_module.c Wed May 20 12:24:05 2020 +0800 @@ -327,7 +327,7 @@ if (rc == NGX_ERROR) { ngx_conf_log_error(NGX_LOG_EMERG, cf, 0, - "invalid parameter \"%V\"", &value[1]); + "invalid parameter \"%V\"", &value[1]); return NGX_CONF_ERROR; } diff -r 10678810de74 -r 3242f9829897 src/http/modules/ngx_http_dav_module.c --- a/src/http/modules/ngx_http_dav_module.c Wed May 20 12:21:32 2020 +0800 +++ b/src/http/modules/ngx_http_dav_module.c Wed May 20 12:24:05 2020 +0800 @@ -1116,10 +1116,10 @@ ngx_http_dav_loc_conf_t *conf = child; ngx_conf_merge_bitmask_value(conf->methods, prev->methods, - (NGX_CONF_BITMASK_SET|NGX_HTTP_DAV_OFF)); + (NGX_CONF_BITMASK_SET|NGX_HTTP_DAV_OFF)); ngx_conf_merge_uint_value(conf->min_delete_depth, - prev->min_delete_depth, 0); + prev->min_delete_depth, 0); ngx_conf_merge_uint_value(conf->access, prev->access, 0600); diff -r 10678810de74 -r 3242f9829897 src/http/modules/ngx_http_fastcgi_module.c --- a/src/http/modules/ngx_http_fastcgi_module.c Wed May 20 12:21:32 2020 +0800 +++ b/src/http/modules/ngx_http_fastcgi_module.c Wed May 20 12:24:05 2020 +0800 @@ -2805,7 +2805,7 @@ if (conf->upstream.store == NGX_CONF_UNSET) { ngx_conf_merge_value(conf->upstream.store, - prev->upstream.store, 0); + prev->upstream.store, 0); conf->upstream.store_lengths = prev->upstream.store_lengths; conf->upstream.store_values = prev->upstream.store_values; @@ -2818,22 +2818,22 @@ prev->upstream.next_upstream_tries, 0); ngx_conf_merge_value(conf->upstream.buffering, - prev->upstream.buffering, 1); + prev->upstream.buffering, 1); ngx_conf_merge_value(conf->upstream.request_buffering, - prev->upstream.request_buffering, 1); + prev->upstream.request_buffering, 1); ngx_conf_merge_value(conf->upstream.ignore_client_abort, - prev->upstream.ignore_client_abort, 0); + prev->upstream.ignore_client_abort, 0); ngx_conf_merge_value(conf->upstream.force_ranges, - prev->upstream.force_ranges, 0); + prev->upstream.force_ranges, 0); ngx_conf_merge_ptr_value(conf->upstream.local, - prev->upstream.local, NULL); + prev->upstream.local, NULL); ngx_conf_merge_value(conf->upstream.socket_keepalive, - prev->upstream.socket_keepalive, 0); + prev->upstream.socket_keepalive, 0); ngx_conf_merge_msec_value(conf->upstream.connect_timeout, prev->upstream.connect_timeout, 60000); @@ -2951,15 +2951,15 @@ ngx_conf_merge_bitmask_value(conf->upstream.ignore_headers, - prev->upstream.ignore_headers, - NGX_CONF_BITMASK_SET); + prev->upstream.ignore_headers, + NGX_CONF_BITMASK_SET); ngx_conf_merge_bitmask_value(conf->upstream.next_upstream, - prev->upstream.next_upstream, - (NGX_CONF_BITMASK_SET - |NGX_HTTP_UPSTREAM_FT_ERROR - |NGX_HTTP_UPSTREAM_FT_TIMEOUT)); + prev->upstream.next_upstream, + (NGX_CONF_BITMASK_SET + |NGX_HTTP_UPSTREAM_FT_ERROR + |NGX_HTTP_UPSTREAM_FT_TIMEOUT)); if (conf->upstream.next_upstream & NGX_HTTP_UPSTREAM_FT_OFF) { conf->upstream.next_upstream = NGX_CONF_BITMASK_SET @@ -2978,7 +2978,7 @@ if (conf->upstream.cache == NGX_CONF_UNSET) { ngx_conf_merge_value(conf->upstream.cache, - prev->upstream.cache, 0); + prev->upstream.cache, 0); conf->upstream.cache_zone = prev->upstream.cache_zone; conf->upstream.cache_value = prev->upstream.cache_value; @@ -3000,13 +3000,13 @@ prev->upstream.cache_min_uses, 1); ngx_conf_merge_off_value(conf->upstream.cache_max_range_offset, - prev->upstream.cache_max_range_offset, - NGX_MAX_OFF_T_VALUE); + prev->upstream.cache_max_range_offset, + NGX_MAX_OFF_T_VALUE); ngx_conf_merge_bitmask_value(conf->upstream.cache_use_stale, - prev->upstream.cache_use_stale, - (NGX_CONF_BITMASK_SET - |NGX_HTTP_UPSTREAM_FT_OFF)); + prev->upstream.cache_use_stale, + (NGX_CONF_BITMASK_SET + |NGX_HTTP_UPSTREAM_FT_OFF)); if (conf->upstream.cache_use_stale & NGX_HTTP_UPSTREAM_FT_OFF) { conf->upstream.cache_use_stale = NGX_CONF_BITMASK_SET @@ -3042,7 +3042,7 @@ } ngx_conf_merge_value(conf->upstream.cache_lock, - prev->upstream.cache_lock, 0); + prev->upstream.cache_lock, 0); ngx_conf_merge_msec_value(conf->upstream.cache_lock_timeout, prev->upstream.cache_lock_timeout, 5000); @@ -3051,20 +3051,20 @@ prev->upstream.cache_lock_age, 5000); ngx_conf_merge_value(conf->upstream.cache_revalidate, - prev->upstream.cache_revalidate, 0); + prev->upstream.cache_revalidate, 0); ngx_conf_merge_value(conf->upstream.cache_background_update, - prev->upstream.cache_background_update, 0); + prev->upstream.cache_background_update, 0); #endif ngx_conf_merge_value(conf->upstream.pass_request_headers, - prev->upstream.pass_request_headers, 1); + prev->upstream.pass_request_headers, 1); ngx_conf_merge_value(conf->upstream.pass_request_body, - prev->upstream.pass_request_body, 1); + prev->upstream.pass_request_body, 1); ngx_conf_merge_value(conf->upstream.intercept_errors, - prev->upstream.intercept_errors, 0); + prev->upstream.intercept_errors, 0); ngx_conf_merge_ptr_value(conf->catch_stderr, prev->catch_stderr, NULL); diff -r 10678810de74 -r 3242f9829897 src/http/modules/ngx_http_grpc_module.c --- a/src/http/modules/ngx_http_grpc_module.c Wed May 20 12:21:32 2020 +0800 +++ b/src/http/modules/ngx_http_grpc_module.c Wed May 20 12:24:05 2020 +0800 @@ -4357,10 +4357,10 @@ ngx_http_core_loc_conf_t *clcf; ngx_conf_merge_ptr_value(conf->upstream.local, - prev->upstream.local, NULL); + prev->upstream.local, NULL); ngx_conf_merge_value(conf->upstream.socket_keepalive, - prev->upstream.socket_keepalive, 0); + prev->upstream.socket_keepalive, 0); ngx_conf_merge_uint_value(conf->upstream.next_upstream_tries, prev->upstream.next_upstream_tries, 0); @@ -4382,14 +4382,14 @@ (size_t) ngx_pagesize); ngx_conf_merge_bitmask_value(conf->upstream.ignore_headers, - prev->upstream.ignore_headers, - NGX_CONF_BITMASK_SET); + prev->upstream.ignore_headers, + NGX_CONF_BITMASK_SET); ngx_conf_merge_bitmask_value(conf->upstream.next_upstream, - prev->upstream.next_upstream, - (NGX_CONF_BITMASK_SET - |NGX_HTTP_UPSTREAM_FT_ERROR - |NGX_HTTP_UPSTREAM_FT_TIMEOUT)); + prev->upstream.next_upstream, + (NGX_CONF_BITMASK_SET + |NGX_HTTP_UPSTREAM_FT_ERROR + |NGX_HTTP_UPSTREAM_FT_TIMEOUT)); if (conf->upstream.next_upstream & NGX_HTTP_UPSTREAM_FT_OFF) { conf->upstream.next_upstream = NGX_CONF_BITMASK_SET @@ -4397,12 +4397,12 @@ } ngx_conf_merge_value(conf->upstream.intercept_errors, - prev->upstream.intercept_errors, 0); + prev->upstream.intercept_errors, 0); #if (NGX_HTTP_SSL) ngx_conf_merge_value(conf->upstream.ssl_session_reuse, - prev->upstream.ssl_session_reuse, 1); + prev->upstream.ssl_session_reuse, 1); ngx_conf_merge_bitmask_value(conf->ssl_protocols, prev->ssl_protocols, (NGX_CONF_BITMASK_SET|NGX_SSL_TLSv1 @@ -4416,19 +4416,19 @@ } ngx_conf_merge_value(conf->upstream.ssl_server_name, - prev->upstream.ssl_server_name, 0); + prev->upstream.ssl_server_name, 0); ngx_conf_merge_value(conf->upstream.ssl_verify, - prev->upstream.ssl_verify, 0); + prev->upstream.ssl_verify, 0); ngx_conf_merge_uint_value(conf->ssl_verify_depth, prev->ssl_verify_depth, 1); ngx_conf_merge_str_value(conf->ssl_trusted_certificate, - prev->ssl_trusted_certificate, ""); + prev->ssl_trusted_certificate, ""); ngx_conf_merge_str_value(conf->ssl_crl, prev->ssl_crl, ""); ngx_conf_merge_str_value(conf->ssl_certificate, - prev->ssl_certificate, ""); + prev->ssl_certificate, ""); ngx_conf_merge_str_value(conf->ssl_certificate_key, - prev->ssl_certificate_key, ""); + prev->ssl_certificate_key, ""); ngx_conf_merge_ptr_value(conf->ssl_passwords, prev->ssl_passwords, NULL); if (conf->ssl && ngx_http_grpc_set_ssl(cf, conf) != NGX_OK) { diff -r 10678810de74 -r 3242f9829897 src/http/modules/ngx_http_gzip_filter_module.c --- a/src/http/modules/ngx_http_gzip_filter_module.c Wed May 20 12:21:32 2020 +0800 +++ b/src/http/modules/ngx_http_gzip_filter_module.c Wed May 20 12:24:05 2020 +0800 @@ -745,10 +745,10 @@ ngx_http_gzip_conf_t *conf; ngx_log_debug6(NGX_LOG_DEBUG_HTTP, r->connection->log, 0, - "deflate in: ni:%p no:%p ai:%ud ao:%ud fl:%d redo:%d", - ctx->zstream.next_in, ctx->zstream.next_out, - ctx->zstream.avail_in, ctx->zstream.avail_out, - ctx->flush, ctx->redo); + "deflate in: ni:%p no:%p ai:%ud ao:%ud fl:%d redo:%d", + ctx->zstream.next_in, ctx->zstream.next_out, + ctx->zstream.avail_in, ctx->zstream.avail_out, + ctx->flush, ctx->redo); rc = deflate(&ctx->zstream, ctx->flush); diff -r 10678810de74 -r 3242f9829897 src/http/modules/ngx_http_image_filter_module.c --- a/src/http/modules/ngx_http_image_filter_module.c Wed May 20 12:21:32 2020 +0800 +++ b/src/http/modules/ngx_http_image_filter_module.c Wed May 20 12:24:05 2020 +0800 @@ -616,9 +616,9 @@ b->last = ngx_sprintf(b->pos, "{ \"img\" : " - "{ \"width\": %uz," - " \"height\": %uz," - " \"type\": \"%s\" } }" CRLF, + "{ \"width\": %uz," + " \"height\": %uz," + " \"type\": \"%s\" } }" CRLF, ctx->width, ctx->height, ngx_http_image_types[ctx->type - 1].data + 6); diff -r 10678810de74 -r 3242f9829897 src/http/modules/ngx_http_memcached_module.c --- a/src/http/modules/ngx_http_memcached_module.c Wed May 20 12:21:32 2020 +0800 +++ b/src/http/modules/ngx_http_memcached_module.c Wed May 20 12:24:05 2020 +0800 @@ -640,10 +640,10 @@ ngx_http_memcached_loc_conf_t *conf = child; ngx_conf_merge_ptr_value(conf->upstream.local, - prev->upstream.local, NULL); + prev->upstream.local, NULL); ngx_conf_merge_value(conf->upstream.socket_keepalive, - prev->upstream.socket_keepalive, 0); + prev->upstream.socket_keepalive, 0); ngx_conf_merge_uint_value(conf->upstream.next_upstream_tries, prev->upstream.next_upstream_tries, 0); @@ -665,10 +665,10 @@ (size_t) ngx_pagesize); ngx_conf_merge_bitmask_value(conf->upstream.next_upstream, - prev->upstream.next_upstream, - (NGX_CONF_BITMASK_SET - |NGX_HTTP_UPSTREAM_FT_ERROR - |NGX_HTTP_UPSTREAM_FT_TIMEOUT)); + prev->upstream.next_upstream, + (NGX_CONF_BITMASK_SET + |NGX_HTTP_UPSTREAM_FT_ERROR + |NGX_HTTP_UPSTREAM_FT_TIMEOUT)); if (conf->upstream.next_upstream & NGX_HTTP_UPSTREAM_FT_OFF) { conf->upstream.next_upstream = NGX_CONF_BITMASK_SET diff -r 10678810de74 -r 3242f9829897 src/http/modules/ngx_http_proxy_module.c --- a/src/http/modules/ngx_http_proxy_module.c Wed May 20 12:21:32 2020 +0800 +++ b/src/http/modules/ngx_http_proxy_module.c Wed May 20 12:24:05 2020 +0800 @@ -2934,7 +2934,7 @@ if (conf->upstream.store == NGX_CONF_UNSET) { ngx_conf_merge_value(conf->upstream.store, - prev->upstream.store, 0); + prev->upstream.store, 0); conf->upstream.store_lengths = prev->upstream.store_lengths; conf->upstream.store_values = prev->upstream.store_values; @@ -2947,22 +2947,22 @@ prev->upstream.next_upstream_tries, 0); ngx_conf_merge_value(conf->upstream.buffering, - prev->upstream.buffering, 1); + prev->upstream.buffering, 1); ngx_conf_merge_value(conf->upstream.request_buffering, - prev->upstream.request_buffering, 1); + prev->upstream.request_buffering, 1); ngx_conf_merge_value(conf->upstream.ignore_client_abort, - prev->upstream.ignore_client_abort, 0); + prev->upstream.ignore_client_abort, 0); ngx_conf_merge_value(conf->upstream.force_ranges, - prev->upstream.force_ranges, 0); + prev->upstream.force_ranges, 0); ngx_conf_merge_ptr_value(conf->upstream.local, - prev->upstream.local, NULL); + prev->upstream.local, NULL); ngx_conf_merge_value(conf->upstream.socket_keepalive, - prev->upstream.socket_keepalive, 0); + prev->upstream.socket_keepalive, 0); ngx_conf_merge_msec_value(conf->upstream.connect_timeout, prev->upstream.connect_timeout, 60000); @@ -3078,15 +3078,15 @@ ngx_conf_merge_bitmask_value(conf->upstream.ignore_headers, - prev->upstream.ignore_headers, - NGX_CONF_BITMASK_SET); + prev->upstream.ignore_headers, + NGX_CONF_BITMASK_SET); ngx_conf_merge_bitmask_value(conf->upstream.next_upstream, - prev->upstream.next_upstream, - (NGX_CONF_BITMASK_SET - |NGX_HTTP_UPSTREAM_FT_ERROR - |NGX_HTTP_UPSTREAM_FT_TIMEOUT)); + prev->upstream.next_upstream, + (NGX_CONF_BITMASK_SET + |NGX_HTTP_UPSTREAM_FT_ERROR + |NGX_HTTP_UPSTREAM_FT_TIMEOUT)); if (conf->upstream.next_upstream & NGX_HTTP_UPSTREAM_FT_OFF) { conf->upstream.next_upstream = NGX_CONF_BITMASK_SET @@ -3106,7 +3106,7 @@ if (conf->upstream.cache == NGX_CONF_UNSET) { ngx_conf_merge_value(conf->upstream.cache, - prev->upstream.cache, 0); + prev->upstream.cache, 0); conf->upstream.cache_zone = prev->upstream.cache_zone; conf->upstream.cache_value = prev->upstream.cache_value; @@ -3128,13 +3128,13 @@ prev->upstream.cache_min_uses, 1); ngx_conf_merge_off_value(conf->upstream.cache_max_range_offset, - prev->upstream.cache_max_range_offset, - NGX_MAX_OFF_T_VALUE); + prev->upstream.cache_max_range_offset, + NGX_MAX_OFF_T_VALUE); ngx_conf_merge_bitmask_value(conf->upstream.cache_use_stale, - prev->upstream.cache_use_stale, - (NGX_CONF_BITMASK_SET - |NGX_HTTP_UPSTREAM_FT_OFF)); + prev->upstream.cache_use_stale, + (NGX_CONF_BITMASK_SET + |NGX_HTTP_UPSTREAM_FT_OFF)); if (conf->upstream.cache_use_stale & NGX_HTTP_UPSTREAM_FT_OFF) { conf->upstream.cache_use_stale = NGX_CONF_BITMASK_SET @@ -3165,7 +3165,7 @@ } ngx_conf_merge_value(conf->upstream.cache_lock, - prev->upstream.cache_lock, 0); + prev->upstream.cache_lock, 0); ngx_conf_merge_msec_value(conf->upstream.cache_lock_timeout, prev->upstream.cache_lock_timeout, 5000); @@ -3174,13 +3174,13 @@ prev->upstream.cache_lock_age, 5000); ngx_conf_merge_value(conf->upstream.cache_revalidate, - prev->upstream.cache_revalidate, 0); + prev->upstream.cache_revalidate, 0); ngx_conf_merge_value(conf->upstream.cache_convert_head, - prev->upstream.cache_convert_head, 1); + prev->upstream.cache_convert_head, 1); ngx_conf_merge_value(conf->upstream.cache_background_update, - prev->upstream.cache_background_update, 0); + prev->upstream.cache_background_update, 0); #endif @@ -3189,17 +3189,17 @@ } ngx_conf_merge_value(conf->upstream.pass_request_headers, - prev->upstream.pass_request_headers, 1); + prev->upstream.pass_request_headers, 1); ngx_conf_merge_value(conf->upstream.pass_request_body, - prev->upstream.pass_request_body, 1); + prev->upstream.pass_request_body, 1); ngx_conf_merge_value(conf->upstream.intercept_errors, - prev->upstream.intercept_errors, 0); + prev->upstream.intercept_errors, 0); #if (NGX_HTTP_SSL) ngx_conf_merge_value(conf->upstream.ssl_session_reuse, - prev->upstream.ssl_session_reuse, 1); + prev->upstream.ssl_session_reuse, 1); ngx_conf_merge_bitmask_value(conf->ssl_protocols, prev->ssl_protocols, (NGX_CONF_BITMASK_SET|NGX_SSL_TLSv1 @@ -3213,19 +3213,19 @@ } ngx_conf_merge_value(conf->upstream.ssl_server_name, - prev->upstream.ssl_server_name, 0); + prev->upstream.ssl_server_name, 0); ngx_conf_merge_value(conf->upstream.ssl_verify, - prev->upstream.ssl_verify, 0); + prev->upstream.ssl_verify, 0); ngx_conf_merge_uint_value(conf->ssl_verify_depth, prev->ssl_verify_depth, 1); ngx_conf_merge_str_value(conf->ssl_trusted_certificate, - prev->ssl_trusted_certificate, ""); + prev->ssl_trusted_certificate, ""); ngx_conf_merge_str_value(conf->ssl_crl, prev->ssl_crl, ""); ngx_conf_merge_str_value(conf->ssl_certificate, - prev->ssl_certificate, ""); + prev->ssl_certificate, ""); ngx_conf_merge_str_value(conf->ssl_certificate_key, - prev->ssl_certificate_key, ""); + prev->ssl_certificate_key, ""); ngx_conf_merge_ptr_value(conf->ssl_passwords, prev->ssl_passwords, NULL); if (conf->ssl && ngx_http_proxy_set_ssl(cf, conf) != NGX_OK) { @@ -3954,7 +3954,7 @@ if (plcf->cookie_paths == NGX_CONF_UNSET_PTR) { plcf->cookie_paths = ngx_array_create(cf->pool, 1, - sizeof(ngx_http_proxy_rewrite_t)); + sizeof(ngx_http_proxy_rewrite_t)); if (plcf->cookie_paths == NULL) { return NGX_CONF_ERROR; } diff -r 10678810de74 -r 3242f9829897 src/http/modules/ngx_http_range_filter_module.c --- a/src/http/modules/ngx_http_range_filter_module.c Wed May 20 12:21:32 2020 +0800 +++ b/src/http/modules/ngx_http_range_filter_module.c Wed May 20 12:24:05 2020 +0800 @@ -509,18 +509,18 @@ } else if (r->headers_out.content_type.len) { ctx->boundary_header.len = ngx_sprintf(ctx->boundary_header.data, - CRLF "--%0muA" CRLF - "Content-Type: %V" CRLF - "Content-Range: bytes ", - boundary, - &r->headers_out.content_type) + CRLF "--%0muA" CRLF + "Content-Type: %V" CRLF + "Content-Range: bytes ", + boundary, + &r->headers_out.content_type) - ctx->boundary_header.data; } else { ctx->boundary_header.len = ngx_sprintf(ctx->boundary_header.data, - CRLF "--%0muA" CRLF - "Content-Range: bytes ", - boundary) + CRLF "--%0muA" CRLF + "Content-Range: bytes ", + boundary) - ctx->boundary_header.data; } diff -r 10678810de74 -r 3242f9829897 src/http/modules/ngx_http_rewrite_module.c --- a/src/http/modules/ngx_http_rewrite_module.c Wed May 20 12:21:32 2020 +0800 +++ b/src/http/modules/ngx_http_rewrite_module.c Wed May 20 12:24:05 2020 +0800 @@ -802,7 +802,7 @@ } fop = ngx_http_script_start_code(cf->pool, &lcf->codes, - sizeof(ngx_http_script_file_code_t)); + sizeof(ngx_http_script_file_code_t)); if (fop == NULL) { return NGX_CONF_ERROR; } diff -r 10678810de74 -r 3242f9829897 src/http/modules/ngx_http_scgi_module.c --- a/src/http/modules/ngx_http_scgi_module.c Wed May 20 12:21:32 2020 +0800 +++ b/src/http/modules/ngx_http_scgi_module.c Wed May 20 12:24:05 2020 +0800 @@ -1292,22 +1292,22 @@ prev->upstream.next_upstream_tries, 0); ngx_conf_merge_value(conf->upstream.buffering, - prev->upstream.buffering, 1); + prev->upstream.buffering, 1); ngx_conf_merge_value(conf->upstream.request_buffering, - prev->upstream.request_buffering, 1); + prev->upstream.request_buffering, 1); ngx_conf_merge_value(conf->upstream.ignore_client_abort, - prev->upstream.ignore_client_abort, 0); + prev->upstream.ignore_client_abort, 0); ngx_conf_merge_value(conf->upstream.force_ranges, - prev->upstream.force_ranges, 0); + prev->upstream.force_ranges, 0); ngx_conf_merge_ptr_value(conf->upstream.local, - prev->upstream.local, NULL); + prev->upstream.local, NULL); ngx_conf_merge_value(conf->upstream.socket_keepalive, - prev->upstream.socket_keepalive, 0); + prev->upstream.socket_keepalive, 0); ngx_conf_merge_msec_value(conf->upstream.connect_timeout, prev->upstream.connect_timeout, 60000); @@ -1372,8 +1372,8 @@ > (conf->upstream.bufs.num - 1) * conf->upstream.bufs.size) { ngx_conf_log_error(NGX_LOG_EMERG, cf, 0, - "\"scgi_busy_buffers_size\" must be less than " - "the size of all \"scgi_buffers\" minus one buffer"); + "\"scgi_busy_buffers_size\" must be less than " + "the size of all \"scgi_buffers\" minus one buffer"); return NGX_CONF_ERROR; } @@ -1452,7 +1452,7 @@ if (conf->upstream.cache == NGX_CONF_UNSET) { ngx_conf_merge_value(conf->upstream.cache, - prev->upstream.cache, 0); + prev->upstream.cache, 0); conf->upstream.cache_zone = prev->upstream.cache_zone; conf->upstream.cache_value = prev->upstream.cache_value; @@ -1474,13 +1474,13 @@ prev->upstream.cache_min_uses, 1); ngx_conf_merge_off_value(conf->upstream.cache_max_range_offset, - prev->upstream.cache_max_range_offset, - NGX_MAX_OFF_T_VALUE); + prev->upstream.cache_max_range_offset, + NGX_MAX_OFF_T_VALUE); ngx_conf_merge_bitmask_value(conf->upstream.cache_use_stale, - prev->upstream.cache_use_stale, - (NGX_CONF_BITMASK_SET - |NGX_HTTP_UPSTREAM_FT_OFF)); + prev->upstream.cache_use_stale, + (NGX_CONF_BITMASK_SET + |NGX_HTTP_UPSTREAM_FT_OFF)); if (conf->upstream.cache_use_stale & NGX_HTTP_UPSTREAM_FT_OFF) { conf->upstream.cache_use_stale = NGX_CONF_BITMASK_SET @@ -1516,7 +1516,7 @@ } ngx_conf_merge_value(conf->upstream.cache_lock, - prev->upstream.cache_lock, 0); + prev->upstream.cache_lock, 0); ngx_conf_merge_msec_value(conf->upstream.cache_lock_timeout, prev->upstream.cache_lock_timeout, 5000); @@ -1525,10 +1525,10 @@ prev->upstream.cache_lock_age, 5000); ngx_conf_merge_value(conf->upstream.cache_revalidate, - prev->upstream.cache_revalidate, 0); + prev->upstream.cache_revalidate, 0); ngx_conf_merge_value(conf->upstream.cache_background_update, - prev->upstream.cache_background_update, 0); + prev->upstream.cache_background_update, 0); #endif diff -r 10678810de74 -r 3242f9829897 src/http/modules/ngx_http_ssl_module.c --- a/src/http/modules/ngx_http_ssl_module.c Wed May 20 12:21:32 2020 +0800 +++ b/src/http/modules/ngx_http_ssl_module.c Wed May 20 12:24:05 2020 +0800 @@ -613,31 +613,31 @@ ngx_conf_merge_value(conf->early_data, prev->early_data, 0); ngx_conf_merge_bitmask_value(conf->protocols, prev->protocols, - (NGX_CONF_BITMASK_SET|NGX_SSL_TLSv1 - |NGX_SSL_TLSv1_1|NGX_SSL_TLSv1_2)); + (NGX_CONF_BITMASK_SET|NGX_SSL_TLSv1 + |NGX_SSL_TLSv1_1|NGX_SSL_TLSv1_2)); ngx_conf_merge_size_value(conf->buffer_size, prev->buffer_size, - NGX_SSL_BUFSIZE); + NGX_SSL_BUFSIZE); ngx_conf_merge_uint_value(conf->verify, prev->verify, 0); ngx_conf_merge_uint_value(conf->verify_depth, prev->verify_depth, 1); ngx_conf_merge_ptr_value(conf->certificates, prev->certificates, NULL); ngx_conf_merge_ptr_value(conf->certificate_keys, prev->certificate_keys, - NULL); + NULL); ngx_conf_merge_ptr_value(conf->passwords, prev->passwords, NULL); ngx_conf_merge_str_value(conf->dhparam, prev->dhparam, ""); ngx_conf_merge_str_value(conf->client_certificate, prev->client_certificate, - ""); + ""); ngx_conf_merge_str_value(conf->trusted_certificate, - prev->trusted_certificate, ""); + prev->trusted_certificate, ""); ngx_conf_merge_str_value(conf->crl, prev->crl, ""); ngx_conf_merge_str_value(conf->ecdh_curve, prev->ecdh_curve, - NGX_DEFAULT_ECDH_CURVE); + NGX_DEFAULT_ECDH_CURVE); ngx_conf_merge_str_value(conf->ciphers, prev->ciphers, NGX_DEFAULT_CIPHERS); @@ -645,7 +645,7 @@ ngx_conf_merge_value(conf->stapling_verify, prev->stapling_verify, 0); ngx_conf_merge_str_value(conf->stapling_file, prev->stapling_file, ""); ngx_conf_merge_str_value(conf->stapling_responder, - prev->stapling_responder, ""); + prev->stapling_responder, ""); conf->ssl.log = cf->log; @@ -834,7 +834,7 @@ #endif ngx_conf_merge_ptr_value(conf->session_ticket_keys, - prev->session_ticket_keys, NULL); + prev->session_ticket_keys, NULL); if (ngx_ssl_session_ticket_keys(cf, &conf->ssl, conf->session_ticket_keys) != NGX_OK) diff -r 10678810de74 -r 3242f9829897 src/http/modules/ngx_http_uwsgi_module.c --- a/src/http/modules/ngx_http_uwsgi_module.c Wed May 20 12:21:32 2020 +0800 +++ b/src/http/modules/ngx_http_uwsgi_module.c Wed May 20 12:24:05 2020 +0800 @@ -1513,22 +1513,22 @@ prev->upstream.next_upstream_tries, 0); ngx_conf_merge_value(conf->upstream.buffering, - prev->upstream.buffering, 1); + prev->upstream.buffering, 1); ngx_conf_merge_value(conf->upstream.request_buffering, - prev->upstream.request_buffering, 1); + prev->upstream.request_buffering, 1); ngx_conf_merge_value(conf->upstream.ignore_client_abort, - prev->upstream.ignore_client_abort, 0); + prev->upstream.ignore_client_abort, 0); ngx_conf_merge_value(conf->upstream.force_ranges, - prev->upstream.force_ranges, 0); + prev->upstream.force_ranges, 0); ngx_conf_merge_ptr_value(conf->upstream.local, - prev->upstream.local, NULL); + prev->upstream.local, NULL); ngx_conf_merge_value(conf->upstream.socket_keepalive, - prev->upstream.socket_keepalive, 0); + prev->upstream.socket_keepalive, 0); ngx_conf_merge_msec_value(conf->upstream.connect_timeout, prev->upstream.connect_timeout, 60000); @@ -1673,7 +1673,7 @@ if (conf->upstream.cache == NGX_CONF_UNSET) { ngx_conf_merge_value(conf->upstream.cache, - prev->upstream.cache, 0); + prev->upstream.cache, 0); conf->upstream.cache_zone = prev->upstream.cache_zone; conf->upstream.cache_value = prev->upstream.cache_value; @@ -1695,12 +1695,12 @@ prev->upstream.cache_min_uses, 1); ngx_conf_merge_off_value(conf->upstream.cache_max_range_offset, - prev->upstream.cache_max_range_offset, - NGX_MAX_OFF_T_VALUE); + prev->upstream.cache_max_range_offset, + NGX_MAX_OFF_T_VALUE); ngx_conf_merge_bitmask_value(conf->upstream.cache_use_stale, - prev->upstream.cache_use_stale, - (NGX_CONF_BITMASK_SET + prev->upstream.cache_use_stale, + (NGX_CONF_BITMASK_SET |NGX_HTTP_UPSTREAM_FT_OFF)); if (conf->upstream.cache_use_stale & NGX_HTTP_UPSTREAM_FT_OFF) { @@ -1737,7 +1737,7 @@ } ngx_conf_merge_value(conf->upstream.cache_lock, - prev->upstream.cache_lock, 0); + prev->upstream.cache_lock, 0); ngx_conf_merge_msec_value(conf->upstream.cache_lock_timeout, prev->upstream.cache_lock_timeout, 5000); @@ -1746,10 +1746,10 @@ prev->upstream.cache_lock_age, 5000); ngx_conf_merge_value(conf->upstream.cache_revalidate, - prev->upstream.cache_revalidate, 0); + prev->upstream.cache_revalidate, 0); ngx_conf_merge_value(conf->upstream.cache_background_update, - prev->upstream.cache_background_update, 0); + prev->upstream.cache_background_update, 0); #endif @@ -1764,7 +1764,7 @@ #if (NGX_HTTP_SSL) ngx_conf_merge_value(conf->upstream.ssl_session_reuse, - prev->upstream.ssl_session_reuse, 1); + prev->upstream.ssl_session_reuse, 1); ngx_conf_merge_bitmask_value(conf->ssl_protocols, prev->ssl_protocols, (NGX_CONF_BITMASK_SET|NGX_SSL_TLSv1 @@ -1778,19 +1778,19 @@ } ngx_conf_merge_value(conf->upstream.ssl_server_name, - prev->upstream.ssl_server_name, 0); + prev->upstream.ssl_server_name, 0); ngx_conf_merge_value(conf->upstream.ssl_verify, - prev->upstream.ssl_verify, 0); + prev->upstream.ssl_verify, 0); ngx_conf_merge_uint_value(conf->ssl_verify_depth, prev->ssl_verify_depth, 1); ngx_conf_merge_str_value(conf->ssl_trusted_certificate, - prev->ssl_trusted_certificate, ""); + prev->ssl_trusted_certificate, ""); ngx_conf_merge_str_value(conf->ssl_crl, prev->ssl_crl, ""); ngx_conf_merge_str_value(conf->ssl_certificate, - prev->ssl_certificate, ""); + prev->ssl_certificate, ""); ngx_conf_merge_str_value(conf->ssl_certificate_key, - prev->ssl_certificate_key, ""); + prev->ssl_certificate_key, ""); ngx_conf_merge_ptr_value(conf->ssl_passwords, prev->ssl_passwords, NULL); if (conf->ssl && ngx_http_uwsgi_set_ssl(cf, conf) != NGX_OK) { diff -r 10678810de74 -r 3242f9829897 src/http/modules/ngx_http_xslt_filter_module.c --- a/src/http/modules/ngx_http_xslt_filter_module.c Wed May 20 12:21:32 2020 +0800 +++ b/src/http/modules/ngx_http_xslt_filter_module.c Wed May 20 12:24:05 2020 +0800 @@ -318,7 +318,7 @@ if (b == NULL) { return ngx_http_filter_finalize_request(r, &ngx_http_xslt_filter_module, - NGX_HTTP_INTERNAL_SERVER_ERROR); + NGX_HTTP_INTERNAL_SERVER_ERROR); } cln = ngx_pool_cleanup_add(r->pool, 0); @@ -326,7 +326,7 @@ if (cln == NULL) { ngx_free(b->pos); return ngx_http_filter_finalize_request(r, &ngx_http_xslt_filter_module, - NGX_HTTP_INTERNAL_SERVER_ERROR); + NGX_HTTP_INTERNAL_SERVER_ERROR); } if (r == r->main) { @@ -704,7 +704,7 @@ p = (u_char *) ngx_strchr(p, '='); if (p == NULL) { ngx_log_error(NGX_LOG_ERR, r->connection->log, 0, - "invalid libxslt parameter \"%s\"", value); + "invalid libxslt parameter \"%s\"", value); return NGX_ERROR; } *p++ = '\0'; Thanks! -------------- next part -------------- An HTML attachment was scrubbed... URL: From mdounin at mdounin.ru Wed May 20 13:48:48 2020 From: mdounin at mdounin.ru (Maxim Dounin) Date: Wed, 20 May 2020 16:48:48 +0300 Subject: Style(core): fix the incorrect front spaces to unite style. In-Reply-To: References: Message-ID: <20200520134848.GW12747@mdounin.ru> Hello! On Wed, May 20, 2020 at 12:29:45PM +0800, Jim T wrote: > Nginx is famous for code readability and neat code format. I use the tool( > https://github.com/openresty/openresty-devel-utils) to try to unify the > incorrect front spaces, Here is my attempt at core. > > # HG changeset patch > # User Jinhua Tan <312841925 at qq.com> > # Date 1589948492 -28800 > # Wed May 20 12:21:32 2020 +0800 > # Node ID 10678810de74edc4c02d5cd7303f0099eee9fac3 > # Parent 3c8082c3f98ab7fdec8a598b55609701452b5254 > Style(core): fix the incorrect front spaces to unite style. > > diff -r 3c8082c3f98a -r 10678810de74 src/core/ngx_conf_file.c > --- a/src/core/ngx_conf_file.c Wed May 13 22:02:47 2020 +0800 > +++ b/src/core/ngx_conf_file.c Wed May 20 12:21:32 2020 +0800 > @@ -544,8 +544,8 @@ > } > > ngx_conf_log_error(NGX_LOG_EMERG, cf, 0, > - "unexpected end of file, " > - "expecting \";\" or \"}\""); > + "unexpected end of file, " > + "expecting \";\" or \"}\""); > return NGX_ERROR; > } > > @@ -1047,9 +1047,9 @@ > > } else { > ngx_conf_log_error(NGX_LOG_EMERG, cf, 0, > - "invalid value \"%s\" in \"%s\" directive, " > - "it must be \"on\" or \"off\"", > - value[1].data, cmd->name.data); > + "invalid value \"%s\" in \"%s\" directive, " > + "it must be \"on\" or \"off\"", > + value[1].data, cmd->name.data); > return NGX_CONF_ERROR; > } > > diff -r 3c8082c3f98a -r 10678810de74 src/core/ngx_cycle.c > --- a/src/core/ngx_cycle.c Wed May 13 22:02:47 2020 +0800 > +++ b/src/core/ngx_cycle.c Wed May 20 12:21:32 2020 +0800 > @@ -1308,9 +1308,9 @@ > > if (tag != shm_zone[i].tag) { > ngx_conf_log_error(NGX_LOG_EMERG, cf, 0, > - "the shared memory zone \"%V\" is " > - "already declared for a different use", > - &shm_zone[i].shm.name); > + "the shared memory zone \"%V\" is " > + "already declared for a different use", > + &shm_zone[i].shm.name); > return NULL; > } > This particular chunk is a good example why simple mechanical whitespace changes are not to be applied. Message here is aligned with the following "the size %uz of shared memory zone..." message, which cannot be aligned according to normal rules due to its length. This is also demonstrates that code readability is not always about following rules. Sometimes it's about breaking rules. It should be also noted that pure style changes are usually to be avoided. For example, FreeBSD's style(9) says it this way: Stylistic changes (including whitespace changes) are hard on the source repository and are to be avoided without good reason. Thank you for your patch, but no. [...] -- Maxim Dounin http://mdounin.ru/ From h312841925 at gmail.com Thu May 21 13:45:24 2020 From: h312841925 at gmail.com (Jim T) Date: Thu, 21 May 2020 21:45:24 +0800 Subject: Core: close pid file while writing it failed. Message-ID: Hello! As far as I understand it, `ngx_create_pidfile` is a function that works independently. There is no action to close the pid file externally, so we need to close the pid file when the writing it failed. There are also reports here https://github.com/nginx/nginx/pull/52. # HG changeset patch # User Jinhua Tan <312841925 at qq.com> # Date 1590068494 -28800 # Thu May 21 21:41:34 2020 +0800 # Node ID 6084ea4d9a4d2ae32f3fc4e2e3b9032ab0b71e30 # Parent 3242f98298975e556a7e87130611ce84799fe935 Core: close pid file while writing it failed. diff -r 3242f9829897 -r 6084ea4d9a4d src/core/ngx_cycle.c --- a/src/core/ngx_cycle.c Wed May 20 12:24:05 2020 +0800 +++ b/src/core/ngx_cycle.c Thu May 21 21:41:34 2020 +0800 @@ -1036,6 +1036,12 @@ len = ngx_snprintf(pid, NGX_INT64_LEN + 2, "%P%N", ngx_pid) - pid; if (ngx_write_file(&file, pid, len, 0) == NGX_ERROR) { + + if (ngx_close_file(file.fd) == NGX_FILE_ERROR) { + ngx_log_error(NGX_LOG_ALERT, log, ngx_errno, + ngx_close_file_n " \"%s\" failed", file.name.data); + } + return NGX_ERROR; } } Thank you! -------------- next part -------------- An HTML attachment was scrubbed... URL: From arut at nginx.com Sat May 23 10:35:41 2020 From: arut at nginx.com (Roman Arutyunyan) Date: Sat, 23 May 2020 10:35:41 +0000 Subject: [nginx] OCSP stapling: moved response verification to a separate function. Message-ID: details: https://hg.nginx.org/nginx/rev/abb6cc8f1dd8 branches: changeset: 7650:abb6cc8f1dd8 user: Roman Arutyunyan date: Wed May 06 21:44:14 2020 +0300 description: OCSP stapling: moved response verification to a separate function. diffstat: src/event/ngx_event_openssl_stapling.c | 292 +++++++++++++++++--------------- 1 files changed, 155 insertions(+), 137 deletions(-) diffs (382 lines): diff -r 3c8082c3f98a -r abb6cc8f1dd8 src/event/ngx_event_openssl_stapling.c --- a/src/event/ngx_event_openssl_stapling.c Wed May 13 22:02:47 2020 +0800 +++ b/src/event/ngx_event_openssl_stapling.c Wed May 06 21:44:14 2020 +0300 @@ -44,9 +44,14 @@ typedef struct { typedef struct ngx_ssl_ocsp_ctx_s ngx_ssl_ocsp_ctx_t; struct ngx_ssl_ocsp_ctx_s { + SSL_CTX *ssl_ctx; + X509 *cert; X509 *issuer; + int status; + time_t valid; + u_char *name; ngx_uint_t naddrs; @@ -74,7 +79,7 @@ struct ngx_ssl_ocsp_ctx_s { ngx_uint_t code; ngx_uint_t count; - + ngx_uint_t flags; ngx_uint_t done; u_char *header_name_start; @@ -120,6 +125,7 @@ static ngx_int_t ngx_ssl_ocsp_parse_stat static ngx_int_t ngx_ssl_ocsp_process_headers(ngx_ssl_ocsp_ctx_t *ctx); static ngx_int_t ngx_ssl_ocsp_parse_header_line(ngx_ssl_ocsp_ctx_t *ctx); static ngx_int_t ngx_ssl_ocsp_process_body(ngx_ssl_ocsp_ctx_t *ctx); +static ngx_int_t ngx_ssl_ocsp_verify(ngx_ssl_ocsp_ctx_t *ctx); static u_char *ngx_ssl_ocsp_log_error(ngx_log_t *log, u_char *buf, size_t len); @@ -564,9 +570,11 @@ ngx_ssl_stapling_update(ngx_ssl_stapling return; } + ctx->ssl_ctx = staple->ssl_ctx; ctx->cert = staple->cert; ctx->issuer = staple->issuer; ctx->name = staple->name; + ctx->flags = (staple->verify ? OCSP_TRUSTOTHER : OCSP_NOVERIFY); ctx->addrs = staple->addrs; ctx->host = staple->host; @@ -589,137 +597,27 @@ ngx_ssl_stapling_update(ngx_ssl_stapling static void ngx_ssl_stapling_ocsp_handler(ngx_ssl_ocsp_ctx_t *ctx) { - int n; - size_t len; - time_t now, valid; - ngx_str_t response; - X509_STORE *store; - const u_char *p; - STACK_OF(X509) *chain; - OCSP_CERTID *id; - OCSP_RESPONSE *ocsp; - OCSP_BASICRESP *basic; - ngx_ssl_stapling_t *staple; - ASN1_GENERALIZEDTIME *thisupdate, *nextupdate; + time_t now; + ngx_str_t response; + ngx_ssl_stapling_t *staple; staple = ctx->data; now = ngx_time(); - ocsp = NULL; - basic = NULL; - id = NULL; - if (ctx->code != 200) { - goto error; - } - - /* check the response */ - - len = ctx->response->last - ctx->response->pos; - p = ctx->response->pos; - - ocsp = d2i_OCSP_RESPONSE(NULL, &p, len); - if (ocsp == NULL) { - ngx_ssl_error(NGX_LOG_ERR, ctx->log, 0, - "d2i_OCSP_RESPONSE() failed"); - goto error; - } - - n = OCSP_response_status(ocsp); - - if (n != OCSP_RESPONSE_STATUS_SUCCESSFUL) { - ngx_log_error(NGX_LOG_ERR, ctx->log, 0, - "OCSP response not successful (%d: %s)", - n, OCSP_response_status_str(n)); - goto error; - } - - basic = OCSP_response_get1_basic(ocsp); - if (basic == NULL) { - ngx_ssl_error(NGX_LOG_ERR, ctx->log, 0, - "OCSP_response_get1_basic() failed"); - goto error; - } - - store = SSL_CTX_get_cert_store(staple->ssl_ctx); - if (store == NULL) { - ngx_ssl_error(NGX_LOG_CRIT, ctx->log, 0, - "SSL_CTX_get_cert_store() failed"); + if (ngx_ssl_ocsp_verify(ctx) != NGX_OK) { goto error; } -#ifdef SSL_CTRL_SELECT_CURRENT_CERT - /* OpenSSL 1.0.2+ */ - SSL_CTX_select_current_cert(staple->ssl_ctx, ctx->cert); -#endif - -#ifdef SSL_CTRL_GET_EXTRA_CHAIN_CERTS - /* OpenSSL 1.0.1+ */ - SSL_CTX_get_extra_chain_certs(staple->ssl_ctx, &chain); -#else - chain = staple->ssl_ctx->extra_certs; -#endif - - if (OCSP_basic_verify(basic, chain, store, - staple->verify ? OCSP_TRUSTOTHER : OCSP_NOVERIFY) - != 1) - { - ngx_ssl_error(NGX_LOG_ERR, ctx->log, 0, - "OCSP_basic_verify() failed"); - goto error; - } - - id = OCSP_cert_to_id(NULL, ctx->cert, ctx->issuer); - if (id == NULL) { - ngx_ssl_error(NGX_LOG_CRIT, ctx->log, 0, - "OCSP_cert_to_id() failed"); - goto error; - } - - if (OCSP_resp_find_status(basic, id, &n, NULL, NULL, - &thisupdate, &nextupdate) - != 1) - { + if (ctx->status != V_OCSP_CERTSTATUS_GOOD) { ngx_log_error(NGX_LOG_ERR, ctx->log, 0, - "certificate status not found in the OCSP response"); + "certificate status \"%s\" in the OCSP response", + OCSP_cert_status_str(ctx->status)); goto error; } - if (n != V_OCSP_CERTSTATUS_GOOD) { - ngx_log_error(NGX_LOG_ERR, ctx->log, 0, - "certificate status \"%s\" in the OCSP response", - OCSP_cert_status_str(n)); - goto error; - } - - if (OCSP_check_validity(thisupdate, nextupdate, 300, -1) != 1) { - ngx_ssl_error(NGX_LOG_ERR, ctx->log, 0, - "OCSP_check_validity() failed"); - goto error; - } - - if (nextupdate) { - valid = ngx_ssl_stapling_time(nextupdate); - if (valid == (time_t) NGX_ERROR) { - ngx_log_error(NGX_LOG_ERR, ctx->log, 0, - "invalid nextUpdate time in certificate status"); - goto error; - } - - } else { - valid = NGX_MAX_TIME_T_VALUE; - } - - OCSP_CERTID_free(id); - OCSP_BASICRESP_free(basic); - OCSP_RESPONSE_free(ocsp); - - id = NULL; - basic = NULL; - ocsp = NULL; - /* copy the response to memory not in ctx->pool */ - response.len = len; + response.len = ctx->response->last - ctx->response->pos; response.data = ngx_alloc(response.len, ctx->log); if (response.data == NULL) { @@ -728,16 +626,12 @@ ngx_ssl_stapling_ocsp_handler(ngx_ssl_oc ngx_memcpy(response.data, ctx->response->pos, response.len); - ngx_log_debug2(NGX_LOG_DEBUG_EVENT, ctx->log, 0, - "ssl ocsp response, %s, %uz", - OCSP_cert_status_str(n), response.len); - if (staple->staple.data) { ngx_free(staple->staple.data); } staple->staple = response; - staple->valid = valid; + staple->valid = ctx->valid; /* * refresh before the response expires, @@ -745,7 +639,7 @@ ngx_ssl_stapling_ocsp_handler(ngx_ssl_oc */ staple->loading = 0; - staple->refresh = ngx_max(ngx_min(valid - 300, now + 3600), now + 300); + staple->refresh = ngx_max(ngx_min(ctx->valid - 300, now + 3600), now + 300); ngx_ssl_ocsp_done(ctx); return; @@ -755,18 +649,6 @@ error: staple->loading = 0; staple->refresh = now + 300; - if (id) { - OCSP_CERTID_free(id); - } - - if (basic) { - OCSP_BASICRESP_free(basic); - } - - if (ocsp) { - OCSP_RESPONSE_free(ocsp); - } - ngx_ssl_ocsp_done(ctx); } @@ -1831,6 +1713,142 @@ ngx_ssl_ocsp_process_body(ngx_ssl_ocsp_c } +static ngx_int_t +ngx_ssl_ocsp_verify(ngx_ssl_ocsp_ctx_t *ctx) +{ + int n; + size_t len; + X509_STORE *store; + const u_char *p; + STACK_OF(X509) *chain; + OCSP_CERTID *id; + OCSP_RESPONSE *ocsp; + OCSP_BASICRESP *basic; + ASN1_GENERALIZEDTIME *thisupdate, *nextupdate; + + ocsp = NULL; + basic = NULL; + id = NULL; + + if (ctx->code != 200) { + goto error; + } + + /* check the response */ + + len = ctx->response->last - ctx->response->pos; + p = ctx->response->pos; + + ocsp = d2i_OCSP_RESPONSE(NULL, &p, len); + if (ocsp == NULL) { + ngx_ssl_error(NGX_LOG_ERR, ctx->log, 0, + "d2i_OCSP_RESPONSE() failed"); + goto error; + } + + n = OCSP_response_status(ocsp); + + if (n != OCSP_RESPONSE_STATUS_SUCCESSFUL) { + ngx_log_error(NGX_LOG_ERR, ctx->log, 0, + "OCSP response not successful (%d: %s)", + n, OCSP_response_status_str(n)); + goto error; + } + + basic = OCSP_response_get1_basic(ocsp); + if (basic == NULL) { + ngx_ssl_error(NGX_LOG_ERR, ctx->log, 0, + "OCSP_response_get1_basic() failed"); + goto error; + } + + store = SSL_CTX_get_cert_store(ctx->ssl_ctx); + if (store == NULL) { + ngx_ssl_error(NGX_LOG_CRIT, ctx->log, 0, + "SSL_CTX_get_cert_store() failed"); + goto error; + } + +#ifdef SSL_CTRL_SELECT_CURRENT_CERT + /* OpenSSL 1.0.2+ */ + SSL_CTX_select_current_cert(ctx->ssl_ctx, ctx->cert); +#endif + +#ifdef SSL_CTRL_GET_EXTRA_CHAIN_CERTS + /* OpenSSL 1.0.1+ */ + SSL_CTX_get_extra_chain_certs(ctx->ssl_ctx, &chain); +#else + chain = ctx->ssl_ctx->extra_certs; +#endif + + if (OCSP_basic_verify(basic, chain, store, ctx->flags) != 1) { + ngx_ssl_error(NGX_LOG_ERR, ctx->log, 0, + "OCSP_basic_verify() failed"); + goto error; + } + + id = OCSP_cert_to_id(NULL, ctx->cert, ctx->issuer); + if (id == NULL) { + ngx_ssl_error(NGX_LOG_CRIT, ctx->log, 0, + "OCSP_cert_to_id() failed"); + goto error; + } + + if (OCSP_resp_find_status(basic, id, &ctx->status, NULL, NULL, + &thisupdate, &nextupdate) + != 1) + { + ngx_log_error(NGX_LOG_ERR, ctx->log, 0, + "certificate status not found in the OCSP response"); + goto error; + } + + if (OCSP_check_validity(thisupdate, nextupdate, 300, -1) != 1) { + ngx_ssl_error(NGX_LOG_ERR, ctx->log, 0, + "OCSP_check_validity() failed"); + goto error; + } + + if (nextupdate) { + ctx->valid = ngx_ssl_stapling_time(nextupdate); + if (ctx->valid == (time_t) NGX_ERROR) { + ngx_log_error(NGX_LOG_ERR, ctx->log, 0, + "invalid nextUpdate time in certificate status"); + goto error; + } + + } else { + ctx->valid = NGX_MAX_TIME_T_VALUE; + } + + OCSP_CERTID_free(id); + OCSP_BASICRESP_free(basic); + OCSP_RESPONSE_free(ocsp); + + ngx_log_debug2(NGX_LOG_DEBUG_EVENT, ctx->log, 0, + "ssl ocsp response, %s, %uz", + OCSP_cert_status_str(ctx->status), len); + + return NGX_OK; + +error: + + if (id) { + OCSP_CERTID_free(id); + } + + if (basic) { + OCSP_BASICRESP_free(basic); + } + + if (ocsp) { + OCSP_RESPONSE_free(ocsp); + } + + return NGX_ERROR; +} + + static u_char * ngx_ssl_ocsp_log_error(ngx_log_t *log, u_char *buf, size_t len) { From arut at nginx.com Sat May 23 10:35:45 2020 From: arut at nginx.com (Roman Arutyunyan) Date: Sat, 23 May 2020 10:35:45 +0000 Subject: [nginx] OCSP stapling: keep extra chain in the staple object. Message-ID: details: https://hg.nginx.org/nginx/rev/6ca8e15caf1f branches: changeset: 7651:6ca8e15caf1f user: Roman Arutyunyan date: Sun May 17 14:24:35 2020 +0300 description: OCSP stapling: keep extra chain in the staple object. diffstat: src/event/ngx_event_openssl_stapling.c | 47 +++++++++++++-------------------- 1 files changed, 18 insertions(+), 29 deletions(-) diffs (107 lines): diff -r abb6cc8f1dd8 -r 6ca8e15caf1f src/event/ngx_event_openssl_stapling.c --- a/src/event/ngx_event_openssl_stapling.c Wed May 06 21:44:14 2020 +0300 +++ b/src/event/ngx_event_openssl_stapling.c Sun May 17 14:24:35 2020 +0300 @@ -30,6 +30,7 @@ typedef struct { X509 *cert; X509 *issuer; + STACK_OF(X509) *chain; u_char *name; @@ -48,6 +49,7 @@ struct ngx_ssl_ocsp_ctx_s { X509 *cert; X509 *issuer; + STACK_OF(X509) *chain; int status; time_t valid; @@ -179,6 +181,18 @@ ngx_ssl_stapling_certificate(ngx_conf_t return NGX_ERROR; } +#ifdef SSL_CTRL_SELECT_CURRENT_CERT + /* OpenSSL 1.0.2+ */ + SSL_CTX_select_current_cert(ssl->ctx, cert); +#endif + +#ifdef SSL_CTRL_GET_EXTRA_CHAIN_CERTS + /* OpenSSL 1.0.1+ */ + SSL_CTX_get_extra_chain_certs(ssl->ctx, &staple->chain); +#else + staple->chain = ssl->ctx->extra_certs; +#endif + staple->ssl_ctx = ssl->ctx; staple->timeout = 60000; staple->verify = verify; @@ -295,29 +309,16 @@ ngx_ssl_stapling_issuer(ngx_conf_t *cf, X509 *cert, *issuer; X509_STORE *store; X509_STORE_CTX *store_ctx; - STACK_OF(X509) *chain; cert = staple->cert; -#ifdef SSL_CTRL_SELECT_CURRENT_CERT - /* OpenSSL 1.0.2+ */ - SSL_CTX_select_current_cert(ssl->ctx, cert); -#endif - -#ifdef SSL_CTRL_GET_EXTRA_CHAIN_CERTS - /* OpenSSL 1.0.1+ */ - SSL_CTX_get_extra_chain_certs(ssl->ctx, &chain); -#else - chain = ssl->ctx->extra_certs; -#endif - - n = sk_X509_num(chain); + n = sk_X509_num(staple->chain); ngx_log_debug1(NGX_LOG_DEBUG_EVENT, ssl->log, 0, "SSL get issuer: %d extra certs", n); for (i = 0; i < n; i++) { - issuer = sk_X509_value(chain, i); + issuer = sk_X509_value(staple->chain, i); if (X509_check_issued(issuer, cert) == X509_V_OK) { #if OPENSSL_VERSION_NUMBER >= 0x10100001L X509_up_ref(issuer); @@ -573,6 +574,7 @@ ngx_ssl_stapling_update(ngx_ssl_stapling ctx->ssl_ctx = staple->ssl_ctx; ctx->cert = staple->cert; ctx->issuer = staple->issuer; + ctx->chain = staple->chain; ctx->name = staple->name; ctx->flags = (staple->verify ? OCSP_TRUSTOTHER : OCSP_NOVERIFY); @@ -1720,7 +1722,6 @@ ngx_ssl_ocsp_verify(ngx_ssl_ocsp_ctx_t * size_t len; X509_STORE *store; const u_char *p; - STACK_OF(X509) *chain; OCSP_CERTID *id; OCSP_RESPONSE *ocsp; OCSP_BASICRESP *basic; @@ -1769,19 +1770,7 @@ ngx_ssl_ocsp_verify(ngx_ssl_ocsp_ctx_t * goto error; } -#ifdef SSL_CTRL_SELECT_CURRENT_CERT - /* OpenSSL 1.0.2+ */ - SSL_CTX_select_current_cert(ctx->ssl_ctx, ctx->cert); -#endif - -#ifdef SSL_CTRL_GET_EXTRA_CHAIN_CERTS - /* OpenSSL 1.0.1+ */ - SSL_CTX_get_extra_chain_certs(ctx->ssl_ctx, &chain); -#else - chain = ctx->ssl_ctx->extra_certs; -#endif - - if (OCSP_basic_verify(basic, chain, store, ctx->flags) != 1) { + if (OCSP_basic_verify(basic, ctx->chain, store, ctx->flags) != 1) { ngx_ssl_error(NGX_LOG_ERR, ctx->log, 0, "OCSP_basic_verify() failed"); goto error; From arut at nginx.com Sat May 23 10:35:47 2020 From: arut at nginx.com (Roman Arutyunyan) Date: Sat, 23 May 2020 10:35:47 +0000 Subject: [nginx] OCSP stapling: iterate over all responder addresses. Message-ID: details: https://hg.nginx.org/nginx/rev/7cffd81015e7 branches: changeset: 7652:7cffd81015e7 user: Roman Arutyunyan date: Fri May 22 20:35:05 2020 +0300 description: OCSP stapling: iterate over all responder addresses. Previously only the first responder address was used per each stapling update. Now, in case of a network or parsing error, next address is used. This also fixes the issue with unsupported responder address families (ticket #1330). diffstat: src/event/ngx_event_openssl_stapling.c | 67 +++++++++++++++++++++++++++------ 1 files changed, 54 insertions(+), 13 deletions(-) diffs (168 lines): diff -r 6ca8e15caf1f -r 7cffd81015e7 src/event/ngx_event_openssl_stapling.c --- a/src/event/ngx_event_openssl_stapling.c Sun May 17 14:24:35 2020 +0300 +++ b/src/event/ngx_event_openssl_stapling.c Fri May 22 20:35:05 2020 +0300 @@ -22,6 +22,7 @@ typedef struct { ngx_msec_t resolver_timeout; ngx_addr_t *addrs; + ngx_uint_t naddrs; ngx_str_t host; ngx_str_t uri; in_port_t port; @@ -57,6 +58,7 @@ struct ngx_ssl_ocsp_ctx_s { u_char *name; ngx_uint_t naddrs; + ngx_uint_t naddr; ngx_addr_t *addrs; ngx_str_t host; @@ -114,6 +116,7 @@ static void ngx_ssl_stapling_cleanup(voi static ngx_ssl_ocsp_ctx_t *ngx_ssl_ocsp_start(void); static void ngx_ssl_ocsp_done(ngx_ssl_ocsp_ctx_t *ctx); +static void ngx_ssl_ocsp_next(ngx_ssl_ocsp_ctx_t *ctx); static void ngx_ssl_ocsp_request(ngx_ssl_ocsp_ctx_t *ctx); static void ngx_ssl_ocsp_resolve_handler(ngx_resolver_ctx_t *resolve); static void ngx_ssl_ocsp_connect(ngx_ssl_ocsp_ctx_t *ctx); @@ -469,6 +472,7 @@ ngx_ssl_stapling_responder(ngx_conf_t *c } staple->addrs = u.addrs; + staple->naddrs = u.naddrs; staple->host = u.host; staple->uri = u.uri; staple->port = u.port; @@ -579,6 +583,7 @@ ngx_ssl_stapling_update(ngx_ssl_stapling ctx->flags = (staple->verify ? OCSP_TRUSTOTHER : OCSP_NOVERIFY); ctx->addrs = staple->addrs; + ctx->naddrs = staple->naddrs; ctx->host = staple->host; ctx->uri = staple->uri; ctx->port = staple->port; @@ -769,6 +774,36 @@ ngx_ssl_ocsp_error(ngx_ssl_ocsp_ctx_t *c static void +ngx_ssl_ocsp_next(ngx_ssl_ocsp_ctx_t *ctx) +{ + ngx_log_debug0(NGX_LOG_DEBUG_EVENT, ctx->log, 0, + "ssl ocsp next"); + + if (++ctx->naddr >= ctx->naddrs) { + ngx_ssl_ocsp_error(ctx); + return; + } + + ctx->request->pos = ctx->request->start; + + if (ctx->response) { + ctx->response->last = ctx->response->pos; + } + + if (ctx->peer.connection) { + ngx_close_connection(ctx->peer.connection); + ctx->peer.connection = NULL; + } + + ctx->state = 0; + ctx->count = 0; + ctx->done = 0; + + ngx_ssl_ocsp_connect(ctx); +} + + +static void ngx_ssl_ocsp_request(ngx_ssl_ocsp_ctx_t *ctx) { ngx_resolver_ctx_t *resolve, temp; @@ -906,16 +941,17 @@ failed: static void ngx_ssl_ocsp_connect(ngx_ssl_ocsp_ctx_t *ctx) { - ngx_int_t rc; + ngx_int_t rc; + ngx_addr_t *addr; - ngx_log_debug0(NGX_LOG_DEBUG_EVENT, ctx->log, 0, - "ssl ocsp connect"); + ngx_log_debug2(NGX_LOG_DEBUG_EVENT, ctx->log, 0, + "ssl ocsp connect %ui/%ui", ctx->naddr, ctx->naddrs); - /* TODO: use all ip addresses */ + addr = &ctx->addrs[ctx->naddr]; - ctx->peer.sockaddr = ctx->addrs[0].sockaddr; - ctx->peer.socklen = ctx->addrs[0].socklen; - ctx->peer.name = &ctx->addrs[0].name; + ctx->peer.sockaddr = addr->sockaddr; + ctx->peer.socklen = addr->socklen; + ctx->peer.name = &addr->name; ctx->peer.get = ngx_event_get_peer; ctx->peer.log = ctx->log; ctx->peer.log_error = NGX_ERROR_ERR; @@ -925,11 +961,16 @@ ngx_ssl_ocsp_connect(ngx_ssl_ocsp_ctx_t ngx_log_debug0(NGX_LOG_DEBUG_EVENT, ctx->log, 0, "ssl ocsp connect peer done"); - if (rc == NGX_ERROR || rc == NGX_BUSY || rc == NGX_DECLINED) { + if (rc == NGX_ERROR) { ngx_ssl_ocsp_error(ctx); return; } + if (rc == NGX_BUSY || rc == NGX_DECLINED) { + ngx_ssl_ocsp_next(ctx); + return; + } + ctx->peer.connection->data = ctx; ctx->peer.connection->pool = ctx->pool; @@ -964,7 +1005,7 @@ ngx_ssl_ocsp_write_handler(ngx_event_t * if (wev->timedout) { ngx_log_error(NGX_LOG_ERR, wev->log, NGX_ETIMEDOUT, "OCSP responder timed out"); - ngx_ssl_ocsp_error(ctx); + ngx_ssl_ocsp_next(ctx); return; } @@ -973,7 +1014,7 @@ ngx_ssl_ocsp_write_handler(ngx_event_t * n = ngx_send(c, ctx->request->pos, size); if (n == NGX_ERROR) { - ngx_ssl_ocsp_error(ctx); + ngx_ssl_ocsp_next(ctx); return; } @@ -1018,7 +1059,7 @@ ngx_ssl_ocsp_read_handler(ngx_event_t *r if (rev->timedout) { ngx_log_error(NGX_LOG_ERR, rev->log, NGX_ETIMEDOUT, "OCSP responder timed out"); - ngx_ssl_ocsp_error(ctx); + ngx_ssl_ocsp_next(ctx); return; } @@ -1042,7 +1083,7 @@ ngx_ssl_ocsp_read_handler(ngx_event_t *r rc = ctx->process(ctx); if (rc == NGX_ERROR) { - ngx_ssl_ocsp_error(ctx); + ngx_ssl_ocsp_next(ctx); return; } @@ -1073,7 +1114,7 @@ ngx_ssl_ocsp_read_handler(ngx_event_t *r ngx_log_error(NGX_LOG_ERR, ctx->log, 0, "OCSP responder prematurely closed connection"); - ngx_ssl_ocsp_error(ctx); + ngx_ssl_ocsp_next(ctx); } From arut at nginx.com Sat May 23 10:35:50 2020 From: arut at nginx.com (Roman Arutyunyan) Date: Sat, 23 May 2020 10:35:50 +0000 Subject: [nginx] SSL: client certificate validation with OCSP (ticket #1534). Message-ID: details: https://hg.nginx.org/nginx/rev/8409f9df6219 branches: changeset: 7653:8409f9df6219 user: Roman Arutyunyan date: Fri May 22 17:30:12 2020 +0300 description: SSL: client certificate validation with OCSP (ticket #1534). OCSP validation for client certificates is enabled by the "ssl_ocsp" directive. OCSP responder can be optionally specified by "ssl_ocsp_responder". When session is reused, peer chain is not available for validation. If the verified chain contains certificates from the peer chain not available at the server, validation will fail. diffstat: src/event/ngx_event_openssl.c | 60 +++- src/event/ngx_event_openssl.h | 14 + src/event/ngx_event_openssl_stapling.c | 550 ++++++++++++++++++++++++++++++++- src/http/modules/ngx_http_ssl_module.c | 64 +++- src/http/modules/ngx_http_ssl_module.h | 3 + src/http/ngx_http_request.c | 12 + 6 files changed, 682 insertions(+), 21 deletions(-) diffs (970 lines): diff -r 7cffd81015e7 -r 8409f9df6219 src/event/ngx_event_openssl.c --- a/src/event/ngx_event_openssl.c Fri May 22 20:35:05 2020 +0300 +++ b/src/event/ngx_event_openssl.c Fri May 22 17:30:12 2020 +0300 @@ -130,6 +130,7 @@ int ngx_ssl_connection_index; int ngx_ssl_server_conf_index; int ngx_ssl_session_cache_index; int ngx_ssl_session_ticket_keys_index; +int ngx_ssl_ocsp_index; int ngx_ssl_certificate_index; int ngx_ssl_next_certificate_index; int ngx_ssl_certificate_name_index; @@ -213,6 +214,13 @@ ngx_ssl_init(ngx_log_t *log) return NGX_ERROR; } + ngx_ssl_ocsp_index = SSL_CTX_get_ex_new_index(0, NULL, NULL, NULL, NULL); + if (ngx_ssl_ocsp_index == -1) { + ngx_ssl_error(NGX_LOG_ALERT, log, 0, + "SSL_CTX_get_ex_new_index() failed"); + return NGX_ERROR; + } + ngx_ssl_certificate_index = SSL_CTX_get_ex_new_index(0, NULL, NULL, NULL, NULL); if (ngx_ssl_certificate_index == -1) { @@ -1594,6 +1602,7 @@ ngx_ssl_handshake(ngx_connection_t *c) { int n, sslerr; ngx_err_t err; + ngx_int_t rc; #ifdef SSL_READ_EARLY_DATA_SUCCESS if (c->ssl->try_early_data) { @@ -1601,6 +1610,10 @@ ngx_ssl_handshake(ngx_connection_t *c) } #endif + if (c->ssl->in_ocsp) { + return ngx_ssl_ocsp_validate(c); + } + ngx_ssl_clear_error(c->log); n = SSL_do_handshake(c->ssl->connection); @@ -1621,8 +1634,6 @@ ngx_ssl_handshake(ngx_connection_t *c) ngx_ssl_handshake_log(c); #endif - c->ssl->handshaked = 1; - c->recv = ngx_ssl_recv; c->send = ngx_ssl_write; c->recv_chain = ngx_ssl_recv_chain; @@ -1641,6 +1652,20 @@ ngx_ssl_handshake(ngx_connection_t *c) #endif #endif + rc = ngx_ssl_ocsp_validate(c); + + if (rc == NGX_ERROR) { + return NGX_ERROR; + } + + if (rc == NGX_AGAIN) { + c->read->handler = ngx_ssl_handshake_handler; + c->write->handler = ngx_ssl_handshake_handler; + return NGX_AGAIN; + } + + c->ssl->handshaked = 1; + return NGX_OK; } @@ -1710,6 +1735,7 @@ ngx_ssl_try_early_data(ngx_connection_t u_char buf; size_t readbytes; ngx_err_t err; + ngx_int_t rc; ngx_ssl_clear_error(c->log); @@ -1744,7 +1770,6 @@ ngx_ssl_try_early_data(ngx_connection_t c->ssl->early_buf = buf; c->ssl->early_preread = 1; - c->ssl->handshaked = 1; c->ssl->in_early = 1; c->recv = ngx_ssl_recv; @@ -1752,6 +1777,20 @@ ngx_ssl_try_early_data(ngx_connection_t c->recv_chain = ngx_ssl_recv_chain; c->send_chain = ngx_ssl_send_chain; + rc = ngx_ssl_ocsp_validate(c); + + if (rc == NGX_ERROR) { + return NGX_ERROR; + } + + if (rc == NGX_AGAIN) { + c->read->handler = ngx_ssl_handshake_handler; + c->write->handler = ngx_ssl_handshake_handler; + return NGX_AGAIN; + } + + c->ssl->handshaked = 1; + return NGX_OK; } @@ -2735,6 +2774,8 @@ ngx_ssl_shutdown(ngx_connection_t *c) int n, sslerr, mode; ngx_err_t err; + ngx_ssl_ocsp_cleanup(c); + if (SSL_in_init(c->ssl->connection)) { /* * OpenSSL 1.0.2f complains if SSL_shutdown() is called during @@ -4894,11 +4935,14 @@ ngx_ssl_get_client_verify(ngx_connection rc = SSL_get_verify_result(c->ssl->connection); if (rc == X509_V_OK) { - ngx_str_set(s, "SUCCESS"); - return NGX_OK; - } - - str = X509_verify_cert_error_string(rc); + if (ngx_ssl_ocsp_get_status(c, &str) == NGX_OK) { + ngx_str_set(s, "SUCCESS"); + return NGX_OK; + } + + } else { + str = X509_verify_cert_error_string(rc); + } s->data = ngx_pnalloc(pool, sizeof("FAILED:") - 1 + ngx_strlen(str)); if (s->data == NULL) { diff -r 7cffd81015e7 -r 8409f9df6219 src/event/ngx_event_openssl.h --- a/src/event/ngx_event_openssl.h Fri May 22 20:35:05 2020 +0300 +++ b/src/event/ngx_event_openssl.h Fri May 22 17:30:12 2020 +0300 @@ -64,6 +64,9 @@ #endif +typedef struct ngx_ssl_ocsp_s ngx_ssl_ocsp_t; + + struct ngx_ssl_s { SSL_CTX *ctx; ngx_log_t *log; @@ -87,6 +90,8 @@ struct ngx_ssl_connection_s { ngx_event_handler_pt saved_read_handler; ngx_event_handler_pt saved_write_handler; + ngx_ssl_ocsp_t *ocsp; + u_char early_buf; unsigned handshaked:1; @@ -97,6 +102,7 @@ struct ngx_ssl_connection_s { unsigned handshake_buffer_set:1; unsigned try_early_data:1; unsigned in_early:1; + unsigned in_ocsp:1; unsigned early_preread:1; unsigned write_blocked:1; }; @@ -180,6 +186,13 @@ ngx_int_t ngx_ssl_stapling(ngx_conf_t *c ngx_str_t *file, ngx_str_t *responder, ngx_uint_t verify); ngx_int_t ngx_ssl_stapling_resolver(ngx_conf_t *cf, ngx_ssl_t *ssl, ngx_resolver_t *resolver, ngx_msec_t resolver_timeout); +ngx_int_t ngx_ssl_ocsp(ngx_conf_t *cf, ngx_ssl_t *ssl, ngx_str_t *responder, + ngx_uint_t depth); +ngx_int_t ngx_ssl_ocsp_resolver(ngx_conf_t *cf, ngx_ssl_t *ssl, + ngx_resolver_t *resolver, ngx_msec_t resolver_timeout); +ngx_int_t ngx_ssl_ocsp_validate(ngx_connection_t *c); +ngx_int_t ngx_ssl_ocsp_get_status(ngx_connection_t *c, const char **s); +void ngx_ssl_ocsp_cleanup(ngx_connection_t *c); RSA *ngx_ssl_rsa512_key_callback(ngx_ssl_conn_t *ssl_conn, int is_export, int key_length); ngx_array_t *ngx_ssl_read_password_file(ngx_conf_t *cf, ngx_str_t *file); @@ -281,6 +294,7 @@ extern int ngx_ssl_connection_index; extern int ngx_ssl_server_conf_index; extern int ngx_ssl_session_cache_index; extern int ngx_ssl_session_ticket_keys_index; +extern int ngx_ssl_ocsp_index; extern int ngx_ssl_certificate_index; extern int ngx_ssl_next_certificate_index; extern int ngx_ssl_certificate_name_index; diff -r 7cffd81015e7 -r 8409f9df6219 src/event/ngx_event_openssl_stapling.c --- a/src/event/ngx_event_openssl_stapling.c Fri May 22 20:35:05 2020 +0300 +++ b/src/event/ngx_event_openssl_stapling.c Fri May 22 17:30:12 2020 +0300 @@ -43,8 +43,35 @@ typedef struct { } ngx_ssl_stapling_t; +typedef struct { + ngx_addr_t *addrs; + ngx_uint_t naddrs; + + ngx_str_t host; + ngx_str_t uri; + in_port_t port; + ngx_uint_t depth; + + ngx_resolver_t *resolver; + ngx_msec_t resolver_timeout; +} ngx_ssl_ocsp_conf_t; + + typedef struct ngx_ssl_ocsp_ctx_s ngx_ssl_ocsp_ctx_t; + +struct ngx_ssl_ocsp_s { + STACK_OF(X509) *certs; + ngx_uint_t ncert; + + int cert_status; + ngx_int_t status; + + ngx_ssl_ocsp_conf_t *conf; + ngx_ssl_ocsp_ctx_t *ctx; +}; + + struct ngx_ssl_ocsp_ctx_s { SSL_CTX *ssl_ctx; @@ -114,7 +141,12 @@ static time_t ngx_ssl_stapling_time(ASN1 static void ngx_ssl_stapling_cleanup(void *data); -static ngx_ssl_ocsp_ctx_t *ngx_ssl_ocsp_start(void); +static void ngx_ssl_ocsp_validate_next(ngx_connection_t *c); +static void ngx_ssl_ocsp_handler(ngx_ssl_ocsp_ctx_t *ctx); +static ngx_int_t ngx_ssl_ocsp_responder(ngx_connection_t *c, + ngx_ssl_ocsp_ctx_t *ctx); + +static ngx_ssl_ocsp_ctx_t *ngx_ssl_ocsp_start(ngx_log_t *log); static void ngx_ssl_ocsp_done(ngx_ssl_ocsp_ctx_t *ctx); static void ngx_ssl_ocsp_next(ngx_ssl_ocsp_ctx_t *ctx); static void ngx_ssl_ocsp_request(ngx_ssl_ocsp_ctx_t *ctx); @@ -570,7 +602,7 @@ ngx_ssl_stapling_update(ngx_ssl_stapling staple->loading = 1; - ctx = ngx_ssl_ocsp_start(); + ctx = ngx_ssl_ocsp_start(ngx_cycle->log); if (ctx == NULL) { return; } @@ -709,14 +741,467 @@ ngx_ssl_stapling_cleanup(void *data) } +ngx_int_t +ngx_ssl_ocsp(ngx_conf_t *cf, ngx_ssl_t *ssl, ngx_str_t *responder, + ngx_uint_t depth) +{ + ngx_url_t u; + ngx_ssl_ocsp_conf_t *ocf; + + ocf = ngx_pcalloc(cf->pool, sizeof(ngx_ssl_ocsp_conf_t)); + if (ocf == NULL) { + return NGX_ERROR; + } + + ocf->depth = depth; + + if (responder->len) { + ngx_memzero(&u, sizeof(ngx_url_t)); + + u.url = *responder; + u.default_port = 80; + u.uri_part = 1; + + if (u.url.len > 7 + && ngx_strncasecmp(u.url.data, (u_char *) "http://", 7) == 0) + { + u.url.len -= 7; + u.url.data += 7; + + } else { + ngx_log_error(NGX_LOG_EMERG, cf->log, 0, + "invalid URL prefix in OCSP responder \"%V\" " + "in \"ssl_ocsp_responder\"", &u.url); + return NGX_ERROR; + } + + if (ngx_parse_url(cf->pool, &u) != NGX_OK) { + if (u.err) { + ngx_log_error(NGX_LOG_EMERG, cf->log, 0, + "%s in OCSP responder \"%V\" " + "in \"ssl_ocsp_responder\"", u.err, &u.url); + } + + return NGX_ERROR; + } + + ocf->addrs = u.addrs; + ocf->naddrs = u.naddrs; + ocf->host = u.host; + ocf->uri = u.uri; + ocf->port = u.port; + } + + if (SSL_CTX_set_ex_data(ssl->ctx, ngx_ssl_ocsp_index, ocf) == 0) { + ngx_ssl_error(NGX_LOG_EMERG, ssl->log, 0, + "SSL_CTX_set_ex_data() failed"); + return NGX_ERROR; + } + + return NGX_OK; +} + + +ngx_int_t +ngx_ssl_ocsp_resolver(ngx_conf_t *cf, ngx_ssl_t *ssl, + ngx_resolver_t *resolver, ngx_msec_t resolver_timeout) +{ + ngx_ssl_ocsp_conf_t *ocf; + + ocf = SSL_CTX_get_ex_data(ssl->ctx, ngx_ssl_ocsp_index); + ocf->resolver = resolver; + ocf->resolver_timeout = resolver_timeout; + + return NGX_OK; +} + + +ngx_int_t +ngx_ssl_ocsp_validate(ngx_connection_t *c) +{ + X509 *cert; + SSL_CTX *ssl_ctx; + ngx_int_t rc; + X509_STORE *store; + X509_STORE_CTX *store_ctx; + STACK_OF(X509) *chain; + ngx_ssl_ocsp_t *ocsp; + ngx_ssl_ocsp_conf_t *ocf; + + if (c->ssl->in_ocsp) { + if (ngx_handle_read_event(c->read, 0) != NGX_OK) { + return NGX_ERROR; + } + + if (ngx_handle_write_event(c->write, 0) != NGX_OK) { + return NGX_ERROR; + } + + return NGX_AGAIN; + } + + ssl_ctx = SSL_get_SSL_CTX(c->ssl->connection); + + ocf = SSL_CTX_get_ex_data(ssl_ctx, ngx_ssl_ocsp_index); + if (ocf == NULL) { + return NGX_OK; + } + + if (SSL_get_verify_result(c->ssl->connection) != X509_V_OK) { + return NGX_OK; + } + + cert = SSL_get_peer_certificate(c->ssl->connection); + if (cert == NULL) { + return NGX_OK; + } + + ocsp = ngx_pcalloc(c->pool, sizeof(ngx_ssl_ocsp_t)); + if (ocsp == NULL) { + return NGX_ERROR; + } + + c->ssl->ocsp = ocsp; + + ocsp->status = NGX_AGAIN; + ocsp->cert_status = V_OCSP_CERTSTATUS_GOOD; + ocsp->conf = ocf; + +#if (OPENSSL_VERSION_NUMBER >= 0x10100000L && !defined LIBRESSL_VERSION_NUMBER) + + ocsp->certs = SSL_get0_verified_chain(c->ssl->connection); + + if (ocsp->certs) { + ocsp->certs = X509_chain_up_ref(ocsp->certs); + if (ocsp->certs == NULL) { + return NGX_ERROR; + } + } + +#endif + + if (ocsp->certs == NULL) { + store = SSL_CTX_get_cert_store(ssl_ctx); + if (store == NULL) { + ngx_ssl_error(NGX_LOG_ERR, c->log, 0, + "SSL_CTX_get_cert_store() failed"); + return NGX_ERROR; + } + + store_ctx = X509_STORE_CTX_new(); + if (store_ctx == NULL) { + ngx_ssl_error(NGX_LOG_ERR, c->log, 0, + "X509_STORE_CTX_new() failed"); + return NGX_ERROR; + } + + chain = SSL_get_peer_cert_chain(c->ssl->connection); + + if (X509_STORE_CTX_init(store_ctx, store, cert, chain) == 0) { + ngx_ssl_error(NGX_LOG_ERR, c->log, 0, + "X509_STORE_CTX_init() failed"); + X509_STORE_CTX_free(store_ctx); + return NGX_ERROR; + } + + rc = X509_verify_cert(store_ctx); + if (rc <= 0) { + ngx_ssl_error(NGX_LOG_ERR, c->log, 0, "X509_verify_cert() failed"); + X509_STORE_CTX_free(store_ctx); + return NGX_ERROR; + } + + ocsp->certs = X509_STORE_CTX_get1_chain(store_ctx); + if (ocsp->certs == NULL) { + ngx_ssl_error(NGX_LOG_ERR, c->log, 0, + "X509_STORE_CTX_get1_chain() failed"); + X509_STORE_CTX_free(store_ctx); + return NGX_ERROR; + } + + X509_STORE_CTX_free(store_ctx); + } + + ngx_log_debug1(NGX_LOG_DEBUG_EVENT, c->log, 0, + "ssl ocsp validate, certs:%i", sk_X509_num(ocsp->certs)); + + ngx_ssl_ocsp_validate_next(c); + + if (ocsp->status == NGX_AGAIN) { + c->ssl->in_ocsp = 1; + return NGX_AGAIN; + } + + return NGX_OK; +} + + +static void +ngx_ssl_ocsp_validate_next(ngx_connection_t *c) +{ + ngx_uint_t n; + ngx_ssl_ocsp_t *ocsp; + ngx_ssl_ocsp_ctx_t *ctx; + ngx_ssl_ocsp_conf_t *ocf; + + ocsp = c->ssl->ocsp; + ocf = ocsp->conf; + + n = sk_X509_num(ocsp->certs); + + for ( ;; ) { + + if (ocsp->ncert == n - 1 || (ocf->depth == 2 && ocsp->ncert == 1)) { + ngx_log_debug1(NGX_LOG_DEBUG_EVENT, c->log, 0, + "ssl ocsp validated, certs:%ui", ocsp->ncert); + goto done; + } + + ngx_log_debug1(NGX_LOG_DEBUG_EVENT, c->log, 0, + "ssl ocsp validate cert:%ui", ocsp->ncert); + + ctx = ngx_ssl_ocsp_start(c->log); + if (ctx == NULL) { + goto failed; + } + + ocsp->ctx = ctx; + + ctx->ssl_ctx = SSL_get_SSL_CTX(c->ssl->connection); + ctx->cert = sk_X509_value(ocsp->certs, ocsp->ncert); + ctx->issuer = sk_X509_value(ocsp->certs, ocsp->ncert + 1); + ctx->chain = ocsp->certs; + + ctx->resolver = ocf->resolver; + ctx->resolver_timeout = ocf->resolver_timeout; + + ctx->handler = ngx_ssl_ocsp_handler; + ctx->data = c; + + ctx->addrs = ocf->addrs; + ctx->naddrs = ocf->naddrs; + ctx->host = ocf->host; + ctx->uri = ocf->uri; + ctx->port = ocf->port; + + if (ngx_ssl_ocsp_responder(c, ctx) != NGX_OK) { + goto failed; + } + + if (ctx->uri.len == 0) { + ngx_str_set(&ctx->uri, "/"); + } + + ocsp->ncert++; + + break; + } + + ngx_ssl_ocsp_request(ctx); + return; + +done: + + ocsp->status = NGX_OK; + return; + +failed: + + ocsp->status = NGX_ERROR; +} + + +static void +ngx_ssl_ocsp_handler(ngx_ssl_ocsp_ctx_t *ctx) +{ + ngx_int_t rc; + ngx_ssl_ocsp_t *ocsp; + ngx_connection_t *c; + + c = ctx->data; + ocsp = c->ssl->ocsp; + ocsp->ctx = NULL; + + rc = ngx_ssl_ocsp_verify(ctx); + if (rc != NGX_OK) { + ocsp->status = rc; + ngx_ssl_ocsp_done(ctx); + goto done; + } + + if (ctx->status != V_OCSP_CERTSTATUS_GOOD) { + ocsp->cert_status = ctx->status; + ocsp->status = NGX_OK; + ngx_ssl_ocsp_done(ctx); + goto done; + } + + ngx_ssl_ocsp_done(ctx); + + ngx_ssl_ocsp_validate_next(c); + +done: + + if (ocsp->status == NGX_AGAIN || !c->ssl->in_ocsp) { + return; + } + + c->ssl->handshaked = 1; + + c->ssl->handler(c); +} + + +static ngx_int_t +ngx_ssl_ocsp_responder(ngx_connection_t *c, ngx_ssl_ocsp_ctx_t *ctx) +{ + char *s; + ngx_str_t responder; + ngx_url_t u; + STACK_OF(OPENSSL_STRING) *aia; + + if (ctx->host.len) { + return NGX_OK; + } + + /* extract OCSP responder URL from certificate */ + + aia = X509_get1_ocsp(ctx->cert); + if (aia == NULL) { + ngx_log_error(NGX_LOG_ERR, c->log, 0, + "no OCSP responder URL in certificate"); + return NGX_ERROR; + } + +#if OPENSSL_VERSION_NUMBER >= 0x10000000L + s = sk_OPENSSL_STRING_value(aia, 0); +#else + s = sk_value(aia, 0); +#endif + if (s == NULL) { + ngx_log_error(NGX_LOG_ERR, c->log, 0, + "no OCSP responder URL in certificate"); + X509_email_free(aia); + return NGX_ERROR; + } + + responder.len = ngx_strlen(s); + responder.data = ngx_palloc(ctx->pool, responder.len); + if (responder.data == NULL) { + X509_email_free(aia); + return NGX_ERROR; + } + + ngx_memcpy(responder.data, s, responder.len); + X509_email_free(aia); + + ngx_memzero(&u, sizeof(ngx_url_t)); + + u.url = responder; + u.default_port = 80; + u.uri_part = 1; + u.no_resolve = 1; + + if (u.url.len > 7 + && ngx_strncasecmp(u.url.data, (u_char *) "http://", 7) == 0) + { + u.url.len -= 7; + u.url.data += 7; + + } else { + ngx_log_error(NGX_LOG_ERR, c->log, 0, + "invalid URL prefix in OCSP responder \"%V\" " + "in certificate", &u.url); + return NGX_ERROR; + } + + if (ngx_parse_url(ctx->pool, &u) != NGX_OK) { + if (u.err) { + ngx_log_error(NGX_LOG_ERR, c->log, 0, + "%s in OCSP responder \"%V\" in certificate", + u.err, &u.url); + } + + return NGX_ERROR; + } + + if (u.host.len == 0) { + ngx_log_error(NGX_LOG_ERR, c->log, 0, + "empty host in OCSP responder in certificate"); + return NGX_ERROR; + } + + ctx->addrs = u.addrs; + ctx->naddrs = u.naddrs; + ctx->host = u.host; + ctx->uri = u.uri; + ctx->port = u.port; + + return NGX_OK; +} + + +ngx_int_t +ngx_ssl_ocsp_get_status(ngx_connection_t *c, const char **s) +{ + ngx_ssl_ocsp_t *ocsp; + + ocsp = c->ssl->ocsp; + if (ocsp == NULL) { + return NGX_OK; + } + + if (ocsp->status == NGX_ERROR) { + *s = "certificate status request failed"; + return NGX_DECLINED; + } + + switch (ocsp->cert_status) { + + case V_OCSP_CERTSTATUS_GOOD: + return NGX_OK; + + case V_OCSP_CERTSTATUS_REVOKED: + *s = "certificate revoked"; + break; + + default: /* V_OCSP_CERTSTATUS_UNKNOWN */ + *s = "certificate status unknown"; + } + + return NGX_DECLINED; +} + + +void +ngx_ssl_ocsp_cleanup(ngx_connection_t *c) +{ + ngx_ssl_ocsp_t *ocsp; + + ocsp = c->ssl->ocsp; + if (ocsp == NULL) { + return; + } + + if (ocsp->ctx) { + ngx_ssl_ocsp_done(ocsp->ctx); + ocsp->ctx = NULL; + } + + if (ocsp->certs) { + sk_X509_pop_free(ocsp->certs, X509_free); + ocsp->certs = NULL; + } +} + + static ngx_ssl_ocsp_ctx_t * -ngx_ssl_ocsp_start(void) +ngx_ssl_ocsp_start(ngx_log_t *log) { - ngx_log_t *log; ngx_pool_t *pool; ngx_ssl_ocsp_ctx_t *ctx; - pool = ngx_create_pool(2048, ngx_cycle->log); + pool = ngx_create_pool(2048, log); if (pool == NULL) { return NULL; } @@ -828,6 +1313,14 @@ ngx_ssl_ocsp_request(ngx_ssl_ocsp_ctx_t } if (resolve == NGX_NO_RESOLVER) { + if (ctx->naddrs == 0) { + ngx_log_error(NGX_LOG_ERR, ctx->log, 0, + "no resolver defined to resolve %V", &ctx->host); + + ngx_ssl_ocsp_error(ctx); + return; + } + ngx_log_error(NGX_LOG_WARN, ctx->log, 0, "no resolver defined to resolve %V", &ctx->host); goto connect; @@ -979,8 +1472,10 @@ ngx_ssl_ocsp_connect(ngx_ssl_ocsp_ctx_t ctx->process = ngx_ssl_ocsp_process_status_line; - ngx_add_timer(ctx->peer.connection->read, ctx->timeout); - ngx_add_timer(ctx->peer.connection->write, ctx->timeout); + if (ctx->timeout) { + ngx_add_timer(ctx->peer.connection->read, ctx->timeout); + ngx_add_timer(ctx->peer.connection->write, ctx->timeout); + } if (rc == NGX_OK) { ngx_ssl_ocsp_write_handler(ctx->peer.connection->write); @@ -1036,7 +1531,7 @@ ngx_ssl_ocsp_write_handler(ngx_event_t * } } - if (!wev->timer_set) { + if (!wev->timer_set && ctx->timeout) { ngx_add_timer(wev, ctx->timeout); } } @@ -1939,4 +2434,43 @@ ngx_ssl_stapling_resolver(ngx_conf_t *cf } +ngx_int_t +ngx_ssl_ocsp(ngx_conf_t *cf, ngx_ssl_t *ssl, ngx_str_t *responder, + ngx_uint_t depth) +{ + ngx_log_error(NGX_LOG_EMERG, ssl->log, 0, + "\"ssl_ocsp\" is not supported on this platform"); + + return NGX_ERROR; +} + + +ngx_int_t +ngx_ssl_ocsp_resolver(ngx_conf_t *cf, ngx_ssl_t *ssl, + ngx_resolver_t *resolver, ngx_msec_t resolver_timeout) +{ + return NGX_OK; +} + + +ngx_int_t +ngx_ssl_ocsp_validate(ngx_connection_t *c) +{ + return NGX_OK; +} + + +ngx_int_t +ngx_ssl_ocsp_get_status(ngx_connection_t *c, const char **s) +{ + return NGX_OK; +} + + +void +ngx_ssl_ocsp_cleanup(ngx_connection_t *c) +{ +} + + #endif diff -r 7cffd81015e7 -r 8409f9df6219 src/http/modules/ngx_http_ssl_module.c --- a/src/http/modules/ngx_http_ssl_module.c Fri May 22 20:35:05 2020 +0300 +++ b/src/http/modules/ngx_http_ssl_module.c Fri May 22 17:30:12 2020 +0300 @@ -74,6 +74,14 @@ static ngx_conf_enum_t ngx_http_ssl_ver }; +static ngx_conf_enum_t ngx_http_ssl_ocsp[] = { + { ngx_string("off"), 0 }, + { ngx_string("on"), 1 }, + { ngx_string("leaf"), 2 }, + { ngx_null_string, 0 } +}; + + static ngx_conf_deprecated_t ngx_http_ssl_deprecated = { ngx_conf_deprecated, "ssl", "listen ... ssl" }; @@ -214,6 +222,20 @@ static ngx_command_t ngx_http_ssl_comma offsetof(ngx_http_ssl_srv_conf_t, crl), NULL }, + { ngx_string("ssl_ocsp"), + NGX_HTTP_MAIN_CONF|NGX_HTTP_SRV_CONF|NGX_CONF_FLAG, + ngx_conf_set_enum_slot, + NGX_HTTP_SRV_CONF_OFFSET, + offsetof(ngx_http_ssl_srv_conf_t, ocsp), + &ngx_http_ssl_ocsp }, + + { ngx_string("ssl_ocsp_responder"), + NGX_HTTP_MAIN_CONF|NGX_HTTP_SRV_CONF|NGX_CONF_TAKE1, + ngx_conf_set_str_slot, + NGX_HTTP_SRV_CONF_OFFSET, + offsetof(ngx_http_ssl_srv_conf_t, ocsp_responder), + NULL }, + { ngx_string("ssl_stapling"), NGX_HTTP_MAIN_CONF|NGX_HTTP_SRV_CONF|NGX_CONF_FLAG, ngx_conf_set_flag_slot, @@ -561,6 +583,7 @@ ngx_http_ssl_create_srv_conf(ngx_conf_t * sscf->crl = { 0, NULL }; * sscf->ciphers = { 0, NULL }; * sscf->shm_zone = NULL; + * sscf->ocsp_responder = { 0, NULL }; * sscf->stapling_file = { 0, NULL }; * sscf->stapling_responder = { 0, NULL }; */ @@ -578,6 +601,7 @@ ngx_http_ssl_create_srv_conf(ngx_conf_t sscf->session_timeout = NGX_CONF_UNSET; sscf->session_tickets = NGX_CONF_UNSET; sscf->session_ticket_keys = NGX_CONF_UNSET_PTR; + sscf->ocsp = NGX_CONF_UNSET_UINT; sscf->stapling = NGX_CONF_UNSET; sscf->stapling_verify = NGX_CONF_UNSET; @@ -641,6 +665,9 @@ ngx_http_ssl_merge_srv_conf(ngx_conf_t * ngx_conf_merge_str_value(conf->ciphers, prev->ciphers, NGX_DEFAULT_CIPHERS); + ngx_conf_merge_uint_value(conf->ocsp, prev->ocsp, 0); + ngx_conf_merge_str_value(conf->ocsp_responder, prev->ocsp_responder, ""); + ngx_conf_merge_value(conf->stapling, prev->stapling, 0); ngx_conf_merge_value(conf->stapling_verify, prev->stapling_verify, 0); ngx_conf_merge_str_value(conf->stapling_file, prev->stapling_file, ""); @@ -802,6 +829,22 @@ ngx_http_ssl_merge_srv_conf(ngx_conf_t * return NGX_CONF_ERROR; } + if (conf->ocsp) { + + if (conf->verify == 3) { + ngx_log_error(NGX_LOG_EMERG, cf->log, 0, + "\"ssl_ocsp\" is incompatible with " + "\"ssl_verify_client optional_no_ca\""); + return NGX_CONF_ERROR; + } + + if (ngx_ssl_ocsp(cf, &conf->ssl, &conf->ocsp_responder, conf->ocsp) + != NGX_OK) + { + return NGX_CONF_ERROR; + } + } + if (ngx_ssl_dhparam(cf, &conf->ssl, &conf->dhparam) != NGX_OK) { return NGX_CONF_ERROR; } @@ -1118,17 +1161,28 @@ ngx_http_ssl_init(ngx_conf_t *cf) sscf = cscfp[s]->ctx->srv_conf[ngx_http_ssl_module.ctx_index]; - if (sscf->ssl.ctx == NULL || !sscf->stapling) { + if (sscf->ssl.ctx == NULL) { continue; } clcf = cscfp[s]->ctx->loc_conf[ngx_http_core_module.ctx_index]; - if (ngx_ssl_stapling_resolver(cf, &sscf->ssl, clcf->resolver, + if (sscf->stapling) { + if (ngx_ssl_stapling_resolver(cf, &sscf->ssl, clcf->resolver, + clcf->resolver_timeout) + != NGX_OK) + { + return NGX_ERROR; + } + } + + if (sscf->ocsp) { + if (ngx_ssl_ocsp_resolver(cf, &sscf->ssl, clcf->resolver, clcf->resolver_timeout) - != NGX_OK) - { - return NGX_ERROR; + != NGX_OK) + { + return NGX_ERROR; + } } } diff -r 7cffd81015e7 -r 8409f9df6219 src/http/modules/ngx_http_ssl_module.h --- a/src/http/modules/ngx_http_ssl_module.h Fri May 22 20:35:05 2020 +0300 +++ b/src/http/modules/ngx_http_ssl_module.h Fri May 22 17:30:12 2020 +0300 @@ -54,6 +54,9 @@ typedef struct { ngx_flag_t session_tickets; ngx_array_t *session_ticket_keys; + ngx_uint_t ocsp; + ngx_str_t ocsp_responder; + ngx_flag_t stapling; ngx_flag_t stapling_verify; ngx_str_t stapling_file; diff -r 7cffd81015e7 -r 8409f9df6219 src/http/ngx_http_request.c --- a/src/http/ngx_http_request.c Fri May 22 20:35:05 2020 +0300 +++ b/src/http/ngx_http_request.c Fri May 22 17:30:12 2020 +0300 @@ -1993,6 +1993,7 @@ ngx_http_process_request(ngx_http_reques if (r->http_connection->ssl) { long rc; X509 *cert; + const char *s; ngx_http_ssl_srv_conf_t *sscf; if (c->ssl == NULL) { @@ -2037,6 +2038,17 @@ ngx_http_process_request(ngx_http_reques X509_free(cert); } + + if (ngx_ssl_ocsp_get_status(c, &s) != NGX_OK) { + ngx_log_error(NGX_LOG_INFO, c->log, 0, + "client SSL certificate verify error: %s", s); + + ngx_ssl_remove_cached_session(c->ssl->session_ctx, + (SSL_get0_session(c->ssl->connection))); + + ngx_http_finalize_request(r, NGX_HTTPS_CERT_ERROR); + return; + } } } From arut at nginx.com Sat May 23 10:35:53 2020 From: arut at nginx.com (Roman Arutyunyan) Date: Sat, 23 May 2020 10:35:53 +0000 Subject: [nginx] OCSP: certificate status cache. Message-ID: details: https://hg.nginx.org/nginx/rev/b56f725dd4bb branches: changeset: 7654:b56f725dd4bb user: Roman Arutyunyan date: Fri May 22 17:25:27 2020 +0300 description: OCSP: certificate status cache. When enabled, certificate status is stored in cache and is used to validate the certificate in future requests. New directive ssl_ocsp_cache is added to configure the cache. diffstat: src/event/ngx_event_openssl.h | 3 +- src/event/ngx_event_openssl_stapling.c | 308 ++++++++++++++++++++++++++++++++- src/http/modules/ngx_http_ssl_module.c | 94 +++++++++- src/http/modules/ngx_http_ssl_module.h | 1 + 4 files changed, 401 insertions(+), 5 deletions(-) diffs (570 lines): diff -r 8409f9df6219 -r b56f725dd4bb src/event/ngx_event_openssl.h --- a/src/event/ngx_event_openssl.h Fri May 22 17:30:12 2020 +0300 +++ b/src/event/ngx_event_openssl.h Fri May 22 17:25:27 2020 +0300 @@ -187,12 +187,13 @@ ngx_int_t ngx_ssl_stapling(ngx_conf_t *c ngx_int_t ngx_ssl_stapling_resolver(ngx_conf_t *cf, ngx_ssl_t *ssl, ngx_resolver_t *resolver, ngx_msec_t resolver_timeout); ngx_int_t ngx_ssl_ocsp(ngx_conf_t *cf, ngx_ssl_t *ssl, ngx_str_t *responder, - ngx_uint_t depth); + ngx_uint_t depth, ngx_shm_zone_t *shm_zone); ngx_int_t ngx_ssl_ocsp_resolver(ngx_conf_t *cf, ngx_ssl_t *ssl, ngx_resolver_t *resolver, ngx_msec_t resolver_timeout); ngx_int_t ngx_ssl_ocsp_validate(ngx_connection_t *c); ngx_int_t ngx_ssl_ocsp_get_status(ngx_connection_t *c, const char **s); void ngx_ssl_ocsp_cleanup(ngx_connection_t *c); +ngx_int_t ngx_ssl_ocsp_cache_init(ngx_shm_zone_t *shm_zone, void *data); RSA *ngx_ssl_rsa512_key_callback(ngx_ssl_conn_t *ssl_conn, int is_export, int key_length); ngx_array_t *ngx_ssl_read_password_file(ngx_conf_t *cf, ngx_str_t *file); diff -r 8409f9df6219 -r b56f725dd4bb src/event/ngx_event_openssl_stapling.c --- a/src/event/ngx_event_openssl_stapling.c Fri May 22 17:30:12 2020 +0300 +++ b/src/event/ngx_event_openssl_stapling.c Fri May 22 17:25:27 2020 +0300 @@ -52,11 +52,28 @@ typedef struct { in_port_t port; ngx_uint_t depth; + ngx_shm_zone_t *shm_zone; + ngx_resolver_t *resolver; ngx_msec_t resolver_timeout; } ngx_ssl_ocsp_conf_t; +typedef struct { + ngx_rbtree_t rbtree; + ngx_rbtree_node_t sentinel; + ngx_queue_t expire_queue; +} ngx_ssl_ocsp_cache_t; + + +typedef struct { + ngx_str_node_t node; + ngx_queue_t queue; + int status; + time_t valid; +} ngx_ssl_ocsp_cache_node_t; + + typedef struct ngx_ssl_ocsp_ctx_s ngx_ssl_ocsp_ctx_t; @@ -100,10 +117,13 @@ struct ngx_ssl_ocsp_ctx_s { void (*handler)(ngx_ssl_ocsp_ctx_t *ctx); void *data; + ngx_str_t key; ngx_buf_t *request; ngx_buf_t *response; ngx_peer_connection_t peer; + ngx_shm_zone_t *shm_zone; + ngx_int_t (*process)(ngx_ssl_ocsp_ctx_t *ctx); ngx_uint_t state; @@ -164,6 +184,10 @@ static ngx_int_t ngx_ssl_ocsp_parse_head static ngx_int_t ngx_ssl_ocsp_process_body(ngx_ssl_ocsp_ctx_t *ctx); static ngx_int_t ngx_ssl_ocsp_verify(ngx_ssl_ocsp_ctx_t *ctx); +static ngx_int_t ngx_ssl_ocsp_cache_lookup(ngx_ssl_ocsp_ctx_t *ctx); +static ngx_int_t ngx_ssl_ocsp_cache_store(ngx_ssl_ocsp_ctx_t *ctx); +static ngx_int_t ngx_ssl_ocsp_create_key(ngx_ssl_ocsp_ctx_t *ctx); + static u_char *ngx_ssl_ocsp_log_error(ngx_log_t *log, u_char *buf, size_t len); @@ -743,7 +767,7 @@ ngx_ssl_stapling_cleanup(void *data) ngx_int_t ngx_ssl_ocsp(ngx_conf_t *cf, ngx_ssl_t *ssl, ngx_str_t *responder, - ngx_uint_t depth) + ngx_uint_t depth, ngx_shm_zone_t *shm_zone) { ngx_url_t u; ngx_ssl_ocsp_conf_t *ocf; @@ -754,6 +778,7 @@ ngx_ssl_ocsp(ngx_conf_t *cf, ngx_ssl_t * } ocf->depth = depth; + ocf->shm_zone = shm_zone; if (responder->len) { ngx_memzero(&u, sizeof(ngx_url_t)); @@ -939,6 +964,7 @@ ngx_ssl_ocsp_validate(ngx_connection_t * static void ngx_ssl_ocsp_validate_next(ngx_connection_t *c) { + ngx_int_t rc; ngx_uint_t n; ngx_ssl_ocsp_t *ocsp; ngx_ssl_ocsp_ctx_t *ctx; @@ -978,6 +1004,8 @@ ngx_ssl_ocsp_validate_next(ngx_connectio ctx->handler = ngx_ssl_ocsp_handler; ctx->data = c; + ctx->shm_zone = ocf->shm_zone; + ctx->addrs = ocf->addrs; ctx->naddrs = ocf->naddrs; ctx->host = ocf->host; @@ -994,7 +1022,28 @@ ngx_ssl_ocsp_validate_next(ngx_connectio ocsp->ncert++; - break; + rc = ngx_ssl_ocsp_cache_lookup(ctx); + + if (rc == NGX_ERROR) { + goto failed; + } + + if (rc == NGX_DECLINED) { + break; + } + + /* rc == NGX_OK */ + + if (ctx->status != V_OCSP_CERTSTATUS_GOOD) { + ngx_log_debug1(NGX_LOG_DEBUG_EVENT, ctx->log, 0, + "ssl ocsp cached status \"%s\"", + OCSP_cert_status_str(ctx->status)); + ocsp->cert_status = ctx->status; + goto done; + } + + ocsp->ctx = NULL; + ngx_ssl_ocsp_done(ctx); } ngx_ssl_ocsp_request(ctx); @@ -1029,6 +1078,13 @@ ngx_ssl_ocsp_handler(ngx_ssl_ocsp_ctx_t goto done; } + rc = ngx_ssl_ocsp_cache_store(ctx); + if (rc != NGX_OK) { + ocsp->status = rc; + ngx_ssl_ocsp_done(ctx); + goto done; + } + if (ctx->status != V_OCSP_CERTSTATUS_GOOD) { ocsp->cert_status = ctx->status; ocsp->status = NGX_OK; @@ -2374,6 +2430,245 @@ error: } +ngx_int_t +ngx_ssl_ocsp_cache_init(ngx_shm_zone_t *shm_zone, void *data) +{ + size_t len; + ngx_slab_pool_t *shpool; + ngx_ssl_ocsp_cache_t *cache; + + if (data) { + shm_zone->data = data; + return NGX_OK; + } + + shpool = (ngx_slab_pool_t *) shm_zone->shm.addr; + + if (shm_zone->shm.exists) { + shm_zone->data = shpool->data; + return NGX_OK; + } + + cache = ngx_slab_alloc(shpool, sizeof(ngx_ssl_ocsp_cache_t)); + if (cache == NULL) { + return NGX_ERROR; + } + + shpool->data = cache; + shm_zone->data = cache; + + ngx_rbtree_init(&cache->rbtree, &cache->sentinel, + ngx_str_rbtree_insert_value); + + ngx_queue_init(&cache->expire_queue); + + len = sizeof(" in OCSP cache \"\"") + shm_zone->shm.name.len; + + shpool->log_ctx = ngx_slab_alloc(shpool, len); + if (shpool->log_ctx == NULL) { + return NGX_ERROR; + } + + ngx_sprintf(shpool->log_ctx, " in OCSP cache \"%V\"%Z", + &shm_zone->shm.name); + + shpool->log_nomem = 0; + + return NGX_OK; +} + + +static ngx_int_t +ngx_ssl_ocsp_cache_lookup(ngx_ssl_ocsp_ctx_t *ctx) +{ + uint32_t hash; + ngx_shm_zone_t *shm_zone; + ngx_slab_pool_t *shpool; + ngx_ssl_ocsp_cache_t *cache; + ngx_ssl_ocsp_cache_node_t *node; + + shm_zone = ctx->shm_zone; + + if (shm_zone == NULL) { + return NGX_DECLINED; + } + + if (ngx_ssl_ocsp_create_key(ctx) != NGX_OK) { + return NGX_ERROR; + } + + ngx_log_debug0(NGX_LOG_DEBUG_EVENT, ctx->log, 0, "ssl ocsp cache lookup"); + + cache = shm_zone->data; + shpool = (ngx_slab_pool_t *) shm_zone->shm.addr; + hash = ngx_hash_key(ctx->key.data, ctx->key.len); + + ngx_shmtx_lock(&shpool->mutex); + + node = (ngx_ssl_ocsp_cache_node_t *) + ngx_str_rbtree_lookup(&cache->rbtree, &ctx->key, hash); + + if (node) { + if (node->valid > ngx_time()) { + ctx->status = node->status; + ngx_shmtx_unlock(&shpool->mutex); + + ngx_log_debug1(NGX_LOG_DEBUG_EVENT, ctx->log, 0, + "ssl ocsp cache hit, %s", + OCSP_cert_status_str(ctx->status)); + + return NGX_OK; + } + + ngx_queue_remove(&node->queue); + ngx_rbtree_delete(&cache->rbtree, &node->node.node); + ngx_slab_free_locked(shpool, node); + + ngx_shmtx_unlock(&shpool->mutex); + + ngx_log_debug0(NGX_LOG_DEBUG_EVENT, ctx->log, 0, + "ssl ocsp cache expired"); + + return NGX_DECLINED; + } + + ngx_shmtx_unlock(&shpool->mutex); + + ngx_log_debug0(NGX_LOG_DEBUG_EVENT, ctx->log, 0, "ssl ocsp cache miss"); + + return NGX_DECLINED; +} + + +static ngx_int_t +ngx_ssl_ocsp_cache_store(ngx_ssl_ocsp_ctx_t *ctx) +{ + time_t now, valid; + uint32_t hash; + ngx_queue_t *q; + ngx_shm_zone_t *shm_zone; + ngx_slab_pool_t *shpool; + ngx_ssl_ocsp_cache_t *cache; + ngx_ssl_ocsp_cache_node_t *node; + + shm_zone = ctx->shm_zone; + + if (shm_zone == NULL) { + return NGX_OK; + } + + valid = ctx->valid; + + now = ngx_time(); + + if (valid < now) { + return NGX_OK; + } + + if (valid == NGX_MAX_TIME_T_VALUE) { + valid = now + 3600; + } + + ngx_log_debug1(NGX_LOG_DEBUG_EVENT, ctx->log, 0, + "ssl ocsp cache store, valid:%T", valid - now); + + cache = shm_zone->data; + shpool = (ngx_slab_pool_t *) shm_zone->shm.addr; + hash = ngx_hash_key(ctx->key.data, ctx->key.len); + + ngx_shmtx_lock(&shpool->mutex); + + node = ngx_slab_calloc_locked(shpool, + sizeof(ngx_ssl_ocsp_cache_node_t) + ctx->key.len); + if (node == NULL) { + + if (!ngx_queue_empty(&cache->expire_queue)) { + q = ngx_queue_last(&cache->expire_queue); + node = ngx_queue_data(q, ngx_ssl_ocsp_cache_node_t, queue); + + ngx_rbtree_delete(&cache->rbtree, &node->node.node); + ngx_queue_remove(q); + ngx_slab_free_locked(shpool, node); + + node = ngx_slab_alloc_locked(shpool, + sizeof(ngx_ssl_ocsp_cache_node_t) + ctx->key.len); + } + + if (node == NULL) { + ngx_shmtx_unlock(&shpool->mutex); + ngx_log_error(NGX_LOG_ALERT, ctx->log, 0, + "could not allocate new entry%s", shpool->log_ctx); + return NGX_ERROR; + } + } + + node->node.str.len = ctx->key.len; + node->node.str.data = (u_char *) node + sizeof(ngx_ssl_ocsp_cache_node_t); + ngx_memcpy(node->node.str.data, ctx->key.data, ctx->key.len); + node->node.node.key = hash; + node->status = ctx->status; + node->valid = valid; + + ngx_rbtree_insert(&cache->rbtree, &node->node.node); + ngx_queue_insert_head(&cache->expire_queue, &node->queue); + + ngx_shmtx_unlock(&shpool->mutex); + + return NGX_OK; +} + + +static ngx_int_t +ngx_ssl_ocsp_create_key(ngx_ssl_ocsp_ctx_t *ctx) +{ + u_char *p; + X509_NAME *name; + ASN1_INTEGER *serial; + + p = ngx_pnalloc(ctx->pool, 60); + if (p == NULL) { + return NGX_ERROR; + } + + ctx->key.data = p; + ctx->key.len = 60; + + name = X509_get_subject_name(ctx->issuer); + if (X509_NAME_digest(name, EVP_sha1(), p, NULL) == 0) { + return NGX_ERROR; + } + + p += 20; + + if (X509_pubkey_digest(ctx->issuer, EVP_sha1(), p, NULL) == 0) { + return NGX_ERROR; + } + + p += 20; + + serial = X509_get_serialNumber(ctx->cert); + if (serial->length > 20) { + return NGX_ERROR; + } + + p = ngx_cpymem(p, serial->data, serial->length); + ngx_memzero(p, 20 - serial->length); + +#if (NGX_DEBUG) + { + u_char buf[120]; + + ngx_hex_dump(buf, ctx->key.data, ctx->key.len); + + ngx_log_debug2(NGX_LOG_DEBUG_EVENT, ctx->log, 0, + "ssl ocsp key %*s", 120, buf); + } +#endif + + return NGX_OK; +} + + static u_char * ngx_ssl_ocsp_log_error(ngx_log_t *log, u_char *buf, size_t len) { @@ -2436,7 +2731,7 @@ ngx_ssl_stapling_resolver(ngx_conf_t *cf ngx_int_t ngx_ssl_ocsp(ngx_conf_t *cf, ngx_ssl_t *ssl, ngx_str_t *responder, - ngx_uint_t depth) + ngx_uint_t depth, ngx_shm_zone_t *shm_zone) { ngx_log_error(NGX_LOG_EMERG, ssl->log, 0, "\"ssl_ocsp\" is not supported on this platform"); @@ -2473,4 +2768,11 @@ ngx_ssl_ocsp_cleanup(ngx_connection_t *c } +ngx_int_t +ngx_ssl_ocsp_cache_init(ngx_shm_zone_t *shm_zone, void *data) +{ + return NGX_OK; +} + + #endif diff -r 8409f9df6219 -r b56f725dd4bb src/http/modules/ngx_http_ssl_module.c --- a/src/http/modules/ngx_http_ssl_module.c Fri May 22 17:30:12 2020 +0300 +++ b/src/http/modules/ngx_http_ssl_module.c Fri May 22 17:25:27 2020 +0300 @@ -50,6 +50,8 @@ static char *ngx_http_ssl_password_file( void *conf); static char *ngx_http_ssl_session_cache(ngx_conf_t *cf, ngx_command_t *cmd, void *conf); +static char *ngx_http_ssl_ocsp_cache(ngx_conf_t *cf, ngx_command_t *cmd, + void *conf); static ngx_int_t ngx_http_ssl_init(ngx_conf_t *cf); @@ -236,6 +238,13 @@ static ngx_command_t ngx_http_ssl_comma offsetof(ngx_http_ssl_srv_conf_t, ocsp_responder), NULL }, + { ngx_string("ssl_ocsp_cache"), + NGX_HTTP_MAIN_CONF|NGX_HTTP_SRV_CONF|NGX_CONF_TAKE1, + ngx_http_ssl_ocsp_cache, + NGX_HTTP_SRV_CONF_OFFSET, + 0, + NULL }, + { ngx_string("ssl_stapling"), NGX_HTTP_MAIN_CONF|NGX_HTTP_SRV_CONF|NGX_CONF_FLAG, ngx_conf_set_flag_slot, @@ -602,6 +611,7 @@ ngx_http_ssl_create_srv_conf(ngx_conf_t sscf->session_tickets = NGX_CONF_UNSET; sscf->session_ticket_keys = NGX_CONF_UNSET_PTR; sscf->ocsp = NGX_CONF_UNSET_UINT; + sscf->ocsp_cache_zone = NGX_CONF_UNSET_PTR; sscf->stapling = NGX_CONF_UNSET; sscf->stapling_verify = NGX_CONF_UNSET; @@ -667,6 +677,8 @@ ngx_http_ssl_merge_srv_conf(ngx_conf_t * ngx_conf_merge_uint_value(conf->ocsp, prev->ocsp, 0); ngx_conf_merge_str_value(conf->ocsp_responder, prev->ocsp_responder, ""); + ngx_conf_merge_ptr_value(conf->ocsp_cache_zone, + prev->ocsp_cache_zone, NULL); ngx_conf_merge_value(conf->stapling, prev->stapling, 0); ngx_conf_merge_value(conf->stapling_verify, prev->stapling_verify, 0); @@ -838,7 +850,8 @@ ngx_http_ssl_merge_srv_conf(ngx_conf_t * return NGX_CONF_ERROR; } - if (ngx_ssl_ocsp(cf, &conf->ssl, &conf->ocsp_responder, conf->ocsp) + if (ngx_ssl_ocsp(cf, &conf->ssl, &conf->ocsp_responder, conf->ocsp, + conf->ocsp_cache_zone) != NGX_OK) { return NGX_CONF_ERROR; @@ -1143,6 +1156,85 @@ invalid: } +static char * +ngx_http_ssl_ocsp_cache(ngx_conf_t *cf, ngx_command_t *cmd, void *conf) +{ + ngx_http_ssl_srv_conf_t *sscf = conf; + + size_t len; + ngx_int_t n; + ngx_str_t *value, name, size; + ngx_uint_t j; + + if (sscf->ocsp_cache_zone != NGX_CONF_UNSET_PTR) { + return "is duplicate"; + } + + value = cf->args->elts; + + if (ngx_strcmp(value[1].data, "off") == 0) { + sscf->ocsp_cache_zone = NULL; + return NGX_CONF_OK; + } + + if (value[1].len <= sizeof("shared:") - 1 + || ngx_strncmp(value[1].data, "shared:", sizeof("shared:") - 1) != 0) + { + goto invalid; + } + + len = 0; + + for (j = sizeof("shared:") - 1; j < value[1].len; j++) { + if (value[1].data[j] == ':') { + break; + } + + len++; + } + + if (len == 0) { + goto invalid; + } + + name.len = len; + name.data = value[1].data + sizeof("shared:") - 1; + + size.len = value[1].len - j - 1; + size.data = name.data + len + 1; + + n = ngx_parse_size(&size); + + if (n == NGX_ERROR) { + goto invalid; + } + + if (n < (ngx_int_t) (8 * ngx_pagesize)) { + ngx_conf_log_error(NGX_LOG_EMERG, cf, 0, + "OCSP cache \"%V\" is too small", &value[1]); + + return NGX_CONF_ERROR; + } + + sscf->ocsp_cache_zone = ngx_shared_memory_add(cf, &name, n, + &ngx_http_ssl_module_ctx); + if (sscf->ocsp_cache_zone == NULL) { + return NGX_CONF_ERROR; + } + + sscf->ocsp_cache_zone->init = ngx_ssl_ocsp_cache_init; + + return NGX_CONF_OK; + +invalid: + + ngx_conf_log_error(NGX_LOG_EMERG, cf, 0, + "invalid OCSP cache \"%V\"", &value[1]); + + return NGX_CONF_ERROR; +} + + static ngx_int_t ngx_http_ssl_init(ngx_conf_t *cf) { diff -r 8409f9df6219 -r b56f725dd4bb src/http/modules/ngx_http_ssl_module.h --- a/src/http/modules/ngx_http_ssl_module.h Fri May 22 17:30:12 2020 +0300 +++ b/src/http/modules/ngx_http_ssl_module.h Fri May 22 17:25:27 2020 +0300 @@ -56,6 +56,7 @@ typedef struct { ngx_uint_t ocsp; ngx_str_t ocsp_responder; + ngx_shm_zone_t *ocsp_cache_zone; ngx_flag_t stapling; ngx_flag_t stapling_verify; From pluknet at nginx.com Mon May 25 10:53:19 2020 From: pluknet at nginx.com (Sergey Kandaurov) Date: Mon, 25 May 2020 10:53:19 +0000 Subject: [nginx] Fixed format specifiers. Message-ID: details: https://hg.nginx.org/nginx/rev/bd4d1b9db0ee branches: changeset: 7655:bd4d1b9db0ee user: Sergey Kandaurov date: Sat May 23 15:53:08 2020 +0300 description: Fixed format specifiers. diffstat: src/event/ngx_event_openssl_stapling.c | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) diffs (21 lines): diff -r b56f725dd4bb -r bd4d1b9db0ee src/event/ngx_event_openssl_stapling.c --- a/src/event/ngx_event_openssl_stapling.c Fri May 22 17:25:27 2020 +0300 +++ b/src/event/ngx_event_openssl_stapling.c Sat May 23 15:53:08 2020 +0300 @@ -948,7 +948,7 @@ ngx_ssl_ocsp_validate(ngx_connection_t * } ngx_log_debug1(NGX_LOG_DEBUG_EVENT, c->log, 0, - "ssl ocsp validate, certs:%i", sk_X509_num(ocsp->certs)); + "ssl ocsp validate, certs:%d", sk_X509_num(ocsp->certs)); ngx_ssl_ocsp_validate_next(c); @@ -2661,7 +2661,7 @@ ngx_ssl_ocsp_create_key(ngx_ssl_ocsp_ctx ngx_hex_dump(buf, ctx->key.data, ctx->key.len); ngx_log_debug2(NGX_LOG_DEBUG_EVENT, ctx->log, 0, - "ssl ocsp key %*s", 120, buf); + "ssl ocsp key %*s", sizeof(buf), buf); } #endif From xeioex at nginx.com Mon May 25 14:23:47 2020 From: xeioex at nginx.com (Dmitry Volyntsev) Date: Mon, 25 May 2020 14:23:47 +0000 Subject: [njs] Version bump. Message-ID: details: https://hg.nginx.org/njs/rev/ab19ee68d945 branches: changeset: 1393:ab19ee68d945 user: Dmitry Volyntsev date: Mon May 25 14:21:14 2020 +0000 description: Version bump. diffstat: src/njs.h | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diffs (12 lines): diff -r 46ec2411fb7c -r ab19ee68d945 src/njs.h --- a/src/njs.h Tue May 19 11:42:11 2020 +0000 +++ b/src/njs.h Mon May 25 14:21:14 2020 +0000 @@ -11,7 +11,7 @@ #include -#define NJS_VERSION "0.4.1" +#define NJS_VERSION "0.4.2" #include /* STDOUT_FILENO, STDERR_FILENO */ From xeioex at nginx.com Mon May 25 14:23:49 2020 From: xeioex at nginx.com (Dmitry Volyntsev) Date: Mon, 25 May 2020 14:23:49 +0000 Subject: [njs] Introduced quick sort implementation. Message-ID: details: https://hg.nginx.org/njs/rev/4fe4575ff3de branches: changeset: 1394:4fe4575ff3de user: Dmitry Volyntsev date: Mon May 25 14:21:20 2020 +0000 description: Introduced quick sort implementation. diffstat: auto/sources | 1 + src/njs_array.c | 6 +- src/njs_main.h | 1 + src/njs_utils.c | 330 ++++++++++++++++++++++++++++++++++++++++++++ src/njs_utils.h | 16 ++ src/test/njs_unit_test.c | 130 +++++++++++++++++ src/test/rbtree_unit_test.c | 6 +- 7 files changed, 484 insertions(+), 6 deletions(-) diffs (576 lines): diff -r ab19ee68d945 -r 4fe4575ff3de auto/sources --- a/auto/sources Mon May 25 14:21:14 2020 +0000 +++ b/auto/sources Mon May 25 14:21:20 2020 +0000 @@ -20,6 +20,7 @@ NJS_LIB_SRCS=" \ src/njs_malloc.c \ src/njs_mp.c \ src/njs_sprintf.c \ + src/njs_utils.c \ src/njs_chb.c \ src/njs_value.c \ src/njs_vm.c \ diff -r ab19ee68d945 -r 4fe4575ff3de src/njs_array.c --- a/src/njs_array.c Mon May 25 14:21:14 2020 +0000 +++ b/src/njs_array.c Mon May 25 14:21:20 2020 +0000 @@ -1522,7 +1522,7 @@ njs_array_prototype_join(njs_vm_t *vm, n static int -njs_array_indices_handler(const void *first, const void *second) +njs_array_indices_handler(const void *first, const void *second, void *ctx) { double num1, num2; int64_t diff; @@ -1572,8 +1572,8 @@ njs_array_keys(njs_vm_t *vm, njs_value_t return NULL; } - qsort(keys->start, keys->length, sizeof(njs_value_t), - njs_array_indices_handler); + njs_qsort(keys->start, keys->length, sizeof(njs_value_t), + njs_array_indices_handler, NULL); return keys; } diff -r ab19ee68d945 -r 4fe4575ff3de src/njs_main.h --- a/src/njs_main.h Mon May 25 14:21:14 2020 +0000 +++ b/src/njs_main.h Mon May 25 14:21:20 2020 +0000 @@ -32,6 +32,7 @@ #include #include #include +#include #include #include diff -r ab19ee68d945 -r 4fe4575ff3de src/njs_utils.c --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/njs_utils.c Mon May 25 14:21:20 2020 +0000 @@ -0,0 +1,330 @@ + +/* + * Copyright (C) Dmitry Volyntsev + * Copyright (C) NGINX, Inc. + */ + + +#include + + +typedef void (*njs_swap_t) (void *a, void *b, size_t size); + + +njs_inline void +njs_swap_u8(void *a, void *b, size_t size) +{ + uint8_t u, *au, *bu; + + au = (uint8_t *) a; + bu = (uint8_t *) b; + + u = au[0]; + au[0] = bu[0]; + bu[0] = u; +} + + +njs_inline void +njs_swap_u16(void *a, void *b, size_t size) +{ + uint16_t u, *au, *bu; + + au = (uint16_t *) a; + bu = (uint16_t *) b; + + u = au[0]; + au[0] = bu[0]; + bu[0] = u; +} + + +njs_inline void +njs_swap_u32(void *a, void *b, size_t size) +{ + uint32_t u, *au, *bu; + + au = (uint32_t *) a; + bu = (uint32_t *) b; + + u = au[0]; + au[0] = bu[0]; + bu[0] = u; +} + + +njs_inline void +njs_swap_u64(void *a, void *b, size_t size) +{ + uint64_t u, *au, *bu; + + au = (uint64_t *) a; + bu = (uint64_t *) b; + + u = au[0]; + au[0] = bu[0]; + bu[0] = u; +} + + +njs_inline void +njs_swap_u128(void *a, void *b, size_t size) +{ + uint64_t u, v, *au, *bu; + + au = (uint64_t *) a; + bu = (uint64_t *) b; + + u = au[0]; + v = au[1]; + au[0] = bu[0]; + au[1] = bu[1]; + bu[0] = u; + bu[1] = v; +} + + +njs_inline void +njs_swap_u128x(void *a, void *b, size_t size) +{ + uint64_t u, v, *au, *bu; + + au = (uint64_t *) a; + bu = (uint64_t *) b; + + do { + u = au[0]; + v = au[1]; + au[0] = bu[0]; + au[1] = bu[1]; + bu[0] = u; + bu[1] = v; + + size -= sizeof(uint64_t) * 2; + + au += 2; + bu += 2; + } while (size != 0); +} + + +njs_inline void +njs_swap_bytes(void *a, void *b, size_t size) +{ + uint8_t u, *au, *bu; + + au = (uint8_t *) a; + bu = (uint8_t *) b; + + while (size-- != 0) { + u = *au; + *au++ = *bu; + *bu++ = u; + } +} + + +njs_inline njs_swap_t +njs_choose_swap(size_t size) +{ + switch (size) { + case 2: + return njs_swap_u16; + case 4: + return njs_swap_u32; + case 8: + return njs_swap_u64; + case 16: + return njs_swap_u128; + default: + if ((size % 16) == 0) { + return njs_swap_u128x; + } + + if (size == 1) { + return njs_swap_u8; + } + } + + return njs_swap_bytes; +} + + +njs_inline void +njs_sift_down(u_char *base, njs_sort_cmp_t cmp, njs_swap_t swap, size_t n, + size_t esize, void *ctx, njs_uint_t i) +{ + njs_uint_t c, m; + + m = i; + + while (1) { + c = 2 * i + esize; + + if (c < n && cmp(base + m, base + c, ctx) < 0) { + m = c; + } + + c += esize; + + if (c < n && cmp(base + m, base + c, ctx) < 0) { + m = c; + } + + if (m == i) { + break; + } + + swap(base + i, base + m, esize); + i = m; + } +} + + +static void +njs_heapsort(u_char *base, size_t n, size_t esize, njs_swap_t swap, + njs_sort_cmp_t cmp, void *ctx) +{ + njs_uint_t i; + + i = (n / 2) * esize; + n = n * esize; + + for ( ;; ) { + njs_sift_down(base, cmp, swap, n, esize, ctx, i); + + if (i == 0) { + break; + } + + i -= esize; + } + + while (n > esize) { + swap(base, base + n - esize, esize); + n -= esize; + + njs_sift_down(base, cmp, swap, n, esize, ctx, 0); + } +} + + +njs_inline void * +njs_pivot(void *a, void *b, void *c, njs_sort_cmp_t cmp, void *ctx) +{ + if (cmp(a, c, ctx) < 0) { + if (cmp(b, c, ctx) < 0) { + return (cmp(a, b, ctx) < 0) ? b : a; + } + + return c; + } + + if (cmp(b, a, ctx) < 0) { + return (cmp(b, c, ctx) < 0) ? c : b; + } + + return a; +} + + +typedef struct { + u_char *base; + njs_uint_t elems; +} njs_qsort_state_t; + + +#define NJS_MAX_DEPTH 16 + + +void +njs_qsort(void *arr, size_t n, size_t esize, njs_sort_cmp_t cmp, void *ctx) +{ + int r; + u_char *base, *lt, *gt, *p, *end; + njs_uint_t m4; + njs_swap_t swap; + njs_qsort_state_t stack[NJS_MAX_DEPTH], *sp; + + if (n < 2) { + return; + } + + swap = njs_choose_swap(esize); + + sp = stack; + sp->base = arr; + sp->elems = n; + sp++; + + while (sp-- > stack) { + base = sp->base; + n = sp->elems; + end = base + n * esize; + + while (n > 6) { + if (njs_slow_path(sp == &stack[NJS_MAX_DEPTH - 1])) { + njs_heapsort(base, n, esize, swap, cmp, ctx); + end = base; + break; + } + + m4 = (n / 4) * esize; + p = njs_pivot(base + m4, base + 2 * m4, base + 3 * m4, cmp, ctx); + swap(base, p, esize); + + /** + * Partition + * < mid | == mid | unprocessed | mid > + * lt p gt + */ + + lt = base; + gt = end; + p = lt + esize; + + while (p < gt) { + r = cmp(p, lt, ctx); + + if (r <= 0) { + if (r < 0) { + swap(lt, p, esize); + lt += esize; + } + + p += esize; + continue; + } + + swap(gt - esize, p, esize); + gt -= esize; + } + + if (lt - base > end - gt) { + sp->base = base; + sp->elems = (lt - base) / esize; + + base = gt; + n = (end - gt) / esize; + + } else { + sp->base = gt; + sp->elems = (end - gt) / esize; + + n = (lt - base) / esize; + } + + end = base + n * esize; + sp++; + } + + /* Insertion sort. */ + + for (p = base + esize; p < end; p += esize) { + while (p > base && cmp(p, p - esize, ctx) < 0) { + swap(p, p - esize, esize); + p -= esize; + } + } + } +} diff -r ab19ee68d945 -r 4fe4575ff3de src/njs_utils.h --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/njs_utils.h Mon May 25 14:21:20 2020 +0000 @@ -0,0 +1,16 @@ + +/* + * Copyright (C) Dmitry Volyntsev + * Copyright (C) NGINX, Inc. + */ + +#ifndef _NJS_UTILS_H_INCLUDED_ +#define _NJS_UTILS_H_INCLUDED_ + + +typedef int (*njs_sort_cmp_t)(const void *, const void *, void *ctx); + +void njs_qsort(void *base, size_t n, size_t size, njs_sort_cmp_t cmp, + void *ctx); + +#endif /* _NJS_UTILS_H_INCLUDED_ */ diff -r ab19ee68d945 -r 4fe4575ff3de src/test/njs_unit_test.c --- a/src/test/njs_unit_test.c Mon May 25 14:21:14 2020 +0000 +++ b/src/test/njs_unit_test.c Mon May 25 14:21:20 2020 +0000 @@ -17917,6 +17917,134 @@ done: } +typedef struct { + size_t size; + uint32_t array[32]; + size_t esize; + uint32_t expected[32]; +} njs_sort_test_t; + + +static int +njs_sort_cmp(const void *a, const void *b, void *ctx) +{ + njs_sort_test_t *c; + + c = ctx; + + switch (c->esize) { + case 1: + return *((uint8_t *) a) - *((uint8_t *) b); + case 2: + return *((uint16_t *) a) - *((uint16_t *) b); + case 4: + default: + return *((uint32_t *) a) - *((uint32_t *) b); + } +} + + +static njs_int_t +njs_sort_test(njs_vm_t *vm, njs_opts_t *opts, njs_stat_t *stat) +{ + u_char *p; + njs_uint_t i, j, k; + njs_sort_test_t *t; + u_char array[sizeof(t->array)]; + uint32_t sorted[sizeof(t->array) / sizeof(t->array[0])]; + + static const njs_sort_test_t tests[] = { + { 1, { 5 }, 1, { 5 } }, + { 3, { 3, 2, 1 }, 1, { 1, 2, 3 } }, + { 4, { 4, 3, 2, 1 }, 1, { 1, 2, 3, 4 } }, + { 5, { 5, 4, 3, 2, 1 }, 1, { 1, 2, 3, 4, 5 } }, + { 5, { 1, 0, 9, 1, 8 }, 1, { 0, 1, 1, 8, 9 } }, + { 8, { 0, 0, 0, 0, 0, 0, 0, 0 }, 1, { 0, 0, 0, 0, 0, 0, 0, 0 } }, + { 8, { 4, 5, 1, 4, 2, 5, 5, 6 }, 1, { 1, 2, 4, 4, 5, 5, 5, 6 } }, + { 4, { 512, 100, 65535, 0 }, 2, { 0, 100, 512, 65535 } }, + { 3, { 65536, 3, 262141 }, 4, { 3, 65536, 262141 } }, + }; + + for (i = 0; i < njs_nitems(tests); i++) { + t = (njs_sort_test_t *) &tests[i]; + + p = array; + for (k = 0; k < t->size; k++) { + switch (t->esize) { + case 1: + *p = (uint8_t) t->array[k]; + break; + case 2: + *(uint16_t *) p = (uint16_t) t->array[k]; + break; + case 4: + default: + *(uint32_t *) p = (uint32_t) t->array[k]; + break; + } + + p += t->esize; + } + + njs_qsort(array, t->size, t->esize, njs_sort_cmp, t); + + p = array; + for (k = 0; k < t->size; k++) { + switch (t->esize) { + case 1: + sorted[k] = *p; + break; + case 2: + sorted[k] = *(uint16_t *) p; + break; + case 4: + default: + sorted[k] = *(uint32_t *) p; + break; + } + + p += t->esize; + } + + + for (k = 0; k < t->size; k++) { + if (sorted[k] != t->expected[k]) { + goto failed; + } + } + + stat->passed++; + continue; + +failed: + + njs_printf("njs_sort_test(["); + for (j = 0; j < t->size; j++) { + njs_printf("%uD%s", t->array[j], + (j < t->size - 1) ? "," : ""); + } + + njs_printf("]):\nexpected: ["); + for (j = 0; j < t->size; j++) { + njs_printf("%uD%s", t->expected[j], + (j < t->size - 1) ? "," : ""); + } + + njs_printf("]\n got: ["); + for (j = 0; j < t->size; j++) { + njs_printf("%uD%s", sorted[j], + (j < t->size - 1) ? "," : ""); + } + + njs_printf("]\n"); + + stat->failed++; + } + + return NJS_OK; +} + + static njs_int_t njs_string_to_index_test(njs_vm_t *vm, njs_opts_t *opts, njs_stat_t *stat) { @@ -18020,6 +18148,8 @@ njs_api_test(njs_opts_t *opts, njs_stat_ njs_str("njs_file_dirname_test") }, { njs_chb_test, njs_str("njs_chb_test") }, + { njs_sort_test, + njs_str("njs_sort_test") }, { njs_string_to_index_test, njs_str("njs_string_to_index_test") }, }; diff -r ab19ee68d945 -r 4fe4575ff3de src/test/rbtree_unit_test.c --- a/src/test/rbtree_unit_test.c Mon May 25 14:21:14 2020 +0000 +++ b/src/test/rbtree_unit_test.c Mon May 25 14:21:20 2020 +0000 @@ -18,7 +18,7 @@ static intptr_t rbtree_unit_test_compari njs_rbtree_node_t *node2); static njs_int_t rbtree_unit_test_compare(uint32_t key1, uint32_t key2); static int njs_cdecl rbtree_unit_test_sort_cmp(const void *one, - const void *two); + const void *two, void *ctx); static njs_int_t @@ -56,7 +56,7 @@ rbtree_unit_test(njs_uint_t n) items[i].key = key; } - qsort(keys, n, sizeof(uint32_t), rbtree_unit_test_sort_cmp); + njs_qsort(keys, n, sizeof(uint32_t), rbtree_unit_test_sort_cmp, NULL); for (i = 0; i < n; i++) { njs_rbtree_insert(&tree, &items[i].node); @@ -161,7 +161,7 @@ rbtree_unit_test_compare(uint32_t key1, static int njs_cdecl -rbtree_unit_test_sort_cmp(const void *one, const void *two) +rbtree_unit_test_sort_cmp(const void *one, const void *two, void *ctx) { const uint32_t *first, *second; From xeioex at nginx.com Mon May 25 14:23:51 2020 From: xeioex at nginx.com (Dmitry Volyntsev) Date: Mon, 25 May 2020 14:23:51 +0000 Subject: [njs] Introduced njs_errno_string(). Message-ID: details: https://hg.nginx.org/njs/rev/6c46a9004ad8 branches: changeset: 1395:6c46a9004ad8 user: Artem S. Povalyukhin date: Mon May 11 09:58:28 2020 +0300 description: Introduced njs_errno_string(). diffstat: src/njs_utils.c | 337 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++ src/njs_utils.h | 2 + 2 files changed, 339 insertions(+), 0 deletions(-) diffs (354 lines): diff -r 4fe4575ff3de -r 6c46a9004ad8 src/njs_utils.c --- a/src/njs_utils.c Mon May 25 14:21:20 2020 +0000 +++ b/src/njs_utils.c Mon May 11 09:58:28 2020 +0300 @@ -328,3 +328,340 @@ njs_qsort(void *arr, size_t n, size_t es } } } + + +#define njs_errno_case(e) \ + case e: \ + return #e; + + +const char* +njs_errno_string(int errnum) +{ + switch (errnum) { +#ifdef EACCES + njs_errno_case(EACCES); +#endif + +#ifdef EADDRINUSE + njs_errno_case(EADDRINUSE); +#endif + +#ifdef EADDRNOTAVAIL + njs_errno_case(EADDRNOTAVAIL); +#endif + +#ifdef EAFNOSUPPORT + njs_errno_case(EAFNOSUPPORT); +#endif + +#ifdef EAGAIN + njs_errno_case(EAGAIN); +#endif + +#ifdef EWOULDBLOCK +#if EAGAIN != EWOULDBLOCK + njs_errno_case(EWOULDBLOCK); +#endif +#endif + +#ifdef EALREADY + njs_errno_case(EALREADY); +#endif + +#ifdef EBADF + njs_errno_case(EBADF); +#endif + +#ifdef EBADMSG + njs_errno_case(EBADMSG); +#endif + +#ifdef EBUSY + njs_errno_case(EBUSY); +#endif + +#ifdef ECANCELED + njs_errno_case(ECANCELED); +#endif + +#ifdef ECHILD + njs_errno_case(ECHILD); +#endif + +#ifdef ECONNABORTED + njs_errno_case(ECONNABORTED); +#endif + +#ifdef ECONNREFUSED + njs_errno_case(ECONNREFUSED); +#endif + +#ifdef ECONNRESET + njs_errno_case(ECONNRESET); +#endif + +#ifdef EDEADLK + njs_errno_case(EDEADLK); +#endif + +#ifdef EDESTADDRREQ + njs_errno_case(EDESTADDRREQ); +#endif + +#ifdef EDOM + njs_errno_case(EDOM); +#endif + +#ifdef EDQUOT + njs_errno_case(EDQUOT); +#endif + +#ifdef EEXIST + njs_errno_case(EEXIST); +#endif + +#ifdef EFAULT + njs_errno_case(EFAULT); +#endif + +#ifdef EFBIG + njs_errno_case(EFBIG); +#endif + +#ifdef EHOSTUNREACH + njs_errno_case(EHOSTUNREACH); +#endif + +#ifdef EIDRM + njs_errno_case(EIDRM); +#endif + +#ifdef EILSEQ + njs_errno_case(EILSEQ); +#endif + +#ifdef EINPROGRESS + njs_errno_case(EINPROGRESS); +#endif + +#ifdef EINTR + njs_errno_case(EINTR); +#endif + +#ifdef EINVAL + njs_errno_case(EINVAL); +#endif + +#ifdef EIO + njs_errno_case(EIO); +#endif + +#ifdef EISCONN + njs_errno_case(EISCONN); +#endif + +#ifdef EISDIR + njs_errno_case(EISDIR); +#endif + +#ifdef ELOOP + njs_errno_case(ELOOP); +#endif + +#ifdef EMFILE + njs_errno_case(EMFILE); +#endif + +#ifdef EMLINK + njs_errno_case(EMLINK); +#endif + +#ifdef EMSGSIZE + njs_errno_case(EMSGSIZE); +#endif + +#ifdef EMULTIHOP + njs_errno_case(EMULTIHOP); +#endif + +#ifdef ENAMETOOLONG + njs_errno_case(ENAMETOOLONG); +#endif + +#ifdef ENETDOWN + njs_errno_case(ENETDOWN); +#endif + +#ifdef ENETRESET + njs_errno_case(ENETRESET); +#endif + +#ifdef ENETUNREACH + njs_errno_case(ENETUNREACH); +#endif + +#ifdef ENFILE + njs_errno_case(ENFILE); +#endif + +#ifdef ENOBUFS + njs_errno_case(ENOBUFS); +#endif + +#ifdef ENODATA + njs_errno_case(ENODATA); +#endif + +#ifdef ENODEV + njs_errno_case(ENODEV); +#endif + +#ifdef ENOENT + njs_errno_case(ENOENT); +#endif + +#ifdef ENOEXEC + njs_errno_case(ENOEXEC); +#endif + +#ifdef ENOLINK + njs_errno_case(ENOLINK); +#endif + +#ifdef ENOLCK +#if ENOLINK != ENOLCK + njs_errno_case(ENOLCK); +#endif +#endif + +#ifdef ENOMEM + njs_errno_case(ENOMEM); +#endif + +#ifdef ENOMSG + njs_errno_case(ENOMSG); +#endif + +#ifdef ENOPROTOOPT + njs_errno_case(ENOPROTOOPT); +#endif + +#ifdef ENOSPC + njs_errno_case(ENOSPC); +#endif + +#ifdef ENOSR + njs_errno_case(ENOSR); +#endif + +#ifdef ENOSTR + njs_errno_case(ENOSTR); +#endif + +#ifdef ENOSYS + njs_errno_case(ENOSYS); +#endif + +#ifdef ENOTCONN + njs_errno_case(ENOTCONN); +#endif + +#ifdef ENOTDIR + njs_errno_case(ENOTDIR); +#endif + +#ifdef ENOTEMPTY +#if ENOTEMPTY != EEXIST + njs_errno_case(ENOTEMPTY); +#endif +#endif + +#ifdef ENOTSOCK + njs_errno_case(ENOTSOCK); +#endif + +#ifdef ENOTSUP + njs_errno_case(ENOTSUP); +#else +#ifdef EOPNOTSUPP + njs_errno_case(EOPNOTSUPP); +#endif +#endif + +#ifdef ENOTTY + njs_errno_case(ENOTTY); +#endif + +#ifdef ENXIO + njs_errno_case(ENXIO); +#endif + +#ifdef EOVERFLOW + njs_errno_case(EOVERFLOW); +#endif + +#ifdef EPERM + njs_errno_case(EPERM); +#endif + +#ifdef EPIPE + njs_errno_case(EPIPE); +#endif + +#ifdef EPROTO + njs_errno_case(EPROTO); +#endif + +#ifdef EPROTONOSUPPORT + njs_errno_case(EPROTONOSUPPORT); +#endif + +#ifdef EPROTOTYPE + njs_errno_case(EPROTOTYPE); +#endif + +#ifdef ERANGE + njs_errno_case(ERANGE); +#endif + +#ifdef EROFS + njs_errno_case(EROFS); +#endif + +#ifdef ESPIPE + njs_errno_case(ESPIPE); +#endif + +#ifdef ESRCH + njs_errno_case(ESRCH); +#endif + +#ifdef ESTALE + njs_errno_case(ESTALE); +#endif + +#ifdef ETIME + njs_errno_case(ETIME); +#endif + +#ifdef ETIMEDOUT + njs_errno_case(ETIMEDOUT); +#endif + +#ifdef ETXTBSY + njs_errno_case(ETXTBSY); +#endif + +#ifdef EXDEV + njs_errno_case(EXDEV); +#endif + + default: + break; + } + + return "UNKNOWN CODE"; +} + + diff -r 4fe4575ff3de -r 6c46a9004ad8 src/njs_utils.h --- a/src/njs_utils.h Mon May 25 14:21:20 2020 +0000 +++ b/src/njs_utils.h Mon May 11 09:58:28 2020 +0300 @@ -13,4 +13,6 @@ typedef int (*njs_sort_cmp_t)(const void void njs_qsort(void *base, size_t n, size_t size, njs_sort_cmp_t cmp, void *ctx); +const char *njs_errno_string(int errnum); + #endif /* _NJS_UTILS_H_INCLUDED_ */ From xeioex at nginx.com Mon May 25 14:23:53 2020 From: xeioex at nginx.com (Dmitry Volyntsev) Date: Mon, 25 May 2020 14:23:53 +0000 Subject: [njs] Introduced fs.Error.code. Message-ID: details: https://hg.nginx.org/njs/rev/0287ae53cbb1 branches: changeset: 1396:0287ae53cbb1 user: Artem S. Povalyukhin date: Mon May 11 09:58:28 2020 +0300 description: Introduced fs.Error.code. diffstat: src/njs_fs.c | 16 ++++++++++++++++ test/js/fs_promises_002.js | 9 +++++---- test/js/fs_promises_004.js | 6 +++--- test/njs_expect_test.exp | 6 +++--- 4 files changed, 27 insertions(+), 10 deletions(-) diffs (128 lines): diff -r 6c46a9004ad8 -r 0287ae53cbb1 src/njs_fs.c --- a/src/njs_fs.c Mon May 11 09:58:28 2020 +0300 +++ b/src/njs_fs.c Mon May 11 09:58:28 2020 +0300 @@ -873,9 +873,11 @@ njs_fs_error(njs_vm_t *vm, const char *s size_t size; njs_int_t ret; njs_value_t value; + const char *code; njs_object_t *error; static const njs_value_t string_errno = njs_string("errno"); + static const njs_value_t string_code = njs_string("code"); static const njs_value_t string_path = njs_string("path"); static const njs_value_t string_syscall = njs_string("syscall"); @@ -900,6 +902,20 @@ njs_fs_error(njs_vm_t *vm, const char *s if (njs_slow_path(ret != NJS_OK)) { return NJS_ERROR; } + + code = njs_errno_string(errn); + size = njs_strlen(code); + + ret = njs_string_new(vm, &value, (u_char *) code, size, size); + if (njs_slow_path(ret != NJS_OK)) { + return NJS_ERROR; + } + + ret = njs_value_property_set(vm, retval, njs_value_arg(&string_code), + &value); + if (njs_slow_path(ret != NJS_OK)) { + return NJS_ERROR; + } } if (path != NULL) { diff -r 6c46a9004ad8 -r 0287ae53cbb1 test/js/fs_promises_002.js --- a/test/js/fs_promises_002.js Mon May 11 09:58:28 2020 +0300 +++ b/test/js/fs_promises_002.js Mon May 11 09:58:28 2020 +0300 @@ -14,8 +14,7 @@ var testSync = new Promise((resolve, rej fs.accessSync(fname + '___'); failed = true; } catch(e) { - failed = (e.syscall != 'access'); - // TODO: e.code != 'ENOENT' + failed = (e.syscall != 'access') || e.code != 'ENOENT'; } resolve(failed); } catch (e) { @@ -33,7 +32,8 @@ var testCallback = new Promise((resolve, fs.access(fname, fs.constants.R_OK | fs.constants.W_OK, (err) => { failed |= (err !== undefined); fs.access(fname + '___', (err) => { - failed |= ((err === undefined) || (err.syscall != 'access')); + failed |= ((err === undefined) || (err.syscall != 'access') + || err.code != 'ENOENT'); resolve(failed); }); }); @@ -66,6 +66,7 @@ Promise.resolve() console.log('testPromise failed'); }) .catch((e) => { - console.log('testPromise ok', (e.syscall == 'access') && (e.path == fname + '___')); + console.log('testPromise ok', (e.syscall == 'access') && (e.path == fname + '___') + && e.code == 'ENOENT'); }) ; diff -r 6c46a9004ad8 -r 0287ae53cbb1 test/js/fs_promises_004.js --- a/test/js/fs_promises_004.js Mon May 11 09:58:28 2020 +0300 +++ b/test/js/fs_promises_004.js Mon May 11 09:58:28 2020 +0300 @@ -23,7 +23,7 @@ var testSync = () => new Promise((resolv fs.realpathSync(fname); throw new Error('fs.realpathSync error 1'); } catch (e) { - if (e.syscall != 'realpath') { // e.code + if (e.syscall != 'realpath' || e.code != 'ENOENT') { throw e; } } @@ -79,7 +79,7 @@ var testCallback = () => new Promise((re reject(new Error('fs.realpath error 1')); return; } - if (err.syscall != 'realpath') { + if (err.syscall != 'realpath' || err.code != 'ENOENT') { reject(err); return; } @@ -165,7 +165,7 @@ Promise.resolve() .then(() => fsp.realpath(fname) .then(() => { throw new Error('fsp.realpath error 1') })) .catch((e) => { - if (e.syscall != 'realpath') { + if (e.syscall != 'realpath' || e.code != 'ENOENT') { throw e; } }) diff -r 6c46a9004ad8 -r 0287ae53cbb1 test/njs_expect_test.exp --- a/test/njs_expect_test.exp Mon May 11 09:58:28 2020 +0300 +++ b/test/njs_expect_test.exp Mon May 11 09:58:28 2020 +0300 @@ -511,7 +511,7 @@ njs_test { {"var fs = require('fs'); \r\n" "undefined\r\n>> "} {"fs.readFile('test/fs/nonexistent', 'utf8', function (e) {console.log(JSON.stringify(e))})\r\n" - "undefined\r\n{\"errno\":2,\"path\":\"test/fs/nonexistent\",\"syscall\":\"open\"}\r\n>> "} + "undefined\r\n{\"errno\":2,\"code\":\"ENOENT\",\"path\":\"test/fs/nonexistent\",\"syscall\":\"open\"}\r\n>> "} } njs_test { @@ -571,7 +571,7 @@ njs_test { {"var fs = require('fs'); \r\n" "undefined\r\n>> "} {"try { fs.readFileSync('test/fs/nonexistent')} catch (e) {console.log(JSON.stringify(e))}\r\n" - "{\"errno\":2,\"path\":\"test/fs/nonexistent\",\"syscall\":\"open\"}\r\nundefined\r\n>> "} + "{\"errno\":2,\"code\":\"ENOENT\",\"path\":\"test/fs/nonexistent\",\"syscall\":\"open\"}\r\nundefined\r\n>> "} } njs_test { @@ -648,7 +648,7 @@ njs_test { {"var fs = require('fs')\r\n" "undefined\r\n>> "} {"fs.writeFile('/invalid_path', 'ABC', function (e) { console.log(JSON.stringify(e))})\r\n" - "undefined\r\n{\"errno\":13,\"path\":\"/invalid_path\",\"syscall\":\"open\"}\r\n>> "} + "undefined\r\n{\"errno\":13,\"code\":\"EACCES\",\"path\":\"/invalid_path\",\"syscall\":\"open\"}\r\n>> "} } # require('fs').writeFileSync() From xeioex at nginx.com Mon May 25 14:23:55 2020 From: xeioex at nginx.com (Dmitry Volyntsev) Date: Mon, 25 May 2020 14:23:55 +0000 Subject: [njs] Fixed Array.prototype.sort() according to the specification. Message-ID: details: https://hg.nginx.org/njs/rev/1d0825906438 branches: changeset: 1397:1d0825906438 user: Dmitry Volyntsev date: Mon May 25 14:21:22 2020 +0000 description: Fixed Array.prototype.sort() according to the specification. diffstat: src/njs_arr.c | 4 +- src/njs_arr.h | 8 +- src/njs_array.c | 363 +++++++++++++++++++++++++++++++++------------- src/test/njs_unit_test.c | 89 ++++++++++- 4 files changed, 344 insertions(+), 120 deletions(-) diffs (565 lines): diff -r 0287ae53cbb1 -r 1d0825906438 src/njs_arr.c --- a/src/njs_arr.c Mon May 11 09:58:28 2020 +0300 +++ b/src/njs_arr.c Mon May 25 14:21:22 2020 +0000 @@ -111,7 +111,7 @@ njs_arr_add_multiple(njs_arr_t *arr, njs old = arr->start; arr->start = start; - memcpy(start, old, (uint32_t) arr->items * arr->item_size); + memcpy(start, old, arr->items * arr->item_size); if (arr->separate == 0) { arr->separate = 1; @@ -121,7 +121,7 @@ njs_arr_add_multiple(njs_arr_t *arr, njs } } - item = (char *) arr->start + (uint32_t) arr->items * arr->item_size; + item = (char *) arr->start + arr->items * arr->item_size; arr->items = items; diff -r 0287ae53cbb1 -r 1d0825906438 src/njs_arr.h --- a/src/njs_arr.h Mon May 11 09:58:28 2020 +0300 +++ b/src/njs_arr.h Mon May 25 14:21:22 2020 +0000 @@ -11,11 +11,11 @@ typedef struct { void *start; /* - * A array can hold no more than 65536 items. - * The item size is no more than 64K. + * A array can hold no more than 2**32 items. + * the item size is no more than 64K. */ - uint16_t items; - uint16_t available; + uint32_t items; + uint32_t available; uint16_t item_size; uint8_t pointer; diff -r 0287ae53cbb1 -r 1d0825906438 src/njs_array.c --- a/src/njs_array.c Mon May 11 09:58:28 2020 +0300 +++ b/src/njs_array.c Mon May 25 14:21:22 2020 +0000 @@ -3046,47 +3046,126 @@ unexpected_args: } -static njs_int_t -njs_array_string_sort(njs_vm_t *vm, njs_value_t *args, njs_uint_t nargs, - njs_index_t unused) +typedef struct { + njs_value_t value; + njs_value_t *str; + int64_t pos; +} njs_array_sort_slot_t; + + +typedef struct { + njs_vm_t *vm; + njs_function_t *function; + njs_bool_t exception; + + njs_arr_t strings; +} njs_array_sort_ctx_t; + + +static int +njs_array_compare(const void *a, const void *b, void *c) { - njs_int_t ret; - njs_uint_t i; - - for (i = 1; i < nargs; i++) { - if (!njs_is_string(&args[i])) { - ret = njs_value_to_string(vm, &args[i], &args[i]); - if (ret != NJS_OK) { - return ret; - } + double num; + njs_int_t ret; + njs_value_t arguments[3], retval; + njs_array_sort_ctx_t *ctx; + njs_array_sort_slot_t *aslot, *bslot; + + ctx = c; + + if (ctx->exception) { + return 0; + } + + aslot = (njs_array_sort_slot_t *) a; + bslot = (njs_array_sort_slot_t *) b; + + if (ctx->function != NULL) { + njs_set_undefined(&arguments[0]); + arguments[1] = aslot->value; + arguments[2] = bslot->value; + + ret = njs_function_apply(ctx->vm, ctx->function, arguments, 3, &retval); + if (njs_slow_path(ret != NJS_OK)) { + goto exception; + } + + ret = njs_value_to_number(ctx->vm, &retval, &num); + if (njs_slow_path(ret != NJS_OK)) { + goto exception; + } + + if (njs_slow_path(isnan(num))) { + return 0; + } + + if (num != 0) { + return (num > 0) - (num < 0); + } + + goto compare_same; + } + + if (aslot->str == NULL) { + aslot->str = njs_arr_add(&ctx->strings); + ret = njs_value_to_string(ctx->vm, aslot->str, &aslot->value); + if (njs_slow_path(ret != NJS_OK)) { + goto exception; } } - ret = njs_string_cmp(&args[1], &args[2]); - - njs_set_number(&vm->retval, ret); - - return NJS_OK; + if (bslot->str == NULL) { + bslot->str = njs_arr_add(&ctx->strings); + ret = njs_value_to_string(ctx->vm, bslot->str, &bslot->value); + if (njs_slow_path(ret != NJS_OK)) { + goto exception; + } + } + + ret = njs_string_cmp(aslot->str, bslot->str); + + if (ret != 0) { + return ret; + } + +compare_same: + + /* Ensures stable sorting. */ + + return (aslot->pos > bslot->pos) - (aslot->pos < bslot->pos); + +exception: + + ctx->exception = 1; + + return 0; } -static const njs_function_t njs_array_string_sort_function = { - .object = { .type = NJS_FUNCTION, .shared = 1, .extensible = 1 }, - .native = 1, - .args_offset = 1, - .u.native = njs_array_string_sort, -}; - - static njs_int_t njs_array_prototype_sort(njs_vm_t *vm, njs_value_t *args, njs_uint_t nargs, njs_index_t unused) { - int64_t n, index, length, current; - njs_int_t ret; - njs_array_t *array; - njs_value_t retval, value, *this, *start, arguments[3]; - njs_function_t *function; + int64_t i, und, inv, len, nlen, length; + njs_int_t ret; + njs_array_t *array; + njs_value_t *this, *comparefn, *start, *strings, value; + njs_array_sort_ctx_t ctx; + njs_array_sort_slot_t *p, *end, *slots, *nslots; + + comparefn = njs_arg(args, nargs, 1); + + if (njs_is_defined(comparefn)) { + if (njs_slow_path(!njs_is_function(comparefn))) { + njs_type_error(vm, "comparefn must be callable or undefined"); + return NJS_ERROR; + } + + ctx.function = njs_function(comparefn); + + } else { + ctx.function = NULL; + } this = njs_argument(args, 0); @@ -3100,93 +3179,167 @@ njs_array_prototype_sort(njs_vm_t *vm, n return ret; } - if (njs_slow_path(length == 0)) { + if (njs_slow_path(length < 2)) { vm->retval = *this; return NJS_OK; } - if (nargs > 1 && njs_is_function(&args[1])) { - function = njs_function(&args[1]); + ctx.vm = vm; + ctx.exception = 0; + + if (njs_fast_path(njs_is_fast_array(this))) { + array = njs_array(this); + start = array->start; + + /** + * Moving undefined and invalid elements to the end. + * x x x | undefineds | invalids + */ + + und = 0; + inv = length; + + for (i = length - 1; i >= 0; i--) { + if (njs_is_undefined(&start[i])) { + value = start[i]; + start[i] = start[inv - und - 1]; + start[inv - und - 1] = value; + und++; + continue; + } + + if (!njs_is_valid(&start[i])) { + value = start[i]; + start[i] = start[inv - und - 1]; + start[inv - und - 1] = njs_value_undefined; + start[inv - 1] = njs_value_invalid; + inv--; + continue; + } + } + + len = inv - und; + + slots = njs_mp_alloc(vm->mem_pool, + sizeof(njs_array_sort_slot_t) * len); + if (njs_slow_path(slots == NULL)) { + return NJS_ERROR; + } + + for (i = 0; i < len; i++) { + slots[i].value = start[i]; + slots[i].pos = i; + slots[i].str = NULL; + } } else { - function = (njs_function_t *) &njs_array_string_sort_function; - } - - if (njs_slow_path(!njs_is_fast_array(this))) { - njs_internal_error(vm, "sort() is not implemented yet for objects"); - return NJS_ERROR; - } - - index = 0; - current = 0; - retval = njs_value_zero; - array = njs_array(&args[0]); - start = array->start; - -start: - - if (njs_is_number(&retval)) { - - /* - * The sort function is implemented with the insertion sort algorithm. - * Its worst and average computational complexity is O^2. This point - * should be considered as return point from comparison function so - * "goto next" moves control to the appropriate step of the algorithm. - * The first iteration also goes there because sort->retval is zero. - */ - if (njs_number(&retval) <= 0) { - goto next; + und = 0; + p = NULL; + end = NULL; + slots = NULL; + + for (i = 0; i < length; i++) { + if (p >= end) { + nlen = njs_min(njs_max((p - slots) * 2, 8), length); + nslots = njs_mp_alloc(vm->mem_pool, + sizeof(njs_array_sort_slot_t) * nlen); + if (njs_slow_path(nslots == NULL)) { + njs_memory_error(vm); + return NJS_ERROR; + } + + p = (void *) njs_cpymem(nslots, slots, + sizeof(njs_array_sort_slot_t) * (p - slots)); + + if (slots != NULL) { + njs_mp_free(vm->mem_pool, slots); + } + + slots = nslots; + end = slots + nlen; + } + + ret = njs_value_property_i64(vm, this, i, &p->value); + if (njs_slow_path(ret == NJS_ERROR)) { + ret = NJS_ERROR; + goto exception; + } + + if (ret == NJS_DECLINED) { + continue; + } + + if (njs_is_undefined(&p->value)) { + und++; + continue; + } + + p->pos = i; + p->str = NULL; + p++; } - n = index; - - swap: - - value = start[n]; - start[n] = start[n - 1]; - n--; - start[n] = value; - - do { - if (n > 0) { - - if (njs_is_valid(&start[n])) { - - if (njs_is_valid(&start[n - 1])) { - njs_set_undefined(&arguments[0]); - - /* GC: array elt, array */ - arguments[1] = start[n - 1]; - arguments[2] = start[n]; - - index = n; - - ret = njs_function_apply(vm, function, arguments, 3, - &retval); - - if (njs_slow_path(ret != NJS_OK)) { - return ret; - } - - goto start; - } - - /* Move invalid values to the end of array. */ - goto swap; + len = p - slots; + } + + strings = njs_arr_init(vm->mem_pool, &ctx.strings, NULL, len + 1, + sizeof(njs_value_t)); + if (njs_slow_path(strings == NULL)) { + ret = NJS_ERROR; + goto exception; + } + + njs_qsort(slots, len, sizeof(njs_array_sort_slot_t), njs_array_compare, + &ctx); + + if (ctx.exception) { + ret = NJS_ERROR; + goto exception; + } + + if (njs_fast_path(njs_is_fast_array(this))) { + array = njs_array(this); + start = array->start; + for (i = 0; i < len; i++) { + start[i] = slots[i].value; + } + + } else { + for (i = 0; i < len; i++) { + if (slots[i].pos != i) { + ret = njs_value_property_i64_set(vm, this, i, &slots[i].value); + if (njs_slow_path(ret == NJS_ERROR)) { + goto exception; } } - - next: - - current++; - n = current; - - } while (n < array->length); + } + + for (i = len; und-- > 0; i++) { + ret = njs_value_property_i64_set(vm, this, i, + njs_value_arg(&njs_value_undefined)); + if (njs_slow_path(ret == NJS_ERROR)) { + goto exception; + } + } + + for (; i < length; i++) { + ret = njs_value_property_i64_delete(vm, this, i, NULL); + if (njs_slow_path(ret == NJS_ERROR)) { + goto exception; + } + } } - vm->retval = args[0]; - - return NJS_OK; + vm->retval = *this; + + ret = NJS_OK; + +exception: + + njs_mp_free(vm->mem_pool, slots); + njs_arr_destroy(&ctx.strings); + + return ret; } diff -r 0287ae53cbb1 -r 1d0825906438 src/test/njs_unit_test.c --- a/src/test/njs_unit_test.c Mon May 11 09:58:28 2020 +0300 +++ b/src/test/njs_unit_test.c Mon May 25 14:21:22 2020 +0000 @@ -5979,8 +5979,55 @@ static njs_unit_test_t njs_test[] = { njs_str("var a = ['1','2','3','4','5','6']; a.sort()"), njs_str("1,2,3,4,5,6") }, + { njs_str("var a = ['a', 'ab', '', 'aa']; a.sort()"), + njs_str(",a,aa,ab") }, + + { njs_str("var a = ['a', 'ab', '', 'aa']; a.sort()"), + njs_str(",a,aa,ab") }, + + { njs_str("var a = [23,1,8,5]; Object.defineProperty(a, '0', {enumerable:false});" + "a.sort((x,y)=>x-y)"), + njs_str("1,5,8,23") }, + + { njs_str("var a = {0:23,1:1,2:8,3:5,length:4};" + "Array.prototype.sort.call(a, (x,y)=>x-y);" + "Array.prototype.join.call(a)"), + njs_str("1,5,8,23") }, + + { njs_str("var a = " NJS_LARGE_ARRAY "; " + "a[100] = 1; a[512] = -1; a[5] = undefined;" + "a.sort();" + "a[0] == -1 && a[1] == 1 && a[2] == undefined"), + njs_str("true") }, + + { njs_str("var a = " NJS_LARGE_ARRAY "; " + "a.fill(1, 256, 512); a.fill(undefined, 1000, 1010);" + "Object.defineProperty(a, '256', {value: a[256], enumerable:false});" + "a.sort();" + "a[0] == 1 && a[255] == 1 && Object.getOwnPropertyDescriptor(a, '256').value == undefined"), + njs_str("true") }, + + { njs_str("var a = [23,1,,undefined,8,,5]; Object.defineProperty(a, '0', {enumerable:false});" + "a.sort((x,y)=>x-y)"), + njs_str("1,5,8,23,,,") }, + { njs_str("var o = { toString: function() { return 5 } };" - "var a = [6,o,4,3,2,1]; a.sort()"), + "var a = [6,o,4,3,2,1]; a.sort(undefined)"), + njs_str("1,2,3,4,5,6") }, + + { njs_str("var a = [undefined,1]; a.sort()"), + njs_str("1,") }, + + { njs_str("var a = [,1]; a.sort()"), + njs_str("1,") }, + + { njs_str("var a = [,1,undefined]; a.sort()"), + njs_str("1,,") }, + + { njs_str("var a = ['1','2','3','4','5','6']; a.sort()"), + njs_str("1,2,3,4,5,6") }, + + { njs_str("var a = [1,2,3,4,5,6]; a.sort()"), njs_str("1,2,3,4,5,6") }, { njs_str("var a = {0:3,1:2,2:1}; Array.prototype.sort.call(a) === a"), @@ -5990,29 +6037,53 @@ static njs_unit_test_t njs_test[] = njs_str("true") }, { njs_str("var a = [1,2,3,4,5,6];" - "a.sort(function(x, y) { return x - y })"), + "a.sort(function(x, y) { return x - y })"), njs_str("1,2,3,4,5,6") }, - { njs_str("var a = [6,5,4,3,2,1];" - "a.sort(function(x, y) { return x - y })"), - njs_str("1,2,3,4,5,6") }, + { njs_str("var a = Array(128).fill().map((v,i,a)=>a.length-i);" + "a.sort((a,b)=>a-b);" + "a.every((v,i,a)=> (i < 1 || v >= a[i-1]))"), + njs_str("true") }, { njs_str("var a = [2,2,2,1,1,1];" - "a.sort(function(x, y) { return x - y })"), + "a.sort(function(x, y) { return x - y })"), njs_str("1,1,1,2,2,2") }, { njs_str("var a = [,,,2,2,2,1,1,1];" - "a.sort(function(x, y) { return x - y })"), + "a.sort(function(x, y) { return x - y })"), njs_str("1,1,1,2,2,2,,,") }, { njs_str("var a = [,,,,];" - "a.sort(function(x, y) { return x - y })"), + "a.sort(function(x, y) { return x - y })"), njs_str(",,,") }, + { njs_str("var a = [,,undefined,undefined,,undefined];" + "a.sort(function(x, y) { return x - y }); njs.dump(a)"), + njs_str("[undefined,undefined,undefined,,,]") }, + + { njs_str("var a = [1,,undefined,8,undefined,,undefined,,2];" + "a.sort(function(x, y) { return x - y }); njs.dump(a)"), + njs_str("[1,2,8,undefined,undefined,undefined,,,]") }, + { njs_str("var a = [1,,];" - "a.sort(function(x, y) { return x - y })"), + "a.sort(function(x, y) { return x - y })"), njs_str("1,") }, + { njs_str("var a = [{ n: 'A', r: 2 }," + " { n: 'B', r: 3 }," + " { n: 'C', r: 2 }," + " { n: 'D', r: 3 }," + " { n: 'E', r: 3 }];" + "a.sort((a, b) => b.r - a.r).map(v=>v.n).join('')"), + njs_str("BDEAC") }, + + { njs_str("var count = 0;" + "[4,3,2,1].sort(function(x, y) { if (count++ == 2) {throw Error('Oops'); }; return x - y })"), + njs_str("Error: Oops") }, + + { njs_str("[1,2].sort(1)"), + njs_str("TypeError: comparefn must be callable or undefined") }, + /* Template literal. */ { njs_str("`"), From mdounin at mdounin.ru Mon May 25 18:03:11 2020 From: mdounin at mdounin.ru (Maxim Dounin) Date: Mon, 25 May 2020 18:03:11 +0000 Subject: [nginx] HTTP/2: invalid connection preface logging (ticket #1981). Message-ID: details: https://hg.nginx.org/nginx/rev/7114d21bc2b1 branches: changeset: 7656:7114d21bc2b1 user: Maxim Dounin date: Mon May 25 18:33:42 2020 +0300 description: HTTP/2: invalid connection preface logging (ticket #1981). Previously, invalid connection preface errors were only logged at debug level, providing no visible feedback, in particular, when a plain text HTTP/2 listening socket is erroneously used for HTTP/1.x connections. Now these are explicitly logged at the info level, much like other client-related errors. diffstat: src/http/v2/ngx_http_v2.c | 10 ++++------ 1 files changed, 4 insertions(+), 6 deletions(-) diffs (27 lines): diff -r bd4d1b9db0ee -r 7114d21bc2b1 src/http/v2/ngx_http_v2.c --- a/src/http/v2/ngx_http_v2.c Sat May 23 15:53:08 2020 +0300 +++ b/src/http/v2/ngx_http_v2.c Mon May 25 18:33:42 2020 +0300 @@ -731,9 +731,8 @@ ngx_http_v2_state_preface(ngx_http_v2_co } if (ngx_memcmp(pos, preface, sizeof(preface) - 1) != 0) { - ngx_log_debug2(NGX_LOG_DEBUG_HTTP, h2c->connection->log, 0, - "invalid http2 connection preface \"%*s\"", - sizeof(preface) - 1, pos); + ngx_log_error(NGX_LOG_INFO, h2c->connection->log, 0, + "invalid connection preface"); return ngx_http_v2_connection_error(h2c, NGX_HTTP_V2_PROTOCOL_ERROR); } @@ -754,9 +753,8 @@ ngx_http_v2_state_preface_end(ngx_http_v } if (ngx_memcmp(pos, preface, sizeof(preface) - 1) != 0) { - ngx_log_debug2(NGX_LOG_DEBUG_HTTP, h2c->connection->log, 0, - "invalid http2 connection preface \"%*s\"", - sizeof(preface) - 1, pos); + ngx_log_error(NGX_LOG_INFO, h2c->connection->log, 0, + "invalid connection preface"); return ngx_http_v2_connection_error(h2c, NGX_HTTP_V2_PROTOCOL_ERROR); } From mdounin at mdounin.ru Mon May 25 19:12:13 2020 From: mdounin at mdounin.ru (Maxim Dounin) Date: Mon, 25 May 2020 19:12:13 +0000 Subject: [nginx] Updated OpenSSL used for win32 builds. Message-ID: details: https://hg.nginx.org/nginx/rev/02f2f129abc0 branches: changeset: 7657:02f2f129abc0 user: Maxim Dounin date: Mon May 25 22:10:37 2020 +0300 description: Updated OpenSSL used for win32 builds. diffstat: misc/GNUmakefile | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diffs (12 lines): diff -r 7114d21bc2b1 -r 02f2f129abc0 misc/GNUmakefile --- a/misc/GNUmakefile Mon May 25 18:33:42 2020 +0300 +++ b/misc/GNUmakefile Mon May 25 22:10:37 2020 +0300 @@ -6,7 +6,7 @@ TEMP = tmp CC = cl OBJS = objs.msvc8 -OPENSSL = openssl-1.1.1f +OPENSSL = openssl-1.1.1g ZLIB = zlib-1.2.11 PCRE = pcre-8.44 From mdounin at mdounin.ru Tue May 26 15:01:19 2020 From: mdounin at mdounin.ru (Maxim Dounin) Date: Tue, 26 May 2020 15:01:19 +0000 Subject: [nginx] nginx-1.19.0-RELEASE Message-ID: details: https://hg.nginx.org/nginx/rev/cbe6ba650211 branches: changeset: 7658:cbe6ba650211 user: Maxim Dounin date: Tue May 26 18:00:20 2020 +0300 description: nginx-1.19.0-RELEASE diffstat: docs/xml/nginx/changes.xml | 45 +++++++++++++++++++++++++++++++++++++++++++++ 1 files changed, 45 insertions(+), 0 deletions(-) diffs (55 lines): diff -r 02f2f129abc0 -r cbe6ba650211 docs/xml/nginx/changes.xml --- a/docs/xml/nginx/changes.xml Mon May 25 22:10:37 2020 +0300 +++ b/docs/xml/nginx/changes.xml Tue May 26 18:00:20 2020 +0300 @@ -5,6 +5,51 @@ + + + + +???????? ?????????? ???????????? ? ??????? OCSP. + + +client certificate validation with OCSP. + + + + + +??? ?????? ? gRPC-????????? +????? ????????? ?????? "upstream sent frame for closed stream". + + +"upstream sent frame for closed stream" errors might occur +when working with gRPC backends. + + + + + +OCSP stapling ??? ?? ????????, +???? ?? ???? ??????? ????????? resolver. + + +OCSP stapling might not work +if the "resolver" directive was not specified. + + + + + +?????????? ? ???????????? HTTP/2 preface ?? ?????????????. + + +connections with incorrect HTTP/2 preface were not logged. + + + + + + From mdounin at mdounin.ru Tue May 26 15:01:22 2020 From: mdounin at mdounin.ru (Maxim Dounin) Date: Tue, 26 May 2020 15:01:22 +0000 Subject: [nginx] release-1.19.0 tag Message-ID: details: https://hg.nginx.org/nginx/rev/d78400b2fa93 branches: changeset: 7659:d78400b2fa93 user: Maxim Dounin date: Tue May 26 18:00:20 2020 +0300 description: release-1.19.0 tag diffstat: .hgtags | 1 + 1 files changed, 1 insertions(+), 0 deletions(-) diffs (8 lines): diff -r cbe6ba650211 -r d78400b2fa93 .hgtags --- a/.hgtags Tue May 26 18:00:20 2020 +0300 +++ b/.hgtags Tue May 26 18:00:20 2020 +0300 @@ -449,3 +449,4 @@ e56295fe0ea76bf53b06bffa77a2d3a9a335cb8c fdacd273711ddf20f778c1fb91529ab53979a454 release-1.17.8 5e8d52bca714d4b85284ddb649d1ba4a3ca978a8 release-1.17.9 c44970de01474f6f3e01b0adea85ec1d03e3a5f2 release-1.17.10 +cbe6ba650211541310618849168631ce0b788f35 release-1.19.0 From gmm at csdoc.com Tue May 26 16:21:55 2020 From: gmm at csdoc.com (Gena Makhomed) Date: Tue, 26 May 2020 19:21:55 +0300 Subject: [PATCH] Contrib: vim syntax, update core and 3rd party module directives. Message-ID: # HG changeset patch # User Gena Makhomed # Date 1590509831 -10800 # Tue May 26 19:17:11 2020 +0300 # Node ID f40470c7b2e00cca839c5f0ec15a404705f7ea0e # Parent 02f2f129abc013707b184cf4bb9ac2d07760be27 Contrib: vim syntax, update core and 3rd party module directives. diff -r 02f2f129abc0 -r f40470c7b2e0 contrib/vim/syntax/nginx.vim --- a/contrib/vim/syntax/nginx.vim Mon May 25 22:10:37 2020 +0300 +++ b/contrib/vim/syntax/nginx.vim Tue May 26 19:17:11 2020 +0300 @@ -141,6 +141,7 @@ syn keyword ngxDirective contained api syn keyword ngxDirective contained auth_basic syn keyword ngxDirective contained auth_basic_user_file +syn keyword ngxDirective contained auth_delay syn keyword ngxDirective contained auth_http syn keyword ngxDirective contained auth_http_header syn keyword ngxDirective contained auth_http_pass_client_cert @@ -332,6 +333,7 @@ syn keyword ngxDirective contained js_access syn keyword ngxDirective contained js_content syn keyword ngxDirective contained js_filter +syn keyword ngxDirective contained js_import syn keyword ngxDirective contained js_include syn keyword ngxDirective contained js_path syn keyword ngxDirective contained js_preread @@ -348,6 +350,7 @@ syn keyword ngxDirective contained least_conn syn keyword ngxDirective contained least_time syn keyword ngxDirective contained limit_conn +syn keyword ngxDirective contained limit_conn_dry_run syn keyword ngxDirective contained limit_conn_log_level syn keyword ngxDirective contained limit_conn_status syn keyword ngxDirective contained limit_conn_zone @@ -595,6 +598,9 @@ syn keyword ngxDirective contained ssl_ecdh_curve syn keyword ngxDirective contained ssl_engine syn keyword ngxDirective contained ssl_handshake_timeout +syn keyword ngxDirective contained ssl_ocsp +syn keyword ngxDirective contained ssl_ocsp_cache +syn keyword ngxDirective contained ssl_ocsp_responder syn keyword ngxDirective contained ssl_password_file syn keyword ngxDirective contained ssl_prefer_server_ciphers syn keyword ngxDirective contained ssl_preread @@ -770,6 +776,7 @@ syn keyword ngxDirectiveThirdParty contained auth_gss_force_realm syn keyword ngxDirectiveThirdParty contained auth_gss_format_full syn keyword ngxDirectiveThirdParty contained auth_gss_keytab +syn keyword ngxDirectiveThirdParty contained auth_gss_map_to_local syn keyword ngxDirectiveThirdParty contained auth_gss_realm syn keyword ngxDirectiveThirdParty contained auth_gss_service_name @@ -791,8 +798,8 @@ " AJP protocol proxy " https://github.com/yaoweibin/nginx_ajp_module +syn keyword ngxDirectiveThirdParty contained ajp_buffers syn keyword ngxDirectiveThirdParty contained ajp_buffer_size -syn keyword ngxDirectiveThirdParty contained ajp_buffers syn keyword ngxDirectiveThirdParty contained ajp_busy_buffers_size syn keyword ngxDirectiveThirdParty contained ajp_cache syn keyword ngxDirectiveThirdParty contained ajp_cache_key @@ -818,6 +825,7 @@ syn keyword ngxDirectiveThirdParty contained ajp_pass_request_body syn keyword ngxDirectiveThirdParty contained ajp_pass_request_headers syn keyword ngxDirectiveThirdParty contained ajp_read_timeout +syn keyword ngxDirectiveThirdParty contained ajp_secret syn keyword ngxDirectiveThirdParty contained ajp_send_lowat syn keyword ngxDirectiveThirdParty contained ajp_send_timeout syn keyword ngxDirectiveThirdParty contained ajp_store @@ -854,8 +862,8 @@ syn keyword ngxDirectiveThirdParty contained content_handler_type syn keyword ngxDirectiveThirdParty contained handler_code syn keyword ngxDirectiveThirdParty contained handler_name +syn keyword ngxDirectiveThirdParty contained handlers_lazy_init syn keyword ngxDirectiveThirdParty contained handler_type -syn keyword ngxDirectiveThirdParty contained handlers_lazy_init syn keyword ngxDirectiveThirdParty contained header_filter_code syn keyword ngxDirectiveThirdParty contained header_filter_name syn keyword ngxDirectiveThirdParty contained header_filter_property @@ -871,6 +879,10 @@ syn keyword ngxDirectiveThirdParty contained jvm_path syn keyword ngxDirectiveThirdParty contained jvm_var syn keyword ngxDirectiveThirdParty contained jvm_workers +syn keyword ngxDirectiveThirdParty contained log_handler_code +syn keyword ngxDirectiveThirdParty contained log_handler_name +syn keyword ngxDirectiveThirdParty contained log_handler_property +syn keyword ngxDirectiveThirdParty contained log_handler_type syn keyword ngxDirectiveThirdParty contained max_balanced_tcp_connections syn keyword ngxDirectiveThirdParty contained rewrite_handler_code syn keyword ngxDirectiveThirdParty contained rewrite_handler_name @@ -879,6 +891,7 @@ syn keyword ngxDirectiveThirdParty contained shared_map syn keyword ngxDirectiveThirdParty contained write_page_size + " Certificate Transparency " https://github.com/grahamedgecombe/nginx-ct syn keyword ngxDirectiveThirdParty contained ssl_ct @@ -942,6 +955,7 @@ syn keyword ngxDirectiveThirdParty contained fancyindex_ignore syn keyword ngxDirectiveThirdParty contained fancyindex_localtime syn keyword ngxDirectiveThirdParty contained fancyindex_name_length +syn keyword ngxDirectiveThirdParty contained fancyindex_show_dotfiles syn keyword ngxDirectiveThirdParty contained fancyindex_show_path syn keyword ngxDirectiveThirdParty contained fancyindex_time_format @@ -991,8 +1005,8 @@ syn keyword ngxDirectiveThirdParty contained nchan_benchmark_subscriber_distribution syn keyword ngxDirectiveThirdParty contained nchan_benchmark_subscribers_per_channel syn keyword ngxDirectiveThirdParty contained nchan_benchmark_time +syn keyword ngxDirectiveThirdParty contained nchan_channel_events_channel_id syn keyword ngxDirectiveThirdParty contained nchan_channel_event_string -syn keyword ngxDirectiveThirdParty contained nchan_channel_events_channel_id syn keyword ngxDirectiveThirdParty contained nchan_channel_group syn keyword ngxDirectiveThirdParty contained nchan_channel_group_accounting syn keyword ngxDirectiveThirdParty contained nchan_channel_id @@ -1000,6 +1014,10 @@ syn keyword ngxDirectiveThirdParty contained nchan_channel_timeout syn keyword ngxDirectiveThirdParty contained nchan_deflate_message_for_websocket syn keyword ngxDirectiveThirdParty contained nchan_eventsource_event +syn keyword ngxDirectiveThirdParty contained nchan_eventsource_ping_comment +syn keyword ngxDirectiveThirdParty contained nchan_eventsource_ping_data +syn keyword ngxDirectiveThirdParty contained nchan_eventsource_ping_event +syn keyword ngxDirectiveThirdParty contained nchan_eventsource_ping_interval syn keyword ngxDirectiveThirdParty contained nchan_group_location syn keyword ngxDirectiveThirdParty contained nchan_group_max_channels syn keyword ngxDirectiveThirdParty contained nchan_group_max_messages @@ -1047,10 +1065,10 @@ syn keyword ngxDirectiveThirdParty contained nchan_stub_status syn keyword ngxDirectiveThirdParty contained nchan_sub_channel_id syn keyword ngxDirectiveThirdParty contained nchan_subscribe_existing_channels_only -syn keyword ngxDirectiveThirdParty contained nchan_subscribe_request syn keyword ngxDirectiveThirdParty contained nchan_subscriber syn keyword ngxDirectiveThirdParty contained nchan_subscriber_channel_id syn keyword ngxDirectiveThirdParty contained nchan_subscriber_compound_etag_message_id +syn keyword ngxDirectiveThirdParty contained nchan_subscribe_request syn keyword ngxDirectiveThirdParty contained nchan_subscriber_first_message syn keyword ngxDirectiveThirdParty contained nchan_subscriber_http_raw_stream_separator syn keyword ngxDirectiveThirdParty contained nchan_subscriber_last_message_id @@ -1987,11 +2005,7 @@ " update upstreams' config by restful interface " https://github.com/yzprofile/ngx_http_dyups_module syn keyword ngxDirectiveThirdParty contained dyups_interface -syn keyword ngxDirectiveThirdParty contained dyups_read_msg_log -syn keyword ngxDirectiveThirdParty contained dyups_read_msg_timeout syn keyword ngxDirectiveThirdParty contained dyups_shm_zone_size -syn keyword ngxDirectiveThirdParty contained dyups_trylock -syn keyword ngxDirectiveThirdParty contained dyups_upstream_conf " add given content to the end of the response according to the condition specified " https://github.com/flygoast/ngx_http_footer_if_filter @@ -2308,6 +2322,62 @@ " https://github.com/flygoast/ngx_http_upstream_ketama_chash syn keyword ngxDirectiveThirdParty contained ketama_chash +" nginx-sticky-module-ng +" https://github.com/ayty-adrianomartins/nginx-sticky-module-ng +syn keyword ngxDirectiveThirdParty contained sticky_no_fallback + +" dynamic linking and call the function of your application +" https://github.com/Taymindis/nginx-link-function +syn keyword ngxDirectiveThirdParty contained ngx_link_func_add_prop +syn keyword ngxDirectiveThirdParty contained ngx_link_func_add_req_header +syn keyword ngxDirectiveThirdParty contained ngx_link_func_ca_cert +syn keyword ngxDirectiveThirdParty contained ngx_link_func_call +syn keyword ngxDirectiveThirdParty contained ngx_link_func_download_link_lib +syn keyword ngxDirectiveThirdParty contained ngx_link_func_lib +syn keyword ngxDirectiveThirdParty contained ngx_link_func_shm_size +syn keyword ngxDirectiveThirdParty contained ngx_link_func_subrequest + +" purge content from FastCGI, proxy, SCGI and uWSGI caches +" https://github.com/torden/ngx_cache_purge +syn keyword ngxDirectiveThirdParty contained cache_purge_response_type + +" set the flags "HttpOnly", "secure" and "SameSite" for cookies +" https://github.com/AirisX/nginx_cookie_flag_module +syn keyword ngxDirectiveThirdParty contained set_cookie_flag + +" Embed websockify into Nginx (convert any tcp connection into websocket) +" https://github.com/tg123/websockify-nginx-module +syn keyword ngxDirectiveThirdParty contained websockify_buffer_size +syn keyword ngxDirectiveThirdParty contained websockify_connect_timeout +syn keyword ngxDirectiveThirdParty contained websockify_pass +syn keyword ngxDirectiveThirdParty contained websockify_read_timeout +syn keyword ngxDirectiveThirdParty contained websockify_send_timeout + +" IP2Location Nginx +" https://github.com/ip2location/ip2location-nginx +syn keyword ngxDirectiveThirdParty contained ip2location +syn keyword ngxDirectiveThirdParty contained ip2location_access_type +syn keyword ngxDirectiveThirdParty contained ip2location_proxy +syn keyword ngxDirectiveThirdParty contained ip2location_proxy_recursive + +" IP2Proxy module for Nginx +" https://github.com/ip2location/ip2proxy-nginx +syn keyword ngxDirectiveThirdParty contained ip2proxy +syn keyword ngxDirectiveThirdParty contained ip2proxy_access_type +syn keyword ngxDirectiveThirdParty contained ip2proxy_as +syn keyword ngxDirectiveThirdParty contained ip2proxy_asn +syn keyword ngxDirectiveThirdParty contained ip2proxy_city +syn keyword ngxDirectiveThirdParty contained ip2proxy_country_long +syn keyword ngxDirectiveThirdParty contained ip2proxy_country_short +syn keyword ngxDirectiveThirdParty contained ip2proxy_database +syn keyword ngxDirectiveThirdParty contained ip2proxy_domain +syn keyword ngxDirectiveThirdParty contained ip2proxy_is_proxy +syn keyword ngxDirectiveThirdParty contained ip2proxy_isp +syn keyword ngxDirectiveThirdParty contained ip2proxy_last_seen +syn keyword ngxDirectiveThirdParty contained ip2proxy_proxy_type +syn keyword ngxDirectiveThirdParty contained ip2proxy_region +syn keyword ngxDirectiveThirdParty contained ip2proxy_reverse_proxy +syn keyword ngxDirectiveThirdParty contained ip2proxy_usage_type From xeioex at nginx.com Tue May 26 18:36:15 2020 From: xeioex at nginx.com (Dmitry Volyntsev) Date: Tue, 26 May 2020 18:36:15 +0000 Subject: [njs] Added fs.mkdir(), fs.rmdir() and friends. Message-ID: details: https://hg.nginx.org/njs/rev/d6fb90ffe4c9 branches: changeset: 1398:d6fb90ffe4c9 user: Artem S. Povalyukhin date: Tue May 12 12:31:19 2020 +0300 description: Added fs.mkdir(), fs.rmdir() and friends. diffstat: src/njs_fs.c | 210 +++++++++++++++++++++++++++++++++++++++++++++ test/js/fs_promises_005.js | 143 ++++++++++++++++++++++++++++++ test/njs_expect_test.exp | 5 + 3 files changed, 358 insertions(+), 0 deletions(-) diffs (393 lines): diff -r 1d0825906438 -r d6fb90ffe4c9 src/njs_fs.c --- a/src/njs_fs.c Mon May 25 14:21:22 2020 +0000 +++ b/src/njs_fs.c Tue May 12 12:31:19 2020 +0300 @@ -729,6 +729,168 @@ done: static njs_int_t +njs_fs_mkdir(njs_vm_t *vm, njs_value_t *args, njs_uint_t nargs, + njs_index_t calltype) +{ + mode_t md; + njs_int_t ret; + const char *file_path; + njs_value_t mode, recursive, retval, *path, *callback, *options; + + static const njs_value_t string_mode = njs_string("mode"); + static const njs_value_t string_recursive = njs_string("recursive"); + + path = njs_arg(args, nargs, 1); + ret = njs_fs_path_arg(vm, &file_path, path, &njs_str_value("path")); + if (njs_slow_path(ret != NJS_OK)) { + return ret; + } + + callback = NULL; + options = njs_arg(args, nargs, 2); + + if (njs_slow_path(calltype == NJS_FS_CALLBACK)) { + callback = njs_arg(args, nargs, njs_min(nargs - 1, 3)); + if (!njs_is_function(callback)) { + njs_type_error(vm, "\"callback\" must be a function"); + return NJS_ERROR; + } + if (options == callback) { + options = njs_value_arg(&njs_value_undefined); + } + } + + njs_set_undefined(&mode); + njs_set_false(&recursive); + + switch (options->type) { + case NJS_NUMBER: + mode = *options; + break; + + case NJS_UNDEFINED: + break; + + default: + if (!njs_is_object(options)) { + njs_type_error(vm, "Unknown options type: \"%s\" " + "(a number or object required)", + njs_type_string(options->type)); + return NJS_ERROR; + } + + ret = njs_value_property(vm, options, njs_value_arg(&string_mode), + &mode); + if (njs_slow_path(ret == NJS_ERROR)) { + return ret; + } + + ret = njs_value_property(vm, options, njs_value_arg(&string_recursive), + &recursive); + if (njs_slow_path(ret == NJS_ERROR)) { + return ret; + } + } + + md = njs_fs_mode(vm, &mode, 0777); + if (njs_slow_path(md == (mode_t) -1)) { + return NJS_ERROR; + } + + if (njs_is_true(&recursive)) { + njs_type_error(vm, "\"options.recursive\" is not supported"); + return NJS_ERROR; + } + + njs_set_undefined(&retval); + + ret = mkdir(file_path, md); + if (njs_slow_path(ret != 0)) { + ret = njs_fs_error(vm, "mkdir", strerror(errno), path, errno, + &retval); + } + + if (ret == NJS_OK) { + return njs_fs_result(vm, &retval, calltype, callback, 1); + } + + return NJS_ERROR; +} + + +static njs_int_t +njs_fs_rmdir(njs_vm_t *vm, njs_value_t *args, njs_uint_t nargs, + njs_index_t calltype) +{ + njs_int_t ret; + const char *file_path; + njs_value_t recursive, retval, *path, *callback, *options; + + static const njs_value_t string_recursive = njs_string("recursive"); + + path = njs_arg(args, nargs, 1); + ret = njs_fs_path_arg(vm, &file_path, path, &njs_str_value("path")); + if (njs_slow_path(ret != NJS_OK)) { + return ret; + } + + callback = NULL; + options = njs_arg(args, nargs, 2); + + if (njs_slow_path(calltype == NJS_FS_CALLBACK)) { + callback = njs_arg(args, nargs, njs_min(nargs - 1, 3)); + if (!njs_is_function(callback)) { + njs_type_error(vm, "\"callback\" must be a function"); + return NJS_ERROR; + } + if (options == callback) { + options = njs_value_arg(&njs_value_undefined); + } + } + + njs_set_false(&recursive); + + switch (options->type) { + case NJS_UNDEFINED: + break; + + default: + if (!njs_is_object(options)) { + njs_type_error(vm, "Unknown options type: \"%s\" " + "(an object required)", + njs_type_string(options->type)); + return NJS_ERROR; + } + + ret = njs_value_property(vm, options, njs_value_arg(&string_recursive), + &recursive); + if (njs_slow_path(ret == NJS_ERROR)) { + return ret; + } + } + + if (njs_is_true(&recursive)) { + njs_type_error(vm, "\"options.recursive\" is not supported"); + return NJS_ERROR; + } + + njs_set_undefined(&retval); + + ret = rmdir(file_path); + if (njs_slow_path(ret != 0)) { + ret = njs_fs_error(vm, "rmdir", strerror(errno), path, errno, + &retval); + } + + if (ret == NJS_OK) { + return njs_fs_result(vm, &retval, calltype, callback, 1); + } + + return NJS_ERROR; +} + + +static njs_int_t njs_fs_fd_read(njs_vm_t *vm, int fd, njs_str_t *data) { u_char *p, *end, *start; @@ -1103,6 +1265,22 @@ static const njs_object_prop_t njs_fs_p { .type = NJS_PROPERTY, + .name = njs_string("mkdir"), + .value = njs_native_function2(njs_fs_mkdir, 0, NJS_FS_PROMISE), + .writable = 1, + .configurable = 1, + }, + + { + .type = NJS_PROPERTY, + .name = njs_string("rmdir"), + .value = njs_native_function2(njs_fs_rmdir, 0, NJS_FS_PROMISE), + .writable = 1, + .configurable = 1, + }, + + { + .type = NJS_PROPERTY, .name = njs_string("symlink"), .value = njs_native_function2(njs_fs_symlink, 0, NJS_FS_PROMISE), .writable = 1, @@ -1330,6 +1508,38 @@ static const njs_object_prop_t njs_fs_o .writable = 1, .configurable = 1, }, + + { + .type = NJS_PROPERTY, + .name = njs_string("mkdir"), + .value = njs_native_function2(njs_fs_mkdir, 0, NJS_FS_CALLBACK), + .writable = 1, + .configurable = 1, + }, + + { + .type = NJS_PROPERTY, + .name = njs_string("mkdirSync"), + .value = njs_native_function2(njs_fs_mkdir, 0, NJS_FS_DIRECT), + .writable = 1, + .configurable = 1, + }, + + { + .type = NJS_PROPERTY, + .name = njs_string("rmdir"), + .value = njs_native_function2(njs_fs_rmdir, 0, NJS_FS_CALLBACK), + .writable = 1, + .configurable = 1, + }, + + { + .type = NJS_PROPERTY, + .name = njs_string("rmdirSync"), + .value = njs_native_function2(njs_fs_rmdir, 0, NJS_FS_DIRECT), + .writable = 1, + .configurable = 1, + }, }; diff -r 1d0825906438 -r d6fb90ffe4c9 test/js/fs_promises_005.js --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/test/js/fs_promises_005.js Tue May 12 12:31:19 2020 +0300 @@ -0,0 +1,143 @@ +var fs = require('fs'); +var fsp = fs.promises; +var rname = './build/test/'; +var dname = rname + 'fs_promises_005'; +var dname_utf8 = rname + 'fs_promises_???_005'; +var fname = (d) => d + '/fs_promises_005_file'; + + +var testSync = () => new Promise((resolve, reject) => { + try { + try { fs.unlinkSync(fname(dname)); } catch (e) {} + try { fs.unlinkSync(fname(dname_utf8)); } catch (e) {} + try { fs.rmdirSync(dname); } catch (e) {} + try { fs.rmdirSync(dname_utf8); } catch (e) {} + + fs.mkdirSync(dname); + + try { + fs.mkdirSync(dname); + + } catch (e) { + if (e.syscall != 'mkdir' || e.code != 'EEXIST') { + throw e; + } + } + + fs.writeFileSync(fname(dname), fname(dname)); + + try { + fs.rmdirSync(dname); + + } catch (e) { + if (e.syscall != 'rmdir' + || (e.code != 'ENOTEMPTY' && e.code != 'EEXIST')) + { + throw e; + } + } + + fs.unlinkSync(fname(dname)); + + fs.rmdirSync(dname); + + fs.mkdirSync(dname_utf8, 0o555); + + try { + fs.writeFileSync(fname(dname_utf8), fname(dname_utf8)); + + } catch (e) { + if (e.syscall != 'open' || e.code != 'EACCES') { + throw e; + } + } + + try { + fs.unlinkSync(dname_utf8); + + } catch (e) { + if (e.syscall != 'unlink' || (e.code != 'EISDIR' && e.code != 'EPERM')) { + throw e; + } + } + + fs.rmdirSync(dname_utf8); + + resolve(); + + } catch (e) { + reject(e); + } +}); + + +var testCallback = () => new Promise((resolve, reject) => { + try { + try { fs.unlinkSync(fname(dname)); } catch (e) {} + try { fs.unlinkSync(fname(dname_utf8)); } catch (e) {} + try { fs.rmdirSync(dname); } catch (e) {} + try { fs.rmdirSync(dname_utf8); } catch (e) {} + + fs.mkdir(dname, (err) => { + if (err) { + reject(err); + } + + fs.mkdir(dname, (err) => { + if (!err || err.code != 'EEXIST') { + reject(new Error('fs.mkdir error 1')); + } + + fs.rmdir(dname, (err) => { + if (err) { + reject(err); + } + + resolve(); + }); + }); + }); + + } catch (e) { + reject(e); + } +}); + + +Promise.resolve() +.then(testSync) +.then(() => { + console.log('test fs.mkdirSync'); +}) +.catch((e) => { + console.log('test fs.mkdirSync failed', JSON.stringify(e)); +}) + +.then(testCallback) +.then(() => { + console.log('test fs.mkdir'); +}) +.catch((e) => { + console.log('test fs.mkdir failed', JSON.stringify(e)); +}) + +.then(() => { + try { fs.unlinkSync(fname(dname)); } catch (e) {} + try { fs.unlinkSync(fname(dname_utf8)); } catch (e) {} + try { fs.rmdirSync(dname); } catch (e) {} + try { fs.rmdirSync(dname_utf8); } catch (e) {} +}) +.then(() => fsp.mkdir(dname)) +.then(() => fsp.mkdir(dname)) +.catch((e) => { + if (e.syscall != 'mkdir' || e.code != 'EEXIST') { + throw e; + } +}) +.then(() => fsp.rmdir(dname)) +.then(() => { + console.log('test fsp.mkdir'); +}) +.catch((e) => { + console.log('test fsp.mkdir failed', JSON.stringify(e)); +}); diff -r 1d0825906438 -r d6fb90ffe4c9 test/njs_expect_test.exp --- a/test/njs_expect_test.exp Mon May 25 14:21:22 2020 +0000 +++ b/test/njs_expect_test.exp Tue May 12 12:31:19 2020 +0300 @@ -1130,3 +1130,8 @@ njs_run {"./test/js/fs_promises_004.js"} "test fs.symlinkSync test fs.symlink test fsp.symlink" + +njs_run {"./test/js/fs_promises_005.js"} \ +"test fs.mkdirSync +test fs.mkdir +test fsp.mkdir" From xeioex at nginx.com Tue May 26 18:36:17 2020 From: xeioex at nginx.com (Dmitry Volyntsev) Date: Tue, 26 May 2020 18:36:17 +0000 Subject: [njs] Improved handling of retvals of system function in "fs". Message-ID: details: https://hg.nginx.org/njs/rev/2e94c512d5c7 branches: changeset: 1399:2e94c512d5c7 user: Dmitry Volyntsev date: Tue May 26 16:21:05 2020 +0000 description: Improved handling of retvals of system function in "fs". diffstat: src/njs_fs.c | 30 ++++++++++-------------------- 1 files changed, 10 insertions(+), 20 deletions(-) diffs (82 lines): diff -r d6fb90ffe4c9 -r 2e94c512d5c7 src/njs_fs.c --- a/src/njs_fs.c Tue May 12 12:31:19 2020 +0300 +++ b/src/njs_fs.c Tue May 26 16:21:05 2020 +0000 @@ -458,16 +458,15 @@ njs_fs_rename_sync(njs_vm_t *vm, njs_val return ret; } + njs_set_undefined(&vm->retval); + ret = rename(old_path, new_path); if (njs_slow_path(ret != 0)) { - (void) njs_fs_error(vm, "rename", strerror(errno), NULL, errno, + ret = njs_fs_error(vm, "rename", strerror(errno), NULL, errno, &vm->retval); - return NJS_ERROR; } - njs_set_undefined(&vm->retval); - - return NJS_OK; + return ret; } @@ -515,16 +514,13 @@ njs_fs_access(njs_vm_t *vm, njs_value_t return NJS_ERROR; } + njs_set_undefined(&retval); + ret = access(file_path, md); if (njs_slow_path(ret != 0)) { ret = njs_fs_error(vm, "access", strerror(errno), path, errno, &retval); - goto done; } - njs_set_undefined(&retval); - -done: - if (ret == NJS_OK) { return njs_fs_result(vm, &retval, calltype, callback, 1); } @@ -573,17 +569,14 @@ njs_fs_symlink(njs_vm_t *vm, njs_value_t return NJS_ERROR; } + njs_set_undefined(&retval); + ret = symlink(target_path, file_path); if (njs_slow_path(ret != 0)) { ret = njs_fs_error(vm, "symlink", strerror(errno), path, errno, &retval); - goto done; } - njs_set_undefined(&retval); - -done: - if (ret == NJS_OK) { return njs_fs_result(vm, &retval, calltype, callback, 1); } @@ -616,16 +609,13 @@ njs_fs_unlink(njs_vm_t *vm, njs_value_t } } + njs_set_undefined(&retval); + ret = unlink(file_path); if (njs_slow_path(ret != 0)) { ret = njs_fs_error(vm, "unlink", strerror(errno), path, errno, &retval); - goto done; } - njs_set_undefined(&retval); - -done: - if (ret == NJS_OK) { return njs_fs_result(vm, &retval, calltype, callback, 1); } From xeioex at nginx.com Tue May 26 18:36:19 2020 From: xeioex at nginx.com (Dmitry Volyntsev) Date: Tue, 26 May 2020 18:36:19 +0000 Subject: [njs] Added friends functions to fs.renameSync(). Message-ID: details: https://hg.nginx.org/njs/rev/b5f4b848c60f branches: changeset: 1400:b5f4b848c60f user: Dmitry Volyntsev date: Tue May 26 16:34:45 2020 +0000 description: Added friends functions to fs.renameSync(). diffstat: src/njs_fs.c | 48 ++++++++++++++++--- test/js/fs_promises_006.js | 104 +++++++++++++++++++++++++++++++++++++++++++++ test/njs_expect_test.exp | 5 ++ 3 files changed, 148 insertions(+), 9 deletions(-) diffs (206 lines): diff -r 2e94c512d5c7 -r b5f4b848c60f src/njs_fs.c --- a/src/njs_fs.c Tue May 26 16:21:05 2020 +0000 +++ b/src/njs_fs.c Tue May 26 16:34:45 2020 +0000 @@ -440,11 +440,22 @@ done: static njs_int_t -njs_fs_rename_sync(njs_vm_t *vm, njs_value_t *args, njs_uint_t nargs, - njs_index_t unused) +njs_fs_rename(njs_vm_t *vm, njs_value_t *args, njs_uint_t nargs, + njs_index_t calltype) { - njs_int_t ret; - const char *old_path, *new_path; + njs_int_t ret; + const char *old_path, *new_path; + njs_value_t retval, *callback; + + callback = NULL; + + if (calltype == NJS_FS_CALLBACK) { + callback = njs_arg(args, nargs, 3); + if (!njs_is_function(callback)) { + njs_type_error(vm, "\"callback\" must be a function"); + return NJS_ERROR; + } + } ret = njs_fs_path_arg(vm, &old_path, njs_arg(args, nargs, 1), &njs_str_value("oldPath")); @@ -458,15 +469,18 @@ njs_fs_rename_sync(njs_vm_t *vm, njs_val return ret; } - njs_set_undefined(&vm->retval); + njs_set_undefined(&retval); ret = rename(old_path, new_path); if (njs_slow_path(ret != 0)) { - ret = njs_fs_error(vm, "rename", strerror(errno), NULL, errno, - &vm->retval); + ret = njs_fs_error(vm, "rename", strerror(errno), NULL, errno, &retval); } - return ret; + if (ret == NJS_OK) { + return njs_fs_result(vm, &retval, calltype, callback, 1); + } + + return NJS_ERROR; } @@ -1263,6 +1277,14 @@ static const njs_object_prop_t njs_fs_p { .type = NJS_PROPERTY, + .name = njs_string("rename"), + .value = njs_native_function2(njs_fs_rename, 0, NJS_FS_PROMISE), + .writable = 1, + .configurable = 1, + }, + + { + .type = NJS_PROPERTY, .name = njs_string("rmdir"), .value = njs_native_function2(njs_fs_rmdir, 0, NJS_FS_PROMISE), .writable = 1, @@ -1445,8 +1467,16 @@ static const njs_object_prop_t njs_fs_o { .type = NJS_PROPERTY, + .name = njs_string("rename"), + .value = njs_native_function2(njs_fs_rename, 0, NJS_FS_CALLBACK), + .writable = 1, + .configurable = 1, + }, + + { + .type = NJS_PROPERTY, .name = njs_string("renameSync"), - .value = njs_native_function(njs_fs_rename_sync, 0), + .value = njs_native_function2(njs_fs_rename, 0, NJS_FS_DIRECT), .writable = 1, .configurable = 1, }, diff -r 2e94c512d5c7 -r b5f4b848c60f test/js/fs_promises_006.js --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/test/js/fs_promises_006.js Tue May 26 16:34:45 2020 +0000 @@ -0,0 +1,104 @@ +var fs = require('fs'); +var fsp = fs.promises; +var dname = './build/test/'; +var fname = (d) => d + '/fs_promises_006_file'; +var fname_utf8 = (d) => d + '/fs_promises_???_006'; + +var testSync = new Promise((resolve, reject) => { + try { + try { fs.unlinkSync(fname(dname)); } catch (e) {} + try { fs.unlinkSync(fname_utf8(dname)); } catch (e) {} + + fs.writeFileSync(fname(dname), fname(dname)); + + fs.renameSync(fname(dname), fname_utf8(dname)); + + fs.accessSync(fname_utf8(dname)); + + try { + fs.renameSync(fname_utf8(dname), dname); + + } catch (e) { + if (e.syscall != 'rename' + || (e.code != 'ENOTDIR' && e.code != 'EISDIR')) + { + throw e; + } + } + + resolve(); + + } catch (e) { + reject(e); + } +}); + +var testCallback = new Promise((resolve, reject) => { + try { + try { fs.unlinkSync(fname(dname)); } catch (e) {} + try { fs.unlinkSync(fname_utf8(dname)); } catch (e) {} + + fs.writeFileSync(fname(dname), fname(dname)); + + fs.rename(fname(dname), fname_utf8(dname), err => { + if (err) { + throw err; + } + }); + + fs.accessSync(fname_utf8(dname)); + + fs.rename(fname_utf8(dname), dname, err => { + if (err.syscall != 'rename' + || (err.code != 'ENOTDIR' && err.code != 'EISDIR')) + { + throw err; + } + }); + + resolve(); + + } catch (e) { + reject(e); + } +}); + +Promise.resolve() +.then(() => testSync) +.then(() => { + console.log('test fs.renameSync'); +}) +.catch((e) => { + console.log('test fs.renameSync failed', JSON.stringify(e)); +}) + +.then(testCallback) +.then(() => { + console.log('test fs.rename'); +}) +.catch((e) => { + console.log('test fs.rename failed', JSON.stringify(e)); +}) + +.then(() => { + try { fs.unlinkSync(fname(dname)); } catch (e) {} + try { fs.unlinkSync(fname_utf8(dname)); } catch (e) {} + + fs.writeFileSync(fname(dname), fname(dname)); +}) +.then(() => fsp.rename(fname(dname), fname_utf8(dname))) +.then(() => fsp.access(fname_utf8(dname))) +.then(() => fsp.rename(fname_utf8(dname), dname)) +.catch(e => { + if (e.syscall != 'rename' + || (e.code != 'ENOTDIR' && e.code != 'EISDIR')) + { + throw e; + } +}) +.then(() => { + console.log('test fsp.rename'); +}) +.catch((e) => { + console.log('test fsp.rename failed', JSON.stringify(e)); +}); diff -r 2e94c512d5c7 -r b5f4b848c60f test/njs_expect_test.exp --- a/test/njs_expect_test.exp Tue May 26 16:21:05 2020 +0000 +++ b/test/njs_expect_test.exp Tue May 26 16:34:45 2020 +0000 @@ -1135,3 +1135,8 @@ njs_run {"./test/js/fs_promises_005.js"} "test fs.mkdirSync test fs.mkdir test fsp.mkdir" + +njs_run {"./test/js/fs_promises_006.js"} \ +"test fs.renameSync +test fs.rename +test fsp.rename" From xeioex at nginx.com Tue May 26 18:36:21 2020 From: xeioex at nginx.com (Dmitry Volyntsev) Date: Tue, 26 May 2020 18:36:21 +0000 Subject: [njs] Fixed Array.prototype.reverse() according to the specification. Message-ID: details: https://hg.nginx.org/njs/rev/db77713e0536 branches: changeset: 1401:db77713e0536 user: Dmitry Volyntsev date: Tue May 26 16:35:11 2020 +0000 description: Fixed Array.prototype.reverse() according to the specification. diffstat: src/njs_array.c | 104 +++++++++++++++++++++++++++++++++++++++------- src/test/njs_unit_test.c | 17 +++++++ 2 files changed, 104 insertions(+), 17 deletions(-) diffs (160 lines): diff -r b5f4b848c60f -r db77713e0536 src/njs_array.c --- a/src/njs_array.c Tue May 26 16:34:45 2020 +0000 +++ b/src/njs_array.c Tue May 26 16:35:11 2020 +0000 @@ -1306,10 +1306,9 @@ static njs_int_t njs_array_prototype_reverse(njs_vm_t *vm, njs_value_t *args, njs_uint_t nargs, njs_index_t unused) { - int64_t length; - njs_int_t ret; - njs_uint_t i, n; - njs_value_t value, *this; + int64_t length, l, h; + njs_int_t ret, lret, hret; + njs_value_t value, lvalue, hvalue, *this; njs_array_t *array; this = njs_argument(args, 0); @@ -1324,31 +1323,102 @@ njs_array_prototype_reverse(njs_vm_t *vm return ret; } - if (njs_slow_path(length == 0)) { + if (njs_slow_path(length < 2)) { vm->retval = *this; return NJS_OK; } - if (njs_slow_path(!njs_is_fast_array(this))) { - njs_internal_error(vm, "reverse() is not implemented yet for objects"); - return NJS_ERROR; - } - if (njs_is_fast_array(this)) { array = njs_array(this); - length = array->length; - - if (length > 1) { - for (i = 0, n = length - 1; i < n; i++, n--) { - value = array->start[i]; - array->start[i] = array->start[n]; - array->start[n] = value; + + for (l = 0, h = length - 1; l < h; l++, h--) { + if (njs_fast_path(njs_is_valid(&array->start[l]))) { + lvalue = array->start[l]; + lret = NJS_OK; + + } else { + lret = njs_value_property_i64(vm, this, l, &lvalue); + if (njs_slow_path(lret == NJS_ERROR)) { + return NJS_ERROR; + } + } + + if (njs_fast_path(njs_is_valid(&array->start[h]))) { + hvalue = array->start[h]; + hret = NJS_OK; + + } else { + hret = njs_value_property_i64(vm, this, h, &hvalue); + if (njs_slow_path(hret == NJS_ERROR)) { + return NJS_ERROR; + } + } + + if (lret == NJS_OK) { + array->start[h] = lvalue; + + if (hret == NJS_OK) { + array->start[l] = hvalue; + + } else { + array->start[h] = njs_value_invalid; + } + + } else if (hret == NJS_OK) { + array->start[l] = hvalue; + array->start[h] = njs_value_invalid; } } njs_set_array(&vm->retval, array); + return NJS_OK; } + for (l = 0, h = length - 1; l < h; l++, h--) { + lret = njs_value_property_i64(vm, this, l, &lvalue); + if (njs_slow_path(lret == NJS_ERROR)) { + return NJS_ERROR; + } + + hret = njs_value_property_i64(vm, this, h, &hvalue); + if (njs_slow_path(hret == NJS_ERROR)) { + return NJS_ERROR; + } + + if (lret == NJS_OK) { + ret = njs_value_property_i64_set(vm, this, h, &lvalue); + if (njs_slow_path(ret == NJS_ERROR)) { + return NJS_ERROR; + } + + if (hret == NJS_OK) { + ret = njs_value_property_i64_set(vm, this, l, &hvalue); + if (njs_slow_path(ret == NJS_ERROR)) { + return NJS_ERROR; + } + + } else { + ret = njs_value_property_i64_delete(vm, this, l, &value); + if (njs_slow_path(ret == NJS_ERROR)) { + return NJS_ERROR; + } + } + + } else if (hret == NJS_OK) { + ret = njs_value_property_i64_set(vm, this, l, &hvalue); + if (njs_slow_path(ret == NJS_ERROR)) { + return NJS_ERROR; + } + + ret = njs_value_property_i64_delete(vm, this, h, &value); + if (njs_slow_path(ret == NJS_ERROR)) { + return NJS_ERROR; + } + } + } + + vm->retval = *this; + return NJS_OK; } diff -r b5f4b848c60f -r db77713e0536 src/test/njs_unit_test.c --- a/src/test/njs_unit_test.c Tue May 26 16:34:45 2020 +0000 +++ b/src/test/njs_unit_test.c Tue May 26 16:35:11 2020 +0000 @@ -4509,6 +4509,23 @@ static njs_unit_test_t njs_test[] = { njs_str("var o = {1:true, 2:'', length:-2}; Array.prototype.reverse.call(o) === o"), njs_str("true") }, + { njs_str("[" + " ['a','b','c']," + " ['a','b','c','d']," + " [,'b','c','d']," + " ['a','b','c',]," + " [,'b','c',]," + "]" + ".map(v=>Object.defineProperty(v, 1, {value:v[1], enumerable:false}))" + ".map(v=>v.reverse().join(''))"), + njs_str("cba,dcba,dcb,cba,cb") }, + + { njs_str("Array.prototype[1] = 1; var x = [0]; x.length = 2; x.reverse(); x"), + njs_str("1,0") }, + + { njs_str("Array.prototype[0] = 0; var x = [,1]; x.reverse(); x"), + njs_str("1,0") }, + { njs_str("var a = [1,2,3,4]; a.indexOf()"), njs_str("-1") }, From mdounin at mdounin.ru Tue May 26 19:13:01 2020 From: mdounin at mdounin.ru (Maxim Dounin) Date: Tue, 26 May 2020 19:13:01 +0000 Subject: [nginx] Version bump. Message-ID: details: https://hg.nginx.org/nginx/rev/d33e17499088 branches: changeset: 7660:d33e17499088 user: Maxim Dounin date: Tue May 26 22:03:00 2020 +0300 description: Version bump. diffstat: src/core/nginx.h | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) diffs (14 lines): diff -r d78400b2fa93 -r d33e17499088 src/core/nginx.h --- a/src/core/nginx.h Tue May 26 18:00:20 2020 +0300 +++ b/src/core/nginx.h Tue May 26 22:03:00 2020 +0300 @@ -9,8 +9,8 @@ #define _NGINX_H_INCLUDED_ -#define nginx_version 1019000 -#define NGINX_VERSION "1.19.0" +#define nginx_version 1019001 +#define NGINX_VERSION "1.19.1" #define NGINX_VER "nginx/" NGINX_VERSION #ifdef NGX_BUILD From mdounin at mdounin.ru Tue May 26 19:13:04 2020 From: mdounin at mdounin.ru (Maxim Dounin) Date: Tue, 26 May 2020 19:13:04 +0000 Subject: [nginx] Contrib: vim syntax, update core and 3rd party module directives. Message-ID: details: https://hg.nginx.org/nginx/rev/8cadaf7e7231 branches: changeset: 7661:8cadaf7e7231 user: Gena Makhomed date: Tue May 26 19:17:11 2020 +0300 description: Contrib: vim syntax, update core and 3rd party module directives. diffstat: contrib/vim/syntax/nginx.vim | 86 +++++++++++++++++++++++++++++++++++++++---- 1 files changed, 78 insertions(+), 8 deletions(-) diffs (208 lines): diff -r d33e17499088 -r 8cadaf7e7231 contrib/vim/syntax/nginx.vim --- a/contrib/vim/syntax/nginx.vim Tue May 26 22:03:00 2020 +0300 +++ b/contrib/vim/syntax/nginx.vim Tue May 26 19:17:11 2020 +0300 @@ -141,6 +141,7 @@ syn keyword ngxDirective contained ancie syn keyword ngxDirective contained api syn keyword ngxDirective contained auth_basic syn keyword ngxDirective contained auth_basic_user_file +syn keyword ngxDirective contained auth_delay syn keyword ngxDirective contained auth_http syn keyword ngxDirective contained auth_http_header syn keyword ngxDirective contained auth_http_pass_client_cert @@ -332,6 +333,7 @@ syn keyword ngxDirective contained ip_ha syn keyword ngxDirective contained js_access syn keyword ngxDirective contained js_content syn keyword ngxDirective contained js_filter +syn keyword ngxDirective contained js_import syn keyword ngxDirective contained js_include syn keyword ngxDirective contained js_path syn keyword ngxDirective contained js_preread @@ -348,6 +350,7 @@ syn keyword ngxDirective contained large syn keyword ngxDirective contained least_conn syn keyword ngxDirective contained least_time syn keyword ngxDirective contained limit_conn +syn keyword ngxDirective contained limit_conn_dry_run syn keyword ngxDirective contained limit_conn_log_level syn keyword ngxDirective contained limit_conn_status syn keyword ngxDirective contained limit_conn_zone @@ -595,6 +598,9 @@ syn keyword ngxDirective contained ssl_e syn keyword ngxDirective contained ssl_ecdh_curve syn keyword ngxDirective contained ssl_engine syn keyword ngxDirective contained ssl_handshake_timeout +syn keyword ngxDirective contained ssl_ocsp +syn keyword ngxDirective contained ssl_ocsp_cache +syn keyword ngxDirective contained ssl_ocsp_responder syn keyword ngxDirective contained ssl_password_file syn keyword ngxDirective contained ssl_prefer_server_ciphers syn keyword ngxDirective contained ssl_preread @@ -770,6 +776,7 @@ syn keyword ngxDirectiveThirdParty conta syn keyword ngxDirectiveThirdParty contained auth_gss_force_realm syn keyword ngxDirectiveThirdParty contained auth_gss_format_full syn keyword ngxDirectiveThirdParty contained auth_gss_keytab +syn keyword ngxDirectiveThirdParty contained auth_gss_map_to_local syn keyword ngxDirectiveThirdParty contained auth_gss_realm syn keyword ngxDirectiveThirdParty contained auth_gss_service_name @@ -791,8 +798,8 @@ syn keyword ngxDirectiveThirdParty conta " AJP protocol proxy " https://github.com/yaoweibin/nginx_ajp_module +syn keyword ngxDirectiveThirdParty contained ajp_buffers syn keyword ngxDirectiveThirdParty contained ajp_buffer_size -syn keyword ngxDirectiveThirdParty contained ajp_buffers syn keyword ngxDirectiveThirdParty contained ajp_busy_buffers_size syn keyword ngxDirectiveThirdParty contained ajp_cache syn keyword ngxDirectiveThirdParty contained ajp_cache_key @@ -818,6 +825,7 @@ syn keyword ngxDirectiveThirdParty conta syn keyword ngxDirectiveThirdParty contained ajp_pass_request_body syn keyword ngxDirectiveThirdParty contained ajp_pass_request_headers syn keyword ngxDirectiveThirdParty contained ajp_read_timeout +syn keyword ngxDirectiveThirdParty contained ajp_secret syn keyword ngxDirectiveThirdParty contained ajp_send_lowat syn keyword ngxDirectiveThirdParty contained ajp_send_timeout syn keyword ngxDirectiveThirdParty contained ajp_store @@ -854,8 +862,8 @@ syn keyword ngxDirectiveThirdParty conta syn keyword ngxDirectiveThirdParty contained content_handler_type syn keyword ngxDirectiveThirdParty contained handler_code syn keyword ngxDirectiveThirdParty contained handler_name +syn keyword ngxDirectiveThirdParty contained handlers_lazy_init syn keyword ngxDirectiveThirdParty contained handler_type -syn keyword ngxDirectiveThirdParty contained handlers_lazy_init syn keyword ngxDirectiveThirdParty contained header_filter_code syn keyword ngxDirectiveThirdParty contained header_filter_name syn keyword ngxDirectiveThirdParty contained header_filter_property @@ -871,6 +879,10 @@ syn keyword ngxDirectiveThirdParty conta syn keyword ngxDirectiveThirdParty contained jvm_path syn keyword ngxDirectiveThirdParty contained jvm_var syn keyword ngxDirectiveThirdParty contained jvm_workers +syn keyword ngxDirectiveThirdParty contained log_handler_code +syn keyword ngxDirectiveThirdParty contained log_handler_name +syn keyword ngxDirectiveThirdParty contained log_handler_property +syn keyword ngxDirectiveThirdParty contained log_handler_type syn keyword ngxDirectiveThirdParty contained max_balanced_tcp_connections syn keyword ngxDirectiveThirdParty contained rewrite_handler_code syn keyword ngxDirectiveThirdParty contained rewrite_handler_name @@ -879,6 +891,7 @@ syn keyword ngxDirectiveThirdParty conta syn keyword ngxDirectiveThirdParty contained shared_map syn keyword ngxDirectiveThirdParty contained write_page_size + " Certificate Transparency " https://github.com/grahamedgecombe/nginx-ct syn keyword ngxDirectiveThirdParty contained ssl_ct @@ -942,6 +955,7 @@ syn keyword ngxDirectiveThirdParty conta syn keyword ngxDirectiveThirdParty contained fancyindex_ignore syn keyword ngxDirectiveThirdParty contained fancyindex_localtime syn keyword ngxDirectiveThirdParty contained fancyindex_name_length +syn keyword ngxDirectiveThirdParty contained fancyindex_show_dotfiles syn keyword ngxDirectiveThirdParty contained fancyindex_show_path syn keyword ngxDirectiveThirdParty contained fancyindex_time_format @@ -991,8 +1005,8 @@ syn keyword ngxDirectiveThirdParty conta syn keyword ngxDirectiveThirdParty contained nchan_benchmark_subscriber_distribution syn keyword ngxDirectiveThirdParty contained nchan_benchmark_subscribers_per_channel syn keyword ngxDirectiveThirdParty contained nchan_benchmark_time +syn keyword ngxDirectiveThirdParty contained nchan_channel_events_channel_id syn keyword ngxDirectiveThirdParty contained nchan_channel_event_string -syn keyword ngxDirectiveThirdParty contained nchan_channel_events_channel_id syn keyword ngxDirectiveThirdParty contained nchan_channel_group syn keyword ngxDirectiveThirdParty contained nchan_channel_group_accounting syn keyword ngxDirectiveThirdParty contained nchan_channel_id @@ -1000,6 +1014,10 @@ syn keyword ngxDirectiveThirdParty conta syn keyword ngxDirectiveThirdParty contained nchan_channel_timeout syn keyword ngxDirectiveThirdParty contained nchan_deflate_message_for_websocket syn keyword ngxDirectiveThirdParty contained nchan_eventsource_event +syn keyword ngxDirectiveThirdParty contained nchan_eventsource_ping_comment +syn keyword ngxDirectiveThirdParty contained nchan_eventsource_ping_data +syn keyword ngxDirectiveThirdParty contained nchan_eventsource_ping_event +syn keyword ngxDirectiveThirdParty contained nchan_eventsource_ping_interval syn keyword ngxDirectiveThirdParty contained nchan_group_location syn keyword ngxDirectiveThirdParty contained nchan_group_max_channels syn keyword ngxDirectiveThirdParty contained nchan_group_max_messages @@ -1047,10 +1065,10 @@ syn keyword ngxDirectiveThirdParty conta syn keyword ngxDirectiveThirdParty contained nchan_stub_status syn keyword ngxDirectiveThirdParty contained nchan_sub_channel_id syn keyword ngxDirectiveThirdParty contained nchan_subscribe_existing_channels_only -syn keyword ngxDirectiveThirdParty contained nchan_subscribe_request syn keyword ngxDirectiveThirdParty contained nchan_subscriber syn keyword ngxDirectiveThirdParty contained nchan_subscriber_channel_id syn keyword ngxDirectiveThirdParty contained nchan_subscriber_compound_etag_message_id +syn keyword ngxDirectiveThirdParty contained nchan_subscribe_request syn keyword ngxDirectiveThirdParty contained nchan_subscriber_first_message syn keyword ngxDirectiveThirdParty contained nchan_subscriber_http_raw_stream_separator syn keyword ngxDirectiveThirdParty contained nchan_subscriber_last_message_id @@ -1987,11 +2005,7 @@ syn keyword ngxDirectiveThirdParty conta " update upstreams' config by restful interface " https://github.com/yzprofile/ngx_http_dyups_module syn keyword ngxDirectiveThirdParty contained dyups_interface -syn keyword ngxDirectiveThirdParty contained dyups_read_msg_log -syn keyword ngxDirectiveThirdParty contained dyups_read_msg_timeout syn keyword ngxDirectiveThirdParty contained dyups_shm_zone_size -syn keyword ngxDirectiveThirdParty contained dyups_trylock -syn keyword ngxDirectiveThirdParty contained dyups_upstream_conf " add given content to the end of the response according to the condition specified " https://github.com/flygoast/ngx_http_footer_if_filter @@ -2308,6 +2322,62 @@ syn keyword ngxDirectiveThirdParty conta " https://github.com/flygoast/ngx_http_upstream_ketama_chash syn keyword ngxDirectiveThirdParty contained ketama_chash +" nginx-sticky-module-ng +" https://github.com/ayty-adrianomartins/nginx-sticky-module-ng +syn keyword ngxDirectiveThirdParty contained sticky_no_fallback + +" dynamic linking and call the function of your application +" https://github.com/Taymindis/nginx-link-function +syn keyword ngxDirectiveThirdParty contained ngx_link_func_add_prop +syn keyword ngxDirectiveThirdParty contained ngx_link_func_add_req_header +syn keyword ngxDirectiveThirdParty contained ngx_link_func_ca_cert +syn keyword ngxDirectiveThirdParty contained ngx_link_func_call +syn keyword ngxDirectiveThirdParty contained ngx_link_func_download_link_lib +syn keyword ngxDirectiveThirdParty contained ngx_link_func_lib +syn keyword ngxDirectiveThirdParty contained ngx_link_func_shm_size +syn keyword ngxDirectiveThirdParty contained ngx_link_func_subrequest + +" purge content from FastCGI, proxy, SCGI and uWSGI caches +" https://github.com/torden/ngx_cache_purge +syn keyword ngxDirectiveThirdParty contained cache_purge_response_type + +" set the flags "HttpOnly", "secure" and "SameSite" for cookies +" https://github.com/AirisX/nginx_cookie_flag_module +syn keyword ngxDirectiveThirdParty contained set_cookie_flag + +" Embed websockify into Nginx (convert any tcp connection into websocket) +" https://github.com/tg123/websockify-nginx-module +syn keyword ngxDirectiveThirdParty contained websockify_buffer_size +syn keyword ngxDirectiveThirdParty contained websockify_connect_timeout +syn keyword ngxDirectiveThirdParty contained websockify_pass +syn keyword ngxDirectiveThirdParty contained websockify_read_timeout +syn keyword ngxDirectiveThirdParty contained websockify_send_timeout + +" IP2Location Nginx +" https://github.com/ip2location/ip2location-nginx +syn keyword ngxDirectiveThirdParty contained ip2location +syn keyword ngxDirectiveThirdParty contained ip2location_access_type +syn keyword ngxDirectiveThirdParty contained ip2location_proxy +syn keyword ngxDirectiveThirdParty contained ip2location_proxy_recursive + +" IP2Proxy module for Nginx +" https://github.com/ip2location/ip2proxy-nginx +syn keyword ngxDirectiveThirdParty contained ip2proxy +syn keyword ngxDirectiveThirdParty contained ip2proxy_access_type +syn keyword ngxDirectiveThirdParty contained ip2proxy_as +syn keyword ngxDirectiveThirdParty contained ip2proxy_asn +syn keyword ngxDirectiveThirdParty contained ip2proxy_city +syn keyword ngxDirectiveThirdParty contained ip2proxy_country_long +syn keyword ngxDirectiveThirdParty contained ip2proxy_country_short +syn keyword ngxDirectiveThirdParty contained ip2proxy_database +syn keyword ngxDirectiveThirdParty contained ip2proxy_domain +syn keyword ngxDirectiveThirdParty contained ip2proxy_is_proxy +syn keyword ngxDirectiveThirdParty contained ip2proxy_isp +syn keyword ngxDirectiveThirdParty contained ip2proxy_last_seen +syn keyword ngxDirectiveThirdParty contained ip2proxy_proxy_type +syn keyword ngxDirectiveThirdParty contained ip2proxy_region +syn keyword ngxDirectiveThirdParty contained ip2proxy_reverse_proxy +syn keyword ngxDirectiveThirdParty contained ip2proxy_usage_type From mdounin at mdounin.ru Tue May 26 19:13:27 2020 From: mdounin at mdounin.ru (Maxim Dounin) Date: Tue, 26 May 2020 22:13:27 +0300 Subject: [PATCH] Contrib: vim syntax, update core and 3rd party module directives. In-Reply-To: References: Message-ID: <20200526191327.GN12747@mdounin.ru> Hello! On Tue, May 26, 2020 at 07:21:55PM +0300, Gena Makhomed wrote: > # HG changeset patch > # User Gena Makhomed > # Date 1590509831 -10800 > # Tue May 26 19:17:11 2020 +0300 > # Node ID f40470c7b2e00cca839c5f0ec15a404705f7ea0e > # Parent 02f2f129abc013707b184cf4bb9ac2d07760be27 > Contrib: vim syntax, update core and 3rd party module directives. [...] Committed, thnx. -- Maxim Dounin http://mdounin.ru/ From xeioex at nginx.com Wed May 27 13:39:47 2020 From: xeioex at nginx.com (Dmitry Volyntsev) Date: Wed, 27 May 2020 13:39:47 +0000 Subject: [njs] Fixed potential undefined behavior in memcpy(). Message-ID: details: https://hg.nginx.org/njs/rev/164801c99e9a branches: changeset: 1402:164801c99e9a user: Dmitry Volyntsev date: Tue May 26 19:02:57 2020 +0000 description: Fixed potential undefined behavior in memcpy(). The issue was introduced in 1d0825906438. Found with Clang Static Analyzer. diffstat: src/njs_array.c | 8 +++++--- 1 files changed, 5 insertions(+), 3 deletions(-) diffs (20 lines): diff -r db77713e0536 -r 164801c99e9a src/njs_array.c --- a/src/njs_array.c Tue May 26 16:35:11 2020 +0000 +++ b/src/njs_array.c Tue May 26 19:02:57 2020 +0000 @@ -3318,11 +3318,13 @@ njs_array_prototype_sort(njs_vm_t *vm, n return NJS_ERROR; } - p = (void *) njs_cpymem(nslots, slots, - sizeof(njs_array_sort_slot_t) * (p - slots)); - if (slots != NULL) { + p = (void *) njs_cpymem(nslots, slots, + sizeof(njs_array_sort_slot_t) * (p - slots)); njs_mp_free(vm->mem_pool, slots); + + } else { + p = nslots; } slots = nslots; From xeioex at nginx.com Wed May 27 13:39:49 2020 From: xeioex at nginx.com (Dmitry Volyntsev) Date: Wed, 27 May 2020 13:39:49 +0000 Subject: [njs] Removed dead store assignment after 1d0825906438. Message-ID: details: https://hg.nginx.org/njs/rev/1b8a0af35e2b branches: changeset: 1403:1b8a0af35e2b user: Dmitry Volyntsev date: Tue May 26 19:03:11 2020 +0000 description: Removed dead store assignment after 1d0825906438. Found with Clang Static Analyzer. diffstat: src/njs_array.c | 6 ++---- 1 files changed, 2 insertions(+), 4 deletions(-) diffs (29 lines): diff -r 164801c99e9a -r 1b8a0af35e2b src/njs_array.c --- a/src/njs_array.c Tue May 26 19:02:57 2020 +0000 +++ b/src/njs_array.c Tue May 26 19:03:11 2020 +0000 @@ -3219,7 +3219,7 @@ njs_array_prototype_sort(njs_vm_t *vm, n int64_t i, und, inv, len, nlen, length; njs_int_t ret; njs_array_t *array; - njs_value_t *this, *comparefn, *start, *strings, value; + njs_value_t *this, *comparefn, *start, *strings; njs_array_sort_ctx_t ctx; njs_array_sort_slot_t *p, *end, *slots, *nslots; @@ -3271,15 +3271,13 @@ njs_array_prototype_sort(njs_vm_t *vm, n for (i = length - 1; i >= 0; i--) { if (njs_is_undefined(&start[i])) { - value = start[i]; start[i] = start[inv - und - 1]; - start[inv - und - 1] = value; + start[inv - und - 1] = njs_value_undefined; und++; continue; } if (!njs_is_valid(&start[i])) { - value = start[i]; start[i] = start[inv - und - 1]; start[inv - und - 1] = njs_value_undefined; start[inv - 1] = njs_value_invalid; From xeioex at nginx.com Wed May 27 13:39:51 2020 From: xeioex at nginx.com (Dmitry Volyntsev) Date: Wed, 27 May 2020 13:39:51 +0000 Subject: [njs] Fixed use-of-uninitialized-value after 1d0825906438. Message-ID: details: https://hg.nginx.org/njs/rev/4fb07fcbb698 branches: changeset: 1404:4fb07fcbb698 user: Dmitry Volyntsev date: Wed May 27 12:02:59 2020 +0000 description: Fixed use-of-uninitialized-value after 1d0825906438. Found by Coverity (CID 1463834). diffstat: src/njs_array.c | 9 +++++++-- 1 files changed, 7 insertions(+), 2 deletions(-) diffs (34 lines): diff -r 1b8a0af35e2b -r 4fb07fcbb698 src/njs_array.c --- a/src/njs_array.c Tue May 26 19:03:11 2020 +0000 +++ b/src/njs_array.c Wed May 27 12:02:59 2020 +0000 @@ -3254,7 +3254,10 @@ njs_array_prototype_sort(njs_vm_t *vm, n return NJS_OK; } + slots = NULL; ctx.vm = vm; + ctx.strings.separate = 0; + ctx.strings.pointer = 0; ctx.exception = 0; if (njs_fast_path(njs_is_fast_array(this))) { @@ -3304,7 +3307,6 @@ njs_array_prototype_sort(njs_vm_t *vm, n und = 0; p = NULL; end = NULL; - slots = NULL; for (i = 0; i < length; i++) { if (p >= end) { @@ -3406,7 +3408,10 @@ njs_array_prototype_sort(njs_vm_t *vm, n exception: - njs_mp_free(vm->mem_pool, slots); + if (slots != NULL) { + njs_mp_free(vm->mem_pool, slots); + } + njs_arr_destroy(&ctx.strings); return ret; From alexander.borisov at nginx.com Wed May 27 15:19:39 2020 From: alexander.borisov at nginx.com (Alexander Borisov) Date: Wed, 27 May 2020 15:19:39 +0000 Subject: [njs] Fixed heap-use-after-free in JSON.parse() function. Message-ID: details: https://hg.nginx.org/njs/rev/9beb9ea093b5 branches: changeset: 1405:9beb9ea093b5 user: Alexander Borisov date: Wed May 27 18:18:40 2020 +0300 description: Fixed heap-use-after-free in JSON.parse() function. diffstat: src/njs_json.c | 24 ++++++++++++++++++++++-- src/test/njs_unit_test.c | 17 +++++++++++++++++ 2 files changed, 39 insertions(+), 2 deletions(-) diffs (64 lines): diff -r 4fb07fcbb698 -r 9beb9ea093b5 src/njs_json.c --- a/src/njs_json.c Wed May 27 12:02:59 2020 +0000 +++ b/src/njs_json.c Wed May 27 18:18:40 2020 +0300 @@ -1018,11 +1018,31 @@ njs_json_parse_iterator_call(njs_vm_t *v return ret; } + /* + * The njs_function_apply() function can convert fast array to object. + * After this conversion, there will be garbage in the value. + */ + + if (njs_fast_path(njs_is_fast_array(&state->value) + && (state->index - 1) < njs_array(&state->value)->length)) + { + if (njs_is_undefined(&parse->retval)) { + njs_set_invalid(value); + + } else { + *value = parse->retval; + } + + break; + } + if (njs_is_undefined(&parse->retval)) { - njs_set_invalid(value); + njs_value_property_i64_delete(vm, &state->value, state->index - 1, + NULL); } else { - *value = parse->retval; + njs_value_property_i64_set(vm, &state->value, state->index - 1, + &parse->retval); } break; diff -r 4fb07fcbb698 -r 9beb9ea093b5 src/test/njs_unit_test.c --- a/src/test/njs_unit_test.c Wed May 27 12:02:59 2020 +0000 +++ b/src/test/njs_unit_test.c Wed May 27 18:18:40 2020 +0300 @@ -15309,6 +15309,23 @@ static njs_unit_test_t njs_test[] = "args.join('|')"), njs_str("0:2|a:3|1:[object Object]|:2,[object Object]") }, + { njs_str("JSON.parse('[0,1,2]', function(k, v) {" + " if (v == 2) {" + " return undefined;" + " }" + " return v;" + "});"), + njs_str("0,1,") }, + + { njs_str("JSON.parse('[0,1,2]', function(k, v) {" + " if (v == 0) {" + " Object.defineProperty(this, '0', {value: undefined, enumerable: false});" + " return undefined;" + " }" + " return v;" + "});"), + njs_str(",1,2") }, + { njs_str("JSON.parse()"), njs_str("SyntaxError: Unexpected token at position 0") }, From balan.pothula at gmail.com Thu May 28 14:15:24 2020 From: balan.pothula at gmail.com (BALAJI POTHULA) Date: Thu, 28 May 2020 19:45:24 +0530 Subject: Requesting Help to Compile NGINX + LuaJIT STATIC Build Message-ID: HI..! Team, I try to Compile NGINX + LuaJIT as STATIC Build, I want to Embedding LuaJIT inside the NGINX. The below script will generate NGINX with size 17MB. https://github.com/balajipothula/nginx/blob/master/compile-nginx-on-ubuntu18.sh . I tried with OpenResty source also. The below script will generate NGINX with LuaJIT (Dynamic Linking) with size 47MB. https://github.com/balajipothula/nginx/blob/master/compile-nginx-lua-on-ubuntu18.sh I am expecting a Lean NGINX + LuaJIT only as a STATIC Build. Drag and Drop the nginx folder, It should work like a charm (If possible any Linux Family OS) I am trying to add More Power to NGINX with luaposix, same time NGINX should be Lean. Could you please Help me. nginx ??? bin ? ??? nginx ??? client_body_temp ??? conf ? ??? mime.types ? ??? nginx.conf ??? fastcgi_temp ??? html ? ??? 50x.html ? ??? index.html ??? log ? ??? access.log ? ??? error.log ? ??? nginx.pid ??? proxy_temp ??? scgi_temp ??? uwsgi_temp Thanks and Regards, BALAJI. -------------- next part -------------- An HTML attachment was scrubbed... URL: From alexander.borisov at nginx.com Fri May 29 11:50:47 2020 From: alexander.borisov at nginx.com (Alexander Borisov) Date: Fri, 29 May 2020 11:50:47 +0000 Subject: [njs] Added return value check forgotten in 9beb9ea093b5. Message-ID: details: https://hg.nginx.org/njs/rev/1d071c0e23e8 branches: changeset: 1406:1d071c0e23e8 user: Alexander Borisov date: Fri May 29 14:49:36 2020 +0300 description: Added return value check forgotten in 9beb9ea093b5. Found by Coverity (CID 1463859, 1463860). diffstat: src/njs_json.c | 12 ++++++++---- 1 files changed, 8 insertions(+), 4 deletions(-) diffs (24 lines): diff -r 9beb9ea093b5 -r 1d071c0e23e8 src/njs_json.c --- a/src/njs_json.c Wed May 27 18:18:40 2020 +0300 +++ b/src/njs_json.c Fri May 29 14:49:36 2020 +0300 @@ -1037,12 +1037,16 @@ njs_json_parse_iterator_call(njs_vm_t *v } if (njs_is_undefined(&parse->retval)) { - njs_value_property_i64_delete(vm, &state->value, state->index - 1, - NULL); + ret = njs_value_property_i64_delete(vm, &state->value, + state->index - 1, NULL); } else { - njs_value_property_i64_set(vm, &state->value, state->index - 1, - &parse->retval); + ret = njs_value_property_i64_set(vm, &state->value, + state->index - 1, &parse->retval); + } + + if (njs_slow_path(ret == NJS_ERROR)) { + return NJS_ERROR; } break; From alexander.borisov at nginx.com Fri May 29 17:01:07 2020 From: alexander.borisov at nginx.com (Alexander Borisov) Date: Fri, 29 May 2020 17:01:07 +0000 Subject: [njs] Parser refactoring. Message-ID: details: https://hg.nginx.org/njs/rev/86f55a7dc4a4 branches: changeset: 1407:86f55a7dc4a4 user: Alexander Borisov date: Fri May 29 20:00:32 2020 +0300 description: Parser refactoring. diffstat: auto/sources | 2 - src/njs_function.c | 7 +- src/njs_lexer.c | 202 +- src/njs_lexer.h | 91 +- src/njs_lexer_tables.h | 491 +- src/njs_module.c | 175 +- src/njs_module.h | 3 +- src/njs_parser.c | 9622 ++++++++++++++++++++++++++++++++++-------- src/njs_parser.h | 316 +- src/njs_parser_expression.c | 1121 ----- src/njs_parser_terminal.c | 1296 ----- src/njs_regexp.c | 95 +- src/njs_regexp.h | 4 +- src/njs_variable.c | 24 +- src/njs_vm.c | 3 +- src/test/njs_unit_test.c | 57 +- test/njs_expect_test.exp | 4 +- utils/lexer_keyword.py | 129 +- 18 files changed, 8745 insertions(+), 4897 deletions(-) diffs (truncated from 14514 to 1000 lines): diff -r 1d071c0e23e8 -r 86f55a7dc4a4 auto/sources --- a/auto/sources Fri May 29 14:49:36 2020 +0300 +++ b/auto/sources Fri May 29 20:00:32 2020 +0300 @@ -49,8 +49,6 @@ NJS_LIB_SRCS=" \ src/njs_lexer.c \ src/njs_lexer_keyword.c \ src/njs_parser.c \ - src/njs_parser_terminal.c \ - src/njs_parser_expression.c \ src/njs_generator.c \ src/njs_disassembler.c \ src/njs_array_buffer.c \ diff -r 1d071c0e23e8 -r 86f55a7dc4a4 src/njs_function.c --- a/src/njs_function.c Fri May 29 14:49:36 2020 +0300 +++ b/src/njs_function.c Fri May 29 20:00:32 2020 +0300 @@ -889,7 +889,9 @@ njs_function_constructor(njs_vm_t *vm, n return ret; } - njs_chb_append_literal(&chain, ","); + if (i != (nargs - 2)) { + njs_chb_append_literal(&chain, ","); + } } njs_chb_append_literal(&chain, "){"); @@ -923,9 +925,10 @@ njs_function_constructor(njs_vm_t *vm, n return ret; } + parser->vm = vm; parser->lexer = &lexer; - ret = njs_parser(vm, parser, NULL); + ret = njs_parser(parser, NULL); if (njs_slow_path(ret != NJS_OK)) { return ret; } diff -r 1d071c0e23e8 -r 86f55a7dc4a4 src/njs_lexer.c --- a/src/njs_lexer.c Fri May 29 14:49:36 2020 +0300 +++ b/src/njs_lexer.c Fri May 29 20:00:32 2020 +0300 @@ -27,10 +27,6 @@ static void njs_lexer_multi(njs_lexer_t const njs_lexer_multi_t *multi, size_t length); static void njs_lexer_division(njs_lexer_t *lexer, njs_lexer_token_t *token); -static njs_lexer_token_t *njs_lexer_token_push(njs_vm_t *vm, - njs_lexer_t *lexer); -static njs_lexer_token_t *njs_lexer_token_pop(njs_lexer_t *lexer); - const njs_lvlhsh_proto_t njs_lexer_hash_proto njs_aligned(64) = @@ -311,102 +307,19 @@ njs_lexer_init(njs_vm_t *vm, njs_lexer_t } -njs_token_type_t -njs_lexer_token(njs_vm_t *vm, njs_lexer_t *lexer) -{ - njs_lexer_token_t *lt; - - lexer->prev_start = lexer->start; - - if (lexer->token != NULL) { - lexer->prev_type = lexer->token->type; - njs_mp_free(vm->mem_pool, lexer->token); - } - - if (njs_queue_is_empty(&lexer->preread)) { - lt = njs_lexer_token_push(vm, lexer); - if (njs_slow_path(lt == NULL)) { - return NJS_TOKEN_ERROR; - } - } - - lexer->token = njs_lexer_token_pop(lexer); - - return lexer->token->type; -} - - -njs_token_type_t -njs_lexer_peek_token(njs_vm_t *vm, njs_lexer_t *lexer, size_t offset) -{ - size_t i; - njs_queue_link_t *link; - njs_lexer_token_t *lt; - - /* GCC and Clang complain about uninitialized lt. */ - lt = NULL; - - link = njs_queue_first(&lexer->preread); - - for (i = 0; i <= offset; i++) { - - if (link != njs_queue_tail(&lexer->preread)) { - - lt = njs_queue_link_data(link, njs_lexer_token_t, link); - - /* NJS_TOKEN_DIVISION stands for regexp literal. */ - - if (lt->type == NJS_TOKEN_DIVISION || lt->type == NJS_TOKEN_END) { - break; - } - - link = njs_queue_next(link); - - } else { - - lt = njs_lexer_token_push(vm, lexer); - - if (njs_slow_path(lt == NULL)) { - return NJS_TOKEN_ERROR; - } - } - } - - return lt->type; -} - - -njs_int_t -njs_lexer_rollback(njs_vm_t *vm, njs_lexer_t *lexer) -{ - njs_lexer_token_t *lt; - - lt = njs_mp_zalloc(vm->mem_pool, sizeof(njs_lexer_token_t)); - if (njs_slow_path(lt == NULL)) { - return NJS_ERROR; - } - - *lt = *lexer->token; - - njs_queue_insert_head(&lexer->preread, <->link); - - return NJS_OK; -} - - -static njs_lexer_token_t * -njs_lexer_token_push(njs_vm_t *vm, njs_lexer_t *lexer) +njs_inline njs_lexer_token_t * +njs_lexer_next_token(njs_lexer_t *lexer) { njs_int_t ret; njs_lexer_token_t *token; - token = njs_mp_zalloc(vm->mem_pool, sizeof(njs_lexer_token_t)); + token = njs_mp_zalloc(lexer->mem_pool, sizeof(njs_lexer_token_t)); if (njs_slow_path(token == NULL)) { return NULL; } do { - ret = njs_lexer_next_token(lexer, token); + ret = njs_lexer_make_token(lexer, token); if (njs_slow_path(ret != NJS_OK)) { return NULL; } @@ -419,20 +332,109 @@ njs_lexer_token_push(njs_vm_t *vm, njs_l } -static njs_lexer_token_t * -njs_lexer_token_pop(njs_lexer_t *lexer) +njs_lexer_token_t * +njs_lexer_token(njs_lexer_t *lexer, njs_bool_t with_end_line) { - njs_queue_link_t *lnk; + njs_queue_link_t *lnk; + njs_lexer_token_t *token; lnk = njs_queue_first(&lexer->preread); - njs_queue_remove(lnk); + + while (lnk != njs_queue_head(&lexer->preread)) { + token = njs_queue_link_data(lnk, njs_lexer_token_t, link); + + if (!with_end_line && token->type == NJS_TOKEN_LINE_END) { + lexer->prev_type = token->type; + + lnk = njs_queue_next(&token->link); + continue; + } + + return token; + } + + do { + token = njs_lexer_next_token(lexer); + if (token == NULL) { + return NULL; + } + + if (!with_end_line && token->type == NJS_TOKEN_LINE_END) { + lexer->prev_type = token->type; + continue; + } + + break; + + } while (1); + + return token; +} + + +njs_lexer_token_t * +njs_lexer_peek_token(njs_lexer_t *lexer, njs_lexer_token_t *current, + njs_bool_t with_end_line) +{ + njs_queue_link_t *lnk; + njs_lexer_token_t *token; + + lnk = njs_queue_next(¤t->link); + + while (lnk != njs_queue_head(&lexer->preread)) { + token = njs_queue_link_data(lnk, njs_lexer_token_t, link); - return njs_queue_link_data(lnk, njs_lexer_token_t, link); + if (!with_end_line && token->type == NJS_TOKEN_LINE_END) { + lnk = njs_queue_next(&token->link); + continue; + } + + return token; + } + + do { + token = njs_lexer_next_token(lexer); + if (token == NULL) { + return NULL; + } + + if (!with_end_line && token->type == NJS_TOKEN_LINE_END) { + continue; + } + + break; + + } while (1); + + return token; +} + + +void +njs_lexer_consume_token(njs_lexer_t *lexer, unsigned length) +{ + njs_queue_link_t *lnk; + njs_lexer_token_t *token; + + while (length > 0) { + lnk = njs_queue_first(&lexer->preread); + token = njs_queue_link_data(lnk, njs_lexer_token_t, link); + + lexer->prev_type = token->type; + + if (token->type != NJS_TOKEN_LINE_END) { + length--; + } + + njs_queue_remove(lnk); + + njs_mp_free(lexer->mem_pool, token); + } } njs_int_t -njs_lexer_next_token(njs_lexer_t *lexer, njs_lexer_token_t *token) +njs_lexer_make_token(njs_lexer_t *lexer, njs_lexer_token_t *token) { u_char c, *p; @@ -446,7 +448,6 @@ njs_lexer_next_token(njs_lexer_t *lexer, } } - lexer->keyword = 0; token->type = njs_tokens[c]; switch (token->type) { @@ -565,6 +566,7 @@ njs_lexer_next_token(njs_lexer_t *lexer, /* Fall through. */ default: + token->line = lexer->line; token->text.start = lexer->start - 1; token->text.length = lexer->start - token->text.start; @@ -694,12 +696,14 @@ njs_lexer_word(njs_lexer_t *lexer, njs_l } token->type = NJS_TOKEN_NAME; + token->keyword_type = NJS_KEYWORD_TYPE_UNDEF; } else { entry = &key_entry->value->entry; token->type = key_entry->value->type; - lexer->keyword = 1; + token->keyword_type = NJS_KEYWORD_TYPE_KEYWORD; + token->keyword_type |= key_entry->value->reserved; } token->unique_id = (uintptr_t) entry; diff -r 1d071c0e23e8 -r 86f55a7dc4a4 src/njs_lexer.h --- a/src/njs_lexer.h Fri May 29 14:49:36 2020 +0300 +++ b/src/njs_lexer.h Fri May 29 20:00:32 2020 +0300 @@ -99,6 +99,7 @@ typedef enum { NJS_TOKEN_COALESCE, NJS_TOKEN_IN, + NJS_TOKEN_OF, NJS_TOKEN_INSTANCEOF, NJS_TOKEN_TYPEOF, NJS_TOKEN_VOID, @@ -178,7 +179,14 @@ typedef enum { NJS_TOKEN_IMPORT, NJS_TOKEN_EXPORT, + NJS_TOKEN_TARGET, + + NJS_TOKEN_FROM, + + NJS_TOKEN_META, + NJS_TOKEN_AWAIT, + NJS_TOKEN_ASYNC, NJS_TOKEN_CLASS, NJS_TOKEN_CONST, NJS_TOKEN_DEBUGGER, @@ -198,6 +206,13 @@ typedef enum { } njs_token_type_t; +typedef enum { + NJS_KEYWORD_TYPE_UNDEF = 0, + NJS_KEYWORD_TYPE_RESERVED = 1, + NJS_KEYWORD_TYPE_KEYWORD = 2 +} njs_keyword_type_t; + + typedef struct { njs_str_t name; } njs_lexer_entry_t; @@ -206,6 +221,7 @@ typedef struct { typedef struct { njs_lexer_entry_t entry; njs_token_type_t type; + njs_bool_t reserved; } njs_keyword_t; @@ -220,6 +236,7 @@ typedef struct { typedef struct { njs_token_type_t type:16; + njs_keyword_type_t keyword_type; uint32_t line; uintptr_t unique_id; njs_str_t text; @@ -231,10 +248,10 @@ typedef struct { typedef struct { njs_lexer_token_t *token; njs_queue_t preread; /* of njs_lexer_token_t */ - uint8_t keyword; u_char *prev_start; njs_token_type_t prev_type:16; + njs_token_type_t last_type:16; uint32_t line; njs_str_t file; @@ -251,12 +268,12 @@ typedef struct { njs_int_t njs_lexer_init(njs_vm_t *vm, njs_lexer_t *lexer, njs_str_t *file, u_char *start, u_char *end); -njs_token_type_t njs_lexer_token(njs_vm_t *vm, njs_lexer_t *lexer); -njs_token_type_t njs_lexer_peek_token(njs_vm_t *vm, njs_lexer_t *lexer, - size_t offset); -njs_int_t njs_lexer_rollback(njs_vm_t *vm, njs_lexer_t *lexer); - -njs_int_t njs_lexer_next_token(njs_lexer_t *lexer, njs_lexer_token_t *token); +njs_lexer_token_t *njs_lexer_token(njs_lexer_t *lexer, + njs_bool_t with_end_line); +njs_lexer_token_t *njs_lexer_peek_token(njs_lexer_t *lexer, + njs_lexer_token_t *current, njs_bool_t with_end_line); +void njs_lexer_consume_token(njs_lexer_t *lexer, unsigned length); +njs_int_t njs_lexer_make_token(njs_lexer_t *lexer, njs_lexer_token_t *token); const njs_lexer_keyword_entry_t *njs_lexer_keyword(const u_char *key, size_t length); @@ -270,6 +287,66 @@ njs_lexer_entry(uintptr_t unique_id) } +njs_inline njs_bool_t +njs_lexer_token_is_keyword(njs_lexer_token_t *token) +{ + return token->keyword_type & NJS_KEYWORD_TYPE_KEYWORD; +} + + +njs_inline njs_bool_t +njs_lexer_token_is_reserved(njs_lexer_token_t *token) +{ + return token->keyword_type & NJS_KEYWORD_TYPE_RESERVED; +} + + +njs_inline njs_bool_t +njs_lexer_token_is_name(njs_lexer_token_t *token) +{ + return token->type == NJS_TOKEN_NAME + || (!njs_lexer_token_is_reserved(token) + && njs_lexer_token_is_keyword(token)); +} + + +njs_inline njs_bool_t +njs_lexer_token_is_identifier_name(njs_lexer_token_t *token) +{ + return token->type == NJS_TOKEN_NAME || njs_lexer_token_is_keyword(token); +} + + +njs_inline njs_bool_t +njs_lexer_token_is_binding_identifier(njs_lexer_token_t *token) +{ + switch (token->type) { + case NJS_TOKEN_NAME: + case NJS_TOKEN_YIELD: + case NJS_TOKEN_AWAIT: + return 1; + + default: + return (!njs_lexer_token_is_reserved(token) + && njs_lexer_token_is_keyword(token)); + }; +} + + +njs_inline njs_bool_t +njs_lexer_token_is_label_identifier(njs_lexer_token_t *token) +{ + return njs_lexer_token_is_binding_identifier(token); +} + + +njs_inline njs_bool_t +njs_lexer_token_is_identifier_reference(njs_lexer_token_t *token) +{ + return njs_lexer_token_is_binding_identifier(token); +} + + extern const njs_lvlhsh_proto_t njs_lexer_hash_proto; diff -r 1d071c0e23e8 -r 86f55a7dc4a4 src/njs_lexer_tables.h --- a/src/njs_lexer_tables.h Fri May 29 14:49:36 2020 +0300 +++ b/src/njs_lexer_tables.h Fri May 29 20:00:32 2020 +0300 @@ -10,135 +10,428 @@ #define _NJS_LEXER_TABLES_H_INCLUDED_ -static const njs_keyword_t njs_lexer_kws[48] = +static const njs_keyword_t njs_lexer_kws[53] = { - { .entry = { njs_str("null") }, .type = NJS_TOKEN_NULL }, - { .entry = { njs_str("false") }, .type = NJS_TOKEN_FALSE }, - { .entry = { njs_str("true") }, .type = NJS_TOKEN_TRUE }, - { .entry = { njs_str("in") }, .type = NJS_TOKEN_IN }, - { .entry = { njs_str("typeof") }, .type = NJS_TOKEN_TYPEOF }, - { .entry = { njs_str("instanceof") }, .type = NJS_TOKEN_INSTANCEOF }, - { .entry = { njs_str("void") }, .type = NJS_TOKEN_VOID }, - { .entry = { njs_str("new") }, .type = NJS_TOKEN_NEW }, - { .entry = { njs_str("delete") }, .type = NJS_TOKEN_DELETE }, - { .entry = { njs_str("yield") }, .type = NJS_TOKEN_YIELD }, - { .entry = { njs_str("var") }, .type = NJS_TOKEN_VAR }, - { .entry = { njs_str("if") }, .type = NJS_TOKEN_IF }, - { .entry = { njs_str("else") }, .type = NJS_TOKEN_ELSE }, - { .entry = { njs_str("while") }, .type = NJS_TOKEN_WHILE }, - { .entry = { njs_str("do") }, .type = NJS_TOKEN_DO }, - { .entry = { njs_str("for") }, .type = NJS_TOKEN_FOR }, - { .entry = { njs_str("break") }, .type = NJS_TOKEN_BREAK }, - { .entry = { njs_str("continue") }, .type = NJS_TOKEN_CONTINUE }, - { .entry = { njs_str("switch") }, .type = NJS_TOKEN_SWITCH }, - { .entry = { njs_str("case") }, .type = NJS_TOKEN_CASE }, - { .entry = { njs_str("default") }, .type = NJS_TOKEN_DEFAULT }, - { .entry = { njs_str("function") }, .type = NJS_TOKEN_FUNCTION }, - { .entry = { njs_str("return") }, .type = NJS_TOKEN_RETURN }, - { .entry = { njs_str("with") }, .type = NJS_TOKEN_WITH }, - { .entry = { njs_str("try") }, .type = NJS_TOKEN_TRY }, - { .entry = { njs_str("catch") }, .type = NJS_TOKEN_CATCH }, - { .entry = { njs_str("finally") }, .type = NJS_TOKEN_FINALLY }, - { .entry = { njs_str("throw") }, .type = NJS_TOKEN_THROW }, - { .entry = { njs_str("import") }, .type = NJS_TOKEN_IMPORT }, - { .entry = { njs_str("export") }, .type = NJS_TOKEN_EXPORT }, - { .entry = { njs_str("this") }, .type = NJS_TOKEN_THIS }, - { .entry = { njs_str("arguments") }, .type = NJS_TOKEN_ARGUMENTS }, - { .entry = { njs_str("eval") }, .type = NJS_TOKEN_EVAL }, - { .entry = { njs_str("await") }, .type = NJS_TOKEN_AWAIT }, - { .entry = { njs_str("class") }, .type = NJS_TOKEN_CLASS }, - { .entry = { njs_str("const") }, .type = NJS_TOKEN_CONST }, - { .entry = { njs_str("debugger") }, .type = NJS_TOKEN_DEBUGGER }, - { .entry = { njs_str("enum") }, .type = NJS_TOKEN_ENUM }, - { .entry = { njs_str("extends") }, .type = NJS_TOKEN_EXTENDS }, - { .entry = { njs_str("implements") }, .type = NJS_TOKEN_IMPLEMENTS }, - { .entry = { njs_str("interface") }, .type = NJS_TOKEN_INTERFACE }, - { .entry = { njs_str("let") }, .type = NJS_TOKEN_LET }, - { .entry = { njs_str("package") }, .type = NJS_TOKEN_PACKAGE }, - { .entry = { njs_str("private") }, .type = NJS_TOKEN_PRIVATE }, - { .entry = { njs_str("protected") }, .type = NJS_TOKEN_PROTECTED }, - { .entry = { njs_str("public") }, .type = NJS_TOKEN_PUBLIC }, - { .entry = { njs_str("static") }, .type = NJS_TOKEN_STATIC }, - { .entry = { njs_str("super") }, .type = NJS_TOKEN_SUPER }, + { + .entry = { njs_str("arguments") }, + .type = NJS_TOKEN_ARGUMENTS, + .reserved = 0 + }, + + { + .entry = { njs_str("async") }, + .type = NJS_TOKEN_ASYNC, + .reserved = 0 + }, + + { + .entry = { njs_str("await") }, + .type = NJS_TOKEN_AWAIT, + .reserved = 1 + }, + + { + .entry = { njs_str("break") }, + .type = NJS_TOKEN_BREAK, + .reserved = 1 + }, + + { + .entry = { njs_str("case") }, + .type = NJS_TOKEN_CASE, + .reserved = 1 + }, + + { + .entry = { njs_str("catch") }, + .type = NJS_TOKEN_CATCH, + .reserved = 1 + }, + + { + .entry = { njs_str("class") }, + .type = NJS_TOKEN_CLASS, + .reserved = 1 + }, + + { + .entry = { njs_str("const") }, + .type = NJS_TOKEN_CONST, + .reserved = 1 + }, + + { + .entry = { njs_str("continue") }, + .type = NJS_TOKEN_CONTINUE, + .reserved = 1 + }, + + { + .entry = { njs_str("debugger") }, + .type = NJS_TOKEN_DEBUGGER, + .reserved = 1 + }, + + { + .entry = { njs_str("default") }, + .type = NJS_TOKEN_DEFAULT, + .reserved = 1 + }, + + { + .entry = { njs_str("delete") }, + .type = NJS_TOKEN_DELETE, + .reserved = 1 + }, + + { + .entry = { njs_str("do") }, + .type = NJS_TOKEN_DO, + .reserved = 1 + }, + + { + .entry = { njs_str("else") }, + .type = NJS_TOKEN_ELSE, + .reserved = 1 + }, + + { + .entry = { njs_str("enum") }, + .type = NJS_TOKEN_ENUM, + .reserved = 1 + }, + + { + .entry = { njs_str("eval") }, + .type = NJS_TOKEN_EVAL, + .reserved = 0 + }, + + { + .entry = { njs_str("export") }, + .type = NJS_TOKEN_EXPORT, + .reserved = 1 + }, + + { + .entry = { njs_str("extends") }, + .type = NJS_TOKEN_EXTENDS, + .reserved = 1 + }, + + { + .entry = { njs_str("false") }, + .type = NJS_TOKEN_FALSE, + .reserved = 1 + }, + + { + .entry = { njs_str("finally") }, + .type = NJS_TOKEN_FINALLY, + .reserved = 1 + }, + + { + .entry = { njs_str("for") }, + .type = NJS_TOKEN_FOR, + .reserved = 1 + }, + + { + .entry = { njs_str("from") }, + .type = NJS_TOKEN_FROM, + .reserved = 0 + }, + + { + .entry = { njs_str("function") }, + .type = NJS_TOKEN_FUNCTION, + .reserved = 1 + }, + + { + .entry = { njs_str("if") }, + .type = NJS_TOKEN_IF, + .reserved = 1 + }, + + { + .entry = { njs_str("implements") }, + .type = NJS_TOKEN_IMPLEMENTS, + .reserved = 0 + }, + + { + .entry = { njs_str("import") }, + .type = NJS_TOKEN_IMPORT, + .reserved = 1 + }, + + { + .entry = { njs_str("in") }, + .type = NJS_TOKEN_IN, + .reserved = 1 + }, + + { + .entry = { njs_str("instanceof") }, + .type = NJS_TOKEN_INSTANCEOF, + .reserved = 1 + }, + + { + .entry = { njs_str("interface") }, + .type = NJS_TOKEN_INTERFACE, + .reserved = 0 + }, + + { + .entry = { njs_str("let") }, + .type = NJS_TOKEN_LET, + .reserved = 0 + }, + + { + .entry = { njs_str("meta") }, + .type = NJS_TOKEN_META, + .reserved = 0 + }, + + { + .entry = { njs_str("new") }, + .type = NJS_TOKEN_NEW, + .reserved = 1 + }, + + { + .entry = { njs_str("null") }, + .type = NJS_TOKEN_NULL, + .reserved = 1 + }, + + { + .entry = { njs_str("of") }, + .type = NJS_TOKEN_OF, + .reserved = 0 + }, + + { + .entry = { njs_str("package") }, + .type = NJS_TOKEN_PACKAGE, + .reserved = 0 + }, + + { + .entry = { njs_str("private") }, + .type = NJS_TOKEN_PRIVATE, + .reserved = 0 + }, + + { + .entry = { njs_str("protected") }, + .type = NJS_TOKEN_PROTECTED, + .reserved = 0 + }, + + { + .entry = { njs_str("public") }, + .type = NJS_TOKEN_PUBLIC, + .reserved = 0 + }, + + { + .entry = { njs_str("return") }, + .type = NJS_TOKEN_RETURN, + .reserved = 1 + }, + + { + .entry = { njs_str("static") }, + .type = NJS_TOKEN_STATIC, + .reserved = 0 + }, + + { + .entry = { njs_str("super") }, + .type = NJS_TOKEN_SUPER, + .reserved = 1 + }, + + { + .entry = { njs_str("switch") }, + .type = NJS_TOKEN_SWITCH, + .reserved = 1 + }, + + { + .entry = { njs_str("target") }, + .type = NJS_TOKEN_TARGET, + .reserved = 0 + }, + + { + .entry = { njs_str("this") }, + .type = NJS_TOKEN_THIS, + .reserved = 1 + }, + + { + .entry = { njs_str("throw") }, + .type = NJS_TOKEN_THROW, + .reserved = 1 + }, + + { + .entry = { njs_str("true") }, + .type = NJS_TOKEN_TRUE, + .reserved = 1 + }, + + { + .entry = { njs_str("try") }, + .type = NJS_TOKEN_TRY, + .reserved = 1 + }, + + { + .entry = { njs_str("typeof") }, + .type = NJS_TOKEN_TYPEOF, + .reserved = 1 + }, + + { + .entry = { njs_str("var") }, + .type = NJS_TOKEN_VAR, + .reserved = 1 + }, + + { + .entry = { njs_str("void") }, + .type = NJS_TOKEN_VOID, + .reserved = 1 + }, + + { + .entry = { njs_str("while") }, + .type = NJS_TOKEN_WHILE, + .reserved = 1 + }, + + { + .entry = { njs_str("with") }, + .type = NJS_TOKEN_WITH, + .reserved = 1 + }, + + { + .entry = { njs_str("yield") }, + .type = NJS_TOKEN_YIELD, + .reserved = 1 + }, }; -static const njs_lexer_keyword_entry_t njs_lexer_keyword_entries[75] = +static const njs_lexer_keyword_entry_t njs_lexer_keyword_entries[99] = { - { NULL, NULL, 74, 0 }, - { "case", &njs_lexer_kws[19], 4, 0 }, - { "continue", &njs_lexer_kws[17], 8, 0 }, - { "do", &njs_lexer_kws[14], 2, 0 }, - { "enum", &njs_lexer_kws[37], 4, 0 }, - { "extends", &njs_lexer_kws[38], 7, 0 }, - { "instanceof", &njs_lexer_kws[5], 10, 0 }, - { "public", &njs_lexer_kws[45], 6, 0 }, - { "static", &njs_lexer_kws[46], 6, 0 }, - { "in", &njs_lexer_kws[3], 2, 0 }, - { "await", &njs_lexer_kws[33], 5, 0 }, - { "private", &njs_lexer_kws[43], 7, 0 }, + { NULL, NULL, 98, 0 }, + { "continue", &njs_lexer_kws[8], 8, 0 }, + { "finally", &njs_lexer_kws[19], 7, 0 }, + { "return", &njs_lexer_kws[38], 6, 0 }, + { "static", &njs_lexer_kws[39], 6, 0 }, + { "async", &njs_lexer_kws[1], 5, 0 }, + { "break", &njs_lexer_kws[3], 5, 0 }, + { "interface", &njs_lexer_kws[28], 9, 0 }, + { "case", &njs_lexer_kws[4], 4, 0 }, + { "import", &njs_lexer_kws[25], 6, 0 }, + { "protected", &njs_lexer_kws[36], 9, 0 }, + { "switch", &njs_lexer_kws[41], 6, 0 }, + { "catch", &njs_lexer_kws[5], 5, 1 }, + { "delete", &njs_lexer_kws[11], 6, 0 }, + { "else", &njs_lexer_kws[13], 4, 0 }, + { "private", &njs_lexer_kws[35], 7, 0 }, + { "extends", &njs_lexer_kws[17], 7, 0 }, + { "this", &njs_lexer_kws[43], 4, 0 }, + { "false", &njs_lexer_kws[18], 5, 0 }, + { "await", &njs_lexer_kws[2], 5, 0 }, { NULL, NULL, 0, 0 }, - { "debugger", &njs_lexer_kws[36], 8, 0 }, - { "for", &njs_lexer_kws[15], 3, 1 }, + { "public", &njs_lexer_kws[37], 6, 0 }, { NULL, NULL, 0, 0 }, - { "catch", &njs_lexer_kws[25], 5, 0 }, + { "class", &njs_lexer_kws[6], 5, 0 }, + { "const", &njs_lexer_kws[7], 5, 4 }, { NULL, NULL, 0, 0 }, - { "super", &njs_lexer_kws[47], 5, 2 }, + { "try", &njs_lexer_kws[46], 3, 0 }, + { "null", &njs_lexer_kws[32], 4, 0 }, { NULL, NULL, 0, 0 }, - { "const", &njs_lexer_kws[35], 5, 0 }, + { "do", &njs_lexer_kws[12], 2, 0 }, + { "var", &njs_lexer_kws[48], 3, 0 }, + { "if", &njs_lexer_kws[23], 2, 7 }, + { "implements", &njs_lexer_kws[24], 10, 0 }, + { "with", &njs_lexer_kws[51], 4, 0 }, { NULL, NULL, 0, 0 }, - { "false", &njs_lexer_kws[1], 5, 0 }, - { "with", &njs_lexer_kws[23], 4, 0 }, - { "implements", &njs_lexer_kws[39], 10, 0 }, - { "this", &njs_lexer_kws[30], 4, 0 }, - { "let", &njs_lexer_kws[41], 3, 0 }, + { "eval", &njs_lexer_kws[15], 4, 9 }, + { NULL, NULL, 0, 0 }, + { "target", &njs_lexer_kws[42], 6, 0 }, + { "enum", &njs_lexer_kws[14], 4, 10 }, + { "instanceof", &njs_lexer_kws[27], 10, 0 }, + { NULL, NULL, 0, 0 }, + { "debugger", &njs_lexer_kws[9], 8, 0 }, { NULL, NULL, 0, 0 }, { NULL, NULL, 0, 0 }, - { "true", &njs_lexer_kws[2], 4, 0 }, + { "default", &njs_lexer_kws[10], 7, 0 }, + { "void", &njs_lexer_kws[49], 4, 0 }, + { NULL, NULL, 0, 0 }, { NULL, NULL, 0, 0 }, - { "export", &njs_lexer_kws[29], 6, 0 }, { NULL, NULL, 0, 0 }, - { "interface", &njs_lexer_kws[40], 9, 0 }, + { "from", &njs_lexer_kws[21], 4, 0 }, + { "package", &njs_lexer_kws[34], 7, 15 }, { NULL, NULL, 0, 0 }, - { "eval", &njs_lexer_kws[32], 4, 0 }, - { "protected", &njs_lexer_kws[44], 9, 0 }, - { "while", &njs_lexer_kws[13], 5, 0 }, + { "yield", &njs_lexer_kws[52], 5, 0 }, { NULL, NULL, 0, 0 }, - { "void", &njs_lexer_kws[6], 4, 0 }, + { NULL, NULL, 0, 0 }, + { "of", &njs_lexer_kws[33], 2, 0 }, { NULL, NULL, 0, 0 }, - { "return", &njs_lexer_kws[22], 6, 0 }, + { "function", &njs_lexer_kws[22], 8, 0 }, { NULL, NULL, 0, 0 }, - { "delete", &njs_lexer_kws[8], 6, 0 }, - { "yield", &njs_lexer_kws[9], 5, 0 }, - { "null", &njs_lexer_kws[0], 4, 0 }, - { "throw", &njs_lexer_kws[27], 5, 0 }, + { "true", &njs_lexer_kws[45], 4, 16 }, + { "new", &njs_lexer_kws[31], 3, 0 }, + { "export", &njs_lexer_kws[16], 6, 0 }, { NULL, NULL, 0, 0 }, { NULL, NULL, 0, 0 }, { NULL, NULL, 0, 0 }, { NULL, NULL, 0, 0 }, - { "import", &njs_lexer_kws[28], 6, 0 }, + { NULL, NULL, 0, 0 }, + { NULL, NULL, 0, 0 }, + { "for", &njs_lexer_kws[20], 3, 0 }, + { "while", &njs_lexer_kws[50], 5, 0 }, { NULL, NULL, 0, 0 }, - { "switch", &njs_lexer_kws[18], 6, 0 }, - { "try", &njs_lexer_kws[24], 3, 0 }, - { "function", &njs_lexer_kws[21], 8, 0 }, + { NULL, NULL, 0, 0 }, + { NULL, NULL, 0, 0 }, { NULL, NULL, 0, 0 }, - { "if", &njs_lexer_kws[11], 2, 0 }, - { "break", &njs_lexer_kws[16], 5, 0 }, + { NULL, NULL, 0, 0 }, + { NULL, NULL, 0, 0 }, { NULL, NULL, 0, 0 }, { NULL, NULL, 0, 0 }, { NULL, NULL, 0, 0 }, - { "var", &njs_lexer_kws[10], 3, 4 }, + { "typeof", &njs_lexer_kws[47], 6, 0 }, + { NULL, NULL, 0, 0 }, + { NULL, NULL, 0, 0 }, + { "super", &njs_lexer_kws[40], 5, 0 }, { NULL, NULL, 0, 0 }, - { "default", &njs_lexer_kws[20], 7, 0 }, - { "arguments", &njs_lexer_kws[31], 9, 6 }, - { "finally", &njs_lexer_kws[26], 7, 0 }, + { NULL, NULL, 0, 0 }, + { NULL, NULL, 0, 0 }, + { "let", &njs_lexer_kws[29], 3, 19 }, + { "in", &njs_lexer_kws[26], 2, 0 }, { NULL, NULL, 0, 0 }, - { "else", &njs_lexer_kws[12], 4, 0 }, - { "class", &njs_lexer_kws[34], 5, 7 }, - { "new", &njs_lexer_kws[7], 3, 8 }, + { NULL, NULL, 0, 0 }, + { "throw", &njs_lexer_kws[44], 5, 0 }, + { "arguments", &njs_lexer_kws[0], 9, 0 }, + { "meta", &njs_lexer_kws[30], 4, 0 }, { NULL, NULL, 0, 0 }, - { "package", &njs_lexer_kws[42], 7, 11 }, - { "typeof", &njs_lexer_kws[4], 6, 0 }, + { NULL, NULL, 0, 0 }, + { NULL, NULL, 0, 0 }, + { NULL, NULL, 0, 0 }, + { NULL, NULL, 0, 0 }, { NULL, NULL, 0, 0 }, }; diff -r 1d071c0e23e8 -r 86f55a7dc4a4 src/njs_module.c --- a/src/njs_module.c Fri May 29 14:49:36 2020 +0300 +++ b/src/njs_module.c Fri May 29 20:00:32 2020 +0300 @@ -15,6 +15,19 @@ typedef struct { } njs_module_info_t; +typedef struct { From xeioex at nginx.com Fri May 29 19:30:59 2020 From: xeioex at nginx.com (Dmitry Volyntsev) Date: Fri, 29 May 2020 19:30:59 +0000 Subject: [njs] Added AST serialization. Message-ID: details: https://hg.nginx.org/njs/rev/d255e73aed3b branches: changeset: 1408:d255e73aed3b user: Dmitry Volyntsev date: Fri May 29 19:29:59 2020 +0000 description: Added AST serialization. diffstat: src/njs.h | 2 + src/njs_parser.c | 256 +++++++++++++++++++++++++++++++++++++++++++++++ src/njs_parser.h | 2 + src/njs_shell.c | 7 + src/njs_vm.c | 21 +++- test/njs_expect_test.exp | 4 + 6 files changed, 291 insertions(+), 1 deletions(-) diffs (391 lines): diff -r 86f55a7dc4a4 -r d255e73aed3b src/njs.h --- a/src/njs.h Fri May 29 20:00:32 2020 +0300 +++ b/src/njs.h Fri May 29 19:29:59 2020 +0000 @@ -206,6 +206,7 @@ typedef struct { * unsafe - enables unsafe language features: * - Function constructors. * module - ES6 "module" mode. Script mode is default. + * ast - print AST. */ uint8_t trailer; /* 1 bit */ @@ -217,6 +218,7 @@ typedef struct { uint8_t sandbox; /* 1 bit */ uint8_t unsafe; /* 1 bit */ uint8_t module; /* 1 bit */ + uint8_t ast; /* 1 bit */ } njs_vm_opt_t; diff -r 86f55a7dc4a4 -r d255e73aed3b src/njs_parser.c --- a/src/njs_parser.c Fri May 29 20:00:32 2020 +0300 +++ b/src/njs_parser.c Fri May 29 19:29:59 2020 +0000 @@ -449,6 +449,11 @@ static njs_token_type_t njs_parser_escap static njs_int_t njs_parser_escape_string_calc_length(njs_parser_t *parser, njs_lexer_token_t *token, size_t *out_size, size_t *out_length); +static void njs_parser_serialize_tree(njs_chb_t *chain, + njs_parser_node_t *node, njs_int_t *ret, size_t indent); +static njs_int_t njs_parser_serialize_node(njs_chb_t *chain, + njs_parser_node_t *node); + #define njs_parser_chain_top(parser) \ ((parser)->scope->top) @@ -8269,3 +8274,254 @@ njs_parser_node_error(njs_vm_t *vm, njs_ njs_parser_scope_error(vm, node->scope, type, node->token_line, fmt, args); va_end(args); } + + +njs_int_t +njs_parser_serialize_ast(njs_parser_node_t *node, njs_chb_t *chain) +{ + njs_int_t ret; + + ret = NJS_OK; + + njs_parser_serialize_tree(chain, node, &ret, 0); + njs_chb_append_literal(chain, "\n"); + + return ret; +} + + +njs_inline void +njs_parser_serialize_indent(njs_chb_t *chain, size_t indent) +{ + size_t i; + + for (i = 0; i < indent; i++) { + njs_chb_append_literal(chain, " "); + } +} + + +static void +njs_parser_serialize_tree(njs_chb_t *chain, njs_parser_node_t *node, + njs_int_t *ret, size_t indent) +{ + njs_str_t str; + + njs_chb_append_literal(chain, "{\"name\": \""); + + *ret |= njs_parser_serialize_node(chain, node); + + njs_chb_append_literal(chain, "\""); + + switch (node->token_type) { + case NJS_TOKEN_NUMBER: + case NJS_TOKEN_STRING: + case NJS_TOKEN_NAME: + case NJS_TOKEN_FUNCTION_CALL: + njs_chb_append_literal(chain, ",\n"); + njs_parser_serialize_indent(chain, indent); + njs_chb_sprintf(chain, 32, " \"index\": \"%p\"", node->index); + + switch (node->token_type) { + case NJS_TOKEN_NUMBER: + case NJS_TOKEN_STRING: + njs_chb_append_literal(chain, ",\n"); + njs_parser_serialize_indent(chain, indent); + + if (node->token_type == NJS_TOKEN_NUMBER) { + njs_chb_sprintf(chain, 32, " \"value\": %f\n", + njs_number(&node->u.value)); + + } else { + njs_string_get(&node->u.value, &str); + njs_chb_sprintf(chain, 32, " \"value\": \"%V\"\n", &str); + } + + break; + + default: + break; + } + + break; + + default: + break; + } + + if (node->left != NULL) { + njs_chb_append_literal(chain, ",\n"); + njs_parser_serialize_indent(chain, indent); + njs_chb_append_literal(chain, " \"left\": "); + + njs_parser_serialize_tree(chain, node->left, ret, indent + 1); + } + + if (node->right != NULL) { + njs_chb_append_literal(chain, ",\n"); + njs_parser_serialize_indent(chain, indent); + njs_chb_append_literal(chain, " \"right\": "); + + njs_parser_serialize_tree(chain, node->right, ret, indent + 1); + } + + njs_chb_append_literal(chain, "}"); +} + + +static njs_int_t +njs_parser_serialize_node(njs_chb_t *chain, njs_parser_node_t *node) +{ + const char *name; + +#define njs_token_serialize(token) \ + case token: \ + name = &njs_stringify(token)[njs_length("NJS_TOKEN_")]; \ + njs_chb_append(chain, name, njs_strlen(name)); \ + break + + switch (node->token_type) { + njs_token_serialize(NJS_TOKEN_END); + /* FIXME: NJS_TOKEN_ILLEGAL should not be present in AST */ + njs_token_serialize(NJS_TOKEN_ILLEGAL); + njs_token_serialize(NJS_TOKEN_COMMA); + njs_token_serialize(NJS_TOKEN_CONDITIONAL); + njs_token_serialize(NJS_TOKEN_ASSIGNMENT); + njs_token_serialize(NJS_TOKEN_ADDITION_ASSIGNMENT); + njs_token_serialize(NJS_TOKEN_SUBSTRACTION_ASSIGNMENT); + njs_token_serialize(NJS_TOKEN_MULTIPLICATION_ASSIGNMENT); + njs_token_serialize(NJS_TOKEN_EXPONENTIATION_ASSIGNMENT); + njs_token_serialize(NJS_TOKEN_DIVISION_ASSIGNMENT); + njs_token_serialize(NJS_TOKEN_REMAINDER_ASSIGNMENT); + njs_token_serialize(NJS_TOKEN_LEFT_SHIFT_ASSIGNMENT); + njs_token_serialize(NJS_TOKEN_RIGHT_SHIFT_ASSIGNMENT); + njs_token_serialize(NJS_TOKEN_UNSIGNED_RIGHT_SHIFT_ASSIGNMENT); + njs_token_serialize(NJS_TOKEN_BITWISE_OR_ASSIGNMENT); + njs_token_serialize(NJS_TOKEN_BITWISE_XOR_ASSIGNMENT); + njs_token_serialize(NJS_TOKEN_BITWISE_AND_ASSIGNMENT); + njs_token_serialize(NJS_TOKEN_EQUAL); + njs_token_serialize(NJS_TOKEN_NOT_EQUAL); + njs_token_serialize(NJS_TOKEN_STRICT_EQUAL); + njs_token_serialize(NJS_TOKEN_STRICT_NOT_EQUAL); + njs_token_serialize(NJS_TOKEN_ADDITION); + njs_token_serialize(NJS_TOKEN_UNARY_PLUS); + njs_token_serialize(NJS_TOKEN_INCREMENT); + njs_token_serialize(NJS_TOKEN_POST_INCREMENT); + njs_token_serialize(NJS_TOKEN_SUBSTRACTION); + njs_token_serialize(NJS_TOKEN_UNARY_NEGATION); + njs_token_serialize(NJS_TOKEN_DECREMENT); + njs_token_serialize(NJS_TOKEN_POST_DECREMENT); + njs_token_serialize(NJS_TOKEN_MULTIPLICATION); + njs_token_serialize(NJS_TOKEN_EXPONENTIATION); + njs_token_serialize(NJS_TOKEN_DIVISION); + njs_token_serialize(NJS_TOKEN_REMAINDER); + njs_token_serialize(NJS_TOKEN_LESS); + njs_token_serialize(NJS_TOKEN_LESS_OR_EQUAL); + njs_token_serialize(NJS_TOKEN_LEFT_SHIFT); + njs_token_serialize(NJS_TOKEN_GREATER); + njs_token_serialize(NJS_TOKEN_GREATER_OR_EQUAL); + njs_token_serialize(NJS_TOKEN_RIGHT_SHIFT); + njs_token_serialize(NJS_TOKEN_UNSIGNED_RIGHT_SHIFT); + njs_token_serialize(NJS_TOKEN_BITWISE_OR); + njs_token_serialize(NJS_TOKEN_LOGICAL_OR); + njs_token_serialize(NJS_TOKEN_BITWISE_XOR); + njs_token_serialize(NJS_TOKEN_BITWISE_AND); + njs_token_serialize(NJS_TOKEN_LOGICAL_AND); + njs_token_serialize(NJS_TOKEN_BITWISE_NOT); + njs_token_serialize(NJS_TOKEN_LOGICAL_NOT); + njs_token_serialize(NJS_TOKEN_COALESCE); + njs_token_serialize(NJS_TOKEN_IN); + njs_token_serialize(NJS_TOKEN_OF); + njs_token_serialize(NJS_TOKEN_INSTANCEOF); + njs_token_serialize(NJS_TOKEN_TYPEOF); + njs_token_serialize(NJS_TOKEN_VOID); + njs_token_serialize(NJS_TOKEN_NEW); + njs_token_serialize(NJS_TOKEN_DELETE); + njs_token_serialize(NJS_TOKEN_YIELD); + + njs_token_serialize(NJS_TOKEN_NULL); + njs_token_serialize(NJS_TOKEN_NUMBER); + njs_token_serialize(NJS_TOKEN_TRUE); + njs_token_serialize(NJS_TOKEN_FALSE); + njs_token_serialize(NJS_TOKEN_STRING); + njs_token_serialize(NJS_TOKEN_TEMPLATE_LITERAL); + njs_token_serialize(NJS_TOKEN_NAME); + njs_token_serialize(NJS_TOKEN_OBJECT); + njs_token_serialize(NJS_TOKEN_OBJECT_VALUE); + njs_token_serialize(NJS_TOKEN_ARRAY); + njs_token_serialize(NJS_TOKEN_REGEXP); + + njs_token_serialize(NJS_TOKEN_PROPERTY); + njs_token_serialize(NJS_TOKEN_PROPERTY_INIT); + njs_token_serialize(NJS_TOKEN_PROPERTY_DELETE); + njs_token_serialize(NJS_TOKEN_PROPERTY_GETTER); + njs_token_serialize(NJS_TOKEN_PROPERTY_SETTER); + + njs_token_serialize(NJS_TOKEN_PROTO_INIT); + + njs_token_serialize(NJS_TOKEN_FUNCTION); + njs_token_serialize(NJS_TOKEN_FUNCTION_EXPRESSION); + njs_token_serialize(NJS_TOKEN_FUNCTION_CALL); + njs_token_serialize(NJS_TOKEN_METHOD_CALL); + + njs_token_serialize(NJS_TOKEN_ARGUMENT); + njs_token_serialize(NJS_TOKEN_RETURN); + njs_token_serialize(NJS_TOKEN_STATEMENT); + njs_token_serialize(NJS_TOKEN_BLOCK); + njs_token_serialize(NJS_TOKEN_VAR); + njs_token_serialize(NJS_TOKEN_LET); + njs_token_serialize(NJS_TOKEN_IF); + njs_token_serialize(NJS_TOKEN_ELSE); + njs_token_serialize(NJS_TOKEN_BRANCHING); + njs_token_serialize(NJS_TOKEN_WHILE); + njs_token_serialize(NJS_TOKEN_DO); + njs_token_serialize(NJS_TOKEN_FOR); + njs_token_serialize(NJS_TOKEN_FOR_IN); + njs_token_serialize(NJS_TOKEN_BREAK); + njs_token_serialize(NJS_TOKEN_CONTINUE); + njs_token_serialize(NJS_TOKEN_SWITCH); + njs_token_serialize(NJS_TOKEN_CASE); + njs_token_serialize(NJS_TOKEN_DEFAULT); + njs_token_serialize(NJS_TOKEN_WITH); + njs_token_serialize(NJS_TOKEN_TRY); + njs_token_serialize(NJS_TOKEN_CATCH); + njs_token_serialize(NJS_TOKEN_FINALLY); + njs_token_serialize(NJS_TOKEN_THROW); + njs_token_serialize(NJS_TOKEN_THIS); + njs_token_serialize(NJS_TOKEN_GLOBAL_OBJECT); + njs_token_serialize(NJS_TOKEN_NON_LOCAL_THIS); + njs_token_serialize(NJS_TOKEN_ARGUMENTS); + njs_token_serialize(NJS_TOKEN_EVAL); + njs_token_serialize(NJS_TOKEN_IMPORT); + njs_token_serialize(NJS_TOKEN_EXPORT); + +#if 0 + + njs_token_serialize(NJS_TOKEN_TARGET); + njs_token_serialize(NJS_TOKEN_META); + njs_token_serialize(NJS_TOKEN_ASYNC); + njs_token_serialize(NJS_TOKEN_AWAIT); + njs_token_serialize(NJS_TOKEN_CONST); + njs_token_serialize(NJS_TOKEN_DEBUGGER); + njs_token_serialize(NJS_TOKEN_ENUM); + + njs_token_serialize(NJS_TOKEN_CLASS); + njs_token_serialize(NJS_TOKEN_EXTENDS); + njs_token_serialize(NJS_TOKEN_IMPLEMENTS); + njs_token_serialize(NJS_TOKEN_INTERFACE); + njs_token_serialize(NJS_TOKEN_PACKAGE); + njs_token_serialize(NJS_TOKEN_PRIVATE); + njs_token_serialize(NJS_TOKEN_PROTECTED); + njs_token_serialize(NJS_TOKEN_PUBLIC); + njs_token_serialize(NJS_TOKEN_STATIC); + njs_token_serialize(NJS_TOKEN_SUPER); + +#endif + + default: + njs_chb_sprintf(chain, 32, "#UNDEF(%d)", (int) node->token_type); + return NJS_DECLINED; + } + + return NJS_OK; +} diff -r 86f55a7dc4a4 -r d255e73aed3b src/njs_parser.h --- a/src/njs_parser.h Fri May 29 20:00:32 2020 +0300 +++ b/src/njs_parser.h Fri May 29 19:29:59 2020 +0000 @@ -128,6 +128,8 @@ void njs_parser_lexer_error(njs_parser_t void njs_parser_node_error(njs_vm_t *vm, njs_parser_node_t *node, njs_object_type_t type, const char *fmt, ...); +njs_int_t njs_parser_serialize_ast(njs_parser_node_t *node, njs_chb_t *chain); + #define njs_parser_restricted_identifier(token) \ (token == NJS_TOKEN_ARGUMENTS || token == NJS_TOKEN_EVAL) diff -r 86f55a7dc4a4 -r d255e73aed3b src/njs_shell.c --- a/src/njs_shell.c Fri May 29 20:00:32 2020 +0300 +++ b/src/njs_shell.c Fri May 29 19:29:59 2020 +0000 @@ -34,6 +34,7 @@ typedef struct { uint8_t sandbox; uint8_t safe; uint8_t version; + uint8_t ast; char *file; char *command; @@ -268,6 +269,7 @@ main(int argc, char **argv) vm_options.external = &njs_console; vm_options.argv = opts.argv; vm_options.argc = opts.argc; + vm_options.ast = opts.ast; if (opts.interactive) { ret = njs_interactive_shell(&opts, &vm_options); @@ -307,6 +309,7 @@ njs_get_options(njs_opts_t *opts, int ar "njs [options] [-c string | script.js | -] [script args]" "\n" "Options:\n" + " -a print AST.\n" " -c specify the command to execute.\n" " -d print disassembled code.\n" " -f disabled denormals mode.\n" @@ -340,6 +343,10 @@ njs_get_options(njs_opts_t *opts, int ar (void) write(STDOUT_FILENO, help, njs_length(help)); return ret; + case 'a': + opts->ast = 1; + break; + case 'c': opts->interactive = 0; diff -r 86f55a7dc4a4 -r d255e73aed3b src/njs_vm.c --- a/src/njs_vm.c Fri May 29 20:00:32 2020 +0300 +++ b/src/njs_vm.c Fri May 29 19:29:59 2020 +0000 @@ -125,6 +125,8 @@ njs_int_t njs_vm_compile(njs_vm_t *vm, u_char **start, u_char *end) { njs_int_t ret; + njs_str_t ast; + njs_chb_t chain; njs_lexer_t lexer; njs_parser_t *parser, *prev; njs_generator_t generator; @@ -202,7 +204,24 @@ njs_vm_compile(njs_vm_t *vm, u_char **st njs_disassembler(vm); } - return NJS_OK; + if (njs_slow_path(vm->options.ast)) { + njs_chb_init(&chain, vm->mem_pool); + ret = njs_parser_serialize_ast(parser->node, &chain); + if (njs_slow_path(ret == NJS_ERROR)) { + return ret; + } + + if (njs_slow_path(njs_chb_join(&chain, &ast) != NJS_OK)) { + return NJS_ERROR; + } + + njs_print(ast.start, ast.length); + + njs_chb_destroy(&chain); + njs_mp_free(vm->mem_pool, ast.start); + } + + return ret; fail: diff -r 86f55a7dc4a4 -r d255e73aed3b test/njs_expect_test.exp --- a/test/njs_expect_test.exp Fri May 29 20:00:32 2020 +0300 +++ b/test/njs_expect_test.exp Fri May 29 19:29:59 2020 +0000 @@ -791,6 +791,10 @@ njs_run {"-p" "test/module" "./test/modu njs_run {"-h"} "Options" +# ast + +njs_run {"-a" "-c" "console.log(1*2)"} "{\"name\": \"END\"" + # command njs_run {"-c" "console.log(\"a b c\")"} "a b c"