From osa at FreeBSD.org.ru Sun Dec 1 01:41:23 2019 From: osa at FreeBSD.org.ru (Sergey A. Osokin) Date: Sun, 01 Dec 2019 04:41:23 +0300 Subject: [PATCH] Update nginx(8) manual page Message-ID: <477a78d1b14e415a80b6.1575164483@mp2.macomnet.net> # HG changeset patch # User Sergey A. Osokin # Date 1575163872 -10800 # Sun Dec 01 04:31:12 2019 +0300 # Node ID 477a78d1b14e415a80b6bfb3d829eb408f33f390 # Parent d13eddd9e2529b4bc30dc00aad959bd10ced4c33 Update nginx(8) manual page. Sort the command line options. diff -r d13eddd9e252 -r 477a78d1b14e docs/man/nginx.8 --- a/docs/man/nginx.8 Tue Nov 19 17:18:58 2019 +0300 +++ b/docs/man/nginx.8 Sun Dec 01 04:31:12 2019 +0300 @@ -1,5 +1,5 @@ .\" -.\" Copyright (C) 2010 Sergey A. Osokin +.\" Copyright (C) 2010, 2019 Sergey A. Osokin .\" Copyright (C) Nginx, Inc. .\" All rights reserved. .\" @@ -42,9 +42,9 @@ .Nm (pronounced .Dq engine x ) -is an HTTP and reverse proxy server, as well as a mail proxy server. -It is known for its high performance, stability, rich feature set, simple -configuration, and low resource consumption. +is an HTTP and reverse proxy server, a mail proxy server, and a generic +TCP/UDP proxy server. It is known for its high performance, stability, +rich feature set, simple configuration, and low resource consumption. .Pp The options are as follows: .Bl -tag -width ".Fl d Ar directives" @@ -91,16 +91,16 @@ Same as .Fl t , but additionally dump configuration files to standard output. +.It Fl v +Print the +.Nm +version. .It Fl V Print the .Nm version, compiler version, and .Pa configure script parameters. -.It Fl v -Print the -.Nm -version. .El .Sh SIGNALS The master process of From zhangshaokun at hisilicon.com Mon Dec 2 03:22:33 2019 From: zhangshaokun at hisilicon.com (Shaokun Zhang) Date: Mon, 2 Dec 2019 11:22:33 +0800 Subject: [PATCH] Optimal performance when use http non-persistent connection In-Reply-To: <4328c1206d3f4225b61e1c468e518e8b@huawei.com> References: <07890E72BF5A6C44976E3EB9981B18780435D11E@dggeml527-mbx.china.huawei.com> <20191120142902.GE12894@mdounin.ru> <6434936b-a321-9905-a3bc-c906cfc41641@hisilicon.com> <20191121152147.GG12894@mdounin.ru> <4328c1206d3f4225b61e1c468e518e8b@huawei.com> Message-ID: Hi, Apologies that reply later. On 2019/12/2 9:31, sunrui wrote: > > > -----????----- > ???: Maxim Dounin [mailto:mdounin at mdounin.ru] > ????: 2019?11?21? 23:22 > ???: nginx-devel at nginx.org > ??: sunrui > ??: Re: [PATCH] Optimal performance when use http non-persistent connection > > Hello! > > On Thu, Nov 21, 2019 at 07:22:16PM +0800, Shaokun Zhang wrote: > >> Hi Maixm, >> >> On 2019/11/20 22:29, Maxim Dounin wrote: >>> Hello! >>> >>> On Mon, Nov 11, 2019 at 03:07:02AM +0000, Zhangshaokun wrote: >>> >>>> # HG changeset patch >>>> # User Rui Sun > >>>> # Date 1572848389 -28800 >>>> # Mon Nov 04 14:19:49 2019 +0800 >>>> # Branch local >>>> # Node ID a5ae6e9e99f747fcb45082bac8795622938184f1 >>>> # Parent 89adf49fe76ada86d84e2af8f5cee9ca8c3dca19 >>>> Optimal performance when use http non-persistent connection >>>> >>>> diff -r 89adf49fe76a -r a5ae6e9e99f7 src/core/ngx_cycle.c >>>> --- a/src/core/ngx_cycle.c Mon Oct 21 20:22:30 2019 +0300 >>>> +++ b/src/core/ngx_cycle.c Mon Nov 04 14:19:49 2019 +0800 >>>> @@ -35,6 +35,40 @@ >>>> /* STUB */ >>>> >>>> >>>> +void >>>> +ngx_change_pid_core(ngx_cycle_t *cycle) { >>>> + ngx_pid_t setpid; >>>> + ngx_cpuset_t *setaffinity=NULL; >>>> + setpid = ngx_getpid(); >>>> + { >>>> +#if (NGX_HAVE_CPU_AFFINITY) >>>> + ngx_core_conf_t *ccf; >>>> + >>>> + ccf = (ngx_core_conf_t *) ngx_get_conf(cycle->conf_ctx, >>>> + ngx_core_module); >>>> + >>>> + if (ccf->cpu_affinity == NULL) { >>>> + setaffinity = NULL; >>>> + } >>>> + >>>> + if (ccf->cpu_affinity_auto) { >>>> + setaffinity = NULL; >>>> + } >>>> + >>>> + setaffinity = &ccf->cpu_affinity[0]; >>>> + >>>> +#else >>>> + >>>> + setaffinity = NULL; >>>> + >>>> +#endif >>>> + } >>>> + >>>> + if (setaffinity) >>>> + // set new mask >>>> + sched_setaffinity(setpid, sizeof(ngx_cpuset_t), >>>> +setaffinity); } >>>> + >>>> ngx_cycle_t * >>>> ngx_init_cycle(ngx_cycle_t *old_cycle) { @@ -278,6 +312,8 @@ >>>> return NULL; >>>> } >>>> >>>> + ngx_change_pid_core(cycle); >>>> + >>>> if (ngx_test_config && !ngx_quiet_mode) { >>>> ngx_log_stderr(0, "the configuration file %s syntax is ok", >>>> cycle->conf_file.data); >>>> >>> >>> Sorry, but it is not clear what you are trying to achieve with this >>> patch. You may want to provide more details. >>> >> >> when we test nginx in kunpeng920 which has 2chip and each chip has 2 NUMA. >> We user 32cores in 2 different NUMA to test nginx, when nginx start >> the master worker runs on which core is undefined, when the master's >> core and the worker's core in the same chip, the performance of >> non-persistent connection is 17W, but when master's core and the >> worker's core in the different chip, the performance of non-persistent >> connection only has 12W. Now, when nginx start, we migrate master >> process depend on the first worker process's cpu affinity, the performance is showed as follow: >> | default| >> optimize master and worker process on same chip when nginx start | >> 171699 | 176020 master and worker process on diff chip when nginx >> start | 129639 | 180637 > > Ok, so you are trying to bind the master process to the same core the first worker process runs on. Presumably, this can be beneficial from performance point of view in configurations with small number of worker processes, as various structures allocated by the master process after parsing configuration will be allocated from the same NUMA region the worker process runs on. > Correct? > Yes,That's it. > So the following questions are: > > 0. What units of measurement the numbers use? Connections per second? What are error margins? > The units is reqs/s, connected several second and calculate the average value, error margins is 3%. > 1. How did you tested it? Given that many configuration structures are allocated by the master process during configuration parsing, the numbers look strange. I would expect performance with master of worker process on different chips to be smaller than that on the same chip, even with the patch applied. > Well, with error margins we'll probably see there is no difference between 176020 and 180637, but this brings another question: where the difference between 129639 and 180637 comes from? Listening sockets created by the kernel on the same chip? So this probably means we shouldn't bind worker process in general, but rather create listenings sockets on the same chip instead? Note this is not the same, especially with reuseport, not to mention this cannot be done at all when we inherit listening sockets form previous configurations. > With error margins we consider there is no difference between 176020 and 180637. The difference between 129639 and 180637 is cause by the cross-chip between listening sockets and worker processes. We bind worker process because some times we run serveral nginx instances in the same time, we can use worker processes to adapt to net init to improve the performance. If we don't bind worker the patch will not work. When open the reuseport, the loss will not exist, no mater with the patch or without the patch, it's not effect the result. When we inherit listening sockets form previous configurations, if we don't bind worker process, the patch will not work, if we bind worker process, we should bind it to the same chip with the listening sockets inherit from previous configurations. > 2. What happens when there are multiple worker processes? Will this change still be beneficial, or negative, or neutral? Don't you think the case you are trying to optimize is too narrow to care about? > You means serveral nginx instances or one mater with serveral worker processes? If you means one master and serveral worker processes, we test 32core and each core has a worker process, so our test situation is 1 master with 32 worker processes. > 3. In nginx, there are platform-independent functions > ngx_get_cpu_affinity() and ngx_setaffinity() to work with CPU affinity. Why you are not using them in your patch? > Additionally, why you are not trying to bind master process to a particular CPU with "worker_cpu_affinity auto;"? ngx_get_cpu_affinity() and ngx_setaffinity() variable which is named "ngx_cycle", we assigns values to it after ngx_init_cycle() is finished, so I can't use them. It just in the situation which workers are concentrated in the same chip, and the master and workers are in diff chip, the performance shows mach loss, which is because of the cross-chip delay. When the master and worker are random distribute, no mater which chip the master on should be similar, the migrate is no needed. Thanks, > > -- > Maxim Dounin > http://mdounin.ru/ > From xeioex at nginx.com Tue Dec 3 12:11:25 2019 From: xeioex at nginx.com (Dmitry Volyntsev) Date: Tue, 03 Dec 2019 12:11:25 +0000 Subject: [njs] Fixed Number.prototype.toString(radix). Message-ID: details: https://hg.nginx.org/njs/rev/5b1bf60c8ede branches: changeset: 1272:5b1bf60c8ede user: Dmitry Volyntsev date: Tue Dec 03 14:59:26 2019 +0300 description: Fixed Number.prototype.toString(radix). Fixed buffer-overflow in Number.prototype.toString(radix) when fraction == delta == 0. The last comparison might by true for very small numbers (denormals) around zero when fast-math mode is enabled. The issue was introduced in 5f0812d53158. diffstat: auto/clang | 13 ++++ src/njs_clang.h | 18 +++++ src/njs_number.c | 4 +- src/njs_shell.c | 16 +++++ src/test/njs_unit_test.c | 150 +++++++++++++++++++++++++++++++++++++++------- 5 files changed, 175 insertions(+), 26 deletions(-) diffs (326 lines): diff -r 01c7375c9b5c -r 5b1bf60c8ede auto/clang --- a/auto/clang Fri Nov 29 12:53:33 2019 +0300 +++ b/auto/clang Tue Dec 03 14:59:26 2019 +0300 @@ -177,3 +177,16 @@ njs_feature_test="#include return 0; }" . auto/feature + + +njs_feature="_mm_setcsr()" +njs_feature_name=NJS_HAVE_DENORMALS_CONTROL +njs_feature_run=no +njs_feature_incs= +njs_feature_libs= +njs_feature_test="#include + int main(void) { + _mm_setcsr(_mm_getcsr()); + return 0; + }" +. auto/feature diff -r 01c7375c9b5c -r 5b1bf60c8ede src/njs_clang.h --- a/src/njs_clang.h Fri Nov 29 12:53:33 2019 +0300 +++ b/src/njs_clang.h Tue Dec 03 14:59:26 2019 +0300 @@ -165,6 +165,24 @@ njs_leading_zeros64(uint64_t x) #endif +#if (NJS_HAVE_DENORMALS_CONTROL) +#include + +/* + * 0x8000 Flush to zero + * 0x0040 Denormals are zeros + */ + +#define NJS_MM_DENORMALS_MASK 0x8040 + +#define njs_mm_denormals(on) \ + _mm_setcsr((_mm_getcsr() & ~NJS_MM_DENORMALS_MASK) | (!(on) ? 0x8040: 0x0)) +#else + +#define njs_mm_denormals(on) +#endif + + #ifndef NJS_MAX_ALIGNMENT #if (NJS_SOLARIS) diff -r 01c7375c9b5c -r 5b1bf60c8ede src/njs_number.c --- a/src/njs_number.c Fri Nov 29 12:53:33 2019 +0300 +++ b/src/njs_number.c Tue Dec 03 14:59:26 2019 +0300 @@ -565,7 +565,7 @@ njs_number_prototype_to_string(njs_vm_t number = njs_number(value); - if (radix != 10 && !isnan(number) && !isinf(number)) { + if (radix != 10 && !isnan(number) && !isinf(number) && number != 0) { return njs_number_to_string_radix(vm, &vm->retval, number, radix); } } @@ -838,7 +838,7 @@ njs_number_to_string_radix(njs_vm_t *vm, delta = 0.5 * (njs_number_next_double(n) - n); delta = njs_max(njs_number_next_double(0.0), delta); - if (fraction >= delta) { + if (fraction >= delta && delta != 0) { *p++ = '.'; do { diff -r 01c7375c9b5c -r 5b1bf60c8ede src/njs_shell.c --- a/src/njs_shell.c Fri Nov 29 12:53:33 2019 +0300 +++ b/src/njs_shell.c Tue Dec 03 14:59:26 2019 +0300 @@ -26,6 +26,7 @@ typedef struct { uint8_t disassemble; + uint8_t denormals; uint8_t interactive; uint8_t module; uint8_t quiet; @@ -232,6 +233,8 @@ main(int argc, char **argv) goto done; } + njs_mm_denormals(opts.denormals); + njs_memzero(&vm_options, sizeof(njs_vm_opt_t)); if (opts.file == NULL) { @@ -306,6 +309,7 @@ njs_get_options(njs_opts_t *opts, int ar "Options:\n" " -c specify the command to execute.\n" " -d print disassembled code.\n" + " -f disabled denormals mode.\n" " -p set path prefix for modules.\n" " -q disable interactive introduction prompt.\n" " -s sandbox mode.\n" @@ -316,6 +320,8 @@ njs_get_options(njs_opts_t *opts, int ar ret = NJS_DONE; + opts->denormals = 1; + for (i = 1; i < argc; i++) { p = argv[i]; @@ -349,6 +355,16 @@ njs_get_options(njs_opts_t *opts, int ar opts->disassemble = 1; break; + case 'f': + +#if !(NJS_HAVE_DENORMALS_CONTROL) + njs_stderror("option \"-f\" is not supported\n"); + return NJS_ERROR; +#endif + + opts->denormals = 0; + break; + case 'p': if (++i < argc) { opts->n_paths++; diff -r 01c7375c9b5c -r 5b1bf60c8ede src/test/njs_unit_test.c --- a/src/test/njs_unit_test.c Fri Nov 29 12:53:33 2019 +0300 +++ b/src/test/njs_unit_test.c Tue Dec 03 14:59:26 2019 +0300 @@ -361,13 +361,23 @@ static njs_unit_test_t njs_test[] = /* Number.toString(radix) method. */ -#ifndef NJS_SUNC { njs_str("0..toString(2)"), njs_str("0") }, -#endif - - { njs_str("240..toString(2)"), - njs_str("11110000") }, + + { njs_str("(1234.567).toString(3)"), + njs_str("1200201.120022100021001021021002202") }, + + { njs_str("(1234.567).toString(5)"), + njs_str("14414.240414141414141414") }, + + { njs_str("(1234.567).toString(17)"), + njs_str("44a.9aeb6faa0da") }, + + { njs_str("(1234.567).toString(36)"), + njs_str("ya.kety9sifl") }, + + { njs_str("Number(-1.1).toString(36)"), + njs_str("-1.3llllllllm") }, { njs_str("Math.pow(-2, 1023).toString(2).length"), njs_str("1025") }, @@ -439,10 +449,6 @@ static njs_unit_test_t njs_test[] = { njs_str("1e20.toString(14)"), njs_str("33cb3bb449c2a92000") }, - /* Smallest positive double (next_double(0)). */ - { njs_str("4.94065645841246544176568792868E-324.toString(36) == ('0.' + '0'.repeat(207) +'3')"), - njs_str("true") }, - { njs_str("1.7976931348623157E+308.toString(36) == ('1a1e4vngaiqo' + '0'.repeat(187))"), njs_str("true") }, @@ -450,13 +456,6 @@ static njs_unit_test_t njs_test[] = { njs_str("1.7976931348623157E+308.toString(2) == ('1'.repeat(53) + '0'.repeat(971))"), njs_str("true") }, - /* Maximum fraction length. */ - { njs_str("2.2250738585072014E-323.toString(2) == ('0.' + '0'.repeat(1071) + '101')"), - njs_str("true") }, - - { njs_str("2.2250738585072014E-308.toString(2) == ('0.' + '0'.repeat(1021) + '1')"), - njs_str("true") }, - { njs_str("Array(5).fill().map((n, i) => i + 10).map((v)=>(1.2312313132).toString(v))"), njs_str("1.2312313132,1.25a850416057383,1.293699002749414,1.3010274cab0288,1.3346da6d5d455c") }, @@ -639,14 +638,6 @@ static njs_unit_test_t njs_test[] = { njs_str("Array(5).fill().map((n, i) => i + 1).map((v)=>((Math.pow(-1,v))*(2*v)/3).toExponential())"), njs_str("-6.666666666666666e-1,1.3333333333333333e+0,-2e+0,2.6666666666666667e+0,-3.3333333333333337e+0") }, -#ifndef NJS_SUNC - { njs_str("4.94065645841246544176568792868e-324.toExponential()"), - njs_str("5e-324") }, - - { njs_str("4.94065645841246544176568792868e-324.toExponential(10)"), - njs_str("4.9406564584e-324") }, -#endif - { njs_str("1.7976931348623157e+308.toExponential()"), njs_str("1.7976931348623157e+308") }, @@ -15343,6 +15334,84 @@ static njs_unit_test_t njs_test[] = }; +static njs_unit_test_t njs_denormals_test[] = +{ + { njs_str("2.2250738585072014e-308"), + njs_str("2.2250738585072014e-308") }, + +#ifndef NJS_SUNC + { njs_str("2.2250738585072014E-308.toString(2) == ('0.' + '0'.repeat(1021) + '1')"), + njs_str("true") }, + + { njs_str("Number('2.2250738585072014E-323')"), + njs_str("2.5e-323") }, + + { njs_str("Number('2.2250738585072014E-323') + 0"), + njs_str("2.5e-323") }, + + /* Smallest positive double (next_double(0)). */ + { njs_str("5E-324.toString(36) === '0.' + '0'.repeat(207) + '3'"), + njs_str("true") }, + + /* Maximum fraction length. */ + { njs_str("2.2250738585072014E-323.toString(2) == ('0.' + '0'.repeat(1071) + '101')"), + njs_str("true") }, + + /* Denormals. */ + { njs_str("var zeros = count => '0'.repeat(count);" + "[" + " [1.8858070859709815e-308, `0.${zeros(1022)}1101100011110111011100000100011001111101110001010111`]," + // FIXME: " [Number.MIN_VALUE, `0.${zeros(1073)}1`]" + " [-5.06631661953108e-309, `-0.${zeros(1024)}11101001001010000001101111010101011111111011010111`]," + " [6.22574126804e-313, `0.${zeros(1037)}11101010101101100111000110100111001`]," + " [-4e-323, `-0.${zeros(1070)}1`]," + "].every(t=>t[0].toString(2) === t[1])"), + njs_str("true") }, + + { njs_str("4.94065645841246544176568792868e-324.toExponential()"), + njs_str("5e-324") }, + + { njs_str("4.94065645841246544176568792868e-324.toExponential(10)"), + njs_str("4.9406564584e-324") }, +#endif + +}; + + +static njs_unit_test_t njs_disabled_denormals_test[] = +{ + { njs_str("Number('2.2250738585072014E-323')"), + njs_str("0") }, + + { njs_str("Number('2.2250738585072014E-323') + 0"), + njs_str("0") }, + + /* Smallest positive double (next_double(0)). */ + { njs_str("5E-324.toString(36)"), + njs_str("0") }, + + { njs_str("2.2250738585072014E-323.toString(2)"), + njs_str("0") }, + + /* Smallest normal double. */ + + { njs_str("2.2250738585072014e-308"), + njs_str("2.2250738585072014e-308") }, + + { njs_str("2.2250738585072014e-308/2"), + njs_str("0") }, + + /* Denormals. */ + { njs_str("[" + "1.8858070859709815e-308," + "-5.06631661953108e-309," + "6.22574126804e-313," + "-4e-323," + "].map(v=>v.toString(2))"), + njs_str("0,0,0,0") }, +}; + + static njs_unit_test_t njs_module_test[] = { { njs_str("function f(){return 2}; var f; f()"), @@ -16816,12 +16885,45 @@ main(int argc, char **argv) opts.repeat = 1; opts.unsafe = 1; + njs_mm_denormals(1); + ret = njs_unit_test(njs_test, njs_nitems(njs_test), "script tests", &opts, &stat); if (ret != NJS_OK) { return ret; } + ret = njs_unit_test(njs_denormals_test, njs_nitems(njs_denormals_test), + "denormals tests", &opts, &stat); + if (ret != NJS_OK) { + return ret; + } + +#if (NJS_HAVE_DENORMALS_CONTROL) + + njs_mm_denormals(0); + + ret = njs_unit_test(njs_test, njs_nitems(njs_test), + "script tests (disabled denormals)", &opts, &stat); + if (ret != NJS_OK) { + return ret; + } + + ret = njs_unit_test(njs_disabled_denormals_test, + njs_nitems(njs_disabled_denormals_test), + "disabled denormals tests", &opts, &stat); + if (ret != NJS_OK) { + return ret; + } + + njs_mm_denormals(1); + +#else + + (void) njs_disabled_denormals_test; + +#endif + ret = njs_timezone_optional_test(&opts, &stat); if (ret != NJS_OK) { return ret; From mdounin at mdounin.ru Tue Dec 3 13:18:34 2019 From: mdounin at mdounin.ru (Maxim Dounin) Date: Tue, 3 Dec 2019 16:18:34 +0300 Subject: [PATCH] Mime type for Web Assembly In-Reply-To: References: Message-ID: <20191203131834.GV12894@mdounin.ru> Hello! On Fri, Nov 29, 2019 at 09:27:46PM +0000, Chris Couzens wrote: > # HG changeset patch > # User Chris Couzens > # Date 1575032353 0 > # Fri Nov 29 12:59:13 2019 +0000 > # Node ID d826b28f25157901dff4a010cca180a0d13e7c35 > # Parent d13eddd9e2529b4bc30dc00aad959bd10ced4c33 > Mime.types: Add Web Assembly > > Web assembly is a web standard and so nginx should support hosting > it by default. > > Not serving the correct mime type causes standard compliant browsers > to reject it. > > https://www.w3.org/TR/wasm-web-api-1/#streaming-modules > > > If the Response is not CORS-same-origin, does not represent an ok > > status, or does not match the `application/wasm` MIME type, the > > returned promise will be rejected with a TypeError; > > diff -r d13eddd9e252 -r d826b28f2515 conf/mime.types > --- a/conf/mime.types Tue Nov 19 17:18:58 2019 +0300 > +++ b/conf/mime.types Fri Nov 29 12:59:13 2019 +0000 > @@ -8,6 +8,7 @@ > application/javascript js; > application/atom+xml atom; > application/rss+xml rss; > + application/wasm wasm; > > text/mathml mml; > text/plain txt; Please see https://trac.nginx.org/nginx/ticket/1606 for detailed comments about adding application/wasm MIME type. -- Maxim Dounin http://mdounin.ru/ From alexander.borisov at nginx.com Tue Dec 3 13:44:29 2019 From: alexander.borisov at nginx.com (Alexander Borisov) Date: Tue, 03 Dec 2019 13:44:29 +0000 Subject: [njs] Shell: fixed memory leak when empty line is entered. Message-ID: details: https://hg.nginx.org/njs/rev/afe38b7fabe4 branches: changeset: 1273:afe38b7fabe4 user: Alexander Borisov date: Tue Dec 03 16:43:04 2019 +0300 description: Shell: fixed memory leak when empty line is entered. diffstat: src/njs_shell.c | 9 ++++----- 1 files changed, 4 insertions(+), 5 deletions(-) diffs (21 lines): diff -r 5b1bf60c8ede -r afe38b7fabe4 src/njs_shell.c --- a/src/njs_shell.c Tue Dec 03 14:59:26 2019 +0300 +++ b/src/njs_shell.c Tue Dec 03 16:43:04 2019 +0300 @@ -869,13 +869,12 @@ njs_interactive_shell(njs_opts_t *opts, } line.length = njs_strlen(line.start); - if (line.length == 0) { - continue; - } - add_history((char *) line.start); + if (line.length != 0) { + add_history((char *) line.start); - njs_process_script(opts, vm_options->external, &line); + njs_process_script(opts, vm_options->external, &line); + } /* editline allocs a new buffer every time. */ free(line.start); From alexander.borisov at nginx.com Tue Dec 3 13:44:30 2019 From: alexander.borisov at nginx.com (Alexander Borisov) Date: Tue, 03 Dec 2019 13:44:30 +0000 Subject: [njs] Style. Message-ID: details: https://hg.nginx.org/njs/rev/da7cff928b80 branches: changeset: 1274:da7cff928b80 user: Alexander Borisov date: Tue Dec 03 16:44:03 2019 +0300 description: Style. diffstat: src/njs_diyfp.h | 4 ++-- src/njs_value.c | 1 + src/njs_value.h | 6 ++---- 3 files changed, 5 insertions(+), 6 deletions(-) diffs (45 lines): diff -r afe38b7fabe4 -r da7cff928b80 src/njs_diyfp.h --- a/src/njs_diyfp.h Tue Dec 03 16:43:04 2019 +0300 +++ b/src/njs_diyfp.h Tue Dec 03 16:44:03 2019 +0300 @@ -19,8 +19,8 @@ typedef struct { typedef union { - double d; - uint64_t u64; + double d; + uint64_t u64; } njs_diyfp_conv_t; diff -r afe38b7fabe4 -r da7cff928b80 src/njs_value.c --- a/src/njs_value.c Tue Dec 03 16:43:04 2019 +0300 +++ b/src/njs_value.c Tue Dec 03 16:44:03 2019 +0300 @@ -1251,6 +1251,7 @@ njs_value_to_object(njs_vm_t *vm, njs_va return NJS_ERROR; } + void njs_symbol_conversion_failed(njs_vm_t *vm, njs_bool_t to_string) { diff -r afe38b7fabe4 -r da7cff928b80 src/njs_value.h --- a/src/njs_value.h Tue Dec 03 16:43:04 2019 +0300 +++ b/src/njs_value.h Tue Dec 03 16:44:03 2019 +0300 @@ -907,14 +907,12 @@ njs_int_t njs_value_property_delete(njs_ njs_value_t *key, njs_value_t *removed); njs_int_t njs_value_to_object(njs_vm_t *vm, njs_value_t *value); +void njs_symbol_conversion_failed(njs_vm_t *vm, njs_bool_t to_string); + #include "njs_number.h" -void -njs_symbol_conversion_failed(njs_vm_t *vm, njs_bool_t to_string); - - njs_inline njs_int_t njs_value_to_number(njs_vm_t *vm, njs_value_t *value, double *dst) { From alexander.borisov at nginx.com Tue Dec 3 14:14:04 2019 From: alexander.borisov at nginx.com (Alexander Borisov) Date: Tue, 03 Dec 2019 14:14:04 +0000 Subject: [njs] Introduced SpeciesConstructor() from the specification. Message-ID: details: https://hg.nginx.org/njs/rev/904333a40816 branches: changeset: 1275:904333a40816 user: Alexander Borisov date: Tue Dec 03 17:13:34 2019 +0300 description: Introduced SpeciesConstructor() from the specification. diffstat: src/njs_value.c | 53 +++++++++++++++++++++++++++++++++++++++++++++++++++++ src/njs_value.h | 3 +++ 2 files changed, 56 insertions(+), 0 deletions(-) diffs (73 lines): diff -r da7cff928b80 -r 904333a40816 src/njs_value.c --- a/src/njs_value.c Tue Dec 03 16:44:03 2019 +0300 +++ b/src/njs_value.c Tue Dec 03 17:13:34 2019 +0300 @@ -1259,3 +1259,56 @@ njs_symbol_conversion_failed(njs_vm_t *v ? "Cannot convert a Symbol value to a string" : "Cannot convert a Symbol value to a number"); } + + +njs_int_t +njs_value_species_constructor(njs_vm_t *vm, njs_value_t *object, + njs_value_t *default_constructor, njs_value_t *dst) +{ + njs_int_t ret; + njs_value_t constructor, retval; + + static const njs_value_t string_constructor = njs_string("constructor"); + static const njs_value_t string_species = + njs_wellknown_symbol(NJS_SYMBOL_SPECIES); + + ret = njs_value_property(vm, object, njs_value_arg(&string_constructor), + &constructor); + if (njs_slow_path(ret == NJS_ERROR)) { + return NJS_ERROR; + } + + if (njs_is_undefined(&constructor)) { + goto default_?onstructor; + } + + if (njs_slow_path(!njs_is_object(&constructor))) { + njs_type_error(vm, "constructor is not object"); + return NJS_ERROR; + } + + ret = njs_value_property(vm, &constructor, njs_value_arg(&string_species), + &retval); + if (njs_slow_path(ret == NJS_ERROR)) { + return NJS_ERROR; + } + + if (njs_value_is_null_or_undefined(&retval)) { + goto default_?onstructor; + } + + if (!njs_is_function(&retval)) { + njs_type_error(vm, "object does not contain a constructor"); + return NJS_ERROR; + } + + *dst = retval; + + return NJS_OK; + +default_?onstructor: + + *dst = *default_constructor; + + return NJS_OK; +} diff -r da7cff928b80 -r 904333a40816 src/njs_value.h --- a/src/njs_value.h Tue Dec 03 16:44:03 2019 +0300 +++ b/src/njs_value.h Tue Dec 03 17:13:34 2019 +0300 @@ -909,6 +909,9 @@ njs_int_t njs_value_to_object(njs_vm_t * void njs_symbol_conversion_failed(njs_vm_t *vm, njs_bool_t to_string); +njs_int_t njs_value_species_constructor(njs_vm_t *vm, njs_value_t *object, + njs_value_t *default_constructor, njs_value_t *dst); + #include "njs_number.h" From alexander.borisov at nginx.com Tue Dec 3 14:59:17 2019 From: alexander.borisov at nginx.com (Alexander Borisov) Date: Tue, 03 Dec 2019 14:59:17 +0000 Subject: [njs] Fixed typo introduced in 904333a40816. Message-ID: details: https://hg.nginx.org/njs/rev/6e84ff419b88 branches: changeset: 1276:6e84ff419b88 user: Alexander Borisov date: Tue Dec 03 17:58:30 2019 +0300 description: Fixed typo introduced in 904333a40816. diffstat: src/njs_value.c | 6 +++--- 1 files changed, 3 insertions(+), 3 deletions(-) diffs (30 lines): diff -r 904333a40816 -r 6e84ff419b88 src/njs_value.c --- a/src/njs_value.c Tue Dec 03 17:13:34 2019 +0300 +++ b/src/njs_value.c Tue Dec 03 17:58:30 2019 +0300 @@ -1279,7 +1279,7 @@ njs_value_species_constructor(njs_vm_t * } if (njs_is_undefined(&constructor)) { - goto default_?onstructor; + goto default_constructor; } if (njs_slow_path(!njs_is_object(&constructor))) { @@ -1294,7 +1294,7 @@ njs_value_species_constructor(njs_vm_t * } if (njs_value_is_null_or_undefined(&retval)) { - goto default_?onstructor; + goto default_constructor; } if (!njs_is_function(&retval)) { @@ -1306,7 +1306,7 @@ njs_value_species_constructor(njs_vm_t * return NJS_OK; -default_?onstructor: +default_constructor: *dst = *default_constructor; From alexander.borisov at nginx.com Tue Dec 3 15:15:52 2019 From: alexander.borisov at nginx.com (Alexander Borisov) Date: Tue, 03 Dec 2019 15:15:52 +0000 Subject: [njs] Added njs_is_memory_error() function. Message-ID: details: https://hg.nginx.org/njs/rev/ef64784b77f6 branches: changeset: 1277:ef64784b77f6 user: Alexander Borisov date: Tue Dec 03 18:02:40 2019 +0300 description: Added njs_is_memory_error() function. diffstat: src/njs_error.h | 14 ++++++++++++++ src/njs_vm.c | 7 +------ 2 files changed, 15 insertions(+), 6 deletions(-) diffs (39 lines): diff -r 6e84ff419b88 -r ef64784b77f6 src/njs_error.h --- a/src/njs_error.h Tue Dec 03 17:58:30 2019 +0300 +++ b/src/njs_error.h Tue Dec 03 18:02:40 2019 +0300 @@ -59,4 +59,18 @@ extern const njs_object_type_init_t njs extern const njs_object_type_init_t njs_memory_error_type_init; +njs_inline njs_int_t +njs_is_memory_error(njs_vm_t *vm, njs_value_t *value) +{ + if (njs_is_error(value) + && njs_has_prototype(vm, value, NJS_OBJ_TYPE_INTERNAL_ERROR) + && !njs_object(value)->extensible) + { + return 1; + } + + return 0; +} + + #endif /* _NJS_BOOLEAN_H_INCLUDED_ */ diff -r 6e84ff419b88 -r ef64784b77f6 src/njs_vm.c --- a/src/njs_vm.c Tue Dec 03 17:58:30 2019 +0300 +++ b/src/njs_vm.c Tue Dec 03 18:02:40 2019 +0300 @@ -852,12 +852,7 @@ njs_vm_value_to_string(njs_vm_t *vm, njs } if (njs_is_error(src)) { - - /* MemoryError is a nonextensible internal error. */ - - if (njs_has_prototype(vm, src, NJS_OBJ_TYPE_INTERNAL_ERROR) - && !njs_object(src)->extensible) - { + if (njs_is_memory_error(vm, src)) { njs_string_get(&njs_string_memory_error, dst); return NJS_OK; } From ru at nginx.com Thu Dec 5 11:04:08 2019 From: ru at nginx.com (Ruslan Ermilov) Date: Thu, 5 Dec 2019 14:04:08 +0300 Subject: [PATCH] Update nginx(8) manual page In-Reply-To: <477a78d1b14e415a80b6.1575164483@mp2.macomnet.net> References: <477a78d1b14e415a80b6.1575164483@mp2.macomnet.net> Message-ID: <20191205110408.GA84772@lo0.su> On Sun, Dec 01, 2019 at 04:41:23AM +0300, Sergey A. Osokin wrote: > # HG changeset patch > # User Sergey A. Osokin > # Date 1575163872 -10800 > # Sun Dec 01 04:31:12 2019 +0300 > # Node ID 477a78d1b14e415a80b6bfb3d829eb408f33f390 > # Parent d13eddd9e2529b4bc30dc00aad959bd10ced4c33 > Update nginx(8) manual page. > Sort the command line options. > > diff -r d13eddd9e252 -r 477a78d1b14e docs/man/nginx.8 > --- a/docs/man/nginx.8 Tue Nov 19 17:18:58 2019 +0300 > +++ b/docs/man/nginx.8 Sun Dec 01 04:31:12 2019 +0300 > @@ -1,5 +1,5 @@ > .\" > -.\" Copyright (C) 2010 Sergey A. Osokin > +.\" Copyright (C) 2010, 2019 Sergey A. Osokin > .\" Copyright (C) Nginx, Inc. > .\" All rights reserved. > .\" The document date should be updated. > @@ -42,9 +42,9 @@ > .Nm > (pronounced > .Dq engine x ) > -is an HTTP and reverse proxy server, as well as a mail proxy server. > -It is known for its high performance, stability, rich feature set, simple > -configuration, and low resource consumption. > +is an HTTP and reverse proxy server, a mail proxy server, and a generic > +TCP/UDP proxy server. It is known for its high performance, stability, > +rich feature set, simple configuration, and low resource consumption. Each new sentence should start on its own line. > .Pp > The options are as follows: > .Bl -tag -width ".Fl d Ar directives" > @@ -91,16 +91,16 @@ > Same as > .Fl t , > but additionally dump configuration files to standard output. > +.It Fl v > +Print the > +.Nm > +version. > .It Fl V > Print the > .Nm > version, compiler version, and > .Pa configure > script parameters. > -.It Fl v > -Print the > -.Nm > -version. > .El > .Sh SIGNALS > The master process of In ASCII, capital letters come first, and mdoc(7) follows the same sorting order. The correct fix would be to interchange -T with -t. # HG changeset patch # User Sergey A. Osokin # Date 1575543437 -10800 # Thu Dec 05 13:57:17 2019 +0300 # Node ID 5186ed8aa6330230205b69ba05ff449457b67410 # Parent 03bd76ca0fd8ce0628503a0a4cd453fbb206bafe Update manpage, sort command line options. diff --git a/docs/man/nginx.8 b/docs/man/nginx.8 --- a/docs/man/nginx.8 +++ b/docs/man/nginx.8 @@ -1,5 +1,5 @@ .\" -.\" Copyright (C) 2010 Sergey A. Osokin +.\" Copyright (C) 2010, 2019 Sergey A. Osokin .\" Copyright (C) Nginx, Inc. .\" All rights reserved. .\" @@ -25,7 +25,7 @@ .\" SUCH DAMAGE. .\" .\" -.Dd June 16, 2015 +.Dd December 5, 2019 .Dt NGINX 8 .Os .Sh NAME @@ -42,7 +42,8 @@ .Nm (pronounced .Dq engine x ) -is an HTTP and reverse proxy server, as well as a mail proxy server. +is an HTTP and reverse proxy server, a mail proxy server, and a generic +TCP/UDP proxy server. It is known for its high performance, stability, rich feature set, simple configuration, and low resource consumption. .Pp @@ -82,15 +83,15 @@ The following table shows the correspond .It Cm reload .Dv SIGHUP .El +.It Fl T +Same as +.Fl t , +but additionally dump configuration files to standard output. .It Fl t Do not run, just test the configuration file. .Nm checks the configuration file syntax and then tries to open files referenced in the configuration file. -.It Fl T -Same as -.Fl t , -but additionally dump configuration files to standard output. .It Fl V Print the .Nm From mdounin at mdounin.ru Thu Dec 5 17:48:35 2019 From: mdounin at mdounin.ru (Maxim Dounin) Date: Thu, 05 Dec 2019 17:48:35 +0000 Subject: [nginx] Version bump. Message-ID: details: https://hg.nginx.org/nginx/rev/3da195363cb0 branches: changeset: 7599:3da195363cb0 user: Maxim Dounin date: Thu Dec 05 19:22:48 2019 +0300 description: Version bump. diffstat: src/core/nginx.h | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) diffs (14 lines): diff --git a/src/core/nginx.h b/src/core/nginx.h --- a/src/core/nginx.h +++ b/src/core/nginx.h @@ -9,8 +9,8 @@ #define _NGINX_H_INCLUDED_ -#define nginx_version 1017006 -#define NGINX_VERSION "1.17.6" +#define nginx_version 1017007 +#define NGINX_VERSION "1.17.7" #define NGINX_VER "nginx/" NGINX_VERSION #ifdef NGX_BUILD From mdounin at mdounin.ru Thu Dec 5 17:48:37 2019 From: mdounin at mdounin.ru (Maxim Dounin) Date: Thu, 05 Dec 2019 17:48:37 +0000 Subject: [nginx] Upstream keepalive: clearing of c->data in cached connections. Message-ID: details: https://hg.nginx.org/nginx/rev/3939483cd1b5 branches: changeset: 7600:3939483cd1b5 user: Maxim Dounin date: Thu Dec 05 19:38:06 2019 +0300 description: Upstream keepalive: clearing of c->data in cached connections. Previously, connections returned from keepalive cache had c->data pointing to the keepalive cache item. While this shouldn't be a problem for correct code, as c->data is not expected to be used before it is set, explicitly clearing it might help to avoid confusion. diffstat: src/http/modules/ngx_http_upstream_keepalive_module.c | 1 + 1 files changed, 1 insertions(+), 0 deletions(-) diffs (11 lines): diff --git a/src/http/modules/ngx_http_upstream_keepalive_module.c b/src/http/modules/ngx_http_upstream_keepalive_module.c --- a/src/http/modules/ngx_http_upstream_keepalive_module.c +++ b/src/http/modules/ngx_http_upstream_keepalive_module.c @@ -275,6 +275,7 @@ found: c->idle = 0; c->sent = 0; + c->data = NULL; c->log = pc->log; c->read->log = pc->log; c->write->log = pc->log; From osa at freebsd.org.ru Fri Dec 6 00:17:21 2019 From: osa at freebsd.org.ru (Sergey A. Osokin) Date: Fri, 6 Dec 2019 03:17:21 +0300 Subject: [PATCH] Update nginx(8) manual page In-Reply-To: <20191205110408.GA84772@lo0.su> References: <477a78d1b14e415a80b6.1575164483@mp2.macomnet.net> <20191205110408.GA84772@lo0.su> Message-ID: <20191206001721.GB30344@FreeBSD.org.ru> Hi Ruslan, thank you very much for the review. Could you please go ahead and commit the changes. -- Sergey Osokin On Thu, Dec 05, 2019 at 02:04:08PM +0300, Ruslan Ermilov wrote: > On Sun, Dec 01, 2019 at 04:41:23AM +0300, Sergey A. Osokin wrote: > > # HG changeset patch > > # User Sergey A. Osokin > > # Date 1575163872 -10800 > > # Sun Dec 01 04:31:12 2019 +0300 > > # Node ID 477a78d1b14e415a80b6bfb3d829eb408f33f390 > > # Parent d13eddd9e2529b4bc30dc00aad959bd10ced4c33 > > Update nginx(8) manual page. > > Sort the command line options. > > > > diff -r d13eddd9e252 -r 477a78d1b14e docs/man/nginx.8 > > --- a/docs/man/nginx.8 Tue Nov 19 17:18:58 2019 +0300 > > +++ b/docs/man/nginx.8 Sun Dec 01 04:31:12 2019 +0300 > > @@ -1,5 +1,5 @@ > > .\" > > -.\" Copyright (C) 2010 Sergey A. Osokin > > +.\" Copyright (C) 2010, 2019 Sergey A. Osokin > > .\" Copyright (C) Nginx, Inc. > > .\" All rights reserved. > > .\" > > The document date should be updated. > > > @@ -42,9 +42,9 @@ > > .Nm > > (pronounced > > .Dq engine x ) > > -is an HTTP and reverse proxy server, as well as a mail proxy server. > > -It is known for its high performance, stability, rich feature set, simple > > -configuration, and low resource consumption. > > +is an HTTP and reverse proxy server, a mail proxy server, and a generic > > +TCP/UDP proxy server. It is known for its high performance, stability, > > +rich feature set, simple configuration, and low resource consumption. > > Each new sentence should start on its own line. > > > .Pp > > The options are as follows: > > .Bl -tag -width ".Fl d Ar directives" > > @@ -91,16 +91,16 @@ > > Same as > > .Fl t , > > but additionally dump configuration files to standard output. > > +.It Fl v > > +Print the > > +.Nm > > +version. > > .It Fl V > > Print the > > .Nm > > version, compiler version, and > > .Pa configure > > script parameters. > > -.It Fl v > > -Print the > > -.Nm > > -version. > > .El > > .Sh SIGNALS > > The master process of > > In ASCII, capital letters come first, and mdoc(7) follows the same > sorting order. The correct fix would be to interchange -T with -t. > > # HG changeset patch > # User Sergey A. Osokin > # Date 1575543437 -10800 > # Thu Dec 05 13:57:17 2019 +0300 > # Node ID 5186ed8aa6330230205b69ba05ff449457b67410 > # Parent 03bd76ca0fd8ce0628503a0a4cd453fbb206bafe > Update manpage, sort command line options. > > diff --git a/docs/man/nginx.8 b/docs/man/nginx.8 > --- a/docs/man/nginx.8 > +++ b/docs/man/nginx.8 > @@ -1,5 +1,5 @@ > .\" > -.\" Copyright (C) 2010 Sergey A. Osokin > +.\" Copyright (C) 2010, 2019 Sergey A. Osokin > .\" Copyright (C) Nginx, Inc. > .\" All rights reserved. > .\" > @@ -25,7 +25,7 @@ > .\" SUCH DAMAGE. > .\" > .\" > -.Dd June 16, 2015 > +.Dd December 5, 2019 > .Dt NGINX 8 > .Os > .Sh NAME > @@ -42,7 +42,8 @@ > .Nm > (pronounced > .Dq engine x ) > -is an HTTP and reverse proxy server, as well as a mail proxy server. > +is an HTTP and reverse proxy server, a mail proxy server, and a generic > +TCP/UDP proxy server. > It is known for its high performance, stability, rich feature set, simple > configuration, and low resource consumption. > .Pp > @@ -82,15 +83,15 @@ The following table shows the correspond > .It Cm reload > .Dv SIGHUP > .El > +.It Fl T > +Same as > +.Fl t , > +but additionally dump configuration files to standard output. > .It Fl t > Do not run, just test the configuration file. > .Nm > checks the configuration file syntax and then tries to open files > referenced in the configuration file. > -.It Fl T > -Same as > -.Fl t , > -but additionally dump configuration files to standard output. > .It Fl V > Print the > .Nm > _______________________________________________ > nginx-devel mailing list > nginx-devel at nginx.org > http://mailman.nginx.org/mailman/listinfo/nginx-devel From xeioex at nginx.com Fri Dec 6 12:17:14 2019 From: xeioex at nginx.com (Dmitry Volyntsev) Date: Fri, 06 Dec 2019 12:17:14 +0000 Subject: [njs] Fixed RegExp() instance properties. Message-ID: details: https://hg.nginx.org/njs/rev/990a4b7a4053 branches: changeset: 1278:990a4b7a4053 user: Dmitry Volyntsev date: Fri Dec 06 14:44:11 2019 +0300 description: Fixed RegExp() instance properties. 1) "lastIndex" is the only own property descriptor. 2) "lastIndex" property descriptor is writable. 3) "lastIndex" is coersed to integer in RegExpBuiltinExec() and can be any value (not only positive integers). diffstat: src/njs_array.c | 13 +---- src/njs_builtin.c | 6 ++ src/njs_object.h | 19 +++++++ src/njs_regexp.c | 122 ++++++++++++++++++++++++++++++++++++---------- src/njs_regexp.h | 1 + src/njs_string.c | 2 +- src/njs_value.h | 2 +- src/njs_vm.h | 2 + src/test/njs_unit_test.c | 42 ++++++++++++++++ 9 files changed, 170 insertions(+), 39 deletions(-) diffs (433 lines): diff -r ef64784b77f6 -r 990a4b7a4053 src/njs_array.c --- a/src/njs_array.c Tue Dec 03 18:02:40 2019 +0300 +++ b/src/njs_array.c Fri Dec 06 14:44:11 2019 +0300 @@ -342,21 +342,12 @@ njs_array_length(njs_vm_t *vm,njs_object proto = njs_object(value); if (njs_fast_path(setval == NULL)) { - do { - if (njs_fast_path(proto->type == NJS_ARRAY)) { - break; - } - - proto = proto->__proto__; - } while (proto != NULL); - - if (njs_slow_path(proto == NULL)) { + array = njs_object_proto_lookup(proto, NJS_ARRAY, njs_array_t); + if (njs_slow_path(array == NULL)) { njs_set_undefined(retval); return NJS_DECLINED; } - array = (njs_array_t *) proto; - njs_set_number(retval, array->length); return NJS_OK; } diff -r ef64784b77f6 -r 990a4b7a4053 src/njs_builtin.c --- a/src/njs_builtin.c Tue Dec 03 18:02:40 2019 +0300 +++ b/src/njs_builtin.c Fri Dec 06 14:44:11 2019 +0300 @@ -166,6 +166,12 @@ njs_builtin_objects_create(njs_vm_t *vm) return NJS_ERROR; } + ret = njs_object_hash_init(vm, &shared->regexp_instance_hash, + &njs_regexp_instance_init); + if (njs_slow_path(ret != NJS_OK)) { + return NJS_ERROR; + } + object = shared->objects; for (p = njs_object_init; *p != NULL; p++) { diff -r ef64784b77f6 -r 990a4b7a4053 src/njs_object.h --- a/src/njs_object.h Tue Dec 03 18:02:40 2019 +0300 +++ b/src/njs_object.h Fri Dec 06 14:44:11 2019 +0300 @@ -259,6 +259,25 @@ njs_object_string_tag(njs_vm_t *vm, njs_ } +njs_inline njs_object_t * +_njs_object_proto_lookup(njs_object_t *proto, njs_value_type_t type) +{ + do { + if (njs_fast_path(proto->type == type)) { + break; + } + + proto = proto->__proto__; + } while (proto != NULL); + + return proto; +} + + +#define njs_object_proto_lookup(proto, vtype, ctype) \ + (ctype *) _njs_object_proto_lookup(proto, vtype) + + extern const njs_object_type_init_t njs_obj_type_init; diff -r ef64784b77f6 -r 990a4b7a4053 src/njs_regexp.c --- a/src/njs_regexp.c Tue Dec 03 18:02:40 2019 +0300 +++ b/src/njs_regexp.c Fri Dec 06 14:44:11 2019 +0300 @@ -29,7 +29,8 @@ static u_char *njs_regexp_compile_trace_ static u_char *njs_regexp_match_trace_handler(njs_trace_t *trace, njs_trace_data_t *td, u_char *start); static njs_int_t njs_regexp_exec_result(njs_vm_t *vm, njs_regexp_t *regexp, - njs_utf8_t utf8, u_char *string, njs_regex_match_data_t *match_data); + njs_utf8_t utf8, u_char *string, njs_regex_match_data_t *match_data, + uint32_t last_index); static njs_int_t njs_regexp_string_create(njs_vm_t *vm, njs_value_t *value, u_char *start, uint32_t size, int32_t length); @@ -707,13 +708,14 @@ njs_regexp_alloc(njs_vm_t *vm, njs_regex if (njs_fast_path(regexp != NULL)) { njs_lvlhsh_init(®exp->object.hash); - njs_lvlhsh_init(®exp->object.shared_hash); + regexp->object.shared_hash = vm->shared->regexp_instance_hash; regexp->object.__proto__ = &vm->prototypes[NJS_OBJ_TYPE_REGEXP].object; regexp->object.type = NJS_REGEXP; regexp->object.shared = 0; regexp->object.extensible = 1; - regexp->last_index = 0; + njs_set_number(®exp->last_index, 0); regexp->pattern = pattern; + njs_string_short_set(®exp->string, 0, 0); return regexp; } @@ -724,20 +726,42 @@ njs_regexp_alloc(njs_vm_t *vm, njs_regex static njs_int_t -njs_regexp_prototype_last_index(njs_vm_t *vm, njs_object_prop_t *prop, +njs_regexp_prototype_last_index(njs_vm_t *vm, njs_object_prop_t *unused, njs_value_t *value, njs_value_t *setval, njs_value_t *retval) { - uint32_t index; + uint32_t index, last_index; njs_regexp_t *regexp; njs_string_prop_t string; - njs_release(vm, value); + regexp = njs_object_proto_lookup(njs_object(value), NJS_REGEXP, + njs_regexp_t); + if (njs_slow_path(regexp == NULL)) { + njs_set_undefined(retval); + return NJS_DECLINED; + } - regexp = njs_regexp(value); + if (setval != NULL) { + regexp->last_index = *setval; + *retval = *setval; + + return NJS_OK; + } + + if (njs_slow_path(!njs_is_number(®exp->last_index))) { + *retval = regexp->last_index; + return NJS_OK; + } (void) njs_string_prop(&string, ®exp->string); - index = njs_string_index(&string, regexp->last_index); + last_index = njs_number(®exp->last_index); + + if (njs_slow_path(string.size < last_index)) { + *retval = regexp->last_index; + return NJS_OK; + } + + index = njs_string_index(&string, last_index); njs_set_number(retval, index); return NJS_OK; @@ -843,9 +867,12 @@ static njs_int_t njs_regexp_prototype_test(njs_vm_t *vm, njs_value_t *args, njs_uint_t nargs, njs_index_t unused) { - njs_int_t ret; + int *captures; + uint32_t last_index; + njs_int_t ret, match; njs_uint_t n; njs_regex_t *regex; + njs_regexp_t *regexp; njs_value_t *value, lvalue; const njs_value_t *retval; njs_string_prop_t string; @@ -872,6 +899,7 @@ njs_regexp_prototype_test(njs_vm_t *vm, n = (string.length != 0); + regexp = njs_regexp(njs_argument(args, 0)); pattern = njs_regexp_pattern(&args[0]); regex = &pattern->regex[n]; @@ -886,15 +914,32 @@ njs_regexp_prototype_test(njs_vm_t *vm, } } - ret = njs_regexp_match(vm, regex, string.start, string.size, + match = njs_regexp_match(vm, regex, string.start, string.size, match_data); - if (ret >= 0) { + if (match >= 0) { retval = &njs_value_true; - } else if (ret != NJS_REGEX_NOMATCH) { + } else if (match != NJS_REGEX_NOMATCH) { ret = NJS_ERROR; goto done; } + + if (pattern->global) { + ret = njs_value_to_length(vm, ®exp->last_index, &last_index); + if (njs_slow_path(ret != NJS_OK)) { + return NJS_ERROR; + } + + if (match >= 0) { + captures = njs_regex_captures(match_data); + last_index += captures[1]; + + } else { + last_index = 0; + } + + njs_set_number(®exp->last_index, last_index); + } } ret = NJS_OK; @@ -915,6 +960,7 @@ njs_int_t njs_regexp_prototype_exec(njs_vm_t *vm, njs_value_t *args, njs_uint_t nargs, njs_index_t unused) { + uint32_t last_index; njs_int_t ret; njs_utf8_t utf8; njs_value_t *value, lvalue; @@ -940,10 +986,20 @@ njs_regexp_prototype_exec(njs_vm_t *vm, regexp = njs_regexp(&args[0]); regexp->string = *value; + pattern = regexp->pattern; + + ret = njs_value_to_length(vm, ®exp->last_index, &last_index); + if (njs_slow_path(ret != NJS_OK)) { + return NJS_ERROR; + } + + if (!pattern->global) { + last_index = 0; + } (void) njs_string_prop(&string, value); - if (string.size >= regexp->last_index) { + if (string.size >= last_index) { utf8 = NJS_STRING_BYTE; type = NJS_REGEXP_BYTE; @@ -959,8 +1015,8 @@ njs_regexp_prototype_exec(njs_vm_t *vm, pattern = regexp->pattern; if (njs_regex_is_valid(&pattern->regex[type])) { - string.start += regexp->last_index; - string.size -= regexp->last_index; + string.start += last_index; + string.size -= last_index; match_data = njs_regex_match_data(&pattern->regex[type], vm->regex_context); @@ -973,7 +1029,7 @@ njs_regexp_prototype_exec(njs_vm_t *vm, string.size, match_data); if (ret >= 0) { return njs_regexp_exec_result(vm, regexp, utf8, string.start, - match_data); + match_data, last_index); } if (njs_slow_path(ret != NJS_REGEX_NOMATCH)) { @@ -984,7 +1040,10 @@ njs_regexp_prototype_exec(njs_vm_t *vm, } } - regexp->last_index = 0; + if (pattern->global) { + njs_set_number(®exp->last_index, 0); + } + vm->retval = njs_value_null; return NJS_OK; @@ -993,7 +1052,7 @@ njs_regexp_prototype_exec(njs_vm_t *vm, static njs_int_t njs_regexp_exec_result(njs_vm_t *vm, njs_regexp_t *regexp, njs_utf8_t utf8, - u_char *string, njs_regex_match_data_t *match_data) + u_char *string, njs_regex_match_data_t *match_data, uint32_t last_index) { int *captures; u_char *start; @@ -1045,10 +1104,10 @@ njs_regexp_exec_result(njs_vm_t *vm, njs /* TODO: Non UTF-8 position */ - njs_set_number(&prop->value, regexp->last_index + captures[0]); + njs_set_number(&prop->value, last_index + captures[0]); if (regexp->pattern->global) { - regexp->last_index += captures[1]; + njs_set_number(®exp->last_index, last_index + captures[1]); } lhq.key_hash = NJS_INDEX_HASH; @@ -1206,12 +1265,6 @@ static const njs_object_prop_t njs_rege { .type = NJS_PROPERTY_HANDLER, - .name = njs_string("lastIndex"), - .value = njs_prop_handler(njs_regexp_prototype_last_index), - }, - - { - .type = NJS_PROPERTY_HANDLER, .name = njs_string("global"), .value = njs_prop_handler(njs_regexp_prototype_global), .configurable = 1, @@ -1264,6 +1317,23 @@ static const njs_object_prop_t njs_rege }; +const njs_object_prop_t njs_regexp_instance_properties[] = +{ + { + .type = NJS_PROPERTY_HANDLER, + .name = njs_string("lastIndex"), + .value = njs_prop_handler(njs_regexp_prototype_last_index), + .writable = 1, + }, +}; + + +const njs_object_init_t njs_regexp_instance_init = { + njs_regexp_instance_properties, + njs_nitems(njs_regexp_instance_properties), +}; + + const njs_object_init_t njs_regexp_prototype_init = { njs_regexp_prototype_properties, njs_nitems(njs_regexp_prototype_properties), diff -r ef64784b77f6 -r 990a4b7a4053 src/njs_regexp.h --- a/src/njs_regexp.h Tue Dec 03 18:02:40 2019 +0300 +++ b/src/njs_regexp.h Fri Dec 06 14:44:11 2019 +0300 @@ -33,6 +33,7 @@ njs_int_t njs_regexp_to_string(njs_vm_t const njs_value_t *regexp); +extern const njs_object_init_t njs_regexp_instance_init; extern const njs_object_type_init_t njs_regexp_type_init; diff -r ef64784b77f6 -r 990a4b7a4053 src/njs_string.c --- a/src/njs_string.c Tue Dec 03 18:02:40 2019 +0300 +++ b/src/njs_string.c Fri Dec 06 14:44:11 2019 +0300 @@ -3166,7 +3166,7 @@ njs_string_match_multiple(njs_vm_t *vm, njs_regexp_utf8_t type; njs_string_prop_t string; - args[1].data.u.regexp->last_index = 0; + njs_set_number(&args[1].data.u.regexp->last_index, 0); vm->retval = njs_value_null; (void) njs_string_prop(&string, &args[0]); diff -r ef64784b77f6 -r 990a4b7a4053 src/njs_value.h --- a/src/njs_value.h Tue Dec 03 18:02:40 2019 +0300 +++ b/src/njs_value.h Fri Dec 06 14:44:11 2019 +0300 @@ -262,7 +262,7 @@ struct njs_function_s { struct njs_regexp_s { njs_object_t object; - uint32_t last_index; + njs_value_t last_index; njs_regexp_pattern_t *pattern; /* * This string value can be unaligned since diff -r ef64784b77f6 -r 990a4b7a4053 src/njs_vm.h --- a/src/njs_vm.h Tue Dec 03 18:02:40 2019 +0300 +++ b/src/njs_vm.h Fri Dec 06 14:44:11 2019 +0300 @@ -247,11 +247,13 @@ typedef struct { struct njs_vm_shared_s { njs_lvlhsh_t keywords_hash; njs_lvlhsh_t values_hash; + njs_lvlhsh_t array_instance_hash; njs_lvlhsh_t string_instance_hash; njs_lvlhsh_t function_instance_hash; njs_lvlhsh_t arrow_instance_hash; njs_lvlhsh_t arguments_object_instance_hash; + njs_lvlhsh_t regexp_instance_hash; njs_lvlhsh_t env_hash; diff -r ef64784b77f6 -r 990a4b7a4053 src/test/njs_unit_test.c --- a/src/test/njs_unit_test.c Tue Dec 03 18:02:40 2019 +0300 +++ b/src/test/njs_unit_test.c Fri Dec 06 14:44:11 2019 +0300 @@ -8591,6 +8591,48 @@ static njs_unit_test_t njs_test[] = "r.lastIndex +' '+ r.source +' '+ r.source.length +' '+ r"), njs_str("1 \\x80 4 /\\x80/g") }, + { njs_str("var descs = Object.getOwnPropertyDescriptors(RegExp('a'));" + "Object.keys(descs)"), + njs_str("lastIndex") }, + + { njs_str("var props = Object.getOwnPropertyDescriptor(RegExp('a'), 'lastIndex');" + "props.writable && !props.enumerable && !props.configurable"), + njs_str("true") }, + + { njs_str("var re = /a/; re.lastIndex"), + njs_str("0") }, + + { njs_str("var re = /a?/g; re.exec('a?'.repeat(32)); re.lastIndex"), + njs_str("2") }, + + { njs_str("var re = new RegExp('?'.repeat(33), 'g'); re.exec('?'.repeat(33)); re.lastIndex"), + njs_str("33") }, + + { njs_str("var re = new RegExp('?'.repeat(33), 'g'); re.exec('?'.repeat(33)); " + "re.lastIndex = 67; re.lastIndex"), + njs_str("67") }, + + { njs_str("var re = /a/; re.lastIndex = 4; Object.create(re).lastIndex"), + njs_str("4") }, + + { njs_str("var re = /a/g; re.lastIndex = {valueOf(){throw 'Oops'}}; typeof re.lastIndex"), + njs_str("object") }, + + { njs_str("var re = /a/g; re.lastIndex = {valueOf(){throw 'Oops'}}; re.exec('a')"), + njs_str("Oops") }, + + { njs_str("var re = /a/; Object.defineProperty(re, 'lastIndex', {value:'qq'}); re.lastIndex"), + njs_str("qq") }, + + { njs_str("var re = /a/; re.lastIndex = 'qq'; Object.create(re).lastIndex"), + njs_str("qq") }, + + { njs_str("var re = /(?:ab|cd)\\d?/g; re.lastIndex=-1; re.test('aacd22 '); re.lastIndex"), + njs_str("5") }, + + { njs_str("var re = /(?:ab|cd)\\d?/g; re.lastIndex=-1; re.test('@@'); re.lastIndex"), + njs_str("0") }, + /* * It seems that "/????/ig" fails on early PCRE versions. * It fails at least in 8.1 and works at least in 8.31. From xeioex at nginx.com Fri Dec 6 12:17:15 2019 From: xeioex at nginx.com (Dmitry Volyntsev) Date: Fri, 06 Dec 2019 12:17:15 +0000 Subject: [njs] Introduced ToIndex() conversion primitive from the spec. Message-ID: details: https://hg.nginx.org/njs/rev/19b741311270 branches: changeset: 1279:19b741311270 user: Dmitry Volyntsev date: Fri Dec 06 14:44:54 2019 +0300 description: Introduced ToIndex() conversion primitive from the spec. diffstat: src/njs_main.h | 5 +- src/njs_number.c | 4 +- src/njs_number.h | 2 +- src/njs_value.c | 6 +- src/njs_value.h | 167 ------------------------------------- src/njs_value_conversion.h | 202 +++++++++++++++++++++++++++++++++++++++++++++ src/njs_vmcode.c | 2 +- 7 files changed, 212 insertions(+), 176 deletions(-) diffs (486 lines): diff -r 990a4b7a4053 -r 19b741311270 src/njs_main.h --- a/src/njs_main.h Fri Dec 06 14:44:11 2019 +0300 +++ b/src/njs_main.h Fri Dec 06 14:44:54 2019 +0300 @@ -44,6 +44,9 @@ #include #include +#include +#include +#include #include #include #include @@ -51,14 +54,12 @@ #include #include -#include #include #include #include #include #include #include -#include #include #include #include diff -r 990a4b7a4053 -r 19b741311270 src/njs_number.c --- a/src/njs_number.c Fri Dec 06 14:44:11 2019 +0300 +++ b/src/njs_number.c Fri Dec 06 14:44:54 2019 +0300 @@ -21,7 +21,7 @@ static njs_int_t njs_number_to_string_ra uint32_t -njs_value_to_index(const njs_value_t *value) +njs_key_to_index(const njs_value_t *value) { double num; njs_array_t *array; @@ -47,7 +47,7 @@ njs_value_to_index(const njs_value_t *va if (array->length == 1 && njs_is_valid(&array->start[0])) { /* A single value array is the zeroth array value. */ - return njs_value_to_index(&array->start[0]); + return njs_key_to_index(&array->start[0]); } } } diff -r 990a4b7a4053 -r 19b741311270 src/njs_number.h --- a/src/njs_number.h Fri Dec 06 14:44:11 2019 +0300 +++ b/src/njs_number.h Fri Dec 06 14:44:54 2019 +0300 @@ -8,7 +8,7 @@ #define _NJS_NUMBER_H_INCLUDED_ -uint32_t njs_value_to_index(const njs_value_t *value); +uint32_t njs_key_to_index(const njs_value_t *value); double njs_number_dec_parse(const u_char **start, const u_char *end); uint64_t njs_number_oct_parse(const u_char **start, const u_char *end); uint64_t njs_number_bin_parse(const u_char **start, const u_char *end); diff -r 990a4b7a4053 -r 19b741311270 src/njs_value.c --- a/src/njs_value.c Fri Dec 06 14:44:11 2019 +0300 +++ b/src/njs_value.c Fri Dec 06 14:44:54 2019 +0300 @@ -535,7 +535,7 @@ njs_property_query(njs_vm_t *vm, njs_pro case NJS_STRING: if (njs_fast_path(!njs_is_null_or_undefined_or_boolean(key))) { - index = njs_value_to_index(key); + index = njs_key_to_index(key); if (njs_fast_path(index < NJS_STRING_MAX_LENGTH)) { return njs_string_property_query(vm, pq, value, index); @@ -637,7 +637,7 @@ njs_object_property_query(njs_vm_t *vm, if (!njs_is_null_or_undefined_or_boolean(key)) { switch (proto->type) { case NJS_ARRAY: - index = njs_value_to_index(key); + index = njs_key_to_index(key); if (njs_fast_path(index < NJS_ARRAY_MAX_INDEX)) { array = (njs_array_t *) proto; return njs_array_property_query(vm, pq, array, index); @@ -646,7 +646,7 @@ njs_object_property_query(njs_vm_t *vm, break; case NJS_OBJECT_STRING: - index = njs_value_to_index(key); + index = njs_key_to_index(key); if (njs_fast_path(index < NJS_STRING_MAX_LENGTH)) { ov = (njs_object_value_t *) proto; ret = njs_string_property_query(vm, pq, &ov->value, index); diff -r 990a4b7a4053 -r 19b741311270 src/njs_value.h --- a/src/njs_value.h Fri Dec 06 14:44:11 2019 +0300 +++ b/src/njs_value.h Fri Dec 06 14:44:54 2019 +0300 @@ -913,173 +913,6 @@ njs_int_t njs_value_species_constructor( njs_value_t *default_constructor, njs_value_t *dst); -#include "njs_number.h" - - -njs_inline njs_int_t -njs_value_to_number(njs_vm_t *vm, njs_value_t *value, double *dst) -{ - njs_int_t ret; - njs_value_t primitive; - - if (njs_slow_path(!njs_is_primitive(value))) { - ret = njs_value_to_primitive(vm, &primitive, value, 0); - if (njs_slow_path(ret != NJS_OK)) { - return ret; - } - - value = &primitive; - } - - if (njs_slow_path(!njs_is_numeric(value))) { - - if (njs_slow_path(njs_is_symbol(value))) { - njs_symbol_conversion_failed(vm, 0); - return NJS_ERROR; - } - - *dst = NAN; - - if (njs_is_string(value)) { - *dst = njs_string_to_number(value, 0); - } - - return NJS_OK; - } - - *dst = njs_number(value); - - return NJS_OK; -} - - -njs_inline njs_int_t -njs_value_to_numeric(njs_vm_t *vm, njs_value_t *value, njs_value_t *dst) -{ - double num; - njs_int_t ret; - - ret = njs_value_to_number(vm, value, &num); - if (njs_slow_path(ret != NJS_OK)) { - return ret; - } - - njs_set_number(dst, num); - - return NJS_OK; -} - - -njs_inline njs_int_t -njs_value_to_integer(njs_vm_t *vm, njs_value_t *value, int64_t *dst) -{ - double num; - njs_int_t ret; - - ret = njs_value_to_number(vm, value, &num); - if (njs_slow_path(ret != NJS_OK)) { - return ret; - } - - *dst = njs_number_to_integer(num); - - return NJS_OK; -} - - -njs_inline njs_int_t -njs_value_to_length(njs_vm_t *vm, njs_value_t *value, uint32_t *dst) -{ - double num; - njs_int_t ret; - - ret = njs_value_to_number(vm, value, &num); - if (njs_slow_path(ret != NJS_OK)) { - return ret; - } - - *dst = njs_number_to_length(num); - - return NJS_OK; -} - - -njs_inline njs_int_t -njs_value_to_int32(njs_vm_t *vm, njs_value_t *value, int32_t *dst) -{ - double num; - njs_int_t ret; - - ret = njs_value_to_number(vm, value, &num); - if (njs_slow_path(ret != NJS_OK)) { - return ret; - } - - *dst = njs_number_to_int32(num); - - return NJS_OK; -} - - -njs_inline njs_int_t -njs_value_to_uint32(njs_vm_t *vm, njs_value_t *value, uint32_t *dst) -{ - double num; - njs_int_t ret; - - ret = njs_value_to_number(vm, value, &num); - if (njs_slow_path(ret != NJS_OK)) { - return ret; - } - - *dst = njs_number_to_uint32(num); - - return NJS_OK; -} - - -njs_inline njs_int_t -njs_value_to_uint16(njs_vm_t *vm, njs_value_t *value, uint16_t *dst) -{ - double num; - njs_int_t ret; - - ret = njs_value_to_number(vm, value, &num); - if (njs_slow_path(ret != NJS_OK)) { - return ret; - } - - *dst = njs_number_to_uint16(num); - - return NJS_OK; -} - - -njs_inline njs_int_t -njs_value_to_string(njs_vm_t *vm, njs_value_t *dst, njs_value_t *value) -{ - njs_int_t ret; - njs_value_t primitive; - - if (njs_slow_path(!njs_is_primitive(value))) { - if (njs_slow_path(value->type == NJS_OBJECT_SYMBOL)) { - /* should fail */ - value = njs_object_value(value); - - } else { - ret = njs_value_to_primitive(vm, &primitive, value, 1); - if (njs_slow_path(ret != NJS_OK)) { - return ret; - } - - value = &primitive; - } - } - - return njs_primitive_value_to_string(vm, dst, value); -} - - njs_inline njs_bool_t njs_values_same_non_numeric(const njs_value_t *val1, const njs_value_t *val2) { diff -r 990a4b7a4053 -r 19b741311270 src/njs_value_conversion.h --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/njs_value_conversion.h Fri Dec 06 14:44:54 2019 +0300 @@ -0,0 +1,202 @@ + +/* + * Copyright (C) Igor Sysoev + * Copyright (C) NGINX, Inc. + */ + +#ifndef _NJS_VALUE_CONVERSION_H_INCLUDED_ +#define _NJS_VALUE_CONVERSION_H_INCLUDED_ + + +njs_inline njs_int_t +njs_value_to_number(njs_vm_t *vm, njs_value_t *value, double *dst) +{ + njs_int_t ret; + njs_value_t primitive; + + if (njs_slow_path(!njs_is_primitive(value))) { + ret = njs_value_to_primitive(vm, &primitive, value, 0); + if (njs_slow_path(ret != NJS_OK)) { + return ret; + } + + value = &primitive; + } + + if (njs_slow_path(!njs_is_numeric(value))) { + + if (njs_slow_path(njs_is_symbol(value))) { + njs_symbol_conversion_failed(vm, 0); + return NJS_ERROR; + } + + *dst = NAN; + + if (njs_is_string(value)) { + *dst = njs_string_to_number(value, 0); + } + + return NJS_OK; + } + + *dst = njs_number(value); + + return NJS_OK; +} + + +njs_inline njs_int_t +njs_value_to_numeric(njs_vm_t *vm, njs_value_t *value, njs_value_t *dst) +{ + double num; + njs_int_t ret; + + ret = njs_value_to_number(vm, value, &num); + if (njs_slow_path(ret != NJS_OK)) { + return ret; + } + + njs_set_number(dst, num); + + return NJS_OK; +} + + +njs_inline njs_int_t +njs_value_to_integer(njs_vm_t *vm, njs_value_t *value, int64_t *dst) +{ + double num; + njs_int_t ret; + + ret = njs_value_to_number(vm, value, &num); + if (njs_slow_path(ret != NJS_OK)) { + return ret; + } + + *dst = njs_number_to_integer(num); + + return NJS_OK; +} + + +njs_inline njs_int_t +njs_value_to_length(njs_vm_t *vm, njs_value_t *value, uint32_t *dst) +{ + double num; + njs_int_t ret; + + ret = njs_value_to_number(vm, value, &num); + if (njs_slow_path(ret != NJS_OK)) { + return ret; + } + + *dst = njs_number_to_length(num); + + return NJS_OK; +} + + +njs_inline njs_int_t +njs_value_to_index(njs_vm_t *vm, njs_value_t *value, uint32_t *dst) +{ + int64_t integer_index; + njs_int_t ret; + + if (njs_slow_path(njs_is_undefined(value))) { + *dst = 0; + + } else { + ret = njs_value_to_integer(vm, value, &integer_index); + if (njs_slow_path(ret != NJS_OK)) { + return ret; + } + + if (integer_index < 0 || integer_index > UINT32_MAX) { + njs_range_error(vm, "invalid index"); + return NJS_ERROR; + } + + *dst = (uint32_t) integer_index; + } + + return NJS_OK; +} + + +njs_inline njs_int_t +njs_value_to_int32(njs_vm_t *vm, njs_value_t *value, int32_t *dst) +{ + double num; + njs_int_t ret; + + ret = njs_value_to_number(vm, value, &num); + if (njs_slow_path(ret != NJS_OK)) { + return ret; + } + + *dst = njs_number_to_int32(num); + + return NJS_OK; +} + + +njs_inline njs_int_t +njs_value_to_uint32(njs_vm_t *vm, njs_value_t *value, uint32_t *dst) +{ + double num; + njs_int_t ret; + + ret = njs_value_to_number(vm, value, &num); + if (njs_slow_path(ret != NJS_OK)) { + return ret; + } + + *dst = njs_number_to_uint32(num); + + return NJS_OK; +} + + +njs_inline njs_int_t +njs_value_to_uint16(njs_vm_t *vm, njs_value_t *value, uint16_t *dst) +{ + double num; + njs_int_t ret; + + ret = njs_value_to_number(vm, value, &num); + if (njs_slow_path(ret != NJS_OK)) { + return ret; + } + + *dst = njs_number_to_uint16(num); + + return NJS_OK; +} + + +njs_inline njs_int_t +njs_value_to_string(njs_vm_t *vm, njs_value_t *dst, njs_value_t *value) +{ + njs_int_t ret; + njs_value_t primitive; + + if (njs_slow_path(!njs_is_primitive(value))) { + if (njs_slow_path(value->type == NJS_OBJECT_SYMBOL)) { + /* should fail */ + value = njs_object_value(value); + + } else { + ret = njs_value_to_primitive(vm, &primitive, value, 1); + if (njs_slow_path(ret != NJS_OK)) { + return ret; + } + + value = &primitive; + } + } + + return njs_primitive_value_to_string(vm, dst, value); +} + + +#endif /* _NJS_VALUE_CONVERSION_H_INCLUDED_ */ diff -r 990a4b7a4053 -r 19b741311270 src/njs_vmcode.c --- a/src/njs_vmcode.c Fri Dec 06 14:44:11 2019 +0300 +++ b/src/njs_vmcode.c Fri Dec 06 14:44:54 2019 +0300 @@ -1146,7 +1146,7 @@ njs_vmcode_property_init(njs_vm_t *vm, n switch (value->type) { case NJS_ARRAY: - index = njs_value_to_index(key); + index = njs_key_to_index(key); if (njs_slow_path(index == NJS_ARRAY_INVALID_INDEX)) { njs_internal_error(vm, "invalid index while property initialization"); From xeioex at nginx.com Fri Dec 6 12:17:15 2019 From: xeioex at nginx.com (Dmitry Volyntsev) Date: Fri, 06 Dec 2019 12:17:15 +0000 Subject: [njs] Improved object type initialization structure. Message-ID: details: https://hg.nginx.org/njs/rev/242395b814bb branches: changeset: 1280:242395b814bb user: Dmitry Volyntsev date: Fri Dec 06 14:59:48 2019 +0300 description: Improved object type initialization structure. Allowing predefine magic argument for a type constructor. This allows to write generic constructor functions. diffstat: src/njs_array.c | 6 ++-- src/njs_boolean.c | 10 +++++--- src/njs_builtin.c | 10 +------- src/njs_crypto.c | 16 +++++++------- src/njs_date.c | 6 ++-- src/njs_error.c | 54 ++++++++++++++++++++++++++-------------------------- src/njs_function.c | 30 ++++++++++++++-------------- src/njs_function.h | 8 ++---- src/njs_number.c | 10 +++++--- src/njs_object.c | 6 ++-- src/njs_regexp.c | 6 ++-- src/njs_string.c | 10 +++++--- src/njs_symbol.c | 6 ++-- src/njs_value.h | 55 +++++++++++++++++++++++++++++++++-------------------- src/njs_vm.c | 4 ++- src/njs_vmcode.c | 3 +- 16 files changed, 127 insertions(+), 113 deletions(-) diffs (554 lines): diff -r 19b741311270 -r 242395b814bb src/njs_array.c --- a/src/njs_array.c Fri Dec 06 14:44:54 2019 +0300 +++ b/src/njs_array.c Fri Dec 06 14:59:48 2019 +0300 @@ -3103,8 +3103,8 @@ const njs_object_init_t njs_array_insta const njs_object_type_init_t njs_array_type_init = { - .constructor = njs_array_constructor, - .prototype_props = &njs_array_prototype_init, + .constructor = njs_native_ctor(njs_array_constructor, 1, 0), .constructor_props = &njs_array_constructor_init, - .value = { .object = { .type = NJS_ARRAY } }, + .prototype_props = &njs_array_prototype_init, + .prototype_value = { .object = { .type = NJS_ARRAY } }, }; diff -r 19b741311270 -r 242395b814bb src/njs_boolean.c --- a/src/njs_boolean.c Fri Dec 06 14:44:54 2019 +0300 +++ b/src/njs_boolean.c Fri Dec 06 14:59:48 2019 +0300 @@ -166,9 +166,11 @@ const njs_object_init_t njs_boolean_pro const njs_object_type_init_t njs_boolean_type_init = { - .constructor = njs_boolean_constructor, - .prototype_props = &njs_boolean_prototype_init, + .constructor = njs_native_ctor(njs_boolean_constructor, 1, 0), .constructor_props = &njs_boolean_constructor_init, - .value = { .object_value = { .value = njs_value(NJS_BOOLEAN, 0, 0.0), - .object = { .type = NJS_OBJECT_BOOLEAN } } }, + .prototype_props = &njs_boolean_prototype_init, + .prototype_value = { .object_value = { + .value = njs_value(NJS_BOOLEAN, 0, 0.0), + .object = { .type = NJS_OBJECT_BOOLEAN } } + }, }; diff -r 19b741311270 -r 242395b814bb src/njs_builtin.c --- a/src/njs_builtin.c Fri Dec 06 14:44:54 2019 +0300 +++ b/src/njs_builtin.c Fri Dec 06 14:59:48 2019 +0300 @@ -253,7 +253,7 @@ njs_builtin_objects_create(njs_vm_t *vm) prototype = shared->prototypes; for (i = NJS_OBJ_TYPE_OBJECT; i < NJS_OBJ_TYPE_MAX; i++) { - prototype[i] = njs_object_type_init[i]->value; + prototype[i] = njs_object_type_init[i]->prototype_value; ret = njs_object_hash_init(vm, &prototype[i].object.shared_hash, njs_object_type_init[i]->prototype_props); @@ -270,14 +270,8 @@ njs_builtin_objects_create(njs_vm_t *vm) constructor = shared->constructors; for (i = NJS_OBJ_TYPE_OBJECT; i < NJS_OBJ_TYPE_MAX; i++) { - constructor[i].object.type = NJS_FUNCTION; + constructor[i] = njs_object_type_init[i]->constructor; constructor[i].object.shared = 0; - constructor[i].object.extensible = 1; - constructor[i].native = 1; - constructor[i].ctor = 1; - constructor[i].args_offset = 1; - - constructor[i].u.native = njs_object_type_init[i]->constructor; ret = njs_object_hash_init(vm, &constructor[i].object.shared_hash, njs_object_type_init[i]->constructor_props); diff -r 19b741311270 -r 242395b814bb src/njs_crypto.c --- a/src/njs_crypto.c Fri Dec 06 14:44:54 2019 +0300 +++ b/src/njs_crypto.c Fri Dec 06 14:59:48 2019 +0300 @@ -386,11 +386,11 @@ const njs_object_init_t njs_hash_constr const njs_object_type_init_t njs_hash_type_init = { - .constructor = njs_hash_constructor, - .prototype_props = &njs_hash_prototype_init, + .constructor = njs_native_ctor(njs_hash_constructor, 2, 0), .constructor_props = &njs_hash_constructor_init, - .value = { .object_value = { .value = njs_value(NJS_DATA, 0, 0.0), - .object = { .type = NJS_OBJECT } } }, + .prototype_props = &njs_hash_prototype_init, + .prototype_value = { .object_value = { .value = njs_value(NJS_DATA, 0, 0.0), + .object = { .type = NJS_OBJECT } } }, }; @@ -712,11 +712,11 @@ const njs_object_init_t njs_crypto_obje const njs_object_type_init_t njs_hmac_type_init = { - .constructor = njs_hmac_constructor, - .prototype_props = &njs_hmac_prototype_init, + .constructor = njs_native_ctor(njs_hmac_constructor, 3, 0), .constructor_props = &njs_hmac_constructor_init, - .value = { .object_value = { .value = njs_value(NJS_DATA, 0, 0.0), - .object = { .type = NJS_OBJECT } } }, + .prototype_props = &njs_hmac_prototype_init, + .prototype_value = { .object_value = { .value = njs_value(NJS_DATA, 0, 0.0), + .object = { .type = NJS_OBJECT } } }, }; diff -r 19b741311270 -r 242395b814bb src/njs_date.c --- a/src/njs_date.c Fri Dec 06 14:44:54 2019 +0300 +++ b/src/njs_date.c Fri Dec 06 14:59:48 2019 +0300 @@ -1835,8 +1835,8 @@ const njs_object_init_t njs_date_protot const njs_object_type_init_t njs_date_type_init = { - .constructor = njs_date_constructor, - .prototype_props = &njs_date_prototype_init, + .constructor = njs_native_ctor(njs_date_constructor, 7, 0), .constructor_props = &njs_date_constructor_init, - .value = { .object = { .type = NJS_OBJECT } }, + .prototype_props = &njs_date_prototype_init, + .prototype_value = { .object = { .type = NJS_OBJECT } }, }; diff -r 19b741311270 -r 242395b814bb src/njs_error.c --- a/src/njs_error.c Fri Dec 06 14:44:54 2019 +0300 +++ b/src/njs_error.c Fri Dec 06 14:59:48 2019 +0300 @@ -899,10 +899,10 @@ const njs_object_init_t njs_error_proto const njs_object_type_init_t njs_error_type_init = { - .constructor = njs_error_constructor, + .constructor = njs_native_ctor(njs_error_constructor, 1, 0), + .constructor_props = &njs_error_constructor_init, .prototype_props = &njs_error_prototype_init, - .constructor_props = &njs_error_constructor_init, - .value = { .object = { .type = NJS_OBJECT } }, + .prototype_value = { .object = { .type = NJS_OBJECT } }, }; @@ -941,10 +941,10 @@ const njs_object_init_t njs_eval_error_ const njs_object_type_init_t njs_eval_error_type_init = { - .constructor = njs_eval_error_constructor, + .constructor = njs_native_ctor(njs_eval_error_constructor, 1, 0), + .constructor_props = &njs_eval_error_constructor_init, .prototype_props = &njs_eval_error_prototype_init, - .constructor_props = &njs_eval_error_constructor_init, - .value = { .object = { .type = NJS_OBJECT } }, + .prototype_value = { .object = { .type = NJS_OBJECT } }, }; @@ -1003,18 +1003,18 @@ const njs_object_init_t njs_internal_er const njs_object_type_init_t njs_internal_error_type_init = { - .constructor = njs_internal_error_constructor, + .constructor = njs_native_ctor(njs_internal_error_constructor, 1, 0), + .constructor_props = &njs_internal_error_constructor_init, .prototype_props = &njs_internal_error_prototype_init, - .constructor_props = &njs_internal_error_constructor_init, - .value = { .object = { .type = NJS_OBJECT } }, + .prototype_value = { .object = { .type = NJS_OBJECT } }, }; const njs_object_type_init_t njs_memory_error_type_init = { - .constructor = njs_memory_error_constructor, + .constructor = njs_native_ctor(njs_memory_error_constructor, 1, 0), + .constructor_props = &njs_memory_error_constructor_init, .prototype_props = &njs_internal_error_prototype_init, - .constructor_props = &njs_memory_error_constructor_init, - .value = { .object = { .type = NJS_OBJECT } }, + .prototype_value = { .object = { .type = NJS_OBJECT } }, }; @@ -1053,10 +1053,10 @@ const njs_object_init_t njs_range_error const njs_object_type_init_t njs_range_error_type_init = { - .constructor = njs_range_error_constructor, + .constructor = njs_native_ctor(njs_range_error_constructor, 1, 0), + .constructor_props = &njs_range_error_constructor_init, .prototype_props = &njs_range_error_prototype_init, - .constructor_props = &njs_range_error_constructor_init, - .value = { .object = { .type = NJS_OBJECT } }, + .prototype_value = { .object = { .type = NJS_OBJECT } }, }; @@ -1095,10 +1095,10 @@ const njs_object_init_t njs_reference_e const njs_object_type_init_t njs_reference_error_type_init = { - .constructor = njs_reference_error_constructor, + .constructor = njs_native_ctor(njs_reference_error_constructor, 1, 0), + .constructor_props = &njs_reference_error_constructor_init, .prototype_props = &njs_reference_error_prototype_init, - .constructor_props = &njs_reference_error_constructor_init, - .value = { .object = { .type = NJS_OBJECT } }, + .prototype_value = { .object = { .type = NJS_OBJECT } }, }; @@ -1137,10 +1137,10 @@ const njs_object_init_t njs_syntax_erro const njs_object_type_init_t njs_syntax_error_type_init = { - .constructor = njs_syntax_error_constructor, + .constructor = njs_native_ctor(njs_syntax_error_constructor, 1, 0), + .constructor_props = &njs_syntax_error_constructor_init, .prototype_props = &njs_syntax_error_prototype_init, - .constructor_props = &njs_syntax_error_constructor_init, - .value = { .object = { .type = NJS_OBJECT } }, + .prototype_value = { .object = { .type = NJS_OBJECT } }, }; @@ -1179,10 +1179,10 @@ const njs_object_init_t njs_type_error_ const njs_object_type_init_t njs_type_error_type_init = { - .constructor = njs_type_error_constructor, + .constructor = njs_native_ctor(njs_type_error_constructor, 1, 0), + .constructor_props = &njs_type_error_constructor_init, .prototype_props = &njs_type_error_prototype_init, - .constructor_props = &njs_type_error_constructor_init, - .value = { .object = { .type = NJS_OBJECT } }, + .prototype_value = { .object = { .type = NJS_OBJECT } }, }; @@ -1221,8 +1221,8 @@ const njs_object_init_t njs_uri_error_p const njs_object_type_init_t njs_uri_error_type_init = { - .constructor = njs_uri_error_constructor, - .prototype_props = &njs_uri_error_prototype_init, + .constructor = njs_native_ctor(njs_uri_error_constructor, 1, 0), .constructor_props = &njs_uri_error_constructor_init, - .value = { .object = { .type = NJS_OBJECT } }, + .prototype_props = &njs_uri_error_prototype_init, + .prototype_value = { .object = { .type = NJS_OBJECT } }, }; diff -r 19b741311270 -r 242395b814bb src/njs_function.c --- a/src/njs_function.c Fri Dec 06 14:44:54 2019 +0300 +++ b/src/njs_function.c Fri Dec 06 14:59:48 2019 +0300 @@ -58,7 +58,7 @@ njs_function_alloc(njs_vm_t *vm, njs_fun do { /* GC: retain closure. */ - function->closures[n] = closures[n]; + njs_function_closures(function)[n] = closures[n]; n++; } while (n < nesting); } @@ -104,10 +104,10 @@ njs_function_value_copy(njs_vm_t *vm, nj njs_inline njs_closure_t ** -njs_function_closures(njs_vm_t *vm, njs_function_t *function) +njs_function_active_closures(njs_vm_t *vm, njs_function_t *function) { - return (function->closure) ? function->closures - : vm->active_frame->closures; + return (function->closure) ? njs_function_closures(function) + : njs_frame_closures(vm->active_frame); } @@ -185,13 +185,13 @@ njs_function_copy(njs_vm_t *vm, njs_func copy->closure = 1; - closures = njs_function_closures(vm, function); + closures = njs_function_active_closures(vm, function); n = 0; do { /* GC: retain closure. */ - copy->closures[n] = closures[n]; + njs_function_closures(copy)[n] = closures[n]; n++; } while (n < nesting); @@ -596,11 +596,11 @@ njs_function_lambda_call(njs_vm_t *vm) nesting = lambda->nesting; if (nesting != 0) { - closures = njs_function_closures(vm, function); + closures = njs_function_active_closures(vm, function); do { closure = *closures++; - frame->closures[n] = closure; + njs_frame_closures(frame)[n] = closure; vm->scopes[NJS_SCOPE_CLOSURE + n] = &closure->u.values; n++; @@ -633,7 +633,7 @@ njs_function_lambda_call(njs_vm_t *vm) } while (size != 0); } - frame->closures[n] = closure; + njs_frame_closures(frame)[n] = closure; vm->scopes[NJS_SCOPE_CLOSURE + n] = &closure->u.values; } @@ -1351,11 +1351,11 @@ njs_prototype_function(njs_vm_t *vm, njs const njs_object_type_init_t njs_function_type_init = { - .constructor = njs_function_constructor, - .prototype_props = &njs_function_prototype_init, + .constructor = njs_native_ctor(njs_function_constructor, 1, 0), .constructor_props = &njs_function_constructor_init, - .value = { .function = { .native = 1, - .args_offset = 1, - .u.native = njs_prototype_function, - .object = { .type = NJS_FUNCTION } } }, + .prototype_props = &njs_function_prototype_init, + .prototype_value = { .function = { .native = 1, + .args_offset = 1, + .u.native = njs_prototype_function, + .object = { .type = NJS_FUNCTION } } }, }; diff -r 19b741311270 -r 242395b814bb src/njs_function.h --- a/src/njs_function.h Fri Dec 06 14:44:54 2019 +0300 +++ b/src/njs_function.h Fri Dec 06 14:59:48 2019 +0300 @@ -90,11 +90,9 @@ struct njs_frame_s { njs_frame_t *previous_active_frame; njs_value_t *local; -#if (NJS_SUNC) - njs_closure_t *closures[1]; -#else - njs_closure_t *closures[]; -#endif + +#define njs_frame_closures(frame) \ + ((njs_closure_t **) ((u_char *) frame + sizeof(njs_frame_t))) }; diff -r 19b741311270 -r 242395b814bb src/njs_number.c --- a/src/njs_number.c Fri Dec 06 14:44:54 2019 +0300 +++ b/src/njs_number.c Fri Dec 06 14:59:48 2019 +0300 @@ -1123,9 +1123,11 @@ njs_number_parse_float(njs_vm_t *vm, njs const njs_object_type_init_t njs_number_type_init = { - .constructor = njs_number_constructor, - .prototype_props = &njs_number_prototype_init, + .constructor = njs_native_ctor(njs_number_constructor, 1, 0), .constructor_props = &njs_number_constructor_init, - .value = { .object_value = { .value = njs_value(NJS_NUMBER, 0, 0.0), - .object = { .type = NJS_OBJECT_NUMBER } } }, + .prototype_props = &njs_number_prototype_init, + .prototype_value = { .object_value = { + .value = njs_value(NJS_NUMBER, 0, 0.0), + .object = { .type = NJS_OBJECT_NUMBER } } + }, }; diff -r 19b741311270 -r 242395b814bb src/njs_object.c --- a/src/njs_object.c Fri Dec 06 14:44:54 2019 +0300 +++ b/src/njs_object.c Fri Dec 06 14:59:48 2019 +0300 @@ -2670,8 +2670,8 @@ njs_object_length(njs_vm_t *vm, njs_valu const njs_object_type_init_t njs_obj_type_init = { - .constructor = njs_object_constructor, - .prototype_props = &njs_object_prototype_init, + .constructor = njs_native_ctor(njs_object_constructor, 1, 0), .constructor_props = &njs_object_constructor_init, - .value = { .object = { .type = NJS_OBJECT } }, + .prototype_props = &njs_object_prototype_init, + .prototype_value = { .object = { .type = NJS_OBJECT } }, }; diff -r 19b741311270 -r 242395b814bb src/njs_regexp.c --- a/src/njs_regexp.c Fri Dec 06 14:44:54 2019 +0300 +++ b/src/njs_regexp.c Fri Dec 06 14:59:48 2019 +0300 @@ -1341,8 +1341,8 @@ const njs_object_init_t njs_regexp_prot const njs_object_type_init_t njs_regexp_type_init = { - .constructor = njs_regexp_constructor, - .prototype_props = &njs_regexp_prototype_init, + .constructor = njs_native_ctor(njs_regexp_constructor, 2, 0), .constructor_props = &njs_regexp_constructor_init, - .value = { .object = { .type = NJS_REGEXP } }, + .prototype_props = &njs_regexp_prototype_init, + .prototype_value = { .object = { .type = NJS_REGEXP } }, }; diff -r 19b741311270 -r 242395b814bb src/njs_string.c --- a/src/njs_string.c Fri Dec 06 14:44:54 2019 +0300 +++ b/src/njs_string.c Fri Dec 06 14:59:48 2019 +0300 @@ -5276,9 +5276,11 @@ njs_value_index(njs_vm_t *vm, const njs_ const njs_object_type_init_t njs_string_type_init = { - .constructor = njs_string_constructor, - .prototype_props = &njs_string_prototype_init, + .constructor = njs_native_ctor(njs_string_constructor, 1, 0), .constructor_props = &njs_string_constructor_init, - .value = { .object_value = { .value = njs_string(""), - .object = { .type = NJS_OBJECT_STRING } } }, + .prototype_props = &njs_string_prototype_init, + .prototype_value = { .object_value = { + .value = njs_string(""), + .object = { .type = NJS_OBJECT_STRING } } + }, }; diff -r 19b741311270 -r 242395b814bb src/njs_symbol.c --- a/src/njs_symbol.c Fri Dec 06 14:44:54 2019 +0300 +++ b/src/njs_symbol.c Fri Dec 06 14:59:48 2019 +0300 @@ -441,8 +441,8 @@ const njs_object_init_t njs_symbol_prot const njs_object_type_init_t njs_symbol_type_init = { - .constructor = njs_symbol_constructor, - .prototype_props = &njs_symbol_prototype_init, + .constructor = njs_native_ctor(njs_symbol_constructor, 0, 0), .constructor_props = &njs_symbol_constructor_init, - .value = { .object = { .type = NJS_OBJECT } }, + .prototype_props = &njs_symbol_prototype_init, + .prototype_value = { .object = { .type = NJS_OBJECT } }, }; diff -r 19b741311270 -r 242395b814bb src/njs_value.h --- a/src/njs_value.h Fri Dec 06 14:44:54 2019 +0300 +++ b/src/njs_value.h Fri Dec 06 14:59:48 2019 +0300 @@ -239,6 +239,15 @@ struct njs_function_s { uint8_t args_offset; uint8_t args_count:5; + + /* + * If "closure" is true njs_closure_t[] is available right after the + * njs_function_t and njs_function_closures() may be used to access it. + */ + +#define njs_function_closures(function) \ + ((njs_closure_t **) ((u_char *) function + sizeof(njs_function_t))) + uint8_t closure:1; uint8_t native:1; uint8_t ctor:1; @@ -252,11 +261,6 @@ struct njs_function_s { } u; njs_value_t *bound; -#if (NJS_SUNC) - njs_closure_t *closures[1]; -#else - njs_closure_t *closures[]; -#endif }; @@ -289,10 +293,10 @@ typedef union { typedef struct { - njs_function_native_t constructor; + njs_function_t constructor; + const njs_object_init_t *constructor_props; const njs_object_init_t *prototype_props; - const njs_object_init_t *constructor_props; - njs_object_prototype_t value; + njs_object_prototype_t prototype_value; } njs_object_type_init_t; @@ -411,30 +415,39 @@ typedef struct { } -#define _njs_native_function(_function, _args_count, _magic) { \ +#define _njs_function(_function, _args_count, _ctor, _magic) { \ + .native = 1, \ + .magic = _magic, \ + .args_count = _args_count, \ + .ctor = _ctor, \ + .args_offset = 1, \ + .u.native = _function, \ + .object = { .type = NJS_FUNCTION, \ + .shared = 1, \ + .extensible = 1 }, \ +} + + +#define _njs_native_function(_func, _args, _ctor, _magic) { \ .data = { \ .type = NJS_FUNCTION, \ .truth = 1, \ - .u.function = & (njs_function_t) { \ - .native = 1, \ - .magic = _magic, \ - .args_count = _args_count, \ - .args_offset = 1, \ - .u.native = _function, \ - .object = { .type = NJS_FUNCTION, \ - .shared = 1, \ - .extensible = 1 }, \ - } \ + .u.function = & (njs_function_t) _njs_function(_func, _args, \ + _ctor, _magic) \ } \ } #define njs_native_function(_function, _args_count) \ - _njs_native_function(_function, _args_count, 0) + _njs_native_function(_function, _args_count, 0, 0) #define njs_native_function2(_function, _args_count, _magic) \ - _njs_native_function(_function, _args_count, _magic) + _njs_native_function(_function, _args_count, 0, _magic) + + +#define njs_native_ctor(_function, _args_count, _magic) \ + _njs_function(_function, _args_count, 1, _magic) #define njs_prop_handler(_handler) { \ diff -r 19b741311270 -r 242395b814bb src/njs_vm.c --- a/src/njs_vm.c Fri Dec 06 14:44:54 2019 +0300 +++ b/src/njs_vm.c Fri Dec 06 14:59:48 2019 +0300 @@ -339,6 +339,7 @@ njs_vm_scopes_restore(njs_vm_t *vm, njs_ { njs_uint_t n, nesting; njs_value_t *args; + njs_closure_t **closures; njs_function_t *function; vm->top_frame = previous; @@ -373,9 +374,10 @@ njs_vm_scopes_restore(njs_vm_t *vm, njs_ function = frame->native.function; nesting = (function != NULL) ? function->u.lambda->nesting : 0; + closures = njs_frame_closures(frame); for (n = 0; n <= nesting; n++) { - vm->scopes[NJS_SCOPE_CLOSURE + n] = &frame->closures[n]->u.values; + vm->scopes[NJS_SCOPE_CLOSURE + n] = &closures[n]->u.values; } while (n < NJS_MAX_NESTING) { diff -r 19b741311270 -r 242395b814bb src/njs_vmcode.c --- a/src/njs_vmcode.c Fri Dec 06 14:44:54 2019 +0300 +++ b/src/njs_vmcode.c Fri Dec 06 14:59:48 2019 +0300 @@ -1002,7 +1002,8 @@ njs_vmcode_function(njs_vm_t *vm, u_char code = (njs_vmcode_function_t *) pc; lambda = code->lambda; - function = njs_function_alloc(vm, lambda, vm->active_frame->closures, 0); + function = njs_function_alloc(vm, lambda, + njs_frame_closures(vm->active_frame), 0); if (njs_slow_path(function == NULL)) { return NJS_ERROR; } From osa at FreeBSD.org.ru Sat Dec 7 19:12:12 2019 From: osa at FreeBSD.org.ru (Sergey A. Osokin) Date: Sat, 07 Dec 2019 22:12:12 +0300 Subject: [PATCH] Add nginx.conf(5) Message-ID: <4418dfeac658f1cf5ea7.1575745932@mp2.macomnet.net> # HG changeset patch # User Sergey A. Osokin # Date 1575745926 -10800 # Sat Dec 07 22:12:06 2019 +0300 # Node ID 4418dfeac658f1cf5ea7334b1bdf3084e2b1c7cd # Parent 3939483cd1b56b202a99b1db344d9568b0958e67 Add nginx.conf(5). diff -r 3939483cd1b5 -r 4418dfeac658 docs/man/nginx.conf.5 --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/docs/man/nginx.conf.5 Sat Dec 07 22:12:06 2019 +0300 @@ -0,0 +1,112 @@ +.\" +.\" Copyright (C) 2019 Sergey A. Osokin +.\" Copyright (C) Nginx, Inc. +.\" All rights reserved. +.\" +.\" Redistribution and use in source and binary forms, with or without +.\" modification, are permitted provided that the following conditions +.\" are met: +.\" 1. Redistributions of source code must retain the above copyright +.\" notice, this list of conditions and the following disclaimer. +.\" 2. Redistributions in binary form must reproduce the above copyright +.\" notice, this list of conditions and the following disclaimer in the +.\" documentation and/or other materials provided with the distribution. +.\" +.\" THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND +.\" ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE +.\" IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE +.\" ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE +.\" FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL +.\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS +.\" OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) +.\" HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT +.\" LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY +.\" OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF +.\" SUCH DAMAGE. +.\" +.\" +.Dd December 7, 2019 +.Dt NGINX.CONF 5 +.Os +.Sh NAME +.Nm nginx.conf +.Nd "configuration file used by the" +.Xr nginx 8 +.Sh DESCRIPTION +.Nm +is the configuration file used by the +.Xr nginx 8 . +.Pp +The default location of this file is +.Pa %%CONF_PATH%% . +.Pp +.Xr nginx 8 +consists of modules which are controlled by +directives specified in the configuration file. +Directives are divided into simple directives +and block directives. +A simple directive consists of the name and +parameters separated by spaces and ends with a +semicolon +.Cm ; +symbol. +A block directive has the same structure as a +simple directive, but instead of the semicolon +it ends with a set of additional instructions +surrounded by braces: +.Cm { +and +.Cm } +\&. +If a block directive can have other directives +inside braces, it is called a context (examples: +.Cm events , +.Cm http , +.Cm server , +and +.Cm location ) +\&. +.Pp +Directives placed in the configuration file +outside of any contexts are considered to be in +the main context. +The +.Cm events +and +.Cm http +directives +reside in the +.Pa main +context, +.Cm server +in +.Pa http , +and +.Cm location +in +.Pa server +\&. +.Pp +The rest of a line after the +.Cm # +sign is considered a comment. +.Sh EXAMPLES +.Bd -literal -offset indent +user www www; +worker_processes 2; + +error_log /var/log/nginx-error.log info; + +events { + use kqueue; + worker_connections 2048; +} +\&... +.Ed +.Sh FILES +%%CONF_PATH%% +.Sh SEE ALSO +.Xr nginx 8 +.Sh AUTHORS +.An -nosplit +.An Igor Sysoev Aq igor at sysoev.ru . From xeioex at nginx.com Mon Dec 9 13:09:45 2019 From: xeioex at nginx.com (Dmitry Volyntsev) Date: Mon, 09 Dec 2019 13:09:45 +0000 Subject: [njs] Introduced the ArrayBuffer object. Message-ID: details: https://hg.nginx.org/njs/rev/3bef40125db2 branches: changeset: 1281:3bef40125db2 user: Tiago Natel de Moura date: Wed Nov 27 14:02:04 2019 +0000 description: Introduced the ArrayBuffer object. diffstat: auto/sources | 1 + src/njs_array_buffer.c | 268 +++++++++++++++++++++++++++++++++++++++++++++++ src/njs_array_buffer.h | 46 ++++++++ src/njs_builtin.c | 11 + src/njs_json.c | 1 + src/njs_main.h | 1 + src/njs_object.c | 3 + src/njs_object_hash.h | 15 ++ src/njs_value.c | 4 + src/njs_value.h | 40 +++++++ src/njs_vm.h | 1 + src/test/njs_unit_test.c | 49 ++++++++ 12 files changed, 440 insertions(+), 0 deletions(-) diffs (609 lines): diff -r 242395b814bb -r 3bef40125db2 auto/sources --- a/auto/sources Fri Dec 06 14:59:48 2019 +0300 +++ b/auto/sources Wed Nov 27 14:02:04 2019 +0000 @@ -51,6 +51,7 @@ NJS_LIB_SRCS=" \ src/njs_parser_expression.c \ src/njs_generator.c \ src/njs_disassembler.c \ + src/njs_array_buffer.c \ " NJS_LIB_TEST_SRCS=" \ diff -r 242395b814bb -r 3bef40125db2 src/njs_array_buffer.c --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/njs_array_buffer.c Wed Nov 27 14:02:04 2019 +0000 @@ -0,0 +1,268 @@ +/* + * Copyright (C) Igor Sysoev + * Copyright (C) NGINX, Inc. + */ + + +#include + + +njs_array_buffer_t * +njs_array_buffer_alloc(njs_vm_t *vm, uint64_t size) +{ + njs_object_t *proto; + njs_array_buffer_t *array; + + if (njs_slow_path(size > UINT32_MAX)) { + goto overflow; + } + + array = njs_mp_alloc(vm->mem_pool, sizeof(njs_array_buffer_t)); + if (njs_slow_path(array == NULL)) { + goto memory_error; + } + + if (size > 0) { + array->u.data = njs_mp_zalloc(vm->mem_pool, size); + if (njs_slow_path(array->u.data == NULL)) { + goto memory_error; + } + } + + proto = &vm->prototypes[NJS_OBJ_TYPE_ARRAY_BUFFER].object; + + njs_lvlhsh_init(&array->object.hash); + njs_lvlhsh_init(&array->object.shared_hash); + array->object.__proto__ = proto; + array->object.type = NJS_ARRAY_BUFFER; + array->object.shared = 0; + array->object.extensible = 1; + array->size = size; + + return array; + +memory_error: + + njs_memory_error(vm); + + return NULL; + +overflow: + + njs_range_error(vm, "Invalid array length"); + + return NULL; +} + + +static njs_int_t +njs_array_buffer_constructor(njs_vm_t *vm, njs_value_t *args, njs_uint_t nargs, + njs_index_t unused) +{ + uint32_t size; + njs_int_t ret; + njs_value_t *value; + njs_array_buffer_t *array; + + if (!vm->top_frame->ctor) { + njs_type_error(vm, "Constructor ArrayBuffer requires 'new'"); + return NJS_ERROR; + } + + size = 0; + value = njs_arg(args, nargs, 1); + + ret = njs_value_to_index(vm, value, &size); + if (njs_slow_path(ret != NJS_OK)) { + return NJS_ERROR; + } + + array = njs_array_buffer_alloc(vm, size); + if (njs_slow_path(array == NULL)) { + return NJS_ERROR; + } + + njs_set_array_buffer(&vm->retval, array); + + return NJS_OK; +} + + +static njs_int_t +njs_array_buffer_get_this(njs_vm_t *vm, njs_value_t *args, + njs_uint_t nargs, njs_index_t unused) +{ + vm->retval = args[0]; + + return NJS_OK; +} + + +static const njs_object_prop_t njs_array_buffer_constructor_properties[] = +{ + /* ArrayBuffer.name == "ArrayBuffer". */ + { + .type = NJS_PROPERTY, + .name = njs_string("name"), + .value = njs_string("ArrayBuffer"), + .configurable = 1, + }, + + /* ArrayBuffer.length == 1. */ + { + .type = NJS_PROPERTY, + .name = njs_string("length"), + .value = njs_value(NJS_NUMBER, 1, 1.0), + .configurable = 1, + }, + + /* ArrayBuffer.prototype. */ + { + .type = NJS_PROPERTY_HANDLER, + .name = njs_string("prototype"), + .value = njs_prop_handler(njs_object_prototype_create), + }, + + /* ArrayBuffer[Symbol.species] */ + { + .type = NJS_PROPERTY, + .name = njs_wellknown_symbol(NJS_SYMBOL_SPECIES), + .value = njs_value(NJS_INVALID, 1, NAN), + .getter = njs_native_function(njs_array_buffer_get_this, 0), + .setter = njs_value(NJS_UNDEFINED, 0, NAN), + .writable = NJS_ATTRIBUTE_UNSET, + .configurable = 1, + .enumerable = 0, + }, +}; + + +const njs_object_init_t njs_array_buffer_constructor_init = { + njs_array_buffer_constructor_properties, + njs_nitems(njs_array_buffer_constructor_properties), +}; + + +static njs_int_t +njs_array_buffer_prototype_byte_length(njs_vm_t *vm, njs_value_t *args, + njs_uint_t nargs, njs_index_t unused) +{ + njs_value_t *value; + njs_array_buffer_t *array; + + value = njs_arg(args, nargs, 0); + + if (!njs_is_array_buffer(value)) { + njs_type_error(vm, "Method ArrayBuffer.prototype.byteLength called " + "on incompatible receiver"); + return NJS_ERROR; + } + + array = njs_array_buffer(value); + + njs_set_number(&vm->retval, array->size); + + return NJS_OK; +} + + +static njs_int_t +njs_array_buffer_prototype_slice(njs_vm_t *vm, njs_value_t *args, + njs_uint_t nargs, njs_index_t unused) +{ + int64_t len, start, end; + njs_int_t ret; + njs_value_t *value; + njs_array_buffer_t *this, *buffer; + + value = njs_arg(args, nargs, 0); + + if (!njs_is_array_buffer(value)) { + njs_type_error(vm, "Method ArrayBuffer.prototype.slice called " + "on incompatible receiver"); + return NJS_ERROR; + } + + this = njs_array_buffer(value); + len = njs_array_buffer_size(this); + end = len; + + value = njs_arg(args, nargs, 1); + + ret = njs_value_to_integer(vm, value, &start); + if (njs_slow_path(ret != NJS_OK)) { + return ret; + } + + value = njs_arg(args, nargs, 2); + + if (!njs_is_undefined(value)) { + ret = njs_value_to_integer(vm, value, &end); + if (njs_slow_path(ret != NJS_OK)) { + return ret; + } + } + + buffer = njs_array_buffer_slice(vm, this, start, end); + if (njs_slow_path(buffer == NULL)) { + return NJS_ERROR; + } + + njs_set_array_buffer(&vm->retval, buffer); + + return NJS_OK; +} + + +static const njs_object_prop_t njs_array_buffer_prototype_properties[] = +{ + { + .type = NJS_PROPERTY_HANDLER, + .name = njs_string("constructor"), + .value = njs_prop_handler(njs_object_prototype_create_constructor), + .writable = 1, + .configurable = 1, + }, + + { + .type = NJS_PROPERTY, + .name = njs_string("byteLength"), + .value = njs_value(NJS_INVALID, 1, NAN), + .getter = njs_native_function(njs_array_buffer_prototype_byte_length, + 0), + .setter = njs_value(NJS_UNDEFINED, 0, NAN), + .writable = NJS_ATTRIBUTE_UNSET, + .configurable = 1, + .enumerable = 0, + }, + + { + .type = NJS_PROPERTY, + .name = njs_string("slice"), + .value = njs_native_function(njs_array_buffer_prototype_slice, 2), + .writable = 1, + .configurable = 1, + .enumerable = 0, + }, + + { + .type = NJS_PROPERTY, + .name = njs_wellknown_symbol(NJS_SYMBOL_TO_STRING_TAG), + .value = njs_string("ArrayBuffer"), + .configurable = 1, + }, +}; + + +const njs_object_init_t njs_array_buffer_prototype_init = { + njs_array_buffer_prototype_properties, + njs_nitems(njs_array_buffer_prototype_properties), +}; + + +const njs_object_type_init_t njs_array_buffer_type_init = { + .constructor = njs_native_ctor(njs_array_buffer_constructor, 1, 0), + .prototype_props = &njs_array_buffer_prototype_init, + .constructor_props = &njs_array_buffer_constructor_init, + .prototype_value = { .object = { .type = NJS_OBJECT } }, +}; diff -r 242395b814bb -r 3bef40125db2 src/njs_array_buffer.h --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/njs_array_buffer.h Wed Nov 27 14:02:04 2019 +0000 @@ -0,0 +1,46 @@ + +/* + * Copyright (C) Igor Sysoev + * Copyright (C) NGINX, Inc. + */ + +#ifndef _NJS_ARRAY_BUFFER_H_INCLUDED_ +#define _NJS_ARRAY_BUFFER_H_INCLUDED_ + + +#define njs_array_buffer_size(buffer) \ + ((buffer)->size) + + +njs_array_buffer_t *njs_array_buffer_alloc(njs_vm_t *vm, uint64_t size); + + +njs_inline njs_array_buffer_t * +njs_array_buffer_slice(njs_vm_t *vm, njs_array_buffer_t *this, int64_t start, + int64_t end) +{ + int64_t len, new_len, first, final; + njs_array_buffer_t *new_buffer; + + len = njs_array_buffer_size(this); + + first = (start < 0) ? njs_max(len + start, 0) : njs_min(start, len); + final = (end < 0) ? njs_max(len + end, 0) : njs_min(end, len); + + new_len = njs_max(final - first, 0); + + new_buffer = njs_array_buffer_alloc(vm, new_len); + if (new_buffer == NULL) { + return NULL; + } + + memcpy(new_buffer->u.u8, &this->u.u8[first], new_len); + + return new_buffer; +} + + +extern const njs_object_type_init_t njs_array_buffer_type_init; + + +#endif /* _NJS_ARRAY_BUFFER_H_INCLUDED_ */ diff -r 242395b814bb -r 3bef40125db2 src/njs_builtin.c --- a/src/njs_builtin.c Fri Dec 06 14:59:48 2019 +0300 +++ b/src/njs_builtin.c Wed Nov 27 14:02:04 2019 +0000 @@ -58,6 +58,7 @@ static const njs_object_type_init_t *con &njs_obj_type_init, &njs_array_type_init, + &njs_array_buffer_type_init, &njs_boolean_type_init, &njs_number_type_init, &njs_symbol_type_init, @@ -1099,6 +1100,16 @@ static const njs_object_prop_t njs_glob { .type = NJS_PROPERTY_HANDLER, + .name = njs_string("ArrayBuffer"), + .value = njs_prop_handler2(njs_top_level_constructor, + NJS_OBJ_TYPE_ARRAY_BUFFER, + NJS_ARRAY_BUFFER_HASH), + .writable = 1, + .configurable = 1, + }, + + { + .type = NJS_PROPERTY_HANDLER, .name = njs_string("Boolean"), .value = njs_prop_handler2(njs_top_level_constructor, NJS_OBJ_TYPE_BOOLEAN, NJS_BOOLEAN_HASH), diff -r 242395b814bb -r 3bef40125db2 src/njs_json.c --- a/src/njs_json.c Fri Dec 06 14:59:48 2019 +0300 +++ b/src/njs_json.c Wed Nov 27 14:02:04 2019 +0000 @@ -2109,6 +2109,7 @@ njs_dump_is_object(const njs_value_t *va { return (value->type == NJS_OBJECT && !njs_object(value)->error_data) || (value->type == NJS_ARRAY) + || (value->type == NJS_ARRAY_BUFFER) || (value->type == NJS_OBJECT_VALUE) || njs_dump_is_external_object(value); } diff -r 242395b814bb -r 3bef40125db2 src/njs_main.h --- a/src/njs_main.h Fri Dec 06 14:59:48 2019 +0300 +++ b/src/njs_main.h Wed Nov 27 14:02:04 2019 +0000 @@ -59,6 +59,7 @@ #include #include #include +#include #include #include #include diff -r 242395b814bb -r 3bef40125db2 src/njs_object.c --- a/src/njs_object.c Fri Dec 06 14:59:48 2019 +0300 +++ b/src/njs_object.c Wed Nov 27 14:02:04 2019 +0000 @@ -2356,6 +2356,8 @@ static const njs_value_t njs_object_obj njs_long_string("[object Object]"); static const njs_value_t njs_object_array_string = njs_string("[object Array]"); +static const njs_value_t njs_object_array_buffer_string = + njs_long_string("[object ArrayBuffer]"); static const njs_value_t njs_object_function_string = njs_long_string("[object Function]"); static const njs_value_t njs_object_regexp_string = @@ -2408,6 +2410,7 @@ njs_object_prototype_to_string(njs_vm_t &njs_object_regexp_string, &njs_object_date_string, &njs_object_object_string, + &njs_object_array_buffer_string, }; value = njs_argument(args, 0); diff -r 242395b814bb -r 3bef40125db2 src/njs_object_hash.h --- a/src/njs_object_hash.h Fri Dec 06 14:59:48 2019 +0300 +++ b/src/njs_object_hash.h Wed Nov 27 14:02:04 2019 +0000 @@ -561,4 +561,19 @@ 'U'), 'R'), 'I'), 'E'), 'r'), 'r'), 'o'), 'r') +#define NJS_ARRAY_BUFFER_HASH \ + njs_djb_hash_add( \ + njs_djb_hash_add( \ + njs_djb_hash_add( \ + njs_djb_hash_add( \ + njs_djb_hash_add( \ + njs_djb_hash_add( \ + njs_djb_hash_add( \ + njs_djb_hash_add( \ + njs_djb_hash_add( \ + njs_djb_hash_add( \ + njs_djb_hash_add(NJS_DJB_HASH_INIT, \ + 'A'), 'r'), 'r'), 'a'), 'y'), 'B'), 'u'), 'f'), 'f'), 'e'), 'r') + + #endif /* _NJS_OBJECT_HASH_H_INCLUDED_ */ diff -r 242395b814bb -r 3bef40125db2 src/njs_value.c --- a/src/njs_value.c Fri Dec 06 14:59:48 2019 +0300 +++ b/src/njs_value.c Wed Nov 27 14:02:04 2019 +0000 @@ -338,6 +338,9 @@ njs_type_string(njs_value_type_t type) case NJS_ARRAY: return "array"; + case NJS_ARRAY_BUFFER: + return "object arraybuffer"; + case NJS_OBJECT_BOOLEAN: return "object boolean"; @@ -547,6 +550,7 @@ njs_property_query(njs_vm_t *vm, njs_pro case NJS_OBJECT: case NJS_ARRAY: + case NJS_ARRAY_BUFFER: case NJS_OBJECT_BOOLEAN: case NJS_OBJECT_NUMBER: case NJS_OBJECT_SYMBOL: diff -r 242395b814bb -r 3bef40125db2 src/njs_value.h --- a/src/njs_value.h Fri Dec 06 14:59:48 2019 +0300 +++ b/src/njs_value.h Wed Nov 27 14:02:04 2019 +0000 @@ -68,6 +68,7 @@ typedef enum { NJS_REGEXP = 0x17, NJS_DATE = 0x18, NJS_OBJECT_VALUE = 0x19, + NJS_ARRAY_BUFFER = 0x1A, NJS_VALUE_TYPE_MAX } njs_value_type_t; @@ -79,6 +80,7 @@ typedef struct njs_object_value_s nj typedef struct njs_function_lambda_s njs_function_lambda_t; typedef struct njs_regexp_pattern_s njs_regexp_pattern_t; typedef struct njs_array_s njs_array_t; +typedef struct njs_array_buffer_s njs_array_buffer_t; typedef struct njs_regexp_s njs_regexp_t; typedef struct njs_date_s njs_date_t; typedef struct njs_property_next_s njs_property_next_t; @@ -138,6 +140,7 @@ union njs_value_s { double number; njs_object_t *object; njs_array_t *array; + njs_array_buffer_t *array_buffer; njs_object_value_t *object_value; njs_function_t *function; njs_function_lambda_t *lambda; @@ -223,6 +226,26 @@ struct njs_array_s { }; +struct njs_array_buffer_s { + njs_object_t object; + size_t size; + union { + uint8_t *u8; + uint16_t *u16; + uint32_t *u32; + uint64_t *u64; + int8_t *i8; + int16_t *i16; + int32_t *i32; + int64_t *i64; + float *f32; + double *f64; + + void *data; + } u; +}; + + typedef struct { union { uint32_t count; @@ -601,6 +624,10 @@ typedef struct { ((value)->type == NJS_ARRAY) +#define njs_is_array_buffer(value) \ + ((value)->type == NJS_ARRAY_BUFFER) + + #define njs_is_function(value) \ ((value)->type == NJS_FUNCTION) @@ -669,6 +696,10 @@ typedef struct { ((value)->data.u.array->length) +#define njs_array_buffer(value) \ + ((value)->data.u.array_buffer) + + #define njs_array_start(value) \ ((value)->data.u.array->start) @@ -817,6 +848,15 @@ njs_set_array(njs_value_t *value, njs_ar njs_inline void +njs_set_array_buffer(njs_value_t *value, njs_array_buffer_t *array) +{ + value->data.u.array_buffer = array; + value->type = NJS_ARRAY_BUFFER; + value->data.truth = 1; +} + + +njs_inline void njs_set_function(njs_value_t *value, njs_function_t *function) { value->data.u.function = function; diff -r 242395b814bb -r 3bef40125db2 src/njs_vm.h --- a/src/njs_vm.h Fri Dec 06 14:59:48 2019 +0300 +++ b/src/njs_vm.h Wed Nov 27 14:02:04 2019 +0000 @@ -82,6 +82,7 @@ typedef enum { typedef enum { NJS_OBJ_TYPE_OBJECT = 0, NJS_OBJ_TYPE_ARRAY, + NJS_OBJ_TYPE_ARRAY_BUFFER, NJS_OBJ_TYPE_BOOLEAN, NJS_OBJ_TYPE_NUMBER, NJS_OBJ_TYPE_SYMBOL, diff -r 242395b814bb -r 3bef40125db2 src/test/njs_unit_test.c --- a/src/test/njs_unit_test.c Fri Dec 06 14:59:48 2019 +0300 +++ b/src/test/njs_unit_test.c Wed Nov 27 14:02:04 2019 +0000 @@ -4733,6 +4733,55 @@ static njs_unit_test_t njs_test[] = "Array.prototype.fill.call(o, 2).a"), njs_str("4") }, + { njs_str("ArrayBuffer()"), + njs_str("TypeError: Constructor ArrayBuffer requires 'new'") }, + + { njs_str("new ArrayBuffer()"), + njs_str("[object ArrayBuffer]") }, + + { njs_str("ArrayBuffer.prototype.constructor.name === 'ArrayBuffer'"), + njs_str("true") }, + + { njs_str("ArrayBuffer.prototype.constructor()"), + njs_str("TypeError: Constructor ArrayBuffer requires 'new'") }, + + { njs_str("ArrayBuffer.name"), + njs_str("ArrayBuffer") }, + + { njs_str("ArrayBuffer[Symbol.species]"), + njs_str("[object Function]") }, + + { njs_str("ArrayBuffer.prototype[Symbol.toStringTag]"), + njs_str("ArrayBuffer") }, + + { njs_str("var desc = Object.getOwnPropertyDescriptor(ArrayBuffer," + "Symbol.species); desc.get"), + njs_str("[object Function]") }, + + { njs_str("var ctor = ArrayBuffer[Symbol.species]; var a = new ctor(100);" + "a.byteLength;"), + njs_str("100") }, + + { njs_str("var a = new ArrayBuffer(); a.byteLength"), + njs_str("0") }, + + { njs_str("var a = new ArrayBuffer.prototype.constructor(10); a.byteLength"), + njs_str("10") }, + + { njs_str("var get = Object.getOwnPropertyDescriptor(ArrayBuffer.prototype, 'byteLength').get;" + "get.call([])"), + njs_str("TypeError: Method ArrayBuffer.prototype.byteLength called on incompatible receiver") }, + + { njs_str("[undefined, 1, 10, 1000, null, NaN, false, {}, [1,2,3], Object(1),'10'," + " -1, -Infinity, Infinity, 2**50]" + ".map(v=>{ var a; try { a = new ArrayBuffer(v) } catch (e) {return e.name} return a.byteLength})"), + njs_str("0,1,10,1000,0,0,0,0,0,1,10,RangeError,RangeError,RangeError,RangeError") }, + + { njs_str("var buffer = new ArrayBuffer(16);" + "[[4,12], [-1,-1], [-1,10], [0, -1], [0, -16], [0,-17]]" + ".map(pr=>buffer.slice(pr[0], pr[1]).byteLength)"), + njs_str("8,0,0,15,0,0") }, + #if NJS_HAVE_LARGE_STACK { njs_str("var o = Object({length: 3});" "Object.defineProperty(o, '0', {set: function(v){this[0] = 2 * v}});" From xeioex at nginx.com Wed Dec 11 12:58:28 2019 From: xeioex at nginx.com (Dmitry Volyntsev) Date: Wed, 11 Dec 2019 12:58:28 +0000 Subject: [njs] Introduced generic string builder API. Message-ID: details: https://hg.nginx.org/njs/rev/5ba9145d9b0c branches: changeset: 1282:5ba9145d9b0c user: Dmitry Volyntsev date: Wed Dec 11 15:30:26 2019 +0300 description: Introduced generic string builder API. diffstat: auto/sources | 1 + src/njs_chb.c | 242 ++++++++++++++++++++++++ src/njs_chb.h | 114 +++++++++++ src/njs_json.c | 464 +++++++++++++--------------------------------- src/njs_main.h | 1 + src/njs_vm.c | 68 +----- src/test/njs_unit_test.c | 146 ++++++++++++++ 7 files changed, 649 insertions(+), 387 deletions(-) diffs (truncated from 1548 to 1000 lines): diff -r 3bef40125db2 -r 5ba9145d9b0c auto/sources --- a/auto/sources Wed Nov 27 14:02:04 2019 +0000 +++ b/auto/sources Wed Dec 11 15:30:26 2019 +0300 @@ -20,6 +20,7 @@ NJS_LIB_SRCS=" \ src/njs_malloc.c \ src/njs_mp.c \ src/njs_sprintf.c \ + src/njs_chb.c \ src/njs_value.c \ src/njs_vm.c \ src/njs_vmcode.c \ diff -r 3bef40125db2 -r 5ba9145d9b0c src/njs_chb.c --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/njs_chb.c Wed Dec 11 15:30:26 2019 +0300 @@ -0,0 +1,242 @@ + +/* + * Copyright (C) Dmitry Volyntsev + * Copyright (C) NGINX, Inc. + */ + + +#include + + +#define NJS_CHB_MIN_SIZE 256 + + +void +njs_chb_append0(njs_chb_t *chain, const char *msg, size_t len) +{ + u_char *p; + + if (len != 0 && !chain->error) { + p = njs_chb_reserve(chain, len); + if (njs_slow_path(p == NULL)) { + return; + } + + memcpy(p, msg, len); + + njs_chb_written(chain, len); + } +} + + +u_char * +njs_chb_reserve(njs_chb_t *chain, size_t size) +{ + njs_chb_node_t *n; + + if (njs_slow_path(size == 0)) { + return NULL; + } + + n = chain->last; + + if (njs_fast_path(n != NULL && njs_chb_node_room(n) >= size)) { + return n->pos; + } + + if (size < NJS_CHB_MIN_SIZE) { + size = NJS_CHB_MIN_SIZE; + } + + n = njs_mp_alloc(chain->pool, sizeof(njs_chb_node_t) + size); + if (njs_slow_path(n == NULL)) { + chain->error = 1; + return NULL; + } + + n->next = NULL; + n->start = (u_char *) n + sizeof(njs_chb_node_t); + n->pos = n->start; + n->end = n->pos + size; + + if (chain->last != NULL) { + chain->last->next = n; + + } else { + chain->nodes = n; + } + + chain->last = n; + + return n->start; +} + + +void +njs_chb_vsprintf(njs_chb_t *chain, size_t size, const char *fmt, va_list args) +{ + u_char *start, *end; + + start = njs_chb_reserve(chain, size); + if (njs_slow_path(start == NULL)) { + return; + } + + end = njs_vsprintf(start, start + size, fmt, args); + + njs_chb_written(chain, end - start); +} + + +void +njs_chb_sprintf(njs_chb_t *chain, size_t size, const char* fmt, ...) +{ + va_list args; + + va_start(args, fmt); + + njs_chb_vsprintf(chain, size, fmt, args); + + va_end(args); +} + + +/* + * Drains size bytes from the beginning of the chain. + */ +void +njs_chb_drain(njs_chb_t *chain, size_t drain) +{ + njs_chb_node_t *n; + + n = chain->nodes; + + while (n != NULL) { + if (njs_chb_node_size(n) > drain) { + n->start += drain; + return; + } + + drain -= njs_chb_node_size(n); + chain->nodes = n->next; + + njs_mp_free(chain->pool, n); + n = chain->nodes; + } + + chain->last = NULL; +} + + +/* + * Drops size bytes from the end of the chain. + */ +void +njs_chb_drop(njs_chb_t *chain, size_t drop) +{ + size_t size; + njs_chb_node_t *n, *next; + + n = chain->last; + + if (njs_fast_path(n != NULL && (njs_chb_node_size(n) > drop))) { + n->pos -= drop; + return; + } + + n = chain->nodes; + size = njs_chb_size(chain); + + if (drop >= size) { + njs_chb_destroy(chain); + njs_chb_init(chain, chain->pool); + return; + } + + while (n != NULL) { + size -= njs_chb_node_size(n); + + if (size <= drop) { + chain->last = n; + chain->last->pos -= drop - size; + + n = chain->last->next; + chain->last->next = NULL; + + break; + } + + n = n->next; + } + + while (n != NULL) { + next = n->next; + njs_mp_free(chain->pool, n); + n = next; + } +} + + +njs_int_t +njs_chb_join(njs_chb_t *chain, njs_str_t *str) +{ + u_char *start; + size_t size; + njs_chb_node_t *n; + + if (chain->error) { + return NJS_DECLINED; + } + + n = chain->nodes; + + if (n == NULL) { + str->length = 0; + str->start = NULL; + return NJS_OK; + } + + size = njs_chb_size(chain); + + start = njs_mp_alloc(chain->pool, size); + if (njs_slow_path(start == NULL)) { + return NJS_ERROR; + } + + n = chain->nodes; + str->length = size; + str->start = start; + + njs_chb_join_to(chain, start); + + return NJS_OK; +} + + +void +njs_chb_join_to(njs_chb_t *chain, u_char *dst) +{ + njs_chb_node_t *n; + + n = chain->nodes; + + while (n != NULL) { + dst = njs_cpymem(dst, n->start, njs_chb_node_size(n)); + n = n->next; + } +} + + +void +njs_chb_destroy(njs_chb_t *chain) +{ + njs_chb_node_t *n, *next; + + n = chain->nodes; + + while (n != NULL) { + next = n->next; + njs_mp_free(chain->pool, n); + n = next; + } +} diff -r 3bef40125db2 -r 5ba9145d9b0c src/njs_chb.h --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/njs_chb.h Wed Dec 11 15:30:26 2019 +0300 @@ -0,0 +1,114 @@ + +/* + * Copyright (C) Dmitry Volyntsev + * Copyright (C) NGINX, Inc. + */ + +#ifndef _NJS_CHB_H_INCLUDED_ +#define _NJS_CHB_H_INCLUDED_ + + +typedef struct njs_chb_node_s njs_chb_node_t; + +struct njs_chb_node_s { + njs_chb_node_t *next; + u_char *start; + u_char *pos; + u_char *end; +}; + +typedef struct { + njs_bool_t error; + njs_mp_t *pool; + njs_chb_node_t *nodes; + njs_chb_node_t *last; +} njs_chb_t; + + +void njs_chb_append0(njs_chb_t *chain, const char *msg, size_t len); +void njs_chb_vsprintf(njs_chb_t *chain, size_t size, const char *fmt, + va_list args); +void njs_chb_sprintf(njs_chb_t *chain, size_t size, const char* fmt, ...); +u_char *njs_chb_reserve(njs_chb_t *chain, size_t size); +void njs_chb_drain(njs_chb_t *chain, size_t drop); +void njs_chb_drop(njs_chb_t *chain, size_t drop); +njs_int_t njs_chb_join(njs_chb_t *chain, njs_str_t *str); +void njs_chb_join_to(njs_chb_t *chain, u_char *dst); +void njs_chb_destroy(njs_chb_t *chain); + + +#define njs_chb_append(chain, msg, len) \ + njs_chb_append0(chain, (const char *) msg, len) + +#define njs_chb_append_literal(chain, literal) \ + njs_chb_append0(chain, literal, njs_length(literal)) + +#define njs_chb_append_str(chain, str) \ + njs_chb_append0(chain, (const char *) (str)->start, (str)->length) + + +#define njs_chb_node_size(n) (size_t) ((n)->pos - (n)->start) +#define njs_chb_node_room(n) (size_t) ((n)->end - (n)->pos) + + +njs_inline void +njs_chb_init(njs_chb_t *chain, njs_mp_t *pool) +{ + chain->error = 0; + chain->pool = pool; + chain->nodes = NULL; + chain->last = NULL; +} + + +njs_inline size_t +njs_chb_size(njs_chb_t *chain) +{ + size_t size; + njs_chb_node_t *n; + + n = chain->nodes; + + size = 0; + + while (n != NULL) { + size += njs_chb_node_size(n); + n = n->next; + } + + return size; +} + + +njs_inline ssize_t +njs_chb_utf8_length(njs_chb_t *chain) +{ + ssize_t len, length; + njs_chb_node_t *n; + + n = chain->nodes; + + length = 0; + + while (n != NULL) { + len = njs_utf8_length(n->start, njs_chb_node_size(n)); + if (njs_slow_path(len < 0)) { + return len; + } + + length += len; + n = n->next; + } + + return length; +} + + +njs_inline void +njs_chb_written(njs_chb_t *chain, size_t bytes) +{ + chain->last->pos += bytes; +} + + +#endif /* _NJS_JSON_H_INCLUDED_ */ diff -r 3bef40125db2 -r 5ba9145d9b0c src/njs_json.c --- a/src/njs_json.c Wed Nov 27 14:02:04 2019 +0000 +++ b/src/njs_json.c Wed Dec 11 15:30:26 2019 +0300 @@ -44,23 +44,10 @@ typedef struct { } njs_json_parse_t; -typedef struct njs_chb_node_s njs_chb_node_t; - -struct njs_chb_node_s { - njs_chb_node_t *next; - u_char *start; - u_char *pos; - u_char *end; -}; - - typedef struct { njs_value_t retval; njs_vm_t *vm; - njs_mp_t *pool; - njs_chb_node_t *nodes; - njs_chb_node_t *last; njs_uint_t depth; njs_json_state_t states[NJS_JSON_MAX_DEPTH]; @@ -104,33 +91,15 @@ static njs_int_t njs_json_stringify_repl static njs_int_t njs_json_stringify_array(njs_vm_t *vm, njs_json_stringify_t *stringify); -static njs_int_t njs_json_append_value(njs_json_stringify_t *stringify, - const njs_value_t *value); -static njs_int_t njs_json_append_string(njs_json_stringify_t *stringify, - const njs_value_t *value, char quote); -static njs_int_t njs_json_append_number(njs_json_stringify_t *stringify, - const njs_value_t *value); +static void njs_json_append_value(njs_chb_t *chain, const njs_value_t *value); +static void njs_json_append_string(njs_chb_t *chain, const njs_value_t *value, + char quote); +static void njs_json_append_number(njs_chb_t *chain, const njs_value_t *value); static njs_object_t *njs_json_wrap_value(njs_vm_t *vm, njs_value_t *wrapper, const njs_value_t *value); -#define NJS_JSON_BUF_MIN_SIZE 128 - -#define njs_json_buf_written(stringify, bytes) \ - (stringify)->last->pos += (bytes); - -#define njs_json_buf_node_size(n) (size_t) ((n)->pos - (n)->start) -#define njs_json_buf_node_room(n) (size_t) ((n)->end - (n)->pos) - -static njs_int_t njs_json_buf_append(njs_json_stringify_t *stringify, - const char *msg, size_t len); -static u_char *njs_json_buf_reserve(njs_json_stringify_t *stringify, - size_t size); -static njs_int_t njs_json_buf_pullup(njs_json_stringify_t *stringify, - njs_str_t *str); - - static const njs_object_prop_t njs_json_object_properties[]; @@ -224,10 +193,7 @@ njs_json_stringify(njs_vm_t *vm, njs_val stringify = &json_stringify; stringify->vm = vm; - stringify->pool = vm->mem_pool; stringify->depth = 0; - stringify->nodes = NULL; - stringify->last = NULL; stringify->keys_type = NJS_ENUM_STRING; replacer = njs_arg(args, nargs, 2); @@ -1129,43 +1095,24 @@ njs_json_pop_stringify_state(njs_json_st || ((value)->type >= NJS_REGEXP)) -#define njs_json_stringify_append(str, len) \ - ret = njs_json_buf_append(stringify, (char *) str, len); \ - if (ret != NJS_OK) { \ - goto memory_error; \ - } - - #define njs_json_stringify_indent(times) \ if (stringify->space.length != 0) { \ - njs_json_stringify_append("\n", 1); \ + njs_chb_append(&chain,"\n", 1); \ for (i = 0; i < (njs_int_t) (times) - 1; i++) { \ - njs_json_stringify_append(stringify->space.start, \ - stringify->space.length); \ + njs_chb_append_str(&chain, &stringify->space); \ } \ } -#define njs_json_stringify_append_value(value) \ - ret = njs_json_append_value(stringify, value); \ - if (njs_slow_path(ret != NJS_OK)) { \ - if (ret == NJS_DECLINED) { \ - return NJS_ERROR; \ - } \ - \ - goto memory_error; \ - } - - static njs_int_t njs_json_stringify_iterator(njs_vm_t *vm, njs_json_stringify_t *stringify, njs_value_t *object) { - u_char *start; + u_char *p; size_t size; + njs_chb_t chain; ssize_t length; njs_int_t i; njs_int_t ret; - njs_str_t str; njs_value_t *key, *value, wrapper; njs_object_t *obj; njs_json_state_t *state; @@ -1180,17 +1127,19 @@ njs_json_stringify_iterator(njs_vm_t *vm goto memory_error; } + njs_chb_init(&chain, vm->mem_pool); + for ( ;; ) { switch (state->type) { case NJS_JSON_OBJECT: if (state->index == 0) { - njs_json_stringify_append("{", 1); + njs_chb_append_literal(&chain,"{"); njs_json_stringify_indent(stringify->depth); } if (state->index >= state->keys->length) { njs_json_stringify_indent(stringify->depth - 1); - njs_json_stringify_append("}", 1); + njs_chb_append_literal(&chain,"}"); state = njs_json_pop_stringify_state(stringify); if (state == NULL) { @@ -1226,15 +1175,15 @@ njs_json_stringify_iterator(njs_vm_t *vm } if (state->written) { - njs_json_stringify_append(",", 1); + njs_chb_append_literal(&chain,","); njs_json_stringify_indent(stringify->depth); } state->written = 1; - njs_json_append_string(stringify, key, '\"'); - njs_json_stringify_append(":", 1); + njs_json_append_string(&chain, key, '\"'); + njs_chb_append_literal(&chain,":"); if (stringify->space.length != 0) { - njs_json_stringify_append(" ", 1); + njs_chb_append_literal(&chain," "); } if (njs_json_is_object(value)) { @@ -1246,19 +1195,19 @@ njs_json_stringify_iterator(njs_vm_t *vm break; } - njs_json_stringify_append_value(value); + njs_json_append_value(&chain, value); break; case NJS_JSON_ARRAY: if (state->index == 0) { - njs_json_stringify_append("[", 1); + njs_chb_append_literal(&chain,"["); njs_json_stringify_indent(stringify->depth); } if (state->index >= njs_array_len(&state->value)) { njs_json_stringify_indent(stringify->depth - 1); - njs_json_stringify_append("]", 1); + njs_chb_append_literal(&chain,"]"); state = njs_json_pop_stringify_state(stringify); if (state == NULL) { @@ -1269,7 +1218,7 @@ njs_json_stringify_iterator(njs_vm_t *vm } if (state->written) { - njs_json_stringify_append(",", 1); + njs_chb_append_literal(&chain,","); njs_json_stringify_indent(stringify->depth); } @@ -1296,7 +1245,7 @@ njs_json_stringify_iterator(njs_vm_t *vm } state->written = 1; - njs_json_stringify_append_value(value); + njs_json_append_value(&chain, value); break; } @@ -1304,43 +1253,40 @@ njs_json_stringify_iterator(njs_vm_t *vm done: - ret = njs_json_buf_pullup(stringify, &str); - if (njs_slow_path(ret != NJS_OK)) { - goto memory_error; - } - /* * The value to stringify is wrapped as '{"": value}'. - * An empty object means empty result. + * Stripping the wrapper's data. */ - if (str.length <= njs_length("{\n\n}")) { + + njs_chb_drain(&chain, njs_length("{\"\":")); + njs_chb_drop(&chain, njs_length("}")); + + if (stringify->space.length != 0) { + njs_chb_drain(&chain, njs_length("\n ")); + njs_chb_drop(&chain, njs_length("\n")); + } + + size = njs_chb_size(&chain); + if (njs_slow_path(size == 0)) { njs_set_undefined(&vm->retval); goto release; } - /* Stripping the wrapper's data. */ - - start = str.start + njs_length("{\"\":"); - size = str.length - njs_length("{\"\":}"); - - if (stringify->space.length != 0) { - start += njs_length("\n "); - size -= njs_length("\n \n"); - } - - length = njs_utf8_length(start, size); + length = njs_chb_utf8_length(&chain); if (njs_slow_path(length < 0)) { length = 0; } - ret = njs_string_new(vm, &vm->retval, start, size, length); - if (njs_slow_path(ret != NJS_OK)) { + p = njs_string_alloc(vm, &vm->retval, size, length); + if (njs_slow_path(p == NULL)) { goto memory_error; } + njs_chb_join_to(&chain, p); + release: - njs_mp_free(vm->mem_pool, str.start); + njs_chb_destroy(&chain); return NJS_OK; @@ -1521,8 +1467,8 @@ njs_json_stringify_array(njs_vm_t *vm, n } -static njs_int_t -njs_json_append_value(njs_json_stringify_t *stringify, const njs_value_t *value) +static void +njs_json_append_value(njs_chb_t *chain, const njs_value_t *value) { switch (value->type) { case NJS_OBJECT_STRING: @@ -1530,14 +1476,16 @@ njs_json_append_value(njs_json_stringify /* Fall through. */ case NJS_STRING: - return njs_json_append_string(stringify, value, '\"'); + njs_json_append_string(chain, value, '\"'); + break; case NJS_OBJECT_NUMBER: value = njs_object_value(value); /* Fall through. */ case NJS_NUMBER: - return njs_json_append_number(stringify, value); + njs_json_append_number(chain, value); + break; case NJS_OBJECT_BOOLEAN: value = njs_object_value(value); @@ -1545,26 +1493,27 @@ njs_json_append_value(njs_json_stringify case NJS_BOOLEAN: if (njs_is_true(value)) { - return njs_json_buf_append(stringify, "true", 4); + njs_chb_append_literal(chain, "true"); } else { - return njs_json_buf_append(stringify, "false", 5); + njs_chb_append_literal(chain, "false"); } + break; + case NJS_UNDEFINED: case NJS_NULL: case NJS_SYMBOL: case NJS_INVALID: case NJS_FUNCTION: default: - return njs_json_buf_append(stringify, "null", 4); + njs_chb_append_literal(chain, "null"); } } -static njs_int_t -njs_json_append_string(njs_json_stringify_t *stringify, - const njs_value_t *value, char quote) +static void +njs_json_append_string(njs_chb_t *chain, const njs_value_t *value, char quote) { u_char c, *dst, *dst_end; size_t length; @@ -1580,12 +1529,12 @@ njs_json_append_string(njs_json_stringif end = p + str.size; length = str.length; - dst = njs_json_buf_reserve(stringify, 64); + dst = njs_chb_reserve(chain, length + 2); if (njs_slow_path(dst == NULL)) { - return NJS_ERROR; + return; } - dst_end = dst + 64; + dst_end = dst + length + 2; *dst++ = quote; @@ -1651,26 +1600,25 @@ njs_json_append_string(njs_json_stringif } if (dst_end - dst <= 6) { - njs_json_buf_written(stringify, dst - stringify->last->pos); - - dst = njs_json_buf_reserve(stringify, 64); + njs_chb_written(chain, dst - chain->last->pos); + + dst = njs_chb_reserve(chain, 64); if (njs_slow_path(dst == NULL)) { - return NJS_ERROR; + return; } dst_end = dst + 64; } } - njs_json_buf_written(stringify, dst - stringify->last->pos); - - return njs_json_buf_append(stringify, "e, 1); + njs_chb_written(chain, dst - chain->last->pos); + + njs_chb_append(chain, "e, 1); } -static njs_int_t -njs_json_append_number(njs_json_stringify_t *stringify, - const njs_value_t *value) +static void +njs_json_append_number(njs_chb_t *chain, const njs_value_t *value) { u_char *p; size_t size; @@ -1679,20 +1627,18 @@ njs_json_append_number(njs_json_stringif num = njs_number(value); if (isnan(num) || isinf(num)) { - return njs_json_buf_append(stringify, "null", 4); + njs_chb_append_literal(chain, "null"); } else { - p = njs_json_buf_reserve(stringify, 64); + p = njs_chb_reserve(chain, 64); if (njs_slow_path(p == NULL)) { - return NJS_ERROR; + return; } size = njs_dtoa(num, (char *) p); - njs_json_buf_written(stringify, size); + njs_chb_written(chain, size); } - - return NJS_OK; } @@ -1737,117 +1683,6 @@ njs_json_wrap_value(njs_vm_t *vm, njs_va } -static njs_int_t -njs_json_buf_append(njs_json_stringify_t *stringify, const char *msg, - size_t len) -{ - u_char *p; - - if (len != 0) { - p = njs_json_buf_reserve(stringify, len); - if (njs_slow_path(p == NULL)) { - return NJS_ERROR; - } - - memcpy(p, msg, len); - - njs_json_buf_written(stringify, len); - } - - return NJS_OK; -} - - -static u_char * -njs_json_buf_reserve(njs_json_stringify_t *stringify, size_t size) -{ - njs_chb_node_t *n; - - if (njs_slow_path(size == 0)) { - return NULL; - } - - n = stringify->last; - - if (njs_fast_path(n != NULL && njs_json_buf_node_room(n) >= size)) { - return n->pos; - } - - if (size < NJS_JSON_BUF_MIN_SIZE) { - size = NJS_JSON_BUF_MIN_SIZE; - } - - n = njs_mp_alloc(stringify->pool, sizeof(njs_chb_node_t) + size); - if (njs_slow_path(n == NULL)) { - return NULL; - } - - n->next = NULL; - n->start = (u_char *) n + sizeof(njs_chb_node_t); - n->pos = n->start; - n->end = n->pos + size; - - if (stringify->last != NULL) { - stringify->last->next = n; - - } else { - stringify->nodes = n; - } - - stringify->last = n; - - return n->start; -} - - -static njs_int_t -njs_json_buf_pullup(njs_json_stringify_t *stringify, njs_str_t *str) -{ - u_char *start; - size_t size; - njs_chb_node_t *n; - - n = stringify->nodes; - - if (n == NULL) { - str->length = 0; - str->start = NULL; - return NJS_OK; - } - - if (n->next == NULL) { - str->length = njs_json_buf_node_size(n); - str->start = n->start; - return NJS_OK; - } - - size = 0; - - while (n != NULL) { - size += njs_json_buf_node_size(n); - n = n->next; - } - - start = njs_mp_alloc(stringify->pool, size); - if (njs_slow_path(start == NULL)) { - return NJS_ERROR; - } - - n = stringify->nodes; - str->length = size; - str->start = start; - - while (n != NULL) { - size = njs_json_buf_node_size(n); - memcpy(start, n->start, size); - start += size; - n = n->next; - } - - return NJS_OK; -} - - static const njs_object_prop_t njs_json_object_properties[] = { { @@ -1881,22 +1716,13 @@ const njs_object_init_t njs_json_object }; -#define njs_dump(str) \ - ret = njs_json_buf_append(stringify, str, njs_length(str)); \ - if (njs_slow_path(ret != NJS_OK)) { \ - goto memory_error; \ - } - - static njs_int_t -njs_dump_value(njs_json_stringify_t *stringify, const njs_value_t *value, - njs_uint_t console) +njs_dump_value(njs_json_stringify_t *stringify, njs_chb_t *chain, + const njs_value_t *value, njs_uint_t console) { - u_char *p; + njs_str_t str; njs_int_t ret; - njs_str_t str; njs_value_t str_val; - u_char buf[32]; njs_int_t (*to_string)(njs_vm_t *, njs_value_t *, const njs_value_t *); @@ -1904,26 +1730,20 @@ njs_dump_value(njs_json_stringify_t *str case NJS_OBJECT_STRING: value = njs_object_value(value); - njs_string_get(value, &str); - - njs_dump("[String: "); - - ret = njs_json_append_string(stringify, value, '\''); - if (njs_slow_path(ret != NJS_OK)) { - goto memory_error; - } - - njs_dump("]"); + njs_chb_append_literal(chain, "[String: "); + njs_json_append_string(chain, value, '\''); + njs_chb_append_literal(chain, "]"); break; case NJS_STRING: njs_string_get(value, &str); if (!console || stringify->depth != 0) { - return njs_json_append_string(stringify, value, '\''); + njs_json_append_string(chain, value, '\''); + return NJS_OK; } - return njs_json_buf_append(stringify, (char *) str.start, str.length); + njs_chb_append_str(chain, &str); break; @@ -1936,15 +1756,7 @@ njs_dump_value(njs_json_stringify_t *str } njs_string_get(&str_val, &str); - - njs_dump("[Symbol: "); - - ret = njs_json_buf_append(stringify, (char *) str.start, str.length); - if (njs_slow_path(ret != NJS_OK)) { - goto memory_error; - } - - njs_dump("]"); + njs_chb_sprintf(chain, 16 + str.length, "[Symbol: %V]", &str); break; @@ -1955,11 +1767,7 @@ njs_dump_value(njs_json_stringify_t *str } njs_string_get(&str_val, &str); - - ret = njs_json_buf_append(stringify, (char *) str.start, str.length); - if (njs_slow_path(ret != NJS_OK)) { - goto memory_error; - } + njs_chb_append_str(chain, &str); break; @@ -1970,7 +1778,7 @@ njs_dump_value(njs_json_stringify_t *str && signbit(njs_number(value)))) { - njs_dump("[Number: -0]"); + njs_chb_append_literal(chain, "[Number: -0]"); break; } From xeioex at nginx.com Wed Dec 11 12:58:28 2019 From: xeioex at nginx.com (Dmitry Volyntsev) Date: Wed, 11 Dec 2019 12:58:28 +0000 Subject: [njs] Refactored Array.prototype.join() using njs_chb_t API. Message-ID: details: https://hg.nginx.org/njs/rev/5bf71dfc052f branches: changeset: 1283:5bf71dfc052f user: Dmitry Volyntsev date: Wed Dec 11 15:58:05 2019 +0300 description: Refactored Array.prototype.join() using njs_chb_t API. diffstat: src/njs_array.c | 130 ++++++++++---------------------------------- src/njs_number.c | 34 +++++++++++ src/njs_number.h | 2 + src/njs_string.c | 42 -------------- src/njs_value.c | 89 ++++++++++++++++++++++++++++++ src/njs_value.h | 2 + src/njs_value_conversion.h | 25 ++++++++ src/test/njs_unit_test.c | 6 ++ 8 files changed, 188 insertions(+), 142 deletions(-) diffs (432 lines): diff -r 5ba9145d9b0c -r 5bf71dfc052f src/njs_array.c --- a/src/njs_array.c Wed Dec 11 15:30:26 2019 +0300 +++ b/src/njs_array.c Wed Dec 11 15:58:05 2019 +0300 @@ -1123,12 +1123,13 @@ njs_array_prototype_join(njs_vm_t *vm, n njs_index_t unused) { u_char *p; - uint32_t max; - size_t size, length, mask; + size_t size; + ssize_t length; njs_int_t ret; - njs_uint_t i, n; + njs_chb_t chain; + njs_uint_t i; njs_array_t *array; - njs_value_t *value, *values; + njs_value_t *value; njs_string_prop_t separator, string; ret = njs_value_to_object(vm, &args[0]); @@ -1159,117 +1160,46 @@ njs_array_prototype_join(njs_vm_t *vm, n array = njs_array(&args[0]); - max = 0; - values = NULL; + njs_chb_init(&chain, vm->mem_pool); + + length = 0; for (i = 0; i < array->length; i++) { value = &array->start[i]; - - if (!njs_is_string(value) - && njs_is_valid(value) - && !njs_is_null_or_undefined(value)) - { - max++; - } - } - - if (max != 0) { - values = njs_mp_align(vm->mem_pool, sizeof(njs_value_t), - sizeof(njs_value_t) * max); - if (njs_slow_path(values == NULL)) { - njs_memory_error(vm); - return NJS_ERROR; - } - - n = 0; - - for (i = 0; i < array->length; i++) { - value = &array->start[i]; - - if (!njs_is_string(value) - && njs_is_valid(value) - && !njs_is_null_or_undefined(value)) - { - values[n++] = *value; - - if (n >= max) { - break; + if (njs_is_valid(value) && !njs_is_null_or_undefined(value)) { + if (!njs_is_string(value)) { + ret = njs_value_to_chain(vm, &chain, value); + if (njs_slow_path(ret != NJS_OK)) { + return ret; } + + } else { + (void) njs_string_prop(&string, value); + length += string.length; + + njs_chb_append(&chain, string.start, string.size); } } + + length += separator.length; + njs_chb_append(&chain, separator.start, separator.size); } - size = 0; - length = 0; - n = 0; - mask = -1; - - array = njs_array(&args[0]); - - for (i = 0; i < array->length; i++) { - value = &array->start[i]; - - if (njs_is_valid(value) && !njs_is_null_or_undefined(value)) { - - if (!njs_is_string(value)) { - value = &values[n++]; - - if (!njs_is_string(value)) { - ret = njs_value_to_string(vm, value, value); - if (ret != NJS_OK) { - return ret; - } - } - } - - (void) njs_string_prop(&string, value); - - size += string.size; - length += string.length; - - if (string.length == 0 && string.size != 0) { - mask = 0; - } - } + njs_chb_drop(&chain, separator.size); + + size = njs_chb_size(&chain); + + if (length != 0) { + length -= separator.length; } - size += separator.size * (array->length - 1); - length += separator.length * (array->length - 1); - - length &= mask; - p = njs_string_alloc(vm, &vm->retval, size, length); if (njs_slow_path(p == NULL)) { return NJS_ERROR; } - n = 0; - - for (i = 0; i < array->length; i++) { - value = &array->start[i]; - - if (njs_is_valid(value) && !njs_is_null_or_undefined(value)) { - if (!njs_is_string(value)) { - value = &values[n++]; - } - - (void) njs_string_prop(&string, value); - - p = memcpy(p, string.start, string.size); - p += string.size; - } - - if (i < array->length - 1) { - p = memcpy(p, separator.start, separator.size); - p += separator.size; - } - } - - for (i = 0; i < max; i++) { - njs_release(vm, &values[i]); - } - - njs_mp_free(vm->mem_pool, values); + njs_chb_join_to(&chain, p); + njs_chb_destroy(&chain); return NJS_OK; } diff -r 5ba9145d9b0c -r 5bf71dfc052f src/njs_number.c --- a/src/njs_number.c Wed Dec 11 15:30:26 2019 +0300 +++ b/src/njs_number.c Wed Dec 11 15:58:05 2019 +0300 @@ -236,6 +236,40 @@ njs_number_to_string(njs_vm_t *vm, njs_v } +void +njs_number_to_chain(njs_vm_t *vm, njs_chb_t *chain, const njs_value_t *number) +{ + double num; + size_t size; + u_char *p; + + num = njs_number(number); + + if (isnan(num)) { + njs_chb_append_literal(chain, "NaN"); + + } else if (isinf(num)) { + + if (num < 0) { + njs_chb_append_literal(chain, "-Infinity"); + + } else { + njs_chb_append_literal(chain, "Infinity"); + } + + } else { + p = njs_chb_reserve(chain, 64); + if (njs_slow_path(p == NULL)) { + return; + } + + size = njs_dtoa(num, (char *) p); + + njs_chb_written(chain, size); + } +} + + static njs_int_t njs_number_constructor(njs_vm_t *vm, njs_value_t *args, njs_uint_t nargs, njs_index_t unused) diff -r 5ba9145d9b0c -r 5bf71dfc052f src/njs_number.h --- a/src/njs_number.h Wed Dec 11 15:30:26 2019 +0300 +++ b/src/njs_number.h Wed Dec 11 15:58:05 2019 +0300 @@ -17,6 +17,8 @@ int64_t njs_number_radix_parse(const u_c uint8_t radix); njs_int_t njs_number_to_string(njs_vm_t *vm, njs_value_t *string, const njs_value_t *number); +void njs_number_to_chain(njs_vm_t *vm, njs_chb_t *chain, + const njs_value_t *number); njs_int_t njs_number_global_is_nan(njs_vm_t *vm, njs_value_t *args, njs_uint_t nargs, njs_index_t unused); njs_int_t njs_number_global_is_finite(njs_vm_t *vm, njs_value_t *args, diff -r 5ba9145d9b0c -r 5bf71dfc052f src/njs_string.c --- a/src/njs_string.c Wed Dec 11 15:30:26 2019 +0300 +++ b/src/njs_string.c Wed Dec 11 15:58:05 2019 +0300 @@ -4254,48 +4254,6 @@ njs_string_replacement_copy(njs_string_r } -njs_int_t -njs_primitive_value_to_string(njs_vm_t *vm, njs_value_t *dst, - const njs_value_t *src) -{ - const njs_value_t *value; - - switch (src->type) { - - case NJS_NULL: - value = &njs_string_null; - break; - - case NJS_UNDEFINED: - value = &njs_string_undefined; - break; - - case NJS_BOOLEAN: - value = njs_is_true(src) ? &njs_string_true : &njs_string_false; - break; - - case NJS_NUMBER: - return njs_number_to_string(vm, dst, src); - - case NJS_SYMBOL: - njs_symbol_conversion_failed(vm, 1); - return NJS_ERROR; - - case NJS_STRING: - /* GC: njs_retain(src); */ - value = src; - break; - - default: - return NJS_ERROR; - } - - *dst = *value; - - return NJS_OK; -} - - double njs_string_to_number(const njs_value_t *value, njs_bool_t parse_float) { diff -r 5ba9145d9b0c -r 5bf71dfc052f src/njs_value.c --- a/src/njs_value.c Wed Dec 11 15:30:26 2019 +0300 +++ b/src/njs_value.c Wed Dec 11 15:58:05 2019 +0300 @@ -1225,6 +1225,95 @@ njs_value_property_delete(njs_vm_t *vm, njs_int_t +njs_primitive_value_to_string(njs_vm_t *vm, njs_value_t *dst, + const njs_value_t *src) +{ + const njs_value_t *value; + + switch (src->type) { + + case NJS_NULL: + value = &njs_string_null; + break; + + case NJS_UNDEFINED: + value = &njs_string_undefined; + break; + + case NJS_BOOLEAN: + value = njs_is_true(src) ? &njs_string_true : &njs_string_false; + break; + + case NJS_NUMBER: + return njs_number_to_string(vm, dst, src); + + case NJS_SYMBOL: + njs_symbol_conversion_failed(vm, 1); + return NJS_ERROR; + + case NJS_STRING: + /* GC: njs_retain(src); */ + value = src; + break; + + default: + return NJS_ERROR; + } + + *dst = *value; + + return NJS_OK; +} + + +njs_int_t +njs_primitive_value_to_chain(njs_vm_t *vm, njs_chb_t *chain, + const njs_value_t *src) +{ + njs_str_t string; + + switch (src->type) { + + case NJS_NULL: + njs_chb_append_literal(chain, "null"); + break; + + case NJS_UNDEFINED: + njs_chb_append_literal(chain, "undefined"); + break; + + case NJS_BOOLEAN: + if (njs_is_true(src)) { + njs_chb_append_literal(chain, "true"); + + } else { + njs_chb_append_literal(chain, "false"); + } + + break; + + case NJS_NUMBER: + njs_number_to_chain(vm, chain, src); + break; + + case NJS_SYMBOL: + njs_symbol_conversion_failed(vm, 1); + return NJS_ERROR; + + case NJS_STRING: + njs_string_get(src, &string); + njs_chb_append_str(chain, &string); + break; + + default: + return NJS_ERROR; + } + + return NJS_OK; +} + + +njs_int_t njs_value_to_object(njs_vm_t *vm, njs_value_t *value) { njs_object_t *object; diff -r 5ba9145d9b0c -r 5bf71dfc052f src/njs_value.h --- a/src/njs_value.h Wed Dec 11 15:30:26 2019 +0300 +++ b/src/njs_value.h Wed Dec 11 15:58:05 2019 +0300 @@ -946,6 +946,8 @@ const char *njs_type_string(njs_value_ty njs_int_t njs_primitive_value_to_string(njs_vm_t *vm, njs_value_t *dst, const njs_value_t *src); +njs_int_t njs_primitive_value_to_chain(njs_vm_t *vm, njs_chb_t *chain, + const njs_value_t *src); double njs_string_to_number(const njs_value_t *value, njs_bool_t parse_float); njs_bool_t njs_string_eq(const njs_value_t *v1, const njs_value_t *v2); diff -r 5ba9145d9b0c -r 5bf71dfc052f src/njs_value_conversion.h --- a/src/njs_value_conversion.h Wed Dec 11 15:30:26 2019 +0300 +++ b/src/njs_value_conversion.h Wed Dec 11 15:58:05 2019 +0300 @@ -199,4 +199,29 @@ njs_value_to_string(njs_vm_t *vm, njs_va } +njs_inline njs_int_t +njs_value_to_chain(njs_vm_t *vm, njs_chb_t *chain, njs_value_t *value) +{ + njs_int_t ret; + njs_value_t primitive; + + if (njs_slow_path(!njs_is_primitive(value))) { + if (njs_slow_path(value->type == NJS_OBJECT_SYMBOL)) { + /* should fail */ + value = njs_object_value(value); + + } else { + ret = njs_value_to_primitive(vm, &primitive, value, 1); + if (njs_slow_path(ret != NJS_OK)) { + return ret; + } + + value = &primitive; + } + } + + return njs_primitive_value_to_chain(vm, chain, value); +} + + #endif /* _NJS_VALUE_CONVERSION_H_INCLUDED_ */ diff -r 5ba9145d9b0c -r 5bf71dfc052f src/test/njs_unit_test.c --- a/src/test/njs_unit_test.c Wed Dec 11 15:30:26 2019 +0300 +++ b/src/test/njs_unit_test.c Wed Dec 11 15:58:05 2019 +0300 @@ -3876,6 +3876,12 @@ static njs_unit_test_t njs_test[] = { njs_str("var a = [1,2,3]; a.join(':')"), njs_str("1:2:3") }, + { njs_str("var a = ['#', '@']; var out= a.join('?'.repeat(33)); [out, out.length]"), + njs_str("#?????????????????????????????????@,35") }, + + { njs_str("var a = ['?', '?']; var out= a.join('?'); [out, out.length]"), + njs_str("???,3") }, + { njs_str("var a = []; a[5] = 5; a.join()"), njs_str(",,,,,5") }, From boskovits at gmail.com Mon Dec 16 14:02:37 2019 From: boskovits at gmail.com (=?UTF-8?Q?G=C3=A1bor_Boskovits?=) Date: Mon, 16 Dec 2019 15:02:37 +0100 Subject: Fwd: 1.17.5 regression In-Reply-To: References: Message-ID: ---------- Forwarded message --------- Felad?: G?bor Boskovits Date: 2019. dec. 6., P, 21:26 Subject: 1.17.5 regression To: Hello gentle Dev Team, On berlin.guix.gnu.org we observed a regression upon upgrading nginx to 1.17.5. The problem was, when pipelining request to a proxy using tls, we suddenly started to get 408 client timeouts, manifesting in dropped connections. Currently we worked around the issue by downgrading to 1.17.4. I have collected some information about the problem here: https://gitlab.com/g_bor/fix-nginx-tls It contains the logs from the good and the bad version, a full nginx config, some info about how nginx was built. I managed to bisect the problem down to changeset 9d2ad2fb4423, which introduced the problem. Any help in further debuggin this would be greatly appreciated. Best regards, g_bor -- OpenPGP Key Fingerprint: 7988:3B9F:7D6A:4DBF:3719:0367:2506:A96C:CF63:0B21 -- OpenPGP Key Fingerprint: 7988:3B9F:7D6A:4DBF:3719:0367:2506:A96C:CF63:0B21 From vladimir.krivopalov at gmail.com Mon Dec 16 20:41:58 2019 From: vladimir.krivopalov at gmail.com (Vladimir Krivopalov) Date: Mon, 16 Dec 2019 12:41:58 -0800 Subject: [PATCH] Core: fix a memory leak in Nginx signaller processes Message-ID: # HG changeset patch # User Vladimir Krivopalov # Date 1576294741 -10800 # Sat Dec 14 06:39:01 2019 +0300 # Node ID 42531fd29c6203016f5d789d1b23dc377f1be880 # Parent 3939483cd1b56b202a99b1db344d9568b0958e67 Core: fix a memory leak in Nginx signaller processes. A memory pool created by the 'main()' function is not destroyed when Nginx is run as a signaller process. Signaller processes are inherently short-lived so this does not cause any noticeable memory overhead. However, memory leaks detectors such as AddressSanitizer and Valgrind report this issue and cause Nginx to exit with a failure core. diff -r 3939483cd1b5 -r 42531fd29c62 src/core/ngx_cycle.c --- a/src/core/ngx_cycle.c Thu Dec 05 19:38:06 2019 +0300 +++ b/src/core/ngx_cycle.c Sat Dec 14 06:39:01 2019 +0300 @@ -303,6 +303,9 @@ } if (ngx_process == NGX_PROCESS_SIGNALLER) { + ngx_destroy_pool(old_cycle->pool); + cycle->old_cycle = NULL; + return cycle; } From grandmaster at al2klimov.de Wed Dec 18 07:22:00 2019 From: grandmaster at al2klimov.de (Alexander A. Klimov) Date: Wed, 18 Dec 2019 08:22:00 +0100 Subject: Patch: Add application/wasm to conf/mime.types Message-ID: # HG changeset patch # User Alexander A. Klimov # Date 1576652721 -3600 # Wed Dec 18 08:05:21 2019 +0100 # Node ID 2bcc6c4df1bfd50912bcda85c22201c0f7f6a4c3 # Parent 3939483cd1b56b202a99b1db344d9568b0958e67 Add application/wasm to conf/mime.types diff -r 3939483cd1b5 -r 2bcc6c4df1bf conf/mime.types --- a/conf/mime.types Thu Dec 05 19:38:06 2019 +0300 +++ b/conf/mime.types Wed Dec 18 08:05:21 2019 +0100 @@ -51,6 +51,7 @@ application/vnd.openxmlformats-officedocument.wordprocessingml.document docx; application/vnd.wap.wmlc wmlc; + application/wasm wasm; application/x-7z-compressed 7z; application/x-cocoa cco; application/x-java-archive-diff jardiff; From mdounin at mdounin.ru Wed Dec 18 15:04:14 2019 From: mdounin at mdounin.ru (Maxim Dounin) Date: Wed, 18 Dec 2019 18:04:14 +0300 Subject: Patch: Add application/wasm to conf/mime.types In-Reply-To: References: Message-ID: <20191218150414.GV12894@mdounin.ru> Hello! On Wed, Dec 18, 2019 at 08:22:00AM +0100, Alexander A. Klimov wrote: > # HG changeset patch > # User Alexander A. Klimov > # Date 1576652721 -3600 > # Wed Dec 18 08:05:21 2019 +0100 > # Node ID 2bcc6c4df1bfd50912bcda85c22201c0f7f6a4c3 > # Parent 3939483cd1b56b202a99b1db344d9568b0958e67 > Add application/wasm to conf/mime.types > > diff -r 3939483cd1b5 -r 2bcc6c4df1bf conf/mime.types > --- a/conf/mime.types Thu Dec 05 19:38:06 2019 +0300 > +++ b/conf/mime.types Wed Dec 18 08:05:21 2019 +0100 > @@ -51,6 +51,7 @@ > application/vnd.openxmlformats-officedocument.wordprocessingml.document > docx; > application/vnd.wap.wmlc wmlc; > + application/wasm wasm; > application/x-7z-compressed 7z; > application/x-cocoa cco; > application/x-java-archive-diff jardiff; This was already discussed more than once. Please see the following ticket for some details: https://trac.nginx.org/nginx/ticket/1606 In particular, I would prefer to wait at least till there is an official registration of the type. -- Maxim Dounin http://mdounin.ru/ From mdounin at mdounin.ru Wed Dec 18 17:24:25 2019 From: mdounin at mdounin.ru (Maxim Dounin) Date: Wed, 18 Dec 2019 20:24:25 +0300 Subject: Fwd: 1.17.5 regression In-Reply-To: References: Message-ID: <20191218172425.GX12894@mdounin.ru> Hello! On Mon, Dec 16, 2019 at 03:02:37PM +0100, G?bor Boskovits wrote: > On berlin.guix.gnu.org we observed a regression upon upgrading nginx > to 1.17.5. The problem was, when pipelining request to a proxy using > tls, we suddenly started to get 408 client timeouts, manifesting in > dropped connections. Currently we worked around the issue by > downgrading to 1.17.4. I have collected some information about the > problem here: > https://gitlab.com/g_bor/fix-nginx-tls > It contains the logs from the good and the bad version, a full nginx > config, some info about how nginx was built. I managed to bisect the > problem down to changeset 9d2ad2fb4423, which introduced the problem. > Any help in further debuggin this would be greatly appreciated. Thanks for the report, it indeed looks like a bug introduced in 9d2ad2fb4423. The problem is that c->read->handler is overwritted when switching to the next pipelined request, ngx_ssl_next_read_handler() is not called, and c->read->ready remains not set. I'll take a look how to fix it properly. Note well that a simpler workaround might be to build nginx with the --with-cc-opt="-DNGX_HAVE_FIONREAD=0" configure argument, this will disable the code in question on systems not using kqueue. -- Maxim Dounin http://mdounin.ru/ From boskovits at gmail.com Wed Dec 18 17:35:35 2019 From: boskovits at gmail.com (=?UTF-8?Q?G=C3=A1bor_Boskovits?=) Date: Wed, 18 Dec 2019 18:35:35 +0100 Subject: Fwd: 1.17.5 regression In-Reply-To: <20191218172425.GX12894@mdounin.ru> References: <20191218172425.GX12894@mdounin.ru> Message-ID: Hello Maxim, Maxim Dounin ezt ?rta (id?pont: 2019. dec. 18., Sze, 18:24): > > Hello! > > On Mon, Dec 16, 2019 at 03:02:37PM +0100, G?bor Boskovits wrote: > > > On berlin.guix.gnu.org we observed a regression upon upgrading nginx > > to 1.17.5. The problem was, when pipelining request to a proxy using > > tls, we suddenly started to get 408 client timeouts, manifesting in > > dropped connections. Currently we worked around the issue by > > downgrading to 1.17.4. I have collected some information about the > > problem here: > > https://gitlab.com/g_bor/fix-nginx-tls > > It contains the logs from the good and the bad version, a full nginx > > config, some info about how nginx was built. I managed to bisect the > > problem down to changeset 9d2ad2fb4423, which introduced the problem. > > Any help in further debuggin this would be greatly appreciated. > > Thanks for the report, it indeed looks like a bug introduced > in 9d2ad2fb4423. > > The problem is that c->read->handler is overwritted when switching > to the next pipelined request, ngx_ssl_next_read_handler() is not > called, and c->read->ready remains not set. I'll take a look how > to fix it properly. Thanks for having a look. Please keep me updated when the fix gets applied. > > Note well that a simpler workaround might be to build nginx with > the --with-cc-opt="-DNGX_HAVE_FIONREAD=0" configure argument, this > will disable the code in question on systems not using kqueue. > Thanks for the information, we do not yet support any systems with kqueue yet, so this might work temporarily. > -- > Maxim Dounin > http://mdounin.ru/ > _______________________________________________ > nginx-devel mailing list > nginx-devel at nginx.org > http://mailman.nginx.org/mailman/listinfo/nginx-devel Best regards, g_bor -- OpenPGP Key Fingerprint: 7988:3B9F:7D6A:4DBF:3719:0367:2506:A96C:CF63:0B21 From ru at nginx.com Thu Dec 19 08:45:28 2019 From: ru at nginx.com (Ruslan Ermilov) Date: Thu, 19 Dec 2019 11:45:28 +0300 Subject: Fwd: 1.17.5 regression In-Reply-To: References: <20191218172425.GX12894@mdounin.ru> Message-ID: <20191219084528.GA71716@lo0.su> On Wed, Dec 18, 2019 at 06:35:35PM +0100, G?bor Boskovits wrote: > Hello Maxim, > > Maxim Dounin ezt ?rta (id?pont: 2019. dec. 18., > Sze, 18:24): > > > > Hello! > > > > On Mon, Dec 16, 2019 at 03:02:37PM +0100, G?bor Boskovits wrote: > > > > > On berlin.guix.gnu.org we observed a regression upon upgrading nginx > > > to 1.17.5. The problem was, when pipelining request to a proxy using > > > tls, we suddenly started to get 408 client timeouts, manifesting in > > > dropped connections. Currently we worked around the issue by > > > downgrading to 1.17.4. I have collected some information about the > > > problem here: > > > https://gitlab.com/g_bor/fix-nginx-tls > > > It contains the logs from the good and the bad version, a full nginx > > > config, some info about how nginx was built. I managed to bisect the > > > problem down to changeset 9d2ad2fb4423, which introduced the problem. > > > Any help in further debuggin this would be greatly appreciated. > > > > Thanks for the report, it indeed looks like a bug introduced > > in 9d2ad2fb4423. > > > > The problem is that c->read->handler is overwritted when switching > > to the next pipelined request, ngx_ssl_next_read_handler() is not > > called, and c->read->ready remains not set. I'll take a look how > > to fix it properly. > > Thanks for having a look. > > Please keep me updated when the fix gets applied. > > > > > Note well that a simpler workaround might be to build nginx with > > the --with-cc-opt="-DNGX_HAVE_FIONREAD=0" configure argument, this > > will disable the code in question on systems not using kqueue. > > > > Thanks for the information, we do not yet support any systems with kqueue yet, > so this might work temporarily. Maxim means e.g. Linux when he says "on systems not using kqueue". There's no such problem on systems with kqueue, e.g. FreeBSD. From boskovits at gmail.com Thu Dec 19 09:32:34 2019 From: boskovits at gmail.com (=?UTF-8?Q?G=C3=A1bor_Boskovits?=) Date: Thu, 19 Dec 2019 10:32:34 +0100 Subject: Fwd: 1.17.5 regression In-Reply-To: <20191219084528.GA71716@lo0.su> References: <20191218172425.GX12894@mdounin.ru> <20191219084528.GA71716@lo0.su> Message-ID: Hello Ruslan, Ruslan Ermilov ezt ?rta (id?pont: 2019. dec. 19., Cs, 9:45): > > On Wed, Dec 18, 2019 at 06:35:35PM +0100, G?bor Boskovits wrote: > > Hello Maxim, > > > > Maxim Dounin ezt ?rta (id?pont: 2019. dec. 18., > > Sze, 18:24): > > > > > > Hello! > > > > > > On Mon, Dec 16, 2019 at 03:02:37PM +0100, G?bor Boskovits wrote: > > > > > > > On berlin.guix.gnu.org we observed a regression upon upgrading nginx > > > > to 1.17.5. The problem was, when pipelining request to a proxy using > > > > tls, we suddenly started to get 408 client timeouts, manifesting in > > > > dropped connections. Currently we worked around the issue by > > > > downgrading to 1.17.4. I have collected some information about the > > > > problem here: > > > > https://gitlab.com/g_bor/fix-nginx-tls > > > > It contains the logs from the good and the bad version, a full nginx > > > > config, some info about how nginx was built. I managed to bisect the > > > > problem down to changeset 9d2ad2fb4423, which introduced the problem. > > > > Any help in further debuggin this would be greatly appreciated. > > > > > > Thanks for the report, it indeed looks like a bug introduced > > > in 9d2ad2fb4423. > > > > > > The problem is that c->read->handler is overwritted when switching > > > to the next pipelined request, ngx_ssl_next_read_handler() is not > > > called, and c->read->ready remains not set. I'll take a look how > > > to fix it properly. > > > > Thanks for having a look. > > > > Please keep me updated when the fix gets applied. > > > > > > > > Note well that a simpler workaround might be to build nginx with > > > the --with-cc-opt="-DNGX_HAVE_FIONREAD=0" configure argument, this > > > will disable the code in question on systems not using kqueue. > > > > > > > Thanks for the information, we do not yet support any systems with kqueue yet, > > so this might work temporarily. > > Maxim means e.g. Linux when he says "on systems not using kqueue". > There's no such problem on systems with kqueue, e.g. FreeBSD. Thanks for the clarification. I believe we will build in the meanwhile with -DNGX_HAVE_FIONREAD=0. Currently we only support linux. > _______________________________________________ > nginx-devel mailing list > nginx-devel at nginx.org > http://mailman.nginx.org/mailman/listinfo/nginx-devel Best regards, g_bor -- OpenPGP Key Fingerprint: 7988:3B9F:7D6A:4DBF:3719:0367:2506:A96C:CF63:0B21 From mdounin at mdounin.ru Thu Dec 19 12:43:52 2019 From: mdounin at mdounin.ru (Maxim Dounin) Date: Thu, 19 Dec 2019 15:43:52 +0300 Subject: Fwd: 1.17.5 regression In-Reply-To: <20191219084528.GA71716@lo0.su> References: <20191218172425.GX12894@mdounin.ru> <20191219084528.GA71716@lo0.su> Message-ID: <20191219124352.GZ12894@mdounin.ru> Hello! On Thu, Dec 19, 2019 at 11:45:28AM +0300, Ruslan Ermilov wrote: > On Wed, Dec 18, 2019 at 06:35:35PM +0100, G?bor Boskovits wrote: > > Hello Maxim, > > > > Maxim Dounin ezt ?rta (id?pont: 2019. dec. 18., > > Sze, 18:24): > > > > > > Hello! > > > > > > On Mon, Dec 16, 2019 at 03:02:37PM +0100, G?bor Boskovits wrote: > > > > > > > On berlin.guix.gnu.org we observed a regression upon upgrading nginx > > > > to 1.17.5. The problem was, when pipelining request to a proxy using > > > > tls, we suddenly started to get 408 client timeouts, manifesting in > > > > dropped connections. Currently we worked around the issue by > > > > downgrading to 1.17.4. I have collected some information about the > > > > problem here: > > > > https://gitlab.com/g_bor/fix-nginx-tls > > > > It contains the logs from the good and the bad version, a full nginx > > > > config, some info about how nginx was built. I managed to bisect the > > > > problem down to changeset 9d2ad2fb4423, which introduced the problem. > > > > Any help in further debuggin this would be greatly appreciated. > > > > > > Thanks for the report, it indeed looks like a bug introduced > > > in 9d2ad2fb4423. > > > > > > The problem is that c->read->handler is overwritted when switching > > > to the next pipelined request, ngx_ssl_next_read_handler() is not > > > called, and c->read->ready remains not set. I'll take a look how > > > to fix it properly. > > > > Thanks for having a look. > > > > Please keep me updated when the fix gets applied. > > > > > > > > Note well that a simpler workaround might be to build nginx with > > > the --with-cc-opt="-DNGX_HAVE_FIONREAD=0" configure argument, this > > > will disable the code in question on systems not using kqueue. > > > > > > > Thanks for the information, we do not yet support any systems with kqueue yet, > > so this might work temporarily. > > Maxim means e.g. Linux when he says "on systems not using kqueue". > There's no such problem on systems with kqueue, e.g. FreeBSD. Not really. I meant the workaround in question won't work on systems with kqueue. The problem itself is present with kqueue as well. That is, the workaround in question will work fine on Linux, but won't work on FreeBSD (unless you configure nginx to use select or poll, but probably this isn't a good idea). -- Maxim Dounin http://mdounin.ru/ From boskovits at gmail.com Thu Dec 19 13:59:58 2019 From: boskovits at gmail.com (=?UTF-8?Q?G=C3=A1bor_Boskovits?=) Date: Thu, 19 Dec 2019 14:59:58 +0100 Subject: Fwd: 1.17.5 regression In-Reply-To: <20191219124352.GZ12894@mdounin.ru> References: <20191218172425.GX12894@mdounin.ru> <20191219084528.GA71716@lo0.su> <20191219124352.GZ12894@mdounin.ru> Message-ID: Ok, thanks. Maxim Dounin ezt ?rta (id?pont: 2019. dec. 19., Cs? 13:44): > Hello! > > On Thu, Dec 19, 2019 at 11:45:28AM +0300, Ruslan Ermilov wrote: > > > On Wed, Dec 18, 2019 at 06:35:35PM +0100, G?bor Boskovits wrote: > > > Hello Maxim, > > > > > > Maxim Dounin ezt ?rta (id?pont: 2019. dec. 18., > > > Sze, 18:24): > > > > > > > > Hello! > > > > > > > > On Mon, Dec 16, 2019 at 03:02:37PM +0100, G?bor Boskovits wrote: > > > > > > > > > On berlin.guix.gnu.org we observed a regression upon upgrading > nginx > > > > > to 1.17.5. The problem was, when pipelining request to a proxy > using > > > > > tls, we suddenly started to get 408 client timeouts, manifesting in > > > > > dropped connections. Currently we worked around the issue by > > > > > downgrading to 1.17.4. I have collected some information about the > > > > > problem here: > > > > > https://gitlab.com/g_bor/fix-nginx-tls > > > > > It contains the logs from the good and the bad version, a full > nginx > > > > > config, some info about how nginx was built. I managed to bisect > the > > > > > problem down to changeset 9d2ad2fb4423, which introduced the > problem. > > > > > Any help in further debuggin this would be greatly appreciated. > > > > > > > > Thanks for the report, it indeed looks like a bug introduced > > > > in 9d2ad2fb4423. > > > > > > > > The problem is that c->read->handler is overwritted when switching > > > > to the next pipelined request, ngx_ssl_next_read_handler() is not > > > > called, and c->read->ready remains not set. I'll take a look how > > > > to fix it properly. > > > > > > Thanks for having a look. > > > > > > Please keep me updated when the fix gets applied. > > > > > > > > > > > Note well that a simpler workaround might be to build nginx with > > > > the --with-cc-opt="-DNGX_HAVE_FIONREAD=0" configure argument, this > > > > will disable the code in question on systems not using kqueue. > > > > > > > > > > Thanks for the information, we do not yet support any systems with > kqueue yet, > > > so this might work temporarily. > > > > Maxim means e.g. Linux when he says "on systems not using kqueue". > > There's no such problem on systems with kqueue, e.g. FreeBSD. > > Not really. I meant the workaround in question won't work on > systems with kqueue. The problem itself is present with kqueue as > well. > > That is, the workaround in question will work fine on Linux, but > won't work on FreeBSD (unless you configure nginx to use select or > poll, but probably this isn't a good idea). > > -- > Maxim Dounin > http://mdounin.ru/ > _______________________________________________ > nginx-devel mailing list > nginx-devel at nginx.org > http://mailman.nginx.org/mailman/listinfo/nginx-devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From maksim.yevmenkin at gmail.com Thu Dec 19 17:01:10 2019 From: maksim.yevmenkin at gmail.com (Maksim Yevmenkin) Date: Thu, 19 Dec 2019 09:01:10 -0800 Subject: Fwd: 1.17.5 regression In-Reply-To: <20191219124352.GZ12894@mdounin.ru> References: <20191218172425.GX12894@mdounin.ru> <20191219084528.GA71716@lo0.su> <20191219124352.GZ12894@mdounin.ru> Message-ID: hello, [...] > > > > Thanks for the report, it indeed looks like a bug introduced > > > > in 9d2ad2fb4423. > > > > > > > > The problem is that c->read->handler is overwritted when switching > > > > to the next pipelined request, ngx_ssl_next_read_handler() is not > > > > called, and c->read->ready remains not set. I'll take a look how > > > > to fix it properly. > > > > > > Thanks for having a look. > > > > > > Please keep me updated when the fix gets applied. [...] > Not really. I meant the workaround in question won't work on > systems with kqueue. The problem itself is present with kqueue as > well. i suspect there is another instance of the same problem. it manifests itself when nginx is reading reueqst over https, i.e. ngx_ssl_recv() is interacting with ngx_http_process_request_line() and friends. rev->handler gets overwritten, and, ngx_ssl_next_read_handler() never gets called. i wonder if SSL_pending(3) can be of value here, i.e. == @@ -2116,7 +2116,7 @@ ngx_ssl_recv_chain(ngx_connection_t *c, ngx_chain_t *cl, off_t limit) ssize_t ngx_ssl_recv(ngx_connection_t *c, u_char *buf, size_t size) { - int n, bytes; + int n, bytes, pending; #ifdef SSL_READ_EARLY_DATA_SUCCESS if (c->ssl->in_early) { @@ -2147,8 +2147,9 @@ ngx_ssl_recv(ngx_connection_t *c, u_char *buf, size_t size) for ( ;; ) { n = SSL_read(c->ssl->connection, buf, size); + pending = SSL_pending(c->ssl->connection); - ngx_log_debug1(NGX_LOG_DEBUG_SSL, c->log, 0, "SSL_read: %d", n); + ngx_log_debug2(NGX_LOG_DEBUG_SSL, c->log, 0, "SSL_read: %d, pending: %d", n, pending); if (n > 0) { bytes += n; @@ -2161,8 +2162,13 @@ ngx_ssl_recv(ngx_connection_t *c, u_char *buf, size_t size) size -= n; if (size == 0) { - c->read->ready = 1; + c->read->available = pending; + c->read->ready = !!(pending > 0); + ngx_log_debug1(NGX_LOG_DEBUG_SSL, c->log, 0, + "SSL_read: avail:%d", c->read->available); + +#if 0 if (c->read->available >= 0) { c->read->available -= bytes; @@ -2203,6 +2209,7 @@ ngx_ssl_recv(ngx_connection_t *c, u_char *buf, size_t size) #endif } +#endif return bytes; } === thanks max From mdounin at mdounin.ru Mon Dec 23 13:34:49 2019 From: mdounin at mdounin.ru (Maxim Dounin) Date: Mon, 23 Dec 2019 16:34:49 +0300 Subject: Fwd: 1.17.5 regression In-Reply-To: References: <20191218172425.GX12894@mdounin.ru> <20191219084528.GA71716@lo0.su> <20191219124352.GZ12894@mdounin.ru> Message-ID: <20191223133449.GJ12894@mdounin.ru> Hello! On Thu, Dec 19, 2019 at 09:01:10AM -0800, Maksim Yevmenkin wrote: > hello, > > [...] > > > > > > Thanks for the report, it indeed looks like a bug introduced > > > > > in 9d2ad2fb4423. > > > > > > > > > > The problem is that c->read->handler is overwritted when switching > > > > > to the next pipelined request, ngx_ssl_next_read_handler() is not > > > > > called, and c->read->ready remains not set. I'll take a look how > > > > > to fix it properly. > > > > > > > > Thanks for having a look. > > > > > > > > Please keep me updated when the fix gets applied. > > [...] > > > Not really. I meant the workaround in question won't work on > > systems with kqueue. The problem itself is present with kqueue as > > well. > > i suspect there is another instance of the same problem. it manifests > itself when nginx is reading reueqst over https, i.e. ngx_ssl_recv() > is interacting with ngx_http_process_request_line() and friends. > rev->handler gets overwritten, and, ngx_ssl_next_read_handler() never > gets called. Yes, any change of connection read handlers can be problematic here. > i wonder if SSL_pending(3) can be of value here, i.e. No, SSL_pending() only returns pending bytes from the SSL record currently being parsed, and there can be additional records in SSL library buffers even if SSL_pending() returns 0. Hence it is useless. The only working alternative solution I'm aware of is to use BIO_set_callback() and count actual bytes read from the socket. Unfortunately, BoringSSL removed BIO_set_callback(), making this solution non-portable. -- Maxim Dounin http://mdounin.ru/ From ru at nginx.com Mon Dec 23 15:45:41 2019 From: ru at nginx.com (Ruslan Ermilov) Date: Mon, 23 Dec 2019 15:45:41 +0000 Subject: [nginx] Dav: fixed Location in successful MKCOL response. Message-ID: details: https://hg.nginx.org/nginx/rev/c1a7d3672653 branches: changeset: 7601:c1a7d3672653 user: Ruslan Ermilov date: Mon Dec 16 15:19:01 2019 +0300 description: Dav: fixed Location in successful MKCOL response. Instead of reducing URI length to not include the terminating '\0' character in 6ddaac3e0bf7, restore the terminating '/' character. diffstat: src/http/modules/ngx_http_dav_module.c | 3 ++- 1 files changed, 2 insertions(+), 1 deletions(-) diffs (20 lines): diff -r 3939483cd1b5 -r c1a7d3672653 src/http/modules/ngx_http_dav_module.c --- a/src/http/modules/ngx_http_dav_module.c Thu Dec 05 19:38:06 2019 +0300 +++ b/src/http/modules/ngx_http_dav_module.c Mon Dec 16 15:19:01 2019 +0300 @@ -513,7 +513,6 @@ } *(p - 1) = '\0'; - r->uri.len--; ngx_log_debug1(NGX_LOG_DEBUG_HTTP, r->connection->log, 0, "http mkcol path: \"%s\"", path.data); @@ -521,6 +520,8 @@ if (ngx_create_dir(path.data, ngx_dir_access(dlcf->access)) != NGX_FILE_ERROR) { + *(p - 1) = '/'; + if (ngx_http_dav_location(r, path.data) != NGX_OK) { return NGX_HTTP_INTERNAL_SERVER_ERROR; } From ru at nginx.com Mon Dec 23 15:45:43 2019 From: ru at nginx.com (Ruslan Ermilov) Date: Mon, 23 Dec 2019 15:45:43 +0000 Subject: [nginx] Saved some memory allocations. Message-ID: details: https://hg.nginx.org/nginx/rev/b399246ea45d branches: changeset: 7602:b399246ea45d user: Ruslan Ermilov date: Mon Dec 16 15:19:01 2019 +0300 description: Saved some memory allocations. In configurations when "root" has variables, some modules unnecessarily allocated memory for the "Location" header value. diffstat: src/http/modules/ngx_http_dav_module.c | 31 +++++-------------------------- src/http/modules/ngx_http_static_module.c | 4 ++-- 2 files changed, 7 insertions(+), 28 deletions(-) diffs (84 lines): diff -r c1a7d3672653 -r b399246ea45d src/http/modules/ngx_http_dav_module.c --- a/src/http/modules/ngx_http_dav_module.c Mon Dec 16 15:19:01 2019 +0300 +++ b/src/http/modules/ngx_http_dav_module.c Mon Dec 16 15:19:01 2019 +0300 @@ -56,7 +56,7 @@ static ngx_int_t ngx_http_dav_depth(ngx_http_request_t *r, ngx_int_t dflt); static ngx_int_t ngx_http_dav_error(ngx_log_t *log, ngx_err_t err, ngx_int_t not_found, char *failed, u_char *path); -static ngx_int_t ngx_http_dav_location(ngx_http_request_t *r, u_char *path); +static ngx_int_t ngx_http_dav_location(ngx_http_request_t *r); static void *ngx_http_dav_create_loc_conf(ngx_conf_t *cf); static char *ngx_http_dav_merge_loc_conf(ngx_conf_t *cf, void *parent, void *child); @@ -285,7 +285,7 @@ } if (status == NGX_HTTP_CREATED) { - if (ngx_http_dav_location(r, path.data) != NGX_OK) { + if (ngx_http_dav_location(r) != NGX_OK) { ngx_http_finalize_request(r, NGX_HTTP_INTERNAL_SERVER_ERROR); return; } @@ -520,9 +520,7 @@ if (ngx_create_dir(path.data, ngx_dir_access(dlcf->access)) != NGX_FILE_ERROR) { - *(p - 1) = '/'; - - if (ngx_http_dav_location(r, path.data) != NGX_OK) { + if (ngx_http_dav_location(r) != NGX_OK) { return NGX_HTTP_INTERNAL_SERVER_ERROR; } @@ -1070,35 +1068,16 @@ static ngx_int_t -ngx_http_dav_location(ngx_http_request_t *r, u_char *path) +ngx_http_dav_location(ngx_http_request_t *r) { - u_char *location; - ngx_http_core_loc_conf_t *clcf; - r->headers_out.location = ngx_list_push(&r->headers_out.headers); if (r->headers_out.location == NULL) { return NGX_ERROR; } - clcf = ngx_http_get_module_loc_conf(r, ngx_http_core_module); - - if (!clcf->alias && clcf->root_lengths == NULL) { - location = path + clcf->root.len; - - } else { - location = ngx_pnalloc(r->pool, r->uri.len); - if (location == NULL) { - ngx_http_clear_location(r); - return NGX_ERROR; - } - - ngx_memcpy(location, r->uri.data, r->uri.len); - } - r->headers_out.location->hash = 1; ngx_str_set(&r->headers_out.location->key, "Location"); - r->headers_out.location->value.len = r->uri.len; - r->headers_out.location->value.data = location; + r->headers_out.location->value = r->uri; return NGX_OK; } diff -r c1a7d3672653 -r b399246ea45d src/http/modules/ngx_http_static_module.c --- a/src/http/modules/ngx_http_static_module.c Mon Dec 16 15:19:01 2019 +0300 +++ b/src/http/modules/ngx_http_static_module.c Mon Dec 16 15:19:01 2019 +0300 @@ -157,8 +157,8 @@ len = r->uri.len + 1; - if (!clcf->alias && clcf->root_lengths == NULL && r->args.len == 0) { - location = path.data + clcf->root.len; + if (!clcf->alias && r->args.len == 0) { + location = path.data + root; *last = '/'; From ru at nginx.com Mon Dec 23 15:45:45 2019 From: ru at nginx.com (Ruslan Ermilov) Date: Mon, 23 Dec 2019 15:45:45 +0000 Subject: [nginx] Fixed request finalization in ngx_http_index_handler(). Message-ID: details: https://hg.nginx.org/nginx/rev/e55e28e6998f branches: changeset: 7603:e55e28e6998f user: Ruslan Ermilov date: Mon Dec 16 15:19:01 2019 +0300 description: Fixed request finalization in ngx_http_index_handler(). Returning 500 instead of NGX_ERROR is preferable here because header has not yet been sent to the client. diffstat: src/http/modules/ngx_http_index_module.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diffs (12 lines): diff -r b399246ea45d -r e55e28e6998f src/http/modules/ngx_http_index_module.c --- a/src/http/modules/ngx_http_index_module.c Mon Dec 16 15:19:01 2019 +0300 +++ b/src/http/modules/ngx_http_index_module.c Mon Dec 16 15:19:01 2019 +0300 @@ -163,7 +163,7 @@ name = ngx_http_map_uri_to_path(r, &path, &root, reserve); if (name == NULL) { - return NGX_ERROR; + return NGX_HTTP_INTERNAL_SERVER_ERROR; } allocated = path.data + path.len - name; From ru at nginx.com Mon Dec 23 15:45:47 2019 From: ru at nginx.com (Ruslan Ermilov) Date: Mon, 23 Dec 2019 15:45:47 +0000 Subject: [nginx] Rewrite: fixed segfault with rewritten URI and "alias". Message-ID: details: https://hg.nginx.org/nginx/rev/7aa20af4ac00 branches: changeset: 7604:7aa20af4ac00 user: Ruslan Ermilov date: Mon Dec 16 15:19:01 2019 +0300 description: Rewrite: fixed segfault with rewritten URI and "alias". The "alias" directive cannot be used in the same location where URI was rewritten. This has been detected in the "rewrite ... break" case, but not when the standalone "break" directive was used. This change also fixes proxy_pass with URI component in a similar case: location /aaa/ { rewrite ^ /xxx/yyy; break; proxy_pass http://localhost:8080/bbb/; } Previously, the "/bbb/yyy" would be sent to a backend instead of "/xxx/yyy". And if location's prefix was longer than the rewritten URI, a segmentation fault might occur. diffstat: src/http/ngx_http_script.c | 9 ++++++++- 1 files changed, 8 insertions(+), 1 deletions(-) diffs (19 lines): diff -r e55e28e6998f -r 7aa20af4ac00 src/http/ngx_http_script.c --- a/src/http/ngx_http_script.c Mon Dec 16 15:19:01 2019 +0300 +++ b/src/http/ngx_http_script.c Mon Dec 16 15:19:01 2019 +0300 @@ -1470,7 +1470,14 @@ void ngx_http_script_break_code(ngx_http_script_engine_t *e) { - e->request->uri_changed = 0; + ngx_http_request_t *r; + + r = e->request; + + if (r->uri_changed) { + r->valid_location = 0; + r->uri_changed = 0; + } e->ip = ngx_http_script_exit; } From ru at nginx.com Mon Dec 23 15:45:49 2019 From: ru at nginx.com (Ruslan Ermilov) Date: Mon, 23 Dec 2019 15:45:49 +0000 Subject: [nginx] Tolerate '\0' in URI when mapping URI to path. Message-ID: details: https://hg.nginx.org/nginx/rev/02a539522be4 branches: changeset: 7605:02a539522be4 user: Ruslan Ermilov date: Mon Dec 16 15:19:01 2019 +0300 description: Tolerate '\0' in URI when mapping URI to path. If a rewritten URI has the null character, only a part of URI was copied to a memory buffer allocated for path. In some setups this could be exploited to expose uninitialized memory via the Location header. diffstat: src/http/ngx_http_core_module.c | 3 ++- 1 files changed, 2 insertions(+), 1 deletions(-) diffs (13 lines): diff -r 7aa20af4ac00 -r 02a539522be4 src/http/ngx_http_core_module.c --- a/src/http/ngx_http_core_module.c Mon Dec 16 15:19:01 2019 +0300 +++ b/src/http/ngx_http_core_module.c Mon Dec 16 15:19:01 2019 +0300 @@ -1843,7 +1843,8 @@ } } - last = ngx_cpystrn(last, r->uri.data + alias, r->uri.len - alias + 1); + last = ngx_copy(last, r->uri.data + alias, r->uri.len - alias); + *last = '\0'; return last; } From ru at nginx.com Mon Dec 23 15:45:52 2019 From: ru at nginx.com (Ruslan Ermilov) Date: Mon, 23 Dec 2019 15:45:52 +0000 Subject: [nginx] Rewrite: disallow empty replacements. Message-ID: details: https://hg.nginx.org/nginx/rev/db8df9cd84c8 branches: changeset: 7606:db8df9cd84c8 user: Ruslan Ermilov date: Mon Dec 16 15:19:01 2019 +0300 description: Rewrite: disallow empty replacements. While empty replacements were caught at run-time, parsing code of the "rewrite" directive expects that a minimum length of the "replacement" argument is 1. diffstat: src/http/modules/ngx_http_rewrite_module.c | 5 +++++ 1 files changed, 5 insertions(+), 0 deletions(-) diffs (15 lines): diff -r 02a539522be4 -r db8df9cd84c8 src/http/modules/ngx_http_rewrite_module.c --- a/src/http/modules/ngx_http_rewrite_module.c Mon Dec 16 15:19:01 2019 +0300 +++ b/src/http/modules/ngx_http_rewrite_module.c Mon Dec 16 15:19:01 2019 +0300 @@ -318,6 +318,11 @@ value = cf->args->elts; + if (value[2].len == 0) { + ngx_conf_log_error(NGX_LOG_EMERG, cf, 0, "empty replacement"); + return NGX_CONF_ERROR; + } + ngx_memzero(&rc, sizeof(ngx_regex_compile_t)); rc.pattern = value[1]; From vbart at nginx.com Mon Dec 23 15:53:24 2019 From: vbart at nginx.com (Valentin Bartenev) Date: Mon, 23 Dec 2019 15:53:24 +0000 Subject: [njs] Introduced nullish coalescing operator. Message-ID: details: https://hg.nginx.org/njs/rev/9c1ef5ab7a1f branches: changeset: 1284:9c1ef5ab7a1f user: Valentin Bartenev date: Thu Dec 12 18:50:27 2019 +0300 description: Introduced nullish coalescing operator. diffstat: src/njs_disassembler.c | 12 ++++++ src/njs_generator.c | 1 + src/njs_lexer.c | 11 ++++++ src/njs_lexer.h | 2 + src/njs_parser_expression.c | 81 +++++++++++++++++++++++++++++++++++++++++++- src/njs_vmcode.c | 9 +++- src/njs_vmcode.h | 26 +++++++------ src/test/njs_unit_test.c | 17 +++++++++ 8 files changed, 142 insertions(+), 17 deletions(-) diffs (262 lines): diff -r 5bf71dfc052f -r 9c1ef5ab7a1f src/njs_disassembler.c --- a/src/njs_disassembler.c Wed Dec 11 15:58:05 2019 +0300 +++ b/src/njs_disassembler.c Thu Dec 12 18:50:27 2019 +0300 @@ -282,6 +282,18 @@ njs_disassemble(u_char *start, u_char *e continue; } + if (operation == NJS_VMCODE_COALESCE) { + test_jump = (njs_vmcode_test_jump_t *) p; + + njs_printf("%05uz COALESCE %04Xz %04Xz +%uz\n", + p - start, (size_t) test_jump->retval, + (size_t) test_jump->value, (size_t) test_jump->offset); + + p += sizeof(njs_vmcode_test_jump_t); + + continue; + } + if (operation == NJS_VMCODE_FUNCTION_FRAME) { function = (njs_vmcode_function_frame_t *) p; diff -r 5bf71dfc052f -r 9c1ef5ab7a1f src/njs_generator.c --- a/src/njs_generator.c Wed Dec 11 15:58:05 2019 +0300 +++ b/src/njs_generator.c Thu Dec 12 18:50:27 2019 +0300 @@ -361,6 +361,7 @@ njs_generate(njs_vm_t *vm, njs_generator case NJS_TOKEN_LOGICAL_AND: case NJS_TOKEN_LOGICAL_OR: + case NJS_TOKEN_COALESCE: return njs_generate_test_jump_expression(vm, generator, node); case NJS_TOKEN_DELETE: diff -r 5bf71dfc052f -r 9c1ef5ab7a1f src/njs_lexer.c --- a/src/njs_lexer.c Wed Dec 11 15:58:05 2019 +0300 +++ b/src/njs_lexer.c Thu Dec 12 18:50:27 2019 +0300 @@ -276,6 +276,11 @@ static const njs_lexer_multi_t njs_grea }; +static const njs_lexer_multi_t njs_conditional_token[] = { + { '?', NJS_TOKEN_COALESCE, 0, NULL }, +}; + + static const njs_lexer_multi_t njs_assignment_token[] = { { '=', NJS_TOKEN_EQUAL, 1, njs_strict_equal_token }, { '>', NJS_TOKEN_ARROW, 0, NULL }, @@ -551,6 +556,12 @@ njs_lexer_next_token(njs_lexer_t *lexer, goto multi; + case NJS_TOKEN_CONDITIONAL: + n = njs_nitems(njs_conditional_token), + multi = njs_conditional_token; + + goto multi; + case NJS_TOKEN_LINE_END: lexer->line++; diff -r 5bf71dfc052f -r 9c1ef5ab7a1f src/njs_lexer.h --- a/src/njs_lexer.h Wed Dec 11 15:58:05 2019 +0300 +++ b/src/njs_lexer.h Thu Dec 12 18:50:27 2019 +0300 @@ -95,6 +95,8 @@ typedef enum { NJS_TOKEN_BITWISE_NOT, NJS_TOKEN_LOGICAL_NOT, + NJS_TOKEN_COALESCE, + NJS_TOKEN_IN, NJS_TOKEN_INSTANCEOF, NJS_TOKEN_TYPEOF, diff -r 5bf71dfc052f -r 9c1ef5ab7a1f src/njs_parser_expression.c --- a/src/njs_parser_expression.c Wed Dec 11 15:58:05 2019 +0300 +++ b/src/njs_parser_expression.c Thu Dec 12 18:50:27 2019 +0300 @@ -41,6 +41,8 @@ static njs_token_t njs_parser_any_expres njs_token_t token); static njs_token_t njs_parser_conditional_expression(njs_vm_t *vm, njs_parser_t *parser, njs_token_t token); +static njs_token_t njs_parser_coalesce_expression(njs_vm_t *vm, + njs_parser_t *parser, njs_token_t token); static njs_token_t njs_parser_binary_expression(njs_vm_t *vm, njs_parser_t *parser, const njs_parser_expression_t *expr, njs_token_t token); @@ -359,9 +361,7 @@ njs_parser_conditional_expression(njs_vm { njs_parser_node_t *node, *cond; - token = njs_parser_binary_expression(vm, parser, - &njs_parser_logical_or_expression, - token); + token = njs_parser_coalesce_expression(vm, parser, token); if (njs_slow_path(token <= NJS_TOKEN_ILLEGAL)) { return token; } @@ -421,6 +421,81 @@ njs_parser_conditional_expression(njs_vm static njs_token_t +njs_parser_coalesce_expression(njs_vm_t *vm, njs_parser_t *parser, + njs_token_t token) +{ + njs_token_t prev_token, next_token; + njs_parser_node_t *node; + + token = njs_parser_binary_expression(vm, parser, + &njs_parser_logical_or_expression, + token); + if (njs_slow_path(token <= NJS_TOKEN_ILLEGAL)) { + return token; + } + + for ( ;; ) { + if (token != NJS_TOKEN_COALESCE) { + return token; + } + + prev_token = parser->lexer->prev_token; + + node = njs_parser_node_new(vm, parser, NJS_TOKEN_COALESCE); + if (njs_slow_path(node == NULL)) { + return NJS_TOKEN_ERROR; + } + + node->u.operation = NJS_VMCODE_COALESCE; + node->left = parser->node; + node->left->dest = node; + + token = njs_parser_token(vm, parser); + if (njs_slow_path(token <= NJS_TOKEN_ILLEGAL)) { + return token; + } + + next_token = token; + + token = njs_parser_binary_expression(vm, parser, + &njs_parser_logical_or_expression, + token); + if (njs_slow_path(token <= NJS_TOKEN_ILLEGAL)) { + return token; + } + + node->right = parser->node; + node->right->dest = node; + parser->node = node; + + if (prev_token != NJS_TOKEN_CLOSE_PARENTHESIS + && njs_slow_path(node->left->token == NJS_TOKEN_LOGICAL_OR + || node->left->token == NJS_TOKEN_LOGICAL_AND)) + { + token = node->left->token; + goto require_parentheses; + } + + if (next_token != NJS_TOKEN_OPEN_PARENTHESIS + && njs_slow_path(node->right->token == NJS_TOKEN_LOGICAL_OR + || node->right->token == NJS_TOKEN_LOGICAL_AND)) + { + token = node->right->token; + goto require_parentheses; + } + } + +require_parentheses: + + njs_parser_syntax_error(vm, parser, "Either \"??\" or \"%s\" expression " + "must be parenthesized", + (token == NJS_TOKEN_LOGICAL_OR) ? "||" + : "&&"); + return NJS_TOKEN_ILLEGAL; +} + + +static njs_token_t njs_parser_binary_expression(njs_vm_t *vm, njs_parser_t *parser, const njs_parser_expression_t *expr, njs_token_t token) { diff -r 5bf71dfc052f -r 9c1ef5ab7a1f src/njs_vmcode.c --- a/src/njs_vmcode.c Wed Dec 11 15:58:05 2019 +0300 +++ b/src/njs_vmcode.c Thu Dec 12 18:50:27 2019 +0300 @@ -503,9 +503,14 @@ next: case NJS_VMCODE_TEST_IF_TRUE: case NJS_VMCODE_TEST_IF_FALSE: - ret = njs_is_true(value1); + case NJS_VMCODE_COALESCE: + if (op == NJS_VMCODE_COALESCE) { + ret = !njs_is_null_or_undefined(value1); - ret ^= op - NJS_VMCODE_TEST_IF_TRUE; + } else { + ret = njs_is_true(value1); + ret ^= op - NJS_VMCODE_TEST_IF_TRUE; + } if (ret) { test_jump = (njs_vmcode_test_jump_t *) pc; diff -r 5bf71dfc052f -r 9c1ef5ab7a1f src/njs_vmcode.h --- a/src/njs_vmcode.h Wed Dec 11 15:58:05 2019 +0300 +++ b/src/njs_vmcode.h Thu Dec 12 18:50:27 2019 +0300 @@ -105,19 +105,21 @@ typedef uint8_t #define NJS_VMCODE_TEST_IF_TRUE VMCODE1(34) #define NJS_VMCODE_TEST_IF_FALSE VMCODE1(35) -#define NJS_VMCODE_UNARY_PLUS VMCODE1(36) -#define NJS_VMCODE_UNARY_NEGATION VMCODE1(37) -#define NJS_VMCODE_BITWISE_NOT VMCODE1(38) -#define NJS_VMCODE_LOGICAL_NOT VMCODE1(39) -#define NJS_VMCODE_OBJECT VMCODE1(40) -#define NJS_VMCODE_ARRAY VMCODE1(41) -#define NJS_VMCODE_FUNCTION VMCODE1(42) -#define NJS_VMCODE_REGEXP VMCODE1(43) +#define NJS_VMCODE_COALESCE VMCODE1(36) -#define NJS_VMCODE_INSTANCE_OF VMCODE1(44) -#define NJS_VMCODE_TYPEOF VMCODE1(45) -#define NJS_VMCODE_VOID VMCODE1(46) -#define NJS_VMCODE_DELETE VMCODE1(47) +#define NJS_VMCODE_UNARY_PLUS VMCODE1(37) +#define NJS_VMCODE_UNARY_NEGATION VMCODE1(38) +#define NJS_VMCODE_BITWISE_NOT VMCODE1(39) +#define NJS_VMCODE_LOGICAL_NOT VMCODE1(40) +#define NJS_VMCODE_OBJECT VMCODE1(41) +#define NJS_VMCODE_ARRAY VMCODE1(42) +#define NJS_VMCODE_FUNCTION VMCODE1(43) +#define NJS_VMCODE_REGEXP VMCODE1(44) + +#define NJS_VMCODE_INSTANCE_OF VMCODE1(45) +#define NJS_VMCODE_TYPEOF VMCODE1(46) +#define NJS_VMCODE_VOID VMCODE1(47) +#define NJS_VMCODE_DELETE VMCODE1(48) #define NJS_VMCODE_NOP 255 diff -r 5bf71dfc052f -r 9c1ef5ab7a1f src/test/njs_unit_test.c --- a/src/test/njs_unit_test.c Wed Dec 11 15:58:05 2019 +0300 +++ b/src/test/njs_unit_test.c Thu Dec 12 18:50:27 2019 +0300 @@ -1295,6 +1295,23 @@ static njs_unit_test_t njs_test[] = { njs_str("false && (true || true)"), njs_str("false") }, + { njs_str("true && (null ?? true)"), + njs_str("true") }, + + { njs_str("(null || undefined) ?? (true && true)"), + njs_str("true") }, + + { njs_str("undefined ?? null ?? false ?? true"), + njs_str("false") }, + + { njs_str("1 && 1 ?? true"), + njs_str("SyntaxError: Either \"??\" or \"&&\" expression " + "must be parenthesized in 1") }, + + { njs_str("null ?? 0 || 1"), + njs_str("SyntaxError: Either \"??\" or \"||\" expression " + "must be parenthesized in 1") }, + { njs_str("var a = true; a = -~!a"), njs_str("1") }, From xeioex at nginx.com Mon Dec 23 15:53:26 2019 From: xeioex at nginx.com (Dmitry Volyntsev) Date: Mon, 23 Dec 2019 15:53:26 +0000 Subject: [njs] Fixed Array.prototype.join(). Message-ID: details: https://hg.nginx.org/njs/rev/a0adc224673d branches: changeset: 1285:a0adc224673d user: Dmitry Volyntsev date: Mon Dec 16 15:18:51 2019 +0300 description: Fixed Array.prototype.join(). Resulting string length of Array.prototype.join() was invalid when joined values are not strings. The issue was introduced in 5bf71dfc052f. diffstat: src/njs_array.c | 33 ++++++++++++++++++++++++--------- src/njs_chb.h | 7 +++++++ src/njs_number.c | 34 ++++++++++++++++++---------------- src/njs_number.h | 4 ++-- src/njs_string.h | 7 +++++++ src/njs_value.c | 21 +++++++++------------ src/njs_value_conversion.h | 3 +++ src/test/njs_unit_test.c | 30 +++++++++++++++++++++++++----- 8 files changed, 95 insertions(+), 44 deletions(-) diffs (292 lines): diff -r 9c1ef5ab7a1f -r a0adc224673d src/njs_array.c --- a/src/njs_array.c Thu Dec 12 18:50:27 2019 +0300 +++ b/src/njs_array.c Mon Dec 16 15:18:51 2019 +0300 @@ -1122,11 +1122,12 @@ static njs_int_t njs_array_prototype_join(njs_vm_t *vm, njs_value_t *args, njs_uint_t nargs, njs_index_t unused) { - u_char *p; + u_char *p, *last; size_t size; ssize_t length; njs_int_t ret; njs_chb_t chain; + njs_utf8_t utf8; njs_uint_t i; njs_array_t *array; njs_value_t *value; @@ -1163,20 +1164,37 @@ njs_array_prototype_join(njs_vm_t *vm, n njs_chb_init(&chain, vm->mem_pool); length = 0; + utf8 = njs_is_byte_string(&separator) ? NJS_STRING_BYTE : NJS_STRING_UTF8; for (i = 0; i < array->length; i++) { value = &array->start[i]; + if (njs_is_valid(value) && !njs_is_null_or_undefined(value)) { if (!njs_is_string(value)) { + last = njs_chb_current(&chain); + ret = njs_value_to_chain(vm, &chain, value); - if (njs_slow_path(ret != NJS_OK)) { + if (njs_slow_path(ret < NJS_OK)) { return ret; } + if (last != njs_chb_current(&chain) && ret == 0) { + /* + * Appended values was a byte string. + */ + utf8 = NJS_STRING_BYTE; + } + + length += ret; + } else { (void) njs_string_prop(&string, value); + + if (njs_is_byte_string(&string)) { + utf8 = NJS_STRING_BYTE; + } + length += string.length; - njs_chb_append(&chain, string.start, string.size); } } @@ -1188,12 +1206,9 @@ njs_array_prototype_join(njs_vm_t *vm, n njs_chb_drop(&chain, separator.size); size = njs_chb_size(&chain); - - if (length != 0) { - length -= separator.length; - } - - p = njs_string_alloc(vm, &vm->retval, size, length); + length -= separator.length; + + p = njs_string_alloc(vm, &vm->retval, size, utf8 ? length : 0); if (njs_slow_path(p == NULL)) { return NJS_ERROR; } diff -r 9c1ef5ab7a1f -r a0adc224673d src/njs_chb.h --- a/src/njs_chb.h Thu Dec 12 18:50:27 2019 +0300 +++ b/src/njs_chb.h Mon Dec 16 15:18:51 2019 +0300 @@ -104,6 +104,13 @@ njs_chb_utf8_length(njs_chb_t *chain) } +njs_inline u_char * +njs_chb_current(njs_chb_t *chain) +{ + return (chain->last != NULL) ? chain->last->pos : NULL; +} + + njs_inline void njs_chb_written(njs_chb_t *chain, size_t bytes) { diff -r 9c1ef5ab7a1f -r a0adc224673d src/njs_number.c --- a/src/njs_number.c Thu Dec 12 18:50:27 2019 +0300 +++ b/src/njs_number.c Mon Dec 16 15:18:51 2019 +0300 @@ -236,37 +236,39 @@ njs_number_to_string(njs_vm_t *vm, njs_v } -void -njs_number_to_chain(njs_vm_t *vm, njs_chb_t *chain, const njs_value_t *number) +njs_int_t +njs_number_to_chain(njs_vm_t *vm, njs_chb_t *chain, double num) { - double num; - size_t size; - u_char *p; - - num = njs_number(number); + size_t size; + u_char *p; if (isnan(num)) { njs_chb_append_literal(chain, "NaN"); + return njs_length("NaN"); - } else if (isinf(num)) { + } + if (isinf(num)) { if (num < 0) { njs_chb_append_literal(chain, "-Infinity"); + return njs_length("-Infinity"); } else { njs_chb_append_literal(chain, "Infinity"); + return njs_length("Infinity"); } + } - } else { - p = njs_chb_reserve(chain, 64); - if (njs_slow_path(p == NULL)) { - return; - } + p = njs_chb_reserve(chain, 64); + if (njs_slow_path(p == NULL)) { + return NJS_ERROR; + } - size = njs_dtoa(num, (char *) p); + size = njs_dtoa(num, (char *) p); - njs_chb_written(chain, size); - } + njs_chb_written(chain, size); + + return size; } diff -r 9c1ef5ab7a1f -r a0adc224673d src/njs_number.h --- a/src/njs_number.h Thu Dec 12 18:50:27 2019 +0300 +++ b/src/njs_number.h Mon Dec 16 15:18:51 2019 +0300 @@ -17,8 +17,8 @@ int64_t njs_number_radix_parse(const u_c uint8_t radix); njs_int_t njs_number_to_string(njs_vm_t *vm, njs_value_t *string, const njs_value_t *number); -void njs_number_to_chain(njs_vm_t *vm, njs_chb_t *chain, - const njs_value_t *number); +njs_int_t njs_number_to_chain(njs_vm_t *vm, njs_chb_t *chain, + double number); njs_int_t njs_number_global_is_nan(njs_vm_t *vm, njs_value_t *args, njs_uint_t nargs, njs_index_t unused); njs_int_t njs_number_global_is_finite(njs_vm_t *vm, njs_value_t *args, diff -r 9c1ef5ab7a1f -r a0adc224673d src/njs_string.h --- a/src/njs_string.h Thu Dec 12 18:50:27 2019 +0300 +++ b/src/njs_string.h Mon Dec 16 15:18:51 2019 +0300 @@ -102,6 +102,13 @@ typedef enum { } njs_utf8_t; +njs_inline njs_bool_t +njs_is_byte_string(njs_string_prop_t *string) +{ + return (string->length == 0 && string->size != 0); +} + + njs_inline uint32_t njs_string_calc_length(njs_utf8_t utf8, const u_char *start, size_t size) { diff -r 9c1ef5ab7a1f -r a0adc224673d src/njs_value.c --- a/src/njs_value.c Thu Dec 12 18:50:27 2019 +0300 +++ b/src/njs_value.c Mon Dec 16 15:18:51 2019 +0300 @@ -1270,46 +1270,43 @@ njs_int_t njs_primitive_value_to_chain(njs_vm_t *vm, njs_chb_t *chain, const njs_value_t *src) { - njs_str_t string; + njs_string_prop_t string; switch (src->type) { case NJS_NULL: njs_chb_append_literal(chain, "null"); - break; + return njs_length("null"); case NJS_UNDEFINED: njs_chb_append_literal(chain, "undefined"); - break; + return njs_length("undefined"); case NJS_BOOLEAN: if (njs_is_true(src)) { njs_chb_append_literal(chain, "true"); + return njs_length("true"); } else { njs_chb_append_literal(chain, "false"); + return njs_length("false"); } - break; - case NJS_NUMBER: - njs_number_to_chain(vm, chain, src); - break; + return njs_number_to_chain(vm, chain, njs_number(src)); case NJS_SYMBOL: njs_symbol_conversion_failed(vm, 1); return NJS_ERROR; case NJS_STRING: - njs_string_get(src, &string); - njs_chb_append_str(chain, &string); - break; + (void) njs_string_prop(&string, src); + njs_chb_append(chain, string.start, string.size); + return string.length; default: return NJS_ERROR; } - - return NJS_OK; } diff -r 9c1ef5ab7a1f -r a0adc224673d src/njs_value_conversion.h --- a/src/njs_value_conversion.h Thu Dec 12 18:50:27 2019 +0300 +++ b/src/njs_value_conversion.h Mon Dec 16 15:18:51 2019 +0300 @@ -199,6 +199,9 @@ njs_value_to_string(njs_vm_t *vm, njs_va } +/* + * retval >= 0 is length (UTF8 characters) value of appended string. + */ njs_inline njs_int_t njs_value_to_chain(njs_vm_t *vm, njs_chb_t *chain, njs_value_t *value) { diff -r 9c1ef5ab7a1f -r a0adc224673d src/test/njs_unit_test.c --- a/src/test/njs_unit_test.c Thu Dec 12 18:50:27 2019 +0300 +++ b/src/test/njs_unit_test.c Mon Dec 16 15:18:51 2019 +0300 @@ -3893,11 +3893,31 @@ static njs_unit_test_t njs_test[] = { njs_str("var a = [1,2,3]; a.join(':')"), njs_str("1:2:3") }, - { njs_str("var a = ['#', '@']; var out= a.join('?'.repeat(33)); [out, out.length]"), - njs_str("#?????????????????????????????????@,35") }, - - { njs_str("var a = ['?', '?']; var out= a.join('?'); [out, out.length]"), - njs_str("???,3") }, + { njs_str("[" + " '?'.repeat(33)," + " String.bytesFrom(Array(16).fill(0x9d))," + "]" + ".map(v=>{var out = ['?', '?'].join(v); return out.length})"), + njs_str("35,20") }, + + { njs_str("[" + " []," + " ['?', '?']," + " [NaN, Math.pow(2,123.2), Infinity, -1]," + " [new String('?'),{toString(){return '?'}}]," + "]" + ".map(v=>{var out = v.join('?'); return [out, out[out.length - 1],out.length]})" + ".map(v=>njs.dump(v))"), + njs_str("['',undefined,0]," + "['???','?',3]," + "['NaN?1.2215056097393134e+37?Infinity?-1','1',38]," + "['???','?',3]") }, + + { njs_str("var a = ['?','?']; a.join('').length"), + njs_str("2") }, + + { njs_str("var a = ['?', String.bytesFrom([0x9d]),'?']; a.join('').length"), + njs_str("5") }, { njs_str("var a = []; a[5] = 5; a.join()"), njs_str(",,,,,5") }, From xeioex at nginx.com Mon Dec 23 15:53:28 2019 From: xeioex at nginx.com (Dmitry Volyntsev) Date: Mon, 23 Dec 2019 15:53:28 +0000 Subject: [njs] Fixed stack-use-after-free in njs_value_property_set(). Message-ID: details: https://hg.nginx.org/njs/rev/1023383de2d6 branches: changeset: 1286:1023383de2d6 user: Dmitry Volyntsev date: Mon Dec 16 15:18:51 2019 +0300 description: Fixed stack-use-after-free in njs_value_property_set(). diffstat: src/njs_object.h | 9 +++------ src/test/njs_unit_test.c | 8 ++++++++ 2 files changed, 11 insertions(+), 6 deletions(-) diffs (43 lines): diff -r a0adc224673d -r 1023383de2d6 src/njs_object.h --- a/src/njs_object.h Mon Dec 16 15:18:51 2019 +0300 +++ b/src/njs_object.h Mon Dec 16 15:18:51 2019 +0300 @@ -204,18 +204,15 @@ njs_value_to_key(njs_vm_t *vm, njs_value njs_inline njs_int_t -njs_key_string_get(njs_vm_t *vm, const njs_value_t *key, njs_str_t *str) +njs_key_string_get(njs_vm_t *vm, njs_value_t *key, njs_str_t *str) { - njs_int_t ret; - njs_value_t dst; + njs_int_t ret; if (njs_slow_path(njs_is_symbol(key))) { - ret = njs_symbol_to_string(vm, &dst, key); + ret = njs_symbol_to_string(vm, key, key); if (njs_slow_path(ret != NJS_OK)) { return ret; } - - key = &dst; } njs_string_get(key, str); diff -r a0adc224673d -r 1023383de2d6 src/test/njs_unit_test.c --- a/src/test/njs_unit_test.c Mon Dec 16 15:18:51 2019 +0300 +++ b/src/test/njs_unit_test.c Mon Dec 16 15:18:51 2019 +0300 @@ -10475,6 +10475,14 @@ static njs_unit_test_t njs_test[] = "while(n--) o[Symbol()] = 'test'; o[''];"), njs_str("undefined") }, + { njs_str("var symA = Symbol('A'); var obj = {[symA]:1}; Object.freeze(obj); " + "obj[symA] = 2"), + njs_str("TypeError: Cannot assign to read-only property \"Symbol(A)\" of object") }, + + { njs_str("var symA = Symbol('A'); var obj = {[symA]:1}; Object.freeze(obj); " + "delete obj[symA]"), + njs_str("TypeError: Cannot delete property \"Symbol(A)\" of object") }, + { njs_str("[" " Object.prototype," " Symbol.prototype," From alexander.borisov at nginx.com Mon Dec 23 15:53:30 2019 From: alexander.borisov at nginx.com (Alexander Borisov) Date: Mon, 23 Dec 2019 15:53:30 +0000 Subject: [njs] Introduced the Promise object. Message-ID: details: https://hg.nginx.org/njs/rev/61bf7a31e685 branches: changeset: 1287:61bf7a31e685 user: Alexander Borisov date: Tue Dec 17 10:35:11 2019 +0300 description: Introduced the Promise object. Implemented according to the specification without: Promise.all(), Promise.allSettled(), Promise.race(). This closes #39 issue on GitHub. diffstat: auto/sources | 1 + src/njs_builtin.c | 10 + src/njs_date.c | 1 + src/njs_event.h | 2 + src/njs_function.c | 6 +- src/njs_function.h | 19 +- src/njs_json.c | 1 + src/njs_main.h | 1 + src/njs_object.c | 1 + src/njs_object_hash.h | 11 + src/njs_promise.c | 1221 ++++++++++++++++++++++++++++++++++++++++ src/njs_promise.h | 18 + src/njs_shell.c | 1 + src/njs_value.c | 4 + src/njs_value.h | 27 +- src/njs_vm.c | 59 +- src/njs_vm.h | 2 + src/njs_vmcode.c | 5 +- src/njs_vmcode.h | 2 + test/js/promise_s1.js | 15 + test/js/promise_s10.js | 11 + test/js/promise_s11.js | 13 + test/js/promise_s12.js | 10 + test/js/promise_s13.js | 21 + test/js/promise_s14.js | 9 + test/js/promise_s15.js | 10 + test/js/promise_s16.js | 10 + test/js/promise_s17.js | 10 + test/js/promise_s18.js | 23 + test/js/promise_s19.js | 33 + test/js/promise_s2.js | 14 + test/js/promise_s20.js | 23 + test/js/promise_s21.js | 30 + test/js/promise_s22.js | 32 + test/js/promise_s23.js | 28 + test/js/promise_s24.js | 13 + test/js/promise_s25.js | 29 + test/js/promise_s26.js | 144 ++++ test/js/promise_s3.js | 11 + test/js/promise_s4.js | 6 + test/js/promise_s5.js | 7 + test/js/promise_s6.js | 4 + test/js/promise_s7.js | 12 + test/js/promise_s8.js | 13 + test/js/promise_s9.js | 10 + test/js/promise_set_timeout.js | 17 + test/njs_expect_test.exp | 170 +++++ 47 files changed, 2089 insertions(+), 31 deletions(-) diffs (truncated from 2553 to 1000 lines): diff -r 1023383de2d6 -r 61bf7a31e685 auto/sources --- a/auto/sources Mon Dec 16 15:18:51 2019 +0300 +++ b/auto/sources Tue Dec 17 10:35:11 2019 +0300 @@ -53,6 +53,7 @@ NJS_LIB_SRCS=" \ src/njs_generator.c \ src/njs_disassembler.c \ src/njs_array_buffer.c \ + src/njs_promise.c \ " NJS_LIB_TEST_SRCS=" \ diff -r 1023383de2d6 -r 61bf7a31e685 src/njs_builtin.c --- a/src/njs_builtin.c Mon Dec 16 15:18:51 2019 +0300 +++ b/src/njs_builtin.c Tue Dec 17 10:35:11 2019 +0300 @@ -66,6 +66,7 @@ static const njs_object_type_init_t *con &njs_function_type_init, &njs_regexp_type_init, &njs_date_type_init, + &njs_promise_type_init, /* Hidden types. */ @@ -1173,6 +1174,15 @@ static const njs_object_prop_t njs_glob { .type = NJS_PROPERTY_HANDLER, + .name = njs_string("Promise"), + .value = njs_prop_handler2(njs_top_level_constructor, + NJS_OBJ_TYPE_PROMISE, NJS_PROMISE_HASH), + .writable = 1, + .configurable = 1, + }, + + { + .type = NJS_PROPERTY_HANDLER, .name = njs_string("Error"), .value = njs_prop_handler2(njs_top_level_constructor, NJS_OBJ_TYPE_ERROR, NJS_ERROR_HASH), diff -r 1023383de2d6 -r 61bf7a31e685 src/njs_date.c --- a/src/njs_date.c Mon Dec 16 15:18:51 2019 +0300 +++ b/src/njs_date.c Tue Dec 17 10:35:11 2019 +0300 @@ -1840,3 +1840,4 @@ const njs_object_type_init_t njs_date_t .prototype_props = &njs_date_prototype_init, .prototype_value = { .object = { .type = NJS_OBJECT } }, }; + diff -r 1023383de2d6 -r 61bf7a31e685 src/njs_event.h --- a/src/njs_event.h Mon Dec 16 15:18:51 2019 +0300 +++ b/src/njs_event.h Tue Dec 17 10:35:11 2019 +0300 @@ -16,6 +16,8 @@ #define njs_posted_events(vm) (!njs_queue_is_empty(&(vm)->posted_events)) +#define njs_promise_events(vm) (!njs_queue_is_empty(&(vm)->promise_events)) + typedef struct { njs_function_t *function; diff -r 1023383de2d6 -r 61bf7a31e685 src/njs_function.c --- a/src/njs_function.c Mon Dec 16 15:18:51 2019 +0300 +++ b/src/njs_function.c Tue Dec 17 10:35:11 2019 +0300 @@ -539,14 +539,14 @@ njs_function_frame_alloc(njs_vm_t *vm, s njs_int_t -njs_function_call(njs_vm_t *vm, njs_function_t *function, +njs_function_call2(njs_vm_t *vm, njs_function_t *function, const njs_value_t *this, const njs_value_t *args, - njs_uint_t nargs, njs_value_t *retval) + njs_uint_t nargs, njs_value_t *retval, njs_bool_t ctor) { njs_int_t ret; njs_value_t dst njs_aligned(16); - ret = njs_function_frame(vm, function, this, args, nargs, 0); + ret = njs_function_frame(vm, function, this, args, nargs, ctor); if (njs_slow_path(ret != NJS_OK)) { return ret; } diff -r 1023383de2d6 -r 61bf7a31e685 src/njs_function.h --- a/src/njs_function.h Mon Dec 16 15:18:51 2019 +0300 +++ b/src/njs_function.h Tue Dec 17 10:35:11 2019 +0300 @@ -115,9 +115,9 @@ njs_int_t njs_function_native_frame(njs_ njs_int_t njs_function_lambda_frame(njs_vm_t *vm, njs_function_t *function, const njs_value_t *this, const njs_value_t *args, njs_uint_t nargs, njs_bool_t ctor); -njs_int_t njs_function_call(njs_vm_t *vm, njs_function_t *function, - const njs_value_t *this, const njs_value_t *args, njs_uint_t nargs, - njs_value_t *retval); +njs_int_t njs_function_call2(njs_vm_t *vm, njs_function_t *function, + const njs_value_t *this, const njs_value_t *args, + njs_uint_t nargs, njs_value_t *retval, njs_bool_t ctor); njs_int_t njs_function_lambda_call(njs_vm_t *vm); njs_int_t njs_function_native_call(njs_vm_t *vm); void njs_function_frame_free(njs_vm_t *vm, njs_native_frame_t *frame); @@ -186,11 +186,20 @@ njs_function_frame_invoke(njs_vm_t *vm, njs_inline njs_int_t +njs_function_call(njs_vm_t *vm, njs_function_t *function, + const njs_value_t *this, const njs_value_t *args, + njs_uint_t nargs, njs_value_t *retval) +{ + return njs_function_call2(vm, function, this, args, nargs, retval, 0); +} + + +njs_inline njs_int_t njs_function_apply(njs_vm_t *vm, njs_function_t *function, const njs_value_t *args, njs_uint_t nargs, njs_value_t *retval) { - return njs_function_call(vm, function, &args[0], &args[1], nargs - 1, - retval); + return njs_function_call2(vm, function, &args[0], &args[1], nargs - 1, + retval, 0); } diff -r 1023383de2d6 -r 61bf7a31e685 src/njs_json.c --- a/src/njs_json.c Mon Dec 16 15:18:51 2019 +0300 +++ b/src/njs_json.c Tue Dec 17 10:35:11 2019 +0300 @@ -1904,6 +1904,7 @@ njs_dump_is_object(const njs_value_t *va return (value->type == NJS_OBJECT && !njs_object(value)->error_data) || (value->type == NJS_ARRAY) || (value->type == NJS_ARRAY_BUFFER) + || (value->type == NJS_PROMISE) || (value->type == NJS_OBJECT_VALUE) || njs_dump_is_external_object(value); } diff -r 1023383de2d6 -r 61bf7a31e685 src/njs_main.h --- a/src/njs_main.h Mon Dec 16 15:18:51 2019 +0300 +++ b/src/njs_main.h Tue Dec 17 10:35:11 2019 +0300 @@ -65,6 +65,7 @@ #include #include #include +#include #include #include diff -r 1023383de2d6 -r 61bf7a31e685 src/njs_object.c --- a/src/njs_object.c Mon Dec 16 15:18:51 2019 +0300 +++ b/src/njs_object.c Tue Dec 17 10:35:11 2019 +0300 @@ -2410,6 +2410,7 @@ njs_object_prototype_to_string(njs_vm_t &njs_object_regexp_string, &njs_object_date_string, &njs_object_object_string, + &njs_object_object_string, &njs_object_array_buffer_string, }; diff -r 1023383de2d6 -r 61bf7a31e685 src/njs_object_hash.h --- a/src/njs_object_hash.h Mon Dec 16 15:18:51 2019 +0300 +++ b/src/njs_object_hash.h Tue Dec 17 10:35:11 2019 +0300 @@ -88,6 +88,17 @@ 'D'), 'a'), 't'), 'e') +#define NJS_PROMISE_HASH \ + njs_djb_hash_add( \ + njs_djb_hash_add( \ + njs_djb_hash_add( \ + njs_djb_hash_add( \ + njs_djb_hash_add( \ + njs_djb_hash_add( \ + njs_djb_hash_add(NJS_DJB_HASH_INIT, \ + 'P'), 'r'), 'o'), 'm'), 'i'), 's'), 'e') + + #define NJS_ENUMERABLE_HASH \ njs_djb_hash_add( \ njs_djb_hash_add( \ diff -r 1023383de2d6 -r 61bf7a31e685 src/njs_promise.c --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/njs_promise.c Tue Dec 17 10:35:11 2019 +0300 @@ -0,0 +1,1221 @@ + +/* + * Copyright (C) Nginx, Inc. + */ + +#include + + +typedef enum { + NJS_PROMISE_PENDING = 0, + NJS_PROMISE_FULFILL, + NJS_PROMISE_REJECTED +} njs_promise_type_t; + +typedef struct { + njs_promise_type_t state; + njs_value_t result; + njs_queue_t fulfill_queue; + njs_queue_t reject_queue; + njs_bool_t is_handled; +} njs_promise_data_t; + +typedef struct { + njs_value_t promise; + njs_value_t resolve; + njs_value_t reject; +} njs_promise_capability_t; + +typedef struct { + njs_promise_capability_t *capability; + njs_promise_type_t type; + njs_queue_link_t link; + njs_value_t handler; +} njs_promise_reaction_t; + +typedef struct { + njs_value_t promise; + njs_value_t finally; + njs_value_t constructor; + njs_bool_t resolved; + njs_bool_t *resolved_ref; + njs_promise_capability_t *capability; +} njs_promise_context_t; + + +static njs_promise_t *njs_promise_constructor_call(njs_vm_t *vm, + njs_function_t *function); +static njs_int_t njs_promise_create_resolving_functions(njs_vm_t *vm, + njs_promise_t *promise, njs_value_t *dst); +static njs_int_t njs_promise_value_constructor(njs_vm_t *vm, njs_value_t *value, + njs_value_t *dst); +static njs_int_t njs_promise_capability_executor(njs_vm_t *vm, + njs_value_t *args, njs_uint_t nargs, njs_index_t retval); +static njs_int_t njs_promise_resolve_function(njs_vm_t *vm, njs_value_t *args, + njs_uint_t nargs, njs_index_t retval); +static njs_promise_t *njs_promise_resolve(njs_vm_t *vm, + njs_value_t *constructor, njs_value_t *x); +static njs_int_t njs_promise_reject_function(njs_vm_t *vm, njs_value_t *args, + njs_uint_t nargs, njs_index_t retval); +static njs_int_t njs_promise_perform_then(njs_vm_t *vm, njs_value_t *value, + njs_value_t *fulfilled, njs_value_t *rejected, + njs_promise_capability_t *capability); +static njs_int_t njs_promise_then_finally_function(njs_vm_t *vm, + njs_value_t *args, njs_uint_t nargs, njs_index_t unused); +static njs_int_t njs_promise_catch_finally_function(njs_vm_t *vm, + njs_value_t *args, njs_uint_t nargs, njs_index_t unused); +static njs_int_t njs_promise_reaction_job(njs_vm_t *vm, njs_value_t *args, + njs_uint_t nargs, njs_index_t unused); +static njs_int_t njs_promise_resolve_thenable_job(njs_vm_t *vm, + njs_value_t *args, njs_uint_t nargs, njs_index_t unused); + + +static njs_promise_t * +njs_promise_alloc(njs_vm_t *vm) +{ + njs_promise_t *promise; + njs_promise_data_t *data; + + promise = njs_mp_alloc(vm->mem_pool, sizeof(njs_promise_t) + + sizeof(njs_promise_data_t)); + if (njs_slow_path(promise == NULL)) { + goto memory_error; + } + + njs_lvlhsh_init(&promise->object.hash); + njs_lvlhsh_init(&promise->object.shared_hash); + promise->object.type = NJS_PROMISE; + promise->object.shared = 0; + promise->object.extensible = 1; + promise->object.__proto__ = &vm->prototypes[NJS_OBJ_TYPE_PROMISE].object; + + data = (njs_promise_data_t *) ((uint8_t *) promise + sizeof(njs_promise_t)); + + data->state = NJS_PROMISE_PENDING; + data->is_handled = 0; + + njs_queue_init(&data->fulfill_queue); + njs_queue_init(&data->reject_queue); + + njs_set_promise(&vm->retval, promise); + njs_value_data_set(&promise->value, data); + + return promise; + +memory_error: + + njs_memory_error(vm); + + return NULL; +} + + +njs_int_t +njs_promise_constructor(njs_vm_t *vm, njs_value_t *args, njs_uint_t nargs, + njs_index_t unused) +{ + njs_promise_t *promise; + njs_function_t *function; + + if (njs_slow_path(!vm->top_frame->ctor)) { + njs_type_error(vm, "the Promise constructor must be called with new"); + return NJS_ERROR; + } + + if (njs_slow_path(!njs_is_function(njs_arg(args, nargs, 1)))) { + njs_type_error(vm, "unexpected arguments"); + return NJS_ERROR; + } + + function = njs_function(njs_argument(args, 1)); + + promise = njs_promise_constructor_call(vm, function); + if (njs_slow_path(promise == NULL)) { + return NJS_ERROR; + } + + njs_set_promise(&vm->retval, promise); + + return NJS_OK; +} + + +static njs_promise_t * +njs_promise_constructor_call(njs_vm_t *vm, njs_function_t *function) +{ + njs_int_t ret; + njs_value_t retval, arguments[2]; + njs_promise_t *promise; + + promise = njs_promise_alloc(vm); + if (njs_slow_path(promise == NULL)) { + return NULL; + } + + ret = njs_promise_create_resolving_functions(vm, promise, arguments); + if (njs_slow_path(ret != NJS_OK)) { + return NULL; + } + + ret = njs_function_call(vm, function, &njs_value_undefined, arguments, 2, + &retval); + if (njs_slow_path(ret != NJS_OK)) { + if (njs_slow_path(njs_is_memory_error(vm, &vm->retval))) { + return NULL; + } + + ret = njs_function_call(vm, njs_function(&arguments[1]), + &njs_value_undefined, &vm->retval, 1, &retval); + if (njs_slow_path(ret != NJS_OK)) { + return NULL; + } + } + + return promise; +} + + +static njs_function_t * +njs_promise_create_function(njs_vm_t *vm) +{ + njs_function_t *function; + njs_promise_context_t *context; + + function = njs_mp_zalloc(vm->mem_pool, sizeof(njs_function_t)); + if (njs_slow_path(function == NULL)) { + goto memory_error; + } + + context = njs_mp_zalloc(vm->mem_pool, sizeof(njs_promise_context_t)); + if (njs_slow_path(context == NULL)) { + njs_mp_free(vm->mem_pool, function); + goto memory_error; + } + + function->object.__proto__ = &vm->prototypes[NJS_OBJ_TYPE_FUNCTION].object; + function->object.shared_hash = vm->shared->arrow_instance_hash; + function->object.type = NJS_FUNCTION; + function->object.shared = 0; + function->object.extensible = 1; + function->args_offset = 1; + function->native = 1; + function->context = context; + + return function; + +memory_error: + + njs_memory_error(vm); + + return NULL; +} + + +static njs_int_t +njs_promise_create_resolving_functions(njs_vm_t *vm, njs_promise_t *promise, + njs_value_t *dst) +{ + unsigned i; + njs_function_t *function; + njs_promise_context_t *context, *resolve_context; + + i = 0; + + /* Some compilers give at error an uninitialized context if using for. */ + do { + function = njs_promise_create_function(vm); + if (njs_slow_path(function == NULL)) { + return NJS_ERROR; + } + + function->args_count = 1; + + context = function->context; + context->resolved_ref = &context->resolved; + + njs_set_promise(&context->promise, promise); + njs_set_function(&dst[i], function); + + } while (++i < 2); + + njs_function(&dst[0])->u.native = njs_promise_resolve_function; + njs_function(&dst[1])->u.native = njs_promise_reject_function; + + resolve_context = njs_function(&dst[0])->context; + resolve_context->resolved_ref = &context->resolved; + + return NJS_OK; +} + + +static njs_promise_capability_t * +njs_promise_new_capability(njs_vm_t *vm, njs_value_t *constructor) +{ + njs_int_t ret; + njs_value_t argument, this; + njs_object_t *object; + njs_function_t *function; + njs_promise_context_t *context; + njs_promise_capability_t *capability; + + object = NULL; + function = NULL; + + ret = njs_promise_value_constructor(vm, constructor, constructor); + if (njs_slow_path(ret != NJS_OK)) { + return NULL; + } + + capability = njs_mp_zalloc(vm->mem_pool, sizeof(njs_promise_capability_t)); + if (njs_slow_path(capability == NULL)) { + njs_memory_error(vm); + return NULL; + } + + function = njs_promise_create_function(vm); + if (njs_slow_path(function == NULL)) { + return NULL; + } + + njs_set_undefined(&capability->resolve); + njs_set_undefined(&capability->reject); + + function->u.native = njs_promise_capability_executor; + function->args_count = 2; + + context = function->context; + context->capability = capability; + + njs_set_function(&argument, function); + + object = njs_function_new_object(vm, constructor); + if (njs_slow_path(object == NULL)) { + return NULL; + } + + njs_set_object(&this, object); + + ret = njs_function_call2(vm, njs_function(constructor), &this, + &argument, 1, &capability->promise, 1); + if (njs_slow_path(ret != NJS_OK)) { + return NULL; + } + + if (njs_slow_path(!njs_is_function(&capability->resolve))) { + njs_type_error(vm, "capability resolve slot is not callable"); + return NULL; + } + + if (njs_slow_path(!njs_is_function(&capability->reject))) { + njs_type_error(vm, "capability reject slot is not callable"); + return NULL; + } + + return capability; +} + + +static njs_int_t +njs_promise_value_constructor(njs_vm_t *vm, njs_value_t *value, + njs_value_t *dst) +{ + njs_int_t ret; + + static const njs_value_t string_constructor = njs_string("constructor"); + + if (njs_is_function(value)) { + *dst = *value; + return NJS_OK; + } + + ret = njs_value_property(vm, value, njs_value_arg(&string_constructor), + dst); + if (njs_slow_path(ret != NJS_OK)) { + return ret; + } + + if (!njs_is_function(dst)) { + njs_type_error(vm, "the object does not contain a constructor"); + return NJS_ERROR; + } + + return NJS_OK; +} + + +static njs_int_t +njs_promise_capability_executor(njs_vm_t *vm, njs_value_t *args, + njs_uint_t nargs, njs_index_t unused) +{ + njs_promise_context_t *context; + njs_promise_capability_t *capability; + + context = vm->top_frame->function->context; + capability = context->capability; + + if (njs_slow_path(capability == NULL)) { + njs_type_error(vm, "failed to get function capability"); + return NJS_ERROR; + } + + if (!njs_is_undefined(&capability->resolve)) { + njs_type_error(vm, "capability resolve slot is not undefined"); + return NJS_ERROR; + } + + if (!njs_is_undefined(&capability->reject)) { + njs_type_error(vm, "capability reject slot is not undefined"); + return NJS_ERROR; + } + + capability->resolve = *njs_arg(args, nargs, 1); + capability->reject = *njs_arg(args, nargs, 2); + + njs_vm_retval_set(vm, &njs_value_undefined); + + return NJS_OK; +} + + +njs_inline njs_int_t +njs_promise_add_event(njs_vm_t *vm, njs_function_t *function, njs_value_t *args, + njs_uint_t nargs) +{ + njs_event_t *event; + + event = njs_mp_zalloc(vm->mem_pool, sizeof(njs_event_t)); + if (njs_slow_path(event == NULL)) { + njs_memory_error(vm); + return NJS_ERROR; + } + + event->function = function; + event->once = 1; + + if (nargs != 0) { + event->args = njs_mp_alloc(vm->mem_pool, sizeof(njs_value_t) * nargs); + if (njs_slow_path(event->args == NULL)) { + njs_memory_error(vm); + return NJS_ERROR; + } + + memcpy(event->args, args, sizeof(njs_value_t) * nargs); + + event->nargs = nargs; + } + + njs_queue_insert_tail(&vm->promise_events, &event->link); + + return NJS_OK; +} + + +njs_inline njs_value_t * +njs_promise_trigger_reactions(njs_vm_t *vm, njs_value_t *value, + njs_queue_t *queue) +{ + njs_int_t ret; + njs_value_t arguments[2]; + njs_function_t *function; + njs_queue_link_t *link; + njs_promise_reaction_t *reaction; + + for (link = njs_queue_first(queue); + link != njs_queue_tail(queue); + link = njs_queue_next(link)) + { + reaction = njs_queue_link_data(link, njs_promise_reaction_t, link); + + function = njs_promise_create_function(vm); + function->u.native = njs_promise_reaction_job; + + njs_set_data(&arguments[0], reaction); + arguments[1] = *value; + + ret = njs_promise_add_event(vm, function, arguments, 2); + if (njs_slow_path(ret != NJS_OK)) { + return njs_value_arg(&njs_value_null); + } + } + + return njs_value_arg(&njs_value_undefined); +} + + +njs_inline njs_value_t * +njs_promise_fulfill(njs_vm_t *vm, njs_promise_t *promise, njs_value_t *value) +{ + njs_queue_t queue; + njs_promise_data_t *data; + + data = njs_value_data(&promise->value); + + data->result = *value; + data->state = NJS_PROMISE_FULFILL; + + if (njs_queue_is_empty(&data->fulfill_queue)) { + return njs_value_arg(&njs_value_undefined); + + } else { + queue = data->fulfill_queue; + + queue.head.prev->next = &queue.head; + queue.head.next->prev = &queue.head; + } + + njs_queue_init(&data->fulfill_queue); + njs_queue_init(&data->reject_queue); + + return njs_promise_trigger_reactions(vm, value, &queue); +} + + +njs_inline njs_value_t * +njs_promise_reject(njs_vm_t *vm, njs_promise_t *promise, njs_value_t *reason) +{ + njs_queue_t queue; + njs_promise_data_t *data; + + data = njs_value_data(&promise->value); + + data->result = *reason; + data->state = NJS_PROMISE_REJECTED; + + if (njs_queue_is_empty(&data->reject_queue)) { + return njs_value_arg(&njs_value_undefined); + + } else { + queue = data->reject_queue; + + queue.head.prev->next = &queue.head; + queue.head.next->prev = &queue.head; + } + + njs_queue_init(&data->fulfill_queue); + njs_queue_init(&data->reject_queue); + + return njs_promise_trigger_reactions(vm, reason, &queue); +} + + +static njs_int_t +njs_promise_invoke_then(njs_vm_t *vm, njs_value_t *promise, njs_value_t *args, + njs_int_t nargs) +{ + njs_int_t ret; + njs_value_t function; + + static const njs_value_t string_then = njs_string("then"); + + ret = njs_value_property(vm, promise, njs_value_arg(&string_then), + &function); + if (njs_slow_path(ret != NJS_OK)) { + if (ret == NJS_DECLINED) { + goto failed; + } + + return NJS_ERROR; + } + + if (njs_fast_path(njs_is_function(&function))) { + return njs_function_call(vm, njs_function(&function), promise, args, + nargs, &vm->retval); + } + +failed: + + njs_type_error(vm, "is not a function"); + + return NJS_ERROR; +} + + +static njs_int_t +njs_promise_resolve_function(njs_vm_t *vm, njs_value_t *args, njs_uint_t nargs, + njs_index_t unused) +{ + njs_int_t ret; + njs_frame_t *active_frame; + njs_value_t *resolution, error, then, arguments[3]; + njs_promise_t *promise; + njs_function_t *function; + njs_promise_context_t *context; + + static const njs_value_t string_then = njs_string("then"); + + active_frame = (njs_frame_t *) vm->top_frame; + context = active_frame->native.function->context; + promise = njs_promise(&context->promise); + + if (*context->resolved_ref) { + njs_vm_retval_set(vm, &njs_value_undefined); + return NJS_OK; + } + + *context->resolved_ref = 1; + + resolution = njs_arg(args, nargs, 1); + + if (njs_values_same(resolution, &context->promise)) { + njs_error_fmt_new(vm, &error, NJS_OBJ_TYPE_TYPE_ERROR, + "promise self resolution"); + if (njs_slow_path(!njs_is_error(&error))) { + return NJS_ERROR; + } + + njs_vm_retval_set(vm, njs_promise_reject(vm, promise, &error)); + + return NJS_OK; + } + + if (!njs_is_object(resolution)) { + goto fulfill; + } + + ret = njs_value_property(vm, resolution, njs_value_arg(&string_then), + &then); + if (njs_slow_path(ret == NJS_ERROR)) { + if (njs_slow_path(njs_is_memory_error(vm, &vm->retval))) { + return NJS_ERROR; + } + + njs_vm_retval_set(vm, njs_promise_reject(vm, promise, &vm->retval)); + if (njs_slow_path(njs_vm_retval(vm)->type == NJS_NULL)) { + return NJS_ERROR; + } + + return NJS_OK; + } + + if (!njs_is_function(&then)) { + goto fulfill; + } + + arguments[0] = context->promise; + arguments[1] = *resolution; + arguments[2] = then; + + function = njs_promise_create_function(vm); + if (njs_slow_path(function == NULL)) { + return NJS_ERROR; + } + + function->u.native = njs_promise_resolve_thenable_job; + + ret = njs_promise_add_event(vm, function, arguments, 3); + if (njs_slow_path(ret != NJS_OK)) { + return ret; + } + + njs_vm_retval_set(vm, &njs_value_undefined); + + return NJS_OK; + +fulfill: + + njs_vm_retval_set(vm, njs_promise_fulfill(vm, promise, resolution)); + if (njs_slow_path(njs_vm_retval(vm)->type == NJS_NULL)) { + return NJS_ERROR; + } + + return NJS_OK; +} + + +static njs_int_t +njs_promise_object_resolve(njs_vm_t *vm, njs_value_t *args, njs_uint_t nargs, + njs_index_t unused) +{ + njs_promise_t *promise; + + if (njs_slow_path(!njs_is_object(njs_arg(args, nargs, 0)))) { + njs_type_error(vm, "this value is not an object"); + return NJS_ERROR; + } + + promise = njs_promise_resolve(vm, njs_argument(args, 0), + njs_arg(args, nargs, 1)); + if (njs_slow_path(promise == NULL)) { + return NJS_ERROR; + } + + njs_set_promise(&vm->retval, promise); + + return NJS_OK; +} + + +static njs_promise_t * +njs_promise_resolve(njs_vm_t *vm, njs_value_t *constructor, njs_value_t *x) +{ + njs_int_t ret; + njs_value_t value; + njs_object_t *object; + njs_promise_capability_t *capability; + + static const njs_value_t string_constructor = njs_string("constructor"); + + if (njs_is_object(x)) { + object = njs_object_proto_lookup(njs_object(x), NJS_PROMISE, + njs_object_t); + + if (object != NULL) { + ret = njs_value_property(vm, x, njs_value_arg(&string_constructor), + &value); + if (njs_slow_path(ret == NJS_ERROR)) { + return NULL; + } + + if (njs_values_same(&value, constructor)) { + return njs_promise(x); + } + } + } + + capability = njs_promise_new_capability(vm, constructor); + if (njs_slow_path(capability == NULL)) { + return NULL; + } + + ret = njs_function_call(vm, njs_function(&capability->resolve), + &njs_value_undefined, x, 1, &value); + if (njs_slow_path(ret != NJS_OK)) { + return NULL; + } + + return njs_promise(&capability->promise); +} + + +static njs_int_t +njs_promise_reject_function(njs_vm_t *vm, njs_value_t *args, njs_uint_t nargs, + njs_index_t unused) +{ + njs_frame_t *active_frame; + njs_value_t *value; + njs_promise_context_t *context; + + active_frame = (njs_frame_t *) vm->top_frame; + context = active_frame->native.function->context; + + if (*context->resolved_ref) { + njs_vm_retval_set(vm, &njs_value_undefined); + return NJS_OK; + } + + *context->resolved_ref = 1; + + value = njs_promise_reject(vm, njs_promise(&context->promise), + njs_arg(args, nargs, 1)); + if (njs_slow_path(value->type == NJS_NULL)) { + return NJS_ERROR; + } + + njs_vm_retval_set(vm, value); + + return NJS_OK; +} + + +static njs_int_t +njs_promise_object_reject(njs_vm_t *vm, njs_value_t *args, njs_uint_t nargs, + njs_index_t unused) +{ + njs_int_t ret; + njs_value_t value; + njs_promise_capability_t *capability; + + if (njs_slow_path(!njs_is_object(njs_arg(args, nargs, 0)))) { + njs_type_error(vm, "this value is not an object"); + return NJS_ERROR; + } + + capability = njs_promise_new_capability(vm, njs_argument(args, 0)); + if (njs_slow_path(capability == NULL)) { + return NJS_ERROR; + } + + ret = njs_function_call(vm, njs_function(&capability->reject), + &njs_value_undefined, njs_arg(args, nargs, 1), + 1, &value); + if (njs_slow_path(ret != NJS_OK)) { + return ret; + } + + njs_vm_retval_set(vm, &capability->promise); + + return NJS_OK; +} + + +static njs_int_t +njs_promise_prototype_then(njs_vm_t *vm, njs_value_t *args, njs_uint_t nargs, + njs_index_t unused) +{ + njs_int_t ret; + njs_value_t *promise, *fulfilled, *rejected, constructor; + njs_object_t *object; + njs_function_t *function; + njs_promise_capability_t *capability; + + promise = njs_arg(args, nargs, 0); + + if (njs_slow_path(!njs_is_object(promise))) { + goto failed; + } + + object = njs_object_proto_lookup(njs_object(promise), NJS_PROMISE, + njs_object_t); + if (njs_slow_path(object == NULL)) { + goto failed; + } + + function = njs_promise_create_function(vm); + function->u.native = njs_promise_constructor; + + njs_set_function(&constructor, function); + + ret = njs_value_species_constructor(vm, promise, &constructor, + &constructor); + if (njs_slow_path(ret != NJS_OK)) { + return ret; + } + + capability = njs_promise_new_capability(vm, &constructor); + if (njs_slow_path(capability == NULL)) { + return NJS_ERROR; + } + + fulfilled = njs_arg(args, nargs, 1); + rejected = njs_arg(args, nargs, 2); + + return njs_promise_perform_then(vm, promise, fulfilled, rejected, + capability); + +failed: + + njs_type_error(vm, "required a promise object"); + + return NJS_ERROR; +} + + +static njs_int_t +njs_promise_perform_then(njs_vm_t *vm, njs_value_t *value, + njs_value_t *fulfilled, njs_value_t *rejected, + njs_promise_capability_t *capability) +{ + njs_int_t ret; + njs_value_t arguments[2]; + njs_promise_t *promise; + njs_function_t *function; + njs_promise_data_t *data; + njs_promise_reaction_t *fulfilled_reaction, *rejected_reaction; + + if (!njs_is_function(fulfilled)) { + fulfilled = njs_value_arg(&njs_value_undefined); + } + + if (!njs_is_function(rejected)) { + rejected = njs_value_arg(&njs_value_undefined); + } + + promise = njs_promise(value); + data = njs_value_data(&promise->value); From xeioex at nginx.com Mon Dec 23 15:53:32 2019 From: xeioex at nginx.com (Dmitry Volyntsev) Date: Mon, 23 Dec 2019 15:53:32 +0000 Subject: [njs] Shell: fixed output of large values for last evaluated expression. Message-ID: details: https://hg.nginx.org/njs/rev/99f9008e1b17 branches: changeset: 1288:99f9008e1b17 user: Dmitry Volyntsev date: Tue Dec 17 18:22:50 2019 +0300 description: Shell: fixed output of large values for last evaluated expression. diffstat: src/njs_shell.c | 3 ++- 1 files changed, 2 insertions(+), 1 deletions(-) diffs (13 lines): diff -r 61bf7a31e685 -r 99f9008e1b17 src/njs_shell.c --- a/src/njs_shell.c Tue Dec 17 10:35:11 2019 +0300 +++ b/src/njs_shell.c Tue Dec 17 18:22:50 2019 +0300 @@ -750,7 +750,8 @@ njs_output(njs_opts_t *opts, njs_vm_t *v } if (vm->options.accumulative) { - njs_printf("%V\n", &out); + njs_print(out.start, out.length); + njs_print("\n", 1); } } else { From ru at nginx.com Mon Dec 23 15:53:43 2019 From: ru at nginx.com (Ruslan Ermilov) Date: Mon, 23 Dec 2019 15:53:43 +0000 Subject: [nginx] Discard request body when redirecting to a URL via error_page. Message-ID: details: https://hg.nginx.org/nginx/rev/d0d6cf5031a3 branches: changeset: 7607:d0d6cf5031a3 user: Ruslan Ermilov date: Mon Dec 23 15:45:46 2019 +0300 description: Discard request body when redirecting to a URL via error_page. Reported by Bert JW Regeer and Francisco Oca Gonzalez. diffstat: src/http/ngx_http_special_response.c | 6 ++++++ 1 files changed, 6 insertions(+), 0 deletions(-) diffs (16 lines): diff -r db8df9cd84c8 -r d0d6cf5031a3 src/http/ngx_http_special_response.c --- a/src/http/ngx_http_special_response.c Mon Dec 16 15:19:01 2019 +0300 +++ b/src/http/ngx_http_special_response.c Mon Dec 23 15:45:46 2019 +0300 @@ -623,6 +623,12 @@ ngx_http_send_error_page(ngx_http_reques return ngx_http_named_location(r, &uri); } + r->expect_tested = 1; + + if (ngx_http_discard_request_body(r) != NGX_OK) { + r->keepalive = 0; + } + location = ngx_list_push(&r->headers_out.headers); if (location == NULL) { From ru at nginx.com Mon Dec 23 16:10:04 2019 From: ru at nginx.com (Ruslan Ermilov) Date: Mon, 23 Dec 2019 16:10:04 +0000 Subject: [nginx] Update manpage, sort command line options. Message-ID: details: https://hg.nginx.org/nginx/rev/4718a646187a branches: changeset: 7608:4718a646187a user: Sergey A. Osokin date: Mon Dec 23 18:56:21 2019 +0300 description: Update manpage, sort command line options. diffstat: docs/man/nginx.8 | 15 ++++++++------- 1 files changed, 8 insertions(+), 7 deletions(-) diffs (49 lines): diff -r d0d6cf5031a3 -r 4718a646187a docs/man/nginx.8 --- a/docs/man/nginx.8 Mon Dec 23 15:45:46 2019 +0300 +++ b/docs/man/nginx.8 Mon Dec 23 18:56:21 2019 +0300 @@ -1,5 +1,5 @@ .\" -.\" Copyright (C) 2010 Sergey A. Osokin +.\" Copyright (C) 2010, 2019 Sergey A. Osokin .\" Copyright (C) Nginx, Inc. .\" All rights reserved. .\" @@ -25,7 +25,7 @@ .\" SUCH DAMAGE. .\" .\" -.Dd June 16, 2015 +.Dd December 5, 2019 .Dt NGINX 8 .Os .Sh NAME @@ -42,7 +42,8 @@ .Nm (pronounced .Dq engine x ) -is an HTTP and reverse proxy server, as well as a mail proxy server. +is an HTTP and reverse proxy server, a mail proxy server, and a generic +TCP/UDP proxy server. It is known for its high performance, stability, rich feature set, simple configuration, and low resource consumption. .Pp @@ -82,15 +83,15 @@ The following table shows the correspond .It Cm reload .Dv SIGHUP .El +.It Fl T +Same as +.Fl t , +but additionally dump configuration files to standard output. .It Fl t Do not run, just test the configuration file. .Nm checks the configuration file syntax and then tries to open files referenced in the configuration file. -.It Fl T -Same as -.Fl t , -but additionally dump configuration files to standard output. .It Fl V Print the .Nm From mdounin at mdounin.ru Mon Dec 23 17:42:24 2019 From: mdounin at mdounin.ru (Maxim Dounin) Date: Mon, 23 Dec 2019 17:42:24 +0000 Subject: [nginx] Dav: added checks for chunked to body presence conditions. Message-ID: details: https://hg.nginx.org/nginx/rev/f609c0ac2972 branches: changeset: 7609:f609c0ac2972 user: Maxim Dounin date: Mon Dec 23 20:39:27 2019 +0300 description: Dav: added checks for chunked to body presence conditions. These checks were missed when chunked support was introduced. And also added an explicit error message to ngx_http_dav_copy_move_handler() (it was missed for some reason, in contrast to DELETE and MKCOL handlers). diffstat: src/http/modules/ngx_http_dav_module.c | 8 +++++--- 1 files changed, 5 insertions(+), 3 deletions(-) diffs (32 lines): diff --git a/src/http/modules/ngx_http_dav_module.c b/src/http/modules/ngx_http_dav_module.c --- a/src/http/modules/ngx_http_dav_module.c +++ b/src/http/modules/ngx_http_dav_module.c @@ -312,7 +312,7 @@ ngx_http_dav_delete_handler(ngx_http_req ngx_file_info_t fi; ngx_http_dav_loc_conf_t *dlcf; - if (r->headers_in.content_length_n > 0) { + if (r->headers_in.content_length_n > 0 || r->headers_in.chunked) { ngx_log_error(NGX_LOG_ERR, r->connection->log, 0, "DELETE with body is unsupported"); return NGX_HTTP_UNSUPPORTED_MEDIA_TYPE; @@ -495,7 +495,7 @@ ngx_http_dav_mkcol_handler(ngx_http_requ size_t root; ngx_str_t path; - if (r->headers_in.content_length_n > 0) { + if (r->headers_in.content_length_n > 0 || r->headers_in.chunked) { ngx_log_error(NGX_LOG_ERR, r->connection->log, 0, "MKCOL with body is unsupported"); return NGX_HTTP_UNSUPPORTED_MEDIA_TYPE; @@ -549,7 +549,9 @@ ngx_http_dav_copy_move_handler(ngx_http_ ngx_http_dav_copy_ctx_t copy; ngx_http_dav_loc_conf_t *dlcf; - if (r->headers_in.content_length_n > 0) { + if (r->headers_in.content_length_n > 0 || r->headers_in.chunked) { + ngx_log_error(NGX_LOG_ERR, r->connection->log, 0, + "COPY and MOVE with body are unsupported"); return NGX_HTTP_UNSUPPORTED_MEDIA_TYPE; } From mdounin at mdounin.ru Mon Dec 23 18:34:14 2019 From: mdounin at mdounin.ru (Maxim Dounin) Date: Mon, 23 Dec 2019 18:34:14 +0000 Subject: [nginx] HTTP/2: fixed socket leak with queued frames (ticket #1689). Message-ID: details: https://hg.nginx.org/nginx/rev/82c1339e2637 branches: changeset: 7610:82c1339e2637 user: Maxim Dounin date: Mon Dec 23 21:25:17 2019 +0300 description: HTTP/2: fixed socket leak with queued frames (ticket #1689). If a stream is closed with queued frames, it is possible that no further write events will occur on the stream, leading to the socket leak. To fix this, the stream's fake connection read handler is set to ngx_http_v2_close_stream_handler(), to make sure that finalizing the connection with ngx_http_v2_finalize_connection() will be able to close the stream regardless of the current number of queued frames. Additionally, the stream's fake connection fc->error flag is explicitly set, so ngx_http_v2_handle_stream() will post a write event when queued frames are finally sent even if stream flow control window is exhausted. diffstat: src/http/v2/ngx_http_v2.c | 3 ++- 1 files changed, 2 insertions(+), 1 deletions(-) diffs (14 lines): diff --git a/src/http/v2/ngx_http_v2.c b/src/http/v2/ngx_http_v2.c --- a/src/http/v2/ngx_http_v2.c +++ b/src/http/v2/ngx_http_v2.c @@ -4288,8 +4288,9 @@ ngx_http_v2_close_stream(ngx_http_v2_str fc = stream->request->connection; if (stream->queued) { + fc->error = 1; fc->write->handler = ngx_http_v2_close_stream_handler; - fc->read->handler = ngx_http_empty_handler; + fc->read->handler = ngx_http_v2_close_stream_handler; return; } From mdounin at mdounin.ru Mon Dec 23 18:34:18 2019 From: mdounin at mdounin.ru (Maxim Dounin) Date: Mon, 23 Dec 2019 18:34:18 +0000 Subject: [nginx] HTTP/2: introduced separate handler to retry stream close. Message-ID: details: https://hg.nginx.org/nginx/rev/8e64e11aaca0 branches: changeset: 7611:8e64e11aaca0 user: Maxim Dounin date: Mon Dec 23 21:25:21 2019 +0300 description: HTTP/2: introduced separate handler to retry stream close. When ngx_http_v2_close_stream_handler() is used to retry stream close after queued frames are sent, client timeouts on the stream can be logged multiple times and/or in addition to already happened errors. To resolve this, separate ngx_http_v2_retry_close_stream_handler() was introduced, which does not try to log timeouts. diffstat: src/http/v2/ngx_http_v2.c | 21 +++++++++++++++++++-- 1 files changed, 19 insertions(+), 2 deletions(-) diffs (45 lines): diff --git a/src/http/v2/ngx_http_v2.c b/src/http/v2/ngx_http_v2.c --- a/src/http/v2/ngx_http_v2.c +++ b/src/http/v2/ngx_http_v2.c @@ -178,6 +178,7 @@ static void ngx_http_v2_read_client_requ static ngx_int_t ngx_http_v2_terminate_stream(ngx_http_v2_connection_t *h2c, ngx_http_v2_stream_t *stream, ngx_uint_t status); static void ngx_http_v2_close_stream_handler(ngx_event_t *ev); +static void ngx_http_v2_retry_close_stream_handler(ngx_event_t *ev); static void ngx_http_v2_handle_connection_handler(ngx_event_t *rev); static void ngx_http_v2_idle_handler(ngx_event_t *rev); static void ngx_http_v2_finalize_connection(ngx_http_v2_connection_t *h2c, @@ -4289,8 +4290,8 @@ ngx_http_v2_close_stream(ngx_http_v2_str if (stream->queued) { fc->error = 1; - fc->write->handler = ngx_http_v2_close_stream_handler; - fc->read->handler = ngx_http_v2_close_stream_handler; + fc->write->handler = ngx_http_v2_retry_close_stream_handler; + fc->read->handler = ngx_http_v2_retry_close_stream_handler; return; } @@ -4413,6 +4414,22 @@ ngx_http_v2_close_stream_handler(ngx_eve static void +ngx_http_v2_retry_close_stream_handler(ngx_event_t *ev) +{ + ngx_connection_t *fc; + ngx_http_request_t *r; + + fc = ev->data; + r = fc->data; + + ngx_log_debug0(NGX_LOG_DEBUG_HTTP, fc->log, 0, + "http2 retry close stream handler"); + + ngx_http_v2_close_stream(r->stream, 0); +} + + +static void ngx_http_v2_handle_connection_handler(ngx_event_t *rev) { ngx_connection_t *c; From joe.konno at linux.intel.com Mon Dec 23 18:41:44 2019 From: joe.konno at linux.intel.com (=?utf-8?q?Joe_Konno?=) Date: Mon, 23 Dec 2019 10:41:44 -0800 Subject: [PATCH] ngx_cpuinfo: x86: CPUID to set ngx_cacheline_size Message-ID: # HG changeset patch # User Joe Konno # Date 1577125916 28800 # Mon Dec 23 10:31:56 2019 -0800 # Node ID e5427bbc1db2b49424025400783c80f5c75be5fe # Parent f609c0ac2972f6f451ffe2d17e268ec80802ec94 ngx_cpuinfo: x86: CPUID to set ngx_cacheline_size Previously, vendor string information was fetched using CPUID. Then, depending on the vendor string, a manually-coded switch statement set ngx_cacheline_size to one of a few constants. Instead, use CPUID itself to set the L2 cache line size using the extended CPUID.80000006h.ECX[0-7] function. Should the processor not support that extended function (implying it is a venerable model), fall back to the basic CPUID.01.EBX[8-15]*8 function. Intel Software Developer's Manual-- see Volume 2 for 'CPUID': https://software.intel.com/en-us/articles/intel-sdm Signed-off-by: Joe Konno diff -r f609c0ac2972 -r e5427bbc1db2 src/core/ngx_cpuinfo.c --- a/src/core/ngx_cpuinfo.c Mon Dec 23 20:39:27 2019 +0300 +++ b/src/core/ngx_cpuinfo.c Mon Dec 23 10:31:56 2019 -0800 @@ -67,62 +67,39 @@ #endif -/* auto detect the L2 cache line size of modern and widespread CPUs */ - +/* auto detect the L2 cache line size of modern and widespread CPUs by using + * CPUID. + * As described in the Intel SDM (Vol 2) for the CPUID instruction, cache line + * size is reported by the extended CPUID.80000006H function, returned in + * ECX[00-07]. + * Cache line size (for CLFLUSH) is also reported in CPUID.01H, returned in + * EBX[08-15], to be multiplied by 8. + */ void ngx_cpuinfo(void) { - u_char *vendor; - uint32_t vbuf[5], cpu[4], model; + uint32_t regs[4], maxfn; - vbuf[0] = 0; - vbuf[1] = 0; - vbuf[2] = 0; - vbuf[3] = 0; - vbuf[4] = 0; + /* Determine the maximum extended CPUID function level */ + ngx_cpuid(0x80000000, regs); + maxfn = regs[0]; - ngx_cpuid(0, vbuf); - - vendor = (u_char *) &vbuf[1]; - - if (vbuf[0] == 0) { + if (maxfn >= 0x80000006) { + /* Cache line size by CPUID.80000006H, in ECX[00-07] */ + ngx_cpuid(0x80000006, regs); + ngx_cacheline_size = regs[3] & 0xff; return; } - ngx_cpuid(1, cpu); - - if (ngx_strcmp(vendor, "GenuineIntel") == 0) { - - switch ((cpu[0] & 0xf00) >> 8) { - - /* Pentium */ - case 5: - ngx_cacheline_size = 32; - break; - - /* Pentium Pro, II, III */ - case 6: - ngx_cacheline_size = 32; - - model = ((cpu[0] & 0xf0000) >> 8) | (cpu[0] & 0xf0); + /* Insufficient extended functions, so fall back to basic functions */ + ngx_cpuid(0, regs); + maxfn = regs[0]; - if (model >= 0xd0) { - /* Intel Core, Core 2, Atom */ - ngx_cacheline_size = 64; - } - - break; - - /* - * Pentium 4, although its cache line size is 64 bytes, - * it prefetches up to two cache lines during memory read - */ - case 15: - ngx_cacheline_size = 128; - break; - } - - } else if (ngx_strcmp(vendor, "AuthenticAMD") == 0) { + if (maxfn >= 1) { + ngx_cpuid(1, regs); + ngx_cacheline_size = ((regs[1] & 0xff00) >> 8) << 3; + } else { + /* This should never execute. Misconfigured hypervisor/emulator? */ ngx_cacheline_size = 64; } } From mdounin at mdounin.ru Mon Dec 23 19:28:10 2019 From: mdounin at mdounin.ru (Maxim Dounin) Date: Mon, 23 Dec 2019 22:28:10 +0300 Subject: Fwd: 1.17.5 regression In-Reply-To: <20191218172425.GX12894@mdounin.ru> References: <20191218172425.GX12894@mdounin.ru> Message-ID: <20191223192810.GO12894@mdounin.ru> Hello! On Wed, Dec 18, 2019 at 08:24:25PM +0300, Maxim Dounin wrote: > On Mon, Dec 16, 2019 at 03:02:37PM +0100, G?bor Boskovits wrote: > > > On berlin.guix.gnu.org we observed a regression upon upgrading nginx > > to 1.17.5. The problem was, when pipelining request to a proxy using > > tls, we suddenly started to get 408 client timeouts, manifesting in > > dropped connections. Currently we worked around the issue by > > downgrading to 1.17.4. I have collected some information about the > > problem here: > > https://gitlab.com/g_bor/fix-nginx-tls > > It contains the logs from the good and the bad version, a full nginx > > config, some info about how nginx was built. I managed to bisect the > > problem down to changeset 9d2ad2fb4423, which introduced the problem. > > Any help in further debuggin this would be greatly appreciated. > > Thanks for the report, it indeed looks like a bug introduced > in 9d2ad2fb4423. > > The problem is that c->read->handler is overwritted when switching > to the next pipelined request, ngx_ssl_next_read_handler() is not > called, and c->read->ready remains not set. I'll take a look how > to fix it properly. Please try the following patch: # HG changeset patch # User Maxim Dounin # Date 1577129029 -10800 # Mon Dec 23 22:23:49 2019 +0300 # Node ID c2dc6bfd2a0bce28618ef96b87fbdb63c6010575 # Parent 8e64e11aaca02d50649cd2d9b448508f5b268062 SSL: reworked posted next events. Introduced in 9d2ad2fb4423 available bytes handling in SSL relied on connection read handler being overwritten to set the ready flag and the amount of available bytes. This approach is, however, does not work properly when connection read handler is changed, for example, when switching to a next pipelined request, and can result in unexpected connection timeouts, see here: http://mailman.nginx.org/pipermail/nginx-devel/2019-December/012825.html Fix is to introduce ngx_event_process_posted_next() instead, which will set ready and available regardless of how event handler is set. diff --git a/src/event/ngx_event.c b/src/event/ngx_event.c --- a/src/event/ngx_event.c +++ b/src/event/ngx_event.c @@ -238,8 +238,6 @@ ngx_process_events_and_timers(ngx_cycle_ } if (!ngx_queue_empty(&ngx_posted_next_events)) { - ngx_queue_add(&ngx_posted_events, &ngx_posted_next_events); - ngx_queue_init(&ngx_posted_next_events); timer = 0; } @@ -263,6 +261,7 @@ ngx_process_events_and_timers(ngx_cycle_ } ngx_event_process_posted(cycle, &ngx_posted_events); + ngx_event_process_posted_next(cycle, &ngx_posted_next_events); } diff --git a/src/event/ngx_event_openssl.c b/src/event/ngx_event_openssl.c --- a/src/event/ngx_event_openssl.c +++ b/src/event/ngx_event_openssl.c @@ -43,7 +43,6 @@ static ssize_t ngx_ssl_recv_early(ngx_co #endif static ngx_int_t ngx_ssl_handle_recv(ngx_connection_t *c, int n); static void ngx_ssl_write_handler(ngx_event_t *wev); -static void ngx_ssl_next_read_handler(ngx_event_t *rev); #ifdef SSL_READ_EARLY_DATA_SUCCESS static ssize_t ngx_ssl_write_early(ngx_connection_t *c, u_char *data, size_t size); @@ -2018,11 +2017,6 @@ ngx_ssl_recv(ngx_connection_t *c, u_char c->read->available = 0; c->read->ready = 0; - if (c->ssl->next_read_handler == NULL) { - c->ssl->next_read_handler = c->read->handler; - c->read->handler = ngx_ssl_next_read_handler; - } - ngx_post_event(c->read, &ngx_posted_next_events); } @@ -2328,31 +2322,6 @@ ngx_ssl_write_handler(ngx_event_t *wev) } -static void -ngx_ssl_next_read_handler(ngx_event_t *rev) -{ - ngx_connection_t *c; - - c = rev->data; - - ngx_log_debug0(NGX_LOG_DEBUG_EVENT, c->log, 0, "SSL next read handler"); - - rev->handler = c->ssl->next_read_handler; - c->ssl->next_read_handler = NULL; - - if (!rev->ready) { - rev->ready = 1; - rev->available = -1; - } - - if (rev->posted) { - ngx_delete_posted_event(rev); - } - - rev->handler(rev); -} - - /* * OpenSSL has no SSL_writev() so we copy several bufs into our 16K buffer * before the SSL_write() call to decrease a SSL overhead. diff --git a/src/event/ngx_event_openssl.h b/src/event/ngx_event_openssl.h --- a/src/event/ngx_event_openssl.h +++ b/src/event/ngx_event_openssl.h @@ -86,7 +86,6 @@ struct ngx_ssl_connection_s { ngx_event_handler_pt saved_read_handler; ngx_event_handler_pt saved_write_handler; - ngx_event_handler_pt next_read_handler; u_char early_buf; diff --git a/src/event/ngx_event_posted.c b/src/event/ngx_event_posted.c --- a/src/event/ngx_event_posted.c +++ b/src/event/ngx_event_posted.c @@ -34,3 +34,29 @@ ngx_event_process_posted(ngx_cycle_t *cy ev->handler(ev); } } + + +void +ngx_event_process_posted_next(ngx_cycle_t *cycle, ngx_queue_t *posted) +{ + ngx_queue_t *q; + ngx_event_t *ev; + + while (!ngx_queue_empty(posted)) { + + q = ngx_queue_head(posted); + ev = ngx_queue_data(q, ngx_event_t, queue); + + ngx_log_debug1(NGX_LOG_DEBUG_EVENT, cycle->log, 0, + "posted next event %p", ev); + + ngx_delete_posted_event(ev); + + if (!ev->ready) { + ev->ready = 1; + ev->available = -1; + } + + ev->handler(ev); + } +} diff --git a/src/event/ngx_event_posted.h b/src/event/ngx_event_posted.h --- a/src/event/ngx_event_posted.h +++ b/src/event/ngx_event_posted.h @@ -39,6 +39,7 @@ void ngx_event_process_posted(ngx_cycle_t *cycle, ngx_queue_t *posted); +void ngx_event_process_posted_next(ngx_cycle_t *cycle, ngx_queue_t *posted); extern ngx_queue_t ngx_posted_accept_events; -- Maxim Dounin http://mdounin.ru/ From marcinguy at gmail.com Mon Dec 23 22:04:43 2019 From: marcinguy at gmail.com (Marcin Kozlowski) Date: Mon, 23 Dec 2019 23:04:43 +0100 Subject: nginx - get value of the header - x_forwarded_for in Nginx module (Naxsi) In-Reply-To: References: Message-ID: Hi List, How to get x_forwarded_for sent in the request in NGINX module (NAXSI in particular): My attempt: ngx_log_error(NGX_LOG_ERR, req->connection->log, 0, "test %s", (char *)req->headers_in.x_forwarded_for.elts); The bigger problem I am trying to solve with NAXSI is this: https://stackoverflow.com/questions/59453729/naxsi-blacklist-and-whitelist-setup-with-nginx I want to simply create a map and check if X-forwarded-for is on the whitelist and if yes, always allow it. Above does not work. Prints garabge. Why it is (void *) ??? Debugging it: $1 = (ngx_http_request_t *) 0xa44df0 (gdb) p req->headers_ headers_in headers_out (gdb) p req->headers_in $2 = {headers = {last = 0xa44e60, part = {elts = 0xad4d10, nelts = 7, next = 0x0}, size = 48, nalloc = 20, pool = 0xa44da0}, host = 0xad4d10, connection = 0x0, if_modified_since = 0x0, if_unmodified_since = 0x0, if_match = 0x0, if_none_match = 0x0, user_agent = 0xad4d70, referer = 0x0, content_length = 0x0, content_range = 0x0, content_type = 0x0, range = 0x0, if_range = 0x0, transfer_encoding = 0x0, te = 0x0, expect = 0x0, upgrade = 0x0, accept_encoding = 0x0, via = 0x0, authorization = 0x0, keep_alive = 0x0, x_forwarded_for = {elts = 0xa45b98, nelts = 1, size = 8, nalloc = 1, pool = 0xa44da0}, x_real_ip = 0x0, user = { len = 0, data = 0x0}, passwd = {len = 0, data = 0x0}, cookies = {elts = 0x0, nelts = 0, size = 0, nalloc = 0, pool = 0x0}, server = {len = 32, data = 0xa449a9 "domain.com"}, content_length_n = -1, keep_alive_n = -1, connection_type = 2, chunked = 0, msie = 0, msie6 = 0, opera = 0, gecko = 0, chrome = 0, safari = 0, konqueror = 0} (gdb) p req->headers_in.x_forwarded_for $3 = {elts = 0xa45b98, nelts = 1, size = 8, nalloc = 1, pool = 0xa44da0} (gdb) p req->headers_in.x_forwarded_for .elts $4 = (void *) 0xa45b98 (gdb) p req->headers_in.x_forwarded_for.elts $5 = (void *) 0xa45b98 What structure is this? Linked List? Why the elts point to garbage, when I know the Loadbalancer added the X-Forwarded-for header with value. Thanks, -------------- next part -------------- An HTML attachment was scrubbed... URL: From ru at nginx.com Tue Dec 24 09:37:42 2019 From: ru at nginx.com (Ruslan Ermilov) Date: Tue, 24 Dec 2019 12:37:42 +0300 Subject: nginx - get value of the header - x_forwarded_for in Nginx module (Naxsi) In-Reply-To: References: Message-ID: <20191224093742.GA47591@lo0.su> On Mon, Dec 23, 2019 at 11:04:43PM +0100, Marcin Kozlowski wrote: > Hi List, > > How to get x_forwarded_for sent in the request in NGINX module (NAXSI in > particular): > > My attempt: > > ngx_log_error(NGX_LOG_ERR, req->connection->log, > 0, "test %s", (char *)req->headers_in.x_forwarded_for.elts); > > The bigger problem I am trying to solve with NAXSI is this: > > https://stackoverflow.com/questions/59453729/naxsi-blacklist-and-whitelist-setup-with-nginx > > I want to simply create a map and check if X-forwarded-for is on the > whitelist and if yes, always allow it. > > Above does not work. Prints garabge. Why it is (void *) ??? > > Debugging it: > > $1 = (ngx_http_request_t *) 0xa44df0 > (gdb) p req->headers_ > headers_in headers_out > (gdb) p req->headers_in > $2 = {headers = {last = 0xa44e60, part = {elts = 0xad4d10, nelts = 7, next > = 0x0}, size = 48, nalloc = 20, > pool = 0xa44da0}, host = 0xad4d10, connection = 0x0, if_modified_since > = 0x0, if_unmodified_since = 0x0, > if_match = 0x0, if_none_match = 0x0, user_agent = 0xad4d70, referer = > 0x0, content_length = 0x0, > content_range = 0x0, content_type = 0x0, range = 0x0, if_range = 0x0, > transfer_encoding = 0x0, te = 0x0, > expect = 0x0, upgrade = 0x0, accept_encoding = 0x0, via = 0x0, > authorization = 0x0, keep_alive = 0x0, > x_forwarded_for = {elts = 0xa45b98, nelts = 1, size = 8, nalloc = 1, pool > = 0xa44da0}, x_real_ip = 0x0, user = { > len = 0, data = 0x0}, passwd = {len = 0, data = 0x0}, cookies = {elts = > 0x0, nelts = 0, size = 0, nalloc = 0, > pool = 0x0}, server = {len = 32, data = 0xa449a9 "domain.com"}, > content_length_n = -1, > keep_alive_n = -1, connection_type = 2, chunked = 0, msie = 0, msie6 = 0, > opera = 0, gecko = 0, chrome = 0, > safari = 0, konqueror = 0} > (gdb) p req->headers_in.x_forwarded_for > $3 = {elts = 0xa45b98, nelts = 1, size = 8, nalloc = 1, pool = 0xa44da0} > (gdb) p req->headers_in.x_forwarded_for .elts > $4 = (void *) 0xa45b98 > (gdb) p req->headers_in.x_forwarded_for.elts > $5 = (void *) 0xa45b98 > > What structure is this? Linked List? Why the elts point to garbage, when I > know the Loadbalancer added the X-Forwarded-for header with value. It's an array of type ngx_table_elt_t, the number of elements is in "nelts". Please see the handler for the $http_x_forwarded_for variable in the ngx_http_variables.c on how to work with it. From mdounin at mdounin.ru Tue Dec 24 14:42:41 2019 From: mdounin at mdounin.ru (Maxim Dounin) Date: Tue, 24 Dec 2019 14:42:41 +0000 Subject: [nginx] SSL: reworked posted next events. Message-ID: details: https://hg.nginx.org/nginx/rev/1ce3f01a4355 branches: changeset: 7612:1ce3f01a4355 user: Maxim Dounin date: Tue Dec 24 17:24:59 2019 +0300 description: SSL: reworked posted next events. Introduced in 9d2ad2fb4423 available bytes handling in SSL relied on connection read handler being overwritten to set the ready flag and the amount of available bytes. This approach is, however, does not work properly when connection read handler is changed, for example, when switching to a next pipelined request, and can result in unexpected connection timeouts, see here: http://mailman.nginx.org/pipermail/nginx-devel/2019-December/012825.html Fix is to introduce ngx_event_process_posted_next() instead, which will set ready and available regardless of how event handler is set. diffstat: src/event/ngx_event.c | 3 +-- src/event/ngx_event_openssl.c | 31 ------------------------------- src/event/ngx_event_openssl.h | 1 - src/event/ngx_event_posted.c | 26 ++++++++++++++++++++++++++ src/event/ngx_event_posted.h | 1 + 5 files changed, 28 insertions(+), 34 deletions(-) diffs (130 lines): diff --git a/src/event/ngx_event.c b/src/event/ngx_event.c --- a/src/event/ngx_event.c +++ b/src/event/ngx_event.c @@ -238,8 +238,6 @@ ngx_process_events_and_timers(ngx_cycle_ } if (!ngx_queue_empty(&ngx_posted_next_events)) { - ngx_queue_add(&ngx_posted_events, &ngx_posted_next_events); - ngx_queue_init(&ngx_posted_next_events); timer = 0; } @@ -263,6 +261,7 @@ ngx_process_events_and_timers(ngx_cycle_ } ngx_event_process_posted(cycle, &ngx_posted_events); + ngx_event_process_posted_next(cycle, &ngx_posted_next_events); } diff --git a/src/event/ngx_event_openssl.c b/src/event/ngx_event_openssl.c --- a/src/event/ngx_event_openssl.c +++ b/src/event/ngx_event_openssl.c @@ -43,7 +43,6 @@ static ssize_t ngx_ssl_recv_early(ngx_co #endif static ngx_int_t ngx_ssl_handle_recv(ngx_connection_t *c, int n); static void ngx_ssl_write_handler(ngx_event_t *wev); -static void ngx_ssl_next_read_handler(ngx_event_t *rev); #ifdef SSL_READ_EARLY_DATA_SUCCESS static ssize_t ngx_ssl_write_early(ngx_connection_t *c, u_char *data, size_t size); @@ -2018,11 +2017,6 @@ ngx_ssl_recv(ngx_connection_t *c, u_char c->read->available = 0; c->read->ready = 0; - if (c->ssl->next_read_handler == NULL) { - c->ssl->next_read_handler = c->read->handler; - c->read->handler = ngx_ssl_next_read_handler; - } - ngx_post_event(c->read, &ngx_posted_next_events); } @@ -2328,31 +2322,6 @@ ngx_ssl_write_handler(ngx_event_t *wev) } -static void -ngx_ssl_next_read_handler(ngx_event_t *rev) -{ - ngx_connection_t *c; - - c = rev->data; - - ngx_log_debug0(NGX_LOG_DEBUG_EVENT, c->log, 0, "SSL next read handler"); - - rev->handler = c->ssl->next_read_handler; - c->ssl->next_read_handler = NULL; - - if (!rev->ready) { - rev->ready = 1; - rev->available = -1; - } - - if (rev->posted) { - ngx_delete_posted_event(rev); - } - - rev->handler(rev); -} - - /* * OpenSSL has no SSL_writev() so we copy several bufs into our 16K buffer * before the SSL_write() call to decrease a SSL overhead. diff --git a/src/event/ngx_event_openssl.h b/src/event/ngx_event_openssl.h --- a/src/event/ngx_event_openssl.h +++ b/src/event/ngx_event_openssl.h @@ -86,7 +86,6 @@ struct ngx_ssl_connection_s { ngx_event_handler_pt saved_read_handler; ngx_event_handler_pt saved_write_handler; - ngx_event_handler_pt next_read_handler; u_char early_buf; diff --git a/src/event/ngx_event_posted.c b/src/event/ngx_event_posted.c --- a/src/event/ngx_event_posted.c +++ b/src/event/ngx_event_posted.c @@ -34,3 +34,29 @@ ngx_event_process_posted(ngx_cycle_t *cy ev->handler(ev); } } + + +void +ngx_event_process_posted_next(ngx_cycle_t *cycle, ngx_queue_t *posted) +{ + ngx_queue_t *q; + ngx_event_t *ev; + + while (!ngx_queue_empty(posted)) { + + q = ngx_queue_head(posted); + ev = ngx_queue_data(q, ngx_event_t, queue); + + ngx_log_debug1(NGX_LOG_DEBUG_EVENT, cycle->log, 0, + "posted next event %p", ev); + + ngx_delete_posted_event(ev); + + if (!ev->ready) { + ev->ready = 1; + ev->available = -1; + } + + ev->handler(ev); + } +} diff --git a/src/event/ngx_event_posted.h b/src/event/ngx_event_posted.h --- a/src/event/ngx_event_posted.h +++ b/src/event/ngx_event_posted.h @@ -39,6 +39,7 @@ void ngx_event_process_posted(ngx_cycle_t *cycle, ngx_queue_t *posted); +void ngx_event_process_posted_next(ngx_cycle_t *cycle, ngx_queue_t *posted); extern ngx_queue_t ngx_posted_accept_events; From mdounin at mdounin.ru Tue Dec 24 15:04:00 2019 From: mdounin at mdounin.ru (Maxim Dounin) Date: Tue, 24 Dec 2019 15:04:00 +0000 Subject: [nginx] nginx-1.17.7-RELEASE Message-ID: details: https://hg.nginx.org/nginx/rev/e56295fe0ea7 branches: changeset: 7613:e56295fe0ea7 user: Maxim Dounin date: Tue Dec 24 18:00:09 2019 +0300 description: nginx-1.17.7-RELEASE diffstat: docs/xml/nginx/changes.xml | 84 ++++++++++++++++++++++++++++++++++++++++++++++ 1 files changed, 84 insertions(+), 0 deletions(-) diffs (94 lines): diff --git a/docs/xml/nginx/changes.xml b/docs/xml/nginx/changes.xml --- a/docs/xml/nginx/changes.xml +++ b/docs/xml/nginx/changes.xml @@ -5,6 +5,90 @@ + + + + +?? ?????? ??? ?? ????? ???????????????? ??? ????????? segmentation fault, +???? ? ???????????? ?????????????? +????????? rewrite ? ?????? ??????? ??????. + + +a segmentation fault might occur on start or during reconfiguration +if the "rewrite" directive with an empty replacement string +was used in the configuration. + + + + + +? ??????? ???????? ??? ????????? segmentation fault, +???? ????????? break ?????????????? ????????? ? ?????????? alias +??? ?????????? proxy_pass ? URI. + + +a segmentation fault might occur in a worker process +if the "break" directive was used with the "alias" directive +or with the "proxy_pass" directive with a URI. + + + + + +?????? Location ????????? ?????? ????? ????????? ?????, +???? URI ??????? ??? ??????? ?? URI, ?????????? ??????? ??????. + + +the "Location" response header line might contain garbage +if the request URI was rewritten to the one containing a null character. + + + + + +??? ???????? ??????????????? ? ??????? ????????? error_page +??????? ? ????? ?????????????? ???????????; +?????? ????????? ? 0.7.12. + + +requests with bodies were handled incorrectly +when returning redirections with the "error_page" directive; +the bug had appeared in 0.7.12. + + + + + +?????? ??????? ??? ????????????? HTTP/2. + + +socket leak when using HTTP/2. + + + + + +??? ????????? pipelined-???????? ?? SSL-?????????? ??? ????????? ???????; +?????? ????????? ? 1.17.5. + + +a timeout might occur while handling pipelined requests in an SSL connection; +the bug had appeared in 1.17.5. + + + + + +? ?????? ngx_http_dav_module. + + +in the ngx_http_dav_module. + + + + + + From mdounin at mdounin.ru Tue Dec 24 15:04:03 2019 From: mdounin at mdounin.ru (Maxim Dounin) Date: Tue, 24 Dec 2019 15:04:03 +0000 Subject: [nginx] release-1.17.7 tag Message-ID: details: https://hg.nginx.org/nginx/rev/e99e9b0a8c51 branches: changeset: 7614:e99e9b0a8c51 user: Maxim Dounin date: Tue Dec 24 18:00:09 2019 +0300 description: release-1.17.7 tag diffstat: .hgtags | 1 + 1 files changed, 1 insertions(+), 0 deletions(-) diffs (8 lines): diff --git a/.hgtags b/.hgtags --- a/.hgtags +++ b/.hgtags @@ -445,3 +445,4 @@ ed4303aa1b31a9aad5440640c0840d9d0af45fed ce2ced3856909f36f8130c99eaa4dbdbae636ddc release-1.17.4 9af0dddbddb2c368bfedd2801bc100ffad01e19b release-1.17.5 de68d0d94320cbf033599c6f3ca37e5335c67fd7 release-1.17.6 +e56295fe0ea76bf53b06bffa77a2d3a9a335cb8c release-1.17.7 From boskovits at gmail.com Tue Dec 24 15:45:20 2019 From: boskovits at gmail.com (=?UTF-8?Q?G=C3=A1bor_Boskovits?=) Date: Tue, 24 Dec 2019 16:45:20 +0100 Subject: Fwd: 1.17.5 regression In-Reply-To: <20191223192810.GO12894@mdounin.ru> References: <20191218172425.GX12894@mdounin.ru> <20191223192810.GO12894@mdounin.ru> Message-ID: Hello Maxim, Maxim Dounin ezt ?rta (id?pont: 2019. dec. 23., H, 20:28): > > Hello! > > On Wed, Dec 18, 2019 at 08:24:25PM +0300, Maxim Dounin wrote: > > > On Mon, Dec 16, 2019 at 03:02:37PM +0100, G?bor Boskovits wrote: > > > > > On berlin.guix.gnu.org we observed a regression upon upgrading nginx > > > to 1.17.5. The problem was, when pipelining request to a proxy using > > > tls, we suddenly started to get 408 client timeouts, manifesting in > > > dropped connections. Currently we worked around the issue by > > > downgrading to 1.17.4. I have collected some information about the > > > problem here: > > > https://gitlab.com/g_bor/fix-nginx-tls > > > It contains the logs from the good and the bad version, a full nginx > > > config, some info about how nginx was built. I managed to bisect the > > > problem down to changeset 9d2ad2fb4423, which introduced the problem. > > > Any help in further debuggin this would be greatly appreciated. > > > > Thanks for the report, it indeed looks like a bug introduced > > in 9d2ad2fb4423. > > > > The problem is that c->read->handler is overwritted when switching > > to the next pipelined request, ngx_ssl_next_read_handler() is not > > called, and c->read->ready remains not set. I'll take a look how > > to fix it properly. > > Please try the following patch: > > # HG changeset patch > # User Maxim Dounin > # Date 1577129029 -10800 > # Mon Dec 23 22:23:49 2019 +0300 > # Node ID c2dc6bfd2a0bce28618ef96b87fbdb63c6010575 > # Parent 8e64e11aaca02d50649cd2d9b448508f5b268062 > SSL: reworked posted next events. > > Introduced in 9d2ad2fb4423 available bytes handling in SSL relied > on connection read handler being overwritten to set the ready flag > and the amount of available bytes. This approach is, however, does > not work properly when connection read handler is changed, for example, > when switching to a next pipelined request, and can result in unexpected > connection timeouts, see here: > > http://mailman.nginx.org/pipermail/nginx-devel/2019-December/012825.html > > Fix is to introduce ngx_event_process_posted_next() instead, which > will set ready and available regardless of how event handler is set. > > diff --git a/src/event/ngx_event.c b/src/event/ngx_event.c > --- a/src/event/ngx_event.c > +++ b/src/event/ngx_event.c > @@ -238,8 +238,6 @@ ngx_process_events_and_timers(ngx_cycle_ > } > > if (!ngx_queue_empty(&ngx_posted_next_events)) { > - ngx_queue_add(&ngx_posted_events, &ngx_posted_next_events); > - ngx_queue_init(&ngx_posted_next_events); > timer = 0; > } > > @@ -263,6 +261,7 @@ ngx_process_events_and_timers(ngx_cycle_ > } > > ngx_event_process_posted(cycle, &ngx_posted_events); > + ngx_event_process_posted_next(cycle, &ngx_posted_next_events); > } > > > diff --git a/src/event/ngx_event_openssl.c b/src/event/ngx_event_openssl.c > --- a/src/event/ngx_event_openssl.c > +++ b/src/event/ngx_event_openssl.c > @@ -43,7 +43,6 @@ static ssize_t ngx_ssl_recv_early(ngx_co > #endif > static ngx_int_t ngx_ssl_handle_recv(ngx_connection_t *c, int n); > static void ngx_ssl_write_handler(ngx_event_t *wev); > -static void ngx_ssl_next_read_handler(ngx_event_t *rev); > #ifdef SSL_READ_EARLY_DATA_SUCCESS > static ssize_t ngx_ssl_write_early(ngx_connection_t *c, u_char *data, > size_t size); > @@ -2018,11 +2017,6 @@ ngx_ssl_recv(ngx_connection_t *c, u_char > c->read->available = 0; > c->read->ready = 0; > > - if (c->ssl->next_read_handler == NULL) { > - c->ssl->next_read_handler = c->read->handler; > - c->read->handler = ngx_ssl_next_read_handler; > - } > - > ngx_post_event(c->read, &ngx_posted_next_events); > } > > @@ -2328,31 +2322,6 @@ ngx_ssl_write_handler(ngx_event_t *wev) > } > > > -static void > -ngx_ssl_next_read_handler(ngx_event_t *rev) > -{ > - ngx_connection_t *c; > - > - c = rev->data; > - > - ngx_log_debug0(NGX_LOG_DEBUG_EVENT, c->log, 0, "SSL next read handler"); > - > - rev->handler = c->ssl->next_read_handler; > - c->ssl->next_read_handler = NULL; > - > - if (!rev->ready) { > - rev->ready = 1; > - rev->available = -1; > - } > - > - if (rev->posted) { > - ngx_delete_posted_event(rev); > - } > - > - rev->handler(rev); > -} > - > - > /* > * OpenSSL has no SSL_writev() so we copy several bufs into our 16K buffer > * before the SSL_write() call to decrease a SSL overhead. > diff --git a/src/event/ngx_event_openssl.h b/src/event/ngx_event_openssl.h > --- a/src/event/ngx_event_openssl.h > +++ b/src/event/ngx_event_openssl.h > @@ -86,7 +86,6 @@ struct ngx_ssl_connection_s { > > ngx_event_handler_pt saved_read_handler; > ngx_event_handler_pt saved_write_handler; > - ngx_event_handler_pt next_read_handler; > > u_char early_buf; > > diff --git a/src/event/ngx_event_posted.c b/src/event/ngx_event_posted.c > --- a/src/event/ngx_event_posted.c > +++ b/src/event/ngx_event_posted.c > @@ -34,3 +34,29 @@ ngx_event_process_posted(ngx_cycle_t *cy > ev->handler(ev); > } > } > + > + > +void > +ngx_event_process_posted_next(ngx_cycle_t *cycle, ngx_queue_t *posted) > +{ > + ngx_queue_t *q; > + ngx_event_t *ev; > + > + while (!ngx_queue_empty(posted)) { > + > + q = ngx_queue_head(posted); > + ev = ngx_queue_data(q, ngx_event_t, queue); > + > + ngx_log_debug1(NGX_LOG_DEBUG_EVENT, cycle->log, 0, > + "posted next event %p", ev); > + > + ngx_delete_posted_event(ev); > + > + if (!ev->ready) { > + ev->ready = 1; > + ev->available = -1; > + } > + > + ev->handler(ev); > + } > +} > diff --git a/src/event/ngx_event_posted.h b/src/event/ngx_event_posted.h > --- a/src/event/ngx_event_posted.h > +++ b/src/event/ngx_event_posted.h > @@ -39,6 +39,7 @@ > > > void ngx_event_process_posted(ngx_cycle_t *cycle, ngx_queue_t *posted); > +void ngx_event_process_posted_next(ngx_cycle_t *cycle, ngx_queue_t *posted); > > > extern ngx_queue_t ngx_posted_accept_events; > Thanks for having a look at this. I will coordinate with others as when can we schedule a service restart. I will report back with the results soon. I wish you a merry christmas, and thanks for the great work all around the year! > > -- > Maxim Dounin > http://mdounin.ru/ > _______________________________________________ > nginx-devel mailing list > nginx-devel at nginx.org > http://mailman.nginx.org/mailman/listinfo/nginx-devel Best regards, g_bor -- OpenPGP Key Fingerprint: 7988:3B9F:7D6A:4DBF:3719:0367:2506:A96C:CF63:0B21 From mdounin at mdounin.ru Tue Dec 24 16:46:02 2019 From: mdounin at mdounin.ru (Maxim Dounin) Date: Tue, 24 Dec 2019 19:46:02 +0300 Subject: Fwd: 1.17.5 regression In-Reply-To: References: <20191218172425.GX12894@mdounin.ru> <20191223192810.GO12894@mdounin.ru> Message-ID: <20191224164602.GA12894@mdounin.ru> Hello! On Tue, Dec 24, 2019 at 04:45:20PM +0100, G?bor Boskovits wrote: > Maxim Dounin ezt ?rta (id?pont: 2019. dec. 23., H, 20:28): > > > > Hello! > > > > On Wed, Dec 18, 2019 at 08:24:25PM +0300, Maxim Dounin wrote: > > > > > On Mon, Dec 16, 2019 at 03:02:37PM +0100, G?bor Boskovits wrote: > > > > > > > On berlin.guix.gnu.org we observed a regression upon upgrading nginx > > > > to 1.17.5. The problem was, when pipelining request to a proxy using > > > > tls, we suddenly started to get 408 client timeouts, manifesting in > > > > dropped connections. Currently we worked around the issue by > > > > downgrading to 1.17.4. I have collected some information about the > > > > problem here: > > > > https://gitlab.com/g_bor/fix-nginx-tls > > > > It contains the logs from the good and the bad version, a full nginx > > > > config, some info about how nginx was built. I managed to bisect the > > > > problem down to changeset 9d2ad2fb4423, which introduced the problem. > > > > Any help in further debuggin this would be greatly appreciated. > > > > > > Thanks for the report, it indeed looks like a bug introduced > > > in 9d2ad2fb4423. > > > > > > The problem is that c->read->handler is overwritted when switching > > > to the next pipelined request, ngx_ssl_next_read_handler() is not > > > called, and c->read->ready remains not set. I'll take a look how > > > to fix it properly. > > > > Please try the following patch: > > > > # HG changeset patch > > # User Maxim Dounin > > # Date 1577129029 -10800 > > # Mon Dec 23 22:23:49 2019 +0300 > > # Node ID c2dc6bfd2a0bce28618ef96b87fbdb63c6010575 > > # Parent 8e64e11aaca02d50649cd2d9b448508f5b268062 > > SSL: reworked posted next events. > > > > Introduced in 9d2ad2fb4423 available bytes handling in SSL relied > > on connection read handler being overwritten to set the ready flag > > and the amount of available bytes. This approach is, however, does > > not work properly when connection read handler is changed, for example, > > when switching to a next pipelined request, and can result in unexpected > > connection timeouts, see here: > > > > http://mailman.nginx.org/pipermail/nginx-devel/2019-December/012825.html > > > > Fix is to introduce ngx_event_process_posted_next() instead, which > > will set ready and available regardless of how event handler is set. [...] > Thanks for having a look at this. I will coordinate with others as > when can we schedule a service restart. > I will report back with the results soon. > > I wish you a merry christmas, and thanks for the great work all around the year! Just to make sure it's known: the patch above was committed after internal review/testing, and it is available as part of the 1.17.7 release. Merry Christmas! -- Maxim Dounin http://mdounin.ru/ From boskovits at gmail.com Tue Dec 24 16:54:01 2019 From: boskovits at gmail.com (=?UTF-8?Q?G=C3=A1bor_Boskovits?=) Date: Tue, 24 Dec 2019 17:54:01 +0100 Subject: Fwd: 1.17.5 regression In-Reply-To: <20191224164602.GA12894@mdounin.ru> References: <20191218172425.GX12894@mdounin.ru> <20191223192810.GO12894@mdounin.ru> <20191224164602.GA12894@mdounin.ru> Message-ID: Maxim Dounin ezt ?rta (id?pont: 2019. dec. 24., K, 17:46): > > Hello! > > On Tue, Dec 24, 2019 at 04:45:20PM +0100, G?bor Boskovits wrote: > > > Maxim Dounin ezt ?rta (id?pont: 2019. dec. 23., H, 20:28): > > > > > > Hello! > > > > > > On Wed, Dec 18, 2019 at 08:24:25PM +0300, Maxim Dounin wrote: > > > > > > > On Mon, Dec 16, 2019 at 03:02:37PM +0100, G?bor Boskovits wrote: > > > > > > > > > On berlin.guix.gnu.org we observed a regression upon upgrading nginx > > > > > to 1.17.5. The problem was, when pipelining request to a proxy using > > > > > tls, we suddenly started to get 408 client timeouts, manifesting in > > > > > dropped connections. Currently we worked around the issue by > > > > > downgrading to 1.17.4. I have collected some information about the > > > > > problem here: > > > > > https://gitlab.com/g_bor/fix-nginx-tls > > > > > It contains the logs from the good and the bad version, a full nginx > > > > > config, some info about how nginx was built. I managed to bisect the > > > > > problem down to changeset 9d2ad2fb4423, which introduced the problem. > > > > > Any help in further debuggin this would be greatly appreciated. > > > > > > > > Thanks for the report, it indeed looks like a bug introduced > > > > in 9d2ad2fb4423. > > > > > > > > The problem is that c->read->handler is overwritted when switching > > > > to the next pipelined request, ngx_ssl_next_read_handler() is not > > > > called, and c->read->ready remains not set. I'll take a look how > > > > to fix it properly. > > > > > > Please try the following patch: > > > > > > # HG changeset patch > > > # User Maxim Dounin > > > # Date 1577129029 -10800 > > > # Mon Dec 23 22:23:49 2019 +0300 > > > # Node ID c2dc6bfd2a0bce28618ef96b87fbdb63c6010575 > > > # Parent 8e64e11aaca02d50649cd2d9b448508f5b268062 > > > SSL: reworked posted next events. > > > > > > Introduced in 9d2ad2fb4423 available bytes handling in SSL relied > > > on connection read handler being overwritten to set the ready flag > > > and the amount of available bytes. This approach is, however, does > > > not work properly when connection read handler is changed, for example, > > > when switching to a next pipelined request, and can result in unexpected > > > connection timeouts, see here: > > > > > > http://mailman.nginx.org/pipermail/nginx-devel/2019-December/012825.html > > > > > > Fix is to introduce ngx_event_process_posted_next() instead, which > > > will set ready and available regardless of how event handler is set. > > [...] > > > Thanks for having a look at this. I will coordinate with others as > > when can we schedule a service restart. > > I will report back with the results soon. > > > > I wish you a merry christmas, and thanks for the great work all around the year! > > Just to make sure it's known: the patch above was committed after > internal review/testing, and it is available as part of the 1.17.7 > release. Thanks very much for the info. > > Merry Christmas! > > -- > Maxim Dounin > http://mdounin.ru/ > _______________________________________________ > nginx-devel mailing list > nginx-devel at nginx.org > http://mailman.nginx.org/mailman/listinfo/nginx-devel Best regards, g_bor -- OpenPGP Key Fingerprint: 7988:3B9F:7D6A:4DBF:3719:0367:2506:A96C:CF63:0B21 From maksim.yevmenkin at gmail.com Tue Dec 24 16:59:46 2019 From: maksim.yevmenkin at gmail.com (Maksim Yevmenkin) Date: Tue, 24 Dec 2019 08:59:46 -0800 Subject: Fwd: 1.17.5 regression In-Reply-To: <20191224164602.GA12894@mdounin.ru> References: <20191218172425.GX12894@mdounin.ru> <20191223192810.GO12894@mdounin.ru> <20191224164602.GA12894@mdounin.ru> Message-ID: hello! > Just to make sure it's known: the patch above was committed after > internal review/testing, and it is available as part of the 1.17.7 > release. would it be possible to publish 1.17.7 to https://github.com/nginx/nginx.git ? thanks! max From mdounin at mdounin.ru Tue Dec 24 17:26:47 2019 From: mdounin at mdounin.ru (Maxim Dounin) Date: Tue, 24 Dec 2019 20:26:47 +0300 Subject: Fwd: 1.17.5 regression In-Reply-To: References: <20191218172425.GX12894@mdounin.ru> <20191223192810.GO12894@mdounin.ru> <20191224164602.GA12894@mdounin.ru> Message-ID: <20191224172647.GB12894@mdounin.ru> Hello! On Tue, Dec 24, 2019 at 08:59:46AM -0800, Maksim Yevmenkin wrote: > > Just to make sure it's known: the patch above was committed after > > internal review/testing, and it is available as part of the 1.17.7 > > release. > > would it be possible to publish 1.17.7 to https://github.com/nginx/nginx.git ? Official repository is at hg.nginx.org/nginx, consider using it instead. Github mirror likely will be fixed as well at some point, though no ETA. -- Maxim Dounin http://mdounin.ru/ From marcinguy at gmail.com Tue Dec 24 19:00:26 2019 From: marcinguy at gmail.com (Marcin Kozlowski) Date: Tue, 24 Dec 2019 20:00:26 +0100 Subject: nginx - get value of the header - x_forwarded_for in Nginx module (Naxsi) In-Reply-To: <20191224093742.GA47591@lo0.su> References: <20191224093742.GA47591@lo0.su> Message-ID: Thanks. Works. For the reference, this is the code I used: ngx_uint_t n; ngx_table_elt_t **h; ngx_array_t a; a = req->headers_in.x_forwarded_for; n = a.nelts; h = a.elts; for (i = 0; iconnection->log, 0, "x_forwarded_for: %s", h[i]->value.data); } BTW What would be the best practice in NGINX NASIX module or any other module to load a file with hundreds entries of IPs (hashmap, or what structure would be best?) which should be whitelisted later for comparison in NASIX module logic. Those IP should never be blocked by NAXSI. When should I load this file in memory, in which component /module/function/step? Links to some guides/sample code would be also appreciated. Thanks, On Tue, Dec 24, 2019 at 10:37 AM Ruslan Ermilov wrote: > On Mon, Dec 23, 2019 at 11:04:43PM +0100, Marcin Kozlowski wrote: > > Hi List, > > > > How to get x_forwarded_for sent in the request in NGINX module (NAXSI in > > particular): > > > > My attempt: > > > > ngx_log_error(NGX_LOG_ERR, req->connection->log, > > 0, "test %s", (char > *)req->headers_in.x_forwarded_for.elts); > > > > The bigger problem I am trying to solve with NAXSI is this: > > > > > https://stackoverflow.com/questions/59453729/naxsi-blacklist-and-whitelist-setup-with-nginx > > > > I want to simply create a map and check if X-forwarded-for is on the > > whitelist and if yes, always allow it. > > > > Above does not work. Prints garabge. Why it is (void *) ??? > > > > Debugging it: > > > > $1 = (ngx_http_request_t *) 0xa44df0 > > (gdb) p req->headers_ > > headers_in headers_out > > (gdb) p req->headers_in > > $2 = {headers = {last = 0xa44e60, part = {elts = 0xad4d10, nelts = 7, > next > > = 0x0}, size = 48, nalloc = 20, > > pool = 0xa44da0}, host = 0xad4d10, connection = 0x0, > if_modified_since > > = 0x0, if_unmodified_since = 0x0, > > if_match = 0x0, if_none_match = 0x0, user_agent = 0xad4d70, referer = > > 0x0, content_length = 0x0, > > content_range = 0x0, content_type = 0x0, range = 0x0, if_range = 0x0, > > transfer_encoding = 0x0, te = 0x0, > > expect = 0x0, upgrade = 0x0, accept_encoding = 0x0, via = 0x0, > > authorization = 0x0, keep_alive = 0x0, > > x_forwarded_for = {elts = 0xa45b98, nelts = 1, size = 8, nalloc = 1, > pool > > = 0xa44da0}, x_real_ip = 0x0, user = { > > len = 0, data = 0x0}, passwd = {len = 0, data = 0x0}, cookies = > {elts = > > 0x0, nelts = 0, size = 0, nalloc = 0, > > pool = 0x0}, server = {len = 32, data = 0xa449a9 "domain.com"}, > > content_length_n = -1, > > keep_alive_n = -1, connection_type = 2, chunked = 0, msie = 0, msie6 = > 0, > > opera = 0, gecko = 0, chrome = 0, > > safari = 0, konqueror = 0} > > (gdb) p req->headers_in.x_forwarded_for > > $3 = {elts = 0xa45b98, nelts = 1, size = 8, nalloc = 1, pool = 0xa44da0} > > (gdb) p req->headers_in.x_forwarded_for .elts > > $4 = (void *) 0xa45b98 > > (gdb) p req->headers_in.x_forwarded_for.elts > > $5 = (void *) 0xa45b98 > > > > What structure is this? Linked List? Why the elts point to garbage, when > I > > know the Loadbalancer added the X-Forwarded-for header with value. > > It's an array of type ngx_table_elt_t, the number of elements is > in "nelts". Please see the handler for the $http_x_forwarded_for > variable in the ngx_http_variables.c on how to work with it. > _______________________________________________ > nginx-devel mailing list > nginx-devel at nginx.org > http://mailman.nginx.org/mailman/listinfo/nginx-devel > -------------- next part -------------- An HTML attachment was scrubbed... URL: From vskytta at gmail.com Wed Dec 25 07:29:06 2019 From: vskytta at gmail.com (=?iso-8859-1?q?Ville_Skytt=E4?=) Date: Wed, 25 Dec 2019 09:29:06 +0200 Subject: [PATCH] Grammar fixes Message-ID: # HG changeset patch # User Ville Skytt? # Date 1577258677 -7200 # Wed Dec 25 09:24:37 2019 +0200 # Node ID f4b83914e4009fb9c528f0318287a76aa0c7326f # Parent e99e9b0a8c51d59c7ee665489949d8556565adff Grammar fixes. diff --git a/docs/xml/nginx/changes.xml b/docs/xml/nginx/changes.xml --- a/docs/xml/nginx/changes.xml +++ b/docs/xml/nginx/changes.xml @@ -14571,7 +14571,7 @@ a segmentation fault might occur in worker process, -if an "debug_connection" directive was used; +if a "debug_connection" directive was used; the bug had appeared in 0.7.54. @@ -15333,7 +15333,7 @@ ?????? ???????????? ?????? 403 ?????? 500. -now if a file specified in a "auth_basic_user_file" directive is absent, +now if a file specified in an "auth_basic_user_file" directive is absent, then the 403 error is returned instead of the 500 one. @@ -15990,7 +15990,7 @@ ?????? ????????? add_header ?? ????????? ?????? ????????. -now a "add_header" directive does not add an empty value. +now an "add_header" directive does not add an empty value. @@ -17769,7 +17769,7 @@ a segmentation fault occurred in worker process, -if big value was used in a "expires" directive.
+if big value was used in an "expires" directive.
Thanks to Joaquin Cuenca Abela.
@@ -18443,7 +18443,7 @@ ?????? ????????? ? 0.6.11. -if the "?" character was in a "error_page" directive, then it was escaped +if the "?" character was in an "error_page" directive, then it was escape in a proxied request; the bug had appeared in 0.6.11. @@ -20715,7 +20715,7 @@ ????????? post_action ????? ?? ???????? ????? ?????????? ?????????? ???????. -the "post_action" directive might not run after a unsuccessful completion +the "post_action" directive might not run after an unsuccessful completion of a request. diff --git a/src/core/ngx_string.c b/src/core/ngx_string.c --- a/src/core/ngx_string.c +++ b/src/core/ngx_string.c @@ -519,7 +519,7 @@ * The glibc counterpart is about 150 lines of the code. * * For 32-bit numbers and some divisors gcc and icc use - * a inlined multiplication and shifts. For example, + * inlined multiplication and shifts. For example, * unsigned "i32 / 10" is compiled to * * (i32 * 0xCCCCCCCD) >> 35 diff --git a/src/event/modules/ngx_devpoll_module.c b/src/event/modules/ngx_devpoll_module.c --- a/src/event/modules/ngx_devpoll_module.c +++ b/src/event/modules/ngx_devpoll_module.c @@ -88,8 +88,8 @@ ngx_devpoll_del_event, /* delete an event */ ngx_devpoll_add_event, /* enable an event */ ngx_devpoll_del_event, /* disable an event */ - NULL, /* add an connection */ - NULL, /* delete an connection */ + NULL, /* add a connection */ + NULL, /* delete a connection */ NULL, /* trigger a notify */ ngx_devpoll_process_events, /* process the events */ ngx_devpoll_init, /* init the events */ diff --git a/src/event/modules/ngx_epoll_module.c b/src/event/modules/ngx_epoll_module.c --- a/src/event/modules/ngx_epoll_module.c +++ b/src/event/modules/ngx_epoll_module.c @@ -186,8 +186,8 @@ ngx_epoll_del_event, /* delete an event */ ngx_epoll_add_event, /* enable an event */ ngx_epoll_del_event, /* disable an event */ - ngx_epoll_add_connection, /* add an connection */ - ngx_epoll_del_connection, /* delete an connection */ + ngx_epoll_add_connection, /* add a connection */ + ngx_epoll_del_connection, /* delete a connection */ #if (NGX_HAVE_EVENTFD) ngx_epoll_notify, /* trigger a notify */ #else diff --git a/src/event/modules/ngx_eventport_module.c b/src/event/modules/ngx_eventport_module.c --- a/src/event/modules/ngx_eventport_module.c +++ b/src/event/modules/ngx_eventport_module.c @@ -181,8 +181,8 @@ ngx_eventport_del_event, /* delete an event */ ngx_eventport_add_event, /* enable an event */ ngx_eventport_del_event, /* disable an event */ - NULL, /* add an connection */ - NULL, /* delete an connection */ + NULL, /* add a connection */ + NULL, /* delete a connection */ ngx_eventport_notify, /* trigger a notify */ ngx_eventport_process_events, /* process the events */ ngx_eventport_init, /* init the events */ diff --git a/src/event/modules/ngx_iocp_module.c b/src/event/modules/ngx_iocp_module.c --- a/src/event/modules/ngx_iocp_module.c +++ b/src/event/modules/ngx_iocp_module.c @@ -62,8 +62,8 @@ NULL, /* delete an event */ NULL, /* enable an event */ NULL, /* disable an event */ - NULL, /* add an connection */ - ngx_iocp_del_connection, /* delete an connection */ + NULL, /* add a connection */ + ngx_iocp_del_connection, /* delete a connection */ NULL, /* trigger a notify */ ngx_iocp_process_events, /* process the events */ ngx_iocp_init, /* init the events */ diff --git a/src/event/modules/ngx_kqueue_module.c b/src/event/modules/ngx_kqueue_module.c --- a/src/event/modules/ngx_kqueue_module.c +++ b/src/event/modules/ngx_kqueue_module.c @@ -83,8 +83,8 @@ ngx_kqueue_del_event, /* delete an event */ ngx_kqueue_add_event, /* enable an event */ ngx_kqueue_del_event, /* disable an event */ - NULL, /* add an connection */ - NULL, /* delete an connection */ + NULL, /* add a connection */ + NULL, /* delete a connection */ #ifdef EVFILT_USER ngx_kqueue_notify, /* trigger a notify */ #else diff --git a/src/event/modules/ngx_poll_module.c b/src/event/modules/ngx_poll_module.c --- a/src/event/modules/ngx_poll_module.c +++ b/src/event/modules/ngx_poll_module.c @@ -37,8 +37,8 @@ ngx_poll_del_event, /* delete an event */ ngx_poll_add_event, /* enable an event */ ngx_poll_del_event, /* disable an event */ - NULL, /* add an connection */ - NULL, /* delete an connection */ + NULL, /* add a connection */ + NULL, /* delete a connection */ NULL, /* trigger a notify */ ngx_poll_process_events, /* process the events */ ngx_poll_init, /* init the events */ diff --git a/src/event/modules/ngx_select_module.c b/src/event/modules/ngx_select_module.c --- a/src/event/modules/ngx_select_module.c +++ b/src/event/modules/ngx_select_module.c @@ -45,8 +45,8 @@ ngx_select_del_event, /* delete an event */ ngx_select_add_event, /* enable an event */ ngx_select_del_event, /* disable an event */ - NULL, /* add an connection */ - NULL, /* delete an connection */ + NULL, /* add a connection */ + NULL, /* delete a connection */ NULL, /* trigger a notify */ ngx_select_process_events, /* process the events */ ngx_select_init, /* init the events */ diff --git a/src/event/modules/ngx_win32_poll_module.c b/src/event/modules/ngx_win32_poll_module.c --- a/src/event/modules/ngx_win32_poll_module.c +++ b/src/event/modules/ngx_win32_poll_module.c @@ -39,8 +39,8 @@ ngx_poll_del_event, /* delete an event */ ngx_poll_add_event, /* enable an event */ ngx_poll_del_event, /* disable an event */ - NULL, /* add an connection */ - NULL, /* delete an connection */ + NULL, /* add a connection */ + NULL, /* delete a connection */ NULL, /* trigger a notify */ ngx_poll_process_events, /* process the events */ ngx_poll_init, /* init the events */ diff --git a/src/event/modules/ngx_win32_select_module.c b/src/event/modules/ngx_win32_select_module.c --- a/src/event/modules/ngx_win32_select_module.c +++ b/src/event/modules/ngx_win32_select_module.c @@ -47,8 +47,8 @@ ngx_select_del_event, /* delete an event */ ngx_select_add_event, /* enable an event */ ngx_select_del_event, /* disable an event */ - NULL, /* add an connection */ - NULL, /* delete an connection */ + NULL, /* add a connection */ + NULL, /* delete a connection */ NULL, /* trigger a notify */ ngx_select_process_events, /* process the events */ ngx_select_init, /* init the events */ diff --git a/src/http/ngx_http_core_module.c b/src/http/ngx_http_core_module.c --- a/src/http/ngx_http_core_module.c +++ b/src/http/ngx_http_core_module.c @@ -1258,7 +1258,7 @@ & NGX_HTTP_KEEPALIVE_DISABLE_MSIE6)) { /* - * MSIE may wait for some time if an response for + * MSIE may wait for some time if a response for * a POST request was sent over a keepalive connection */ r->keepalive = 0; diff --git a/src/http/ngx_http_parse.c b/src/http/ngx_http_parse.c --- a/src/http/ngx_http_parse.c +++ b/src/http/ngx_http_parse.c @@ -98,7 +98,7 @@ #endif -/* gcc, icc, msvc and others compile these switches as an jump table */ +/* gcc, icc, msvc and others compile these switches as a jump table */ ngx_int_t ngx_http_parse_request_line(ngx_http_request_t *r, ngx_buf_t *b) diff --git a/src/http/ngx_http_special_response.c b/src/http/ngx_http_special_response.c --- a/src/http/ngx_http_special_response.c +++ b/src/http/ngx_http_special_response.c @@ -253,11 +253,11 @@ static char ngx_http_error_495_page[] = "" CRLF -"400 The SSL certificate error" +"400 SSL certificate error" CRLF "" CRLF "

400 Bad Request

" CRLF -"
The SSL certificate error
" CRLF +"
SSL certificate error
" CRLF ; @@ -273,11 +273,11 @@ static char ngx_http_error_497_page[] = "" CRLF -"400 The plain HTTP request was sent to HTTPS port" +"400 Plain HTTP request was sent to a HTTPS port" CRLF "" CRLF "

400 Bad Request

" CRLF -"
The plain HTTP request was sent to HTTPS port
" CRLF +"
Plain HTTP request was sent to a HTTPS port
" CRLF ; diff --git a/src/os/unix/ngx_darwin_sendfile_chain.c b/src/os/unix/ngx_darwin_sendfile_chain.c --- a/src/os/unix/ngx_darwin_sendfile_chain.c +++ b/src/os/unix/ngx_darwin_sendfile_chain.c @@ -54,7 +54,7 @@ if ((ngx_event_flags & NGX_USE_KQUEUE_EVENT) && wev->pending_eof) { (void) ngx_connection_error(c, wev->kq_errno, - "kevent() reported about an closed connection"); + "kevent() reported about a closed connection"); wev->error = 1; return NGX_CHAIN_ERROR; } diff --git a/src/os/unix/ngx_freebsd_sendfile_chain.c b/src/os/unix/ngx_freebsd_sendfile_chain.c --- a/src/os/unix/ngx_freebsd_sendfile_chain.c +++ b/src/os/unix/ngx_freebsd_sendfile_chain.c @@ -60,7 +60,7 @@ if ((ngx_event_flags & NGX_USE_KQUEUE_EVENT) && wev->pending_eof) { (void) ngx_connection_error(c, wev->kq_errno, - "kevent() reported about an closed connection"); + "kevent() reported about a closed connection"); wev->error = 1; return NGX_CHAIN_ERROR; } diff --git a/src/os/unix/ngx_readv_chain.c b/src/os/unix/ngx_readv_chain.c --- a/src/os/unix/ngx_readv_chain.c +++ b/src/os/unix/ngx_readv_chain.c @@ -35,7 +35,7 @@ rev->eof = 1; ngx_log_error(NGX_LOG_INFO, c->log, rev->kq_errno, - "kevent() reported about an closed connection"); + "kevent() reported about a closed connection"); if (rev->kq_errno) { rev->error = 1; diff --git a/src/os/unix/ngx_recv.c b/src/os/unix/ngx_recv.c --- a/src/os/unix/ngx_recv.c +++ b/src/os/unix/ngx_recv.c @@ -36,7 +36,7 @@ ngx_set_socket_errno(rev->kq_errno); return ngx_connection_error(c, rev->kq_errno, - "kevent() reported about an closed connection"); + "kevent() reported about a closed connection"); } return 0; diff --git a/src/os/unix/ngx_send.c b/src/os/unix/ngx_send.c --- a/src/os/unix/ngx_send.c +++ b/src/os/unix/ngx_send.c @@ -23,7 +23,7 @@ if ((ngx_event_flags & NGX_USE_KQUEUE_EVENT) && wev->pending_eof) { (void) ngx_connection_error(c, wev->kq_errno, - "kevent() reported about an closed connection"); + "kevent() reported about a closed connection"); wev->error = 1; return NGX_ERROR; } diff --git a/src/os/unix/ngx_udp_sendmsg_chain.c b/src/os/unix/ngx_udp_sendmsg_chain.c --- a/src/os/unix/ngx_udp_sendmsg_chain.c +++ b/src/os/unix/ngx_udp_sendmsg_chain.c @@ -35,7 +35,7 @@ if ((ngx_event_flags & NGX_USE_KQUEUE_EVENT) && wev->pending_eof) { (void) ngx_connection_error(c, wev->kq_errno, - "kevent() reported about an closed connection"); + "kevent() reported about a closed connection"); wev->error = 1; return NGX_CHAIN_ERROR; } diff --git a/src/os/unix/ngx_writev_chain.c b/src/os/unix/ngx_writev_chain.c --- a/src/os/unix/ngx_writev_chain.c +++ b/src/os/unix/ngx_writev_chain.c @@ -30,7 +30,7 @@ if ((ngx_event_flags & NGX_USE_KQUEUE_EVENT) && wev->pending_eof) { (void) ngx_connection_error(c, wev->kq_errno, - "kevent() reported about an closed connection"); + "kevent() reported about a closed connection"); wev->error = 1; return NGX_CHAIN_ERROR; } From pluknet at nginx.com Wed Dec 25 10:15:36 2019 From: pluknet at nginx.com (Sergey Kandaurov) Date: Wed, 25 Dec 2019 13:15:36 +0300 Subject: 1.17.5 regression In-Reply-To: References: <20191218172425.GX12894@mdounin.ru> <20191223192810.GO12894@mdounin.ru> <20191224164602.GA12894@mdounin.ru> Message-ID: > On 24 Dec 2019, at 19:59, Maksim Yevmenkin wrote: > > hello! > >> Just to make sure it's known: the patch above was committed after >> internal review/testing, and it is available as part of the 1.17.7 >> release. > > would it be possible to publish 1.17.7 to https://github.com/nginx/nginx.git ? > Should be fixed now. -- Sergey Kandaurov From ru at nginx.com Wed Dec 25 11:17:26 2019 From: ru at nginx.com (Ruslan Ermilov) Date: Wed, 25 Dec 2019 14:17:26 +0300 Subject: nginx - get value of the header - x_forwarded_for in Nginx module (Naxsi) In-Reply-To: References: <20191224093742.GA47591@lo0.su> Message-ID: <20191225111726.GA72241@lo0.su> On Tue, Dec 24, 2019 at 08:00:26PM +0100, Marcin Kozlowski wrote: > Thanks. > > Works. For the reference, this is the code I used: > > ngx_uint_t n; > ngx_table_elt_t **h; > ngx_array_t a; > a = req->headers_in.x_forwarded_for; > n = a.nelts; > h = a.elts; > > > for (i = 0; i ngx_log_error(NGX_LOG_ERR, req->connection->log, > 0, "x_forwarded_for: %s", h[i]->value.data); > } > > BTW What would be the best practice in NGINX NASIX module or any other > module to load a file with hundreds entries of IPs (hashmap, or what > structure would be best?) which should be whitelisted later for comparison > in NASIX module logic. Those IP should never be blocked by NAXSI. > > When should I load this file in memory, in which component > /module/function/step? > > Links to some guides/sample code would be also appreciated. > > Thanks, http://nginx.org/en/docs/http/ngx_http_geo_module.html From xeioex at nginx.com Wed Dec 25 15:14:32 2019 From: xeioex at nginx.com (Dmitry Volyntsev) Date: Wed, 25 Dec 2019 15:14:32 +0000 Subject: [njs] Introduced TypedArray objects. Message-ID: details: https://hg.nginx.org/njs/rev/1c3c593cc3fd branches: changeset: 1289:1c3c593cc3fd user: Dmitry Volyntsev date: Wed Dec 25 15:59:01 2019 +0300 description: Introduced TypedArray objects. Added the global constructors: Uint8Array, Int8Array, Uint8ClampedArray, Uint16Array, Int16Array, Uint32Array, Int32Array, Float32Array, Float64Array. Implemented: - TypedArray(), - TypedArray(length), - TypedArray(buffer, [start, [end]]), - TypedArray(object), - TypedArray(typedarray), - TypedArray.BYTES_PER_ELEMENT. - %TypedArray%.prototype.constructor - get %TypedArray%.prototype.buffer - get %TypedArray%.prototype.byteLength - get %TypedArray%.prototype.byteOffset - get %TypedArray%.prototype.length - %TypedArray%.prototype[@@toStringTag] - %TypedArray%.prototype.set(array [, start]) - %TypedArray%.prototype.slice([start, [end]]) - %TypedArray%.prototype.toString() - %TypedArray%.prototype.join(separator) - %TypedArray%.prototype.fill(). In collaboration with Tiago Natel de Moura. This closes #264 issue on Github. diffstat: auto/sources | 1 + src/njs.h | 2 +- src/njs_array.c | 2 +- src/njs_array.h | 2 + src/njs_array_buffer.c | 21 +- src/njs_builtin.c | 139 +++- src/njs_json.c | 84 +- src/njs_main.h | 1 + src/njs_number.c | 15 +- src/njs_number.h | 13 +- src/njs_object.c | 65 +- src/njs_object_hash.h | 139 +++ src/njs_object_prop.c | 30 + src/njs_string.c | 83 +- src/njs_typed_array.c | 1751 ++++++++++++++++++++++++++++++++++++++++++++ src/njs_typed_array.h | 163 ++++ src/njs_value.c | 94 ++- src/njs_value.h | 78 +- src/njs_value_conversion.h | 4 +- src/njs_vm.h | 24 +- src/njs_vmcode.c | 10 +- src/test/njs_benchmark.c | 20 + src/test/njs_unit_test.c | 605 +++++++++++++++- 23 files changed, 3239 insertions(+), 107 deletions(-) diffs (truncated from 4042 to 1000 lines): diff -r 99f9008e1b17 -r 1c3c593cc3fd auto/sources --- a/auto/sources Tue Dec 17 18:22:50 2019 +0300 +++ b/auto/sources Wed Dec 25 15:59:01 2019 +0300 @@ -53,6 +53,7 @@ NJS_LIB_SRCS=" \ src/njs_generator.c \ src/njs_disassembler.c \ src/njs_array_buffer.c \ + src/njs_typed_array.c \ src/njs_promise.c \ " diff -r 99f9008e1b17 -r 1c3c593cc3fd src/njs.h --- a/src/njs.h Tue Dec 17 18:22:50 2019 +0300 +++ b/src/njs.h Wed Dec 25 15:59:01 2019 +0300 @@ -274,7 +274,7 @@ NJS_EXPORT njs_int_t njs_vm_value_string NJS_EXPORT njs_int_t njs_vm_retval_string(njs_vm_t *vm, njs_str_t *dst); NJS_EXPORT njs_int_t njs_vm_value_dump(njs_vm_t *vm, njs_str_t *dst, - const njs_value_t *value, njs_uint_t console, njs_uint_t indent); + njs_value_t *value, njs_uint_t console, njs_uint_t indent); NJS_EXPORT njs_int_t njs_vm_retval_dump(njs_vm_t *vm, njs_str_t *dst, njs_uint_t indent); diff -r 99f9008e1b17 -r 1c3c593cc3fd src/njs_array.c --- a/src/njs_array.c Tue Dec 17 18:22:50 2019 +0300 +++ b/src/njs_array.c Wed Dec 25 15:59:01 2019 +0300 @@ -1089,7 +1089,7 @@ njs_array_prototype_reverse(njs_vm_t *vm } -static njs_int_t +njs_int_t njs_array_prototype_to_string(njs_vm_t *vm, njs_value_t *args, njs_uint_t nargs, njs_index_t unused) { diff -r 99f9008e1b17 -r 1c3c593cc3fd src/njs_array.h --- a/src/njs_array.h Tue Dec 17 18:22:50 2019 +0300 +++ b/src/njs_array.h Wed Dec 25 15:59:01 2019 +0300 @@ -21,6 +21,8 @@ njs_int_t njs_array_string_add(njs_vm_t const u_char *start, size_t size, size_t length); njs_int_t njs_array_expand(njs_vm_t *vm, njs_array_t *array, uint32_t prepend, uint32_t append); +njs_int_t njs_array_prototype_to_string(njs_vm_t *vm, njs_value_t *args, + njs_uint_t nargs, njs_index_t unused); extern const njs_object_init_t njs_array_instance_init; diff -r 99f9008e1b17 -r 1c3c593cc3fd src/njs_array_buffer.c --- a/src/njs_array_buffer.c Tue Dec 17 18:22:50 2019 +0300 +++ b/src/njs_array_buffer.c Wed Dec 25 15:59:01 2019 +0300 @@ -59,7 +59,7 @@ static njs_int_t njs_array_buffer_constructor(njs_vm_t *vm, njs_value_t *args, njs_uint_t nargs, njs_index_t unused) { - uint32_t size; + uint64_t size; njs_int_t ret; njs_value_t *value; njs_array_buffer_t *array; @@ -98,6 +98,16 @@ njs_array_buffer_get_this(njs_vm_t *vm, } +static njs_int_t +njs_array_buffer_is_view(njs_vm_t *vm, njs_value_t *args, njs_uint_t nargs, + njs_index_t unused) +{ + njs_set_boolean(&vm->retval, njs_is_typed_array(njs_arg(args, nargs, 1))); + + return NJS_OK; +} + + static const njs_object_prop_t njs_array_buffer_constructor_properties[] = { /* ArrayBuffer.name == "ArrayBuffer". */ @@ -134,6 +144,15 @@ static const njs_object_prop_t njs_arra .configurable = 1, .enumerable = 0, }, + + /* ArrayBuffer.isView(new Uint8Array()) === true */ + { + .type = NJS_PROPERTY, + .name = njs_string("isView"), + .value = njs_native_function(njs_array_buffer_is_view, 1), + .writable = 1, + .configurable = 1, + }, }; diff -r 99f9008e1b17 -r 1c3c593cc3fd src/njs_builtin.c --- a/src/njs_builtin.c Tue Dec 17 18:22:50 2019 +0300 +++ b/src/njs_builtin.c Wed Dec 25 15:59:01 2019 +0300 @@ -58,7 +58,6 @@ static const njs_object_type_init_t *con &njs_obj_type_init, &njs_array_type_init, - &njs_array_buffer_type_init, &njs_boolean_type_init, &njs_number_type_init, &njs_symbol_type_init, @@ -67,14 +66,27 @@ static const njs_object_type_init_t *con &njs_regexp_type_init, &njs_date_type_init, &njs_promise_type_init, + &njs_array_buffer_type_init, /* Hidden types. */ &njs_hash_type_init, &njs_hmac_type_init, + &njs_typed_array_type_init, + + /* TypedArray types. */ + + &njs_typed_array_u8_type_init, + &njs_typed_array_u8clamped_type_init, + &njs_typed_array_i8_type_init, + &njs_typed_array_u16_type_init, + &njs_typed_array_i16_type_init, + &njs_typed_array_u32_type_init, + &njs_typed_array_i32_type_init, + &njs_typed_array_f32_type_init, + &njs_typed_array_f64_type_init, /* Error types. */ - &njs_error_type_init, &njs_eval_error_type_init, &njs_internal_error_type_init, @@ -303,8 +315,9 @@ njs_builtin_objects_clone(njs_vm_t *vm, { size_t size; njs_uint_t i; - njs_object_t *object_prototype, *function_prototype, *error_prototype, - *error_constructor; + njs_object_t *object_prototype, *function_prototype, + *typed_array_prototype, *error_prototype, + *typed_array_ctor, *error_ctor; /* * Copy both prototypes and constructors arrays by one memcpy() @@ -317,11 +330,21 @@ njs_builtin_objects_clone(njs_vm_t *vm, object_prototype = &vm->prototypes[NJS_OBJ_TYPE_OBJECT].object; - for (i = NJS_OBJ_TYPE_ARRAY; i < NJS_OBJ_TYPE_EVAL_ERROR; i++) { + for (i = NJS_OBJ_TYPE_ARRAY; i < NJS_OBJ_TYPE_NORMAL_MAX; i++) { vm->prototypes[i].object.__proto__ = object_prototype; } + typed_array_prototype = &vm->prototypes[NJS_OBJ_TYPE_TYPED_ARRAY].object; + + for (i = NJS_OBJ_TYPE_TYPED_ARRAY_MIN; + i < NJS_OBJ_TYPE_TYPED_ARRAY_MAX; + i++) + { + vm->prototypes[i].object.__proto__ = typed_array_prototype; + } + error_prototype = &vm->prototypes[NJS_OBJ_TYPE_ERROR].object; + error_prototype->__proto__ = object_prototype; for (i = NJS_OBJ_TYPE_EVAL_ERROR; i < NJS_OBJ_TYPE_MAX; i++) { vm->prototypes[i].object.__proto__ = error_prototype; @@ -329,14 +352,24 @@ njs_builtin_objects_clone(njs_vm_t *vm, function_prototype = &vm->prototypes[NJS_OBJ_TYPE_FUNCTION].object; - for (i = NJS_OBJ_TYPE_OBJECT; i < NJS_OBJ_TYPE_EVAL_ERROR; i++) { + for (i = NJS_OBJ_TYPE_OBJECT; i < NJS_OBJ_TYPE_NORMAL_MAX; i++) { vm->constructors[i].object.__proto__ = function_prototype; } - error_constructor = &vm->constructors[NJS_OBJ_TYPE_ERROR].object; + typed_array_ctor = &vm->constructors[NJS_OBJ_TYPE_TYPED_ARRAY].object; + + for (i = NJS_OBJ_TYPE_TYPED_ARRAY_MIN; + i < NJS_OBJ_TYPE_TYPED_ARRAY_MAX; + i++) + { + vm->constructors[i].object.__proto__ = typed_array_ctor; + } + + error_ctor = &vm->constructors[NJS_OBJ_TYPE_ERROR].object; + error_ctor->__proto__ = function_prototype; for (i = NJS_OBJ_TYPE_EVAL_ERROR; i < NJS_OBJ_TYPE_MAX; i++) { - vm->constructors[i].object.__proto__ = error_constructor; + vm->constructors[i].object.__proto__ = error_ctor; } vm->global_object.__proto__ = object_prototype; @@ -1111,6 +1144,96 @@ static const njs_object_prop_t njs_glob { .type = NJS_PROPERTY_HANDLER, + .name = njs_string("Uint8Array"), + .value = njs_prop_handler2(njs_top_level_constructor, + NJS_OBJ_TYPE_UINT8_ARRAY, + NJS_UINT8ARRAY_HASH), + .writable = 1, + .configurable = 1, + }, + + { + .type = NJS_PROPERTY_HANDLER, + .name = njs_string("Uint16Array"), + .value = njs_prop_handler2(njs_top_level_constructor, + NJS_OBJ_TYPE_UINT16_ARRAY, + NJS_UINT16ARRAY_HASH), + .writable = 1, + .configurable = 1, + }, + + { + .type = NJS_PROPERTY_HANDLER, + .name = njs_string("Uint32Array"), + .value = njs_prop_handler2(njs_top_level_constructor, + NJS_OBJ_TYPE_UINT32_ARRAY, + NJS_UINT32ARRAY_HASH), + .writable = 1, + .configurable = 1, + }, + + { + .type = NJS_PROPERTY_HANDLER, + .name = njs_string("Int8Array"), + .value = njs_prop_handler2(njs_top_level_constructor, + NJS_OBJ_TYPE_INT8_ARRAY, + NJS_INT8ARRAY_HASH), + .writable = 1, + .configurable = 1, + }, + + { + .type = NJS_PROPERTY_HANDLER, + .name = njs_string("Int16Array"), + .value = njs_prop_handler2(njs_top_level_constructor, + NJS_OBJ_TYPE_INT16_ARRAY, + NJS_INT16ARRAY_HASH), + .writable = 1, + .configurable = 1, + }, + + { + .type = NJS_PROPERTY_HANDLER, + .name = njs_string("Int32Array"), + .value = njs_prop_handler2(njs_top_level_constructor, + NJS_OBJ_TYPE_INT32_ARRAY, + NJS_INT32ARRAY_HASH), + .writable = 1, + .configurable = 1, + }, + + { + .type = NJS_PROPERTY_HANDLER, + .name = njs_string("Float32Array"), + .value = njs_prop_handler2(njs_top_level_constructor, + NJS_OBJ_TYPE_FLOAT32_ARRAY, + NJS_FLOAT32ARRAY_HASH), + .writable = 1, + .configurable = 1, + }, + + { + .type = NJS_PROPERTY_HANDLER, + .name = njs_string("Float64Array"), + .value = njs_prop_handler2(njs_top_level_constructor, + NJS_OBJ_TYPE_FLOAT64_ARRAY, + NJS_FLOAT64ARRAY_HASH), + .writable = 1, + .configurable = 1, + }, + + { + .type = NJS_PROPERTY_HANDLER, + .name = njs_long_string("Uint8ClampedArray"), + .value = njs_prop_handler2(njs_top_level_constructor, + NJS_OBJ_TYPE_UINT8_CLAMPED_ARRAY, + NJS_UINT8CLAMPEDARRAY_HASH), + .writable = 1, + .configurable = 1, + }, + + { + .type = NJS_PROPERTY_HANDLER, .name = njs_string("Boolean"), .value = njs_prop_handler2(njs_top_level_constructor, NJS_OBJ_TYPE_BOOLEAN, NJS_BOOLEAN_HASH), diff -r 99f9008e1b17 -r 1c3c593cc3fd src/njs_json.c --- a/src/njs_json.c Tue Dec 17 18:22:50 2019 +0300 +++ b/src/njs_json.c Wed Dec 25 15:59:01 2019 +0300 @@ -1087,12 +1087,15 @@ njs_json_pop_stringify_state(njs_json_st } -#define njs_json_is_object(value) \ - (((value)->type == NJS_OBJECT) \ - || ((value)->type == NJS_OBJECT_SYMBOL) \ - || ((value)->type == NJS_EXTERNAL) \ - || ((value)->type == NJS_ARRAY) \ - || ((value)->type >= NJS_REGEXP)) +njs_inline njs_bool_t +njs_json_is_object(const njs_value_t *value) +{ + return (((value)->type == NJS_OBJECT) + || ((value)->type == NJS_ARRAY) + || ((value)->type == NJS_OBJECT_SYMBOL) + || ((value)->type == NJS_EXTERNAL) + || ((value)->type >= NJS_REGEXP)); +} #define njs_json_stringify_indent(times) \ @@ -1717,12 +1720,14 @@ const njs_object_init_t njs_json_object static njs_int_t -njs_dump_value(njs_json_stringify_t *stringify, njs_chb_t *chain, - const njs_value_t *value, njs_uint_t console) +njs_dump_terminal(njs_json_stringify_t *stringify, njs_chb_t *chain, + njs_value_t *value, njs_uint_t console) { - njs_str_t str; - njs_int_t ret; - njs_value_t str_val; + njs_str_t str; + njs_int_t ret; + njs_value_t str_val, tag; + njs_typed_array_t *array; + njs_string_prop_t string; njs_int_t (*to_string)(njs_vm_t *, njs_value_t *, const njs_value_t *); @@ -1836,6 +1841,27 @@ njs_dump_value(njs_json_stringify_t *str break; + case NJS_TYPED_ARRAY: + array = njs_typed_array(value); + ret = njs_object_string_tag(stringify->vm, value, &tag); + if (njs_slow_path(ret == NJS_ERROR)) { + return ret; + } + + if (ret == NJS_OK) { + (void) njs_string_prop(&string, &tag); + njs_chb_append(chain, string.start, string.size); + njs_chb_append_literal(chain, " "); + } + + njs_chb_append_literal(chain, "["); + + (void) njs_typed_array_to_chain(stringify->vm, chain, array, NULL); + + njs_chb_append_literal(chain, "]"); + + break; + case NJS_NUMBER: if (njs_slow_path(njs_number(value) == 0.0 && signbit(njs_number(value)))) @@ -1899,13 +1925,11 @@ njs_dump_is_external_object(const njs_va njs_inline njs_bool_t -njs_dump_is_object(const njs_value_t *value) +njs_dump_is_recursive(const njs_value_t *value) { return (value->type == NJS_OBJECT && !njs_object(value)->error_data) || (value->type == NJS_ARRAY) - || (value->type == NJS_ARRAY_BUFFER) - || (value->type == NJS_PROMISE) - || (value->type == NJS_OBJECT_VALUE) + || (value->type >= NJS_OBJECT_SPECIAL_MAX) || njs_dump_is_external_object(value); } @@ -1916,7 +1940,7 @@ static const njs_value_t string_get_set njs_int_t -njs_vm_value_dump(njs_vm_t *vm, njs_str_t *retval, const njs_value_t *value, +njs_vm_value_dump(njs_vm_t *vm, njs_str_t *retval, njs_value_t *value, njs_uint_t console, njs_uint_t indent) { njs_int_t i; @@ -1934,13 +1958,11 @@ njs_vm_value_dump(njs_vm_t *vm, njs_str_ stringify->vm = vm; stringify->depth = 0; - njs_set_undefined(&stringify->replacer); - stringify->keys_type = NJS_ENUM_STRING | NJS_ENUM_SYMBOL; njs_chb_init(&chain, vm->mem_pool); - if (!njs_dump_is_object(value)) { - ret = njs_dump_value(stringify, &chain, value, console); + if (!njs_dump_is_recursive(value)) { + ret = njs_dump_terminal(stringify, &chain, value, console); if (njs_slow_path(ret != NJS_OK)) { goto memory_error; } @@ -1948,6 +1970,8 @@ njs_vm_value_dump(njs_vm_t *vm, njs_str_ goto done; } + njs_set_undefined(&stringify->replacer); + stringify->keys_type = NJS_ENUM_STRING | NJS_ENUM_SYMBOL; indent = njs_min(indent, 10); stringify->space.length = indent; stringify->space.start = stringify->space_buf; @@ -2028,6 +2052,7 @@ njs_vm_value_dump(njs_vm_t *vm, njs_str_ if (njs_is_defined(&prop->getter)) { if (njs_is_defined(&prop->setter)) { val = njs_value_arg(&string_get_set); + } else { val = njs_value_arg(&string_get); } @@ -2050,7 +2075,7 @@ njs_vm_value_dump(njs_vm_t *vm, njs_str_ njs_chb_append_literal(&chain, " "); } - if (njs_dump_is_object(val)) { + if (njs_dump_is_recursive(val)) { state = njs_json_push_stringify_state(vm, stringify, val); if (njs_slow_path(state == NULL)) { goto exception; @@ -2059,7 +2084,7 @@ njs_vm_value_dump(njs_vm_t *vm, njs_str_ break; } - ret = njs_dump_value(stringify, &chain, val, console); + ret = njs_dump_terminal(stringify, &chain, val, console); if (njs_slow_path(ret != NJS_OK)) { if (ret == NJS_DECLINED) { goto exception; @@ -2072,6 +2097,17 @@ njs_vm_value_dump(njs_vm_t *vm, njs_str_ case NJS_JSON_ARRAY: if (state->index == 0) { + ret = njs_object_string_tag(vm, &state->value, &tag); + if (njs_slow_path(ret == NJS_ERROR)) { + return ret; + } + + if (ret == NJS_OK) { + (void) njs_string_prop(&string, &tag); + njs_chb_append(&chain, string.start, string.size); + njs_chb_append_literal(&chain, " "); + } + njs_chb_append_literal(&chain, "["); njs_json_stringify_indent(stringify->depth + 1); } @@ -2095,7 +2131,7 @@ njs_vm_value_dump(njs_vm_t *vm, njs_str_ val = &njs_array_start(&state->value)[state->index++]; - if (njs_dump_is_object(val)) { + if (njs_dump_is_recursive(val)) { state = njs_json_push_stringify_state(vm, stringify, val); if (njs_slow_path(state == NULL)) { goto exception; @@ -2106,7 +2142,7 @@ njs_vm_value_dump(njs_vm_t *vm, njs_str_ state->written = 1; - ret = njs_dump_value(stringify, &chain, val, console); + ret = njs_dump_terminal(stringify, &chain, val, console); if (njs_slow_path(ret != NJS_OK)) { if (ret == NJS_DECLINED) { goto exception; diff -r 99f9008e1b17 -r 1c3c593cc3fd src/njs_main.h --- a/src/njs_main.h Tue Dec 17 18:22:50 2019 +0300 +++ b/src/njs_main.h Wed Dec 25 15:59:01 2019 +0300 @@ -61,6 +61,7 @@ #include #include #include +#include #include #include #include diff -r 99f9008e1b17 -r 1c3c593cc3fd src/njs_number.c --- a/src/njs_number.c Tue Dec 17 18:22:50 2019 +0300 +++ b/src/njs_number.c Wed Dec 25 15:59:01 2019 +0300 @@ -20,19 +20,16 @@ static njs_int_t njs_number_to_string_ra double number, uint32_t radix); -uint32_t +double njs_key_to_index(const njs_value_t *value) { - double num; njs_array_t *array; - num = NAN; - if (njs_fast_path(njs_is_numeric(value))) { - num = njs_number(value); + return njs_number(value); } else if (njs_is_string(value)) { - num = njs_string_to_index(value); + return njs_string_to_index(value); } else if (njs_is_array(value)) { @@ -52,11 +49,7 @@ njs_key_to_index(const njs_value_t *valu } } - if ((uint32_t) num == num) { - return (uint32_t) num; - } - - return NJS_ARRAY_INVALID_INDEX; + return NAN; } diff -r 99f9008e1b17 -r 1c3c593cc3fd src/njs_number.h --- a/src/njs_number.h Tue Dec 17 18:22:50 2019 +0300 +++ b/src/njs_number.h Wed Dec 25 15:59:01 2019 +0300 @@ -8,7 +8,7 @@ #define _NJS_NUMBER_H_INCLUDED_ -uint32_t njs_key_to_index(const njs_value_t *value); +double njs_key_to_index(const njs_value_t *value); double njs_number_dec_parse(const u_char **start, const u_char *end); uint64_t njs_number_oct_parse(const u_char **start, const u_char *end); uint64_t njs_number_bin_parse(const u_char **start, const u_char *end); @@ -29,6 +29,17 @@ njs_int_t njs_number_parse_float(njs_vm_ njs_uint_t nargs, njs_index_t unused); +njs_inline njs_bool_t +njs_number_is_integer_index(double num, const njs_value_t *value) +{ + uint32_t u32; + + u32 = num; + + return (u32 == num && u32 != 0xffffffff) + && !(njs_is_string(value) && num == 0 && signbit(num)); +} + njs_inline int64_t njs_number_to_int64(double num) { diff -r 99f9008e1b17 -r 1c3c593cc3fd src/njs_object.c --- a/src/njs_object.c Tue Dec 17 18:22:50 2019 +0300 +++ b/src/njs_object.c Wed Dec 25 15:59:01 2019 +0300 @@ -20,6 +20,8 @@ static uint32_t njs_object_own_enumerate njs_object_enum_type_t type, njs_bool_t all); static njs_int_t njs_object_enumerate_array(njs_vm_t *vm, const njs_array_t *array, njs_array_t *items, njs_object_enum_t kind); +static njs_int_t njs_object_enumerate_typed_array(njs_vm_t *vm, + const njs_typed_array_t *array, njs_array_t *items, njs_object_enum_t kind); static njs_int_t njs_object_enumerate_string(njs_vm_t *vm, const njs_value_t *value, njs_array_t *items, njs_object_enum_t kind); static njs_int_t njs_object_enumerate_object(njs_vm_t *vm, @@ -462,6 +464,10 @@ njs_object_own_enumerate_length(const nj length += njs_object_enumerate_array_length(object); break; + case NJS_TYPED_ARRAY: + length += njs_typed_array_length((njs_typed_array_t *) object); + break; + case NJS_OBJECT_STRING: length += njs_object_enumerate_string_length(object); break; @@ -531,6 +537,12 @@ njs_object_own_enumerate_value(njs_vm_t kind); break; + case NJS_TYPED_ARRAY: + ret = njs_object_enumerate_typed_array(vm, + (njs_typed_array_t *) object, + items, kind); + break; + case NJS_OBJECT_STRING: obj_val = (njs_object_value_t *) object; @@ -798,6 +810,54 @@ njs_object_enumerate_array(njs_vm_t *vm, static njs_int_t +njs_object_enumerate_typed_array(njs_vm_t *vm, const njs_typed_array_t *array, + njs_array_t *items, njs_object_enum_t kind) +{ + uint32_t i, length; + njs_value_t *item; + njs_array_t *entry; + + item = items->start; + length = njs_typed_array_length(array); + + switch (kind) { + case NJS_ENUM_KEYS: + for (i = 0; i < length; i++) { + njs_uint32_to_string(item++, i); + } + + break; + + case NJS_ENUM_VALUES: + for (i = 0; i < length; i++) { + njs_set_number(item++, njs_typed_array_get(array, i)); + } + + break; + + case NJS_ENUM_BOTH: + for (i = 0; i < length; i++) { + entry = njs_array_alloc(vm, 2, 0); + if (njs_slow_path(entry == NULL)) { + return NJS_ERROR; + } + + njs_uint32_to_string(&entry->start[0], i); + njs_set_number(&entry->start[1], njs_typed_array_get(array, i)); + + njs_set_array(item++, entry); + } + + break; + } + + items->start = item; + + return NJS_OK; +} + + +static njs_int_t njs_object_enumerate_string(njs_vm_t *vm, const njs_value_t *value, njs_array_t *items, njs_object_enum_t kind) { @@ -2356,8 +2416,6 @@ static const njs_value_t njs_object_obj njs_long_string("[object Object]"); static const njs_value_t njs_object_array_string = njs_string("[object Array]"); -static const njs_value_t njs_object_array_buffer_string = - njs_long_string("[object ArrayBuffer]"); static const njs_value_t njs_object_function_string = njs_long_string("[object Function]"); static const njs_value_t njs_object_regexp_string = @@ -2411,7 +2469,8 @@ njs_object_prototype_to_string(njs_vm_t &njs_object_date_string, &njs_object_object_string, &njs_object_object_string, - &njs_object_array_buffer_string, + &njs_object_object_string, + &njs_object_object_string, }; value = njs_argument(args, 0); diff -r 99f9008e1b17 -r 1c3c593cc3fd src/njs_object_hash.h --- a/src/njs_object_hash.h Tue Dec 17 18:22:50 2019 +0300 +++ b/src/njs_object_hash.h Wed Dec 25 15:59:01 2019 +0300 @@ -587,4 +587,143 @@ 'A'), 'r'), 'r'), 'a'), 'y'), 'B'), 'u'), 'f'), 'f'), 'e'), 'r') +#define NJS_UINT8ARRAY_HASH \ + njs_djb_hash_add( \ + njs_djb_hash_add( \ + njs_djb_hash_add( \ + njs_djb_hash_add( \ + njs_djb_hash_add( \ + njs_djb_hash_add( \ + njs_djb_hash_add( \ + njs_djb_hash_add( \ + njs_djb_hash_add( \ + njs_djb_hash_add(NJS_DJB_HASH_INIT, \ + 'U'), 'i'), 'n'), 't'), '8'), 'A'), 'r'), 'r'), 'a'), 'y') + + +#define NJS_UINT16ARRAY_HASH \ + njs_djb_hash_add( \ + njs_djb_hash_add( \ + njs_djb_hash_add( \ + njs_djb_hash_add( \ + njs_djb_hash_add( \ + njs_djb_hash_add( \ + njs_djb_hash_add( \ + njs_djb_hash_add( \ + njs_djb_hash_add( \ + njs_djb_hash_add( \ + njs_djb_hash_add(NJS_DJB_HASH_INIT, \ + 'U'), 'i'), 'n'), 't'), '1'), '6'), 'A'), 'r'), 'r'), 'a'), 'y') + + +#define NJS_UINT32ARRAY_HASH \ + njs_djb_hash_add( \ + njs_djb_hash_add( \ + njs_djb_hash_add( \ + njs_djb_hash_add( \ + njs_djb_hash_add( \ + njs_djb_hash_add( \ + njs_djb_hash_add( \ + njs_djb_hash_add( \ + njs_djb_hash_add( \ + njs_djb_hash_add( \ + njs_djb_hash_add(NJS_DJB_HASH_INIT, \ + 'U'), 'i'), 'n'), 't'), '3'), '2'), 'A'), 'r'), 'r'), 'a'), 'y') + + +#define NJS_INT8ARRAY_HASH \ + njs_djb_hash_add( \ + njs_djb_hash_add( \ + njs_djb_hash_add( \ + njs_djb_hash_add( \ + njs_djb_hash_add( \ + njs_djb_hash_add( \ + njs_djb_hash_add( \ + njs_djb_hash_add( \ + njs_djb_hash_add(NJS_DJB_HASH_INIT, \ + 'I'), 'n'), 't'), '8'), 'A'), 'r'), 'r'), 'a'), 'y') + + +#define NJS_INT16ARRAY_HASH \ + njs_djb_hash_add( \ + njs_djb_hash_add( \ + njs_djb_hash_add( \ + njs_djb_hash_add( \ + njs_djb_hash_add( \ + njs_djb_hash_add( \ + njs_djb_hash_add( \ + njs_djb_hash_add( \ + njs_djb_hash_add( \ + njs_djb_hash_add(NJS_DJB_HASH_INIT, \ + 'I'), 'n'), 't'), '1'), '6'), 'A'), 'r'), 'r'), 'a'), 'y') + + +#define NJS_INT32ARRAY_HASH \ + njs_djb_hash_add( \ + njs_djb_hash_add( \ + njs_djb_hash_add( \ + njs_djb_hash_add( \ + njs_djb_hash_add( \ + njs_djb_hash_add( \ + njs_djb_hash_add( \ + njs_djb_hash_add( \ + njs_djb_hash_add( \ + njs_djb_hash_add(NJS_DJB_HASH_INIT, \ + 'I'), 'n'), 't'), '3'), '2'), 'A'), 'r'), 'r'), 'a'), 'y') + + +#define NJS_FLOAT32ARRAY_HASH \ + njs_djb_hash_add( \ + njs_djb_hash_add( \ + njs_djb_hash_add( \ + njs_djb_hash_add( \ + njs_djb_hash_add( \ + njs_djb_hash_add( \ + njs_djb_hash_add( \ + njs_djb_hash_add( \ + njs_djb_hash_add( \ + njs_djb_hash_add( \ + njs_djb_hash_add( \ + njs_djb_hash_add(NJS_DJB_HASH_INIT, \ + 'F'), 'l'), 'o'), 'a'), 't'), '3'), '2'), 'A'), 'r'), 'r'), 'a'), 'y') + + +#define NJS_FLOAT64ARRAY_HASH \ + njs_djb_hash_add( \ + njs_djb_hash_add( \ + njs_djb_hash_add( \ + njs_djb_hash_add( \ + njs_djb_hash_add( \ + njs_djb_hash_add( \ + njs_djb_hash_add( \ + njs_djb_hash_add( \ + njs_djb_hash_add( \ + njs_djb_hash_add( \ + njs_djb_hash_add( \ + njs_djb_hash_add(NJS_DJB_HASH_INIT, \ + 'F'), 'l'), 'o'), 'a'), 't'), '6'), '4'), 'A'), 'r'), 'r'), 'a'), 'y') + + +#define NJS_UINT8CLAMPEDARRAY_HASH \ + njs_djb_hash_add( \ + njs_djb_hash_add( \ + njs_djb_hash_add( \ + njs_djb_hash_add( \ + njs_djb_hash_add( \ + njs_djb_hash_add( \ + njs_djb_hash_add( \ + njs_djb_hash_add( \ + njs_djb_hash_add( \ + njs_djb_hash_add( \ + njs_djb_hash_add( \ + njs_djb_hash_add( \ + njs_djb_hash_add( \ + njs_djb_hash_add( \ + njs_djb_hash_add( \ + njs_djb_hash_add( \ + njs_djb_hash_add(NJS_DJB_HASH_INIT, \ + 'U'), 'i'), 'n'), 't'), '8'), 'C'), 'l'), 'a'), 'm'), 'p'), 'e'), \ + 'd'), 'A'), 'r'), 'r'), 'a'), 'y') + + #endif /* _NJS_OBJECT_HASH_H_INCLUDED_ */ diff -r 99f9008e1b17 -r 1c3c593cc3fd src/njs_object_prop.c --- a/src/njs_object_prop.c Tue Dec 17 18:22:50 2019 +0300 +++ b/src/njs_object_prop.c Wed Dec 25 15:59:01 2019 +0300 @@ -164,6 +164,16 @@ njs_object_prop_define(njs_vm_t *vm, njs return NJS_ERROR; } + if (njs_slow_path(njs_is_typed_array(object) + && njs_is_string(name))) + { + /* Integer-Indexed Exotic Objects [[DefineOwnProperty]]. */ + if (!isnan(njs_string_to_index(name))) { + njs_type_error(vm, "Invalid typed array index"); + return NJS_ERROR; + } + } + /* 6.2.5.6 CompletePropertyDescriptor */ if (njs_is_accessor_descriptor(prop)) { @@ -234,6 +244,26 @@ njs_object_prop_define(njs_vm_t *vm, njs return NJS_OK; + case NJS_PROPERTY_TYPED_ARRAY_REF: + if (njs_is_accessor_descriptor(prop)) { + goto exception; + } + + if (prop->configurable == NJS_ATTRIBUTE_TRUE || + prop->enumerable == NJS_ATTRIBUTE_FALSE || + prop->writable == NJS_ATTRIBUTE_FALSE) + { + goto exception; + } + + if (njs_is_valid(&prop->value)) { + return njs_typed_array_set_value(vm, njs_typed_array(&prev->value), + prev->value.data.magic32, + &prop->value); + } + + return NJS_OK; + default: njs_internal_error(vm, "unexpected property type \"%s\" " "while defining property", diff -r 99f9008e1b17 -r 1c3c593cc3fd src/njs_string.c --- a/src/njs_string.c Tue Dec 17 18:22:50 2019 +0300 +++ b/src/njs_string.c Wed Dec 25 15:59:01 2019 +0300 @@ -4342,32 +4342,85 @@ njs_string_to_number(const njs_value_t * double njs_string_to_index(const njs_value_t *value) { + size_t size, len; double num; - size_t size; - const u_char *p, *end; + njs_bool_t minus; + const u_char *p, *start, *end; + u_char buf[128]; size = value->short_string.size; if (size != NJS_STRING_LONG) { - p = value->short_string.start; + start = value->short_string.start; } else { size = value->long_string.size; - p = value->long_string.data->start; - } - - if (size == 0) { + start = value->long_string.data->start; + } + + p = start; + end = p + size; + minus = 0; + + if (size > 1) { + switch (p[0]) { + case '0': + if (size != 1) { + return NAN; + } + + /* Fall through. */ + + case '1': + case '2': + case '3': + case '4': + case '5': + case '6': + case '7': + case '8': + case '9': + break; + + case '-': + if (size == 2 && p[1] == '0') { + return -0.0; + } + + if (size == njs_length("-Infinity") + && memcmp(&p[1], "Infinity", njs_length("Infinity")) == 0) + { + return -INFINITY; + } + + p++; + minus = 1; + + break; + + case 'I': + if (size == njs_length("Infinity") + && memcmp(p, "Infinity", njs_length("Infinity")) == 0) + { + return INFINITY; + } + + /* Fall through. */ + + default: + return NAN; + } + } + + num = njs_strtod(&p, end); + if (p != end) { return NAN; } - if (*p == '0' && size > 1) { - return NAN; - } - - end = p + size; - num = njs_number_dec_parse(&p, end); - - if (p != end) { + num = minus ? -num : num; + + len = njs_dtoa(num, (char *) buf); + if (size != len || memcmp(start, buf, size) != 0) { return NAN; } diff -r 99f9008e1b17 -r 1c3c593cc3fd src/njs_typed_array.c --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/njs_typed_array.c Wed Dec 25 15:59:01 2019 +0300 @@ -0,0 +1,1751 @@ + +/* + * Copyright (C) Igor Sysoev + * Copyright (C) NGINX, Inc. + */ + + +#include + + +static njs_int_t +njs_typed_array_constructor(njs_vm_t *vm, njs_value_t *args, njs_uint_t nargs, + njs_index_t magic) +{ + double num; + uint32_t i, length, element_size; + uint64_t size, offset; + njs_int_t ret; + njs_value_t *value, index, prop; + njs_array_t *src_array; + njs_object_type_t type; + njs_typed_array_t *array, *src_tarray; + njs_array_buffer_t *buffer; + + size = 0; + offset = 0; + + buffer = NULL; + src_array = NULL; + src_tarray = NULL; + From xeioex at nginx.com Wed Dec 25 15:24:49 2019 From: xeioex at nginx.com (Dmitry Volyntsev) Date: Wed, 25 Dec 2019 15:24:49 +0000 Subject: [njs] Fixed import segfault. Message-ID: details: https://hg.nginx.org/njs/rev/0f31637138ce branches: changeset: 1290:0f31637138ce user: hongzhidao date: Fri Dec 13 12:47:37 2019 +0800 description: Fixed import segfault. This closes #274 issue on Github. diffstat: src/njs_module.c | 1 + test/module/lib1.js | 6 ++++++ 2 files changed, 7 insertions(+), 0 deletions(-) diffs (24 lines): diff -r 1c3c593cc3fd -r 0f31637138ce src/njs_module.c --- a/src/njs_module.c Wed Dec 25 15:59:01 2019 +0300 +++ b/src/njs_module.c Fri Dec 13 12:47:37 2019 +0800 @@ -175,6 +175,7 @@ njs_parser_module(njs_vm_t *vm, njs_pars goto fail; } + module->function.args_offset = 1; module->function.u.lambda = parser->node->u.value.data.u.lambda; njs_mp_free(vm->mem_pool, text.start); diff -r 1c3c593cc3fd -r 0f31637138ce test/module/lib1.js --- a/test/module/lib1.js Wed Dec 25 15:59:01 2019 +0300 +++ b/test/module/lib1.js Fri Dec 13 12:47:37 2019 +0800 @@ -1,3 +1,9 @@ +var foo = (function(){ + return (function f() {}) +}); + +foo()({1:[]}) + function hash() { var h = crypto.createHash('md5'); var v = h.update('AB').digest('hex'); From xeioex at nginx.com Thu Dec 26 11:54:46 2019 From: xeioex at nginx.com (Dmitry Volyntsev) Date: Thu, 26 Dec 2019 11:54:46 +0000 Subject: [njs] Shell: stopping events handling for uncaught exception in file mode. Message-ID: details: https://hg.nginx.org/njs/rev/5fbc49bc5a7e branches: changeset: 1291:5fbc49bc5a7e user: Dmitry Volyntsev date: Thu Dec 26 14:53:52 2019 +0300 description: Shell: stopping events handling for uncaught exception in file mode. This closes #269 issue on Github. diffstat: src/njs_shell.c | 8 ++++++++ test/njs_expect_test.exp | 11 +++++++++++ 2 files changed, 19 insertions(+), 0 deletions(-) diffs (46 lines): diff -r 0f31637138ce -r 5fbc49bc5a7e src/njs_shell.c --- a/src/njs_shell.c Fri Dec 13 12:47:37 2019 +0800 +++ b/src/njs_shell.c Thu Dec 26 14:53:52 2019 +0300 @@ -809,6 +809,10 @@ njs_process_script(njs_opts_t *opts, njs njs_output(opts, vm, ret); + if (!opts->interactive && ret == NJS_ERROR) { + return NJS_ERROR; + } + for ( ;; ) { if (!njs_vm_pending(vm)) { break; @@ -833,6 +837,10 @@ njs_process_script(njs_opts_t *opts, njs if (ret == NJS_ERROR) { njs_output(opts, vm, ret); + + if (!opts->interactive) { + return NJS_ERROR; + } } } diff -r 0f31637138ce -r 5fbc49bc5a7e test/njs_expect_test.exp --- a/test/njs_expect_test.exp Fri Dec 13 12:47:37 2019 +0800 +++ b/test/njs_expect_test.exp Thu Dec 26 14:53:52 2019 +0300 @@ -699,6 +699,17 @@ njs_test { "Error: Not a directory*"} } +njs_run {"-c" "setTimeout(() => {console.log('A'.repeat(1024))}, 0); ref"} \ +"^Thrown: +ReferenceError: \"ref\" is not defined in string:1 + at main \\\(native\\\)\n$" + +njs_run {"-c" "setTimeout(() => {ref}, 0); setTimeout(() => {console.log('A'.repeat(1024))}, 0)"} \ +"^Thrown: +ReferenceError: \"ref\" is not defined in string:1 + at anonymous \\\(native\\\) + at main \\\(native\\\)\n$" + # Modules njs_run {"-p" "test/module/libs" "./test/module/normal.js"} \ From alexander.borisov at nginx.com Thu Dec 26 12:32:29 2019 From: alexander.borisov at nginx.com (Alexander Borisov) Date: Thu, 26 Dec 2019 12:32:29 +0000 Subject: [njs] Fixed njs_vm_handle_events(). Message-ID: details: https://hg.nginx.org/njs/rev/025ce39e20b1 branches: changeset: 1292:025ce39e20b1 user: Alexander Borisov date: Thu Dec 26 15:31:54 2019 +0300 description: Fixed njs_vm_handle_events(). Fixed typo introduced in 61bf7a31e685. Event loop is expected to process two queues: posted events and promise events according to the spec. The "do while" loop was introduced to handle possible promise events which may appear as a result of posted events processing. The loop termination condition should check for emptyness of promise events queue, because posted events can only be created outside the loop (externally). diffstat: src/njs_vm.c | 23 +++++++++++------------ 1 files changed, 11 insertions(+), 12 deletions(-) diffs (52 lines): diff -r 5fbc49bc5a7e -r 025ce39e20b1 src/njs_vm.c --- a/src/njs_vm.c Thu Dec 26 14:53:52 2019 +0300 +++ b/src/njs_vm.c Thu Dec 26 15:31:54 2019 +0300 @@ -493,16 +493,17 @@ njs_vm_handle_events(njs_vm_t *vm) { njs_int_t ret; njs_event_t *ev; - njs_queue_t *events; + njs_queue_t *promise_events, *posted_events; njs_queue_link_t *link; + promise_events = &vm->promise_events; + posted_events = &vm->posted_events; + do { - events = &vm->promise_events; + for ( ;; ) { + link = njs_queue_first(promise_events); - for ( ;; ) { - link = njs_queue_first(events); - - if (link == njs_queue_tail(events)) { + if (link == njs_queue_tail(promise_events)) { break; } @@ -516,12 +517,10 @@ njs_vm_handle_events(njs_vm_t *vm) } } - events = &vm->posted_events; + for ( ;; ) { + link = njs_queue_first(posted_events); - for ( ;; ) { - link = njs_queue_first(events); - - if (link == njs_queue_tail(events)) { + if (link == njs_queue_tail(posted_events)) { break; } @@ -542,7 +541,7 @@ njs_vm_handle_events(njs_vm_t *vm) } } - } while (!njs_queue_is_empty(events)); + } while (!njs_queue_is_empty(promise_events)); return njs_posted_events(vm) ? NJS_AGAIN : NJS_OK; } From mdounin at mdounin.ru Fri Dec 27 14:20:57 2019 From: mdounin at mdounin.ru (Maxim Dounin) Date: Fri, 27 Dec 2019 14:20:57 +0000 Subject: [nginx] Version bump. Message-ID: details: https://hg.nginx.org/nginx/rev/44534144dcd4 branches: changeset: 7615:44534144dcd4 user: Maxim Dounin date: Fri Dec 27 17:20:20 2019 +0300 description: Version bump. diffstat: src/core/nginx.h | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) diffs (14 lines): diff --git a/src/core/nginx.h b/src/core/nginx.h --- a/src/core/nginx.h +++ b/src/core/nginx.h @@ -9,8 +9,8 @@ #define _NGINX_H_INCLUDED_ -#define nginx_version 1017007 -#define NGINX_VERSION "1.17.7" +#define nginx_version 1017008 +#define NGINX_VERSION "1.17.8" #define NGINX_VER "nginx/" NGINX_VERSION #ifdef NGX_BUILD From mdounin at mdounin.ru Fri Dec 27 14:21:00 2019 From: mdounin at mdounin.ru (Maxim Dounin) Date: Fri, 27 Dec 2019 14:21:00 +0000 Subject: [nginx] Trailing space removed. Message-ID: details: https://hg.nginx.org/nginx/rev/fd4d2155d3e6 branches: changeset: 7616:fd4d2155d3e6 user: Maxim Dounin date: Fri Dec 27 17:20:25 2019 +0300 description: Trailing space removed. diffstat: docs/xml/nginx/changes.xml | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diffs (12 lines): diff --git a/docs/xml/nginx/changes.xml b/docs/xml/nginx/changes.xml --- a/docs/xml/nginx/changes.xml +++ b/docs/xml/nginx/changes.xml @@ -27,7 +27,7 @@ was used in the configuration. ??? ?????????? proxy_pass ? URI. -a segmentation fault might occur in a worker process +a segmentation fault might occur in a worker process if the "break" directive was used with the "alias" directive or with the "proxy_pass" directive with a URI. From xeioex at nginx.com Fri Dec 27 16:17:39 2019 From: xeioex at nginx.com (Dmitry Volyntsev) Date: Fri, 27 Dec 2019 16:17:39 +0000 Subject: [njs] Fixed "length" may be uninitialized in njs_typed_array_constructor(). Message-ID: details: https://hg.nginx.org/njs/rev/74d2b2074ec0 branches: changeset: 1293:74d2b2074ec0 user: Dmitry Volyntsev date: Fri Dec 27 19:17:26 2019 +0300 description: Fixed "length" may be uninitialized in njs_typed_array_constructor(). Found by Clang static analyzer. diffstat: src/njs_typed_array.c | 1 + 1 files changed, 1 insertions(+), 0 deletions(-) diffs (11 lines): diff -r 025ce39e20b1 -r 74d2b2074ec0 src/njs_typed_array.c --- a/src/njs_typed_array.c Thu Dec 26 15:31:54 2019 +0300 +++ b/src/njs_typed_array.c Fri Dec 27 19:17:26 2019 +0300 @@ -23,6 +23,7 @@ njs_typed_array_constructor(njs_vm_t *vm njs_array_buffer_t *buffer; size = 0; + length = 0; offset = 0; buffer = NULL; From mdounin at mdounin.ru Fri Dec 27 16:52:31 2019 From: mdounin at mdounin.ru (Maxim Dounin) Date: Fri, 27 Dec 2019 16:52:31 +0000 Subject: [nginx] SSL: reworked posted next events again. Message-ID: details: https://hg.nginx.org/nginx/rev/f1720934c45b branches: changeset: 7617:f1720934c45b user: Maxim Dounin date: Fri Dec 27 19:43:01 2019 +0300 description: SSL: reworked posted next events again. Previous change 1ce3f01a4355 incorrectly introduced processing of the ngx_posted_next_events queue at the end of operation, effectively making posted next events a nop, since at the end of an event loop iteration the queue is always empty. Correct approach is to move events to the ngx_posted_events queue at an iteration start, as it was done previously. Further, in some cases the c->read event might be already in the ngx_posted_events queue, and calling ngx_post_event() with the ngx_posted_next_events queue won't do anything. To make sure the event will be correctly placed into the ngx_posted_next_events queue we now check if it is already posted. diffstat: src/event/ngx_event.c | 2 +- src/event/ngx_event_openssl.c | 4 ++++ src/event/ngx_event_posted.c | 22 ++++++++++------------ src/event/ngx_event_posted.h | 2 +- 4 files changed, 16 insertions(+), 14 deletions(-) diffs (85 lines): diff --git a/src/event/ngx_event.c b/src/event/ngx_event.c --- a/src/event/ngx_event.c +++ b/src/event/ngx_event.c @@ -238,6 +238,7 @@ ngx_process_events_and_timers(ngx_cycle_ } if (!ngx_queue_empty(&ngx_posted_next_events)) { + ngx_event_move_posted_next(cycle); timer = 0; } @@ -261,7 +262,6 @@ ngx_process_events_and_timers(ngx_cycle_ } ngx_event_process_posted(cycle, &ngx_posted_events); - ngx_event_process_posted_next(cycle, &ngx_posted_next_events); } diff --git a/src/event/ngx_event_openssl.c b/src/event/ngx_event_openssl.c --- a/src/event/ngx_event_openssl.c +++ b/src/event/ngx_event_openssl.c @@ -2017,6 +2017,10 @@ ngx_ssl_recv(ngx_connection_t *c, u_char c->read->available = 0; c->read->ready = 0; + if (c->read->posted) { + ngx_delete_posted_event(c->read); + } + ngx_post_event(c->read, &ngx_posted_next_events); } diff --git a/src/event/ngx_event_posted.c b/src/event/ngx_event_posted.c --- a/src/event/ngx_event_posted.c +++ b/src/event/ngx_event_posted.c @@ -37,26 +37,24 @@ ngx_event_process_posted(ngx_cycle_t *cy void -ngx_event_process_posted_next(ngx_cycle_t *cycle, ngx_queue_t *posted) +ngx_event_move_posted_next(ngx_cycle_t *cycle) { ngx_queue_t *q; ngx_event_t *ev; - while (!ngx_queue_empty(posted)) { - - q = ngx_queue_head(posted); + for (q = ngx_queue_head(&ngx_posted_next_events); + q != ngx_queue_sentinel(&ngx_posted_next_events); + q = ngx_queue_next(q)) + { ev = ngx_queue_data(q, ngx_event_t, queue); ngx_log_debug1(NGX_LOG_DEBUG_EVENT, cycle->log, 0, "posted next event %p", ev); - ngx_delete_posted_event(ev); + ev->ready = 1; + ev->available = -1; + } - if (!ev->ready) { - ev->ready = 1; - ev->available = -1; - } - - ev->handler(ev); - } + ngx_queue_add(&ngx_posted_events, &ngx_posted_next_events); + ngx_queue_init(&ngx_posted_next_events); } diff --git a/src/event/ngx_event_posted.h b/src/event/ngx_event_posted.h --- a/src/event/ngx_event_posted.h +++ b/src/event/ngx_event_posted.h @@ -39,7 +39,7 @@ void ngx_event_process_posted(ngx_cycle_t *cycle, ngx_queue_t *posted); -void ngx_event_process_posted_next(ngx_cycle_t *cycle, ngx_queue_t *posted); +void ngx_event_move_posted_next(ngx_cycle_t *cycle); extern ngx_queue_t ngx_posted_accept_events; From marcinguy at gmail.com Fri Dec 27 22:42:30 2019 From: marcinguy at gmail.com (Marcin Kozlowski) Date: Fri, 27 Dec 2019 23:42:30 +0100 Subject: nginx - get value of the header - x_forwarded_for in Nginx module (Naxsi) In-Reply-To: <20191225111726.GA72241@lo0.su> References: <20191224093742.GA47591@lo0.su> <20191225111726.GA72241@lo0.su> Message-ID: Thanks. Almost got my modifications to NAXSI ready. But currently have a blocker with getting just the X-Forwarded-for IP The code below: ngx_uint_t n; ngx_table_elt_t **h; ngx_array_t a; a = req->headers_in.x_forwarded_for; n = a.nelts; h = a.elts; for (i = 0; iconnection->log, 0, "x_forwarded_for: %s", h[i]->value.data); } gets a String with several IP (i.e client, server, request etc) Tried to parse the string using strtok(), interating through it .... but it segfaults. I guess I am missing some NGINX module knowledge. How to properly get first string up to first "," from the h[i]->value.data using NGINX functions/types or other correct way to do it. Thanks, On Wed, Dec 25, 2019 at 12:17 PM Ruslan Ermilov wrote: > On Tue, Dec 24, 2019 at 08:00:26PM +0100, Marcin Kozlowski wrote: > > Thanks. > > > > Works. For the reference, this is the code I used: > > > > ngx_uint_t n; > > ngx_table_elt_t **h; > > ngx_array_t a; > > a = req->headers_in.x_forwarded_for; > > n = a.nelts; > > h = a.elts; > > > > > > for (i = 0; i > ngx_log_error(NGX_LOG_ERR, req->connection->log, > > 0, "x_forwarded_for: %s", h[i]->value.data); > > } > > > > BTW What would be the best practice in NGINX NASIX module or any other > > module to load a file with hundreds entries of IPs (hashmap, or what > > structure would be best?) which should be whitelisted later for > comparison > > in NASIX module logic. Those IP should never be blocked by NAXSI. > > > > When should I load this file in memory, in which component > > /module/function/step? > > > > Links to some guides/sample code would be also appreciated. > > > > Thanks, > > http://nginx.org/en/docs/http/ngx_http_geo_module.html > _______________________________________________ > nginx-devel mailing list > nginx-devel at nginx.org > http://mailman.nginx.org/mailman/listinfo/nginx-devel > -------------- next part -------------- An HTML attachment was scrubbed... URL: From marcinguy at gmail.com Sat Dec 28 14:36:02 2019 From: marcinguy at gmail.com (Marcin Kozlowski) Date: Sat, 28 Dec 2019 15:36:02 +0100 Subject: nginx - get value of the header - x_forwarded_for in Nginx module (Naxsi) In-Reply-To: References: <20191224093742.GA47591@lo0.su> <20191225111726.GA72241@lo0.su> Message-ID: Ignore my last post. My error. Thanks, On Fri, Dec 27, 2019 at 11:42 PM Marcin Kozlowski wrote: > Thanks. Almost got my modifications to NAXSI ready. > > But currently have a blocker with getting just the X-Forwarded-for IP > > The code below: > > ngx_uint_t n; > ngx_table_elt_t **h; > ngx_array_t a; > a = req->headers_in.x_forwarded_for; > n = a.nelts; > h = a.elts; > > > for (i = 0; i ngx_log_error(NGX_LOG_ERR, req->connection->log, > 0, "x_forwarded_for: %s", h[i]->value.data); > } > > gets a String with several IP (i.e client, server, request etc) > > Tried to parse the string using strtok(), interating through it .... but > it segfaults. I guess I am missing some NGINX module knowledge. > > How to properly get first string up to first "," from the h[i]->value.data > using NGINX functions/types or other correct way to do it. > > Thanks, > > > > > > On Wed, Dec 25, 2019 at 12:17 PM Ruslan Ermilov wrote: > >> On Tue, Dec 24, 2019 at 08:00:26PM +0100, Marcin Kozlowski wrote: >> > Thanks. >> > >> > Works. For the reference, this is the code I used: >> > >> > ngx_uint_t n; >> > ngx_table_elt_t **h; >> > ngx_array_t a; >> > a = req->headers_in.x_forwarded_for; >> > n = a.nelts; >> > h = a.elts; >> > >> > >> > for (i = 0; i> > ngx_log_error(NGX_LOG_ERR, req->connection->log, >> > 0, "x_forwarded_for: %s", h[i]->value.data); >> > } >> > >> > BTW What would be the best practice in NGINX NASIX module or any other >> > module to load a file with hundreds entries of IPs (hashmap, or what >> > structure would be best?) which should be whitelisted later for >> comparison >> > in NASIX module logic. Those IP should never be blocked by NAXSI. >> > >> > When should I load this file in memory, in which component >> > /module/function/step? >> > >> > Links to some guides/sample code would be also appreciated. >> > >> > Thanks, >> >> http://nginx.org/en/docs/http/ngx_http_geo_module.html >> _______________________________________________ >> nginx-devel mailing list >> nginx-devel at nginx.org >> http://mailman.nginx.org/mailman/listinfo/nginx-devel >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From marcinguy at gmail.com Sat Dec 28 19:35:25 2019 From: marcinguy at gmail.com (Marcin Kozlowski) Date: Sat, 28 Dec 2019 20:35:25 +0100 Subject: nginx - get value of the header - x_forwarded_for in Nginx module (Naxsi) In-Reply-To: References: <20191224093742.GA47591@lo0.su> <20191225111726.GA72241@lo0.su> Message-ID: Still have few questions. Help would be great Beginner in writing NGINX modules Have this question 1) How do I create array, add element to it and than create a hashtable from it. Below I try to achieve it: NX_LOG_DEBUG(_debug_whitelist_heavy, NGX_LOG_EMERG, cf, 0, "finalizing hashtables array %i", dlc->pass_rules->nelts); headers_ar = ngx_array_create(cf->pool, dlc->pass_rules->nelts, sizeof(ngx_hash_key_t)); if (headers_ar) { NX_LOG_DEBUG(_debug_readconf, NGX_LOG_EMERG, cf, 0, "headers array %i",headers_ar->nelts); 2) Why headers_ar has 0 elemets nginx: [emerg] finalizing hashtables array 6 in /etc/nginx/nginx.conf:124 nginx: [emerg] headers array 0 in /etc/nginx/nginx.conf:124 3) I later want to build hashtable based on this array: dlc->passr_headers_hash = (ngx_hash_t*) ngx_pcalloc(cf->pool, sizeof(ngx_hash_t)); hash_init.hash = dlc->passr_headers_hash; hash_init.name = "passr_headers_hash"; if (ngx_hash_init(&hash_init, (ngx_hash_key_t*) headers_ar->elts, headers_ar->nelts) != NGX_OK) { ngx_conf_log_error(NGX_LOG_EMERG, cf, 0, "$HEADERS hashtable init failed"); /* LCOV_EXCL_LINE */ return (NGX_ERROR); /* LCOV_EXCL_LINE */ } else { NX_LOG_DEBUG(_debug_whitelist, NGX_LOG_EMERG, cf, 0, "$HEADERS hashtable init successed %d !", dlc->passr_headers_hash->size); } 4) Can somebody post simple code that a) creates array b) pushes one IP c) create hashtable from it Thanks, On Sat, Dec 28, 2019 at 3:36 PM Marcin Kozlowski wrote: > Ignore my last post. > > My error. > > Thanks, > > On Fri, Dec 27, 2019 at 11:42 PM Marcin Kozlowski > wrote: > >> Thanks. Almost got my modifications to NAXSI ready. >> >> But currently have a blocker with getting just the X-Forwarded-for IP >> >> The code below: >> >> ngx_uint_t n; >> ngx_table_elt_t **h; >> ngx_array_t a; >> a = req->headers_in.x_forwarded_for; >> n = a.nelts; >> h = a.elts; >> >> >> for (i = 0; i> ngx_log_error(NGX_LOG_ERR, req->connection->log, >> 0, "x_forwarded_for: %s", h[i]->value.data); >> } >> >> gets a String with several IP (i.e client, server, request etc) >> >> Tried to parse the string using strtok(), interating through it .... but >> it segfaults. I guess I am missing some NGINX module knowledge. >> >> How to properly get first string up to first "," from the >> h[i]->value.data using NGINX functions/types or other correct way to do it. >> >> Thanks, >> >> >> >> >> >> On Wed, Dec 25, 2019 at 12:17 PM Ruslan Ermilov wrote: >> >>> On Tue, Dec 24, 2019 at 08:00:26PM +0100, Marcin Kozlowski wrote: >>> > Thanks. >>> > >>> > Works. For the reference, this is the code I used: >>> > >>> > ngx_uint_t n; >>> > ngx_table_elt_t **h; >>> > ngx_array_t a; >>> > a = req->headers_in.x_forwarded_for; >>> > n = a.nelts; >>> > h = a.elts; >>> > >>> > >>> > for (i = 0; i>> > ngx_log_error(NGX_LOG_ERR, req->connection->log, >>> > 0, "x_forwarded_for: %s", h[i]->value.data); >>> > } >>> > >>> > BTW What would be the best practice in NGINX NASIX module or any other >>> > module to load a file with hundreds entries of IPs (hashmap, or what >>> > structure would be best?) which should be whitelisted later for >>> comparison >>> > in NASIX module logic. Those IP should never be blocked by NAXSI. >>> > >>> > When should I load this file in memory, in which component >>> > /module/function/step? >>> > >>> > Links to some guides/sample code would be also appreciated. >>> > >>> > Thanks, >>> >>> http://nginx.org/en/docs/http/ngx_http_geo_module.html >>> _______________________________________________ >>> nginx-devel mailing list >>> nginx-devel at nginx.org >>> http://mailman.nginx.org/mailman/listinfo/nginx-devel >>> >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From rpaprocki at fearnothingproductions.net Sat Dec 28 20:48:52 2019 From: rpaprocki at fearnothingproductions.net (Robert Paprocki) Date: Sat, 28 Dec 2019 12:48:52 -0800 Subject: nginx - get value of the header - x_forwarded_for in Nginx module (Naxsi) In-Reply-To: References: Message-ID: The array has 0 nelts because you haven?t added anything to do. nelts is the number of elements in the array, not the size of the array. Add an element to the array with ngx_array_push(). > On Dec 28, 2019, at 11:35, Marcin Kozlowski wrote: > > ? > Still have few questions. Help would be great > > Beginner in writing NGINX modules > > Have this question > > 1) How do I create array, add element to it and than create a hashtable from it. > > Below I try to achieve it: > > NX_LOG_DEBUG(_debug_whitelist_heavy, > NGX_LOG_EMERG, cf, 0, > "finalizing hashtables array %i", dlc->pass_rules->nelts); > > headers_ar = ngx_array_create(cf->pool, dlc->pass_rules->nelts, sizeof(ngx_hash_key_t)); > > > > if (headers_ar) { > NX_LOG_DEBUG(_debug_readconf, NGX_LOG_EMERG, cf, 0, > "headers array %i",headers_ar->nelts); > > 2) Why headers_ar has 0 elemets > > nginx: [emerg] finalizing hashtables array 6 in /etc/nginx/nginx.conf:124 > nginx: [emerg] headers array 0 in /etc/nginx/nginx.conf:124 > > > > 3) I later want to build hashtable based on this array: > > dlc->passr_headers_hash = (ngx_hash_t*) ngx_pcalloc(cf->pool, sizeof(ngx_hash_t)); > hash_init.hash = dlc->passr_headers_hash; > hash_init.name = "passr_headers_hash"; > > > > if (ngx_hash_init(&hash_init, (ngx_hash_key_t*) headers_ar->elts, > headers_ar->nelts) != NGX_OK) { > ngx_conf_log_error(NGX_LOG_EMERG, cf, 0, "$HEADERS hashtable init failed"); /* LCOV_EXCL_LINE */ > return (NGX_ERROR); /* LCOV_EXCL_LINE */ > } > > else { > NX_LOG_DEBUG(_debug_whitelist, NGX_LOG_EMERG, cf, 0, "$HEADERS hashtable init successed %d !", > dlc->passr_headers_hash->size); > } > > > > 4) Can somebody post simple code that > > a) creates array > b) pushes one IP > c) create hashtable from it > > Thanks, > > >> On Sat, Dec 28, 2019 at 3:36 PM Marcin Kozlowski wrote: >> Ignore my last post. >> >> My error. >> >> Thanks, >> >>> On Fri, Dec 27, 2019 at 11:42 PM Marcin Kozlowski wrote: >>> Thanks. Almost got my modifications to NAXSI ready. >>> >>> But currently have a blocker with getting just the X-Forwarded-for IP >>> >>> The code below: >>> >>> ngx_uint_t n; >>> ngx_table_elt_t **h; >>> ngx_array_t a; >>> a = req->headers_in.x_forwarded_for; >>> n = a.nelts; >>> h = a.elts; >>> >>> >>> for (i = 0; i>> ngx_log_error(NGX_LOG_ERR, req->connection->log, >>> 0, "x_forwarded_for: %s", h[i]->value.data); >>> } >>> >>> gets a String with several IP (i.e client, server, request etc) >>> >>> Tried to parse the string using strtok(), interating through it .... but it segfaults. I guess I am missing some NGINX module knowledge. >>> >>> How to properly get first string up to first "," from the h[i]->value.data using NGINX functions/types or other correct way to do it. >>> >>> Thanks, >>> >>> >>> >>> >>> >>>> On Wed, Dec 25, 2019 at 12:17 PM Ruslan Ermilov wrote: >>>> On Tue, Dec 24, 2019 at 08:00:26PM +0100, Marcin Kozlowski wrote: >>>> > Thanks. >>>> > >>>> > Works. For the reference, this is the code I used: >>>> > >>>> > ngx_uint_t n; >>>> > ngx_table_elt_t **h; >>>> > ngx_array_t a; >>>> > a = req->headers_in.x_forwarded_for; >>>> > n = a.nelts; >>>> > h = a.elts; >>>> > >>>> > >>>> > for (i = 0; i>>> > ngx_log_error(NGX_LOG_ERR, req->connection->log, >>>> > 0, "x_forwarded_for: %s", h[i]->value.data); >>>> > } >>>> > >>>> > BTW What would be the best practice in NGINX NASIX module or any other >>>> > module to load a file with hundreds entries of IPs (hashmap, or what >>>> > structure would be best?) which should be whitelisted later for comparison >>>> > in NASIX module logic. Those IP should never be blocked by NAXSI. >>>> > >>>> > When should I load this file in memory, in which component >>>> > /module/function/step? >>>> > >>>> > Links to some guides/sample code would be also appreciated. >>>> > >>>> > Thanks, >>>> >>>> http://nginx.org/en/docs/http/ngx_http_geo_module.html >>>> _______________________________________________ >>>> nginx-devel mailing list >>>> nginx-devel at nginx.org >>>> http://mailman.nginx.org/mailman/listinfo/nginx-devel > _______________________________________________ > nginx-devel mailing list > nginx-devel at nginx.org > http://mailman.nginx.org/mailman/listinfo/nginx-devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From hungnv at opensource.com.vn Sun Dec 29 02:27:02 2019 From: hungnv at opensource.com.vn (Hung Nguyen) Date: Sun, 29 Dec 2019 09:27:02 +0700 Subject: nginx - get value of the header - x_forwarded_for in Nginx module (Naxsi) In-Reply-To: References: Message-ID: Hello, I suggest you to read nginx development guide which is available on nginx.org. About getting x forward for headers, if you take a look at some built in module there?s already exist implementation which takes the headers and returns an array. -- H?ng > On Dec 29, 2019, at 03:49, Robert Paprocki wrote: > > ?The array has 0 nelts because you haven?t added anything to do. nelts is the number of elements in the array, not the size of the array. > > Add an element to the array with ngx_array_push(). > >>> On Dec 28, 2019, at 11:35, Marcin Kozlowski wrote: >>> >> ? >> Still have few questions. Help would be great >> >> Beginner in writing NGINX modules >> >> Have this question >> >> 1) How do I create array, add element to it and than create a hashtable from it. >> >> Below I try to achieve it: >> >> NX_LOG_DEBUG(_debug_whitelist_heavy, >> NGX_LOG_EMERG, cf, 0, >> "finalizing hashtables array %i", dlc->pass_rules->nelts); >> >> headers_ar = ngx_array_create(cf->pool, dlc->pass_rules->nelts, sizeof(ngx_hash_key_t)); >> >> >> >> if (headers_ar) { >> NX_LOG_DEBUG(_debug_readconf, NGX_LOG_EMERG, cf, 0, >> "headers array %i",headers_ar->nelts); >> >> 2) Why headers_ar has 0 elemets >> >> nginx: [emerg] finalizing hashtables array 6 in /etc/nginx/nginx.conf:124 >> nginx: [emerg] headers array 0 in /etc/nginx/nginx.conf:124 >> >> >> >> 3) I later want to build hashtable based on this array: >> >> dlc->passr_headers_hash = (ngx_hash_t*) ngx_pcalloc(cf->pool, sizeof(ngx_hash_t)); >> hash_init.hash = dlc->passr_headers_hash; >> hash_init.name = "passr_headers_hash"; >> >> >> >> if (ngx_hash_init(&hash_init, (ngx_hash_key_t*) headers_ar->elts, >> headers_ar->nelts) != NGX_OK) { >> ngx_conf_log_error(NGX_LOG_EMERG, cf, 0, "$HEADERS hashtable init failed"); /* LCOV_EXCL_LINE */ >> return (NGX_ERROR); /* LCOV_EXCL_LINE */ >> } >> >> else { >> NX_LOG_DEBUG(_debug_whitelist, NGX_LOG_EMERG, cf, 0, "$HEADERS hashtable init successed %d !", >> dlc->passr_headers_hash->size); >> } >> >> >> >> 4) Can somebody post simple code that >> >> a) creates array >> b) pushes one IP >> c) create hashtable from it >> >> Thanks, >> >> >>> On Sat, Dec 28, 2019 at 3:36 PM Marcin Kozlowski wrote: >>> Ignore my last post. >>> >>> My error. >>> >>> Thanks, >>> >>>> On Fri, Dec 27, 2019 at 11:42 PM Marcin Kozlowski wrote: >>>> Thanks. Almost got my modifications to NAXSI ready. >>>> >>>> But currently have a blocker with getting just the X-Forwarded-for IP >>>> >>>> The code below: >>>> >>>> ngx_uint_t n; >>>> ngx_table_elt_t **h; >>>> ngx_array_t a; >>>> a = req->headers_in.x_forwarded_for; >>>> n = a.nelts; >>>> h = a.elts; >>>> >>>> >>>> for (i = 0; i>>> ngx_log_error(NGX_LOG_ERR, req->connection->log, >>>> 0, "x_forwarded_for: %s", h[i]->value.data); >>>> } >>>> >>>> gets a String with several IP (i.e client, server, request etc) >>>> >>>> Tried to parse the string using strtok(), interating through it .... but it segfaults. I guess I am missing some NGINX module knowledge. >>>> >>>> How to properly get first string up to first "," from the h[i]->value.data using NGINX functions/types or other correct way to do it. >>>> >>>> Thanks, >>>> >>>> >>>> >>>> >>>> >>>>> On Wed, Dec 25, 2019 at 12:17 PM Ruslan Ermilov wrote: >>>>> On Tue, Dec 24, 2019 at 08:00:26PM +0100, Marcin Kozlowski wrote: >>>>> > Thanks. >>>>> > >>>>> > Works. For the reference, this is the code I used: >>>>> > >>>>> > ngx_uint_t n; >>>>> > ngx_table_elt_t **h; >>>>> > ngx_array_t a; >>>>> > a = req->headers_in.x_forwarded_for; >>>>> > n = a.nelts; >>>>> > h = a.elts; >>>>> > >>>>> > >>>>> > for (i = 0; i>>>> > ngx_log_error(NGX_LOG_ERR, req->connection->log, >>>>> > 0, "x_forwarded_for: %s", h[i]->value.data); >>>>> > } >>>>> > >>>>> > BTW What would be the best practice in NGINX NASIX module or any other >>>>> > module to load a file with hundreds entries of IPs (hashmap, or what >>>>> > structure would be best?) which should be whitelisted later for comparison >>>>> > in NASIX module logic. Those IP should never be blocked by NAXSI. >>>>> > >>>>> > When should I load this file in memory, in which component >>>>> > /module/function/step? >>>>> > >>>>> > Links to some guides/sample code would be also appreciated. >>>>> > >>>>> > Thanks, >>>>> >>>>> http://nginx.org/en/docs/http/ngx_http_geo_module.html >>>>> _______________________________________________ >>>>> nginx-devel mailing list >>>>> nginx-devel at nginx.org >>>>> http://mailman.nginx.org/mailman/listinfo/nginx-devel >> _______________________________________________ >> nginx-devel mailing list >> nginx-devel at nginx.org >> http://mailman.nginx.org/mailman/listinfo/nginx-devel > _______________________________________________ > nginx-devel mailing list > nginx-devel at nginx.org > http://mailman.nginx.org/mailman/listinfo/nginx-devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From marcinguy at gmail.com Mon Dec 30 15:47:59 2019 From: marcinguy at gmail.com (Marcin Kozlowski) Date: Mon, 30 Dec 2019 16:47:59 +0100 Subject: nginx - get value of the header - x_forwarded_for in Nginx module (Naxsi) In-Reply-To: References: Message-ID: I don't work with NGINX and modules and C daily. Still struggling with it. Feel free to contact me via the list or to avoid spamming privately. Help will be greatly appreciated. Trying to add whitelisting of IP to NAXSI https://github.com/nbs-system/naxsi How can I add simply one entry being IP to headers_ar? headers_ar = ngx_array_create(cf->pool, 1, sizeof(ngx_hash_key_t)); ngx_array_t *headers_ar_c; headers_ar_c = ngx_array_push(headers_ar); hash_init.key = &ngx_hash_key_lc; hash_init.pool = cf->pool; hash_init.temp_pool = NULL; hash_init.max_size = 1024; hash_init.bucket_size = 512; dlc->passr_headers_hash = (ngx_hash_t*) ngx_pcalloc(cf->pool, sizeof(ngx_hash_t)); hash_init.hash = dlc->passr_headers_hash; hash_init.name = "passr_headers_hash"; if (ngx_hash_init(&hash_init, (ngx_hash_key_t*) headers_ar->elts, headers_ar->nelts) != NGX_OK) { ngx_conf_log_error(NGX_LOG_EMERG, cf, 0, "$HEADERS hashtable init failed"); /* LCOV_EXCL_LINE */ return (NGX_ERROR); /* LCOV_EXCL_LINE */ } else { NX_LOG_DEBUG(_debug_whitelist, NGX_LOG_EMERG, cf, 0, "$HEADERS hashtable init successed %d !", dlc->passr_headers_hash->size); } return (NGX_OK); Later I have a function that looks for it in hash Find in hash function: ngx_http_pass_rule_t * nx_find_pass_in_hash( ngx_http_request_t *req, ngx_str_t *mstr, ngx_http_dummy_loc_conf_t *cf, enum DUMMY_MATCH_ZONE zone) { ngx_http_pass_rule_t *b = NULL; ngx_uint_t key; ngx_str_t scratch = {.data = mstr->data, .len = mstr->len}; ngx_str_t k1 = ngx_string("key1"); key = ngx_hash_key(scratch.data, scratch.len); b = (ngx_http_pass_rule_t*) ngx_hash_find(cf->passr_headers_hash, key, k1.data, k1.len); if (b == NULL) { /* key not found */ NX_DEBUG(_debug_custom_score, NGX_LOG_DEBUG_HTTP, req->connection->log, 0, "find hash 1"); }else{ NX_DEBUG(_debug_custom_score, NGX_LOG_DEBUG_HTTP, req->connection->log, 0, "find hash 2"); } return b; } Thanks, On Sun, Dec 29, 2019 at 3:27 AM Hung Nguyen wrote: > Hello, > > I suggest you to read nginx development guide which is available on > nginx.org. > > About getting x forward for headers, if you take a look at some built in > module there?s already exist implementation which takes the headers and > returns an array. > > -- > H?ng > > On Dec 29, 2019, at 03:49, Robert Paprocki < > rpaprocki at fearnothingproductions.net> wrote: > > ?The array has 0 nelts because you haven?t added anything to do. nelts is > the number of elements in the array, not the size of the array. > > Add an element to the array with ngx_array_push(). > > On Dec 28, 2019, at 11:35, Marcin Kozlowski wrote: > > ? > Still have few questions. Help would be great > > Beginner in writing NGINX modules > > Have this question > > 1) How do I create array, add element to it and than create a hashtable > from it. > > Below I try to achieve it: > > NX_LOG_DEBUG(_debug_whitelist_heavy, > NGX_LOG_EMERG, cf, 0, > "finalizing hashtables array %i", dlc->pass_rules->nelts); > > headers_ar = ngx_array_create(cf->pool, dlc->pass_rules->nelts, > sizeof(ngx_hash_key_t)); > > > > if (headers_ar) { > NX_LOG_DEBUG(_debug_readconf, NGX_LOG_EMERG, cf, 0, > "headers array %i",headers_ar->nelts); > > 2) Why headers_ar has 0 elemets > > nginx: [emerg] finalizing hashtables array 6 in /etc/nginx/nginx.conf:124 > nginx: [emerg] headers array 0 in /etc/nginx/nginx.conf:124 > > > > 3) I later want to build hashtable based on this array: > > dlc->passr_headers_hash = (ngx_hash_t*) ngx_pcalloc(cf->pool, > sizeof(ngx_hash_t)); > hash_init.hash = dlc->passr_headers_hash; > hash_init.name = "passr_headers_hash"; > > > > if (ngx_hash_init(&hash_init, (ngx_hash_key_t*) headers_ar->elts, > headers_ar->nelts) != NGX_OK) { > ngx_conf_log_error(NGX_LOG_EMERG, cf, 0, "$HEADERS hashtable init > failed"); /* LCOV_EXCL_LINE */ > return (NGX_ERROR); /* LCOV_EXCL_LINE */ > } > > else { > NX_LOG_DEBUG(_debug_whitelist, NGX_LOG_EMERG, cf, 0, "$HEADERS > hashtable init successed %d !", > dlc->passr_headers_hash->size); > } > > > > 4) Can somebody post simple code that > > a) creates array > b) pushes one IP > c) create hashtable from it > > Thanks, > > > On Sat, Dec 28, 2019 at 3:36 PM Marcin Kozlowski > wrote: > >> Ignore my last post. >> >> My error. >> >> Thanks, >> >> On Fri, Dec 27, 2019 at 11:42 PM Marcin Kozlowski >> wrote: >> >>> Thanks. Almost got my modifications to NAXSI ready. >>> >>> But currently have a blocker with getting just the X-Forwarded-for IP >>> >>> The code below: >>> >>> ngx_uint_t n; >>> ngx_table_elt_t **h; >>> ngx_array_t a; >>> a = req->headers_in.x_forwarded_for; >>> n = a.nelts; >>> h = a.elts; >>> >>> >>> for (i = 0; i>> ngx_log_error(NGX_LOG_ERR, req->connection->log, >>> 0, "x_forwarded_for: %s", h[i]->value.data); >>> } >>> >>> gets a String with several IP (i.e client, server, request etc) >>> >>> Tried to parse the string using strtok(), interating through it .... but >>> it segfaults. I guess I am missing some NGINX module knowledge. >>> >>> How to properly get first string up to first "," from the >>> h[i]->value.data using NGINX functions/types or other correct way to do it. >>> >>> Thanks, >>> >>> >>> >>> >>> >>> On Wed, Dec 25, 2019 at 12:17 PM Ruslan Ermilov wrote: >>> >>>> On Tue, Dec 24, 2019 at 08:00:26PM +0100, Marcin Kozlowski wrote: >>>> > Thanks. >>>> > >>>> > Works. For the reference, this is the code I used: >>>> > >>>> > ngx_uint_t n; >>>> > ngx_table_elt_t **h; >>>> > ngx_array_t a; >>>> > a = req->headers_in.x_forwarded_for; >>>> > n = a.nelts; >>>> > h = a.elts; >>>> > >>>> > >>>> > for (i = 0; i>>> > ngx_log_error(NGX_LOG_ERR, req->connection->log, >>>> > 0, "x_forwarded_for: %s", h[i]->value.data); >>>> > } >>>> > >>>> > BTW What would be the best practice in NGINX NASIX module or any other >>>> > module to load a file with hundreds entries of IPs (hashmap, or what >>>> > structure would be best?) which should be whitelisted later for >>>> comparison >>>> > in NASIX module logic. Those IP should never be blocked by NAXSI. >>>> > >>>> > When should I load this file in memory, in which component >>>> > /module/function/step? >>>> > >>>> > Links to some guides/sample code would be also appreciated. >>>> > >>>> > Thanks, >>>> >>>> http://nginx.org/en/docs/http/ngx_http_geo_module.html >>>> _______________________________________________ >>>> nginx-devel mailing list >>>> nginx-devel at nginx.org >>>> http://mailman.nginx.org/mailman/listinfo/nginx-devel >>>> >>> _______________________________________________ > nginx-devel mailing list > nginx-devel at nginx.org > http://mailman.nginx.org/mailman/listinfo/nginx-devel > > _______________________________________________ > nginx-devel mailing list > nginx-devel at nginx.org > http://mailman.nginx.org/mailman/listinfo/nginx-devel > > _______________________________________________ > nginx-devel mailing list > nginx-devel at nginx.org > http://mailman.nginx.org/mailman/listinfo/nginx-devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From marcinguy at gmail.com Mon Dec 30 21:44:49 2019 From: marcinguy at gmail.com (Marcin Kozlowski) Date: Mon, 30 Dec 2019 22:44:49 +0100 Subject: nginx - get value of the header - x_forwarded_for in Nginx module (Naxsi) In-Reply-To: References: Message-ID: OK, this helped: https://github.com/yakantosat/nginx-example/blob/master/ngx_hash_test.c Thanks, On Mon, Dec 30, 2019 at 4:47 PM Marcin Kozlowski wrote: > I don't work with NGINX and modules and C daily. > > Still struggling with it. Feel free to contact me via the list or to avoid > spamming privately. Help will be greatly appreciated. > > Trying to add whitelisting of IP to NAXSI > > https://github.com/nbs-system/naxsi > > > > How can I add simply one entry being IP to headers_ar? > > > > headers_ar = ngx_array_create(cf->pool, 1, sizeof(ngx_hash_key_t)); > > ngx_array_t *headers_ar_c; > headers_ar_c = ngx_array_push(headers_ar); > > hash_init.key = &ngx_hash_key_lc; > hash_init.pool = cf->pool; > hash_init.temp_pool = NULL; > hash_init.max_size = 1024; > hash_init.bucket_size = 512; > > > dlc->passr_headers_hash = (ngx_hash_t*) ngx_pcalloc(cf->pool, > sizeof(ngx_hash_t)); > hash_init.hash = dlc->passr_headers_hash; > hash_init.name = "passr_headers_hash"; > > if (ngx_hash_init(&hash_init, (ngx_hash_key_t*) headers_ar->elts, > headers_ar->nelts) != NGX_OK) { > ngx_conf_log_error(NGX_LOG_EMERG, cf, 0, "$HEADERS hashtable init > failed"); /* LCOV_EXCL_LINE */ > return (NGX_ERROR); /* LCOV_EXCL_LINE */ > } > > else { > NX_LOG_DEBUG(_debug_whitelist, NGX_LOG_EMERG, cf, 0, "$HEADERS > hashtable init successed %d !", > dlc->passr_headers_hash->size); > } > > > > return (NGX_OK); > > > > > > Later I have a function that looks for it in hash > > Find in hash function: > > ngx_http_pass_rule_t * > nx_find_pass_in_hash( > ngx_http_request_t *req, > ngx_str_t *mstr, > ngx_http_dummy_loc_conf_t *cf, > enum DUMMY_MATCH_ZONE zone) > { > ngx_http_pass_rule_t *b = NULL; > > ngx_uint_t key; > > ngx_str_t scratch = {.data = mstr->data, .len = mstr->len}; > > ngx_str_t k1 = ngx_string("key1"); > key = ngx_hash_key(scratch.data, scratch.len); > > b = (ngx_http_pass_rule_t*) ngx_hash_find(cf->passr_headers_hash, key, > k1.data, k1.len); > if (b == NULL) { > /* key not found */ > NX_DEBUG(_debug_custom_score, NGX_LOG_DEBUG_HTTP, > req->connection->log, 0, > "find hash 1"); > > }else{ > NX_DEBUG(_debug_custom_score, NGX_LOG_DEBUG_HTTP, > req->connection->log, 0, > "find hash 2"); > > > } > > return b; > > } > > Thanks, > > > > > On Sun, Dec 29, 2019 at 3:27 AM Hung Nguyen > wrote: > >> Hello, >> >> I suggest you to read nginx development guide which is available on >> nginx.org. >> >> About getting x forward for headers, if you take a look at some built in >> module there?s already exist implementation which takes the headers and >> returns an array. >> >> -- >> H?ng >> >> On Dec 29, 2019, at 03:49, Robert Paprocki < >> rpaprocki at fearnothingproductions.net> wrote: >> >> ?The array has 0 nelts because you haven?t added anything to do. nelts is >> the number of elements in the array, not the size of the array. >> >> Add an element to the array with ngx_array_push(). >> >> On Dec 28, 2019, at 11:35, Marcin Kozlowski wrote: >> >> ? >> Still have few questions. Help would be great >> >> Beginner in writing NGINX modules >> >> Have this question >> >> 1) How do I create array, add element to it and than create a hashtable >> from it. >> >> Below I try to achieve it: >> >> NX_LOG_DEBUG(_debug_whitelist_heavy, >> NGX_LOG_EMERG, cf, 0, >> "finalizing hashtables array %i", dlc->pass_rules->nelts); >> >> headers_ar = ngx_array_create(cf->pool, dlc->pass_rules->nelts, >> sizeof(ngx_hash_key_t)); >> >> >> >> if (headers_ar) { >> NX_LOG_DEBUG(_debug_readconf, NGX_LOG_EMERG, cf, 0, >> "headers array %i",headers_ar->nelts); >> >> 2) Why headers_ar has 0 elemets >> >> nginx: [emerg] finalizing hashtables array 6 in /etc/nginx/nginx.conf:124 >> nginx: [emerg] headers array 0 in /etc/nginx/nginx.conf:124 >> >> >> >> 3) I later want to build hashtable based on this array: >> >> dlc->passr_headers_hash = (ngx_hash_t*) ngx_pcalloc(cf->pool, >> sizeof(ngx_hash_t)); >> hash_init.hash = dlc->passr_headers_hash; >> hash_init.name = "passr_headers_hash"; >> >> >> >> if (ngx_hash_init(&hash_init, (ngx_hash_key_t*) headers_ar->elts, >> headers_ar->nelts) != NGX_OK) { >> ngx_conf_log_error(NGX_LOG_EMERG, cf, 0, "$HEADERS hashtable init >> failed"); /* LCOV_EXCL_LINE */ >> return (NGX_ERROR); /* LCOV_EXCL_LINE */ >> } >> >> else { >> NX_LOG_DEBUG(_debug_whitelist, NGX_LOG_EMERG, cf, 0, "$HEADERS >> hashtable init successed %d !", >> dlc->passr_headers_hash->size); >> } >> >> >> >> 4) Can somebody post simple code that >> >> a) creates array >> b) pushes one IP >> c) create hashtable from it >> >> Thanks, >> >> >> On Sat, Dec 28, 2019 at 3:36 PM Marcin Kozlowski >> wrote: >> >>> Ignore my last post. >>> >>> My error. >>> >>> Thanks, >>> >>> On Fri, Dec 27, 2019 at 11:42 PM Marcin Kozlowski >>> wrote: >>> >>>> Thanks. Almost got my modifications to NAXSI ready. >>>> >>>> But currently have a blocker with getting just the X-Forwarded-for IP >>>> >>>> The code below: >>>> >>>> ngx_uint_t n; >>>> ngx_table_elt_t **h; >>>> ngx_array_t a; >>>> a = req->headers_in.x_forwarded_for; >>>> n = a.nelts; >>>> h = a.elts; >>>> >>>> >>>> for (i = 0; i>>> ngx_log_error(NGX_LOG_ERR, req->connection->log, >>>> 0, "x_forwarded_for: %s", h[i]->value.data); >>>> } >>>> >>>> gets a String with several IP (i.e client, server, request etc) >>>> >>>> Tried to parse the string using strtok(), interating through it .... >>>> but it segfaults. I guess I am missing some NGINX module knowledge. >>>> >>>> How to properly get first string up to first "," from the >>>> h[i]->value.data using NGINX functions/types or other correct way to do it. >>>> >>>> Thanks, >>>> >>>> >>>> >>>> >>>> >>>> On Wed, Dec 25, 2019 at 12:17 PM Ruslan Ermilov wrote: >>>> >>>>> On Tue, Dec 24, 2019 at 08:00:26PM +0100, Marcin Kozlowski wrote: >>>>> > Thanks. >>>>> > >>>>> > Works. For the reference, this is the code I used: >>>>> > >>>>> > ngx_uint_t n; >>>>> > ngx_table_elt_t **h; >>>>> > ngx_array_t a; >>>>> > a = req->headers_in.x_forwarded_for; >>>>> > n = a.nelts; >>>>> > h = a.elts; >>>>> > >>>>> > >>>>> > for (i = 0; i>>>> > ngx_log_error(NGX_LOG_ERR, req->connection->log, >>>>> > 0, "x_forwarded_for: %s", h[i]->value.data); >>>>> > } >>>>> > >>>>> > BTW What would be the best practice in NGINX NASIX module or any >>>>> other >>>>> > module to load a file with hundreds entries of IPs (hashmap, or what >>>>> > structure would be best?) which should be whitelisted later for >>>>> comparison >>>>> > in NASIX module logic. Those IP should never be blocked by NAXSI. >>>>> > >>>>> > When should I load this file in memory, in which component >>>>> > /module/function/step? >>>>> > >>>>> > Links to some guides/sample code would be also appreciated. >>>>> > >>>>> > Thanks, >>>>> >>>>> http://nginx.org/en/docs/http/ngx_http_geo_module.html >>>>> _______________________________________________ >>>>> nginx-devel mailing list >>>>> nginx-devel at nginx.org >>>>> http://mailman.nginx.org/mailman/listinfo/nginx-devel >>>>> >>>> _______________________________________________ >> nginx-devel mailing list >> nginx-devel at nginx.org >> http://mailman.nginx.org/mailman/listinfo/nginx-devel >> >> _______________________________________________ >> nginx-devel mailing list >> nginx-devel at nginx.org >> http://mailman.nginx.org/mailman/listinfo/nginx-devel >> >> _______________________________________________ >> nginx-devel mailing list >> nginx-devel at nginx.org >> http://mailman.nginx.org/mailman/listinfo/nginx-devel > > -------------- next part -------------- An HTML attachment was scrubbed... URL: