From rian at alum.mit.edu Sun Sep 1 18:45:20 2019 From: rian at alum.mit.edu (Rian Hunter) Date: Sun, 01 Sep 2019 11:45:20 -0700 Subject: Potential bug in ngx_event.c In-Reply-To: <20190831182822.GM1877@mdounin.ru> References: <797fde8d89aa12a26a84dfd52428a8f5@thelig.ht> <20190830141929.GL1877@mdounin.ru> <6fc40c129d36c08bb56fde854f4ffc91@thelig.ht> <20190831182822.GM1877@mdounin.ru> Message-ID: <07223a007c7d5051261e1251bcb2e64e@thelig.ht> On 2019-08-31 11:28, Maxim Dounin wrote: > Hello! > > On Fri, Aug 30, 2019 at 06:26:57PM -0700, Rian Hunter wrote: > >> On 2019-08-30 07:19, Maxim Dounin wrote: >> > Yes, this is intentional. >> > >> > The first agument of the ngx_shmtx_create() function is a pointer >> > to the ngx_shmtx_t structure, which is not expected to be shared >> > between processes. Copy of the structure as created by fork() is >> > enough. >> >> The POSIX sem_t member (called "sem") needs to reside in shared memory >> if sem_wait()/sem_post() are to have an effect across processes. >> Memory >> copied across fork is not sufficient. > > No, fork() is explicitly documented to preserve semaphores > (http://pubs.opengroup.org/onlinepubs/9699919799/functions/fork.html): > > : Any semaphores that are open in the parent process shall also be > : open in the child process. That's only true for semaphores created with sem_open(). For semaphores created with sem_init(pshared=1) the caller is responsible for using supplying memory that resides in a shared memory segment (e.g. created from mm ap(flags=MAP_SHARED|...)). See: https://blog.superpat.com/2010/07/14/semaphores-on-linux-sem_init-vs-sem_open/ Here is a program that illustrates my point (also here: https://gist.github.com/rianhunter/a0bd4c9e8ab550ecadbe2464995726a8): #include #include #include #include #include #include #include #include #include #include int test_semaphore(sem_t *sem) { pid_t pid; pid = fork(); if (pid < 0) { fprintf(stderr, "failed to fork: %s\n", strerror(errno)); return -1; } if (pid > 0) { int status; /* after posting, child should exit successfully */ sem_post(sem); printf("Waiting for child to react to sem_post()...\n"); waitpid(pid, &status, 0); if (!WIFEXITED(status) || WEXITSTATUS(status) > 1) { return -1; } return !WEXITSTATUS(status); } else { int ret; struct timespec to; ret = clock_gettime(CLOCK_REALTIME, &to); if (ret < 0) { fprintf(stderr, "failed to clock_gettime(): %s\n", strerror(errno)); return 255; } /* wait for sem_post() for 5 seconds */ to.tv_sec += 5; ret = sem_timedwait(sem, &to); if (ret < 0 && errno != ETIMEDOUT) { fprintf(stderr, "failed to timedwait: %s\n", strerror(errno)); return 255; } exit(!ret ? EXIT_SUCCESS : EXIT_FAILURE); } /* notreached */ } int main() { int ret; sem_t stack_allocated, *mmap_shared; sem_init(&stack_allocated, 1, 0); ret = test_semaphore(&stack_allocated); if (ret < 0) { return EXIT_FAILURE; } if (!ret) { printf("Stack-allocated semaphore across fork() is broken\n"); } else { printf("Stack-allocated semaphore across fork() works\n"); } printf("\n"); mmap_shared = mmap(NULL, getpagesize(), PROT_READ | PROT_WRITE, MAP_SHARED | MAP_ANONYMOUS, -1, 0); if (mmap_shared == MAP_FAILED) { return EXIT_FAILURE; } sem_init(mmap_shared, 1, 0); ret = test_semaphore(mmap_shared); if (ret < 0) { return EXIT_FAILURE; } if (!ret) { printf("Mapped semaphore across fork() is broken\n"); } else { printf("Mapped semaphore across fork() works\n"); } return EXIT_SUCCESS; } From guillaume-nginx at outters.eu Mon Sep 2 03:09:45 2019 From: guillaume-nginx at outters.eu (Guillaume Outters) Date: Mon, 02 Sep 2019 05:09:45 +0200 Subject: [PATCH] ngx_conf_file: "include x.conf nearby; " acts relative to the currently parsed file Message-ID: <3165025ac6d67c0895fe.1567393785@bas> Hello Maxim (and anyone interested in fact), here is my re-attempt at easing cascade includes, that is, an included file can include snippets (that it knows are located next to him) without having to know the path through which itself was included. I have abandoned the (definitely obscure and error prone) "./" prefix to use a conventional flag (I have chosen "nearby" which seems more natural than either "local" or "relative"). Use case: in nginx.conf: include /var/www/*/conf/nginx/main.conf; in /var/www/outters.eu/conf/nginx/main.conf: include nearby snippets/phpfpm.conf; instead of: - either including /var/www/outters.eu/conf/nginx/snippets/phpfpm.conf (requires templating when the webapp has no fixed delivery path) - or inlining the contents of phpfpm.conf (which would be a pity if it has to be included multiple times). As "nearby" plugs into ngx_conf_file.c, it works gracefully with wildcards includes too. Hoping this patch will this time be both helpful and clean, -- Guillaume # HG changeset patch # User Guillaume Outters # Date 1567393678 -7200 # Mon Sep 02 05:07:58 2019 +0200 # Node ID 3165025ac6d67c0895fe2b7f110ba7d00c73d485 # Parent 9f1f9d6e056a4f85907957ef263f78a426ae4f9c ngx_conf_file: "include x.conf nearby;" acts relative to the currently parsed file "include x.conf nearby;" found in d/0.conf will include d/x.conf. Eases modular configurations, where includees do not need to know their absolute path to reach helper config files next to them. diff -r 9f1f9d6e056a -r 3165025ac6d6 src/core/ngx_conf_file.c --- a/src/core/ngx_conf_file.c Mon Aug 19 15:16:06 2019 +0300 +++ b/src/core/ngx_conf_file.c Mon Sep 02 05:07:58 2019 +0200 @@ -10,16 +10,19 @@ #define NGX_CONF_BUFFER 4096 +#define NGX_CONF_FLAG_RELATIVE "nearby" + static ngx_int_t ngx_conf_add_dump(ngx_conf_t *cf, ngx_str_t *filename); static ngx_int_t ngx_conf_handler(ngx_conf_t *cf, ngx_int_t last); static ngx_int_t ngx_conf_read_token(ngx_conf_t *cf); static void ngx_conf_flush_files(ngx_cycle_t *cycle); +ngx_int_t ngx_conf_full_name_rel(ngx_conf_t *cf, ngx_cycle_t *cycle, ngx_str_t *name, ngx_uint_t conf_prefix, ngx_uint_t relative); static ngx_command_t ngx_conf_commands[] = { { ngx_string("include"), - NGX_ANY_CONF|NGX_CONF_TAKE1, + NGX_ANY_CONF|NGX_CONF_TAKE12, ngx_conf_include, 0, 0, @@ -824,13 +827,32 @@ ngx_int_t n; ngx_str_t *value, file, name; ngx_glob_t gl; + ngx_uint_t relative; value = cf->args->elts; file = value[1]; + relative = 0; + if (cf->args->nelts == 3) { + if (ngx_strcmp(value[2].data, NGX_CONF_FLAG_RELATIVE) == 0) { + relative = 1; + + } else if (ngx_strcmp(value[1].data, NGX_CONF_FLAG_RELATIVE) == 0) { + relative = 1; + file = value[2]; + + } else { + ngx_conf_log_error(NGX_LOG_EMERG, cf, 0, + "invalid value \"%s\" in \"%s\" directive, " + "expected \"%s\"", value[2].data, + cmd->name.data, NGX_CONF_FLAG_RELATIVE); + return NGX_CONF_ERROR; + } + } + ngx_log_debug1(NGX_LOG_DEBUG_CORE, cf->log, 0, "include %s", file.data); - if (ngx_conf_full_name(cf->cycle, &file, 1) != NGX_OK) { + if (ngx_conf_full_name_rel(cf, cf->cycle, &file, 1, relative) != NGX_OK) { return NGX_CONF_ERROR; } @@ -884,16 +906,44 @@ ngx_int_t -ngx_conf_full_name(ngx_cycle_t *cycle, ngx_str_t *name, ngx_uint_t conf_prefix) +ngx_conf_full_name_rel(ngx_conf_t *cf, ngx_cycle_t *cycle, ngx_str_t *name, ngx_uint_t conf_prefix, ngx_uint_t relative) { ngx_str_t *prefix; + ngx_str_t local_prefix; - prefix = conf_prefix ? &cycle->conf_prefix : &cycle->prefix; + if (relative) { + if (name->len >= 1 && name->data[0] == '/') { + ngx_conf_log_error(NGX_LOG_EMERG, cf, 0, + "relative path \"%s\" " + "should not start with \"/\"", + name); + return NGX_ERROR; + } + + local_prefix.data = cf->conf_file->file.name.data; + for (local_prefix.len = cf->conf_file->file.name.len; + --local_prefix.len >= 0 + && local_prefix.data[local_prefix.len] != '/'; + /* void */ ) + { /* void */ } + ++local_prefix.len; + prefix = &local_prefix; + + } else { + prefix = conf_prefix ? &cycle->conf_prefix : &cycle->prefix; + } return ngx_get_full_name(cycle->pool, prefix, name); } +ngx_int_t +ngx_conf_full_name(ngx_cycle_t *cycle, ngx_str_t *name, ngx_uint_t conf_prefix) +{ + return ngx_conf_full_name_rel(NULL, cycle, name, conf_prefix, 0); +} + + ngx_open_file_t * ngx_conf_open_file(ngx_cycle_t *cycle, ngx_str_t *name) { From xeioex at nginx.com Mon Sep 2 15:25:37 2019 From: xeioex at nginx.com (Dmitry Volyntsev) Date: Mon, 02 Sep 2019 15:25:37 +0000 Subject: [njs] Added new Function() support. Message-ID: details: https://hg.nginx.org/njs/rev/98d3dd617ed7 branches: changeset: 1155:98d3dd617ed7 user: hongzhidao date: Fri Aug 23 13:25:50 2019 -0400 description: Added new Function() support. diffstat: src/njs.h | 18 +++++- src/njs_disassembler.c | 7 +- src/njs_function.c | 114 +++++++++++++++++++++++++++++++++++++++- src/njs_shell.c | 36 ++++++----- src/njs_vm.c | 4 + src/test/njs_interactive_test.c | 3 + src/test/njs_unit_test.c | 84 +++++++++++++++++++++++++++-- test/njs_expect_test.exp | 8 ++ 8 files changed, 244 insertions(+), 30 deletions(-) diffs (530 lines): diff -r ac633d007ac5 -r 98d3dd617ed7 src/njs.h --- a/src/njs.h Thu Aug 29 19:18:53 2019 +0300 +++ b/src/njs.h Fri Aug 23 13:25:50 2019 -0400 @@ -146,13 +146,28 @@ typedef struct { char **argv; njs_uint_t argc; +/* + * accumulative - enables "accumulative" mode to support incremental compiling. + * (REPL). Allows starting parent VM without cloning. + * disassemble - enables disassemble. + * backtrace - enables backtraces. + * quiet - removes filenames from backtraces. To produce comparable + test262 diffs. + * sandbox - "sandbox" mode. Disables file access. + * unsafe - enables unsafe language features: + * - Function constructors. + * module - ES6 "module" mode. Script mode is default. + */ + uint8_t trailer; /* 1 bit */ uint8_t init; /* 1 bit */ uint8_t accumulative; /* 1 bit */ + uint8_t disassemble; /* 1 bit */ uint8_t backtrace; /* 1 bit */ + uint8_t quiet; /* 1 bit */ uint8_t sandbox; /* 1 bit */ + uint8_t unsafe; /* 1 bit */ uint8_t module; /* 1 bit */ - uint8_t quiet; /* 1 bit */ } njs_vm_opt_t; @@ -225,6 +240,7 @@ NJS_EXPORT njs_external_ptr_t njs_vm_ext const njs_value_t *value); NJS_EXPORT void njs_disassembler(njs_vm_t *vm); +NJS_EXPORT void njs_disassemble(u_char *start, u_char *end); NJS_EXPORT const njs_value_t *njs_vm_value(njs_vm_t *vm, const njs_str_t *name); NJS_EXPORT njs_function_t *njs_vm_function(njs_vm_t *vm, const njs_str_t *name); diff -r ac633d007ac5 -r 98d3dd617ed7 src/njs_disassembler.c --- a/src/njs_disassembler.c Thu Aug 29 19:18:53 2019 +0300 +++ b/src/njs_disassembler.c Fri Aug 23 13:25:50 2019 -0400 @@ -8,9 +8,6 @@ #include -static void njs_disassemble(u_char *start, u_char *end); - - typedef struct { njs_vmcode_operation_t operation; size_t size; @@ -152,10 +149,12 @@ njs_disassembler(njs_vm_t *vm) code++; n--; } + + njs_printf("\n"); } -static void +void njs_disassemble(u_char *start, u_char *end) { u_char *p; diff -r ac633d007ac5 -r 98d3dd617ed7 src/njs_function.c --- a/src/njs_function.c Thu Aug 29 19:18:53 2019 +0300 +++ b/src/njs_function.c Fri Aug 23 13:25:50 2019 -0400 @@ -874,9 +874,119 @@ njs_int_t njs_function_constructor(njs_vm_t *vm, njs_value_t *args, njs_uint_t nargs, njs_index_t unused) { - njs_internal_error(vm, "Not implemented"); + size_t size; + u_char *start, *end; + njs_int_t ret; + njs_str_t str, file; + njs_uint_t i; + njs_lexer_t lexer; + njs_parser_t *parser; + njs_generator_t generator; + njs_parser_scope_t *scope; + + if (!vm->options.unsafe) { + njs_type_error(vm, "function constructor is disabled in \"safe\" mode"); + return NJS_ERROR; + } + + if (nargs < 2) { + start = (u_char *) "(function(){})"; + end = start + njs_length("(function(){})"); + + } else { + size = njs_length("(function(") + nargs + njs_length("){})"); + + for (i = 1; i < nargs; i++) { + if (!njs_is_string(&args[i])) { + ret = njs_value_to_string(vm, &args[i], &args[i]); + if (ret != NJS_OK) { + return ret; + } + } + + njs_string_get(&args[i], &str); + size += str.length; + } + + start = njs_mp_alloc(vm->mem_pool, size); + if (njs_slow_path(start == NULL)) { + return NJS_ERROR; + } + + end = njs_cpymem(start, "(function(", njs_length("(function(")); + + for (i = 1; i < nargs - 1; i++) { + if (i != 1) { + *end++ = ','; + } + + njs_string_get(&args[i], &str); + end = njs_cpymem(end, str.start, str.length); + } + + *end++ = ')'; + *end++ = '{'; + + njs_string_get(&args[nargs - 1], &str); + end = njs_cpymem(end, str.start, str.length); - return NJS_ERROR; + *end++ = '}'; + *end++ = ')'; + } + + vm->options.accumulative = 1; + + parser = njs_mp_zalloc(vm->mem_pool, sizeof(njs_parser_t)); + if (njs_slow_path(parser == NULL)) { + return NJS_ERROR; + } + + vm->parser = parser; + + file = njs_str_value("runtime"); + + ret = njs_lexer_init(vm, &lexer, &file, start, end); + if (njs_slow_path(ret != NJS_OK)) { + return ret; + } + + parser->lexer = &lexer; + + ret = njs_parser(vm, parser, NULL); + if (njs_slow_path(ret != NJS_OK)) { + return ret; + } + + scope = parser->scope; + + ret = njs_variables_copy(vm, &scope->variables, &vm->variables_hash); + if (njs_slow_path(ret != NJS_OK)) { + return ret; + } + + ret = njs_variables_scope_reference(vm, scope); + if (njs_slow_path(ret != NJS_OK)) { + return ret; + } + + njs_memzero(&generator, sizeof(njs_generator_t)); + + ret = njs_generate_scope(vm, &generator, scope, &njs_entry_anonymous); + if (njs_slow_path(ret != NJS_OK)) { + return ret; + } + + if (vm->options.disassemble) { + njs_printf("new Function:runtime\n"); + njs_disassemble(generator.code_start, generator.code_end); + } + + ret = njs_vmcode_interpreter(vm, generator.code_start); + if (njs_slow_path(ret != NJS_OK)) { + return ret; + } + + return NJS_OK; } diff -r ac633d007ac5 -r 98d3dd617ed7 src/njs_shell.c --- a/src/njs_shell.c Thu Aug 29 19:18:53 2019 +0300 +++ b/src/njs_shell.c Fri Aug 23 13:25:50 2019 -0400 @@ -25,6 +25,7 @@ typedef struct { uint8_t module; uint8_t quiet; uint8_t sandbox; + uint8_t safe; uint8_t version; char *file; @@ -74,7 +75,7 @@ static njs_int_t njs_interactive_shell(n njs_vm_opt_t *vm_options); static njs_vm_t *njs_create_vm(njs_opts_t *opts, njs_vm_opt_t *vm_options); static njs_int_t njs_process_file(njs_opts_t *opts, njs_vm_opt_t *vm_options); -static njs_int_t njs_process_script(njs_console_t *console, njs_opts_t *opts, +static njs_int_t njs_process_script(njs_console_t *console, const njs_str_t *script); static njs_int_t njs_editline_init(void); static char **njs_completion_handler(const char *text, int start, int end); @@ -247,9 +248,11 @@ main(int argc, char **argv) vm_options.init = !opts.interactive; vm_options.accumulative = opts.interactive; + vm_options.disassemble = opts.disassemble; vm_options.backtrace = 1; vm_options.quiet = opts.quiet; vm_options.sandbox = opts.sandbox; + vm_options.unsafe = !opts.safe; vm_options.module = opts.module; vm_options.ops = &njs_console_ops; @@ -265,7 +268,7 @@ main(int argc, char **argv) if (vm != NULL) { command.start = (u_char *) opts.command; command.length = njs_strlen(opts.command); - ret = njs_process_script(vm_options.external, &opts, &command); + ret = njs_process_script(vm_options.external, &command); } } else { @@ -299,6 +302,7 @@ njs_get_options(njs_opts_t *opts, int ar " -s sandbox mode.\n" " -t script|module source code type (script is default).\n" " -v print njs version and exit.\n" + " -u disable \"unsafe\" mode.\n" " | - run code from a file or stdin.\n"; ret = NJS_DONE; @@ -382,6 +386,10 @@ njs_get_options(njs_opts_t *opts, int ar opts->version = 1; break; + case 'u': + opts->safe = 1; + break; + default: njs_stderror("Unknown argument: \"%s\" " "try \"%s -h\" for available options\n", argv[i], @@ -488,7 +496,7 @@ njs_interactive_shell(njs_opts_t *opts, add_history((char *) line.start); - njs_process_script(vm_options->external, opts, &line); + njs_process_script(vm_options->external, &line); /* editline allocs a new buffer every time. */ free(line.start); @@ -607,7 +615,7 @@ njs_process_file(njs_opts_t *opts, njs_v } } - ret = njs_process_script(vm_options->external, opts, &script); + ret = njs_process_script(vm_options->external, &script); if (ret != NJS_OK) { ret = NJS_ERROR; goto done; @@ -691,7 +699,7 @@ njs_create_vm(njs_opts_t *opts, njs_vm_o static void -njs_output(njs_vm_t *vm, njs_opts_t *opts, njs_int_t ret) +njs_output(njs_vm_t *vm, njs_int_t ret) { njs_str_t out; @@ -703,7 +711,7 @@ njs_output(njs_vm_t *vm, njs_opts_t *opt if (ret != NJS_OK) { njs_stderror("%V\n", &out); - } else if (opts->interactive) { + } else if (vm->options.accumulative) { njs_print(out.start, out.length); njs_printf("\n"); } @@ -711,7 +719,7 @@ njs_output(njs_vm_t *vm, njs_opts_t *opt static njs_int_t -njs_process_events(njs_console_t *console, njs_opts_t *opts) +njs_process_events(njs_console_t *console) { njs_ev_t *ev; njs_queue_t *events; @@ -740,8 +748,7 @@ njs_process_events(njs_console_t *consol static njs_int_t -njs_process_script(njs_console_t *console, njs_opts_t *opts, - const njs_str_t *script) +njs_process_script(njs_console_t *console, const njs_str_t *script) { u_char *start; njs_vm_t *vm; @@ -753,22 +760,17 @@ njs_process_script(njs_console_t *consol ret = njs_vm_compile(vm, &start, start + script->length); if (ret == NJS_OK) { - if (opts->disassemble) { - njs_disassembler(vm); - njs_printf("\n"); - } - ret = njs_vm_start(vm); } - njs_output(vm, opts, ret); + njs_output(vm, ret); for ( ;; ) { if (!njs_vm_pending(vm)) { break; } - ret = njs_process_events(console, opts); + ret = njs_process_events(console); if (njs_slow_path(ret != NJS_OK)) { njs_stderror("njs_process_events() failed\n"); ret = NJS_ERROR; @@ -786,7 +788,7 @@ njs_process_script(njs_console_t *consol ret = njs_vm_run(vm); if (ret == NJS_ERROR) { - njs_output(vm, opts, ret); + njs_output(vm, ret); } } diff -r ac633d007ac5 -r 98d3dd617ed7 src/njs_vm.c --- a/src/njs_vm.c Thu Aug 29 19:18:53 2019 +0300 +++ b/src/njs_vm.c Fri Aug 23 13:25:50 2019 -0400 @@ -224,6 +224,10 @@ njs_vm_compile(njs_vm_t *vm, u_char **st } } + if (vm->options.disassemble) { + njs_disassembler(vm); + } + return NJS_OK; fail: diff -r ac633d007ac5 -r 98d3dd617ed7 src/test/njs_interactive_test.c --- a/src/test/njs_interactive_test.c Thu Aug 29 19:18:53 2019 +0300 +++ b/src/test/njs_interactive_test.c Fri Aug 23 13:25:50 2019 -0400 @@ -173,6 +173,9 @@ static njs_interactive_test_t njs_test[ " at eval (native)\n" " at main (native)\n") }, + { njs_str("new Function(\n\n@)" ENTER), + njs_str("SyntaxError: Unexpected token \"@\" in 3") }, + { njs_str("require()" ENTER), njs_str("TypeError: missing path\n" " at require (native)\n" diff -r ac633d007ac5 -r 98d3dd617ed7 src/test/njs_unit_test.c --- a/src/test/njs_unit_test.c Thu Aug 29 19:18:53 2019 +0300 +++ b/src/test/njs_unit_test.c Fri Aug 23 13:25:50 2019 -0400 @@ -8,6 +8,9 @@ #include +#define NJS_HAVE_LARGE_STACK (!NJS_HAVE_ADDRESS_SANITIZER && !NJS_HAVE_MEMORY_SANITIZER) + + typedef struct { njs_str_t script; njs_str_t ret; @@ -4225,7 +4228,7 @@ static njs_unit_test_t njs_test[] = "Array.prototype.fill.call(o, 2).a"), njs_str("4") }, -#if (!NJS_HAVE_ADDRESS_SANITIZER && !NJS_HAVE_MEMORY_SANITIZER) /* limited stack size */ +#if NJS_HAVE_LARGE_STACK { njs_str("var o = Object({length: 3});" "Object.defineProperty(o, '0', {set: function(v){this[0] = 2 * v}});" "Array.prototype.fill.call(o, 2)"), @@ -6558,7 +6561,7 @@ static njs_unit_test_t njs_test[] = { njs_str("{ function f() {} { var f }}"), njs_str("SyntaxError: \"f\" has already been declared in 1") }, -#if (!NJS_HAVE_ADDRESS_SANITIZER && !NJS_HAVE_MEMORY_SANITIZER) /* limited stack size */ +#if NJS_HAVE_LARGE_STACK { njs_str("function f() { return f() } f()"), njs_str("RangeError: Maximum call stack size exceeded") }, #endif @@ -8837,9 +8840,6 @@ static njs_unit_test_t njs_test[] = { njs_str("var ex; try {({}) instanceof this} catch (e) {ex = e}; ex"), njs_str("TypeError: right argument is not callable") }, - { njs_str("Function.call(this, 'var x / = 1;')"), - njs_str("InternalError: Not implemented") }, - { njs_str("njs"), njs_str("[object Object]") }, @@ -9590,7 +9590,76 @@ static njs_unit_test_t njs_test[] = njs_str("true") }, { njs_str("Function()"), - njs_str("InternalError: Not implemented") }, + njs_str("[object Function]") }, + + { njs_str("new Function();"), + njs_str("[object Function]") }, + + { njs_str("(function(){}).constructor === Function"), + njs_str("true") }, + +#if NJS_HAVE_LARGE_STACK + { njs_str("new Function(\"(\".repeat(2**13));"), + njs_str("RangeError: Maximum call stack size exceeded") }, + + { njs_str("new Function(\"{\".repeat(2**13));"), + njs_str("RangeError: Maximum call stack size exceeded") }, + + { njs_str("new Function(\"[\".repeat(2**13));"), + njs_str("RangeError: Maximum call stack size exceeded") }, + + { njs_str("new Function(\"`\".repeat(2**13));"), + njs_str("RangeError: Maximum call stack size exceeded") }, + + { njs_str("new Function(\"{[\".repeat(2**13));"), + njs_str("RangeError: Maximum call stack size exceeded") }, + + { njs_str("new Function(\"{;\".repeat(2**13));"), + njs_str("RangeError: Maximum call stack size exceeded") }, + + { njs_str("new Function(\"1;\".repeat(2**13));"), + njs_str("RangeError: Maximum call stack size exceeded") }, + + { njs_str("new Function(\"~\".repeat(2**13));"), + njs_str("RangeError: Maximum call stack size exceeded") }, + + { njs_str("new Function(\"new \".repeat(2**13));"), + njs_str("RangeError: Maximum call stack size exceeded") }, + + { njs_str("new Function(\"typeof \".repeat(2**13));"), + njs_str("RangeError: Maximum call stack size exceeded") }, + + { njs_str("new Function(\"1\" + \"** 1\".repeat(2**13));"), + njs_str("RangeError: Maximum call stack size exceeded") }, + + { njs_str("new Function(\"var a; a\" + \"= a\".repeat(2**13));"), + njs_str("RangeError: Maximum call stack size exceeded") }, +#endif + + { njs_str("var f = new Function('return 1;'); f();"), + njs_str("1") }, + + { njs_str("var sum = new Function('a', 'b', 'return a + b');" + "sum(2, 4);"), + njs_str("6") }, + + { njs_str("var sum = new Function('a, b', 'return a + b');" + "sum(2, 4);"), + njs_str("6") }, + + { njs_str("var sum = new Function('a, b', 'c', 'return a + b + c');" + "sum(2, 4, 4);"), + njs_str("10") }, + + { njs_str("(new Function({ toString() { return '...a'; }}, { toString() { return 'return a;' }}))(1,2,3)"), + njs_str("1,2,3") }, + + { njs_str("var x = 10; function foo() { var x = 20; return new Function('return x;'); }" + "var f = foo(); f()"), + njs_str("10") }, + + { njs_str("var fn = (function() { return new Function('return this'); }).call({}), o = {}; fn.call(o) == o && fn.bind(o).call(this) == o"), + njs_str("true") }, { njs_str("RegExp()"), njs_str("/(?:)/") }, @@ -14535,6 +14604,7 @@ njs_externals_init(njs_vm_t *vm) typedef struct { njs_bool_t disassemble; njs_bool_t verbose; + njs_bool_t unsafe; njs_bool_t module; njs_uint_t repeat; } njs_opts_t; @@ -14588,6 +14658,7 @@ njs_unit_test(njs_unit_test_t tests[], s njs_memzero(&options, sizeof(njs_vm_opt_t)); options.module = opts->module; + options.unsafe = opts->unsafe; vm = njs_vm_create(&options); if (vm == NULL) { @@ -15112,6 +15183,7 @@ main(int argc, char **argv) njs_memzero(&stat, sizeof(njs_stat_t)); opts.repeat = 1; + opts.unsafe = 1; ret = njs_unit_test(njs_test, njs_nitems(njs_test), "script tests", &opts, &stat); diff -r ac633d007ac5 -r 98d3dd617ed7 test/njs_expect_test.exp --- a/test/njs_expect_test.exp Thu Aug 29 19:18:53 2019 +0300 +++ b/test/njs_expect_test.exp Fri Aug 23 13:25:50 2019 -0400 @@ -799,6 +799,14 @@ njs_test { } "-s" +# safe mode + +njs_test { + {"new Function()\r\n" + "TypeError: function constructor is disabled in \"safe\" mode\r\n"} +} "-u" + + # source type njs_test { From mdounin at mdounin.ru Mon Sep 2 16:24:01 2019 From: mdounin at mdounin.ru (Maxim Dounin) Date: Mon, 2 Sep 2019 19:24:01 +0300 Subject: Potential bug in ngx_event.c In-Reply-To: <07223a007c7d5051261e1251bcb2e64e@thelig.ht> References: <797fde8d89aa12a26a84dfd52428a8f5@thelig.ht> <20190830141929.GL1877@mdounin.ru> <6fc40c129d36c08bb56fde854f4ffc91@thelig.ht> <20190831182822.GM1877@mdounin.ru> <07223a007c7d5051261e1251bcb2e64e@thelig.ht> Message-ID: <20190902162401.GW1877@mdounin.ru> Hello! On Sun, Sep 01, 2019 at 11:45:20AM -0700, Rian Hunter wrote: > On 2019-08-31 11:28, Maxim Dounin wrote: > > Hello! > > > > On Fri, Aug 30, 2019 at 06:26:57PM -0700, Rian Hunter wrote: > > > >> On 2019-08-30 07:19, Maxim Dounin wrote: > >> > Yes, this is intentional. > >> > > >> > The first agument of the ngx_shmtx_create() function is a pointer > >> > to the ngx_shmtx_t structure, which is not expected to be shared > >> > between processes. Copy of the structure as created by fork() is > >> > enough. > >> > >> The POSIX sem_t member (called "sem") needs to reside in shared memory > >> if sem_wait()/sem_post() are to have an effect across processes. > >> Memory > >> copied across fork is not sufficient. > > > > No, fork() is explicitly documented to preserve semaphores > > (http://pubs.opengroup.org/onlinepubs/9699919799/functions/fork.html): > > > > : Any semaphores that are open in the parent process shall also be > > : open in the child process. > > That's only true for semaphores created with sem_open(). For semaphores > created with sem_init(pshared=1) the caller is responsible for using > supplying memory that resides in a shared memory segment (e.g. created > from mm ap(flags=MAP_SHARED|...)). > > See: > https://blog.superpat.com/2010/07/14/semaphores-on-linux-sem_init-vs-sem_open/ > > Here is a program that illustrates my point (also here: > https://gist.github.com/rianhunter/a0bd4c9e8ab550ecadbe2464995726a8): [...] My bad, you are right here. This isn't a problem though, as accept mutex does not use sem_wait(), but only uses ngx_shmtx_trylock() - that is, only atomic operations. And for other mutexes we allocate ngx_shmtx_t structures from shared memory, so it works fine. -- Maxim Dounin http://mdounin.ru/ From rian at alum.mit.edu Mon Sep 2 16:48:46 2019 From: rian at alum.mit.edu (Rian Hunter) Date: Mon, 02 Sep 2019 09:48:46 -0700 Subject: Potential bug in ngx_event.c In-Reply-To: <20190902162401.GW1877@mdounin.ru> References: <797fde8d89aa12a26a84dfd52428a8f5@thelig.ht> <20190830141929.GL1877@mdounin.ru> <6fc40c129d36c08bb56fde854f4ffc91@thelig.ht> <20190831182822.GM1877@mdounin.ru> <07223a007c7d5051261e1251bcb2e64e@thelig.ht> <20190902162401.GW1877@mdounin.ru> Message-ID: <5e3f492cd08e704be1511adaa8d02de4@thelig.ht> On 2019-09-02 09:24, Maxim Dounin wrote: > My bad, you are right here. > > This isn't a problem though, as accept mutex does not use > sem_wait(), but only uses ngx_shmtx_trylock() - that is, only > atomic operations. > > And for other mutexes we allocate ngx_shmtx_t structures from > shared memory, so it works fine. Got it, very much appreciate the effort looking into this and following up. Cheers! From mdounin at mdounin.ru Mon Sep 2 17:55:01 2019 From: mdounin at mdounin.ru (Maxim Dounin) Date: Mon, 2 Sep 2019 20:55:01 +0300 Subject: Persistent buffers allocation In-Reply-To: References: Message-ID: <20190902175501.GY1877@mdounin.ru> Hello! On Thu, Aug 22, 2019 at 02:11:18PM +0000, Eran Kornblau wrote: > Hi all, > > I noticed today that I'm missing something basic... I wanted to allocate some buffers that will live throughout the lifetime of the process = > not associated with any specific request. I believe I've already done something like that in the past, and I used the cycle pool for that. > However, after digging a bit in the code, I found that in single process mode, if nginx is reloaded, a new cycle is created and the > old one destroyed. > > If my understanding is correct, this has many implications far exceeding the simple question I started with... > For example, since the configuration is allocated on the cycle pool, it means that modules should not read the configuration > of the request in any asynchronous callback, because there is a chance that the configuration was already freed since the request > object was created. > > I then found this issue - https://trac.nginx.org/nginx/ticket/945 so I guess that this problem is known and ignored because > 'master_process off' is only for dev, and no need to support reload there... > > So, back to my original question... is using ngx_cycle->pool the correct way to allocate such "persistent" buffers? > Another option is to use ngx_alloc directly (if the process quits, it doesn't matter...), but cycle pool sounds a bit more elegant > (and won't have valgrind report leaks...) Using cycle's pool is usually a way to go. Note though, that depending on the particular point your code does the allocation, using ngx_cycle->pool might be wrong. For example, during configuration parsing ngx_cycle points to the previous cycle, not the current one, and you have to use cf->cycle->pool instead (or cf->pool, which is the same). -- Maxim Dounin http://mdounin.ru/ From pluknet at nginx.com Tue Sep 3 14:28:29 2019 From: pluknet at nginx.com (Sergey Kandaurov) Date: Tue, 03 Sep 2019 14:28:29 +0000 Subject: [nginx] Detect runaway chunks in ngx_http_parse_chunked(). Message-ID: details: https://hg.nginx.org/nginx/rev/52b5ee64fe11 branches: changeset: 7562:52b5ee64fe11 user: Sergey Kandaurov date: Tue Sep 03 17:26:56 2019 +0300 description: Detect runaway chunks in ngx_http_parse_chunked(). As defined in HTTP/1.1, body chunks have the following ABNF: chunk = chunk-size [ chunk-ext ] CRLF chunk-data CRLF where chunk-data is a sequence of chunk-size octets. With this change, chunk-data that doesn't end up with CRLF at chunk-size offset will be treated as invalid, such as in the example provided below: 4 SEE-THIS-AND- 4 THAT 0 diffstat: src/http/ngx_http_parse.c | 3 +++ 1 files changed, 3 insertions(+), 0 deletions(-) diffs (13 lines): diff -r 9f1f9d6e056a -r 52b5ee64fe11 src/http/ngx_http_parse.c --- a/src/http/ngx_http_parse.c Mon Aug 19 15:16:06 2019 +0300 +++ b/src/http/ngx_http_parse.c Tue Sep 03 17:26:56 2019 +0300 @@ -2268,6 +2268,9 @@ ngx_http_parse_chunked(ngx_http_request_ break; case LF: state = sw_chunk_start; + break; + default: + goto invalid; } break; From xeioex at nginx.com Tue Sep 3 14:36:59 2019 From: xeioex at nginx.com (Dmitry Volyntsev) Date: Tue, 03 Sep 2019 14:36:59 +0000 Subject: [njs] Added Number.prototype.toFixed(). Message-ID: details: https://hg.nginx.org/njs/rev/d46a332c9c4d branches: changeset: 1156:d46a332c9c4d user: Dmitry Volyntsev date: Tue Sep 03 17:31:45 2019 +0300 description: Added Number.prototype.toFixed(). This closes #29 issue on Github. diffstat: auto/sources | 1 + src/njs_dtoa_fixed.c | 456 +++++++++++++++++++++++++++++++++++++++++++++++ src/njs_dtoa_fixed.h | 13 + src/njs_main.h | 1 + src/njs_number.c | 104 ++++++++++ src/test/njs_unit_test.c | 79 ++++++++ 6 files changed, 654 insertions(+), 0 deletions(-) diffs (709 lines): diff -r 98d3dd617ed7 -r d46a332c9c4d auto/sources --- a/auto/sources Fri Aug 23 13:25:50 2019 -0400 +++ b/auto/sources Tue Sep 03 17:31:45 2019 +0300 @@ -1,6 +1,7 @@ NJS_LIB_SRCS=" \ src/njs_diyfp.c \ src/njs_dtoa.c \ + src/njs_dtoa_fixed.c \ src/njs_strtod.c \ src/njs_murmur_hash.c \ src/njs_djb_hash.c \ diff -r 98d3dd617ed7 -r d46a332c9c4d src/njs_dtoa_fixed.c --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/njs_dtoa_fixed.c Tue Sep 03 17:31:45 2019 +0300 @@ -0,0 +1,456 @@ + +/* + * Copyright (C) Dmitry Volyntsev + * Copyright (C) NGINX, Inc. + * + * An internal fixed_dtoa() implementation based upon V8 + * src/numbers/fixed-dtoa.cc without bignum support. + * + * Copyright 2011 the V8 project authors. All rights reserved. + * Use of this source code is governed by a BSD-style license + * that can be found in the LICENSE file. + */ + +#include +#include + + +typedef struct { + uint64_t high; + uint64_t low; +} njs_diyu128_t; + + +#define njs_diyu128(_h, _l) (njs_diyu128_t) { .high = (_h), .low = (_l) } + + +njs_inline njs_diyu128_t +njs_diyu128_mul(njs_diyu128_t v, uint32_t multiplicand) +{ + uint32_t part; + uint64_t accumulator; + + accumulator = (v.low & UINT32_MAX) * multiplicand; + part = (uint32_t) (accumulator & UINT32_MAX); + + accumulator >>= 32; + accumulator = accumulator + (v.low >> 32) * multiplicand; + v.low = (accumulator << 32) + part; + + accumulator >>= 32; + accumulator = accumulator + (v.high & UINT32_MAX) * multiplicand; + part = (uint32_t) (accumulator & UINT32_MAX); + + accumulator >>= 32; + accumulator = accumulator + (v.high >> 32) * multiplicand; + v.high = (accumulator << 32) + part; + + return v; +} + + +njs_inline njs_diyu128_t +njs_diyu128_shift(njs_diyu128_t v, njs_int_t shift) +{ + /* -64 <= shift <= 64.*/ + + if (shift < 0) { + if (shift == -64) { + v.high = v.low; + v.low = 0; + + } else { + v.high <<= -shift; + v.high += v.low >> (64 + shift); + v.low <<= -shift; + } + + return v; + } + + if (shift > 0) { + if (shift == 64) { + v.low = v.high; + v.high = 0; + + } else { + v.low >>= shift; + v.low += v.high << (64 - shift); + v.high >>= shift; + } + } + + return v; +} + + +njs_inline njs_int_t +njs_diyu128_div_mod_pow2(njs_diyu128_t *v, njs_int_t power) +{ + uint64_t part_low, part_high; + njs_int_t result; + + if (power >= 64) { + result = (int) (v->high >> (power - 64)); + v->high -= (uint64_t) result << (power - 64); + + return result; + } + + part_low = v->low >> power; + part_high = v->high << (64 - power); + + result = (int) (part_low + part_high); + + v->low -= part_low << power; + v->high = 0; + + return result; +} + + +njs_inline njs_bool_t +njs_diyu128_is_zero(njs_diyu128_t v) +{ + if (v.low == 0 && v.high == 0) { + return 1; + } + + return 0; +} + + +njs_inline njs_uint_t +njs_diyu128_bit_at(njs_diyu128_t v, njs_uint_t pos) +{ + if (pos >= 64) { + return (njs_uint_t) (v.high >> (pos - 64)) & 1; + } + + return (njs_uint_t) (v.low >> pos) & 1; +} + + +static size_t +njs_fill_digits32(uint32_t number, char *start, size_t length) +{ + char c; + size_t i, j, n; + njs_int_t digit; + + n = 0; + + while (number != 0) { + digit = number % 10; + number /= 10; + start[length + n] = '0' + digit; + n++; + } + + i = length; + j = length + n - 1; + + while (i < j) { + c = start[i]; + start[i] = start[j]; + start[j] = c; + + i++; + j--; + } + + return length + n; +} + + +njs_inline size_t +njs_fill_digits32_fixed_length(uint32_t number, size_t requested_length, + char *start, size_t length) +{ + size_t i; + + i = requested_length; + + while (i-- > 0) { + start[length + i] = '0' + number % 10; + number /= 10; + } + + return length + requested_length; +} + + +njs_inline size_t +njs_fill_digits64(uint64_t number, char *start, size_t length) +{ + uint32_t part0, part1, part2, ten7; + + ten7 = 10000000; + + part2 = (uint32_t) (number % ten7); + number /= ten7; + + part1 = (uint32_t) (number % ten7); + part0 = (uint32_t) (number / ten7); + + if (part0 != 0) { + length = njs_fill_digits32(part0, start, length); + length = njs_fill_digits32_fixed_length(part1, 7, start, length); + return njs_fill_digits32_fixed_length(part2, 7, start, length); + } + + if (part1 != 0) { + length = njs_fill_digits32(part1, start, length); + return njs_fill_digits32_fixed_length(part2, 7, start, length); + } + + return njs_fill_digits32(part2, start, length); +} + + +njs_inline size_t +njs_fill_digits64_fixed_length(uint64_t number, char *start, size_t length) +{ + uint32_t part0, part1, part2, ten7; + + ten7 = 10000000; + + part2 = (uint32_t) (number % ten7); + number /= ten7; + + part1 = (uint32_t) (number % ten7); + part0 = (uint32_t) (number / ten7); + + length = njs_fill_digits32_fixed_length(part0, 3, start, length); + length = njs_fill_digits32_fixed_length(part1, 7, start, length); + + return njs_fill_digits32_fixed_length(part2, 7, start, length); +} + + +njs_inline size_t +njs_dtoa_round_up(char *start, size_t length, njs_int_t *point) +{ + size_t i; + + if (length == 0) { + start[0] = '1'; + *point = 1; + return 1; + } + + start[length - 1]++; + + for (i = length - 1; i > 0; --i) { + if (start[i] != '0' + 10) { + return length; + } + + start[i] = '0'; + start[i - 1]++; + } + + if (start[0] == '0' + 10) { + start[0] = '1'; + (*point)++; + } + + return length; +} + + +static size_t +njs_fill_fractionals(uint64_t fractionals, int exponent, njs_uint_t frac, + char *start, size_t length, njs_int_t *point) +{ + njs_int_t n, digit; + njs_uint_t i; + njs_diyu128_t fractionals128; + + /* + * 128 <= exponent <= 0. + * 0 <= fractionals * 2^exponent < 1. + */ + + if (-exponent <= 64) { + /* fractionals <= 2^56. */ + + n = -exponent; + + for (i = 0; i < frac && fractionals != 0; ++i) { + /* + * Multiplication by 10 is replaced with multiplication by 5 and + * point location adjustment. To avoid integer-overflow. + */ + + fractionals *= 5; + n--; + + digit = (njs_int_t) (fractionals >> n); + fractionals -= (uint64_t) digit << n; + + start[length++] = '0' + digit; + } + + if (n > 0 && ((fractionals >> (n - 1)) & 1)) { + length = njs_dtoa_round_up(start, length, point); + } + + } else { + + fractionals128 = njs_diyu128(fractionals, 0); + fractionals128 = njs_diyu128_shift(fractionals128, -exponent - 64); + + n = 128; + + for (i = 0; i < frac && !njs_diyu128_is_zero(fractionals128); ++i) { + /* + * Multiplication by 10 is replaced with multiplication by 5 and + * point location adjustment. To avoid integer-overflow. + */ + + fractionals128 = njs_diyu128_mul(fractionals128, 5); + n--; + + digit = njs_diyu128_div_mod_pow2(&fractionals128, n); + + start[length++] = '0' + digit; + } + + if (njs_diyu128_bit_at(fractionals128, n - 1)) { + length = njs_dtoa_round_up(start, length, point); + } + } + + return length; +} + + +njs_inline size_t +njs_trim_zeroes(char *start, size_t length, njs_int_t *point) +{ + size_t i, n; + + while (length > 0 && start[length - 1] == '0') { + length--; + } + + n = 0; + + while (n < length && start[n] == '0') { + n++; + } + + if (n != 0) { + for (i = n; i < length; ++i) { + start[i - n] = start[i]; + } + + length -= n; + *point -= n; + } + + return length; +} + + +size_t +njs_fixed_dtoa(double value, njs_uint_t frac, char *start, njs_int_t *point) +{ + size_t length; + uint32_t quotient; + uint64_t significand, divisor, dividend, remainder, integral, fract; + njs_int_t exponent; + njs_diyfp_t v; + + length = 0; + v = njs_d2diyfp(value); + + significand = v.significand; + exponent = v.exp; + + /* exponent <= 19. */ + + if (exponent + NJS_SIGNIFICAND_SIZE > 64) { + /* exponent > 11. */ + + divisor = njs_uint64(0xB1, 0xA2BC2EC5); /* 5 ^ 17 */ + + dividend = significand; + + /* + * Let v = f * 2^e with f == significand and e == exponent. + * Then need q (quotient) and r (remainder) as follows: + * f * 2^e = q * 5^17 * 2^17 + r + * If e > 17 then + * f * 2^(e-17) = q * 5^17 + r/2^17 + * else + * f = q * 5^17 * 2^(17-e) + r/2^e + */ + + if (exponent > 17) { + /* (e - 17) <= 3. */ + dividend <<= exponent - 17; + + quotient = (uint32_t) (dividend / divisor); + remainder = (dividend % divisor) << 17; + + } else { + divisor <<= 17 - exponent; + + quotient = (uint32_t) (dividend / divisor); + remainder = (dividend % divisor) << exponent; + } + + length = njs_fill_digits32(quotient, start, length); + length = njs_fill_digits64_fixed_length(remainder, start, length); + *point = (njs_int_t) length; + + } else if (exponent >= 0) { + /* 0 <= exponent <= 11. */ + + significand <<= exponent; + length = njs_fill_digits64(significand, start, length); + *point = (njs_int_t) length; + + } else if (exponent > -NJS_SIGNIFICAND_SIZE) { + /* -53 < exponent < 0. */ + + integral = significand >> -exponent; + fract = significand - (integral << -exponent); + + if (integral > UINT32_MAX) { + length = njs_fill_digits64(integral, start, length); + + } else { + length = njs_fill_digits32((uint32_t) integral, start, length); + } + + *point = (njs_int_t) length; + length = njs_fill_fractionals(fract, exponent, frac, start, length, + point); + + } else if (exponent < -128) { + /* Valid for frac =< 20 only. TODO: bignum support. */ + + start[0] = '\0'; + + *point = -frac; + + } else { + /* -128 <= exponent <= -53. */ + + *point = 0; + length = njs_fill_fractionals(significand, exponent, frac, start, + length, point); + } + + length = njs_trim_zeroes(start, length, point); + start[length] = '\0'; + + if (length == 0) { + *point = -frac; + } + + return length; +} diff -r 98d3dd617ed7 -r d46a332c9c4d src/njs_dtoa_fixed.h --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/src/njs_dtoa_fixed.h Tue Sep 03 17:31:45 2019 +0300 @@ -0,0 +1,13 @@ + +/* + * Copyright (C) Dmitry Volyntsev + * Copyright (C) Nginx, Inc. + */ + +#ifndef _NJS_DTOA_FIXED_H_INCLUDED_ +#define _NJS_DTOA_FIXED_H_INCLUDED_ + +NJS_EXPORT size_t njs_fixed_dtoa(double value, njs_uint_t frac, char *start, + njs_int_t *point); + +#endif /* _NJS_DTOA_FIXED_H_INCLUDED_ */ diff -r 98d3dd617ed7 -r d46a332c9c4d src/njs_main.h --- a/src/njs_main.h Fri Aug 23 13:25:50 2019 -0400 +++ b/src/njs_main.h Tue Sep 03 17:31:45 2019 +0300 @@ -16,6 +16,7 @@ #include #include #include +#include #include #include #include diff -r 98d3dd617ed7 -r d46a332c9c4d src/njs_number.c --- a/src/njs_number.c Fri Aug 23 13:25:50 2019 -0400 +++ b/src/njs_number.c Tue Sep 03 17:31:45 2019 +0300 @@ -541,6 +541,101 @@ njs_number_prototype_to_string(njs_vm_t } +static njs_int_t +njs_number_prototype_to_fixed(njs_vm_t *vm, njs_value_t *args, + njs_uint_t nargs, njs_index_t unused) +{ + u_char *p; + int32_t frac; + double number; + size_t length, size; + njs_int_t point, prefix, postfix; + njs_value_t *value; + u_char buf[128], buf2[128]; + + /* 128 > 100 + 21 + njs_length(".-\0"). */ + + value = &args[0]; + + if (value->type != NJS_NUMBER) { + if (value->type == NJS_OBJECT_NUMBER) { + value = njs_object_value(value); + + } else { + njs_type_error(vm, "unexpected value type:%s", + njs_type_string(value->type)); + return NJS_ERROR; + } + } + + frac = njs_primitive_value_to_integer(njs_arg(args, nargs, 1)); + if (njs_slow_path(frac < 0 || frac > 100)) { + njs_range_error(vm, "digits argument must be between 0 and 100"); + return NJS_ERROR; + } + + number = njs_number(value); + + if (njs_slow_path(isnan(number) || fabs(number) >= 1e21)) { + return njs_number_to_string(vm, &vm->retval, value); + } + + point = 0; + length = njs_fixed_dtoa(number, frac, (char *) buf, &point); + + prefix = 0; + postfix = 0; + + if (point <= 0) { + prefix = -point + 1; + point = 1; + } + + if (prefix + (njs_int_t) length < point + frac) { + postfix = point + frac - length - prefix; + } + + size = prefix + length + postfix + !!(number < 0); + + if (frac > 0) { + size += njs_length("."); + } + + p = buf2; + + while (--prefix >= 0) { + *p++ = '0'; + } + + if (length != 0) { + p = njs_cpymem(p, buf, length); + } + + while (--postfix >= 0) { + *p++ = '0'; + } + + p = njs_string_alloc(vm, &vm->retval, size, size); + if (njs_slow_path(p == NULL)) { + return NJS_ERROR; + } + + if (number < 0) { + *p++ = '-'; + } + + p = njs_cpymem(p, buf2, point); + + if (frac > 0) { + *p++ = '.'; + + p = njs_cpymem(p, &buf2[point], frac); + } + + return NJS_OK; +} + + /* * The radix equal to 2 produces the longest intergral value of a number * and the maximum value consists of 1024 digits and minus sign. @@ -641,6 +736,15 @@ static const njs_object_prop_t njs_numb .writable = 1, .configurable = 1, }, + + { + .type = NJS_PROPERTY, + .name = njs_string("toFixed"), + .value = njs_native_function(njs_number_prototype_to_fixed, + NJS_SKIP_ARG, NJS_INTEGER_ARG), + .writable = 1, + .configurable = 1, + }, }; diff -r 98d3dd617ed7 -r d46a332c9c4d src/test/njs_unit_test.c --- a/src/test/njs_unit_test.c Fri Aug 23 13:25:50 2019 -0400 +++ b/src/test/njs_unit_test.c Tue Sep 03 17:31:45 2019 +0300 @@ -427,6 +427,85 @@ static njs_unit_test_t njs_test[] = { njs_str("NaN.toString(NaN)"), njs_str("RangeError") }, + /* Number.prototype.toFixed(frac) method. */ + + { njs_str("(900.1).toFixed(1)"), + njs_str("900.1") }, + + { njs_str("(0).toFixed(0)"), + njs_str("0") }, + + { njs_str("(0).toFixed(3)"), + njs_str("0.000") }, + + { njs_str("(7).toFixed()"), + njs_str("7") }, + + { njs_str("(7).toFixed(0)"), + njs_str("7") }, + + { njs_str("(7).toFixed(2)"), + njs_str("7.00") }, + + { njs_str("(-900.1).toFixed(3.3)"), + njs_str("-900.100") }, + + { njs_str("(900.123).toFixed(5)"), + njs_str("900.12300") }, + + { njs_str("(1/3).toFixed(5)"), + njs_str("0.33333") }, + + { njs_str("(new Number(1/3)).toFixed(5)"), + njs_str("0.33333") }, + + { njs_str("(new Number(1/3)).toFixed(5)"), + njs_str("0.33333") }, + + { njs_str("(1/3).toFixed({toString(){return '5'}})"), + njs_str("0.33333") }, + + { njs_str("(1/3).toFixed(100)"), + njs_str("0.3333333333333333148296162562473909929394721984863281250000000000000000000000000000000000000000000000") }, + + { njs_str("(1.23e+20).toFixed(2)"), + njs_str("123000000000000000000.00") }, + + { njs_str("(1.23e-10).toFixed(2)"), + njs_str("0.00") }, + + { njs_str("(1.23e-10).toFixed(15)"), + njs_str("0.000000000123000") }, + + { njs_str("(1.23e-10).toFixed(100)"), + njs_str("0.0000000001229999999999999888422768137255427361997917046210204716771841049194335937500000000000000000") }, + + { njs_str("NaN.toFixed(1)"), + njs_str("NaN") }, + +#if 0 /* FIXME: bignum support is requred to support frac >= 20 */ + { njs_str("(2 ** -100).toFixed(100)"), + njs_str("0.0000000000000000000000000000007888609052210118054117285652827862296732064351090230047702789306640625") }, +#endif + + { njs_str("(1000000000000000128).toString()"), + njs_str("1000000000000000100") }, + + { njs_str("(1000000000000000128).toFixed(0)"), + njs_str("1000000000000000128") }, + + { njs_str("(1e21).toFixed(1)"), + njs_str("1e+21") }, + + { njs_str("Number.prototype.toFixed.call({})"), + njs_str("TypeError: unexpected value type:object") }, + + { njs_str("(0).toFixed(-1)"), + njs_str("RangeError: digits argument must be between 0 and 100") }, + + { njs_str("(0).toFixed(101)"), + njs_str("RangeError: digits argument must be between 0 and 100") }, + /* An object "valueOf/toString" methods. */ { njs_str("var a = { valueOf: function() { return 1 } }; +a"), From mdounin at mdounin.ru Tue Sep 3 14:39:00 2019 From: mdounin at mdounin.ru (Maxim Dounin) Date: Tue, 3 Sep 2019 17:39:00 +0300 Subject: [PATCH] ngx_conf_file: "include ./" acts relative to currently parsed file In-Reply-To: References: <704100d8a8772a4bc2fa.1567058681@bas> <20190830140221.GK1877@mdounin.ru> Message-ID: <20190903143900.GA1877@mdounin.ru> Hello! On Fri, Aug 30, 2019 at 04:28:00PM +0200, Guillaume Outters wrote: > Le 2019-08-30 16:02, Maxim Dounin a ?crit : > > > Changing this to resolve relative paths from the current included > > file instead is possible, but would be a major change - I suspect > > it will break a lot of configurations. Not sure we are going to > > do this. > > > > On Thu, Aug 29, 2019 at 08:04:41AM +0200, Guillaume Outters wrote: > > > >> The following patch adds a simple heuristic to include: if the > >> includee starts with "./", it is considered relative to the > >> current file. If not, the current heuristic applies [?] > > > > Certainly I'm against this approach, as it breaks POLA. The > > "include ./example.conf;" construct shouldn't be handled > > differently from "include example.conf", these are clearly the > > same thing. > > Hmm, you're right. Perhaps an unused diacritical would be better there, > as: > include @example.conf; > or an explicit flag: > include example.conf local; > include example.conf -l; # Hmm, maybe too Apache-styled? > include example.conf relative; > include example.conf from_there; > This would be even better than the diacritical: prevents crash of > existing configurations who use strangely- at -prefixed names, more > explicit, and more respectful of nginx' config principles (optional > flags go at the end). > On the other hand, this is longer to type, would make config reading > more prone to missing the keyword (from my point of view, as I tend to > read left-to-right and lazily stop when I have gathered what I was > looking for) and? looks more complex to implement (this patch was my > first peak ever at nginx' source). > But even to me, the "pros" seems to overthrow the "cons". > > I would be glad to know what you think of this last solution (and which > keyword you would choose then). I can't say I like either of the variants. Additionally, all variants with an additional explicit flags won't work in other cases where a configuration prefix is currently used, such as ssl_certificate or auth_basic_user_file. On the other hand, obviously enough it should be possible to resolve from the current included file all paths which are currently resolved from the configuration prefix. (Further, for things like auth_basic_user_file, which is evaluated at runtime, it would be non-trivial to actually implement resolving from the current included configuration file - we'll have to remember which file was current. Approach which is used now is much simpler.) -- Maxim Dounin http://mdounin.ru/ From guillaume-nginx at outters.eu Tue Sep 3 16:21:47 2019 From: guillaume-nginx at outters.eu (Guillaume Outters) Date: Tue, 03 Sep 2019 18:21:47 +0200 Subject: [PATCH] ngx_conf_file: "include ./" acts relative to currently parsed file In-Reply-To: <20190903143900.GA1877@mdounin.ru> References: <704100d8a8772a4bc2fa.1567058681@bas> <20190830140221.GK1877@mdounin.ru> <20190903143900.GA1877@mdounin.ru> Message-ID: <7fe74e4e34196c91a5c859ec5ac4e47e@outters.eu> Le 2019-09-03 16:39, Maxim Dounin a ?crit : >> include @example.conf; >> include example.conf local; >> include example.conf -l; >> include example.conf relative; >> include example.conf from_there; >> include example.conf nearby; > > I can't say I like either of the variants. ? Neither do I (although the "nearby", that I submitted in my last patch that you perhaps haven't read yet, seems the most natural one). > Additionally, all variants with an additional explicit flags won't > work in other cases where a configuration prefix is currently > used, such as ssl_certificate or auth_basic_user_file. On the > other hand, obviously enough it should be possible to resolve from > the current included file all paths which are currently resolved > from the configuration prefix. The big difference between include and all other directives is that include does not do variable resolution. This allows the misuse of (request-time) variables to store absolute paths, with things such as: map $http_host $app_root { default /var/www/app1; } # Or a set, or $document_root/.., whatever. ssl_certificate $app_root/certs/thissite.pem; This makes an unnecessary variable evaluation at runtime, but as it's technically possible, and as such I'm quite certain there exists a ton of such configurations in the world (if only to reduce typing, and thus copy-paste errors). But include has no such configurability. Both of these advocate in favor of config-time variables: - include would benefit of a bit of configurability: webapps containing their nginx config could be dropped by a sysadm wherever he wanted, set a (config-time) $app_root, and the app would work out of the box by using $app_root to reference snippets indepently of where it has been put. - other directives (that for now misuse request-time variables) would have a clear separation between config-time resolved parts and request-time ones, with a performance gain on the config-time ones But this would require a bit of thinking (about where variables do get stored during the config reading, do they persist until the request-time, attached to each block with the last value they acquired during config reading, to serve as default values for the request-time variables, and so on). In the meantime (and who can tell the meantime's duration?), the "nearby" keyword is a simple, semi-elegant placeholder for that. (Hum? in fact, even with a way to "setenv $app_dir; include $app_dir/nginx/snippet.conf;", I think I would continue to just "include snippet.conf nearby;") -- Guillaume From thibaultcha at fastmail.com Tue Sep 3 19:07:53 2019 From: thibaultcha at fastmail.com (Thibault Charbonnier) Date: Tue, 03 Sep 2019 12:07:53 -0700 Subject: [PATCH] HTTP: added the preserve_method option to the error_page directive. Message-ID: # HG changeset patch # User Thibault Charbonnier # Date 1567537546 25200 # Tue Sep 03 12:05:46 2019 -0700 # Node ID 68ba3d36bff4213e3fedc538021e8cbece85e508 # Parent 52b5ee64fe11ec267a0767cbb9874c8cae652299 HTTP: added the preserve_method option to the error_page directive. As of today, the `error_page` directive will override the request method if redirecting to a non-named location and if said method is not `HEAD`. As for using named location (which preserve the request method), some error cases won't be able to perform the redirect (e.g. HTTP 414). This patch allows users who wish to set a single "catch-all" error location block to preserve the client method for logging purposes. A new option `preserve_method` is added to the directive: error_page 400 preserve_method /error_handler; diff -r 52b5ee64fe11 -r 68ba3d36bff4 src/http/ngx_http_core_module.c --- a/src/http/ngx_http_core_module.c Tue Sep 03 17:26:56 2019 +0300 +++ b/src/http/ngx_http_core_module.c Tue Sep 03 12:05:46 2019 -0700 @@ -4587,6 +4587,7 @@ u_char *p; ngx_int_t overwrite; + ngx_flag_t preserve_method; ngx_str_t *value, uri, args; ngx_uint_t i, n; ngx_http_err_page_t *err; @@ -4601,35 +4602,40 @@ } } + overwrite = -1; + preserve_method = 0; + + n = 1; value = cf->args->elts; - i = cf->args->nelts - 2; - - if (value[i].data[0] == '=') { - if (i == 1) { + for (i = cf->args->nelts - 3; i < cf->args->nelts - 1; i++) { + if (value[i].data[0] == '=') { + if (value[i].len > 1) { + overwrite = ngx_atoi(&value[i].data[1], value[i].len - 1); + if (overwrite == NGX_ERROR) { + ngx_conf_log_error(NGX_LOG_EMERG, cf, 0, + "invalid value \"%V\"", &value[i]); + return NGX_CONF_ERROR; + } + + n++; + + } else { + overwrite = 0; + } + + n++; + + } else if (ngx_strncmp(value[i].data, "preserve_method", 15) == 0) { + preserve_method = 1; + n++; + } + + if (n > 1 && i == 1) { ngx_conf_log_error(NGX_LOG_EMERG, cf, 0, "invalid value \"%V\"", &value[i]); return NGX_CONF_ERROR; } - - if (value[i].len > 1) { - overwrite = ngx_atoi(&value[i].data[1], value[i].len - 1); - - if (overwrite == NGX_ERROR) { - ngx_conf_log_error(NGX_LOG_EMERG, cf, 0, - "invalid value \"%V\"", &value[i]); - return NGX_CONF_ERROR; - } - - } else { - overwrite = 0; - } - - n = 2; - - } else { - overwrite = -1; - n = 1; } uri = value[cf->args->nelts - 1]; @@ -4680,6 +4686,7 @@ } err->overwrite = overwrite; + err->preserve_method = preserve_method; if (overwrite == -1) { switch (err->status) { diff -r 52b5ee64fe11 -r 68ba3d36bff4 src/http/ngx_http_core_module.h --- a/src/http/ngx_http_core_module.h Tue Sep 03 17:26:56 2019 +0300 +++ b/src/http/ngx_http_core_module.h Tue Sep 03 12:05:46 2019 -0700 @@ -294,6 +294,7 @@ ngx_int_t overwrite; ngx_http_complex_value_t value; ngx_str_t args; + ngx_flag_t preserve_method; } ngx_http_err_page_t; diff -r 52b5ee64fe11 -r 68ba3d36bff4 src/http/ngx_http_special_response.c --- a/src/http/ngx_http_special_response.c Tue Sep 03 17:26:56 2019 +0300 +++ b/src/http/ngx_http_special_response.c Tue Sep 03 12:05:46 2019 -0700 @@ -611,7 +611,7 @@ args = err_page->args; } - if (r->method != NGX_HTTP_HEAD) { + if (!err_page->preserve_method && r->method != NGX_HTTP_HEAD) { r->method = NGX_HTTP_GET; r->method_name = ngx_http_core_get_method; } -------------- next part -------------- A non-text attachment was scrubbed... Name: patch Type: application/octet-stream Size: 4168 bytes Desc: not available URL: From pluknet at nginx.com Wed Sep 4 12:01:28 2019 From: pluknet at nginx.com (Sergey Kandaurov) Date: Wed, 04 Sep 2019 12:01:28 +0000 Subject: [nginx] Fixed "return" with discarding invalid chunked body. Message-ID: details: https://hg.nginx.org/nginx/rev/a7e8f953408e branches: changeset: 7563:a7e8f953408e user: Sergey Kandaurov date: Wed Sep 04 13:33:51 2019 +0300 description: Fixed "return" with discarding invalid chunked body. When ngx_http_discard_request_body() call was added to ngx_http_send_response(), there were no return codes other than NGX_OK and NGX_HTTP_INTERNAL_SERVER_ERROR. Now it can also return NGX_HTTP_BAD_REQUEST, but ngx_http_send_response() still incorrectly transforms it to NGX_HTTP_INTERNAL_SERVER_ERROR. The fix is to propagate ngx_http_discard_request_body() errors. diffstat: src/http/ngx_http_core_module.c | 6 ++++-- 1 files changed, 4 insertions(+), 2 deletions(-) diffs (16 lines): diff -r 52b5ee64fe11 -r a7e8f953408e src/http/ngx_http_core_module.c --- a/src/http/ngx_http_core_module.c Tue Sep 03 17:26:56 2019 +0300 +++ b/src/http/ngx_http_core_module.c Wed Sep 04 13:33:51 2019 +0300 @@ -1660,8 +1660,10 @@ ngx_http_send_response(ngx_http_request_ ngx_buf_t *b; ngx_chain_t out; - if (ngx_http_discard_request_body(r) != NGX_OK) { - return NGX_HTTP_INTERNAL_SERVER_ERROR; + rc = ngx_http_discard_request_body(r); + + if (rc != NGX_OK) { + return rc; } r->headers_out.status = status; From mdounin at mdounin.ru Wed Sep 4 12:01:55 2019 From: mdounin at mdounin.ru (Maxim Dounin) Date: Wed, 4 Sep 2019 15:01:55 +0300 Subject: [PATCH] HTTP: added the preserve_method option to the error_page directive. In-Reply-To: References: Message-ID: <20190904120155.GF1877@mdounin.ru> Hello! On Tue, Sep 03, 2019 at 12:07:53PM -0700, Thibault Charbonnier wrote: > # HG changeset patch > # User Thibault Charbonnier > # Date 1567537546 25200 > # Tue Sep 03 12:05:46 2019 -0700 > # Node ID 68ba3d36bff4213e3fedc538021e8cbece85e508 > # Parent 52b5ee64fe11ec267a0767cbb9874c8cae652299 > HTTP: added the preserve_method option to the error_page directive. > > As of today, the `error_page` directive will override the request method > if redirecting to a non-named location and if said method is not `HEAD`. > > As for using named location (which preserve the request method), some > error cases won't be able to perform the redirect (e.g. HTTP 414). > > This patch allows users who wish to set a single "catch-all" error > location block to preserve the client method for logging purposes. > > A new option `preserve_method` is added to the directive: > > error_page 400 preserve_method /error_handler; No, thank you. This seems to complicate things for no real reason. For logging purposes consider using $request variable, which constains original request line, including request method, as long as it is available. Note well that it is generally a bad idea to redirect 400 errors. While it may work in some cases, error 400 indicate a serious error with the request, and it might not be possible to properly handle such requests in a normal request processing pipeline due to lack of various request fields normally available. -- Maxim Dounin http://mdounin.ru/ From phillip.odam at rosettahealth.com Wed Sep 4 12:35:05 2019 From: phillip.odam at rosettahealth.com (Phillip Odam) Date: Wed, 4 Sep 2019 08:35:05 -0400 Subject: Needing TLS handshake to fail Message-ID: <4af9586e-195f-fd88-2ac9-ceb4f3a5d92a@rosettahealth.com> Hi, I tried asking the following on the general mailing list but I'm guessing this is tending more towards development. I have a project that involves mutual / two way TLS and one of the requirements is that the TLS handshake must fail ie. be terminated before completion if the handshake is in anyway unsuccessful, eg. no client certificate provided or client certificate not trusted. After having no success getting nginx (v1.16.1) & openssl (v1.0.2k-fips) to fail the handshake I ended up looking at the nginx source code, in particular src/event/ngx_event_openssl.c, and from what I read here https://www.openssl.org/docs/man1.0.2/man3/SSL_CTX_set_verify.html I think a small but necessary code change is required. Some possible approaches when choosing to remain using nginx as the server end of the mutual TLS connection * in *static int ngx_ssl_verify_callback(int ok, X509_STORE_CTX *x509_store)* make it configurable whether *1* is always returned or the value of *ok* * in *ngx_int_t ngx_ssl_client_certificate(ngx_conf_t *cf, ngx_ssl_t *ssl, ngx_str_t *cert, ngx_int_t depth)* make it configurable whether *SSL_CTX_set_verify(ssl->ctx, SSL_VERIFY_PEER, ngx_ssl_verify_callback);* is called or *SSL_CTX_set_verify(ssl->ctx, SSL_VERIFY_PEER, NULL);* Is a code change required or is there a way for the handshake failure to be 'enabled' as opposed to ending up with a successfully established TLS connection. Admittedly within nginx there's all the detail that the TLS connection doesn't conform to the configured requirements of the TLS connection but this doesn't satisfy the requirements for the project. I won't bother going in to the details of the project but will just say it's a third party certification body that requires the TLS handshake to be terminated before completion if the handshake is in anyway unsuccessful. We're currently looking at alternate software but would really love to be able to pull this back in to nginx when/if the day comes that nginx supports this. Assuming development is required, is this something already on the backlog? My only comment for enabling the TLS handshake failure is it'd be really nice if it were configurable at the level of the virtual host. Regards, Phillip -------------- next part -------------- An HTML attachment was scrubbed... URL: From mdounin at mdounin.ru Wed Sep 4 12:48:26 2019 From: mdounin at mdounin.ru (Maxim Dounin) Date: Wed, 4 Sep 2019 15:48:26 +0300 Subject: Needing TLS handshake to fail In-Reply-To: <4af9586e-195f-fd88-2ac9-ceb4f3a5d92a@rosettahealth.com> References: <4af9586e-195f-fd88-2ac9-ceb4f3a5d92a@rosettahealth.com> Message-ID: <20190904124826.GG1877@mdounin.ru> Hello! On Wed, Sep 04, 2019 at 08:35:05AM -0400, Phillip Odam wrote: > Hi, > > I tried asking the following on the general mailing list but I'm > guessing this is tending more towards development. > > I have a project that involves mutual / two way TLS and one of the > requirements is that the TLS handshake must fail ie. be terminated > before completion if the handshake is in anyway unsuccessful, eg. no > client certificate provided or client certificate not trusted. > > After having no success getting nginx (v1.16.1) & openssl (v1.0.2k-fips) > to fail the handshake I ended up looking at the nginx source code, in > particular src/event/ngx_event_openssl.c, and from what I read here > https://www.openssl.org/docs/man1.0.2/man3/SSL_CTX_set_verify.html I > think a small but necessary code change is required. > > Some possible approaches when choosing to remain using nginx as the > server end of the mutual TLS connection > > * in *static int ngx_ssl_verify_callback(int ok, X509_STORE_CTX > *x509_store)* make it configurable whether *1* is always returned or > the value of *ok* > * in *ngx_int_t ngx_ssl_client_certificate(ngx_conf_t *cf, ngx_ssl_t > *ssl, ngx_str_t *cert, ngx_int_t depth)* make it configurable > whether *SSL_CTX_set_verify(ssl->ctx, SSL_VERIFY_PEER, > ngx_ssl_verify_callback);* is called or > *SSL_CTX_set_verify(ssl->ctx, SSL_VERIFY_PEER, NULL);* > > Is a code change required or is there a way for the handshake failure to > be 'enabled' as opposed to ending up with a successfully established TLS > connection. Admittedly within nginx there's all the detail that the TLS > connection doesn't conform to the configured requirements of the TLS > connection but this doesn't satisfy the requirements for the project. > > I won't bother going in to the details of the project but will just say > it's a third party certification body that requires the TLS handshake to > be terminated before completion if the handshake is in anyway > unsuccessful. We're currently looking at alternate software but would > really love to be able to pull this back in to nginx when/if the day > comes that nginx supports this. > > Assuming development is required, is this something already on the > backlog? My only comment for enabling the TLS handshake failure is it'd > be really nice if it were configurable at the level of the virtual host. No, there is nothing like this in the backlog. As you can see from the code, nginx approach is to proceed with handshake regardless of client certificate verification results, and then reject requests on HTTP level (and/or just make verification results available for logging). This approach is believed to be more user friendly - and, in particular, it allows one to configure an error page explaining the problem. Note well that patches to change this are likely to be rejected unless there is some justification better than "there is a third party that requires this". -- Maxim Dounin http://mdounin.ru/ From phillip.odam at nitorgroup.com Wed Sep 4 13:25:57 2019 From: phillip.odam at nitorgroup.com (Phillip Odam) Date: Wed, 4 Sep 2019 09:25:57 -0400 Subject: Needing TLS handshake to fail In-Reply-To: <20190904124826.GG1877@mdounin.ru> References: <4af9586e-195f-fd88-2ac9-ceb4f3a5d92a@rosettahealth.com> <20190904124826.GG1877@mdounin.ru> Message-ID: <395432e1-a913-47a7-4502-31bcb8970bdf@nitorgroup.com> Hi Maxim Thanks for the prompt feedback. My understanding for requiring the TLS itself to fail, as opposed to doing exactly what you described which is also exactly what we've done for other endponts... I quite like nginx's ability here, is that it prevents being able to take advantage of exploits further down the chain even if these are perceived abilities to exploit. What I'm referring to is if the TLS handshake succeeds and a HTTP response is returned was any part of the HTTP request read in to buffers by nginx? I'm no networking / security expert so can't talk in more detail and about other possible lines of attack. While I certainly get the default being a more user friendly response it'd certainly be convenient if nginx were able to be configured to be rigid. Also, the following reference was provided providing a basis for the TLS handshake requirement, sections 7.2.1 and 7.2.2 - https://tools.ietf.org/html/rfc5246#section-7.2.1. Admittedly production implementations vs RFCs don't always work well together and I'm certainly not nginx expert but from this lay persons perspective I certainly see value in allowing nginx to be configurable and fail the TLS handshake. My C is something like 20yrs old and rusty and certainly have no desire to have to build nginx from source each time we want to release a new version. I can certainly understand there's no desire within the existing development community to make the change but if the community is willing to adopt the change from the perspective of the idea I'm certainly interested in giving it a crack and seeing if I can neat and tidily work it in to the code base and offer up a patch. Cheers Phillip On 9/4/19 8:48 AM, Maxim Dounin wrote: > Hello! > > On Wed, Sep 04, 2019 at 08:35:05AM -0400, Phillip Odam wrote: > >> Hi, >> >> I tried asking the following on the general mailing list but I'm >> guessing this is tending more towards development. >> >> I have a project that involves mutual / two way TLS and one of the >> requirements is that the TLS handshake must fail ie. be terminated >> before completion if the handshake is in anyway unsuccessful, eg. no >> client certificate provided or client certificate not trusted. >> >> After having no success getting nginx (v1.16.1) & openssl (v1.0.2k-fips) >> to fail the handshake I ended up looking at the nginx source code, in >> particular src/event/ngx_event_openssl.c, and from what I read here >> https://www.openssl.org/docs/man1.0.2/man3/SSL_CTX_set_verify.html I >> think a small but necessary code change is required. >> >> Some possible approaches when choosing to remain using nginx as the >> server end of the mutual TLS connection >> >> * in *static int ngx_ssl_verify_callback(int ok, X509_STORE_CTX >> *x509_store)* make it configurable whether *1* is always returned or >> the value of *ok* >> * in *ngx_int_t ngx_ssl_client_certificate(ngx_conf_t *cf, ngx_ssl_t >> *ssl, ngx_str_t *cert, ngx_int_t depth)* make it configurable >> whether *SSL_CTX_set_verify(ssl->ctx, SSL_VERIFY_PEER, >> ngx_ssl_verify_callback);* is called or >> *SSL_CTX_set_verify(ssl->ctx, SSL_VERIFY_PEER, NULL);* >> >> Is a code change required or is there a way for the handshake failure to >> be 'enabled' as opposed to ending up with a successfully established TLS >> connection. Admittedly within nginx there's all the detail that the TLS >> connection doesn't conform to the configured requirements of the TLS >> connection but this doesn't satisfy the requirements for the project. >> >> I won't bother going in to the details of the project but will just say >> it's a third party certification body that requires the TLS handshake to >> be terminated before completion if the handshake is in anyway >> unsuccessful. We're currently looking at alternate software but would >> really love to be able to pull this back in to nginx when/if the day >> comes that nginx supports this. >> >> Assuming development is required, is this something already on the >> backlog? My only comment for enabling the TLS handshake failure is it'd >> be really nice if it were configurable at the level of the virtual host. > No, there is nothing like this in the backlog. As you can see > from the code, nginx approach is to proceed with handshake > regardless of client certificate verification results, and then > reject requests on HTTP level (and/or just make verification > results available for logging). This approach is believed to be > more user friendly - and, in particular, it allows one to > configure an error page explaining the problem. > > Note well that patches to change this are likely to be rejected > unless there is some justification better than "there is a third > party that requires this". > From mdounin at mdounin.ru Wed Sep 4 14:38:04 2019 From: mdounin at mdounin.ru (Maxim Dounin) Date: Wed, 4 Sep 2019 17:38:04 +0300 Subject: Needing TLS handshake to fail In-Reply-To: <395432e1-a913-47a7-4502-31bcb8970bdf@nitorgroup.com> References: <4af9586e-195f-fd88-2ac9-ceb4f3a5d92a@rosettahealth.com> <20190904124826.GG1877@mdounin.ru> <395432e1-a913-47a7-4502-31bcb8970bdf@nitorgroup.com> Message-ID: <20190904143804.GH1877@mdounin.ru> Hello! On Wed, Sep 04, 2019 at 09:25:57AM -0400, Phillip Odam wrote: [...] > Also, the following reference was provided providing a basis for the TLS > handshake requirement, sections 7.2.1 and 7.2.2 - > https://tools.ietf.org/html/rfc5246#section-7.2.1. Admittedly production > implementations vs RFCs don't always work well together and I'm > certainly not nginx expert but from this lay persons perspective I > certainly see value in allowing nginx to be configurable and fail the > TLS handshake. Just a note: these sections do not require implimentations to terminate the TLS handshake. Rather, they describe what to do if you want to terminate the TLS handshake. -- Maxim Dounin http://mdounin.ru/ From mdounin at mdounin.ru Wed Sep 4 16:01:24 2019 From: mdounin at mdounin.ru (Maxim Dounin) Date: Wed, 4 Sep 2019 19:01:24 +0300 Subject: [PATCH] Core: monitoring anonymous map on Darwin platform In-Reply-To: References: Message-ID: <20190904160124.GI1877@mdounin.ru> Hello! On Fri, Aug 23, 2019 at 06:00:29AM +0100, David Carlier wrote: > # HG changeset patch > # User David Carlier > # Date 1566493379 -3600 > # Thu Aug 22 18:02:59 2019 +0100 > # Node ID adc68231e590554860b11ee851b293e46ba652db > # Parent 9f1f9d6e056a4f85907957ef263f78a426ae4f9c > Core: monitoring anonymous map on Darwin platform. > > Allows to highligh those pages from nginx process with > this identifier. > > diff -r 9f1f9d6e056a -r adc68231e590 auto/os/darwin > --- a/auto/os/darwin Mon Aug 19 15:16:06 2019 +0300 > +++ b/auto/os/darwin Thu Aug 22 18:02:59 2019 +0100 > @@ -118,3 +118,21 @@ > ngx_feature_test="int32_t lock = 0; > if (!OSAtomicCompareAndSwap32Barrier(0, 1, &lock)) return 1" > . auto/feature > + > +# Darwin can tag anonymous pages to be tracked > +# with tools like vmmap > +# free ID are guaranteed to be free from 240 to 255 > +# for userland applications Could you please clarify how this is expected to help? As far as I can see, relevant memory regions can be easily identified in vmmap even without tags. [...] -- Maxim Dounin http://mdounin.ru/ From devnexen at gmail.com Wed Sep 4 16:04:15 2019 From: devnexen at gmail.com (David CARLIER) Date: Wed, 4 Sep 2019 16:04:15 +0000 Subject: [PATCH] Core: monitoring anonymous map on Darwin platform In-Reply-To: <20190904160124.GI1877@mdounin.ru> References: <20190904160124.GI1877@mdounin.ru> Message-ID: Well it highlight them a little bit more, easier to parse, easier to identify in case with plethora of third party modules. Regards. On Wed, 4 Sep 2019 at 16:01, Maxim Dounin wrote: > > Hello! > > On Fri, Aug 23, 2019 at 06:00:29AM +0100, David Carlier wrote: > > > # HG changeset patch > > # User David Carlier > > # Date 1566493379 -3600 > > # Thu Aug 22 18:02:59 2019 +0100 > > # Node ID adc68231e590554860b11ee851b293e46ba652db > > # Parent 9f1f9d6e056a4f85907957ef263f78a426ae4f9c > > Core: monitoring anonymous map on Darwin platform. > > > > Allows to highligh those pages from nginx process with > > this identifier. > > > > diff -r 9f1f9d6e056a -r adc68231e590 auto/os/darwin > > --- a/auto/os/darwin Mon Aug 19 15:16:06 2019 +0300 > > +++ b/auto/os/darwin Thu Aug 22 18:02:59 2019 +0100 > > @@ -118,3 +118,21 @@ > > ngx_feature_test="int32_t lock = 0; > > if (!OSAtomicCompareAndSwap32Barrier(0, 1, &lock)) return 1" > > . auto/feature > > + > > +# Darwin can tag anonymous pages to be tracked > > +# with tools like vmmap > > +# free ID are guaranteed to be free from 240 to 255 > > +# for userland applications > > Could you please clarify how this is expected to help? As far as > I can see, relevant memory regions can be easily identified in > vmmap even without tags. > > [...] > > -- > Maxim Dounin > http://mdounin.ru/ > _______________________________________________ > nginx-devel mailing list > nginx-devel at nginx.org > http://mailman.nginx.org/mailman/listinfo/nginx-devel From xeioex at nginx.com Fri Sep 6 12:47:38 2019 From: xeioex at nginx.com (Dmitry Volyntsev) Date: Fri, 06 Sep 2019 12:47:38 +0000 Subject: [njs] Style. Message-ID: details: https://hg.nginx.org/njs/rev/12da61ec4adb branches: changeset: 1157:12da61ec4adb user: Dmitry Volyntsev date: Fri Sep 06 15:47:12 2019 +0300 description: Style. diffstat: src/njs_dtoa.c | 14 +++++++------- src/njs_dtoa.h | 2 +- src/njs_dtoa_fixed.c | 8 ++++---- src/njs_shell.c | 7 ++++--- 4 files changed, 16 insertions(+), 15 deletions(-) diffs (131 lines): diff -r d46a332c9c4d -r 12da61ec4adb src/njs_dtoa.c --- a/src/njs_dtoa.c Tue Sep 03 17:31:45 2019 +0300 +++ b/src/njs_dtoa.c Fri Sep 06 15:47:12 2019 +0300 @@ -236,7 +236,7 @@ njs_grisu2(double value, char *start, in njs_inline size_t -njs_write_exponent(int exp, char* start) +njs_write_exponent(int exp, char *start) { char *p; size_t len; @@ -297,7 +297,7 @@ njs_prettify(char *start, size_t len, in memmove(&start[kk + 1], &start[kk], length - kk); start[kk] = '.'; - return (length + 1); + return length + 1; } else if (-6 < kk && kk <= 0) { @@ -313,7 +313,7 @@ njs_prettify(char *start, size_t len, in njs_memset(&start[2], '0', offset - 2); } - return (length + offset); + return length + offset; } else if (length == 1) { @@ -323,7 +323,7 @@ njs_prettify(char *start, size_t len, in size = njs_write_exponent(kk - 1, &start[2]); - return (size + 2); + return size + 2; } @@ -335,7 +335,7 @@ njs_prettify(char *start, size_t len, in size = njs_write_exponent(kk - 1, &start[length + 2]); - return (size + length + 2); + return size + length + 2; } @@ -354,7 +354,7 @@ njs_dtoa(double value, char *start) if (value == 0) { *p++ = '0'; - return (p - start); + return p - start; } if (signbit(value)) { @@ -367,5 +367,5 @@ njs_dtoa(double value, char *start) length = njs_prettify(p, length, dec_exp); - return (minus + length); + return minus + length; } diff -r d46a332c9c4d -r 12da61ec4adb src/njs_dtoa.h --- a/src/njs_dtoa.h Tue Sep 03 17:31:45 2019 +0300 +++ b/src/njs_dtoa.h Fri Sep 06 15:47:12 2019 +0300 @@ -7,6 +7,6 @@ #ifndef _NJS_DTOA_H_INCLUDED_ #define _NJS_DTOA_H_INCLUDED_ -NJS_EXPORT size_t njs_dtoa(double value, char* buffer); +NJS_EXPORT size_t njs_dtoa(double value, char *buffer); #endif /* _NJS_DTOA_H_INCLUDED_ */ diff -r d46a332c9c4d -r 12da61ec4adb src/njs_dtoa_fixed.c --- a/src/njs_dtoa_fixed.c Tue Sep 03 17:31:45 2019 +0300 +++ b/src/njs_dtoa_fixed.c Fri Sep 06 15:47:12 2019 +0300 @@ -134,9 +134,9 @@ njs_diyu128_bit_at(njs_diyu128_t v, njs_ static size_t njs_fill_digits32(uint32_t number, char *start, size_t length) { - char c; - size_t i, j, n; - njs_int_t digit; + char c; + size_t i, j, n; + njs_int_t digit; n = 0; @@ -268,7 +268,7 @@ njs_fill_fractionals(uint64_t fractional njs_diyu128_t fractionals128; /* - * 128 <= exponent <= 0. + * -128 <= exponent <= 0. * 0 <= fractionals * 2^exponent < 1. */ diff -r d46a332c9c4d -r 12da61ec4adb src/njs_shell.c --- a/src/njs_shell.c Tue Sep 03 17:31:45 2019 +0300 +++ b/src/njs_shell.c Fri Sep 06 15:47:12 2019 +0300 @@ -286,9 +286,9 @@ done: static njs_int_t -njs_get_options(njs_opts_t *opts, int argc, char** argv) +njs_get_options(njs_opts_t *opts, int argc, char **argv) { - char *p, **paths; + char *p, **paths; njs_int_t i, ret; static const char help[] = @@ -511,13 +511,14 @@ njs_process_file(njs_opts_t *opts, njs_v { int fd; char *file; - u_char buf[4096], *p, *end, *start; + u_char *p, *end, *start; size_t size; ssize_t n; njs_vm_t *vm; njs_int_t ret; njs_str_t source, script; struct stat sb; + u_char buf[4096]; file = opts->file; From mdounin at mdounin.ru Mon Sep 9 10:48:22 2019 From: mdounin at mdounin.ru (Maxim Dounin) Date: Mon, 9 Sep 2019 13:48:22 +0300 Subject: [PATCH] ngx_conf_file: "include ./" acts relative to currently parsed file In-Reply-To: <7fe74e4e34196c91a5c859ec5ac4e47e@outters.eu> References: <704100d8a8772a4bc2fa.1567058681@bas> <20190830140221.GK1877@mdounin.ru> <20190903143900.GA1877@mdounin.ru> <7fe74e4e34196c91a5c859ec5ac4e47e@outters.eu> Message-ID: <20190909104822.GN1877@mdounin.ru> Hello! On Tue, Sep 03, 2019 at 06:21:47PM +0200, Guillaume Outters wrote: > Le 2019-09-03 16:39, Maxim Dounin a ?crit : > > >> include @example.conf; > >> include example.conf local; > >> include example.conf -l; > >> include example.conf relative; > >> include example.conf from_there; > >> include example.conf nearby; > > > > I can't say I like either of the variants. > > ? Neither do I (although the "nearby", that I submitted in my last > patch that you perhaps haven't read yet, seems the most natural one). > > > Additionally, all variants with an additional explicit flags won't > > work in other cases where a configuration prefix is currently > > used, such as ssl_certificate or auth_basic_user_file. On the > > other hand, obviously enough it should be possible to resolve from > > the current included file all paths which are currently resolved > > from the configuration prefix. > > The big difference between include and all other directives is that > include does not do variable resolution. > > This allows the misuse of (request-time) variables to store absolute > paths, with things such as: > map $http_host $app_root { default /var/www/app1; } # Or a set, or > $document_root/.., whatever. > ssl_certificate $app_root/certs/thissite.pem; > > This makes an unnecessary variable evaluation at runtime, but as it's > technically possible, and as such I'm quite certain there exists a ton > of such configurations in the world (if only to reduce typing, and thus > copy-paste errors). > > But include has no such configurability. There were no way to use variables in ssl_certificate till nginx 1.15.9 released this year. There are no variables support in the ssl_stapling_file directive and various other directives, such as ssl_client_certificate, ssl_ecdh_param, and so on. Yet all these directives currently use consistent path resolution - from configuration prefix. And preserving this consistency is important. That's why don't like the idea to introduce yet another incompatible machanism to resolve file names. > Both of these advocate in favor of config-time variables: > - include would benefit of a bit of configurability: > webapps containing their nginx config could be dropped by a sysadm > wherever > he wanted, set a (config-time) $app_root, and the app would work out > of the > box by using $app_root to reference snippets indepently of where it > has been > put. > - other directives (that for now misuse request-time variables) would > have a > clear separation between config-time resolved parts and request-time > ones, > with a performance gain on the config-time ones > But this would require a bit of thinking (about where variables do get > stored > during the config reading, do they persist until the request-time, > attached to > each block with the last value they acquired during config reading, to > serve as > default values for the request-time variables, and so on). As http://nginx.org/en/docs/faq/variables_in_config.html mentions, it is more or less trivial to introduce configurations macros yourself. > In the meantime (and who can tell the meantime's duration?), the > "nearby" keyword > is a simple, semi-elegant placeholder for that. > (Hum? in fact, even with a way to "setenv $app_dir; include > $app_dir/nginx/snippet.conf;", > I think I would continue to just "include snippet.conf nearby;") See above. While it may be simple to implement (not really, as your patch misses at least a couple of module-specific "include" implementations), it is not that simple from consistency point of view. -- Maxim Dounin http://mdounin.ru/ From guillaume-nginx at outters.eu Mon Sep 9 16:14:47 2019 From: guillaume-nginx at outters.eu (Guillaume Outters) Date: Mon, 09 Sep 2019 18:14:47 +0200 Subject: [PATCH] ngx_conf_file: "include ./" acts relative to currently parsed file In-Reply-To: <20190909104822.GN1877@mdounin.ru> References: <704100d8a8772a4bc2fa.1567058681@bas> <20190830140221.GK1877@mdounin.ru> <20190903143900.GA1877@mdounin.ru> <7fe74e4e34196c91a5c859ec5ac4e47e@outters.eu> <20190909104822.GN1877@mdounin.ru> Message-ID: Maxim Dounin a ?crit: > There were no way to use variables in ssl_certificate till nginx > 1.15.9 released this year. There are no variables support in the > ssl_stapling_file directive and various other directives, such as > ssl_client_certificate, ssl_ecdh_param, and so on. Yet all these > directives currently use consistent path resolution - from > configuration prefix. And preserving this consistency is > important. That's why don't like the idea to introduce yet > another incompatible machanism to resolve file names. Thank you Maxim for your return on that; in parallel, I arrived to the same conclusion while reviewing a file mixing includes and ssl_certificates: mixing "short" nearby syntax (from my patch), and long syntax for non-nearby directives, immediately looked incoherent. I'll try to experiment with other syntaxes (? and implementations, which may take a bit of time); I really feel the need for a simpler, cleaner, more readable and less error-prone, syntax for self-contained (app + config) web folders. Evaluation cost at config compile time could be neglectable, and ease deployment by removing a templating step (every added step being a potential source of errors). -- Guillaume From ru at nginx.com Tue Sep 10 14:36:31 2019 From: ru at nginx.com (Ruslan Ermilov) Date: Tue, 10 Sep 2019 14:36:31 +0000 Subject: [nginx] HTTP/2: close connection on frames with self-dependency. Message-ID: details: https://hg.nginx.org/nginx/rev/29b2dc731503 branches: changeset: 7564:29b2dc731503 user: Ruslan Ermilov date: Tue Sep 10 15:33:37 2019 +0300 description: HTTP/2: close connection on frames with self-dependency. Don't waste server resources by sending RST_STREAM frames. Instead, reject HEADERS and PRIORITY frames with self-dependency by closing connection with PROTOCOL_ERROR. diffstat: src/http/v2/ngx_http_v2.c | 40 +++++++++------------------------------- 1 files changed, 9 insertions(+), 31 deletions(-) diffs (64 lines): diff -r a7e8f953408e -r 29b2dc731503 src/http/v2/ngx_http_v2.c --- a/src/http/v2/ngx_http_v2.c Wed Sep 04 13:33:51 2019 +0300 +++ b/src/http/v2/ngx_http_v2.c Tue Sep 10 15:33:37 2019 +0300 @@ -1107,6 +1107,14 @@ ngx_http_v2_state_headers(ngx_http_v2_co return ngx_http_v2_connection_error(h2c, NGX_HTTP_V2_PROTOCOL_ERROR); } + if (depend == h2c->state.sid) { + ngx_log_error(NGX_LOG_INFO, h2c->connection->log, 0, + "client sent HEADERS frame for stream %ui " + "with incorrect dependency", h2c->state.sid); + + return ngx_http_v2_connection_error(h2c, NGX_HTTP_V2_PROTOCOL_ERROR); + } + h2c->last_sid = h2c->state.sid; h2c->state.pool = ngx_create_pool(1024, h2c->connection->log); @@ -1114,15 +1122,6 @@ ngx_http_v2_state_headers(ngx_http_v2_co return ngx_http_v2_connection_error(h2c, NGX_HTTP_V2_INTERNAL_ERROR); } - if (depend == h2c->state.sid) { - ngx_log_error(NGX_LOG_INFO, h2c->connection->log, 0, - "client sent HEADERS frame for stream %ui " - "with incorrect dependency", h2c->state.sid); - - status = NGX_HTTP_V2_PROTOCOL_ERROR; - goto rst_stream; - } - h2scf = ngx_http_get_module_srv_conf(h2c->http_connection->conf_ctx, ngx_http_v2_module); @@ -1849,28 +1848,7 @@ ngx_http_v2_state_priority(ngx_http_v2_c "client sent PRIORITY frame for stream %ui " "with incorrect dependency", h2c->state.sid); - node = ngx_http_v2_get_node_by_id(h2c, h2c->state.sid, 0); - - if (node && node->stream) { - if (ngx_http_v2_terminate_stream(h2c, node->stream, - NGX_HTTP_V2_PROTOCOL_ERROR) - == NGX_ERROR) - { - return ngx_http_v2_connection_error(h2c, - NGX_HTTP_V2_INTERNAL_ERROR); - } - - } else { - if (ngx_http_v2_send_rst_stream(h2c, h2c->state.sid, - NGX_HTTP_V2_PROTOCOL_ERROR) - == NGX_ERROR) - { - return ngx_http_v2_connection_error(h2c, - NGX_HTTP_V2_INTERNAL_ERROR); - } - } - - return ngx_http_v2_state_complete(h2c, pos, end); + return ngx_http_v2_connection_error(h2c, NGX_HTTP_V2_PROTOCOL_ERROR); } node = ngx_http_v2_get_node_by_id(h2c, h2c->state.sid, 1); From ru at nginx.com Tue Sep 10 14:36:33 2019 From: ru at nginx.com (Ruslan Ermilov) Date: Tue, 10 Sep 2019 14:36:33 +0000 Subject: [nginx] HTTP/2: close connection on zero WINDOW_UPDATE. Message-ID: details: https://hg.nginx.org/nginx/rev/fd6dcc6f8a49 branches: changeset: 7565:fd6dcc6f8a49 user: Ruslan Ermilov date: Tue Sep 10 15:33:38 2019 +0300 description: HTTP/2: close connection on zero WINDOW_UPDATE. Don't waste server resources by sending RST_STREAM frames. Instead, reject WINDOW_UPDATE frames with invalid zero increment by closing connection with PROTOCOL_ERROR. diffstat: src/http/v2/ngx_http_v2.c | 38 ++++---------------------------------- 1 files changed, 4 insertions(+), 34 deletions(-) diffs (49 lines): diff -r 29b2dc731503 -r fd6dcc6f8a49 src/http/v2/ngx_http_v2.c --- a/src/http/v2/ngx_http_v2.c Tue Sep 10 15:33:37 2019 +0300 +++ b/src/http/v2/ngx_http_v2.c Tue Sep 10 15:33:38 2019 +0300 @@ -2231,41 +2231,11 @@ ngx_http_v2_state_window_update(ngx_http h2c->state.sid, window); if (window == 0) { - if (h2c->state.sid == 0) { - ngx_log_error(NGX_LOG_INFO, h2c->connection->log, 0, - "client sent WINDOW_UPDATE frame " - "with incorrect window increment 0"); - - return ngx_http_v2_connection_error(h2c, - NGX_HTTP_V2_PROTOCOL_ERROR); - } - ngx_log_error(NGX_LOG_INFO, h2c->connection->log, 0, - "client sent WINDOW_UPDATE frame for stream %ui " - "with incorrect window increment 0", h2c->state.sid); - - node = ngx_http_v2_get_node_by_id(h2c, h2c->state.sid, 0); - - if (node && node->stream) { - if (ngx_http_v2_terminate_stream(h2c, node->stream, - NGX_HTTP_V2_PROTOCOL_ERROR) - == NGX_ERROR) - { - return ngx_http_v2_connection_error(h2c, - NGX_HTTP_V2_INTERNAL_ERROR); - } - - } else { - if (ngx_http_v2_send_rst_stream(h2c, h2c->state.sid, - NGX_HTTP_V2_PROTOCOL_ERROR) - == NGX_ERROR) - { - return ngx_http_v2_connection_error(h2c, - NGX_HTTP_V2_INTERNAL_ERROR); - } - } - - return ngx_http_v2_state_complete(h2c, pos, end); + "client sent WINDOW_UPDATE frame " + "with incorrect window increment 0"); + + return ngx_http_v2_connection_error(h2c, NGX_HTTP_V2_PROTOCOL_ERROR); } if (h2c->state.sid) { From pluknet at nginx.com Tue Sep 10 14:40:23 2019 From: pluknet at nginx.com (Sergey Kandaurov) Date: Tue, 10 Sep 2019 14:40:23 +0000 Subject: [nginx] Resolver: fixed possible use-after-free while resolving PTR. Message-ID: details: https://hg.nginx.org/nginx/rev/571383f75a9a branches: changeset: 7566:571383f75a9a user: Sergey Kandaurov date: Tue Sep 10 15:42:34 2019 +0300 description: Resolver: fixed possible use-after-free while resolving PTR. Previously, if a response to the PTR request was cached, and ngx_resolver_dup() failed to allocate memory for the resulting name, then the original node was freed but left in expire_queue. A subsequent address resolving would end up in a use-after-free memory access of the node either in ngx_resolver_expire() or ngx_resolver_process_ptr(), when accessing it through expire_queue. The fix is to leave the resolver node intact. diffstat: src/core/ngx_resolver.c | 3 ++- 1 files changed, 2 insertions(+), 1 deletions(-) diffs (13 lines): diff -r fd6dcc6f8a49 -r 571383f75a9a src/core/ngx_resolver.c --- a/src/core/ngx_resolver.c Tue Sep 10 15:33:38 2019 +0300 +++ b/src/core/ngx_resolver.c Tue Sep 10 15:42:34 2019 +0300 @@ -972,7 +972,8 @@ ngx_resolve_addr(ngx_resolver_ctx_t *ctx name = ngx_resolver_dup(r, rn->name, rn->nlen); if (name == NULL) { - goto failed; + ngx_resolver_free(r, ctx); + return NGX_ERROR; } ctx->name.len = rn->nlen; From ecotoper at redhat.com Tue Sep 10 15:41:13 2019 From: ecotoper at redhat.com (Eloy Coto Pereiro) Date: Tue, 10 Sep 2019 17:41:13 +0200 Subject: proxy_cache min/max TTL Message-ID: Hi, Last week I was playing with Nginx proxy caching, and I have the following questions regarding some changes that I would like to introduce. At the moment, the cache always uses the `Cache-control: max-age=60` or other headers, but there is no way to say in a request that the maximum time is 15seconds for example. I would like to introduce a new parameter called `proxy_cache_max_ttl` and `proxy_cache_min_ttl` where the TTL can be defined in a variable and if the cache-control is bigger (or smaller) than that will be overwritten. If min is set and no header, the cache will be set to this value. The changes will happen on `ngx_http_upstream_process_header` function, but I'm not sure if this change will be accepted. Do these changes make sense for Nginx? Any reason why these changes will not accept? Can this be achieved in any other way? Regards. -------------- next part -------------- An HTML attachment was scrubbed... URL: From alexander.borisov at nginx.com Wed Sep 11 16:47:29 2019 From: alexander.borisov at nginx.com (Alexander Borisov) Date: Wed, 11 Sep 2019 16:47:29 +0000 Subject: [njs] Fixed handing of accessor descriptors in Object.freeze(). Message-ID: details: https://hg.nginx.org/njs/rev/bdeb67672e58 branches: changeset: 1158:bdeb67672e58 user: Alexander Borisov date: Wed Sep 11 19:46:30 2019 +0300 description: Fixed handing of accessor descriptors in Object.freeze(). This closes #211 issue on GitHub. diffstat: src/njs_object.c | 5 ++++- src/test/njs_unit_test.c | 11 +++++++++++ 2 files changed, 15 insertions(+), 1 deletions(-) diffs (36 lines): diff -r 12da61ec4adb -r bdeb67672e58 src/njs_object.c --- a/src/njs_object.c Fri Sep 06 15:47:12 2019 +0300 +++ b/src/njs_object.c Wed Sep 11 19:46:30 2019 +0300 @@ -1345,7 +1345,10 @@ njs_object_freeze(njs_vm_t *vm, njs_valu break; } - prop->writable = 0; + if (!njs_is_accessor_descriptor(prop)) { + prop->writable = 0; + } + prop->configurable = 0; } diff -r 12da61ec4adb -r bdeb67672e58 src/test/njs_unit_test.c --- a/src/test/njs_unit_test.c Fri Sep 06 15:47:12 2019 +0300 +++ b/src/test/njs_unit_test.c Wed Sep 11 19:46:30 2019 +0300 @@ -10849,6 +10849,17 @@ static njs_unit_test_t njs_test[] = { njs_str("var r = Object.freeze(new RegExp('')); r.a = 1"), njs_str("TypeError: Cannot add property \"a\", object is not extensible") }, + { njs_str("var o = Object.freeze({ get x() { return 10; } }); o.x"), + njs_str("10") }, + + { njs_str("var o = Object.freeze({ get x() { return 10; } });" + "Object.getOwnPropertyDescriptors(o).x.get"), + njs_str("[object Function]") }, + + { njs_str("var o = Object.freeze({ get x() { return 10; } });" + "Object.getOwnPropertyDescriptor(o, 'x').writable"), + njs_str("undefined") }, + { njs_str("Object.isFrozen({a:1})"), njs_str("false") }, From alexander.borisov at nginx.com Mon Sep 16 14:22:03 2019 From: alexander.borisov at nginx.com (Alexander Borisov) Date: Mon, 16 Sep 2019 14:22:03 +0000 Subject: [njs] Fixed String.prototype.replace() when first argument is not a string. Message-ID: details: https://hg.nginx.org/njs/rev/50cc68d71326 branches: changeset: 1159:50cc68d71326 user: Alexander Borisov date: Mon Sep 16 17:21:36 2019 +0300 description: Fixed String.prototype.replace() when first argument is not a string. This closes #208 issue on GitHub. diffstat: src/njs_string.c | 121 ++++++++++++++++++++++++++++++---------------- src/test/njs_unit_test.c | 3 + 2 files changed, 81 insertions(+), 43 deletions(-) diffs (317 lines): diff -r bdeb67672e58 -r 50cc68d71326 src/njs_string.c --- a/src/njs_string.c Wed Sep 11 19:46:30 2019 +0300 +++ b/src/njs_string.c Mon Sep 16 17:21:36 2019 +0300 @@ -78,16 +78,17 @@ static njs_int_t njs_string_match_multip njs_regexp_pattern_t *pattern); static njs_int_t njs_string_split_part_add(njs_vm_t *vm, njs_array_t *array, njs_utf8_t utf8, const u_char *start, size_t size); -static njs_int_t njs_string_replace_regexp(njs_vm_t *vm, njs_value_t *args, - njs_string_replace_t *r); +static njs_int_t njs_string_replace_regexp(njs_vm_t *vm, njs_value_t *this, + njs_value_t *regex, njs_string_replace_t *r); static njs_int_t njs_string_replace_regexp_function(njs_vm_t *vm, - njs_value_t *args, njs_string_replace_t *r, int *captures, njs_uint_t n); + njs_value_t *this, njs_value_t *regex, njs_string_replace_t *r, + int *captures, njs_uint_t n); static njs_int_t njs_string_replace_regexp_join(njs_vm_t *vm, njs_string_replace_t *r); -static njs_int_t njs_string_replace_search(njs_vm_t *vm, njs_value_t *args, - njs_string_replace_t *r); +static njs_int_t njs_string_replace_search(njs_vm_t *vm, njs_value_t *this, + njs_value_t *search, njs_string_replace_t *r); static njs_int_t njs_string_replace_search_function(njs_vm_t *vm, - njs_value_t *args, njs_string_replace_t *r); + njs_value_t *this, njs_value_t *search, njs_string_replace_t *r); static njs_int_t njs_string_replace_parse(njs_vm_t *vm, njs_string_replace_t *r, u_char *p, u_char *end, size_t size, njs_uint_t ncaptures); @@ -3067,15 +3068,34 @@ njs_string_prototype_replace(njs_vm_t *v u_char *p, *start, *end; njs_int_t ret; njs_uint_t ncaptures; + njs_value_t *this, *search, *replace; njs_regex_t *regex; njs_string_prop_t string; njs_string_replace_t *r, string_replace; + this = njs_arg(args, nargs, 0); + + if (njs_slow_path(njs_value_is_null_or_undefined(this))) { + njs_type_error(vm, "\"this\" argument cannot be " + "undefined or null value"); + return NJS_ERROR; + } + + if (!njs_is_string(this)) { + ret = njs_value_to_string(vm, this, this); + if (njs_slow_path(ret != NJS_OK)) { + return ret; + } + } + if (nargs == 1) { goto original; } - (void) njs_string_prop(&string, &args[0]); + search = njs_arg(args, nargs, 1); + replace = njs_arg(args, nargs, 2); + + (void) njs_string_prop(&string, this); if (string.size == 0) { goto original; @@ -3095,8 +3115,8 @@ njs_string_prototype_replace(njs_vm_t *v } } - if (njs_is_regexp(&args[1])) { - regex = &njs_regexp_pattern(&args[1])->regex[r->type]; + if (njs_is_regexp(search)) { + regex = &njs_regexp_pattern(search)->regex[r->type]; if (!njs_regex_is_valid(regex)) { goto original; @@ -3121,8 +3141,18 @@ njs_string_prototype_replace(njs_vm_t *v if (nargs == 2) { njs_string_replacement_copy(&r->part[1], &njs_string_undefined); - } else if (njs_is_string(&args[2])) { - njs_string_replacement_copy(&r->part[1], &args[2]); + } else if (njs_is_function(replace)) { + r->function = njs_function(replace); + + } else { + if (!njs_is_string(replace)) { + ret = njs_value_to_string(vm, replace, replace); + if (njs_slow_path(ret != NJS_OK)) { + return ret; + } + } + + njs_string_replacement_copy(&r->part[1], replace); start = r->part[1].start; @@ -3146,9 +3176,6 @@ njs_string_prototype_replace(njs_vm_t *v break; } } - - } else { - r->function = njs_function(&args[2]); } r->part[0].start = string.start; @@ -3161,21 +3188,28 @@ njs_string_prototype_replace(njs_vm_t *v return NJS_ERROR; } - return njs_string_replace_regexp(vm, args, r); + return njs_string_replace_regexp(vm, this, search, r); } - return njs_string_replace_search(vm, args, r); + if (!njs_is_string(search)) { + ret = njs_value_to_string(vm, search, search); + if (njs_slow_path(ret != NJS_OK)) { + return ret; + } + } + + return njs_string_replace_search(vm, this, search, r); original: - njs_string_copy(&vm->retval, &args[0]); + njs_string_copy(&vm->retval, this); return NJS_OK; } static njs_int_t -njs_string_replace_regexp(njs_vm_t *vm, njs_value_t *args, +njs_string_replace_regexp(njs_vm_t *vm, njs_value_t *this, njs_value_t *regex, njs_string_replace_t *r) { int *captures; @@ -3185,7 +3219,7 @@ njs_string_replace_regexp(njs_vm_t *vm, njs_regexp_pattern_t *pattern; njs_string_replace_part_t replace; - pattern = njs_regexp_pattern(&args[1]); + pattern = njs_regexp_pattern(regex); end = r->part[0].start + r->part[0].size; replace = r->part[1]; @@ -3257,7 +3291,7 @@ njs_string_replace_regexp(njs_vm_t *vm, } if (r->function != NULL) { - return njs_string_replace_regexp_function(vm, args, r, + return njs_string_replace_regexp_function(vm, this, regex, r, captures, ret); } @@ -3269,7 +3303,7 @@ njs_string_replace_regexp(njs_vm_t *vm, njs_set_invalid(&r->part[2].value); if (r->function != NULL) { - return njs_string_replace_regexp_function(vm, args, r, + return njs_string_replace_regexp_function(vm, this, regex, r, captures, ret); } @@ -3294,15 +3328,15 @@ njs_string_replace_regexp(njs_vm_t *vm, njs_arr_destroy(&r->parts); - njs_string_copy(&vm->retval, &args[0]); + njs_string_copy(&vm->retval, this); return NJS_OK; } static njs_int_t -njs_string_replace_regexp_function(njs_vm_t *vm, njs_value_t *args, - njs_string_replace_t *r, int *captures, njs_uint_t n) +njs_string_replace_regexp_function(njs_vm_t *vm, njs_value_t *this, + njs_value_t *regex, njs_string_replace_t *r, int *captures, njs_uint_t n) { u_char *start; size_t size, length; @@ -3343,7 +3377,7 @@ njs_string_replace_regexp_function(njs_v /* The whole string being examined. */ length = njs_string_calc_length(r->utf8, r->part[0].start, r->part[0].size); - (void) njs_string_prop(&string, &args[0]); + (void) njs_string_prop(&string, this); ret = njs_string_new(vm, &arguments[n + 2], string.start, string.size, length); @@ -3360,19 +3394,19 @@ njs_string_replace_regexp_function(njs_v return ret; } - (void) njs_string_prop(&string, &args[0]); + (void) njs_string_prop(&string, this); if (njs_is_string(&r->retval)) { njs_string_replacement_copy(&r->part[r->empty ? 0 : 1], &r->retval); - if (njs_regexp_pattern(&args[1])->global) { + if (njs_regexp_pattern(regex)->global) { r->part += 2; if (r->part[0].start > (string.start + string.size)) { return njs_string_replace_regexp_join(vm, r); } - return njs_string_replace_regexp(vm, args, r); + return njs_string_replace_regexp(vm, this, regex, r); } return njs_string_replace_regexp_join(vm, r); @@ -3397,26 +3431,26 @@ njs_string_replace_regexp_join(njs_vm_t static njs_int_t -njs_string_replace_search(njs_vm_t *vm, njs_value_t *args, +njs_string_replace_search(njs_vm_t *vm, njs_value_t *this, njs_value_t *search, njs_string_replace_t *r) { int captures[2]; u_char *p, *end; size_t size; njs_int_t ret; - njs_str_t search; - - njs_string_get(&args[1], &search); + njs_str_t string; + + njs_string_get(search, &string); p = r->part[0].start; - end = (p + r->part[0].size) - (search.length - 1); + end = (p + r->part[0].size) - (string.length - 1); while (p < end) { - if (memcmp(p, search.start, search.length) == 0) { + if (memcmp(p, string.start, string.length) == 0) { if (r->substitutions != NULL) { captures[0] = p - r->part[0].start; - captures[1] = captures[0] + search.length; + captures[1] = captures[0] + string.length; ret = njs_string_replace_substitute(vm, r, captures); if (njs_slow_path(ret != NJS_OK)) { @@ -3424,14 +3458,15 @@ njs_string_replace_search(njs_vm_t *vm, } } else { - r->part[2].start = p + search.length; + r->part[2].start = p + string.length; size = p - r->part[0].start; - r->part[2].size = r->part[0].size - size - search.length; + r->part[2].size = r->part[0].size - size - string.length; r->part[0].size = size; njs_set_invalid(&r->part[2].value); if (r->function != NULL) { - return njs_string_replace_search_function(vm, args, r); + return njs_string_replace_search_function(vm, this, search, + r); } } @@ -3446,15 +3481,15 @@ njs_string_replace_search(njs_vm_t *vm, } } - njs_string_copy(&vm->retval, &args[0]); + njs_string_copy(&vm->retval, this); return NJS_OK; } static njs_int_t -njs_string_replace_search_function(njs_vm_t *vm, njs_value_t *args, - njs_string_replace_t *r) +njs_string_replace_search_function(njs_vm_t *vm, njs_value_t *this, + njs_value_t *search, njs_string_replace_t *r) { njs_int_t ret; njs_value_t string; @@ -3465,13 +3500,13 @@ njs_string_replace_search_function(njs_v /* GC, args[0], args[1] */ /* Matched substring, it is the same as the args[1]. */ - arguments[1] = args[1]; + arguments[1] = *search; /* The offset of the matched substring. */ njs_set_number(&arguments[2], r->part[0].size); /* The whole string being examined. */ - arguments[3] = args[0]; + arguments[3] = *this; ret = njs_function_apply(vm, r->function, arguments, 4, &r->retval); diff -r bdeb67672e58 -r 50cc68d71326 src/test/njs_unit_test.c --- a/src/test/njs_unit_test.c Wed Sep 11 19:46:30 2019 +0300 +++ b/src/test/njs_unit_test.c Mon Sep 16 17:21:36 2019 +0300 @@ -6081,6 +6081,9 @@ static njs_unit_test_t njs_test[] = { njs_str("'ABC'.replace(/((A)B)/g, '($1|$&|$2)')"), njs_str("(AB|AB|A)C") }, + { njs_str("'undefined'.replace(void 0, 'x')"), + njs_str("x") }, + { njs_str("/]/"), njs_str("/\\]/") }, From pluknet at nginx.com Mon Sep 16 16:27:54 2019 From: pluknet at nginx.com (Sergey Kandaurov) Date: Mon, 16 Sep 2019 16:27:54 +0000 Subject: [nginx] SSL: fixed ssl_verify_client error message. Message-ID: details: https://hg.nginx.org/nginx/rev/ef7ee19776db branches: changeset: 7567:ef7ee19776db user: Sergey Kandaurov date: Mon Sep 16 19:26:42 2019 +0300 description: SSL: fixed ssl_verify_client error message. diffstat: src/http/modules/ngx_http_ssl_module.c | 2 +- src/mail/ngx_mail_ssl_module.c | 2 +- src/stream/ngx_stream_ssl_module.c | 2 +- 3 files changed, 3 insertions(+), 3 deletions(-) diffs (36 lines): diff -r 571383f75a9a -r ef7ee19776db src/http/modules/ngx_http_ssl_module.c --- a/src/http/modules/ngx_http_ssl_module.c Tue Sep 10 15:42:34 2019 +0300 +++ b/src/http/modules/ngx_http_ssl_module.c Mon Sep 16 19:26:42 2019 +0300 @@ -777,7 +777,7 @@ ngx_http_ssl_merge_srv_conf(ngx_conf_t * if (conf->client_certificate.len == 0 && conf->verify != 3) { ngx_log_error(NGX_LOG_EMERG, cf->log, 0, - "no ssl_client_certificate for ssl_client_verify"); + "no ssl_client_certificate for ssl_verify_client"); return NGX_CONF_ERROR; } diff -r 571383f75a9a -r ef7ee19776db src/mail/ngx_mail_ssl_module.c --- a/src/mail/ngx_mail_ssl_module.c Tue Sep 10 15:42:34 2019 +0300 +++ b/src/mail/ngx_mail_ssl_module.c Mon Sep 16 19:26:42 2019 +0300 @@ -388,7 +388,7 @@ ngx_mail_ssl_merge_conf(ngx_conf_t *cf, if (conf->client_certificate.len == 0 && conf->verify != 3) { ngx_log_error(NGX_LOG_EMERG, cf->log, 0, - "no ssl_client_certificate for ssl_client_verify"); + "no ssl_client_certificate for ssl_verify_client"); return NGX_CONF_ERROR; } diff -r 571383f75a9a -r ef7ee19776db src/stream/ngx_stream_ssl_module.c --- a/src/stream/ngx_stream_ssl_module.c Tue Sep 10 15:42:34 2019 +0300 +++ b/src/stream/ngx_stream_ssl_module.c Mon Sep 16 19:26:42 2019 +0300 @@ -745,7 +745,7 @@ ngx_stream_ssl_merge_conf(ngx_conf_t *cf if (conf->client_certificate.len == 0 && conf->verify != 3) { ngx_log_error(NGX_LOG_EMERG, cf->log, 0, - "no ssl_client_certificate for ssl_client_verify"); + "no ssl_client_certificate for ssl_verify_client"); return NGX_CONF_ERROR; } From alexander.borisov at nginx.com Tue Sep 17 06:21:24 2019 From: alexander.borisov at nginx.com (Alexander Borisov) Date: Tue, 17 Sep 2019 06:21:24 +0000 Subject: [njs] Fixed njs_value_index(). Message-ID: details: https://hg.nginx.org/njs/rev/d0d4fa8918ac branches: changeset: 1160:d0d4fa8918ac user: Alexander Borisov date: Tue Sep 17 09:20:24 2019 +0300 description: Fixed njs_value_index(). Previous, there were several issues with njs_values_hash_test(). 1) The function assumed string types of left and right values are identical. (If current value is a long string it assumed the second value is also a long string). Long vs short strings collision. 2) The function assumed if hashes are identical value length are equal. Long vs long string collision. The fix is to always check value sizes. In addition, for short strings njs_value_index() calculated hash value including padding bytes (which values are undefined). It could result in identical short strings (but with different padding bytes) treated as different strings. diffstat: src/njs_string.c | 48 +++++++++++++++++++++++++----------------------- src/test/njs_unit_test.c | 16 ++++++++++++++++ 2 files changed, 41 insertions(+), 23 deletions(-) diffs (123 lines): diff -r 50cc68d71326 -r d0d4fa8918ac src/njs_string.c --- a/src/njs_string.c Mon Sep 16 17:21:36 2019 +0300 +++ b/src/njs_string.c Tue Sep 17 09:20:24 2019 +0300 @@ -4723,23 +4723,22 @@ uri_error: static njs_int_t njs_values_hash_test(njs_lvlhsh_query_t *lhq, void *data) { - u_char *start; + njs_str_t string; njs_value_t *value; value = data; - if (lhq->key.length == sizeof(njs_value_t)) { - start = (u_char *) value; + if (njs_is_string(value)) { + njs_string_get(value, &string); } else { - /* - * Only primitive values are added into values_hash. - * If size != sizeof(njs_value_t) it is a long string. - */ - start = value->long_string.data->start; + string.start = (u_char *) value; + string.length = sizeof(njs_value_t); } - if (memcmp(lhq->key.start, start, lhq->key.length) == 0) { + if (lhq->key.length == string.length + && memcmp(lhq->key.start, string.start, string.length) == 0) + { return NJS_OK; } @@ -4768,21 +4767,28 @@ njs_value_index(njs_vm_t *vm, const njs_ u_char *start; uint32_t value_size, size, length; njs_int_t ret; + njs_str_t str; njs_bool_t long_string; njs_value_t *value; njs_string_t *string; njs_lvlhsh_t *values_hash; njs_lvlhsh_query_t lhq; - long_string = src->type == NJS_STRING - && src->short_string.size == NJS_STRING_LONG; - - if (long_string) { - size = src->long_string.size; - start = src->long_string.data->start; + long_string = 0; + value_size = sizeof(njs_value_t); + + if (njs_is_string(src)) { + njs_string_get(src, &str); + + size = (uint32_t) str.length; + start = str.start; + + if (src->short_string.size == NJS_STRING_LONG) { + long_string = 1; + } } else { - size = sizeof(njs_value_t); + size = value_size; start = (u_char *) src; } @@ -4798,22 +4804,18 @@ njs_value_index(njs_vm_t *vm, const njs_ value = lhq.value; } else { - value_size = 0; - if (long_string) { - /* Long string value is allocated together with string. */ - value_size = sizeof(njs_value_t) + sizeof(njs_string_t); - length = src->long_string.data->length; if (size != length && length > NJS_STRING_MAP_STRIDE) { size = njs_string_map_offset(size) + njs_string_map_size(length); } + + value_size += sizeof(njs_string_t) + size; } - value = njs_mp_align(vm->mem_pool, sizeof(njs_value_t), - value_size + size); + value = njs_mp_align(vm->mem_pool, sizeof(njs_value_t), value_size); if (njs_slow_path(value == NULL)) { return NJS_INDEX_NONE; } diff -r 50cc68d71326 -r d0d4fa8918ac src/test/njs_unit_test.c --- a/src/test/njs_unit_test.c Mon Sep 16 17:21:36 2019 +0300 +++ b/src/test/njs_unit_test.c Tue Sep 17 09:20:24 2019 +0300 @@ -9593,6 +9593,22 @@ static njs_unit_test_t njs_test[] = { njs_str("String.length"), njs_str("1") }, + /* values_hash long vs long string collision. */ + { njs_str("'XXXXXXXXXXXXXXXQWEEAB' + 'XXXXXXXXXXXXXXXZHGP'"), + njs_str("XXXXXXXXXXXXXXXQWEEABXXXXXXXXXXXXXXXZHGP") }, + + /* values_hash short vs long string collision. */ + { njs_str("'SHAAAB' + 'XXXXXXXXXXXXXXXUETBF'"), + njs_str("SHAAABXXXXXXXXXXXXXXXUETBF") }, + + /* values_hash long vs short string collision. */ + { njs_str("'XXXXXXXXXXXXXXXUETBF' + 'SHAAAB'"), + njs_str("XXXXXXXXXXXXXXXUETBFSHAAAB") }, + + /* values_hash short vs short string collision. */ + { njs_str("'XUBAAAB' + 'XGYXKY'"), + njs_str("XUBAAABXGYXKY") }, + { njs_str("String.__proto__ === Function.prototype"), njs_str("true") }, From alexander.borisov at nginx.com Tue Sep 17 08:30:12 2019 From: alexander.borisov at nginx.com (Alexander Borisov) Date: Tue, 17 Sep 2019 08:30:12 +0000 Subject: [njs] Fixed stack-use-after-scope in Array.prototype.map(). Message-ID: details: https://hg.nginx.org/njs/rev/1293f464dcc7 branches: changeset: 1161:1293f464dcc7 user: Alexander Borisov date: Tue Sep 17 11:29:10 2019 +0300 description: Fixed stack-use-after-scope in Array.prototype.map(). In the njs_array_iterator() an args.value is replaced to value on stack for non-object strings. diffstat: src/njs_array.c | 6 +++--- src/test/njs_unit_test.c | 3 +++ 2 files changed, 6 insertions(+), 3 deletions(-) diffs (32 lines): diff -r d0d4fa8918ac -r 1293f464dcc7 src/njs_array.c --- a/src/njs_array.c Tue Sep 17 09:20:24 2019 +0300 +++ b/src/njs_array.c Tue Sep 17 11:29:10 2019 +0300 @@ -1917,12 +1917,12 @@ njs_array_prototype_map(njs_vm_t *vm, nj return ret; } - if (njs_is_array(iargs.value) - && njs_object_hash_is_empty(iargs.value)) + if (njs_is_array(&args[0]) + && njs_object_hash_is_empty(&args[0])) { array = iargs.array; - for (i = njs_array_len(iargs.value); i < length; i++) { + for (i = njs_array_len(&args[0]); i < length; i++) { njs_set_invalid(&array->start[i]); } } diff -r d0d4fa8918ac -r 1293f464dcc7 src/test/njs_unit_test.c --- a/src/test/njs_unit_test.c Tue Sep 17 09:20:24 2019 +0300 +++ b/src/test/njs_unit_test.c Tue Sep 17 11:29:10 2019 +0300 @@ -4506,6 +4506,9 @@ static njs_unit_test_t njs_test[] = ".every(x => x === true)"), njs_str("true") }, + { njs_str("Array.prototype.map.call('abcdef', (val, idx, obj) => {return val === 100})"), + njs_str("false,false,false,false,false,false") }, + { njs_str("var a = [];" "a.reduce(function(p, v, i, a) { return p + v })"), njs_str("TypeError: invalid index") }, From mdounin at mdounin.ru Wed Sep 18 18:42:42 2019 From: mdounin at mdounin.ru (Maxim Dounin) Date: Wed, 18 Sep 2019 18:42:42 +0000 Subject: [nginx] HTTP/2: switched back to RST_STREAM with NO_ERROR. Message-ID: details: https://hg.nginx.org/nginx/rev/2e61e4b6bcd9 branches: changeset: 7568:2e61e4b6bcd9 user: Maxim Dounin date: Wed Sep 18 20:28:09 2019 +0300 description: HTTP/2: switched back to RST_STREAM with NO_ERROR. In 8df664ebe037, we've switched to maximizing stream window instead of sending RST_STREAM. Since then handling of RST_STREAM with NO_ERROR was fixed at least in Chrome, hence we switch back to using RST_STREAM. This allows more effective rejecting of large bodies, and also minimizes non-payload traffic to be accounted in the next patch. diffstat: src/http/v2/ngx_http_v2.c | 22 ---------------------- 1 files changed, 0 insertions(+), 22 deletions(-) diffs (37 lines): diff --git a/src/http/v2/ngx_http_v2.c b/src/http/v2/ngx_http_v2.c --- a/src/http/v2/ngx_http_v2.c +++ b/src/http/v2/ngx_http_v2.c @@ -4284,33 +4284,11 @@ ngx_http_v2_close_stream(ngx_http_v2_str } } else if (!stream->in_closed) { -#if 0 if (ngx_http_v2_send_rst_stream(h2c, node->id, NGX_HTTP_V2_NO_ERROR) != NGX_OK) { h2c->connection->error = 1; } -#else - /* - * At the time of writing at least the latest versions of Chrome - * do not properly handle RST_STREAM with NO_ERROR status. - * - * See: https://bugs.chromium.org/p/chromium/issues/detail?id=603182 - * - * As a workaround, the stream window is maximized before closing - * the stream. This allows a client to send up to 2 GB of data - * before getting blocked on flow control. - */ - - if (stream->recv_window < NGX_HTTP_V2_MAX_WINDOW - && ngx_http_v2_send_window_update(h2c, node->id, - NGX_HTTP_V2_MAX_WINDOW - - stream->recv_window) - != NGX_OK) - { - h2c->connection->error = 1; - } -#endif } } From mdounin at mdounin.ru Wed Sep 18 18:42:43 2019 From: mdounin at mdounin.ru (Maxim Dounin) Date: Wed, 18 Sep 2019 18:42:43 +0000 Subject: [nginx] HTTP/2: traffic-based flood detection. Message-ID: details: https://hg.nginx.org/nginx/rev/80359395b345 branches: changeset: 7569:80359395b345 user: Maxim Dounin date: Wed Sep 18 20:28:12 2019 +0300 description: HTTP/2: traffic-based flood detection. With this patch, all traffic over an HTTP/2 connection is counted in the h2c->total_bytes field, and payload traffic is counted in the h2c->payload_bytes field. As long as total traffic is many times larger than payload traffic, we consider this to be a flood. diffstat: src/http/v2/ngx_http_v2.c | 21 +++++++++++++++++++-- src/http/v2/ngx_http_v2.h | 3 +++ src/http/v2/ngx_http_v2_filter_module.c | 13 ++++++++++++- 3 files changed, 34 insertions(+), 3 deletions(-) diffs (130 lines): diff --git a/src/http/v2/ngx_http_v2.c b/src/http/v2/ngx_http_v2.c --- a/src/http/v2/ngx_http_v2.c +++ b/src/http/v2/ngx_http_v2.c @@ -419,6 +419,14 @@ ngx_http_v2_read_handler(ngx_event_t *re } while (p != end); + h2c->total_bytes += n; + + if (h2c->total_bytes / 8 > h2c->payload_bytes + 1048576) { + ngx_log_error(NGX_LOG_INFO, c->log, 0, "http2 flood detected"); + ngx_http_v2_finalize_connection(h2c, NGX_HTTP_V2_NO_ERROR); + return; + } + } while (rev->ready); if (ngx_handle_read_event(rev, 0) != NGX_OK) { @@ -963,6 +971,8 @@ ngx_http_v2_state_read_data(ngx_http_v2_ stream->in_closed = h2c->state.flags & NGX_HTTP_V2_END_STREAM_FLAG; } + h2c->payload_bytes += size; + if (r->request_body) { rc = ngx_http_v2_process_request_body(r, pos, size, stream->in_closed); @@ -2909,9 +2919,9 @@ ngx_http_v2_get_frame(ngx_http_v2_connec "requested control frame is too large: %uz", length); return NULL; } +#endif frame->length = length; -#endif buf->last = ngx_http_v2_write_len_and_type(buf->pos, length, type); @@ -2938,6 +2948,8 @@ ngx_http_v2_frame_handler(ngx_http_v2_co frame->next = h2c->free_frames; h2c->free_frames = frame; + h2c->total_bytes += NGX_HTTP_V2_FRAME_HEADER_SIZE + frame->length; + return NGX_OK; } @@ -3723,7 +3735,8 @@ ngx_http_v2_construct_cookie_header(ngx_ static void ngx_http_v2_run_request(ngx_http_request_t *r) { - ngx_connection_t *fc; + ngx_connection_t *fc; + ngx_http_v2_connection_t *h2c; fc = r->connection; @@ -3755,6 +3768,10 @@ ngx_http_v2_run_request(ngx_http_request r->headers_in.chunked = 1; } + h2c = r->stream->connection; + + h2c->payload_bytes += r->request_length; + ngx_http_process_request(r); failed: diff --git a/src/http/v2/ngx_http_v2.h b/src/http/v2/ngx_http_v2.h --- a/src/http/v2/ngx_http_v2.h +++ b/src/http/v2/ngx_http_v2.h @@ -119,6 +119,9 @@ struct ngx_http_v2_connection_s { ngx_connection_t *connection; ngx_http_connection_t *http_connection; + off_t total_bytes; + off_t payload_bytes; + ngx_uint_t processing; ngx_uint_t frames; ngx_uint_t idle; diff --git a/src/http/v2/ngx_http_v2_filter_module.c b/src/http/v2/ngx_http_v2_filter_module.c --- a/src/http/v2/ngx_http_v2_filter_module.c +++ b/src/http/v2/ngx_http_v2_filter_module.c @@ -1877,6 +1877,8 @@ ngx_http_v2_headers_frame_handler(ngx_ht stream->request->header_size += NGX_HTTP_V2_FRAME_HEADER_SIZE + frame->length; + h2c->payload_bytes += frame->length; + ngx_http_v2_handle_frame(stream, frame); ngx_http_v2_handle_stream(h2c, stream); @@ -1931,6 +1933,8 @@ ngx_http_v2_push_frame_handler(ngx_http_ stream->request->header_size += NGX_HTTP_V2_FRAME_HEADER_SIZE + frame->length; + h2c->payload_bytes += frame->length; + ngx_http_v2_handle_frame(stream, frame); ngx_http_v2_handle_stream(h2c, stream); @@ -2024,6 +2028,8 @@ done: stream->request->header_size += NGX_HTTP_V2_FRAME_HEADER_SIZE; + h2c->payload_bytes += frame->length; + ngx_http_v2_handle_frame(stream, frame); ngx_http_v2_handle_stream(h2c, stream); @@ -2036,12 +2042,17 @@ static ngx_inline void ngx_http_v2_handle_frame(ngx_http_v2_stream_t *stream, ngx_http_v2_out_frame_t *frame) { - ngx_http_request_t *r; + ngx_http_request_t *r; + ngx_http_v2_connection_t *h2c; r = stream->request; r->connection->sent += NGX_HTTP_V2_FRAME_HEADER_SIZE + frame->length; + h2c = stream->connection; + + h2c->total_bytes += NGX_HTTP_V2_FRAME_HEADER_SIZE + frame->length; + if (frame->fin) { stream->out_closed = 1; } From alexander.borisov at nginx.com Thu Sep 19 07:20:13 2019 From: alexander.borisov at nginx.com (Alexander Borisov) Date: Thu, 19 Sep 2019 07:20:13 +0000 Subject: [njs] Added njs_value_property_delete(). Message-ID: details: https://hg.nginx.org/njs/rev/a07722caaee3 branches: changeset: 1163:a07722caaee3 user: Alexander Borisov date: Thu Sep 19 10:19:01 2019 +0300 description: Added njs_value_property_delete(). diffstat: src/njs_value.c | 65 +++++++++++++++++++++++++++++++++++++++++++ src/njs_value.h | 2 + src/njs_vmcode.c | 83 ++++--------------------------------------------------- 3 files changed, 74 insertions(+), 76 deletions(-) diffs (191 lines): diff -r e0f6fc8d0a3c -r a07722caaee3 src/njs_value.c --- a/src/njs_value.c Thu Sep 19 10:19:00 2019 +0300 +++ b/src/njs_value.c Thu Sep 19 10:19:01 2019 +0300 @@ -1133,3 +1133,68 @@ found: return NJS_OK; } + + +njs_int_t +njs_value_property_delete(njs_vm_t *vm, njs_value_t *value, njs_value_t *key, + njs_value_t *removed) +{ + njs_int_t ret; + njs_object_prop_t *prop; + njs_property_query_t pq; + + njs_property_query_init(&pq, NJS_PROPERTY_QUERY_DELETE, 1); + + ret = njs_property_query(vm, &pq, value, key); + if (njs_slow_path(ret != NJS_OK)) { + return ret; + } + + prop = pq.lhq.value; + + if (njs_slow_path(!prop->configurable)) { + njs_type_error(vm, "Cannot delete property \"%V\" of %s", + &pq.lhq.key, njs_type_string(value->type)); + return NJS_ERROR; + } + + switch (prop->type) { + case NJS_PROPERTY_HANDLER: + if (njs_is_external(value)) { + ret = prop->value.data.u.prop_handler(vm, value, NULL, NULL); + if (njs_slow_path(ret != NJS_OK)) { + return NJS_ERROR; + } + + return NJS_OK; + } + + /* Fall through. */ + + case NJS_PROPERTY: + break; + + case NJS_PROPERTY_REF: + if (removed != NULL) { + *removed = *prop->value.data.u.value; + } + + njs_set_invalid(prop->value.data.u.value); + return NJS_OK; + + default: + njs_internal_error(vm, "unexpected property type \"%s\" " + "while deleting", njs_prop_type_string(prop->type)); + return NJS_ERROR; + } + + /* GC: release value. */ + if (removed != NULL) { + *removed = prop->value; + } + + prop->type = NJS_WHITEOUT; + njs_set_invalid(&prop->value); + + return NJS_OK; +} diff -r e0f6fc8d0a3c -r a07722caaee3 src/njs_value.h --- a/src/njs_value.h Thu Sep 19 10:19:00 2019 +0300 +++ b/src/njs_value.h Thu Sep 19 10:19:01 2019 +0300 @@ -840,6 +840,8 @@ njs_int_t njs_value_property(njs_vm_t *v njs_value_t *key, njs_value_t *retval); njs_int_t njs_value_property_set(njs_vm_t *vm, njs_value_t *value, njs_value_t *key, njs_value_t *setval); +njs_int_t njs_value_property_delete(njs_vm_t *vm, njs_value_t *value, + njs_value_t *key, njs_value_t *removed); njs_inline njs_int_t diff -r e0f6fc8d0a3c -r a07722caaee3 src/njs_vmcode.c --- a/src/njs_vmcode.c Thu Sep 19 10:19:00 2019 +0300 +++ b/src/njs_vmcode.c Thu Sep 19 10:19:01 2019 +0300 @@ -29,8 +29,6 @@ static njs_jump_off_t njs_vmcode_proto_i njs_value_t *key, njs_value_t *retval); static njs_jump_off_t njs_vmcode_property_in(njs_vm_t *vm, njs_value_t *value, njs_value_t *key); -static njs_jump_off_t njs_vmcode_property_delete(njs_vm_t *vm, - njs_value_t *value, njs_value_t *key); static njs_jump_off_t njs_vmcode_property_foreach(njs_vm_t *vm, njs_value_t *object, njs_value_t *invld, u_char *pc); static njs_jump_off_t njs_vmcode_property_next(njs_vm_t *vm, @@ -453,7 +451,13 @@ next: break; case NJS_VMCODE_PROPERTY_DELETE: - ret = njs_vmcode_property_delete(vm, value1, value2); + ret = njs_value_property_delete(vm, value1, value2, NULL); + if (njs_fast_path(ret != NJS_ERROR)) { + vm->retval = njs_value_true; + + ret = sizeof(njs_vmcode_3addr_t); + } + break; case NJS_VMCODE_PROPERTY_FOREACH: @@ -1280,79 +1284,6 @@ njs_vmcode_property_in(njs_vm_t *vm, njs static njs_jump_off_t -njs_vmcode_property_delete(njs_vm_t *vm, njs_value_t *value, njs_value_t *key) -{ - njs_jump_off_t ret; - njs_object_prop_t *prop; - njs_property_query_t pq; - - njs_property_query_init(&pq, NJS_PROPERTY_QUERY_DELETE, 1); - - ret = njs_property_query(vm, &pq, value, key); - - switch (ret) { - - case NJS_OK: - prop = pq.lhq.value; - - if (njs_slow_path(!prop->configurable)) { - njs_type_error(vm, "Cannot delete property \"%V\" of %s", - &pq.lhq.key, njs_type_string(value->type)); - return NJS_ERROR; - } - - switch (prop->type) { - case NJS_PROPERTY_HANDLER: - if (njs_is_external(value)) { - ret = prop->value.data.u.prop_handler(vm, value, NULL, NULL); - if (njs_slow_path(ret != NJS_OK)) { - return NJS_ERROR; - } - - goto done; - } - - /* Fall through. */ - - case NJS_PROPERTY: - break; - - case NJS_PROPERTY_REF: - njs_set_invalid(prop->value.data.u.value); - goto done; - - default: - njs_internal_error(vm, "unexpected property type \"%s\" " - "while deleting", - njs_prop_type_string(prop->type)); - - return NJS_ERROR; - } - - /* GC: release value. */ - prop->type = NJS_WHITEOUT; - njs_set_invalid(&prop->value); - - break; - - case NJS_DECLINED: - break; - - case NJS_ERROR: - default: - - return ret; - } - -done: - - vm->retval = njs_value_true; - - return sizeof(njs_vmcode_3addr_t); -} - - -static njs_jump_off_t njs_vmcode_property_foreach(njs_vm_t *vm, njs_value_t *object, njs_value_t *invld, u_char *pc) { From alexander.borisov at nginx.com Thu Sep 19 07:20:12 2019 From: alexander.borisov at nginx.com (Alexander Borisov) Date: Thu, 19 Sep 2019 07:20:12 +0000 Subject: [njs] Fixed Array prototype functions according to the specification. Message-ID: details: https://hg.nginx.org/njs/rev/e0f6fc8d0a3c branches: changeset: 1162:e0f6fc8d0a3c user: Alexander Borisov date: Thu Sep 19 10:19:00 2019 +0300 description: Fixed Array prototype functions according to the specification. The following fuctions were fixed: includes, indexOf, lastIndexOf, reduceRight. diffstat: src/njs_array.c | 906 ++++++++++++++++++++++++++++------------------ src/test/njs_unit_test.c | 60 ++- 2 files changed, 610 insertions(+), 356 deletions(-) diffs (truncated from 1228 to 1000 lines): diff -r 1293f464dcc7 -r e0f6fc8d0a3c src/njs_array.c --- a/src/njs_array.c Tue Sep 17 11:29:10 2019 +0300 +++ b/src/njs_array.c Thu Sep 19 10:19:00 2019 +0300 @@ -9,11 +9,14 @@ typedef struct { - njs_function_t *function; - njs_value_t *this_arg; - njs_value_t *value; - - njs_array_t *array; + njs_function_t *function; + njs_value_t *argument; + njs_value_t *value; + + njs_array_t *array; + + uint32_t from; + uint32_t to; } njs_array_iterator_args_t; @@ -1009,6 +1012,278 @@ njs_array_prototype_join(njs_vm_t *vm, n } +njs_inline njs_int_t +njs_array_iterator(njs_vm_t *vm, njs_array_iterator_args_t *args, + njs_array_iterator_handler_t handler) +{ + uint32_t length, i, from, to; + njs_int_t ret; + njs_value_t *entry, *value, character, index, string_obj, prop; + njs_object_t *object; + const u_char *p, *end, *pos; + njs_string_prop_t string_prop; + + value = args->value; + from = args->from; + to = args->to; + + if (njs_is_array(value)) { + if (njs_slow_path(!njs_object_hash_is_empty(value))) { + goto process_object; + } + + for (i = from; i < to; i++) { + entry = &njs_array_start(value)[i]; + + ret = handler(vm, args, entry, i); + if (njs_slow_path(ret != NJS_OK)) { + if (ret > 0) { + return NJS_DECLINED; + } + + return NJS_ERROR; + } + + to = njs_min(to, njs_array_len(value)); + } + + return NJS_OK; + } + + if (njs_is_string(value) || njs_is_object_string(value)) { + + if (njs_is_string(value)) { + object = njs_object_value_alloc(vm, value, NJS_STRING); + if (njs_slow_path(object == NULL)) { + return NJS_ERROR; + } + + njs_set_type_object(&string_obj, object, NJS_OBJECT_STRING); + + args->value = &string_obj; + } + else { + value = njs_object_value(value); + } + + length = (uint32_t) njs_string_prop(&string_prop, value); + + p = string_prop.start; + end = p + string_prop.size; + + if (length == string_prop.size) { + /* Byte or ASCII string. */ + + for (i = from; i < to; i++) { + /* This cannot fail. */ + (void) njs_string_new(vm, &character, p + i, 1, 1); + + ret = handler(vm, args, &character, i); + if (njs_slow_path(ret != NJS_OK)) { + if (ret > 0) { + return NJS_DECLINED; + } + + return NJS_ERROR; + } + } + + } else { + /* UTF-8 string. */ + + for (i = from; i < to; i++) { + pos = njs_utf8_next(p, end); + + /* This cannot fail. */ + (void) njs_string_new(vm, &character, p, pos - p, 1); + + ret = handler(vm, args, &character, i); + if (njs_slow_path(ret != NJS_OK)) { + if (ret > 0) { + return NJS_DECLINED; + } + + return NJS_ERROR; + } + + p = pos; + } + } + + return NJS_OK; + } + + if (!njs_is_object(value)) { + return NJS_OK; + } + +process_object: + + for (i = from; i < to; i++) { + njs_uint32_to_string(&index, i); + + ret = njs_value_property(vm, value, &index, &prop); + if (njs_slow_path(ret == NJS_ERROR)) { + return ret; + } + + if (ret != NJS_DECLINED) { + ret = handler(vm, args, &prop, i); + if (njs_slow_path(ret != NJS_OK)) { + if (ret > 0) { + return NJS_DECLINED; + } + + return NJS_ERROR; + } + } + } + + return NJS_OK; +} + + +njs_inline njs_int_t +njs_array_reverse_iterator(njs_vm_t *vm, njs_array_iterator_args_t *args, + njs_array_iterator_handler_t handler) +{ + uint32_t i, from, to, length; + njs_int_t ret; + njs_value_t *entry, *value, character, index, string_obj, prop; + njs_object_t *object; + const u_char *p, *end, *pos; + njs_string_prop_t string_prop; + + value = args->value; + from = args->from; + to = args->to; + + if (njs_is_array(value)) { + if (njs_slow_path(!njs_object_hash_is_empty(value))) { + goto process_object; + } + + i = from + 1; + + while (i-- > to) { + entry = &njs_array_start(value)[i]; + + ret = handler(vm, args, entry, i); + if (njs_slow_path(ret != NJS_OK)) { + if (ret > 0) { + return NJS_DECLINED; + } + + return NJS_ERROR; + } + } + + return NJS_OK; + } + + if (njs_is_string(value) || njs_is_object_string(value)) { + + if (njs_is_string(value)) { + object = njs_object_value_alloc(vm, value, NJS_STRING); + if (njs_slow_path(object == NULL)) { + return NJS_ERROR; + } + + njs_set_type_object(&string_obj, object, NJS_OBJECT_STRING); + + args->value = &string_obj; + } + else { + value = njs_object_value(value); + } + + length = (uint32_t) njs_string_prop(&string_prop, value); + end = string_prop.start + string_prop.size; + + if (length == string_prop.size) { + /* Byte or ASCII string. */ + + p = string_prop.start + from; + + i = from + 1; + + while (i-- > to) { + /* This cannot fail. */ + (void) njs_string_new(vm, &character, p, 1, 1); + + ret = handler(vm, args, &character, i); + if (njs_slow_path(ret != NJS_OK)) { + if (ret > 0) { + return NJS_DECLINED; + } + + return NJS_ERROR; + } + + p--; + } + + } else { + /* UTF-8 string. */ + + p = njs_string_offset(string_prop.start, end, from + 1); + + i = from + 1; + + while (i-- > to) { + pos = njs_utf8_prev(p); + + /* This cannot fail. */ + (void) njs_string_new(vm, &character, pos, p - pos , 1); + + ret = handler(vm, args, &character, i); + if (njs_slow_path(ret != NJS_OK)) { + if (ret > 0) { + return NJS_DECLINED; + } + + return NJS_ERROR; + } + + p = pos; + } + } + + return NJS_OK; + } + + if (!njs_is_object(value)) { + return NJS_OK; + } + +process_object: + + i = from + 1; + + while (i-- > to) { + njs_uint32_to_string(&index, i); + + ret = njs_value_property(vm, value, &index, &prop); + if (njs_slow_path(ret == NJS_ERROR)) { + return ret; + } + + if (ret != NJS_DECLINED) { + ret = handler(vm, args, &prop, i); + if (njs_slow_path(ret != NJS_OK)) { + if (ret > 0) { + return NJS_DECLINED; + } + + return NJS_ERROR; + } + } + } + + return NJS_OK; +} + + static njs_int_t njs_array_prototype_concat(njs_vm_t *vm, njs_value_t *args, njs_uint_t nargs, njs_index_t unused) @@ -1069,62 +1344,68 @@ njs_array_copy(njs_value_t *dst, njs_val static njs_int_t +njs_array_handler_index_of(njs_vm_t *vm, njs_array_iterator_args_t *args, + njs_value_t *entry, uint32_t n) +{ + if (njs_values_strict_equal(args->argument, entry)) { + njs_set_number(&vm->retval, n); + + return 1; + } + + return NJS_OK; +} + + +static njs_int_t njs_array_prototype_index_of(njs_vm_t *vm, njs_value_t *args, njs_uint_t nargs, njs_index_t unused) { - njs_int_t i, index, length; - njs_value_t *value, *start; - njs_array_t *array; - - index = -1; - - if (nargs < 2 || !njs_is_array(&args[0])) { - goto done; - } - - array = njs_array(&args[0]); - length = array->length; - - if (length == 0) { - goto done; + int64_t from; + uint32_t length; + njs_int_t ret; + njs_array_iterator_args_t iargs; + + if (njs_slow_path(njs_is_null_or_undefined(njs_arg(args, nargs, 0)))) { + njs_type_error(vm, "unexpected iterator arguments"); + return NJS_ERROR; } - i = 0; - - if (nargs > 2) { - i = njs_number(&args[2]); - - if (i >= length) { - goto done; - } - - if (i < 0) { - i += length; - - if (i < 0) { - i = 0; - } + iargs.value = njs_argument(args, 0); + iargs.argument = njs_arg(args, nargs, 1); + + ret = njs_value_length(vm, iargs.value, &length); + if (njs_slow_path(ret != NJS_OK)) { + return ret; + } + + from = njs_primitive_value_to_integer(njs_arg(args, nargs, 2)); + + if (length == 0 || from >= (int64_t) length) { + goto not_found; + } + + if (from < 0) { + from = length + from; + + if (from < 0) { + from = 0; } } - value = &args[1]; - start = array->start; - - do { - if (njs_values_strict_equal(value, &start[i])) { - index = i; - break; - } - - i++; - - } while (i < length); - -done: - - njs_set_number(&vm->retval, index); - - return NJS_OK; + iargs.from = (uint32_t) from; + iargs.to = length; + + ret = njs_array_iterator(vm, &iargs, njs_array_handler_index_of); + if (njs_fast_path(ret == NJS_DECLINED)) { + return NJS_OK; + } + +not_found: + + njs_set_number(&vm->retval, -1); + + return ret; } @@ -1132,60 +1413,85 @@ static njs_int_t njs_array_prototype_last_index_of(njs_vm_t *vm, njs_value_t *args, njs_uint_t nargs, njs_index_t unused) { - njs_int_t k, n, index, length; - njs_value_t *start; - njs_array_t *array; - njs_value_t *this, *value; - - index = -1; - - this = njs_arg(args, nargs, 0); - - if (!njs_is_array(this)) { - goto done; + int64_t from; + uint32_t length; + njs_int_t ret; + njs_array_iterator_args_t iargs; + + if (njs_slow_path(njs_is_null_or_undefined(njs_arg(args, nargs, 0)))) { + njs_type_error(vm, "unexpected iterator arguments"); + return NJS_ERROR; } - array = njs_array(this); - length = array->length; + iargs.value = njs_argument(args, 0); + iargs.argument = njs_arg(args, nargs, 1); + + ret = njs_value_length(vm, iargs.value, &length); + if (njs_slow_path(ret != NJS_OK)) { + return ret; + } if (length == 0) { - goto done; + goto not_found; } if (nargs > 2) { - n = njs_primitive_value_to_integer(njs_argument(args, 2)); + from = njs_primitive_value_to_integer(njs_arg(args, nargs, 2)); } else { - n = length - 1; + from = length - 1; } - if (n >= 0) { - k = njs_min(n, length - 1); - - } else { - k = n + length; - - if (k < 0) { - goto done; + if (from >= 0) { + from = njs_min(from, length - 1); + + } else if (from < 0) { + from += length; + + if (from <= 0) { + goto not_found; } } - value = njs_arg(args, nargs, 1); - start = array->start; - - do { - if (njs_values_strict_equal(value, &start[k])) { - index = k; - break; - } - - k--; - - } while (k >= 0); - -done: - - njs_set_number(&vm->retval, index); + iargs.from = from; + iargs.to = 0; + + ret = njs_array_reverse_iterator(vm, &iargs, njs_array_handler_index_of); + if (njs_fast_path(ret == NJS_DECLINED)) { + return NJS_OK; + } + +not_found: + + njs_set_number(&vm->retval, -1); + + return ret; +} + + +static njs_int_t +njs_array_handler_includes(njs_vm_t *vm, njs_array_iterator_args_t *args, + njs_value_t *entry, uint32_t n) +{ + if (njs_values_strict_equal(args->argument, entry)) { + njs_set_true(&vm->retval); + + return 1; + } + + return NJS_OK; +} + + +static njs_int_t +njs_array_handler_includes_nan(njs_vm_t *vm, njs_array_iterator_args_t *args, + njs_value_t *entry, uint32_t n) +{ + if (njs_is_numeric(entry) && isnan(njs_number(entry))) { + njs_set_true(&vm->retval); + + return 1; + } return NJS_OK; } @@ -1195,74 +1501,57 @@ static njs_int_t njs_array_prototype_includes(njs_vm_t *vm, njs_value_t *args, njs_uint_t nargs, njs_index_t unused) { - njs_int_t i, length; - njs_value_t *value, *start; - njs_array_t *array; - const njs_value_t *retval; - - retval = &njs_value_false; - - if (nargs < 2 || !njs_is_array(&args[0])) { - goto done; + int64_t from; + uint32_t length; + njs_int_t ret; + njs_array_iterator_args_t iargs; + + if (njs_slow_path(njs_is_null_or_undefined(njs_arg(args, nargs, 0)))) { + njs_type_error(vm, "unexpected iterator arguments"); + return NJS_ERROR; } - array = njs_array(&args[0]); - length = array->length; + iargs.value = njs_argument(args, 0); + iargs.argument = njs_arg(args, nargs, 1); + + ret = njs_value_length(vm, iargs.value, &length); + if (njs_slow_path(ret != NJS_OK)) { + return ret; + } if (length == 0) { - goto done; + goto not_found; } - i = 0; - - if (nargs > 2) { - i = njs_number(&args[2]); - - if (i >= length) { - goto done; - } - - if (i < 0) { - i += length; - - if (i < 0) { - i = 0; - } + from = njs_primitive_value_to_integer(njs_arg(args, nargs, 2)); + + if (from < 0) { + from += length; + + if (from < 0) { + from = 0; } } - start = array->start; - value = &args[1]; - - if (njs_is_number(value) && isnan(njs_number(value))) { - - do { - value = &start[i]; - - if (njs_is_number(value) && isnan(njs_number(value))) { - retval = &njs_value_true; - break; - } - - i++; - - } while (i < length); + iargs.from = (uint32_t) from; + iargs.to = length; + + if (njs_is_numeric(iargs.argument) && isnan(njs_number(iargs.argument))) { + ret = njs_array_iterator(vm, &iargs, njs_array_handler_includes_nan); + if (njs_fast_path(ret == NJS_DECLINED)) { + return NJS_OK; + } } else { - do { - if (njs_values_strict_equal(value, &start[i])) { - retval = &njs_value_true; - break; - } - - i++; - - } while (i < length); + ret = njs_array_iterator(vm, &iargs, njs_array_handler_includes); + if (njs_fast_path(ret == NJS_DECLINED)) { + return NJS_OK; + } } -done: - - vm->retval = *retval; +not_found: + + njs_set_false(&vm->retval); return NJS_OK; } @@ -1375,149 +1664,11 @@ njs_array_iterator_call(njs_vm_t *vm, nj njs_set_number(&arguments[1], n); arguments[2] = *args->value; - return njs_function_call(vm, args->function, args->this_arg, arguments, 3, + return njs_function_call(vm, args->function, args->argument, arguments, 3, &vm->retval); } -njs_inline njs_int_t -njs_array_iterator(njs_vm_t *vm, njs_array_iterator_args_t *args, - njs_array_iterator_handler_t handler, uint32_t length) -{ - uint32_t i; - njs_int_t ret; - njs_value_t *entry, *value, character, index, string_obj, prop; - njs_object_t *object; - const u_char *p, *end, *pos; - njs_string_prop_t string_prop; - - value = args->value; - - if (njs_is_array(value)) { - if (njs_slow_path(!njs_object_hash_is_empty(value))) { - goto process_object; - } - - length = njs_array_len(value); - - for (i = 0; i < length; i++) { - entry = &njs_array_start(value)[i]; - - ret = handler(vm, args, entry, i); - if (njs_slow_path(ret != NJS_OK)) { - if (ret > 0) { - return NJS_DECLINED; - } - - return NJS_ERROR; - } - - length = njs_min(length, njs_array_len(value)); - } - - return NJS_OK; - } - - if (njs_is_string(value) || njs_is_object_string(value)) { - - if (njs_is_string(value)) { - object = njs_object_value_alloc(vm, value, NJS_STRING); - if (njs_slow_path(object == NULL)) { - return NJS_ERROR; - } - - njs_set_type_object(&string_obj, object, NJS_OBJECT_STRING); - - args->value = &string_obj; - } - else { - value = njs_object_value(value); - } - - length = (uint32_t) njs_string_prop(&string_prop, value); - - p = string_prop.start; - end = p + string_prop.size; - - if (length == string_prop.size) { - /* Byte or ASCII string. */ - - for (i = 0; i < length; i++) { - /* This cannot fail. */ - (void) njs_string_new(vm, &character, p++, 1, 1); - - ret = handler(vm, args, &character, i); - if (njs_slow_path(ret != NJS_OK)) { - if (ret > 0) { - return NJS_DECLINED; - } - - return NJS_ERROR; - } - } - - } else { - /* UTF-8 string. */ - - for (i = 0; i < length; i++) { - pos = njs_utf8_next(p, end); - - /* This cannot fail. */ - (void) njs_string_new(vm, &character, p, pos - p, 1); - - ret = handler(vm, args, &character, i); - if (njs_slow_path(ret != NJS_OK)) { - if (ret > 0) { - return NJS_DECLINED; - } - - return NJS_ERROR; - } - - p = pos; - } - } - - return NJS_OK; - } - - if (!njs_is_object(value)) { - return NJS_OK; - } - -process_object: - - if (length > NJS_ARRAY_MAX_LENGTH) { - ret = njs_object_length(vm, value, &length); - if (njs_slow_path(ret != NJS_OK)) { - return NJS_ERROR; - } - } - - for (i = 0; i < length; i++) { - njs_uint32_to_string(&index, i); - - ret = njs_value_property(vm, value, &index, &prop); - if (njs_slow_path(ret == NJS_ERROR)) { - return ret; - } - - if (ret != NJS_DECLINED) { - ret = handler(vm, args, &prop, i); - if (njs_slow_path(ret != NJS_OK)) { - if (ret > 0) { - return NJS_DECLINED; - } - - return NJS_ERROR; - } - } - } - - return NJS_OK; -} - - static njs_int_t njs_array_handler_for_each(njs_vm_t *vm, njs_array_iterator_args_t *args, njs_value_t *entry, uint32_t n) @@ -1546,10 +1697,16 @@ njs_array_prototype_for_each(njs_vm_t *v iargs.value = njs_argument(args, 0); iargs.function = njs_function(&args[1]); - iargs.this_arg = njs_arg(args, nargs, 2); - - ret = njs_array_iterator(vm, &iargs, njs_array_handler_for_each, - NJS_ARRAY_MAX_LENGTH + 1); + iargs.argument = njs_arg(args, nargs, 2); + + iargs.from = 0; + + ret = njs_value_length(vm, iargs.value, &iargs.to); + if (njs_slow_path(ret != NJS_OK)) { + return ret; + } + + ret = njs_array_iterator(vm, &iargs, njs_array_handler_for_each); if (njs_slow_path(ret != NJS_OK)) { return ret; } @@ -1599,10 +1756,16 @@ njs_array_prototype_some(njs_vm_t *vm, n iargs.value = njs_argument(args, 0); iargs.function = njs_function(&args[1]); - iargs.this_arg = njs_arg(args, nargs, 2); - - ret = njs_array_iterator(vm, &iargs, njs_array_handler_some, - NJS_ARRAY_MAX_LENGTH + 1); + iargs.argument = njs_arg(args, nargs, 2); + + iargs.from = 0; + + ret = njs_value_length(vm, iargs.value, &iargs.to); + if (njs_slow_path(ret != NJS_OK)) { + return ret; + } + + ret = njs_array_iterator(vm, &iargs, njs_array_handler_some); if (njs_slow_path(ret == NJS_ERROR)) { return ret; } @@ -1654,10 +1817,16 @@ njs_array_prototype_every(njs_vm_t *vm, iargs.value = njs_argument(args, 0); iargs.function = njs_function(&args[1]); - iargs.this_arg = njs_arg(args, nargs, 2); - - ret = njs_array_iterator(vm, &iargs, njs_array_handler_every, - NJS_ARRAY_MAX_LENGTH + 1); + iargs.argument = njs_arg(args, nargs, 2); + + iargs.from = 0; + + ret = njs_value_length(vm, iargs.value, &iargs.to); + if (njs_slow_path(ret != NJS_OK)) { + return ret; + } + + ret = njs_array_iterator(vm, &iargs, njs_array_handler_every); if (njs_slow_path(ret == NJS_ERROR)) { return ret; } @@ -1714,15 +1883,21 @@ njs_array_prototype_filter(njs_vm_t *vm, iargs.value = njs_argument(args, 0); iargs.function = njs_function(&args[1]); - iargs.this_arg = njs_arg(args, nargs, 2); + iargs.argument = njs_arg(args, nargs, 2); + + iargs.from = 0; + + ret = njs_value_length(vm, iargs.value, &iargs.to); + if (njs_slow_path(ret != NJS_OK)) { + return ret; + } iargs.array = njs_array_alloc(vm, 0, NJS_ARRAY_SPARE); if (njs_slow_path(iargs.array == NULL)) { return NJS_ERROR; } - ret = njs_array_iterator(vm, &iargs, njs_array_handler_filter, - NJS_ARRAY_MAX_LENGTH + 1); + ret = njs_array_iterator(vm, &iargs, njs_array_handler_filter); if (njs_slow_path(ret != NJS_OK)) { return ret; } @@ -1778,10 +1953,16 @@ njs_array_prototype_find(njs_vm_t *vm, n iargs.value = njs_argument(args, 0); iargs.function = njs_function(&args[1]); - iargs.this_arg = njs_arg(args, nargs, 2); - - ret = njs_array_iterator(vm, &iargs, njs_array_handler_find, - NJS_ARRAY_MAX_LENGTH + 1); + iargs.argument = njs_arg(args, nargs, 2); + + iargs.from = 0; + + ret = njs_value_length(vm, iargs.value, &iargs.to); + if (njs_slow_path(ret != NJS_OK)) { + return ret; + } + + ret = njs_array_iterator(vm, &iargs, njs_array_handler_find); if (njs_slow_path(ret == NJS_ERROR)) { return ret; } @@ -1839,10 +2020,16 @@ njs_array_prototype_find_index(njs_vm_t iargs.value = njs_argument(args, 0); iargs.function = njs_function(&args[1]); - iargs.this_arg = njs_arg(args, nargs, 2); - - ret = njs_array_iterator(vm, &iargs, njs_array_handler_find_index, - NJS_ARRAY_MAX_LENGTH + 1); + iargs.argument = njs_arg(args, nargs, 2); + + iargs.from = 0; + + ret = njs_value_length(vm, iargs.value, &iargs.to); + if (njs_slow_path(ret != NJS_OK)) { + return ret; + } + + ret = njs_array_iterator(vm, &iargs, njs_array_handler_find_index); if (njs_slow_path(ret == NJS_ERROR)) { return ret; } @@ -1899,7 +2086,7 @@ njs_array_prototype_map(njs_vm_t *vm, nj iargs.value = njs_argument(args, 0); iargs.function = njs_function(&args[1]); - iargs.this_arg = njs_arg(args, nargs, 2); + iargs.argument = njs_arg(args, nargs, 2); ret = njs_value_length(vm, iargs.value, &length); if (njs_slow_path(ret != NJS_OK)) { @@ -1912,7 +2099,10 @@ njs_array_prototype_map(njs_vm_t *vm, nj } if (length > 0) { - ret = njs_array_iterator(vm, &iargs, njs_array_handler_map, length); + iargs.from = 0; + iargs.to = length; + + ret = njs_array_iterator(vm, &iargs, njs_array_handler_map); if (njs_slow_path(ret != NJS_OK)) { return ret; } @@ -1943,12 +2133,12 @@ njs_array_iterator_reduce(njs_vm_t *vm, /* GC: array elt, array */ njs_set_undefined(&arguments[0]); - arguments[1] = *args->this_arg; + arguments[1] = *args->argument; arguments[2] = *entry; njs_set_number(&arguments[3], n); arguments[4] = *args->value; - return njs_function_apply(vm, args->function, arguments, 5, args->this_arg); + return njs_function_apply(vm, args->function, arguments, 5, args->argument); } @@ -1960,8 +2150,8 @@ njs_array_handler_reduce(njs_vm_t *vm, n if (njs_is_valid(entry)) { - if (!njs_is_valid(args->this_arg)) { - *(args->this_arg) = *entry; + if (!njs_is_valid(args->argument)) { + *(args->argument) = *entry; return NJS_OK; } @@ -1998,16 +2188,22 @@ njs_array_prototype_reduce(njs_vm_t *vm, iargs.value = njs_argument(args, 0); iargs.function = njs_function(&args[1]); - iargs.this_arg = &accumulator; - - ret = njs_array_iterator(vm, &iargs, njs_array_handler_reduce, - NJS_ARRAY_MAX_LENGTH + 1); + iargs.argument = &accumulator; + + iargs.from = 0; + + ret = njs_value_length(vm, iargs.value, &iargs.to); + if (njs_slow_path(ret != NJS_OK)) { From alexander.borisov at nginx.com Thu Sep 19 07:20:13 2019 From: alexander.borisov at nginx.com (Alexander Borisov) Date: Thu, 19 Sep 2019 07:20:13 +0000 Subject: [njs] Fixed Array prototype functions according to the specification. Message-ID: details: https://hg.nginx.org/njs/rev/893fc436a363 branches: changeset: 1164:893fc436a363 user: Alexander Borisov date: Thu Sep 19 10:19:02 2019 +0300 description: Fixed Array prototype functions according to the specification. The following fuctions were fixed: pop, push, shift, unshift. diffstat: src/njs_array.c | 229 ++++++++++++++++++++++++++++++++++++++++++---- src/njs_object.h | 14 ++ src/test/njs_unit_test.c | 51 ++++++++++ 3 files changed, 273 insertions(+), 21 deletions(-) diffs (407 lines): diff -r a07722caaee3 -r 893fc436a363 src/njs_array.c --- a/src/njs_array.c Thu Sep 19 10:19:01 2019 +0300 +++ b/src/njs_array.c Thu Sep 19 10:19:02 2019 +0300 @@ -589,9 +589,19 @@ static njs_int_t njs_array_prototype_push(njs_vm_t *vm, njs_value_t *args, njs_uint_t nargs, njs_index_t unused) { + uint32_t length; njs_int_t ret; njs_uint_t i; njs_array_t *array; + njs_value_t *value, index; + + value = njs_arg(args, nargs, 0); + length = 0; + + if (njs_slow_path(njs_is_null_or_undefined(value))) { + njs_type_error(vm, "Cannot convert undefined or null to object"); + return NJS_ERROR; + } if (njs_is_array(&args[0])) { array = njs_array(&args[0]); @@ -609,8 +619,31 @@ njs_array_prototype_push(njs_vm_t *vm, n } njs_set_number(&vm->retval, array->length); + + return NJS_OK; } + ret = njs_object_length(vm, value, &length); + if (njs_slow_path(ret == NJS_ERROR)) { + return ret; + } + + for (i = 1; i < nargs; i++) { + njs_uint32_to_string(&index, length++); + + ret = njs_value_property_set(vm, value, &index, &args[i]); + if (njs_slow_path(ret == NJS_ERROR)) { + return ret; + } + } + + ret = njs_object_length_set(vm, value, length); + if (njs_slow_path(ret == NJS_ERROR)) { + return ret; + } + + njs_set_number(&vm->retval, length); + return NJS_OK; } @@ -619,25 +652,53 @@ static njs_int_t njs_array_prototype_pop(njs_vm_t *vm, njs_value_t *args, njs_uint_t nargs, njs_index_t unused) { - njs_array_t *array; - const njs_value_t *retval, *value; - - retval = &njs_value_undefined; + uint32_t length; + njs_int_t ret; + njs_array_t *array; + njs_value_t *value, *entry, index; + + value = njs_arg(args, nargs, 0); + + if (njs_slow_path(njs_is_null_or_undefined(value))) { + njs_type_error(vm, "Cannot convert undefined or null to object"); + return NJS_ERROR; + } + + njs_set_undefined(&vm->retval); if (njs_is_array(&args[0])) { array = njs_array(&args[0]); if (array->length != 0) { array->length--; - value = &array->start[array->length]; - - if (njs_is_valid(value)) { - retval = value; + entry = &array->start[array->length]; + + if (njs_is_valid(entry)) { + vm->retval = *entry; } } + + return NJS_OK; } - vm->retval = *retval; + ret = njs_object_length(vm, value, &length); + if (njs_slow_path(ret == NJS_ERROR)) { + return ret; + } + + if (length != 0) { + njs_uint32_to_string(&index, --length); + + ret = njs_value_property_delete(vm, value, &index, &vm->retval); + if (njs_slow_path(ret != NJS_OK)) { + return ret; + } + } + + ret = njs_object_length_set(vm, value, length); + if (njs_slow_path(ret == NJS_ERROR)) { + return ret; + } return NJS_OK; } @@ -647,13 +708,28 @@ static njs_int_t njs_array_prototype_unshift(njs_vm_t *vm, njs_value_t *args, njs_uint_t nargs, njs_index_t unused) { + uint32_t from, to, length; njs_int_t ret; njs_uint_t n; njs_array_t *array; - - if (njs_is_array(&args[0])) { - array = njs_array(&args[0]); - n = nargs - 1; + njs_value_t *value, entry, index; + + value = njs_arg(args, nargs, 0); + length = 0; + n = nargs - 1; + + if (njs_slow_path(njs_is_null_or_undefined(value))) { + njs_type_error(vm, "Cannot convert undefined or null to object"); + return NJS_ERROR; + } + + if (njs_is_array(value)) { + array = njs_array(value); + + if (array->length > (UINT32_MAX - n)) { + njs_type_error(vm, "Invalid length"); + return NJS_ERROR; + } if (n != 0) { ret = njs_array_expand(vm, array, n, 0); @@ -673,8 +749,66 @@ njs_array_prototype_unshift(njs_vm_t *vm } njs_set_number(&vm->retval, array->length); + + return NJS_OK; } + ret = njs_object_length(vm, value, &length); + if (njs_slow_path(ret == NJS_ERROR)) { + return ret; + } + + if (n == 0) { + goto done; + } + + if (length > (UINT32_MAX - n)) { + njs_type_error(vm, "Invalid length"); + return NJS_ERROR; + } + + from = length; + length += n; + to = length; + + while (from > 0) { + njs_uint32_to_string(&index, --from); + + ret = njs_value_property_delete(vm, value, &index, &entry); + if (njs_slow_path(ret == NJS_ERROR)) { + return ret; + } + + to--; + + if (ret == NJS_OK) { + njs_uint32_to_string(&index, to); + + ret = njs_value_property_set(vm, value, &index, &entry); + if (njs_slow_path(ret == NJS_ERROR)) { + return ret; + } + } + } + + for (n = 1; n < nargs; n++) { + njs_uint32_to_string(&index, n - 1); + + ret = njs_value_property_set(vm, value, &index, &args[n]); + if (njs_slow_path(ret == NJS_ERROR)) { + return ret; + } + } + +done: + + ret = njs_object_length_set(vm, value, length); + if (njs_slow_path(ret == NJS_ERROR)) { + return ret; + } + + njs_set_number(&vm->retval, length); + return NJS_OK; } @@ -683,10 +817,20 @@ static njs_int_t njs_array_prototype_shift(njs_vm_t *vm, njs_value_t *args, njs_uint_t nargs, njs_index_t unused) { - njs_array_t *array; - const njs_value_t *retval, *value; - - retval = &njs_value_undefined; + uint32_t i, length; + njs_int_t ret; + njs_array_t *array; + njs_value_t *value, *item, entry, index; + + value = njs_arg(args, nargs, 0); + length = 0; + + if (njs_slow_path(njs_is_null_or_undefined(value))) { + njs_type_error(vm, "Cannot convert undefined or null to object"); + return NJS_ERROR; + } + + njs_set_undefined(&vm->retval); if (njs_is_array(&args[0])) { array = njs_array(&args[0]); @@ -694,16 +838,59 @@ njs_array_prototype_shift(njs_vm_t *vm, if (array->length != 0) { array->length--; - value = &array->start[0]; + item = &array->start[0]; array->start++; - if (njs_is_valid(value)) { - retval = value; + if (njs_is_valid(item)) { + vm->retval = *item; + } + } + + return NJS_OK; + } + + ret = njs_object_length(vm, value, &length); + if (njs_slow_path(ret == NJS_ERROR)) { + return ret; + } + + if (length == 0) { + goto done; + } + + njs_uint32_to_string(&index, 0); + + ret = njs_value_property_delete(vm, value, &index, &vm->retval); + if (njs_slow_path(ret != NJS_OK)) { + return ret; + } + + for (i = 1; i < length; i++) { + njs_uint32_to_string(&index, i); + + ret = njs_value_property_delete(vm, value, &index, &entry); + if (njs_slow_path(ret == NJS_ERROR)) { + return ret; + } + + if (ret == NJS_OK) { + njs_uint32_to_string(&index, i - 1); + + ret = njs_value_property_set(vm, value, &index, &entry); + if (njs_slow_path(ret == NJS_ERROR)) { + return ret; } } } - vm->retval = *retval; + length--; + +done: + + ret = njs_object_length_set(vm, value, length); + if (njs_slow_path(ret == NJS_ERROR)) { + return ret; + } return NJS_OK; } diff -r a07722caaee3 -r 893fc436a363 src/njs_object.h --- a/src/njs_object.h Thu Sep 19 10:19:01 2019 +0300 +++ b/src/njs_object.h Thu Sep 19 10:19:02 2019 +0300 @@ -86,4 +86,18 @@ extern const njs_object_init_t njs_obje extern const njs_object_init_t njs_object_prototype_init; +njs_inline njs_int_t +njs_object_length_set(njs_vm_t *vm, njs_value_t *value, uint32_t length) +{ + njs_value_t index; + + static const njs_value_t string_length = njs_string("length"); + + njs_value_number_set(&index, length); + + return njs_value_property_set(vm, value, njs_value_arg(&string_length), + &index); +} + + #endif /* _NJS_OBJECT_H_INCLUDED_ */ diff -r a07722caaee3 -r 893fc436a363 src/test/njs_unit_test.c --- a/src/test/njs_unit_test.c Thu Sep 19 10:19:01 2019 +0300 +++ b/src/test/njs_unit_test.c Thu Sep 19 10:19:02 2019 +0300 @@ -3907,6 +3907,17 @@ static njs_unit_test_t njs_test[] = { njs_str("Array.prototype.pop()"), njs_str("undefined") }, + { njs_str("var o = {0: 'a', 1: 'b', 2: 'c', 'length': 3};" + "Array.prototype.pop.call(o); var res = o.length;" + "Array.prototype.forEach.call(o, (v) => {res += ', ' + v}); res"), + njs_str("2, a, b") }, + + { njs_str("var obj = {}; obj.pop = Array.prototype.pop; obj.pop(); obj.length === 0"), + njs_str("true") }, + + { njs_str("var o = Object.freeze({0: 0, 1: 1, length: 2}); Array.prototype.pop.call(o)"), + njs_str("TypeError: Cannot delete property \"1\" of object") }, + { njs_str("Array.prototype.shift()"), njs_str("undefined") }, @@ -3939,9 +3950,22 @@ static njs_unit_test_t njs_test[] = "len +' '+ a.pop() +' '+ a"), njs_str("5 5 1,2,3,4") }, + { njs_str("var x = {'0': 'a', '1': 'b', '2': 'c', 'length': 3};" + "Array.prototype.push.call(x, 'x', 'y', 'z', 123) +' '+ x[0] +' '+ x.length"), + njs_str("7 a 7") }, + + { njs_str("var x = {'0': 'a', '1': 'b', '2': 'c', 'length': 3}; var a = [];" + "Array.prototype.push.call(x, 'x', 'y', 'z', 123);" + "Array.prototype.forEach.call(x, (v) => a.push(v)); a"), + njs_str("a,b,c,x,y,z,123") }, + { njs_str("var a = [1,2,3]; a.shift() +' '+ a[0] +' '+ a.length"), njs_str("1 2 2") }, + { njs_str("var x = {'0': 'x', '1': 'y', '2': 'z', 'length': 3};" + "Array.prototype.shift.call(x) +' '+ x[0] +' '+ x.length"), + njs_str("x y 2") }, + { njs_str("var a = [1,2], len = a.unshift(3);" "len +' '+ a +' '+ a.shift()"), njs_str("3 3,1,2 3") }, @@ -3950,6 +3974,33 @@ static njs_unit_test_t njs_test[] = "len +' '+ a +' '+ a.shift()"), njs_str("5 3,4,5,1,2 3") }, + { njs_str("var x = {'0': 'x', '1': 'y', '2': 'z', 'length': 3}; var a = [];" + "Array.prototype.unshift.call(x, 'a', 'b', 'c');" + "Array.prototype.forEach.call(x, (v) => a.push(v)); a + ', ' + x.length"), + njs_str("a,b,c,x,y,z, 6") }, + + { njs_str("var x = {}; x.length = 1; var a = [];" + "Array.prototype.unshift.call(x, 'a', 'b', 1025, 'c', 'oh my');" + "Array.prototype.forEach.call(x, (v) => a.push(v)); a + ', ' + x.length"), + njs_str("a,b,1025,c,oh my, 6") }, + + { njs_str("var x = {5: 1, 6: 2, length: 7}; var a = [];" + "Array.prototype.unshift.call(x, '0');" + "Array.prototype.forEach.call(x, (v) => a.push(v)); a + ', ' + x.length"), + njs_str("0,1,2, 8") }, + + { njs_str("var x = {5: 2, 10: 3, 11: 4, 12: 5, 20: 6, length: 21}; var a = [];" + "Array.prototype.unshift.call(x, '0', '1');" + "Array.prototype.forEach.call(x, (v, k) => a.push(k + ':' + v)); a + ', ' + x.length"), + njs_str("0:0,1:1,7:2,12:3,13:4,14:5,22:6, 23") }, + + { njs_str("var x = {0: 0, length: 4294967294};" + "Array.prototype.unshift.call(x, '0', '1');"), + njs_str("TypeError: Invalid length") }, + + { njs_str("var x = {0: 0}; Array.prototype.unshift.call(x); x.length"), + njs_str("0") }, + { njs_str("var a=[0], n = 64; while(--n) {a.push(n); a.shift()}; a"), njs_str("1") }, From alexander.borisov at nginx.com Thu Sep 19 07:20:14 2019 From: alexander.borisov at nginx.com (Alexander Borisov) Date: Thu, 19 Sep 2019 07:20:14 +0000 Subject: [njs] Improved iteration over objects indexes in Array functions. Message-ID: details: https://hg.nginx.org/njs/rev/f516c8332495 branches: changeset: 1165:f516c8332495 user: Alexander Borisov date: Thu Sep 19 10:19:02 2019 +0300 description: Improved iteration over objects indexes in Array functions. For example, unshift function for object with large length: var arrayLike = {}; arrayLike.length = 2 ** 53 - 1; Array.prototype.unshift.call(arrayLike); diffstat: src/njs_array.c | 174 ++++++++++++++++++++++++++++++++++++++++++++++- src/test/njs_unit_test.c | 23 ++++++ 2 files changed, 196 insertions(+), 1 deletions(-) diffs (298 lines): diff -r 893fc436a363 -r f516c8332495 src/njs_array.c --- a/src/njs_array.c Thu Sep 19 10:19:02 2019 +0300 +++ b/src/njs_array.c Thu Sep 19 10:19:02 2019 +0300 @@ -8,6 +8,9 @@ #include +#define NJS_ARRAY_LARGE_OBJECT_LENGTH 4096 + + typedef struct { njs_function_t *function; njs_value_t *argument; @@ -27,6 +30,7 @@ typedef njs_int_t (*njs_array_iterator_h static njs_int_t njs_array_prototype_slice_copy(njs_vm_t *vm, njs_value_t *this, int64_t start, int64_t length); static njs_value_t *njs_array_copy(njs_value_t *dst, njs_value_t *src); +static njs_array_t *njs_object_indexes(njs_vm_t *vm, njs_value_t *object); njs_array_t * @@ -708,10 +712,11 @@ static njs_int_t njs_array_prototype_unshift(njs_vm_t *vm, njs_value_t *args, njs_uint_t nargs, njs_index_t unused) { + double idx; uint32_t from, to, length; njs_int_t ret; njs_uint_t n; - njs_array_t *array; + njs_array_t *array, *keys; njs_value_t *value, entry, index; value = njs_arg(args, nargs, 0); @@ -767,6 +772,38 @@ njs_array_prototype_unshift(njs_vm_t *vm return NJS_ERROR; } + if (length > NJS_ARRAY_LARGE_OBJECT_LENGTH) { + keys = njs_object_indexes(vm, value); + if (njs_slow_path(keys == NULL)) { + return NJS_ERROR; + } + + from = keys->length; + + while (from > 0) { + ret = njs_value_property_delete(vm, value, &keys->start[--from], + &entry); + if (njs_slow_path(ret == NJS_ERROR)) { + return ret; + } + + if (ret == NJS_OK) { + idx = njs_string_to_index(&keys->start[from]); + + njs_uint32_to_string(&index, (uint32_t) idx + nargs - 1); + + ret = njs_value_property_set(vm, value, &index, &entry); + if (njs_slow_path(ret == NJS_ERROR)) { + return ret; + } + } + } + + length += nargs - 1; + + goto copy; + } + from = length; length += n; to = length; @@ -791,6 +828,8 @@ njs_array_prototype_unshift(njs_vm_t *vm } } +copy: + for (n = 1; n < nargs; n++) { njs_uint32_to_string(&index, n - 1); @@ -1199,12 +1238,75 @@ njs_array_prototype_join(njs_vm_t *vm, n } +static int +njs_object_indexes_handler(const void *first, const void *second) +{ + double num1, num2; + njs_str_t str1, str2; + const njs_value_t *val1, *val2; + + val1 = first; + val2 = second; + + num1 = njs_string_to_index(val1); + num2 = njs_string_to_index(val2); + + if (!isnan(num1) || !isnan(num2)) { + if (isnan(num1)) { + return 1; + } + + if (isnan(num2)) { + return -1; + } + + return (int) (num1 - num2); + } + + njs_string_get(val1, &str1); + njs_string_get(val2, &str2); + + return strncmp((const char *) str1.start, (const char *) str2.start, + njs_min(str1.length, str2.length)); +} + + +static njs_array_t * +njs_object_indexes(njs_vm_t *vm, njs_value_t *object) +{ + double idx; + uint32_t i; + njs_array_t *keys; + + keys = njs_value_own_enumerate(vm, object, NJS_ENUM_KEYS, 0); + if (njs_slow_path(keys == NULL)) { + return NULL; + } + + qsort(keys->start, keys->length, sizeof(njs_value_t), + njs_object_indexes_handler); + + for (i = 0; i < keys->length; i++) { + idx = njs_string_to_index(&keys->start[i]); + + if (isnan(idx)) { + keys->length = i; + break; + } + } + + return keys; +} + + njs_inline njs_int_t njs_array_iterator(njs_vm_t *vm, njs_array_iterator_args_t *args, njs_array_iterator_handler_t handler) { + double idx; uint32_t length, i, from, to; njs_int_t ret; + njs_array_t *keys; njs_value_t *entry, *value, character, index, string_obj, prop; njs_object_t *object; const u_char *p, *end, *pos; @@ -1306,6 +1408,39 @@ njs_array_iterator(njs_vm_t *vm, njs_arr process_object: + if ((to - from) > NJS_ARRAY_LARGE_OBJECT_LENGTH) { + keys = njs_object_indexes(vm, value); + if (njs_slow_path(keys == NULL)) { + return NJS_ERROR; + } + + for (i = 0; i < keys->length; i++) { + idx = njs_string_to_index(&keys->start[i]); + + if (idx < from || idx > to) { + continue; + } + + ret = njs_value_property(vm, value, &keys->start[i], &prop); + if (njs_slow_path(ret == NJS_ERROR)) { + return ret; + } + + if (ret != NJS_DECLINED) { + ret = handler(vm, args, &prop, i); + if (njs_slow_path(ret != NJS_OK)) { + if (ret > 0) { + return NJS_DECLINED; + } + + return NJS_ERROR; + } + } + } + + return NJS_OK; + } + for (i = from; i < to; i++) { njs_uint32_to_string(&index, i); @@ -1334,8 +1469,10 @@ njs_inline njs_int_t njs_array_reverse_iterator(njs_vm_t *vm, njs_array_iterator_args_t *args, njs_array_iterator_handler_t handler) { + double idx; uint32_t i, from, to, length; njs_int_t ret; + njs_array_t *keys; njs_value_t *entry, *value, character, index, string_obj, prop; njs_object_t *object; const u_char *p, *end, *pos; @@ -1445,6 +1582,41 @@ njs_array_reverse_iterator(njs_vm_t *vm, process_object: + if ((from - to) > NJS_ARRAY_LARGE_OBJECT_LENGTH) { + keys = njs_object_indexes(vm, value); + if (njs_slow_path(keys == NULL)) { + return NJS_ERROR; + } + + i = keys->length; + + while (i > 0) { + idx = njs_string_to_index(&keys->start[--i]); + + if (idx < to || idx > from) { + continue; + } + + ret = njs_value_property(vm, value, &keys->start[i], &prop); + if (njs_slow_path(ret == NJS_ERROR)) { + return ret; + } + + if (ret != NJS_DECLINED) { + ret = handler(vm, args, &prop, idx); + if (njs_slow_path(ret != NJS_OK)) { + if (ret > 0) { + return NJS_DECLINED; + } + + return NJS_ERROR; + } + } + } + + return NJS_OK; + } + i = from + 1; while (i-- > to) { diff -r 893fc436a363 -r f516c8332495 src/test/njs_unit_test.c --- a/src/test/njs_unit_test.c Thu Sep 19 10:19:02 2019 +0300 +++ b/src/test/njs_unit_test.c Thu Sep 19 10:19:02 2019 +0300 @@ -4001,6 +4001,12 @@ static njs_unit_test_t njs_test[] = { njs_str("var x = {0: 0}; Array.prototype.unshift.call(x); x.length"), njs_str("0") }, + { njs_str("var obj = {'10000000': 'x', '10000001': 'y', '10000002': 'z'}; var a = [];" + "obj.length = 90000000;" + "Array.prototype.unshift.call(obj, 'a', 'b', 'c');" + "Array.prototype.forEach.call(obj, (v) => a.push(v)); a"), + njs_str("a,b,c,x,y,z")}, + { njs_str("var a=[0], n = 64; while(--n) {a.push(n); a.shift()}; a"), njs_str("1") }, @@ -4156,6 +4162,11 @@ static njs_unit_test_t njs_test[] = "Array.prototype.lastIndexOf.call(o, 'd')"), njs_str("3") }, + { njs_str("var obj = {'10000000': 'x', '10000001': 'y', '10000002': 'z'}; var a = [];" + "obj.length = 90000000;" + "Array.prototype.lastIndexOf.call(obj, 'y');"), + njs_str("10000001")}, + { njs_str("[1,2,3,4].includes()"), njs_str("false") }, @@ -4192,6 +4203,18 @@ static njs_unit_test_t njs_test[] = "Array.prototype.includes.call(o, 'd')"), njs_str("true") }, + { njs_str("var obj = {'0': 'a', '1': 'b', '10000000': 'c', '10000001': 'd', '10000002': 'e'};" + "var fromIndex = 1;" + "obj.length = 90000000;" + "Array.prototype.includes.call(obj, 'c', fromIndex);"), + njs_str("true") }, + + { njs_str("var obj = {'0': 'a', '1': 'b', '10000000': 'c', '10000001': 'd', '10000002': 'e'};" + "var fromIndex = 1;" + "obj.length = 90000000;" + "Array.prototype.includes.call(obj, 'a', fromIndex);"), + njs_str("false") }, + { njs_str("var a = []; var s = { sum: 0 };" "a.forEach(function(v, i, a) { this.sum += v }, s); s.sum"), njs_str("0") }, From ru at nginx.com Mon Sep 23 12:46:24 2019 From: ru at nginx.com (Ruslan Ermilov) Date: Mon, 23 Sep 2019 12:46:24 +0000 Subject: [nginx] HTTP/2: fixed possible alert about left open socket on shutdown. Message-ID: details: https://hg.nginx.org/nginx/rev/d6cf51af8a3d branches: changeset: 7570:d6cf51af8a3d user: Ruslan Ermilov date: Mon Sep 23 15:45:32 2019 +0300 description: HTTP/2: fixed possible alert about left open socket on shutdown. This could happen when graceful shutdown configured by worker_shutdown_timeout times out and is then followed by another timeout such as proxy_read_timeout. In this case, the HEADERS frame is added to the output queue, but attempt to send it fails (due to c->error forcibly set during graceful shutdown timeout). This triggers request finalization which attempts to close the stream. But the stream cannot be closed because there is a frame in the output queue, and the connection cannot be finalized. This leaves the connection open without any timer events leading to alert. The fix is to post write event when sending output queue fails on c->error. That will finalize the connection. diffstat: src/http/v2/ngx_http_v2.c | 7 +++---- 1 files changed, 3 insertions(+), 4 deletions(-) diffs (19 lines): diff -r 80359395b345 -r d6cf51af8a3d src/http/v2/ngx_http_v2.c --- a/src/http/v2/ngx_http_v2.c Wed Sep 18 20:28:12 2019 +0300 +++ b/src/http/v2/ngx_http_v2.c Mon Sep 23 15:45:32 2019 +0300 @@ -513,12 +513,11 @@ ngx_http_v2_send_output_queue(ngx_http_v ngx_http_core_loc_conf_t *clcf; c = h2c->connection; + wev = c->write; if (c->error) { - return NGX_ERROR; - } - - wev = c->write; + goto error; + } if (!wev->ready) { return NGX_AGAIN; From ru at nginx.com Mon Sep 23 12:46:26 2019 From: ru at nginx.com (Ruslan Ermilov) Date: Mon, 23 Sep 2019 12:46:26 +0000 Subject: [nginx] HTTP/2: fixed worker_shutdown_timeout. Message-ID: details: https://hg.nginx.org/nginx/rev/ab5cac9d3f00 branches: changeset: 7571:ab5cac9d3f00 user: Ruslan Ermilov date: Mon Sep 23 15:45:36 2019 +0300 description: HTTP/2: fixed worker_shutdown_timeout. diffstat: src/http/v2/ngx_http_v2.c | 5 +++++ 1 files changed, 5 insertions(+), 0 deletions(-) diffs (15 lines): diff -r d6cf51af8a3d -r ab5cac9d3f00 src/http/v2/ngx_http_v2.c --- a/src/http/v2/ngx_http_v2.c Mon Sep 23 15:45:32 2019 +0300 +++ b/src/http/v2/ngx_http_v2.c Mon Sep 23 15:45:36 2019 +0300 @@ -354,6 +354,11 @@ ngx_http_v2_read_handler(ngx_event_t *re if (c->close) { c->close = 0; + if (c->error) { + ngx_http_v2_finalize_connection(h2c, 0); + return; + } + if (!h2c->goaway) { h2c->goaway = 1; From mdounin at mdounin.ru Tue Sep 24 14:09:10 2019 From: mdounin at mdounin.ru (Maxim Dounin) Date: Tue, 24 Sep 2019 14:09:10 +0000 Subject: [nginx] Updated OpenSSL used for win32 builds. Message-ID: details: https://hg.nginx.org/nginx/rev/1e9bde25d804 branches: changeset: 7572:1e9bde25d804 user: Maxim Dounin date: Tue Sep 24 16:30:03 2019 +0300 description: Updated OpenSSL used for win32 builds. diffstat: misc/GNUmakefile | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diffs (12 lines): diff --git a/misc/GNUmakefile b/misc/GNUmakefile --- a/misc/GNUmakefile +++ b/misc/GNUmakefile @@ -6,7 +6,7 @@ TEMP = tmp CC = cl OBJS = objs.msvc8 -OPENSSL = openssl-1.1.1c +OPENSSL = openssl-1.1.1d ZLIB = zlib-1.2.11 PCRE = pcre-8.43 From mdounin at mdounin.ru Tue Sep 24 15:10:44 2019 From: mdounin at mdounin.ru (Maxim Dounin) Date: Tue, 24 Sep 2019 15:10:44 +0000 Subject: [nginx] nginx-1.17.4-RELEASE Message-ID: details: https://hg.nginx.org/nginx/rev/ce2ced385690 branches: changeset: 7573:ce2ced385690 user: Maxim Dounin date: Tue Sep 24 18:08:48 2019 +0300 description: nginx-1.17.4-RELEASE diffstat: docs/xml/nginx/changes.xml | 89 ++++++++++++++++++++++++++++++++++++++++++++++ 1 files changed, 89 insertions(+), 0 deletions(-) diffs (99 lines): diff --git a/docs/xml/nginx/changes.xml b/docs/xml/nginx/changes.xml --- a/docs/xml/nginx/changes.xml +++ b/docs/xml/nginx/changes.xml @@ -5,6 +5,95 @@ + + + + +???????? ?????????????? ????????????? ????????? ???????? ? HTTP/2. + + +better detection of incorrect client behavior in HTTP/2. + + + + + +? ????????? ?????????????? ???? ??????? +??? ???????? ?????? ? HTTP/2. + + +in handling of not fully read client request body +when returning errors in HTTP/2. + + + + + +????????? worker_shutdown_timeout ????? ?? ???????? +??? ????????????? HTTP/2. + + +the "worker_shutdown_timeout" directive might not work +when using HTTP/2. + + + + + +??? ????????????? HTTP/2 ? ????????? proxy_request_buffering +? ??????? ???????? ??? ????????? segmentation fault. + + +a segmentation fault might occur in a worker process +when using HTTP/2 and the "proxy_request_buffering" directive. + + + + + +?? Windows ??? ????????????? SSL +??????? ?????? ? ??? ?????? ECONNABORTED ??? "crit" ?????? "error". + + +the ECONNABORTED error log level was "crit" instead of "error" +on Windows when using SSL. + + + + + +nginx ??????????? ?????? ?????? ??? ????????????? chunked transfer encoding. + + +nginx ignored extra data when using chunked transfer encoding. + + + + + +???? ?????????????? ????????? return ? +??? ?????? ???? ??????? ????????? ??????, +nginx ?????? ????????? ?????? 500. + + +nginx always returned the 500 error +if the "return" directive was used +and an error occurred during reading client request body. + + + + + +? ????????? ?????? ????????? ??????. + + +in memory allocation error handling. + + + + + + From mdounin at mdounin.ru Tue Sep 24 15:10:45 2019 From: mdounin at mdounin.ru (Maxim Dounin) Date: Tue, 24 Sep 2019 15:10:45 +0000 Subject: [nginx] release-1.17.4 tag Message-ID: details: https://hg.nginx.org/nginx/rev/7ec8ef26e5e4 branches: changeset: 7574:7ec8ef26e5e4 user: Maxim Dounin date: Tue Sep 24 18:08:48 2019 +0300 description: release-1.17.4 tag diffstat: .hgtags | 1 + 1 files changed, 1 insertions(+), 0 deletions(-) diffs (8 lines): diff --git a/.hgtags b/.hgtags --- a/.hgtags +++ b/.hgtags @@ -442,3 +442,4 @@ 054c1c46395caff79bb4caf16f40b331f71bb6dd 7816bd7dabf6ee86c53c073b90a7143161546e06 release-1.17.1 2fc9f853a6b7cd29dc84e0af2ed3cf78e0da6ca8 release-1.17.2 ed4303aa1b31a9aad5440640c0840d9d0af45fed release-1.17.3 +ce2ced3856909f36f8130c99eaa4dbdbae636ddc release-1.17.4 From vbart at nginx.com Sat Sep 28 11:53:53 2019 From: vbart at nginx.com (Valentin Bartenev) Date: Sat, 28 Sep 2019 11:53:53 +0000 Subject: [njs] Date.prototype.toUTCString() format was aligned to ES9. Message-ID: details: https://hg.nginx.org/njs/rev/5c22e2f006fe branches: changeset: 1166:5c22e2f006fe user: Valentin Bartenev date: Fri Sep 27 22:21:55 2019 +0300 description: Date.prototype.toUTCString() format was aligned to ES9. This closes #224 issue on GitHub. diffstat: src/njs_date.c | 12 +++++++----- src/test/njs_unit_test.c | 4 ++-- 2 files changed, 9 insertions(+), 7 deletions(-) diffs (57 lines): diff -r f516c8332495 -r 5c22e2f006fe src/njs_date.c --- a/src/njs_date.c Thu Sep 19 10:19:02 2019 +0300 +++ b/src/njs_date.c Fri Sep 27 22:21:55 2019 +0300 @@ -14,7 +14,9 @@ * FreeBSD and MacOSX timegm() cannot handle years before 1900. */ -#define NJS_ISO_DATE_TIME_LEN sizeof("+001970-09-28T12:00:00.000Z") +#define NJS_ISO_DATE_TIME_LEN sizeof("+001970-09-28T12:00:00.000Z") + +#define NJS_HTTP_DATE_TIME_LEN sizeof("Mon, 28 Sep 1970 12:00:00 GMT") #define NJS_DATE_TIME_LEN \ sizeof("Mon Sep 28 1970 12:00:00 GMT+0600 (XXXXX)") @@ -1009,7 +1011,7 @@ njs_date_prototype_to_utc_string(njs_vm_ { double time; time_t clock; - u_char buf[NJS_DATE_TIME_LEN], *p; + u_char buf[NJS_HTTP_DATE_TIME_LEN], *p; struct tm tm; static const char *week[] = { "Sun", "Mon", "Tue", "Wed", @@ -1024,9 +1026,9 @@ njs_date_prototype_to_utc_string(njs_vm_ clock = time / 1000; gmtime_r(&clock, &tm); - p = njs_sprintf(buf, buf + NJS_DATE_TIME_LEN, - "%s %s %02d %4d %02d:%02d:%02d GMT", - week[tm.tm_wday], month[tm.tm_mon], tm.tm_mday, + p = njs_sprintf(buf, buf + NJS_HTTP_DATE_TIME_LEN, + "%s, %02d %s %4d %02d:%02d:%02d GMT", + week[tm.tm_wday], tm.tm_mday, month[tm.tm_mon], tm.tm_year + 1900, tm.tm_hour, tm.tm_min, tm.tm_sec); return njs_string_new(vm, &vm->retval, buf, p - buf, p - buf); diff -r f516c8332495 -r 5c22e2f006fe src/test/njs_unit_test.c --- a/src/test/njs_unit_test.c Thu Sep 19 10:19:02 2019 +0300 +++ b/src/test/njs_unit_test.c Fri Sep 27 22:21:55 2019 +0300 @@ -11323,7 +11323,7 @@ static njs_unit_test_t njs_test[] = njs_str("18:45:00 GMT+0000") }, { njs_str("var d = new Date(2011, 5, 24, 18, 45); d.toUTCString()"), - njs_str("Fri Jun 24 2011 18:45:00 GMT") }, + njs_str("Fri, 24 Jun 2011 18:45:00 GMT") }, { njs_str("var d = new Date(2011, 5, 24, 18, 45, 12, 625);" "d.toISOString()"), @@ -14074,7 +14074,7 @@ static njs_unit_test_t njs_tz_test[] = njs_str("18:45:00 GMT+1245") }, { njs_str("var d = new Date(2011, 5, 24, 18, 45); d.toUTCString()"), - njs_str("Fri Jun 24 2011 06:00:00 GMT") }, + njs_str("Fri, 24 Jun 2011 06:00:00 GMT") }, { njs_str("var d = new Date(2011, 5, 24, 18, 45, 12, 625);" "d.toISOString()"), From vl at nginx.com Mon Sep 30 13:49:11 2019 From: vl at nginx.com (Vladimir Homutov) Date: Mon, 30 Sep 2019 13:49:11 +0000 Subject: [nginx] Version bump. Message-ID: details: https://hg.nginx.org/nginx/rev/d34a81b49214 branches: changeset: 7575:d34a81b49214 user: Vladimir Homutov date: Mon Sep 30 16:43:16 2019 +0300 description: Version bump. diffstat: src/core/nginx.h | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) diffs (14 lines): diff -r 7ec8ef26e5e4 -r d34a81b49214 src/core/nginx.h --- a/src/core/nginx.h Tue Sep 24 18:08:48 2019 +0300 +++ b/src/core/nginx.h Mon Sep 30 16:43:16 2019 +0300 @@ -9,8 +9,8 @@ #define _NGINX_H_INCLUDED_ -#define nginx_version 1017004 -#define NGINX_VERSION "1.17.4" +#define nginx_version 1017005 +#define NGINX_VERSION "1.17.5" #define NGINX_VER "nginx/" NGINX_VERSION #ifdef NGX_BUILD From vl at nginx.com Mon Sep 30 13:49:12 2019 From: vl at nginx.com (Vladimir Homutov) Date: Mon, 30 Sep 2019 13:49:12 +0000 Subject: [nginx] Core: removed dead code in ngx_rbtree_delete(). Message-ID: details: https://hg.nginx.org/nginx/rev/7fdcf308e0f0 branches: changeset: 7576:7fdcf308e0f0 user: Vladimir Homutov date: Mon Sep 30 16:39:20 2019 +0300 description: Core: removed dead code in ngx_rbtree_delete(). The result of ngx_rbtree_min() is always a node with the left child equal to sentinel, thus the check is unnecessary. diffstat: src/core/ngx_rbtree.c | 7 +------ 1 files changed, 1 insertions(+), 6 deletions(-) diffs (17 lines): diff -r d34a81b49214 -r 7fdcf308e0f0 src/core/ngx_rbtree.c --- a/src/core/ngx_rbtree.c Mon Sep 30 16:43:16 2019 +0300 +++ b/src/core/ngx_rbtree.c Mon Sep 30 16:39:20 2019 +0300 @@ -174,12 +174,7 @@ ngx_rbtree_delete(ngx_rbtree_t *tree, ng } else { subst = ngx_rbtree_min(node->right, sentinel); - - if (subst->left != sentinel) { - temp = subst->left; - } else { - temp = subst->right; - } + temp = subst->right; } if (subst == *root) { From auke-jan.h.kok at intel.com Mon Sep 30 18:46:02 2019 From: auke-jan.h.kok at intel.com (Auke Kok) Date: Mon, 30 Sep 2019 11:46:02 -0700 Subject: Out-of-tree module building (PoC) for Linux Distro's. Message-ID: <6ebddc4d-a5e1-ab06-f2c5-49311fae265f@intel.com> Hi folks, In Clear Linux OS, we generally prefer to build DSO modules separately from the server code. We do this for e.g. php modules using `phpize`. For nginx, this is 3/4 of the way there, except it fails on the last part - modules instructions clearly indicate that modules, with all their dependencies, need to build together. But in reality, there is no technical reason for this. For Clear Linux therefore, I've come up with the following approach and it works really well, and therefore I'd like this to be considered as a PoC for possible inclusion/consideration so other distributions can benefit from the simplicity that it offers to Linux distributions. https://github.com/clearlinux-pkgs/nginx-mainline/blob/master/0003-Add-nginx-module-build-install-script.patch Now, this script is mostly still a terrible hack. I'm not properly fetching the nginx module names for instance. However, the method of installing header files saves us from having to include nginx source code in *every* nginx module, which will prevent lots of problems later on. Obviously a permanent solution would be for the module build system to properly -I/usr/include/nginx instead of copying headers, as well. Cheers, Auke PS: Example of how an (automated) module build looks using this: https://github.com/clearlinux-pkgs/ngx_brotli/blob/master/ngx_brotli.spec From mdounin at mdounin.ru Mon Sep 30 19:54:57 2019 From: mdounin at mdounin.ru (Maxim Dounin) Date: Mon, 30 Sep 2019 22:54:57 +0300 Subject: Out-of-tree module building (PoC) for Linux Distro's. In-Reply-To: <6ebddc4d-a5e1-ab06-f2c5-49311fae265f@intel.com> References: <6ebddc4d-a5e1-ab06-f2c5-49311fae265f@intel.com> Message-ID: <20190930195456.GO1877@mdounin.ru> Hello! On Mon, Sep 30, 2019 at 11:46:02AM -0700, Auke Kok wrote: > Hi folks, > > In Clear Linux OS, we generally prefer to build DSO modules separately > from the server code. We do this for e.g. php modules using `phpize`. > > For nginx, this is 3/4 of the way there, except it fails on the last > part - modules instructions clearly indicate that modules, with all > their dependencies, need to build together. But in reality, there is no > technical reason for this. > > For Clear Linux therefore, I've come up with the following approach and > it works really well, and therefore I'd like this to be considered as a > PoC for possible inclusion/consideration so other distributions can > benefit from the simplicity that it offers to Linux distributions. > > https://github.com/clearlinux-pkgs/nginx-mainline/blob/master/0003-Add-nginx-module-build-install-script.patch > > Now, this script is mostly still a terrible hack. I'm not properly > fetching the nginx module names for instance. However, the method of > installing header files saves us from having to include nginx source > code in *every* nginx module, which will prevent lots of problems later on. > > Obviously a permanent solution would be for the module build system to > properly -I/usr/include/nginx instead of copying headers, as well. Building nginx module requires nginx source code. You can, however, build nginx modules independently - that is, without building other modules (except when there are dependencies), and even nginx itself. There is a special "modules" target in the top-level Makefile as generated by configure, which builds all the modules configured in a particular source tree, but not nginx itself - and it looks like you are aware of this make target. The most important part to make sure that nginx and independently built modules will work together is to keep configure arguments compatible between builds. Most simple solution is to use `--with-compat` configure flag when building nginx itself and all the modules. There are no plans to make it possible to build modules without nginx source code. It is believed that nginx source code is small enough to make little to no difference with dedicated "header files only" install, and we don't want to bother with maintaining such an install. Note well that simply using all *.h files might not be enough. For example, building with Sun C compiler uses *.il files for assembler code. -- Maxim Dounin http://mdounin.ru/