From mdounin at mdounin.ru Wed Apr 1 15:20:04 2020 From: mdounin at mdounin.ru (Maxim Dounin) Date: Wed, 1 Apr 2020 18:20:04 +0300 Subject: [PATCH] When cache file is expired, nginx can make a lot of upstream requests. In-Reply-To: References: Message-ID: <20200401152004.GC20357@mdounin.ru> Hello! On Tue, Mar 31, 2020 at 01:59:31PM +0900, Sangdeuk Kwon wrote: > Thank you for your explanation > > I have a question. > Are there any directives for preventing multiple upstream and serving new > content? > > expect case: > 1st request: get new content from origin and serve new content. > 2nd request: serve new content. > 3rd request: serve new content. > > "proxy_cache_use stale updating" and "proxy_cache_background_update off" > case: > 1st request: get new content from origin and serve new content. > 2nd request: serve stale content. > finish 1st request's upstream. > 3rd request: serve new content. > > "proxy_cache_use stale updating" and "proxy_cache_background_update on" > case: > 1st request: serve stale content and make subrequest for getting new > content. > 2nd request: serve stale content. > finish 1st request's subrequest. > 3rd request: serve new content. As of now the answer is "no", you cannot configure nginx to avoid multiple upstream requests when updating cache items and avoid serving stale responses at the same time. There were some attempts in the past to expand proxy_cache_lock to updating cache items though. If you are interested, you may want to check this patch: http://mailman.nginx.org/pipermail/nginx-devel/2018-December/011710.html -- Maxim Dounin http://mdounin.ru/ From agile6v at agile6v.com Wed Apr 1 23:57:20 2020 From: agile6v at agile6v.com (=?gb18030?B?0ru9rbS6y66jrM/ytqvB9w==?=) Date: Thu, 2 Apr 2020 07:57:20 +0800 Subject: [PATCH] Mirror directive supports variable In-Reply-To: <20200331060237.amviji4g3tlf73h5@Romans-MacBook-Pro.local> References: <20200330123611.l47zqd636rxrht6i@Romans-MacBook-Pro.local> <20200331060237.amviji4g3tlf73h5@Romans-MacBook-Pro.local> Message-ID: Hi Roman Arutyunyan, Yes. If mirror directive supports a variable and its behavior should be different from static mirror. That mirror directive support variables can dynamically decide whether they need to be turned on and the configuration will be more concise. If it is described clearly in the document, it should also be easy to understand. Thanks, agile6v ------------------ Original ------------------ From: "Roman Arutyunyan";; Send time: Tuesday, Mar 31, 2020 2:02 PM To: "nginx-devel"; Cc: "dev"; Subject: Re: [PATCH] Mirror directive supports variable On Mon, Mar 30, 2020 at 03:36:11PM +0300, Roman Arutyunyan wrote: > Hello, > > On Fri, Mar 13, 2020 at 11:29:51PM +0800, ???????? wrote: > > # HG changeset patch > > # User agile6v > > # Date 1584110606 -28800 > > # Fri Mar 13 22:43:26 2020 +0800 > > # Node ID e4a0277cab79865fde6fefeed9374154449e6948 > > # Parent 4eb606b4a5b521603c23223cf8863f3999df743c > > mirror directive supports variable. Also, what is the reason why you need variables in the mirror directive? You can do the same thing by creating a static mirror and using rewrite to change uri. [..] -- Roman Arutyunyan -------------- next part -------------- An HTML attachment was scrubbed... URL: From mdounin at mdounin.ru Fri Apr 3 11:23:21 2020 From: mdounin at mdounin.ru (Maxim Dounin) Date: Fri, 3 Apr 2020 14:23:21 +0300 Subject: [PATCH] Mirror directive supports variable In-Reply-To: References: <20200330123611.l47zqd636rxrht6i@Romans-MacBook-Pro.local> <20200331060237.amviji4g3tlf73h5@Romans-MacBook-Pro.local> Message-ID: <20200403112321.GI20357@mdounin.ru> Hello! On Thu, Apr 02, 2020 at 07:57:20AM +0800, ???????? wrote: > That mirror directive support variables can dynamically decide > whether they need to be turned on and the configuration will be > more concise. You may want to be more specific here. For now it looks like a mostly unneeded feature, as existing mechanisms can be used to do the same. -- Maxim Dounin http://mdounin.ru/ From xeioex at nginx.com Fri Apr 3 16:07:19 2020 From: xeioex at nginx.com (Dmitry Volyntsev) Date: Fri, 03 Apr 2020 16:07:19 +0000 Subject: [njs] Fixed building unit tests by SunC 5.15. Message-ID: details: https://hg.nginx.org/njs/rev/78b8cfd5b1e9 branches: changeset: 1365:78b8cfd5b1e9 user: Dmitry Volyntsev date: Fri Apr 03 16:06:55 2020 +0000 description: Fixed building unit tests by SunC 5.15. Fixed warning "initializer will be sign-extended". diffstat: src/test/njs_externals_test.c | 4 ++-- src/test/njs_unit_test.c | 6 +++--- 2 files changed, 5 insertions(+), 5 deletions(-) diffs (40 lines): diff -r 2a03cd71561a -r 78b8cfd5b1e9 src/test/njs_externals_test.c --- a/src/test/njs_externals_test.c Thu Mar 26 17:12:16 2020 +0300 +++ b/src/test/njs_externals_test.c Fri Apr 03 16:06:55 2020 +0000 @@ -614,8 +614,8 @@ static njs_unit_test_req_init_t njs_test { njs_str("$shared"), { .uri = njs_str("shared"), - .a = -1, - .d = -2, + .a = 11, + .d = 13, }, { { njs_string("r"), njs_string("rval") }, diff -r 2a03cd71561a -r 78b8cfd5b1e9 src/test/njs_unit_test.c --- a/src/test/njs_unit_test.c Thu Mar 26 17:12:16 2020 +0300 +++ b/src/test/njs_unit_test.c Fri Apr 03 16:06:55 2020 +0000 @@ -16845,11 +16845,11 @@ static njs_unit_test_t njs_shared_test[ njs_str("{a:'1',b:42,c:{d:1024}}") }, { njs_str("njs.dump($shared.props)"), - njs_str("{a:'4294967295',b:42,c:{d:4294967294}}") }, + njs_str("{a:'11',b:42,c:{d:13}}") }, { njs_str("var r = JSON.parse(JSON.stringify($shared));" "[r.uri, r.host, r.props.a, njs.dump(r.vars), njs.dump(r.consts), r.header['02']]"), - njs_str("shared,???????????,4294967295,{},{},02|???") }, + njs_str("shared,???????????,11,{},{},02|???") }, { njs_str("$shared.toString()"), njs_str("[object External]") }, @@ -16864,7 +16864,7 @@ static njs_unit_test_t njs_shared_test[ njs_str("1") }, { njs_str("$shared.method = function() {return this.props.a;}; $shared.method()"), - njs_str("4294967295") }, + njs_str("11") }, { njs_str("var r; for (var i = 0; i < 2**10; i++) {r = $r.create('XXX').uri;}"), njs_str("undefined") }, From xeioex at nginx.com Fri Apr 3 16:10:47 2020 From: xeioex at nginx.com (Dmitry Volyntsev) Date: Fri, 03 Apr 2020 16:10:47 +0000 Subject: [njs] Added API for working with arrays. Message-ID: details: https://hg.nginx.org/njs/rev/9812e7d99e7c branches: changeset: 1366:9812e7d99e7c user: Dmitry Volyntsev date: Fri Apr 03 16:10:00 2020 +0000 description: Added API for working with arrays. diffstat: nginx/ngx_http_js_module.c | 9 ++-- nginx/ngx_stream_js_module.c | 5 +- src/njs.h | 8 ++++- src/njs_value.c | 7 ++++ src/njs_vm.c | 76 +++++++++++++++++++++++++++++++++++++------ 5 files changed, 87 insertions(+), 18 deletions(-) diffs (218 lines): diff -r 78b8cfd5b1e9 -r 9812e7d99e7c nginx/ngx_http_js_module.c --- a/nginx/ngx_http_js_module.c Fri Apr 03 16:06:55 2020 +0000 +++ b/nginx/ngx_http_js_module.c Fri Apr 03 16:10:00 2020 +0000 @@ -1806,6 +1806,7 @@ ngx_http_js_ext_subrequest(njs_vm_t *vm, njs_value_t *value, *arg, *options; njs_function_t *callback; ngx_http_js_ctx_t *ctx; + njs_opaque_value_t lvalue; ngx_http_request_t *r, *sr; ngx_http_request_body_t *rb; @@ -1891,7 +1892,7 @@ ngx_http_js_ext_subrequest(njs_vm_t *vm, } if (options != NULL) { - value = njs_vm_object_prop(vm, options, &args_key); + value = njs_vm_object_prop(vm, options, &args_key, &lvalue); if (value != NULL) { if (ngx_http_js_string(vm, value, &args_arg) != NJS_OK) { njs_vm_error(vm, "failed to convert options.args"); @@ -1899,12 +1900,12 @@ ngx_http_js_ext_subrequest(njs_vm_t *vm, } } - value = njs_vm_object_prop(vm, options, &detached_key); + value = njs_vm_object_prop(vm, options, &detached_key, &lvalue); if (value != NULL) { detached = njs_value_bool(value); } - value = njs_vm_object_prop(vm, options, &method_key); + value = njs_vm_object_prop(vm, options, &method_key, &lvalue); if (value != NULL) { if (ngx_http_js_string(vm, value, &method_name) != NJS_OK) { njs_vm_error(vm, "failed to convert options.method"); @@ -1924,7 +1925,7 @@ ngx_http_js_ext_subrequest(njs_vm_t *vm, } } - value = njs_vm_object_prop(vm, options, &body_key); + value = njs_vm_object_prop(vm, options, &body_key, &lvalue); if (value != NULL) { if (ngx_http_js_string(vm, value, &body_arg) != NJS_OK) { njs_vm_error(vm, "failed to convert options.body"); diff -r 78b8cfd5b1e9 -r 9812e7d99e7c nginx/ngx_stream_js_module.c --- a/nginx/ngx_stream_js_module.c Fri Apr 03 16:06:55 2020 +0000 +++ b/nginx/ngx_stream_js_module.c Fri Apr 03 16:10:00 2020 +0000 @@ -1064,6 +1064,7 @@ ngx_stream_js_ext_send(njs_vm_t *vm, njs njs_value_t *flags, *value; ngx_chain_t *cl; ngx_connection_t *c; + njs_opaque_value_t lvalue; ngx_stream_js_ctx_t *ctx; ngx_stream_session_t *s; @@ -1096,12 +1097,12 @@ ngx_stream_js_ext_send(njs_vm_t *vm, njs flags = njs_arg(args, nargs, 2); if (njs_value_is_object(flags)) { - value = njs_vm_object_prop(vm, flags, &flush_key); + value = njs_vm_object_prop(vm, flags, &flush_key, &lvalue); if (value != NULL) { flush = njs_value_bool(value); } - value = njs_vm_object_prop(vm, flags, &last_key); + value = njs_vm_object_prop(vm, flags, &last_key, &lvalue); if (value != NULL) { last_buf = njs_value_bool(value); } diff -r 78b8cfd5b1e9 -r 9812e7d99e7c src/njs.h --- a/src/njs.h Fri Apr 03 16:06:55 2020 +0000 +++ b/src/njs.h Fri Apr 03 16:10:00 2020 +0000 @@ -356,15 +356,21 @@ NJS_EXPORT njs_int_t njs_value_is_number NJS_EXPORT njs_int_t njs_value_is_valid_number(const njs_value_t *value); NJS_EXPORT njs_int_t njs_value_is_string(const njs_value_t *value); NJS_EXPORT njs_int_t njs_value_is_object(const njs_value_t *value); +NJS_EXPORT njs_int_t njs_value_is_array(const njs_value_t *value); NJS_EXPORT njs_int_t njs_value_is_function(const njs_value_t *value); NJS_EXPORT njs_int_t njs_vm_object_alloc(njs_vm_t *vm, njs_value_t *retval, ...); NJS_EXPORT njs_value_t *njs_vm_object_prop(njs_vm_t *vm, - const njs_value_t *value, const njs_str_t *key); + njs_value_t *value, const njs_str_t *key, njs_opaque_value_t *retval); NJS_EXPORT njs_int_t njs_vm_array_alloc(njs_vm_t *vm, njs_value_t *retval, uint32_t spare); +NJS_EXPORT njs_int_t njs_vm_array_length(njs_vm_t *vm, njs_value_t *value, + int64_t *length); +NJS_EXPORT njs_value_t *njs_vm_array_start(njs_vm_t *vm, njs_value_t *value); +NJS_EXPORT njs_value_t *njs_vm_array_prop(njs_vm_t *vm, + njs_value_t *value, int64_t index, njs_opaque_value_t *retval); NJS_EXPORT njs_value_t *njs_vm_array_push(njs_vm_t *vm, njs_value_t *value); NJS_EXPORT njs_int_t njs_vm_json_parse(njs_vm_t *vm, njs_value_t *args, diff -r 78b8cfd5b1e9 -r 9812e7d99e7c src/njs_value.c --- a/src/njs_value.c Fri Apr 03 16:06:55 2020 +0000 +++ b/src/njs_value.c Fri Apr 03 16:10:00 2020 +0000 @@ -465,6 +465,13 @@ njs_value_is_object(const njs_value_t *v njs_int_t +njs_value_is_array(const njs_value_t *value) +{ + return njs_is_array(value); +} + + +njs_int_t njs_value_is_function(const njs_value_t *value) { return njs_is_function(value); diff -r 78b8cfd5b1e9 -r 9812e7d99e7c src/njs_vm.c --- a/src/njs_vm.c Fri Apr 03 16:06:55 2020 +0000 +++ b/src/njs_vm.c Fri Apr 03 16:10:00 2020 +0000 @@ -826,7 +826,7 @@ njs_vm_array_alloc(njs_vm_t *vm, njs_val { njs_array_t *array; - array = njs_array_alloc(vm, 0, 0, spare); + array = njs_array_alloc(vm, 1, 0, spare); if (njs_slow_path(array == NULL)) { return NJS_ERROR; @@ -861,28 +861,82 @@ njs_vm_array_push(njs_vm_t *vm, njs_valu njs_value_t * -njs_vm_object_prop(njs_vm_t *vm, const njs_value_t *value, const njs_str_t *key) +njs_vm_object_prop(njs_vm_t *vm, njs_value_t *value, const njs_str_t *prop, + njs_opaque_value_t *retval) { - njs_int_t ret; - njs_object_prop_t *prop; - njs_lvlhsh_query_t lhq; + njs_int_t ret; + njs_value_t key; if (njs_slow_path(!njs_is_object(value))) { + njs_type_error(vm, "njs_vm_object_prop() argument is not object"); return NULL; } - lhq.key = *key; - lhq.key_hash = njs_djb_hash(lhq.key.start, lhq.key.length); - lhq.proto = &njs_object_hash_proto; + ret = njs_vm_value_string_set(vm, &key, prop->start, prop->length); + if (njs_slow_path(ret != NJS_OK)) { + return NULL; + } - ret = njs_lvlhsh_find(njs_object_hash(value), &lhq); + ret = njs_value_property(vm, value, &key, njs_value_arg(retval)); if (njs_slow_path(ret != NJS_OK)) { return NULL; } - prop = lhq.value; + return njs_value_arg(retval); +} + + +njs_value_t * +njs_vm_array_prop(njs_vm_t *vm, njs_value_t *value, int64_t index, + njs_opaque_value_t *retval) +{ + njs_int_t ret; + njs_array_t *array; + + if (njs_slow_path(!njs_is_object(value))) { + njs_type_error(vm, "njs_vm_array_prop() argument is not object"); + return NULL; + } + + if (njs_fast_path(njs_is_fast_array(value))) { + array = njs_array(value); + + if (index >= 0 && index < array->length) { + return &array->start[index]; + } + + return NULL; + } - return &prop->value; + ret = njs_value_property_i64(vm, value, index, njs_value_arg(retval)); + if (njs_slow_path(ret != NJS_OK)) { + return NULL; + } + + return njs_value_arg(retval); +} + + +njs_value_t * +njs_vm_array_start(njs_vm_t *vm, njs_value_t *value) +{ + if (njs_slow_path(!njs_is_fast_array(value))) { + njs_type_error(vm, "njs_vm_array_start() argument is not a fast array"); + return NULL; + } + + return njs_array(value)->start; +} + + +njs_int_t +njs_vm_array_length(njs_vm_t *vm, njs_value_t *value, int64_t *length) +{ + if (njs_fast_path(njs_is_array(value))) { + *length = njs_array(value)->length; + } + + return njs_object_length(vm, value, length); } From ru at nginx.com Tue Apr 7 22:04:40 2020 From: ru at nginx.com (Ruslan Ermilov) Date: Tue, 07 Apr 2020 22:04:40 +0000 Subject: [nginx] The new auth_delay directive for delaying unauthorized requests. Message-ID: details: https://hg.nginx.org/nginx/rev/681b78a98a52 branches: changeset: 7638:681b78a98a52 user: Ruslan Ermilov date: Wed Apr 08 01:02:17 2020 +0300 description: The new auth_delay directive for delaying unauthorized requests. The request processing is delayed by a timer. Since nginx updates internal time once at the start of each event loop iteration, this normally ensures constant time delay, adding a mitigation from time-based attacks. A notable exception to this is the case when there are no additional events before the timer expires. To ensure constant-time processing in this case as well, we trigger an additional event loop iteration by posting a dummy event for the next event loop iteration. diffstat: src/http/ngx_http_core_module.c | 82 ++++++++++++++++++++++++++++++++++++++++- src/http/ngx_http_core_module.h | 1 + 2 files changed, 82 insertions(+), 1 deletions(-) diffs (150 lines): diff -r 0cb942c1c1aa -r 681b78a98a52 src/http/ngx_http_core_module.c --- a/src/http/ngx_http_core_module.c Fri Mar 13 02:12:10 2020 +0300 +++ b/src/http/ngx_http_core_module.c Wed Apr 08 01:02:17 2020 +0300 @@ -21,6 +21,9 @@ typedef struct { #define NGX_HTTP_REQUEST_BODY_FILE_CLEAN 2 +static ngx_int_t ngx_http_core_auth_delay(ngx_http_request_t *r); +static void ngx_http_core_auth_delay_handler(ngx_http_request_t *r); + static ngx_int_t ngx_http_core_find_location(ngx_http_request_t *r); static ngx_int_t ngx_http_core_find_static_location(ngx_http_request_t *r, ngx_http_location_tree_node_t *node); @@ -520,6 +523,13 @@ static ngx_command_t ngx_http_core_comm offsetof(ngx_http_core_loc_conf_t, satisfy), &ngx_http_core_satisfy }, + { ngx_string("auth_delay"), + NGX_HTTP_MAIN_CONF|NGX_HTTP_SRV_CONF|NGX_HTTP_LOC_CONF|NGX_CONF_TAKE1, + ngx_conf_set_msec_slot, + NGX_HTTP_LOC_CONF_OFFSET, + offsetof(ngx_http_core_loc_conf_t, auth_delay), + NULL }, + { ngx_string("internal"), NGX_HTTP_LOC_CONF|NGX_CONF_NOARGS, ngx_http_core_internal, @@ -1124,6 +1134,10 @@ ngx_http_core_access_phase(ngx_http_requ /* rc == NGX_ERROR || rc == NGX_HTTP_... */ + if (rc == NGX_HTTP_UNAUTHORIZED) { + return ngx_http_core_auth_delay(r); + } + ngx_http_finalize_request(r, rc); return NGX_OK; } @@ -1141,12 +1155,17 @@ ngx_http_core_post_access_phase(ngx_http access_code = r->access_code; if (access_code) { + r->access_code = 0; + if (access_code == NGX_HTTP_FORBIDDEN) { ngx_log_error(NGX_LOG_ERR, r->connection->log, 0, "access forbidden by rule"); } - r->access_code = 0; + if (access_code == NGX_HTTP_UNAUTHORIZED) { + return ngx_http_core_auth_delay(r); + } + ngx_http_finalize_request(r, access_code); return NGX_OK; } @@ -1156,6 +1175,65 @@ ngx_http_core_post_access_phase(ngx_http } +static ngx_int_t +ngx_http_core_auth_delay(ngx_http_request_t *r) +{ + ngx_http_core_loc_conf_t *clcf; + + clcf = ngx_http_get_module_loc_conf(r, ngx_http_core_module); + + if (clcf->auth_delay == 0) { + ngx_http_finalize_request(r, NGX_HTTP_UNAUTHORIZED); + return NGX_OK; + } + + ngx_log_error(NGX_LOG_INFO, r->connection->log, 0, + "delaying unauthorized request"); + + if (ngx_handle_read_event(r->connection->read, 0) != NGX_OK) { + return NGX_HTTP_INTERNAL_SERVER_ERROR; + } + + r->read_event_handler = ngx_http_test_reading; + r->write_event_handler = ngx_http_core_auth_delay_handler; + + r->connection->write->delayed = 1; + ngx_add_timer(r->connection->write, clcf->auth_delay); + + /* + * trigger an additional event loop iteration + * to ensure constant-time processing + */ + + ngx_post_event(r->connection->write, &ngx_posted_next_events); + + return NGX_OK; +} + + +static void +ngx_http_core_auth_delay_handler(ngx_http_request_t *r) +{ + ngx_event_t *wev; + + ngx_log_debug0(NGX_LOG_DEBUG_HTTP, r->connection->log, 0, + "auth delay handler"); + + wev = r->connection->write; + + if (wev->delayed) { + + if (ngx_handle_write_event(wev, 0) != NGX_OK) { + ngx_http_finalize_request(r, NGX_HTTP_INTERNAL_SERVER_ERROR); + } + + return; + } + + ngx_http_finalize_request(r, NGX_HTTP_UNAUTHORIZED); +} + + ngx_int_t ngx_http_core_content_phase(ngx_http_request_t *r, ngx_http_phase_handler_t *ph) @@ -3394,6 +3472,7 @@ ngx_http_core_create_loc_conf(ngx_conf_t clcf->client_body_buffer_size = NGX_CONF_UNSET_SIZE; clcf->client_body_timeout = NGX_CONF_UNSET_MSEC; clcf->satisfy = NGX_CONF_UNSET_UINT; + clcf->auth_delay = NGX_CONF_UNSET_MSEC; clcf->if_modified_since = NGX_CONF_UNSET_UINT; clcf->max_ranges = NGX_CONF_UNSET_UINT; clcf->client_body_in_file_only = NGX_CONF_UNSET_UINT; @@ -3609,6 +3688,7 @@ ngx_http_core_merge_loc_conf(ngx_conf_t |NGX_HTTP_KEEPALIVE_DISABLE_MSIE6)); ngx_conf_merge_uint_value(conf->satisfy, prev->satisfy, NGX_HTTP_SATISFY_ALL); + ngx_conf_merge_msec_value(conf->auth_delay, prev->auth_delay, 0); ngx_conf_merge_uint_value(conf->if_modified_since, prev->if_modified_since, NGX_HTTP_IMS_EXACT); ngx_conf_merge_uint_value(conf->max_ranges, prev->max_ranges, diff -r 0cb942c1c1aa -r 681b78a98a52 src/http/ngx_http_core_module.h --- a/src/http/ngx_http_core_module.h Fri Mar 13 02:12:10 2020 +0300 +++ b/src/http/ngx_http_core_module.h Wed Apr 08 01:02:17 2020 +0300 @@ -363,6 +363,7 @@ struct ngx_http_core_loc_conf_s { ngx_msec_t lingering_time; /* lingering_time */ ngx_msec_t lingering_timeout; /* lingering_timeout */ ngx_msec_t resolver_timeout; /* resolver_timeout */ + ngx_msec_t auth_delay; /* auth_delay */ ngx_resolver_t *resolver; /* resolver */ From xeioex at nginx.com Wed Apr 8 13:15:57 2020 From: xeioex at nginx.com (Dmitry Volyntsev) Date: Wed, 08 Apr 2020 13:15:57 +0000 Subject: [njs] Linking global variables and global object properties. Message-ID: details: https://hg.nginx.org/njs/rev/7ccb8b32cc02 branches: changeset: 1367:7ccb8b32cc02 user: Dmitry Volyntsev date: Wed Apr 08 13:15:02 2020 +0000 description: Linking global variables and global object properties. diffstat: src/njs.h | 3 +- src/njs_array.c | 4 + src/njs_builtin.c | 65 +++++++++++++++ src/njs_object_prop.c | 37 +++++--- src/njs_variable.c | 47 ----------- src/njs_vm.c | 59 ++++++++++++++ src/njs_vm.h | 2 + src/test/njs_unit_test.c | 193 ++++++++++++++++++++++++++++++++++++++++++++++- 8 files changed, 345 insertions(+), 65 deletions(-) diffs (578 lines): diff -r 9812e7d99e7c -r 7ccb8b32cc02 src/njs.h --- a/src/njs.h Fri Apr 03 16:10:00 2020 +0000 +++ b/src/njs.h Wed Apr 08 13:15:02 2020 +0000 @@ -294,7 +294,8 @@ NJS_EXPORT void njs_disassemble(u_char * NJS_EXPORT njs_int_t njs_vm_bind(njs_vm_t *vm, const njs_str_t *var_name, const njs_value_t *value, njs_bool_t shared); -NJS_EXPORT const njs_value_t *njs_vm_value(njs_vm_t *vm, const njs_str_t *name); +NJS_EXPORT njs_int_t njs_vm_value(njs_vm_t *vm, const njs_str_t *path, + njs_value_t *retval); NJS_EXPORT njs_function_t *njs_vm_function(njs_vm_t *vm, const njs_str_t *name); NJS_EXPORT njs_value_t *njs_vm_retval(njs_vm_t *vm); diff -r 9812e7d99e7c -r 7ccb8b32cc02 src/njs_array.c --- a/src/njs_array.c Fri Apr 03 16:10:00 2020 +0000 +++ b/src/njs_array.c Wed Apr 08 13:15:02 2020 +0000 @@ -519,6 +519,10 @@ njs_array_length(njs_vm_t *vm,njs_object return NJS_DECLINED; } + if (njs_slow_path(!njs_is_valid(setval))) { + return NJS_DECLINED; + } + ret = njs_value_to_number(vm, setval, &num); if (njs_slow_path(ret != NJS_OK)) { return ret; diff -r 9812e7d99e7c -r 7ccb8b32cc02 src/njs_builtin.c --- a/src/njs_builtin.c Fri Apr 03 16:10:00 2020 +0000 +++ b/src/njs_builtin.c Wed Apr 08 13:15:02 2020 +0000 @@ -22,6 +22,9 @@ typedef struct { } njs_builtin_traverse_t; +static njs_int_t njs_global_this_prop_handler(njs_vm_t *vm, + njs_object_prop_t *self, njs_value_t *global, njs_value_t *setval, + njs_value_t *retval); static njs_arr_t *njs_vm_expression_completions(njs_vm_t *vm, njs_str_t *expression); static njs_arr_t *njs_object_completions(njs_vm_t *vm, njs_value_t *object); @@ -288,6 +291,13 @@ njs_builtin_objects_create(njs_vm_t *vm) } } + shared->global_slots.prop_handler = njs_global_this_prop_handler; + shared->global_slots.writable = 1; + shared->global_slots.configurable = 1; + shared->global_slots.enumerable = 1; + + shared->objects[0].slots = &shared->global_slots; + vm->global_object = shared->objects[0]; vm->global_object.shared = 0; @@ -778,6 +788,50 @@ njs_dump_value(njs_vm_t *vm, njs_value_t static njs_int_t +njs_global_this_prop_handler(njs_vm_t *vm, njs_object_prop_t *prop, + njs_value_t *global, njs_value_t *setval, njs_value_t *retval) +{ + njs_int_t ret; + njs_value_t *value; + njs_rbtree_node_t *rb_node; + njs_lvlhsh_query_t lhq; + njs_variable_node_t *node, var_node; + + if (retval == NULL) { + return NJS_DECLINED; + } + + njs_string_get(&prop->name, &lhq.key); + lhq.key_hash = njs_djb_hash(lhq.key.start, lhq.key.length); + lhq.proto = &njs_lexer_hash_proto; + + ret = njs_lvlhsh_find(&vm->shared->keywords_hash, &lhq); + + if (njs_slow_path(ret != NJS_OK || lhq.value == NULL)) { + return NJS_DECLINED; + } + + var_node.key = (uintptr_t) lhq.value; + + rb_node = njs_rbtree_find(vm->variables_hash, &var_node.node); + if (rb_node == NULL) { + return NJS_DECLINED; + } + + node = (njs_variable_node_t *) rb_node; + value = njs_vmcode_operand(vm, node->variable->index); + + if (setval != NULL) { + *value = *setval; + } + + *retval = *value; + + return NJS_OK; +} + + +static njs_int_t njs_global_this_object(njs_vm_t *vm, njs_object_prop_t *self, njs_value_t *global, njs_value_t *setval, njs_value_t *retval) { @@ -789,6 +843,9 @@ njs_global_this_object(njs_vm_t *vm, njs if (njs_slow_path(setval != NULL)) { *retval = *setval; + + } else if (njs_slow_path(retval == NULL)) { + return NJS_DECLINED; } prop = njs_object_prop_alloc(vm, &self->name, retval, 1); @@ -831,6 +888,10 @@ njs_top_level_object(njs_vm_t *vm, njs_o *retval = *setval; } else { + if (njs_slow_path(retval == NULL)) { + return NJS_DECLINED; + } + njs_set_object(retval, &vm->shared->objects[self->value.data.magic16]); object = njs_object_value_copy(vm, retval); @@ -879,6 +940,10 @@ njs_top_level_constructor(njs_vm_t *vm, *retval = *setval; } else { + if (njs_slow_path(retval == NULL)) { + return NJS_DECLINED; + } + ctor = &vm->constructors[self->value.data.magic16]; njs_set_function(retval, ctor); diff -r 9812e7d99e7c -r 7ccb8b32cc02 src/njs_object_prop.c --- a/src/njs_object_prop.c Fri Apr 03 16:10:00 2020 +0000 +++ b/src/njs_object_prop.c Wed Apr 08 13:15:02 2020 +0000 @@ -196,6 +196,8 @@ again: if (njs_fast_path(ret == NJS_DECLINED)) { +set_prop: + if (!njs_object(object)->extensible) { njs_key_string_get(vm, &pq.key, &pq.lhq.key); njs_type_error(vm, "Cannot add property \"%V\", " @@ -412,28 +414,20 @@ again: done: - /* - * 9. For each field of Desc that is present, set the corresponding - * attribute of the property named P of object O to the value of the field. - */ - - if (njs_is_valid(&prop->getter)) { - prev->getter = prop->getter; - } - - if (njs_is_valid(&prop->setter)) { - prev->setter = prop->setter; - } - - if (njs_is_valid(&prop->value)) { + if (njs_is_valid(&prop->value) || njs_is_accessor_descriptor(prop)) { if (prev->type == NJS_PROPERTY_HANDLER) { - if (njs_is_data_descriptor(prev) && prev->writable) { + if (prev->writable) { ret = prev->value.data.u.prop_handler(vm, prev, object, &prop->value, &vm->retval); if (njs_slow_path(ret == NJS_ERROR)) { return ret; } + + if (ret == NJS_DECLINED) { + pq.lhq.value = NULL; + goto set_prop; + } } } else { @@ -450,6 +444,19 @@ done: } } + /* + * 9. For each field of Desc that is present, set the corresponding + * attribute of the property named P of object O to the value of the field. + */ + + if (njs_is_valid(&prop->getter)) { + prev->getter = prop->getter; + } + + if (njs_is_valid(&prop->setter)) { + prev->setter = prop->setter; + } + if (prop->writable != NJS_ATTRIBUTE_UNSET) { prev->writable = prop->writable; } diff -r 9812e7d99e7c -r 7ccb8b32cc02 src/njs_variable.c --- a/src/njs_variable.c Fri Apr 03 16:10:00 2020 +0000 +++ b/src/njs_variable.c Wed Apr 08 13:15:02 2020 +0000 @@ -587,50 +587,3 @@ njs_name_copy(njs_vm_t *vm, njs_str_t *d return NJS_ERROR; } - - -const njs_value_t * -njs_vm_value(njs_vm_t *vm, const njs_str_t *name) -{ - njs_int_t ret; - njs_rbtree_node_t *rb_node; - njs_lvlhsh_query_t lhq; - njs_variable_node_t *node, var_node; - - lhq.key = *name; - lhq.key_hash = njs_djb_hash(name->start, name->length); - lhq.proto = &njs_lexer_hash_proto; - - ret = njs_lvlhsh_find(&vm->shared->keywords_hash, &lhq); - - if (njs_slow_path(ret != NJS_OK || lhq.value == NULL)) { - return &njs_value_undefined; - } - - var_node.key = (uintptr_t) lhq.value; - - rb_node = njs_rbtree_find(vm->variables_hash, &var_node.node); - - if (rb_node != NULL) { - node = (njs_variable_node_t *) rb_node; - - return njs_vmcode_operand(vm, node->variable->index); - } - - return &njs_value_undefined; -} - - -njs_function_t * -njs_vm_function(njs_vm_t *vm, const njs_str_t *name) -{ - const njs_value_t *value; - - value = njs_vm_value(vm, name); - - if (njs_is_function(value)) { - return njs_function(value); - } - - return NULL; -} diff -r 9812e7d99e7c -r 7ccb8b32cc02 src/njs_vm.c --- a/src/njs_vm.c Fri Apr 03 16:10:00 2020 +0000 +++ b/src/njs_vm.c Wed Apr 08 13:15:02 2020 +0000 @@ -580,6 +580,50 @@ njs_vm_retval_set(njs_vm_t *vm, const nj njs_int_t +njs_vm_value(njs_vm_t *vm, const njs_str_t *path, njs_value_t *retval) +{ + u_char *start, *p, *end; + size_t size; + njs_int_t ret; + njs_value_t value, key; + + start = path->start; + end = start + path->length; + + njs_set_object(&value, &vm->global_object); + + for ( ;; ) { + p = njs_strchr(start, '.'); + + size = ((p != NULL) ? p : end) - start; + if (njs_slow_path(size == 0)) { + njs_type_error(vm, "empty path element"); + return NJS_ERROR; + } + + ret = njs_string_set(vm, &key, start, size); + if (njs_slow_path(ret != NJS_OK)) { + return NJS_ERROR; + } + + ret = njs_value_property(vm, &value, &key, njs_value_arg(retval)); + if (njs_slow_path(ret == NJS_ERROR)) { + return NJS_ERROR; + } + + if (p == NULL) { + break; + } + + start = p + 1; + value = *retval; + } + + return NJS_OK; +} + + +njs_int_t njs_vm_bind(njs_vm_t *vm, const njs_str_t *var_name, const njs_value_t *value, njs_bool_t shared) { @@ -634,6 +678,21 @@ njs_vm_value_string_alloc(njs_vm_t *vm, } +njs_function_t * +njs_vm_function(njs_vm_t *vm, const njs_str_t *path) +{ + njs_int_t ret; + njs_value_t retval; + + ret = njs_vm_value(vm, path, &retval); + if (njs_slow_path(ret != NJS_OK || !njs_is_function(&retval))) { + return NULL; + } + + return njs_function(&retval); +} + + uint16_t njs_vm_prop_magic16(njs_object_prop_t *prop) { diff -r 9812e7d99e7c -r 7ccb8b32cc02 src/njs_vm.h --- a/src/njs_vm.h Fri Apr 03 16:10:00 2020 +0000 +++ b/src/njs_vm.h Wed Apr 08 13:15:02 2020 +0000 @@ -272,6 +272,8 @@ struct njs_vm_shared_s { njs_object_t string_object; njs_object_t objects[NJS_OBJECT_MAX]; + njs_exotic_slots_t global_slots; + /* * The prototypes and constructors arrays must be togther because they are * copied to njs_vm_t by single memcpy() in njs_builtin_objects_clone(). diff -r 9812e7d99e7c -r 7ccb8b32cc02 src/test/njs_unit_test.c --- a/src/test/njs_unit_test.c Fri Apr 03 16:10:00 2020 +0000 +++ b/src/test/njs_unit_test.c Wed Apr 08 13:15:02 2020 +0000 @@ -10174,6 +10174,41 @@ static njs_unit_test_t njs_test[] = { njs_str("this.a = ()=>1; a()"), njs_str("1") }, + { njs_str("var a = 1; this.a = 42; a"), + njs_str("42") }, + + { njs_str("var a = 1; global.a = 42; a"), + njs_str("42") }, + + { njs_str("var a = 1; globalThis.a = 42; a"), + njs_str("42") }, + + { njs_str("global.a = 1; globalThis.a"), + njs_str("1") }, + + { njs_str("var a = {}; globalThis.a === a"), + njs_str("true") }, + + { njs_str("globalThis.a = 1; var a; a"), + njs_str("1") }, + + { njs_str("var count = 0; function f() {return ++count}; [f(), global.f()]"), + njs_str("1,2") }, + + { njs_str("Object.defineProperty(global, 'a', {value:1}); a"), + njs_str("1") }, + + { njs_str("Object.defineProperty(global, 'a', {get:()=>123}); a"), + njs_str("123") }, + + { njs_str("Object.defineProperties(global, {a:{value:1}, b:{value:2}}); [a,b]"), + njs_str("1,2") }, + +#if 0 /* FIXME: for scope. */ + { njs_str("var r1 = global.a; for (var a = 1; false;) {}; [r1, global.a]"), + njs_str(",") }, +#endif + { njs_str("var global = this;" "function isImmutableConstant(v) {" " var d = Object.getOwnPropertyDescriptor(global, v);" @@ -11455,6 +11490,10 @@ static njs_unit_test_t njs_test[] = { njs_str("(new Function('return this'))() === globalThis"), njs_str("true") }, + { njs_str("var f = Function.call(this, 'return this.a');" + "var r1 = f(); var a = 1; var r2 = f(); [r1,r2]"), + njs_str(",1") }, + { njs_str("(new Function('a', 'return a')).length"), njs_str("1") }, @@ -17373,8 +17412,8 @@ njs_vm_json_test(njs_opts_t *opts, njs_s } args[0] = vm->retval; - args[1] = *njs_vm_value(vm, &fname); - args[2] = *njs_vm_value(vm, &iname); + njs_vm_value(vm, &fname, &args[1]); + njs_vm_value(vm, &iname, &args[2]); ret = njs_vm_json_stringify(vm, args, 3); if (ret != NJS_OK) { @@ -17429,6 +17468,151 @@ done: static njs_int_t +njs_vm_value_test(njs_opts_t *opts, njs_stat_t *stat) +{ + njs_vm_t *vm; + njs_int_t ret; + njs_str_t s, *script; + njs_uint_t i; + njs_bool_t success; + njs_stat_t prev; + njs_vm_opt_t options; + + static struct { + njs_str_t script; + njs_str_t path; + njs_str_t ret; + } tests[] = { + { + .script = njs_str("var o = {a:1}"), + .path = njs_str("o.a"), + .ret = njs_str("1"), + }, + + { + .script = njs_str("var aaaaabbbbbcccccddddd = {e:2}"), + .path = njs_str("aaaaabbbbbcccccddddd.e"), + .ret = njs_str("2"), + }, + + { + .script = njs_str("var o = {a:{b:3}}"), + .path = njs_str("o.a.b"), + .ret = njs_str("3"), + }, + + { + .script = njs_str("var o = 1"), + .path = njs_str("o.a"), + .ret = njs_str("undefined"), + }, + + { + .script = njs_str(""), + .path = njs_str("o"), + .ret = njs_str("undefined"), + }, + + { + .script = njs_str("var o = {'':1}"), + .path = njs_str("."), + .ret = njs_str("TypeError: empty path element"), + }, + + { + .script = njs_str("var o = {'':1}"), + .path = njs_str("o."), + .ret = njs_str("TypeError: empty path element"), + }, + { + .script = njs_str("var o = {'':1}"), + .path = njs_str("o.."), + .ret = njs_str("TypeError: empty path element"), + }, + }; + + vm = NULL; + + prev = *stat; + + ret = NJS_ERROR; + + for (i = 0; i < njs_nitems(tests); i++) { + + memset(&options, 0, sizeof(njs_vm_opt_t)); + options.init = 1; + + vm = njs_vm_create(&options); + if (vm == NULL) { + njs_printf("njs_vm_create() failed\n"); + goto done; + } + + script = &tests[i].script; + + ret = njs_vm_compile(vm, &script->start, + script->start + script->length); + + if (ret != NJS_OK) { + njs_printf("njs_vm_compile() failed\n"); + goto done; + } + + ret = njs_vm_start(vm); + if (ret != NJS_OK) { + njs_printf("njs_vm_run() failed\n"); + goto done; + } + + ret = njs_vm_value(vm, &tests[i].path, &vm->retval); + + if (njs_vm_retval_string(vm, &s) != NJS_OK) { + njs_printf("njs_vm_retval_string() failed\n"); + goto done; + } + + success = njs_strstr_eq(&tests[i].ret, &s); + + if (!success) { + njs_printf("njs_vm_value_test(\"%V\")\n" + "expected: \"%V\"\n got: \"%V\"\n", script, + &tests[i].ret, &s); + + stat->failed++; + + } else { + stat->passed++; + } + + njs_vm_destroy(vm); + vm = NULL; + + } + + ret = NJS_OK; + +done: + + if (ret != NJS_OK) { + if (njs_vm_retval_string(vm, &s) != NJS_OK) { + njs_printf("njs_vm_retval_string() failed\n"); + + } else { + njs_printf("%V\n", &s); + } + } + + njs_unit_test_report("njs_vm_value() tests", &prev, stat); + + if (vm != NULL) { + njs_vm_destroy(vm); + } + + return ret; +} + + +static njs_int_t njs_vm_object_alloc_test(njs_vm_t *vm, njs_opts_t *opts, njs_stat_t *stat) { njs_int_t ret; @@ -17931,6 +18115,11 @@ main(int argc, char **argv) return ret; } + ret = njs_vm_value_test(&opts, &stat); + if (ret != NJS_OK) { + return ret; + } + ret = njs_api_test(&opts, &stat); if (ret != NJS_OK) { return ret; From sander at hoentjen.eu Wed Apr 8 18:26:49 2020 From: sander at hoentjen.eu (Sander Hoentjen) Date: Wed, 8 Apr 2020 20:26:49 +0200 Subject: [PATCH] Added support for proxying managesieve protocol Message-ID: <58016a3a-c11d-bb31-2b05-916cb8124598@hoentjen.eu> # HG changeset patch # User Sander Hoentjen # Date 1586369831 -7200 # Wed Apr 08 20:17:11 2020 +0200 # Node ID f1dffaf619688aaab90caf31781ebe27c3f79598 # Parent 0cb942c1c1aa98118076e72e0b89940e85e6291c Added support for proxying managesieve protocol Sieve is a widely used protocol to implement server-side filtering. Some servers and clients that support this are listed at . This commit is tested with Dovecot as the server and Roundcube as the client, but I don't see a readon why it not would work with others. It is a nice addition to nginx because it adds the possibility to add server side filtering to setups that use IMAP behind an nginx proxy. Fixes https://trac.nginx.org/nginx/ticket/1697 diff -r 0cb942c1c1aa -r f1dffaf61968 auto/modules --- a/auto/modules Fri Mar 13 02:12:10 2020 +0300 +++ b/auto/modules Wed Apr 08 20:17:11 2020 +0200 @@ -965,6 +965,15 @@ . auto/module fi + if [ $MAIL_SIEVE = YES ]; then + ngx_module_name=ngx_mail_sieve_module + ngx_module_deps=src/mail/ngx_mail_sieve_module.h + ngx_module_srcs="src/mail/ngx_mail_sieve_module.c \ + src/mail/ngx_mail_sieve_handler.c" + + . auto/module + fi + if [ $MAIL_SMTP = YES ]; then ngx_module_name=ngx_mail_smtp_module ngx_module_deps=src/mail/ngx_mail_smtp_module.h diff -r 0cb942c1c1aa -r f1dffaf61968 auto/options --- a/auto/options Fri Mar 13 02:12:10 2020 +0300 +++ b/auto/options Wed Apr 08 20:17:11 2020 +0200 @@ -112,6 +112,7 @@ MAIL_SSL=NO MAIL_POP3=YES MAIL_IMAP=YES +MAIL_SIEVE=YES MAIL_SMTP=YES STREAM=NO @@ -305,6 +306,8 @@ ;; --without-mail_pop3_module) MAIL_POP3=NO ;; --without-mail_imap_module) MAIL_IMAP=NO ;; + --without-mail_sieve_module) + MAIL_SIEVE=NO ;; --without-mail_smtp_module) MAIL_SMTP=NO ;; --with-stream) STREAM=YES ;; @@ -517,11 +520,12 @@ --without-http disable HTTP server --without-http-cache disable HTTP cache - --with-mail enable POP3/IMAP4/SMTP proxy module - --with-mail=dynamic enable dynamic POP3/IMAP4/SMTP proxy module + --with-mail enable POP3/IMAP4/SIEVE/SMTP proxy module + --with-mail=dynamic enable dynamic POP3/IMAP4/SIEVE/SMTP proxy module --with-mail_ssl_module enable ngx_mail_ssl_module --without-mail_pop3_module disable ngx_mail_pop3_module --without-mail_imap_module disable ngx_mail_imap_module + --without-mail_sieve_module disable ngx_mail_sieve_module --without-mail_smtp_module disable ngx_mail_smtp_module --with-stream enable TCP/UDP proxy module diff -r 0cb942c1c1aa -r f1dffaf61968 contrib/vim/syntax/nginx.vim --- a/contrib/vim/syntax/nginx.vim Fri Mar 13 02:12:10 2020 +0300 +++ b/contrib/vim/syntax/nginx.vim Wed Apr 08 20:17:11 2020 +0200 @@ -571,6 +571,9 @@ syn keyword ngxDirective contained session_log_format syn keyword ngxDirective contained session_log_zone syn keyword ngxDirective contained set_real_ip_from +syn keyword ngxDirective contained sieve_auth +syn keyword ngxDirective contained sieve_capabilities +syn keyword ngxDirective contained sieve_client_buffer syn keyword ngxDirective contained slice syn keyword ngxDirective contained smtp_auth syn keyword ngxDirective contained smtp_capabilities diff -r 0cb942c1c1aa -r f1dffaf61968 src/mail/ngx_mail.h --- a/src/mail/ngx_mail.h Fri Mar 13 02:12:10 2020 +0300 +++ b/src/mail/ngx_mail.h Wed Apr 08 20:17:11 2020 +0200 @@ -102,6 +102,7 @@ #define NGX_MAIL_POP3_PROTOCOL 0 #define NGX_MAIL_IMAP_PROTOCOL 1 #define NGX_MAIL_SMTP_PROTOCOL 2 +#define NGX_MAIL_SIEVE_PROTOCOL 3 typedef struct ngx_mail_protocol_s ngx_mail_protocol_t; @@ -154,6 +155,21 @@ typedef enum { + ngx_sieve_start = 0, + ngx_sieve_starttls, + ngx_sieve_auth_login_username, + ngx_sieve_auth_login_password, + ngx_sieve_auth_plain, + ngx_sieve_auth_cram_md5, + ngx_sieve_auth_external, + ngx_sieve_login, + ngx_sieve_login_capabilities, + ngx_sieve_user, + ngx_sieve_passwd +} ngx_sieve_state_e; + + +typedef enum { ngx_smtp_start = 0, ngx_smtp_auth_login_username, ngx_smtp_auth_login_password, @@ -227,7 +243,7 @@ ngx_uint_t login_attempt; - /* used to parse POP3/IMAP/SMTP command */ + /* used to parse POP3/IMAP/SIEVE/SMTP command */ ngx_uint_t state; u_char *cmd_start; @@ -271,6 +287,14 @@ #define NGX_IMAP_AUTHENTICATE 7 +#define NGX_SIEVE_LOGOUT 1 +#define NGX_SIEVE_CAPABILITY 2 +#define NGX_SIEVE_NOOP 3 +#define NGX_SIEVE_STARTTLS 4 +#define NGX_SIEVE_NEXT 5 +#define NGX_SIEVE_AUTHENTICATE 6 + + #define NGX_SMTP_HELO 1 #define NGX_SMTP_EHLO 2 #define NGX_SMTP_AUTH 3 @@ -383,7 +407,7 @@ ngx_int_t ngx_mail_auth_login_password(ngx_mail_session_t *s, ngx_connection_t *c); ngx_int_t ngx_mail_auth_cram_md5_salt(ngx_mail_session_t *s, - ngx_connection_t *c, char *prefix, size_t len); + ngx_connection_t *c, char *prefix, size_t len, ngx_uint_t quote); ngx_int_t ngx_mail_auth_cram_md5(ngx_mail_session_t *s, ngx_connection_t *c); ngx_int_t ngx_mail_auth_external(ngx_mail_session_t *s, ngx_connection_t *c, ngx_uint_t n); diff -r 0cb942c1c1aa -r f1dffaf61968 src/mail/ngx_mail_auth_http_module.c --- a/src/mail/ngx_mail_auth_http_module.c Fri Mar 13 02:12:10 2020 +0300 +++ b/src/mail/ngx_mail_auth_http_module.c Wed Apr 08 20:17:11 2020 +0200 @@ -533,6 +533,11 @@ + sizeof(CRLF) - 1; break; + case NGX_MAIL_SIEVE_PROTOCOL: + size = sizeof("BYE \"\"") - 1 + len + + sizeof(CRLF) - 1; + break; + default: /* NGX_MAIL_SMTP_PROTOCOL */ ctx->err = ctx->errmsg; continue; @@ -559,11 +564,27 @@ *p++ = 'N'; *p++ = 'O'; *p++ = ' '; break; + case NGX_MAIL_SIEVE_PROTOCOL: + p = ngx_cpymem(p, s->tag.data, s->tag.len); + *p++ = 'B'; *p++ = 'Y'; *p++ = 'E'; *p++ = ' '; *p++ = '"'; + break; + default: /* NGX_MAIL_SMTP_PROTOCOL */ break; } p = ngx_cpymem(p, ctx->header_start, len); + + switch (s->protocol) { + + case NGX_MAIL_SIEVE_PROTOCOL: + *p++ = '"'; + break; + + default: + break; + } + *p++ = CR; *p++ = LF; ctx->err.len = p - ctx->err.data; diff -r 0cb942c1c1aa -r f1dffaf61968 src/mail/ngx_mail_handler.c --- a/src/mail/ngx_mail_handler.c Fri Mar 13 02:12:10 2020 +0300 +++ b/src/mail/ngx_mail_handler.c Wed Apr 08 20:17:11 2020 +0200 @@ -520,28 +520,39 @@ ngx_int_t ngx_mail_auth_cram_md5_salt(ngx_mail_session_t *s, ngx_connection_t *c, - char *prefix, size_t len) + char *prefix, size_t len, ngx_uint_t quote) { u_char *p; ngx_str_t salt; - ngx_uint_t n; + ngx_uint_t n = 0; + if (quote) { + len += 2; + } p = ngx_pnalloc(c->pool, len + ngx_base64_encoded_length(s->salt.len) + 2); if (p == NULL) { return NGX_ERROR; } - salt.data = ngx_cpymem(p, prefix, len); + if (quote) { + len -= 2; + salt.data = ngx_cpymem(p, "\"", 1); + n++; + } + salt.data = ngx_cpymem(p+ n, prefix, len); s->salt.len -= 2; ngx_encode_base64(&salt, &s->salt); s->salt.len += 2; - n = len + salt.len; + n += len + salt.len; + if (quote) { + p[n++] = '"'; + } p[n++] = CR; p[n++] = LF; s->out.len = n; - s->out.data = p; + s->out.data = p--; return NGX_OK; } diff -r 0cb942c1c1aa -r f1dffaf61968 src/mail/ngx_mail_imap_handler.c --- a/src/mail/ngx_mail_imap_handler.c Fri Mar 13 02:12:10 2020 +0300 +++ b/src/mail/ngx_mail_imap_handler.c Wed Apr 08 20:17:11 2020 +0200 @@ -397,7 +397,7 @@ } } - if (ngx_mail_auth_cram_md5_salt(s, c, "+ ", 2) == NGX_OK) { + if (ngx_mail_auth_cram_md5_salt(s, c, "+ ", 2, 0) == NGX_OK) { s->mail_state = ngx_imap_auth_cram_md5; return NGX_OK; } diff -r 0cb942c1c1aa -r f1dffaf61968 src/mail/ngx_mail_parse.c --- a/src/mail/ngx_mail_parse.c Fri Mar 13 02:12:10 2020 +0300 +++ b/src/mail/ngx_mail_parse.c Wed Apr 08 20:17:11 2020 +0200 @@ -11,6 +11,7 @@ #include #include #include +#include #include @@ -620,6 +621,364 @@ ngx_int_t +ngx_mail_sieve_parse_command(ngx_mail_session_t *s) +{ + u_char ch, *p, *c; + ngx_str_t *arg; + enum { + sw_start = 0, + sw_spaces_before_argument, + sw_argument, + sw_backslash, + sw_literal, + sw_no_sync_literal_argument, + sw_start_literal_argument, + sw_literal_argument, + sw_end_literal_argument, + sw_almost_done + } state; + + state = s->state; + + for (p = s->buffer->pos; p < s->buffer->last; p++) { + ch = *p; + + switch (state) { + + /* SIEVE command */ + case sw_start: + if (ch == ' ' || ch == CR || ch == LF) { + + c = s->buffer->start; + + switch (p - c) { + + case 4: + if ((c[0] == 'N' || c[0] == 'n') + && (c[1] == 'O'|| c[1] == 'o') + && (c[2] == 'O'|| c[2] == 'o') + && (c[3] == 'P'|| c[3] == 'p')) + { + s->command = NGX_SIEVE_NOOP; + + } else { + goto invalid; + } + break; + + case 6: + if ((c[0] == 'L'|| c[0] == 'l') + && (c[1] == 'O'|| c[1] == 'o') + && (c[2] == 'G'|| c[2] == 'g') + && (c[3] == 'O'|| c[3] == 'o') + && (c[4] == 'U'|| c[4] == 'u') + && (c[5] == 'T'|| c[5] == 't')) + { + s->command = NGX_SIEVE_LOGOUT; + + } else { + goto invalid; + } + break; + +#if (NGX_MAIL_SSL) + case 8: + if ((c[0] == 'S'|| c[0] == 's') + && (c[1] == 'T'|| c[1] == 't') + && (c[2] == 'A'|| c[2] == 'a') + && (c[3] == 'R'|| c[3] == 'r') + && (c[4] == 'T'|| c[4] == 't') + && (c[5] == 'T'|| c[5] == 't') + && (c[6] == 'L'|| c[6] == 'l') + && (c[7] == 'S'|| c[7] == 's')) + { + s->command = NGX_SIEVE_STARTTLS; + + } else { + goto invalid; + } + break; +#endif + + case 10: + if ((c[0] == 'C'|| c[0] == 'c') + && (c[1] == 'A'|| c[1] == 'a') + && (c[2] == 'P'|| c[2] == 'p') + && (c[3] == 'A'|| c[3] == 'a') + && (c[4] == 'B'|| c[4] == 'b') + && (c[5] == 'I'|| c[5] == 'i') + && (c[6] == 'L'|| c[6] == 'l') + && (c[7] == 'I'|| c[7] == 'i') + && (c[8] == 'T'|| c[8] == 't') + && (c[9] == 'Y'|| c[9] == 'y')) + { + s->command = NGX_SIEVE_CAPABILITY; + + } else { + goto invalid; + } + break; + + case 12: + if ((c[0] == 'A'|| c[0] == 'a') + && (c[1] == 'U'|| c[1] == 'u') + && (c[2] == 'T'|| c[2] == 't') + && (c[3] == 'H'|| c[3] == 'h') + && (c[4] == 'E'|| c[4] == 'e') + && (c[5] == 'N'|| c[5] == 'n') + && (c[6] == 'T'|| c[6] == 't') + && (c[7] == 'I'|| c[7] == 'i') + && (c[8] == 'C'|| c[8] == 'c') + && (c[9] == 'A'|| c[9] == 'a') + && (c[10] == 'T'|| c[10] == 't') + && (c[11] == 'E'|| c[11] == 'e')) + { + s->command = NGX_SIEVE_AUTHENTICATE; + + } else { + goto invalid; + } + break; + + default: + goto invalid; + } + + switch (ch) { + case ' ': + state = sw_spaces_before_argument; + break; + case CR: + state = sw_almost_done; + break; + case LF: + goto done; + } + break; + } + + if ((ch < 'A' || ch > 'Z') && (ch < 'a' || ch > 'z')) { + goto invalid; + } + + break; + + case sw_spaces_before_argument: + switch (ch) { + case ' ': + break; + case CR: + state = sw_almost_done; + s->arg_end = p; + break; + case LF: + s->arg_end = p; + goto done; + case '"': + if (s->args.nelts <= 2) { + s->quoted = 1; + s->arg_start = p + 1; + state = sw_argument; + break; + } + goto invalid; + case '{': + if (s->args.nelts <= 2) { + state = sw_literal; + break; + } + goto invalid; + default: + if (s->args.nelts <= 2) { + s->arg_start = p; + state = sw_argument; + break; + } + goto invalid; + } + break; + + case sw_argument: + if (ch == ' ' && s->quoted) { + break; + } + + switch (ch) { + case '"': + if (!s->quoted) { + break; + } + s->quoted = 0; + /* fall through */ + case ' ': + case CR: + case LF: + arg = ngx_array_push(&s->args); + if (arg == NULL) { + return NGX_ERROR; + } + arg->len = p - s->arg_start; + arg->data = s->arg_start; + s->arg_start = NULL; + + switch (ch) { + case '"': + case ' ': + state = sw_spaces_before_argument; + break; + case CR: + state = sw_almost_done; + break; + case LF: + goto done; + } + break; + case '\\': + if (s->quoted) { + s->backslash = 1; + state = sw_backslash; + } + break; + } + break; + + case sw_backslash: + switch (ch) { + case CR: + case LF: + goto invalid; + default: + state = sw_argument; + } + break; + + case sw_literal: + if (ch >= '0' && ch <= '9') { + s->literal_len = s->literal_len * 10 + (ch - '0'); + break; + } + if (ch == '}') { + state = sw_start_literal_argument; + break; + } + if (ch == '+') { + state = sw_no_sync_literal_argument; + break; + } + goto invalid; + + case sw_no_sync_literal_argument: + if (ch == '}') { + s->no_sync_literal = 1; + state = sw_start_literal_argument; + break; + } + goto invalid; + + case sw_start_literal_argument: + switch (ch) { + case CR: + break; + case LF: + s->buffer->pos = p + 1; + s->arg_start = p + 1; + if (s->no_sync_literal == 0) { + s->state = sw_literal_argument; + return NGX_SIEVE_NEXT; + } + state = sw_literal_argument; + s->no_sync_literal = 0; + break; + default: + goto invalid; + } + break; + + case sw_literal_argument: + if (s->literal_len && --s->literal_len) { + break; + } + + arg = ngx_array_push(&s->args); + if (arg == NULL) { + return NGX_ERROR; + } + arg->len = p + 1 - s->arg_start; + arg->data = s->arg_start; + s->arg_start = NULL; + state = sw_end_literal_argument; + + break; + + case sw_end_literal_argument: + switch (ch) { + case '{': + if (s->args.nelts <= 2) { + state = sw_literal; + break; + } + goto invalid; + case CR: + state = sw_almost_done; + break; + case LF: + goto done; + default: + state = sw_spaces_before_argument; + break; + } + break; + + case sw_almost_done: + switch (ch) { + case LF: + goto done; + default: + goto invalid; + } + } + } + + s->buffer->pos = p; + s->state = state; + + return NGX_AGAIN; + +done: + + s->buffer->pos = p + 1; + + if (s->arg_start) { + arg = ngx_array_push(&s->args); + if (arg == NULL) { + return NGX_ERROR; + } + arg->len = s->arg_end - s->arg_start; + arg->data = s->arg_start; + + s->arg_start = NULL; + s->cmd_start = NULL; + s->quoted = 0; + s->no_sync_literal = 0; + s->literal_len = 0; + } + + s->state = (s->command != NGX_SIEVE_AUTHENTICATE) ? sw_start : sw_spaces_before_argument; + + return NGX_OK; + +invalid: + + s->state = sw_start; + s->quoted = 0; + s->no_sync_literal = 0; + s->literal_len = 0; + + return NGX_MAIL_PARSE_INVALID_COMMAND; +} + + +ngx_int_t ngx_mail_smtp_parse_command(ngx_mail_session_t *s) { u_char ch, *p, *c, c0, c1, c2, c3; diff -r 0cb942c1c1aa -r f1dffaf61968 src/mail/ngx_mail_pop3_handler.c --- a/src/mail/ngx_mail_pop3_handler.c Fri Mar 13 02:12:10 2020 +0300 +++ b/src/mail/ngx_mail_pop3_handler.c Wed Apr 08 20:17:11 2020 +0200 @@ -492,7 +492,7 @@ return NGX_MAIL_PARSE_INVALID_COMMAND; } - if (ngx_mail_auth_cram_md5_salt(s, c, "+ ", 2) == NGX_OK) { + if (ngx_mail_auth_cram_md5_salt(s, c, "+ ", 2, 0) == NGX_OK) { s->mail_state = ngx_pop3_auth_cram_md5; return NGX_OK; } diff -r 0cb942c1c1aa -r f1dffaf61968 src/mail/ngx_mail_proxy_module.c --- a/src/mail/ngx_mail_proxy_module.c Fri Mar 13 02:12:10 2020 +0300 +++ b/src/mail/ngx_mail_proxy_module.c Wed Apr 08 20:17:11 2020 +0200 @@ -24,6 +24,7 @@ static void ngx_mail_proxy_block_read(ngx_event_t *rev); static void ngx_mail_proxy_pop3_handler(ngx_event_t *rev); static void ngx_mail_proxy_imap_handler(ngx_event_t *rev); +static void ngx_mail_proxy_sieve_handler(ngx_event_t *rev); static void ngx_mail_proxy_smtp_handler(ngx_event_t *rev); static void ngx_mail_proxy_dummy_handler(ngx_event_t *ev); static ngx_int_t ngx_mail_proxy_read_response(ngx_mail_session_t *s, @@ -173,6 +174,11 @@ s->mail_state = ngx_imap_start; break; + case NGX_MAIL_SIEVE_PROTOCOL: + p->upstream.connection->read->handler = ngx_mail_proxy_sieve_handler; + s->mail_state = ngx_sieve_start; + break; + default: /* NGX_MAIL_SMTP_PROTOCOL */ p->upstream.connection->read->handler = ngx_mail_proxy_smtp_handler; s->mail_state = ngx_smtp_start; @@ -446,6 +452,144 @@ static void +ngx_mail_proxy_sieve_handler(ngx_event_t *rev) +{ + u_char *p; + ngx_int_t rc; + ngx_str_t line; + ngx_str_t userpass; + ngx_str_t b64_userpass; + ngx_connection_t *c; + ngx_mail_session_t *s; + ngx_mail_proxy_conf_t *pcf; + + ngx_log_debug0(NGX_LOG_DEBUG_MAIL, rev->log, 0, + "mail proxy sieve auth handler"); + + c = rev->data; + s = c->data; + + if (rev->timedout) { + ngx_log_error(NGX_LOG_INFO, c->log, NGX_ETIMEDOUT, + "upstream timed out"); + c->timedout = 1; + ngx_mail_proxy_internal_server_error(s); + return; + } + + rc = ngx_mail_proxy_read_response(s, s->mail_state); + + if (rc == NGX_AGAIN) { + return; + } + + if (rc == NGX_ERROR) { + ngx_mail_proxy_upstream_error(s); + return; + } + + switch (s->mail_state) { + + case ngx_sieve_start: + ngx_log_debug0(NGX_LOG_DEBUG_MAIL, rev->log, 0, + "mail proxy send login"); + + s->connection->log->action = "sending LOGIN command to upstream"; + + line.len = s->tag.len + sizeof("AUTHENTICATE ") - 1 + + 1 + NGX_SIZE_T_LEN + 1 + 2; + line.data = ngx_pnalloc(c->pool, line.len); + if (line.data == NULL) { + ngx_mail_proxy_internal_server_error(s); + return; + } + + userpass.len = s->login.len + s->passwd.len + 2; + userpass.data = ngx_pnalloc(c->pool, userpass.len); + if (userpass.data == NULL) { + ngx_mail_proxy_internal_server_error(s); + return; + } + b64_userpass.len = ngx_base64_encoded_length(userpass.len) + 2; + b64_userpass.data = ngx_pnalloc(c->pool, b64_userpass.len); + if (b64_userpass.data == NULL) { + ngx_mail_proxy_internal_server_error(s); + return; + } + + p = userpass.data; + *p++ = '\0'; + p = ngx_cpymem(p, s->login.data, s->login.len); + *p++ = '\0'; + p = ngx_cpymem(p, s->passwd.data, s->passwd.len); + + ngx_encode_base64(&b64_userpass, &userpass); + line.len = ngx_sprintf(line.data, "AUTHENTICATE \"PLAIN\" \"%V\"" CRLF, + &b64_userpass) + - line.data; + + s->mail_state = ngx_sieve_login; + break; + + case ngx_sieve_login: + ngx_log_debug0(NGX_LOG_DEBUG_MAIL, rev->log, 0, + "mail proxy send capabilities"); + + s->connection->log->action = "sending CAPABILITY command to upstream"; + + line.len = s->tag.len + sizeof("AUTHENTICATE ") - 1 + + 1 + NGX_SIZE_T_LEN + 1 + 2; + line.data = ngx_pnalloc(c->pool, line.len); + if (line.data == NULL) { + ngx_mail_proxy_internal_server_error(s); + return; + } + + line.len = ngx_sprintf(line.data, "CAPABILITY" CRLF) + - line.data; + + s->mail_state = ngx_sieve_login_capabilities; + break; + + case ngx_sieve_login_capabilities: + s->connection->read->handler = ngx_mail_proxy_handler; + s->connection->write->handler = ngx_mail_proxy_handler; + rev->handler = ngx_mail_proxy_handler; + c->write->handler = ngx_mail_proxy_handler; + + pcf = ngx_mail_get_module_srv_conf(s, ngx_mail_proxy_module); + ngx_add_timer(s->connection->read, pcf->timeout); + ngx_del_timer(c->read); + + c->log->action = NULL; + ngx_log_error(NGX_LOG_INFO, c->log, 0, "client logged in"); + + ngx_mail_proxy_handler(s->connection->write); + + return; + + default: +#if (NGX_SUPPRESS_WARN) + ngx_str_null(&line); +#endif + break; + } + + if (c->send(c, line.data, line.len) < (ssize_t) line.len) { + /* + * we treat the incomplete sending as NGX_ERROR + * because it is very strange here + */ + ngx_mail_proxy_internal_server_error(s); + return; + } + + s->proxy->buffer->pos = s->proxy->buffer->start; + s->proxy->buffer->last = s->proxy->buffer->start; +} + + +static void ngx_mail_proxy_smtp_handler(ngx_event_t *rev) { u_char *p; @@ -793,6 +937,42 @@ break; + case NGX_MAIL_SIEVE_PROTOCOL: + if (p[0] == '"') { + m = b->last - (sizeof(CRLF "OK" CRLF) - 1); + + while (m > p) { + if (m[0] == CR && m[1] == LF) { + break; + } + m--; + } + + if (m <= p || m[0] == '"') { + return NGX_AGAIN; + } + p = m + 2; + } + + switch (state) { + + case ngx_sieve_start: + case ngx_sieve_login: + ngx_log_error(NGX_LOG_INFO, s->connection->log, 0, "1 upstream sent: \"%s\"", p); + if (p[0] == 'O' && p[1] == 'K') { + return NGX_OK; + } + break; + case ngx_sieve_login_capabilities: + ngx_log_error(NGX_LOG_INFO, s->connection->log, 0, "2 upstream sent: \"%s\"", p); + if (p[0] == 'O' && p[1] == 'K') { + return NGX_OK; + } + break; + } + + break; + default: /* NGX_MAIL_SMTP_PROTOCOL */ if (p[3] == '-') { diff -r 0cb942c1c1aa -r f1dffaf61968 src/mail/ngx_mail_smtp_handler.c --- a/src/mail/ngx_mail_smtp_handler.c Fri Mar 13 02:12:10 2020 +0300 +++ b/src/mail/ngx_mail_smtp_handler.c Wed Apr 08 20:17:11 2020 +0200 @@ -693,7 +693,7 @@ } } - if (ngx_mail_auth_cram_md5_salt(s, c, "334 ", 4) == NGX_OK) { + if (ngx_mail_auth_cram_md5_salt(s, c, "334 ", 4, 0) == NGX_OK) { s->mail_state = ngx_smtp_auth_cram_md5; return NGX_OK; } diff -r 0cb942c1c1aa -r f1dffaf61968 src/misc/ngx_cpp_test_module.cpp --- a/src/misc/ngx_cpp_test_module.cpp Fri Mar 13 02:12:10 2020 +0300 +++ b/src/misc/ngx_cpp_test_module.cpp Wed Apr 08 20:17:11 2020 +0200 @@ -13,6 +13,7 @@ #include #include #include + #include #include } From sander at hoentjen.eu Wed Apr 8 18:30:33 2020 From: sander at hoentjen.eu (Sander Hoentjen) Date: Wed, 8 Apr 2020 20:30:33 +0200 Subject: [PATCH] Tests: added sieve tests In-Reply-To: <58016a3a-c11d-bb31-2b05-916cb8124598@hoentjen.eu> References: <58016a3a-c11d-bb31-2b05-916cb8124598@hoentjen.eu> Message-ID: # HG changeset patch # User Sander Hoentjen # Date 1586370310 -7200 # Wed Apr 08 20:25:10 2020 +0200 # Node ID 3d35b19abfa72d1d09e23d02917df7fbdee0970c # Parent 9e5d38da765152a20098642e37b0afe56312f794 Tests: added sieve tests diff -r 9e5d38da7651 -r 3d35b19abfa7 lib/Test/Nginx.pm --- a/lib/Test/Nginx.pm Fri Mar 20 16:32:06 2020 +0300 +++ b/lib/Test/Nginx.pm Wed Apr 08 20:25:10 2020 +0200 @@ -165,6 +165,7 @@ cache => '(?s)^(?!.*--without-http-cache)', pop3 => '(?s)^(?!.*--without-mail_pop3_module)', imap => '(?s)^(?!.*--without-mail_imap_module)', + sieve => '(?s)^(?!.*--without-mail_sieve_module)', smtp => '(?s)^(?!.*--without-mail_smtp_module)', pcre => '(?s)^(?!.*--without-pcre)', split_clients diff -r 9e5d38da7651 -r 3d35b19abfa7 lib/Test/Nginx/SIEVE.pm --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/lib/Test/Nginx/SIEVE.pm Wed Apr 08 20:25:10 2020 +0200 @@ -0,0 +1,156 @@ +package Test::Nginx::SIEVE; + +# (C) Maxim Dounin + +# Module for nginx sieve tests. + +############################################################################### + +use warnings; +use strict; + +use Test::More qw//; +use IO::Select; +use IO::Socket; +use Socket qw/ CRLF /; + +use Test::Nginx; + +sub new { + my $self = {}; + bless $self, shift @_; + + $self->{_socket} = IO::Socket::INET->new( + Proto => "tcp", + PeerAddr => "127.0.0.1:" . port(8200), + @_ + ) + or die "Can't connect to nginx: $!\n"; + + if ({@_}->{'SSL'}) { + require IO::Socket::SSL; + IO::Socket::SSL->start_SSL($self->{_socket}, @_) + or die $IO::Socket::SSL::SSL_ERROR . "\n"; + } + + $self->{_socket}->autoflush(1); + + return $self; +} + +sub eof { + my $self = shift; + return $self->{_socket}->eof(); +} + +sub print { + my ($self, $cmd) = @_; + log_out($cmd); + $self->{_socket}->print($cmd); +} + +sub send { + my ($self, $cmd) = @_; + #warn "\n>>>$cmd\n"; + log_out($cmd); + $self->{_socket}->print($cmd . CRLF); +} + +sub read { + my ($self) = @_; + my $socket = $self->{_socket}; + eval { + local $SIG{ALRM} = sub { die "timeout\n" }; + alarm(8); + while (<$socket>) { + #warn "\n====\n<<<$_\n===\n"; + log_in($_); + # XXX + last; + } + alarm(0); + }; + alarm(0); + if ($@) { + log_in("died: $@"); + return undef; + } + return $_; +} + +sub read_ok { + my ($self) = @_; + my $socket = $self->{_socket}; + eval { + local $SIG{ALRM} = sub { die "timeout\n" }; + alarm(8); + while (<$socket>) { + #warn "\n====\n<<<$_\n===\n"; + log_in($_); + # XXX + next if m/^"/; + last; + } + alarm(0); + }; + alarm(0); + if ($@) { + log_in("died: $@"); + return undef; + } + return $_; +} + +sub check { + my ($self, $regex, $name) = @_; + Test::More->builder->like($self->read(), $regex, $name); +} + +sub ok { + my $self = shift; + Test::More->builder->like($self->read_ok(), qr/^OK/, @_); +} + +sub can_read { + my ($self, $timo) = @_; + IO::Select->new($self->{_socket})->can_read($timo || 3); +} + +############################################################################### + +sub sieve_test_daemon { + my ($port) = @_; + + my $server = IO::Socket::INET->new( + Proto => 'tcp', + LocalAddr => '127.0.0.1:' . ($port || port(8201)), + Listen => 5, + Reuse => 1 + ) + or die "Can't create listening socket: $!\n"; + + while (my $client = $server->accept()) { + $client->autoflush(1); + print $client "OK fake sieve server ready" . CRLF; + + while (<$client>) { + if (/^logout/i) { + print $client 'OK logout ok' . CRLF; + } elsif (/^AUTHENTICATE /i) { + print $client 'OK login ok' . CRLF; + } elsif (/^CAPABILITY/i) { + print $client 'OK capabilty ok' . CRLF; + } else { + print $client 'NO unknown command ' . $_ . CRLF; + } + } + + close $client; + } +} + +############################################################################### + +1; + +############################################################################### diff -r 9e5d38da7651 -r 3d35b19abfa7 mail_capability.t --- a/mail_capability.t Fri Mar 20 16:32:06 2020 +0300 +++ b/mail_capability.t Wed Apr 08 20:25:10 2020 +0200 @@ -3,7 +3,7 @@ # (C) Sergey Kandaurov # (C) Nginx, Inc. -# Tests for imap/pop3/smtp capabilities. +# Tests for imap/pop3/sieve/smtp capabilities. ############################################################################### @@ -18,6 +18,7 @@ use Test::Nginx; use Test::Nginx::IMAP; use Test::Nginx::POP3; +use Test::Nginx::SIEVE; use Test::Nginx::SMTP; ############################################################################### @@ -25,8 +26,8 @@ select STDERR; $| = 1; select STDOUT; $| = 1; -my $t = Test::Nginx->new()->has(qw/mail mail_ssl imap pop3 smtp/) - ->has_daemon('openssl')->plan(17); +my $t = Test::Nginx->new()->has(qw/mail mail_ssl imap pop3 sieve smtp/) + ->has_daemon('openssl')->plan(25); $t->write_file_expand('nginx.conf', <<'EOF'); @@ -81,6 +82,24 @@ } server { + listen 127.0.0.1:8200; + protocol sieve; + sieve_capabilities '"SEE-THIS"'; + } + + server { + listen 127.0.0.1:8201; + protocol sieve; + starttls on; + } + + server { + listen 127.0.0.1:8202; + protocol sieve; + starttls only; + } + + server { listen 127.0.0.1:8025; protocol smtp; starttls off; @@ -188,6 +207,38 @@ unlike($caps, qr/SASL/, 'pop3 starttls only - no methods'); like($caps, qr/STLS/, 'pop3 startls only - stls'); +# sieve, custom capabilities + +$s = Test::Nginx::SIEVE->new(PeerAddr => '127.0.0.1:' . port(8200)); +$s->ok('sieve connection completed'); + +$s->send('CAPABILITY'); +$s->check(qr/^"SEE-THIS"/, 'sieve capability custom'); +$s->check(qr/^"SASL" "PLAIN"/, 'sieve capability sasl'); +$s->ok('sieve capability completed'); + +# sieve starttls + +$s = Test::Nginx::SIEVE->new(PeerAddr => '127.0.0.1:' . port(8201)); +$s->read(); +$s->read(); +$s->read(); +$s->check(qr/^"SASL" "PLAIN"/, + 'sieve capability starttls has plain'); +$s->check(qr/^"STARTTLS"/, + 'sieve capability starttls'); + +# sieve starttls only + +$s = Test::Nginx::SIEVE->new(PeerAddr => '127.0.0.1:' . port(8202)); +$s->read(); +$s->read(); +$s->read(); +$s->check(qr/^"SASL" ""/, + 'sieve capability starttls only not has plain'); +$s->check(qr/^"STARTTLS"/, + 'sieve capability starttls only'); + # smtp $s = Test::Nginx::SMTP->new(PeerAddr => '127.0.0.1:' . port(8025)); diff -r 9e5d38da7651 -r 3d35b19abfa7 mail_sieve.t --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/mail_sieve.t Wed Apr 08 20:25:10 2020 +0200 @@ -0,0 +1,176 @@ +#!/usr/bin/perl + +# (C) Sander Hoentjen +# (C) Antagonist B.V. + +# Tests for nginx mail sieve module. + +############################################################################### + +use warnings; +use strict; + +use Test::More; + +use MIME::Base64; + +BEGIN { use FindBin; chdir($FindBin::Bin); } + +use lib 'lib'; +use Test::Nginx; +use Test::Nginx::SIEVE; + +############################################################################### + +select STDERR; $| = 1; +select STDOUT; $| = 1; + +local $SIG{PIPE} = 'IGNORE'; + +my $t = Test::Nginx->new()->has(qw/mail sieve http rewrite/) + ->write_file_expand('nginx.conf', <<'EOF'); + +%%TEST_GLOBALS%% + +daemon off; + +events { +} + +mail { + proxy_pass_error_message on; + auth_http http://127.0.0.1:8080/mail/auth; + + server { + listen 127.0.0.1:8200; + protocol sieve; + sieve_auth plain cram-md5 external; + } +} + +http { + %%TEST_GLOBALS_HTTP%% + + server { + listen 127.0.0.1:8080; + server_name localhost; + + location = /mail/auth { + set $reply ERROR; + set $passw ""; + + if ($http_auth_smtp_to ~ example.com) { + set $reply OK; + } + + set $userpass "$http_auth_user:$http_auth_pass"; + if ($userpass ~ '^test at example.com:secret$') { + set $reply OK; + } + + set $userpass "$http_auth_user:$http_auth_salt:$http_auth_pass"; + if ($userpass ~ '^test at example.com:<.*@.*>:0{32}$') { + set $reply OK; + set $passw secret; + } + + set $userpass "$http_auth_method:$http_auth_user:$http_auth_pass"; + if ($userpass ~ '^external:test at example.com:$') { + set $reply OK; + set $passw secret; + } + + add_header Auth-Status $reply; + add_header Auth-Server 127.0.0.1; + add_header Auth-Port %%PORT_8201%%; + add_header Auth-Pass $passw; + add_header Auth-Wait 1; + return 204; + } + } +} + +EOF + +$t->run_daemon(\&Test::Nginx::SIEVE::sieve_test_daemon); +$t->run()->plan(15); + +$t->waitforsocket('127.0.0.1:' . port(8201)); + +############################################################################### + +my $s = Test::Nginx::SIEVE->new(); +$s->ok('greeting'); + +# bad auth + +$s->send('AUTHENTICATE'); +$s->check(qr/^NO/, 'auth without arguments'); + +# auth plain + +$s->send('AUTHENTICATE "PLAIN" "' . encode_base64("\0test\@example.com\0bad", '') . '"'); +$s->check(qr/^BYE/, 'auth plain with bad password'); + +$s = Test::Nginx::SIEVE->new(); +$s->ok('greeting'); +$s->send('AUTHENTICATE "PLAIN" "' . encode_base64("\0test\@example.com\0secret", '') . '"'); +#$s->check(qr/auth plain/, 'blaat'); +$s->ok('auth plain'); + +# auth login simple + +$s = Test::Nginx::SIEVE->new(); +$s->read_ok(); + +$s->send('AUTHENTICATE "LOGIN"'); +$s->check(qr/"VXNlcm5hbWU6"/, 'auth login username challenge'); + +$s->send('"' . encode_base64('test at example.com', '') . '"'); +$s->check(qr/"UGFzc3dvcmQ6"/, 'auth login password challenge'); + +$s->send('"' . encode_base64('secret', '') . '"'); +$s->ok('auth login simple'); + +# auth login with username + +$s = Test::Nginx::SIEVE->new(); +$s->read_ok(); + +$s->send('AUTHENTICATE "LOGIN" "' . encode_base64('test at example.com', '') . '"'); +$s->check(qr/"UGFzc3dvcmQ6"/, 'auth login with username password challenge'); + +$s->send('"' . encode_base64('secret', '') . '"'); +$s->ok('auth login with username'); + +# auth cram-md5 + +$s = Test::Nginx::SIEVE->new(); +$s->read_ok(); + +$s->send('AUTHENTICATE "CRAM-MD5"'); +$s->check(qr/"/, 'auth cram-md5 challenge'); + +$s->send('"' . encode_base64('test at example.com ' . ('0' x 32), '') . '"'); +$s->ok('auth cram-md5'); + +# auth external + +$s = Test::Nginx::SIEVE->new(); +$s->read_ok(); + +$s->send('AUTHENTICATE "EXTERNAL"'); +$s->check(qr/"VXNlcm5hbWU6"/, 'auth external challenge'); + +$s->send('"' . encode_base64('test at example.com', '') . '"'); +$s->ok('auth external'); + +# auth external with username + +$s = Test::Nginx::SIEVE->new(); +$s->read_ok(); + +$s->send('AUTHENTICATE "EXTERNAL" "' . encode_base64('test at example.com', '') . '"'); +$s->ok('auth external with username'); + +############################################################################### From sander at hoentjen.eu Wed Apr 8 18:33:23 2020 From: sander at hoentjen.eu (Sander Hoentjen) Date: Wed, 8 Apr 2020 20:33:23 +0200 Subject: [PATCH] Added support for proxying managesieve protocol In-Reply-To: <58016a3a-c11d-bb31-2b05-916cb8124598@hoentjen.eu> References: <58016a3a-c11d-bb31-2b05-916cb8124598@hoentjen.eu> Message-ID: Hello list, This is my attempt at adding support for the managesieve protocol. I hope this is something that you would consider to add. Comments on the code are very welcome! Also, I hope I submitted this the right way. If I need to change anything, please let me know. Kind regards, Sander Hoentjen On 4/8/20 8:26 PM, Sander Hoentjen wrote: > # HG changeset patch > # User Sander Hoentjen > # Date 1586369831 -7200 > # Wed Apr 08 20:17:11 2020 +0200 > # Node ID f1dffaf619688aaab90caf31781ebe27c3f79598 > # Parent 0cb942c1c1aa98118076e72e0b89940e85e6291c > Added support for proxying managesieve protocol > > Sieve is a widely used protocol to implement server-side filtering. Some > servers and clients that support this are listed at > . This > commit is tested with Dovecot as the server and Roundcube as the > client, but I > don't see a readon why it not would work with others. It is a nice > addition to > nginx because it adds the possibility to add server side filtering to > setups > that use IMAP behind an nginx proxy. > > Fixes https://trac.nginx.org/nginx/ticket/1697 > > diff -r 0cb942c1c1aa -r f1dffaf61968 auto/modules > --- a/auto/modules Fri Mar 13 02:12:10 2020 +0300 > +++ b/auto/modules Wed Apr 08 20:17:11 2020 +0200 > @@ -965,6 +965,15 @@ > . auto/module > fi > + if [ $MAIL_SIEVE = YES ]; then > + ngx_module_name=ngx_mail_sieve_module > + ngx_module_deps=src/mail/ngx_mail_sieve_module.h > + ngx_module_srcs="src/mail/ngx_mail_sieve_module.c \ > + src/mail/ngx_mail_sieve_handler.c" > + > + . auto/module > + fi > + > if [ $MAIL_SMTP = YES ]; then > ngx_module_name=ngx_mail_smtp_module > ngx_module_deps=src/mail/ngx_mail_smtp_module.h > diff -r 0cb942c1c1aa -r f1dffaf61968 auto/options > --- a/auto/options Fri Mar 13 02:12:10 2020 +0300 > +++ b/auto/options Wed Apr 08 20:17:11 2020 +0200 > @@ -112,6 +112,7 @@ > MAIL_SSL=NO > MAIL_POP3=YES > MAIL_IMAP=YES > +MAIL_SIEVE=YES > MAIL_SMTP=YES > STREAM=NO > @@ -305,6 +306,8 @@ > ;; > --without-mail_pop3_module) MAIL_POP3=NO ;; > --without-mail_imap_module) MAIL_IMAP=NO ;; > + --without-mail_sieve_module) > + MAIL_SIEVE=NO ;; > --without-mail_smtp_module) MAIL_SMTP=NO ;; > --with-stream) STREAM=YES ;; > @@ -517,11 +520,12 @@ > --without-http disable HTTP server > --without-http-cache disable HTTP cache > - --with-mail enable POP3/IMAP4/SMTP proxy module > - --with-mail=dynamic enable dynamic POP3/IMAP4/SMTP proxy module > + --with-mail enable POP3/IMAP4/SIEVE/SMTP proxy module > + --with-mail=dynamic enable dynamic POP3/IMAP4/SIEVE/SMTP proxy module > --with-mail_ssl_module enable ngx_mail_ssl_module > --without-mail_pop3_module disable ngx_mail_pop3_module > --without-mail_imap_module disable ngx_mail_imap_module > + --without-mail_sieve_module disable ngx_mail_sieve_module > --without-mail_smtp_module disable ngx_mail_smtp_module > --with-stream enable TCP/UDP proxy module > diff -r 0cb942c1c1aa -r f1dffaf61968 contrib/vim/syntax/nginx.vim > --- a/contrib/vim/syntax/nginx.vim Fri Mar 13 02:12:10 2020 +0300 > +++ b/contrib/vim/syntax/nginx.vim Wed Apr 08 20:17:11 2020 +0200 > @@ -571,6 +571,9 @@ > syn keyword ngxDirective contained session_log_format > syn keyword ngxDirective contained session_log_zone > syn keyword ngxDirective contained set_real_ip_from > +syn keyword ngxDirective contained sieve_auth > +syn keyword ngxDirective contained sieve_capabilities > +syn keyword ngxDirective contained sieve_client_buffer > syn keyword ngxDirective contained slice > syn keyword ngxDirective contained smtp_auth > syn keyword ngxDirective contained smtp_capabilities > diff -r 0cb942c1c1aa -r f1dffaf61968 src/mail/ngx_mail.h > --- a/src/mail/ngx_mail.h Fri Mar 13 02:12:10 2020 +0300 > +++ b/src/mail/ngx_mail.h Wed Apr 08 20:17:11 2020 +0200 > @@ -102,6 +102,7 @@ > #define NGX_MAIL_POP3_PROTOCOL 0 > #define NGX_MAIL_IMAP_PROTOCOL 1 > #define NGX_MAIL_SMTP_PROTOCOL 2 > +#define NGX_MAIL_SIEVE_PROTOCOL 3 > typedef struct ngx_mail_protocol_s ngx_mail_protocol_t; > @@ -154,6 +155,21 @@ > typedef enum { > + ngx_sieve_start = 0, > + ngx_sieve_starttls, > + ngx_sieve_auth_login_username, > + ngx_sieve_auth_login_password, > + ngx_sieve_auth_plain, > + ngx_sieve_auth_cram_md5, > + ngx_sieve_auth_external, > + ngx_sieve_login, > + ngx_sieve_login_capabilities, > + ngx_sieve_user, > + ngx_sieve_passwd > +} ngx_sieve_state_e; > + > + > +typedef enum { > ngx_smtp_start = 0, > ngx_smtp_auth_login_username, > ngx_smtp_auth_login_password, > @@ -227,7 +243,7 @@ > ngx_uint_t login_attempt; > - /* used to parse POP3/IMAP/SMTP command */ > + /* used to parse POP3/IMAP/SIEVE/SMTP command */ > ngx_uint_t state; > u_char *cmd_start; > @@ -271,6 +287,14 @@ > #define NGX_IMAP_AUTHENTICATE 7 > +#define NGX_SIEVE_LOGOUT 1 > +#define NGX_SIEVE_CAPABILITY 2 > +#define NGX_SIEVE_NOOP 3 > +#define NGX_SIEVE_STARTTLS 4 > +#define NGX_SIEVE_NEXT 5 > +#define NGX_SIEVE_AUTHENTICATE 6 > + > + > #define NGX_SMTP_HELO 1 > #define NGX_SMTP_EHLO 2 > #define NGX_SMTP_AUTH 3 > @@ -383,7 +407,7 @@ > ngx_int_t ngx_mail_auth_login_password(ngx_mail_session_t *s, > ngx_connection_t *c); > ngx_int_t ngx_mail_auth_cram_md5_salt(ngx_mail_session_t *s, > - ngx_connection_t *c, char *prefix, size_t len); > + ngx_connection_t *c, char *prefix, size_t len, ngx_uint_t quote); > ngx_int_t ngx_mail_auth_cram_md5(ngx_mail_session_t *s, > ngx_connection_t *c); > ngx_int_t ngx_mail_auth_external(ngx_mail_session_t *s, > ngx_connection_t *c, > ngx_uint_t n); > diff -r 0cb942c1c1aa -r f1dffaf61968 src/mail/ngx_mail_auth_http_module.c > --- a/src/mail/ngx_mail_auth_http_module.c Fri Mar 13 02:12:10 2020 +0300 > +++ b/src/mail/ngx_mail_auth_http_module.c Wed Apr 08 20:17:11 2020 +0200 > @@ -533,6 +533,11 @@ > + sizeof(CRLF) - 1; > break; > + case NGX_MAIL_SIEVE_PROTOCOL: > + size = sizeof("BYE \"\"") - 1 + len > + + sizeof(CRLF) - 1; > + break; > + > default: /* NGX_MAIL_SMTP_PROTOCOL */ > ctx->err = ctx->errmsg; > continue; > @@ -559,11 +564,27 @@ > *p++ = 'N'; *p++ = 'O'; *p++ = ' '; > break; > + case NGX_MAIL_SIEVE_PROTOCOL: > + p = ngx_cpymem(p, s->tag.data, s->tag.len); > + *p++ = 'B'; *p++ = 'Y'; *p++ = 'E'; *p++ = ' '; *p++ = '"'; > + break; > + > default: /* NGX_MAIL_SMTP_PROTOCOL */ > break; > } > p = ngx_cpymem(p, ctx->header_start, len); > + > + switch (s->protocol) { > + > + case NGX_MAIL_SIEVE_PROTOCOL: > + *p++ = '"'; > + break; > + > + default: > + break; > + } > + > *p++ = CR; *p++ = LF; > ctx->err.len = p - ctx->err.data; > diff -r 0cb942c1c1aa -r f1dffaf61968 src/mail/ngx_mail_handler.c > --- a/src/mail/ngx_mail_handler.c Fri Mar 13 02:12:10 2020 +0300 > +++ b/src/mail/ngx_mail_handler.c Wed Apr 08 20:17:11 2020 +0200 > @@ -520,28 +520,39 @@ > ngx_int_t > ngx_mail_auth_cram_md5_salt(ngx_mail_session_t *s, ngx_connection_t *c, > - char *prefix, size_t len) > + char *prefix, size_t len, ngx_uint_t quote) > { > u_char *p; > ngx_str_t salt; > - ngx_uint_t n; > + ngx_uint_t n = 0; > + if (quote) { > + len += 2; > + } > p = ngx_pnalloc(c->pool, len + ngx_base64_encoded_length(s->salt.len) > + 2); > if (p == NULL) { > return NGX_ERROR; > } > - salt.data = ngx_cpymem(p, prefix, len); > + if (quote) { > + len -= 2; > + salt.data = ngx_cpymem(p, "\"", 1); > + n++; > + } > + salt.data = ngx_cpymem(p+ n, prefix, len); > s->salt.len -= 2; > ngx_encode_base64(&salt, &s->salt); > s->salt.len += 2; > - n = len + salt.len; > + n += len + salt.len; > + if (quote) { > + p[n++] = '"'; > + } > p[n++] = CR; p[n++] = LF; > s->out.len = n; > - s->out.data = p; > + s->out.data = p--; > return NGX_OK; > } > diff -r 0cb942c1c1aa -r f1dffaf61968 src/mail/ngx_mail_imap_handler.c > --- a/src/mail/ngx_mail_imap_handler.c Fri Mar 13 02:12:10 2020 +0300 > +++ b/src/mail/ngx_mail_imap_handler.c Wed Apr 08 20:17:11 2020 +0200 > @@ -397,7 +397,7 @@ > } > } > - if (ngx_mail_auth_cram_md5_salt(s, c, "+ ", 2) == NGX_OK) { > + if (ngx_mail_auth_cram_md5_salt(s, c, "+ ", 2, 0) == NGX_OK) { > s->mail_state = ngx_imap_auth_cram_md5; > return NGX_OK; > } > diff -r 0cb942c1c1aa -r f1dffaf61968 src/mail/ngx_mail_parse.c > --- a/src/mail/ngx_mail_parse.c Fri Mar 13 02:12:10 2020 +0300 > +++ b/src/mail/ngx_mail_parse.c Wed Apr 08 20:17:11 2020 +0200 > @@ -11,6 +11,7 @@ > #include > #include > #include > +#include > #include > @@ -620,6 +621,364 @@ > ngx_int_t > +ngx_mail_sieve_parse_command(ngx_mail_session_t *s) > +{ > + u_char ch, *p, *c; > + ngx_str_t *arg; > + enum { > + sw_start = 0, > + sw_spaces_before_argument, > + sw_argument, > + sw_backslash, > + sw_literal, > + sw_no_sync_literal_argument, > + sw_start_literal_argument, > + sw_literal_argument, > + sw_end_literal_argument, > + sw_almost_done > + } state; > + > + state = s->state; > + > + for (p = s->buffer->pos; p < s->buffer->last; p++) { > + ch = *p; > + > + switch (state) { > + > + /* SIEVE command */ > + case sw_start: > + if (ch == ' ' || ch == CR || ch == LF) { > + > + c = s->buffer->start; > + > + switch (p - c) { > + > + case 4: > + if ((c[0] == 'N' || c[0] == 'n') > + && (c[1] == 'O'|| c[1] == 'o') > + && (c[2] == 'O'|| c[2] == 'o') > + && (c[3] == 'P'|| c[3] == 'p')) > + { > + s->command = NGX_SIEVE_NOOP; > + > + } else { > + goto invalid; > + } > + break; > + > + case 6: > + if ((c[0] == 'L'|| c[0] == 'l') > + && (c[1] == 'O'|| c[1] == 'o') > + && (c[2] == 'G'|| c[2] == 'g') > + && (c[3] == 'O'|| c[3] == 'o') > + && (c[4] == 'U'|| c[4] == 'u') > + && (c[5] == 'T'|| c[5] == 't')) > + { > + s->command = NGX_SIEVE_LOGOUT; > + > + } else { > + goto invalid; > + } > + break; > + > +#if (NGX_MAIL_SSL) > + case 8: > + if ((c[0] == 'S'|| c[0] == 's') > + && (c[1] == 'T'|| c[1] == 't') > + && (c[2] == 'A'|| c[2] == 'a') > + && (c[3] == 'R'|| c[3] == 'r') > + && (c[4] == 'T'|| c[4] == 't') > + && (c[5] == 'T'|| c[5] == 't') > + && (c[6] == 'L'|| c[6] == 'l') > + && (c[7] == 'S'|| c[7] == 's')) > + { > + s->command = NGX_SIEVE_STARTTLS; > + > + } else { > + goto invalid; > + } > + break; > +#endif > + > + case 10: > + if ((c[0] == 'C'|| c[0] == 'c') > + && (c[1] == 'A'|| c[1] == 'a') > + && (c[2] == 'P'|| c[2] == 'p') > + && (c[3] == 'A'|| c[3] == 'a') > + && (c[4] == 'B'|| c[4] == 'b') > + && (c[5] == 'I'|| c[5] == 'i') > + && (c[6] == 'L'|| c[6] == 'l') > + && (c[7] == 'I'|| c[7] == 'i') > + && (c[8] == 'T'|| c[8] == 't') > + && (c[9] == 'Y'|| c[9] == 'y')) > + { > + s->command = NGX_SIEVE_CAPABILITY; > + > + } else { > + goto invalid; > + } > + break; > + > + case 12: > + if ((c[0] == 'A'|| c[0] == 'a') > + && (c[1] == 'U'|| c[1] == 'u') > + && (c[2] == 'T'|| c[2] == 't') > + && (c[3] == 'H'|| c[3] == 'h') > + && (c[4] == 'E'|| c[4] == 'e') > + && (c[5] == 'N'|| c[5] == 'n') > + && (c[6] == 'T'|| c[6] == 't') > + && (c[7] == 'I'|| c[7] == 'i') > + && (c[8] == 'C'|| c[8] == 'c') > + && (c[9] == 'A'|| c[9] == 'a') > + && (c[10] == 'T'|| c[10] == 't') > + && (c[11] == 'E'|| c[11] == 'e')) > + { > + s->command = NGX_SIEVE_AUTHENTICATE; > + > + } else { > + goto invalid; > + } > + break; > + > + default: > + goto invalid; > + } > + > + switch (ch) { > + case ' ': > + state = sw_spaces_before_argument; > + break; > + case CR: > + state = sw_almost_done; > + break; > + case LF: > + goto done; > + } > + break; > + } > + > + if ((ch < 'A' || ch > 'Z') && (ch < 'a' || ch > 'z')) { > + goto invalid; > + } > + > + break; > + > + case sw_spaces_before_argument: > + switch (ch) { > + case ' ': > + break; > + case CR: > + state = sw_almost_done; > + s->arg_end = p; > + break; > + case LF: > + s->arg_end = p; > + goto done; > + case '"': > + if (s->args.nelts <= 2) { > + s->quoted = 1; > + s->arg_start = p + 1; > + state = sw_argument; > + break; > + } > + goto invalid; > + case '{': > + if (s->args.nelts <= 2) { > + state = sw_literal; > + break; > + } > + goto invalid; > + default: > + if (s->args.nelts <= 2) { > + s->arg_start = p; > + state = sw_argument; > + break; > + } > + goto invalid; > + } > + break; > + > + case sw_argument: > + if (ch == ' ' && s->quoted) { > + break; > + } > + > + switch (ch) { > + case '"': > + if (!s->quoted) { > + break; > + } > + s->quoted = 0; > + /* fall through */ > + case ' ': > + case CR: > + case LF: > + arg = ngx_array_push(&s->args); > + if (arg == NULL) { > + return NGX_ERROR; > + } > + arg->len = p - s->arg_start; > + arg->data = s->arg_start; > + s->arg_start = NULL; > + > + switch (ch) { > + case '"': > + case ' ': > + state = sw_spaces_before_argument; > + break; > + case CR: > + state = sw_almost_done; > + break; > + case LF: > + goto done; > + } > + break; > + case '\\': > + if (s->quoted) { > + s->backslash = 1; > + state = sw_backslash; > + } > + break; > + } > + break; > + > + case sw_backslash: > + switch (ch) { > + case CR: > + case LF: > + goto invalid; > + default: > + state = sw_argument; > + } > + break; > + > + case sw_literal: > + if (ch >= '0' && ch <= '9') { > + s->literal_len = s->literal_len * 10 + (ch - '0'); > + break; > + } > + if (ch == '}') { > + state = sw_start_literal_argument; > + break; > + } > + if (ch == '+') { > + state = sw_no_sync_literal_argument; > + break; > + } > + goto invalid; > + > + case sw_no_sync_literal_argument: > + if (ch == '}') { > + s->no_sync_literal = 1; > + state = sw_start_literal_argument; > + break; > + } > + goto invalid; > + > + case sw_start_literal_argument: > + switch (ch) { > + case CR: > + break; > + case LF: > + s->buffer->pos = p + 1; > + s->arg_start = p + 1; > + if (s->no_sync_literal == 0) { > + s->state = sw_literal_argument; > + return NGX_SIEVE_NEXT; > + } > + state = sw_literal_argument; > + s->no_sync_literal = 0; > + break; > + default: > + goto invalid; > + } > + break; > + > + case sw_literal_argument: > + if (s->literal_len && --s->literal_len) { > + break; > + } > + > + arg = ngx_array_push(&s->args); > + if (arg == NULL) { > + return NGX_ERROR; > + } > + arg->len = p + 1 - s->arg_start; > + arg->data = s->arg_start; > + s->arg_start = NULL; > + state = sw_end_literal_argument; > + > + break; > + > + case sw_end_literal_argument: > + switch (ch) { > + case '{': > + if (s->args.nelts <= 2) { > + state = sw_literal; > + break; > + } > + goto invalid; > + case CR: > + state = sw_almost_done; > + break; > + case LF: > + goto done; > + default: > + state = sw_spaces_before_argument; > + break; > + } > + break; > + > + case sw_almost_done: > + switch (ch) { > + case LF: > + goto done; > + default: > + goto invalid; > + } > + } > + } > + > + s->buffer->pos = p; > + s->state = state; > + > + return NGX_AGAIN; > + > +done: > + > + s->buffer->pos = p + 1; > + > + if (s->arg_start) { > + arg = ngx_array_push(&s->args); > + if (arg == NULL) { > + return NGX_ERROR; > + } > + arg->len = s->arg_end - s->arg_start; > + arg->data = s->arg_start; > + > + s->arg_start = NULL; > + s->cmd_start = NULL; > + s->quoted = 0; > + s->no_sync_literal = 0; > + s->literal_len = 0; > + } > + > + s->state = (s->command != NGX_SIEVE_AUTHENTICATE) ? sw_start : > sw_spaces_before_argument; > + > + return NGX_OK; > + > +invalid: > + > + s->state = sw_start; > + s->quoted = 0; > + s->no_sync_literal = 0; > + s->literal_len = 0; > + > + return NGX_MAIL_PARSE_INVALID_COMMAND; > +} > + > + > +ngx_int_t > ngx_mail_smtp_parse_command(ngx_mail_session_t *s) > { > u_char ch, *p, *c, c0, c1, c2, c3; > diff -r 0cb942c1c1aa -r f1dffaf61968 src/mail/ngx_mail_pop3_handler.c > --- a/src/mail/ngx_mail_pop3_handler.c Fri Mar 13 02:12:10 2020 +0300 > +++ b/src/mail/ngx_mail_pop3_handler.c Wed Apr 08 20:17:11 2020 +0200 > @@ -492,7 +492,7 @@ > return NGX_MAIL_PARSE_INVALID_COMMAND; > } > - if (ngx_mail_auth_cram_md5_salt(s, c, "+ ", 2) == NGX_OK) { > + if (ngx_mail_auth_cram_md5_salt(s, c, "+ ", 2, 0) == NGX_OK) { > s->mail_state = ngx_pop3_auth_cram_md5; > return NGX_OK; > } > diff -r 0cb942c1c1aa -r f1dffaf61968 src/mail/ngx_mail_proxy_module.c > --- a/src/mail/ngx_mail_proxy_module.c Fri Mar 13 02:12:10 2020 +0300 > +++ b/src/mail/ngx_mail_proxy_module.c Wed Apr 08 20:17:11 2020 +0200 > @@ -24,6 +24,7 @@ > static void ngx_mail_proxy_block_read(ngx_event_t *rev); > static void ngx_mail_proxy_pop3_handler(ngx_event_t *rev); > static void ngx_mail_proxy_imap_handler(ngx_event_t *rev); > +static void ngx_mail_proxy_sieve_handler(ngx_event_t *rev); > static void ngx_mail_proxy_smtp_handler(ngx_event_t *rev); > static void ngx_mail_proxy_dummy_handler(ngx_event_t *ev); > static ngx_int_t ngx_mail_proxy_read_response(ngx_mail_session_t *s, > @@ -173,6 +174,11 @@ > s->mail_state = ngx_imap_start; > break; > + case NGX_MAIL_SIEVE_PROTOCOL: > + p->upstream.connection->read->handler = ngx_mail_proxy_sieve_handler; > + s->mail_state = ngx_sieve_start; > + break; > + > default: /* NGX_MAIL_SMTP_PROTOCOL */ > p->upstream.connection->read->handler = ngx_mail_proxy_smtp_handler; > s->mail_state = ngx_smtp_start; > @@ -446,6 +452,144 @@ > static void > +ngx_mail_proxy_sieve_handler(ngx_event_t *rev) > +{ > + u_char *p; > + ngx_int_t rc; > + ngx_str_t line; > + ngx_str_t userpass; > + ngx_str_t b64_userpass; > + ngx_connection_t *c; > + ngx_mail_session_t *s; > + ngx_mail_proxy_conf_t *pcf; > + > + ngx_log_debug0(NGX_LOG_DEBUG_MAIL, rev->log, 0, > + "mail proxy sieve auth handler"); > + > + c = rev->data; > + s = c->data; > + > + if (rev->timedout) { > + ngx_log_error(NGX_LOG_INFO, c->log, NGX_ETIMEDOUT, > + "upstream timed out"); > + c->timedout = 1; > + ngx_mail_proxy_internal_server_error(s); > + return; > + } > + > + rc = ngx_mail_proxy_read_response(s, s->mail_state); > + > + if (rc == NGX_AGAIN) { > + return; > + } > + > + if (rc == NGX_ERROR) { > + ngx_mail_proxy_upstream_error(s); > + return; > + } > + > + switch (s->mail_state) { > + > + case ngx_sieve_start: > + ngx_log_debug0(NGX_LOG_DEBUG_MAIL, rev->log, 0, > + "mail proxy send login"); > + > + s->connection->log->action = "sending LOGIN command to upstream"; > + > + line.len = s->tag.len + sizeof("AUTHENTICATE ") - 1 > + + 1 + NGX_SIZE_T_LEN + 1 + 2; > + line.data = ngx_pnalloc(c->pool, line.len); > + if (line.data == NULL) { > + ngx_mail_proxy_internal_server_error(s); > + return; > + } > + > + userpass.len = s->login.len + s->passwd.len + 2; > + userpass.data = ngx_pnalloc(c->pool, userpass.len); > + if (userpass.data == NULL) { > + ngx_mail_proxy_internal_server_error(s); > + return; > + } > + b64_userpass.len = ngx_base64_encoded_length(userpass.len) + 2; > + b64_userpass.data = ngx_pnalloc(c->pool, b64_userpass.len); > + if (b64_userpass.data == NULL) { > + ngx_mail_proxy_internal_server_error(s); > + return; > + } > + > + p = userpass.data; > + *p++ = '\0'; > + p = ngx_cpymem(p, s->login.data, s->login.len); > + *p++ = '\0'; > + p = ngx_cpymem(p, s->passwd.data, s->passwd.len); > + > + ngx_encode_base64(&b64_userpass, &userpass); > + line.len = ngx_sprintf(line.data, "AUTHENTICATE \"PLAIN\" \"%V\"" CRLF, > + &b64_userpass) > + - line.data; > + > + s->mail_state = ngx_sieve_login; > + break; > + > + case ngx_sieve_login: > + ngx_log_debug0(NGX_LOG_DEBUG_MAIL, rev->log, 0, > + "mail proxy send capabilities"); > + > + s->connection->log->action = "sending CAPABILITY command to upstream"; > + > + line.len = s->tag.len + sizeof("AUTHENTICATE ") - 1 > + + 1 + NGX_SIZE_T_LEN + 1 + 2; > + line.data = ngx_pnalloc(c->pool, line.len); > + if (line.data == NULL) { > + ngx_mail_proxy_internal_server_error(s); > + return; > + } > + > + line.len = ngx_sprintf(line.data, "CAPABILITY" CRLF) > + - line.data; > + > + s->mail_state = ngx_sieve_login_capabilities; > + break; > + > + case ngx_sieve_login_capabilities: > + s->connection->read->handler = ngx_mail_proxy_handler; > + s->connection->write->handler = ngx_mail_proxy_handler; > + rev->handler = ngx_mail_proxy_handler; > + c->write->handler = ngx_mail_proxy_handler; > + > + pcf = ngx_mail_get_module_srv_conf(s, ngx_mail_proxy_module); > + ngx_add_timer(s->connection->read, pcf->timeout); > + ngx_del_timer(c->read); > + > + c->log->action = NULL; > + ngx_log_error(NGX_LOG_INFO, c->log, 0, "client logged in"); > + > + ngx_mail_proxy_handler(s->connection->write); > + > + return; > + > + default: > +#if (NGX_SUPPRESS_WARN) > + ngx_str_null(&line); > +#endif > + break; > + } > + > + if (c->send(c, line.data, line.len) < (ssize_t) line.len) { > + /* > + * we treat the incomplete sending as NGX_ERROR > + * because it is very strange here > + */ > + ngx_mail_proxy_internal_server_error(s); > + return; > + } > + > + s->proxy->buffer->pos = s->proxy->buffer->start; > + s->proxy->buffer->last = s->proxy->buffer->start; > +} > + > + > +static void > ngx_mail_proxy_smtp_handler(ngx_event_t *rev) > { > u_char *p; > @@ -793,6 +937,42 @@ > break; > + case NGX_MAIL_SIEVE_PROTOCOL: > + if (p[0] == '"') { > + m = b->last - (sizeof(CRLF "OK" CRLF) - 1); > + > + while (m > p) { > + if (m[0] == CR && m[1] == LF) { > + break; > + } > + m--; > + } > + > + if (m <= p || m[0] == '"') { > + return NGX_AGAIN; > + } > + p = m + 2; > + } > + > + switch (state) { > + > + case ngx_sieve_start: > + case ngx_sieve_login: > + ngx_log_error(NGX_LOG_INFO, s->connection->log, 0, "1 upstream sent: > \"%s\"", p); > + if (p[0] == 'O' && p[1] == 'K') { > + return NGX_OK; > + } > + break; > + case ngx_sieve_login_capabilities: > + ngx_log_error(NGX_LOG_INFO, s->connection->log, 0, "2 upstream sent: > \"%s\"", p); > + if (p[0] == 'O' && p[1] == 'K') { > + return NGX_OK; > + } > + break; > + } > + > + break; > + > default: /* NGX_MAIL_SMTP_PROTOCOL */ > if (p[3] == '-') { > diff -r 0cb942c1c1aa -r f1dffaf61968 src/mail/ngx_mail_smtp_handler.c > --- a/src/mail/ngx_mail_smtp_handler.c Fri Mar 13 02:12:10 2020 +0300 > +++ b/src/mail/ngx_mail_smtp_handler.c Wed Apr 08 20:17:11 2020 +0200 > @@ -693,7 +693,7 @@ > } > } > - if (ngx_mail_auth_cram_md5_salt(s, c, "334 ", 4) == NGX_OK) { > + if (ngx_mail_auth_cram_md5_salt(s, c, "334 ", 4, 0) == NGX_OK) { > s->mail_state = ngx_smtp_auth_cram_md5; > return NGX_OK; > } > diff -r 0cb942c1c1aa -r f1dffaf61968 src/misc/ngx_cpp_test_module.cpp > --- a/src/misc/ngx_cpp_test_module.cpp Fri Mar 13 02:12:10 2020 +0300 > +++ b/src/misc/ngx_cpp_test_module.cpp Wed Apr 08 20:17:11 2020 +0200 > @@ -13,6 +13,7 @@ > #include > #include > #include > + #include > #include > } > > > _______________________________________________ > nginx-devel mailing list > nginx-devel at nginx.org > http://mailman.nginx.org/mailman/listinfo/nginx-devel > From sander at hoentjen.eu Wed Apr 8 18:38:20 2020 From: sander at hoentjen.eu (Sander Hoentjen) Date: Wed, 8 Apr 2020 20:38:20 +0200 Subject: [PATCH] Added support for proxying managesieve protocol (fixed format) In-Reply-To: <58016a3a-c11d-bb31-2b05-916cb8124598@hoentjen.eu> References: <58016a3a-c11d-bb31-2b05-916cb8124598@hoentjen.eu> Message-ID: <8ffb9d91-7fdd-cc01-c775-5faa0f911296@hoentjen.eu> # HG changeset patch # User Sander Hoentjen # Date 1586369831 -7200 # Wed Apr 08 20:17:11 2020 +0200 # Node ID f1dffaf619688aaab90caf31781ebe27c3f79598 # Parent 0cb942c1c1aa98118076e72e0b89940e85e6291c Added support for proxying managesieve protocol Sieve is a widely used protocol to implement server-side filtering. Some servers and clients that support this are listed at. This commit is tested with Dovecot as the server and Roundcube as the client, but I don't see a readon why it not would work with others. It is a nice addition to nginx because it adds the possibility to add server side filtering to setups that use IMAP behind an nginx proxy. Fixeshttps://trac.nginx.org/nginx/ticket/1697 diff -r 0cb942c1c1aa -r f1dffaf61968 auto/modules --- a/auto/modules Fri Mar 13 02:12:10 2020 +0300 +++ b/auto/modules Wed Apr 08 20:17:11 2020 +0200 @@ -965,6 +965,15 @@ . auto/module fi + if [ $MAIL_SIEVE = YES ]; then + ngx_module_name=ngx_mail_sieve_module + ngx_module_deps=src/mail/ngx_mail_sieve_module.h + ngx_module_srcs="src/mail/ngx_mail_sieve_module.c \ + src/mail/ngx_mail_sieve_handler.c" + + . auto/module + fi + if [ $MAIL_SMTP = YES ]; then ngx_module_name=ngx_mail_smtp_module ngx_module_deps=src/mail/ngx_mail_smtp_module.h diff -r 0cb942c1c1aa -r f1dffaf61968 auto/options --- a/auto/options Fri Mar 13 02:12:10 2020 +0300 +++ b/auto/options Wed Apr 08 20:17:11 2020 +0200 @@ -112,6 +112,7 @@ MAIL_SSL=NO MAIL_POP3=YES MAIL_IMAP=YES +MAIL_SIEVE=YES MAIL_SMTP=YES STREAM=NO @@ -305,6 +306,8 @@ ;; --without-mail_pop3_module) MAIL_POP3=NO ;; --without-mail_imap_module) MAIL_IMAP=NO ;; + --without-mail_sieve_module) + MAIL_SIEVE=NO ;; --without-mail_smtp_module) MAIL_SMTP=NO ;; --with-stream) STREAM=YES ;; @@ -517,11 +520,12 @@ --without-http disable HTTP server --without-http-cache disable HTTP cache - --with-mail enable POP3/IMAP4/SMTP proxy module - --with-mail=dynamic enable dynamic POP3/IMAP4/SMTP proxy module + --with-mail enable POP3/IMAP4/SIEVE/SMTP proxy module + --with-mail=dynamic enable dynamic POP3/IMAP4/SIEVE/SMTP proxy module --with-mail_ssl_module enable ngx_mail_ssl_module --without-mail_pop3_module disable ngx_mail_pop3_module --without-mail_imap_module disable ngx_mail_imap_module + --without-mail_sieve_module disable ngx_mail_sieve_module --without-mail_smtp_module disable ngx_mail_smtp_module --with-stream enable TCP/UDP proxy module diff -r 0cb942c1c1aa -r f1dffaf61968 contrib/vim/syntax/nginx.vim --- a/contrib/vim/syntax/nginx.vim Fri Mar 13 02:12:10 2020 +0300 +++ b/contrib/vim/syntax/nginx.vim Wed Apr 08 20:17:11 2020 +0200 @@ -571,6 +571,9 @@ syn keyword ngxDirective contained session_log_format syn keyword ngxDirective contained session_log_zone syn keyword ngxDirective contained set_real_ip_from +syn keyword ngxDirective contained sieve_auth +syn keyword ngxDirective contained sieve_capabilities +syn keyword ngxDirective contained sieve_client_buffer syn keyword ngxDirective contained slice syn keyword ngxDirective contained smtp_auth syn keyword ngxDirective contained smtp_capabilities diff -r 0cb942c1c1aa -r f1dffaf61968 src/mail/ngx_mail.h --- a/src/mail/ngx_mail.h Fri Mar 13 02:12:10 2020 +0300 +++ b/src/mail/ngx_mail.h Wed Apr 08 20:17:11 2020 +0200 @@ -102,6 +102,7 @@ #define NGX_MAIL_POP3_PROTOCOL 0 #define NGX_MAIL_IMAP_PROTOCOL 1 #define NGX_MAIL_SMTP_PROTOCOL 2 +#define NGX_MAIL_SIEVE_PROTOCOL 3 typedef struct ngx_mail_protocol_s ngx_mail_protocol_t; @@ -154,6 +155,21 @@ typedef enum { + ngx_sieve_start = 0, + ngx_sieve_starttls, + ngx_sieve_auth_login_username, + ngx_sieve_auth_login_password, + ngx_sieve_auth_plain, + ngx_sieve_auth_cram_md5, + ngx_sieve_auth_external, + ngx_sieve_login, + ngx_sieve_login_capabilities, + ngx_sieve_user, + ngx_sieve_passwd +} ngx_sieve_state_e; + + +typedef enum { ngx_smtp_start = 0, ngx_smtp_auth_login_username, ngx_smtp_auth_login_password, @@ -227,7 +243,7 @@ ngx_uint_t login_attempt; - /* used to parse POP3/IMAP/SMTP command */ + /* used to parse POP3/IMAP/SIEVE/SMTP command */ ngx_uint_t state; u_char *cmd_start; @@ -271,6 +287,14 @@ #define NGX_IMAP_AUTHENTICATE 7 +#define NGX_SIEVE_LOGOUT 1 +#define NGX_SIEVE_CAPABILITY 2 +#define NGX_SIEVE_NOOP 3 +#define NGX_SIEVE_STARTTLS 4 +#define NGX_SIEVE_NEXT 5 +#define NGX_SIEVE_AUTHENTICATE 6 + + #define NGX_SMTP_HELO 1 #define NGX_SMTP_EHLO 2 #define NGX_SMTP_AUTH 3 @@ -383,7 +407,7 @@ ngx_int_t ngx_mail_auth_login_password(ngx_mail_session_t *s, ngx_connection_t *c); ngx_int_t ngx_mail_auth_cram_md5_salt(ngx_mail_session_t *s, - ngx_connection_t *c, char *prefix, size_t len); + ngx_connection_t *c, char *prefix, size_t len, ngx_uint_t quote); ngx_int_t ngx_mail_auth_cram_md5(ngx_mail_session_t *s, ngx_connection_t *c); ngx_int_t ngx_mail_auth_external(ngx_mail_session_t *s, ngx_connection_t *c, ngx_uint_t n); diff -r 0cb942c1c1aa -r f1dffaf61968 src/mail/ngx_mail_auth_http_module.c --- a/src/mail/ngx_mail_auth_http_module.c Fri Mar 13 02:12:10 2020 +0300 +++ b/src/mail/ngx_mail_auth_http_module.c Wed Apr 08 20:17:11 2020 +0200 @@ -533,6 +533,11 @@ + sizeof(CRLF) - 1; break; + case NGX_MAIL_SIEVE_PROTOCOL: + size = sizeof("BYE \"\"") - 1 + len + + sizeof(CRLF) - 1; + break; + default: /* NGX_MAIL_SMTP_PROTOCOL */ ctx->err = ctx->errmsg; continue; @@ -559,11 +564,27 @@ *p++ = 'N'; *p++ = 'O'; *p++ = ' '; break; + case NGX_MAIL_SIEVE_PROTOCOL: + p = ngx_cpymem(p, s->tag.data, s->tag.len); + *p++ = 'B'; *p++ = 'Y'; *p++ = 'E'; *p++ = ' '; *p++ = '"'; + break; + default: /* NGX_MAIL_SMTP_PROTOCOL */ break; } p = ngx_cpymem(p, ctx->header_start, len); + + switch (s->protocol) { + + case NGX_MAIL_SIEVE_PROTOCOL: + *p++ = '"'; + break; + + default: + break; + } + *p++ = CR; *p++ = LF; ctx->err.len = p - ctx->err.data; diff -r 0cb942c1c1aa -r f1dffaf61968 src/mail/ngx_mail_handler.c --- a/src/mail/ngx_mail_handler.c Fri Mar 13 02:12:10 2020 +0300 +++ b/src/mail/ngx_mail_handler.c Wed Apr 08 20:17:11 2020 +0200 @@ -520,28 +520,39 @@ ngx_int_t ngx_mail_auth_cram_md5_salt(ngx_mail_session_t *s, ngx_connection_t *c, - char *prefix, size_t len) + char *prefix, size_t len, ngx_uint_t quote) { u_char *p; ngx_str_t salt; - ngx_uint_t n; + ngx_uint_t n = 0; + if (quote) { + len += 2; + } p = ngx_pnalloc(c->pool, len + ngx_base64_encoded_length(s->salt.len) + 2); if (p == NULL) { return NGX_ERROR; } - salt.data = ngx_cpymem(p, prefix, len); + if (quote) { + len -= 2; + salt.data = ngx_cpymem(p, "\"", 1); + n++; + } + salt.data = ngx_cpymem(p+ n, prefix, len); s->salt.len -= 2; ngx_encode_base64(&salt, &s->salt); s->salt.len += 2; - n = len + salt.len; + n += len + salt.len; + if (quote) { + p[n++] = '"'; + } p[n++] = CR; p[n++] = LF; s->out.len = n; - s->out.data = p; + s->out.data = p--; return NGX_OK; } diff -r 0cb942c1c1aa -r f1dffaf61968 src/mail/ngx_mail_imap_handler.c --- a/src/mail/ngx_mail_imap_handler.c Fri Mar 13 02:12:10 2020 +0300 +++ b/src/mail/ngx_mail_imap_handler.c Wed Apr 08 20:17:11 2020 +0200 @@ -397,7 +397,7 @@ } } - if (ngx_mail_auth_cram_md5_salt(s, c, "+ ", 2) == NGX_OK) { + if (ngx_mail_auth_cram_md5_salt(s, c, "+ ", 2, 0) == NGX_OK) { s->mail_state = ngx_imap_auth_cram_md5; return NGX_OK; } diff -r 0cb942c1c1aa -r f1dffaf61968 src/mail/ngx_mail_parse.c --- a/src/mail/ngx_mail_parse.c Fri Mar 13 02:12:10 2020 +0300 +++ b/src/mail/ngx_mail_parse.c Wed Apr 08 20:17:11 2020 +0200 @@ -11,6 +11,7 @@ #include #include #include +#include #include @@ -620,6 +621,364 @@ ngx_int_t +ngx_mail_sieve_parse_command(ngx_mail_session_t *s) +{ + u_char ch, *p, *c; + ngx_str_t *arg; + enum { + sw_start = 0, + sw_spaces_before_argument, + sw_argument, + sw_backslash, + sw_literal, + sw_no_sync_literal_argument, + sw_start_literal_argument, + sw_literal_argument, + sw_end_literal_argument, + sw_almost_done + } state; + + state = s->state; + + for (p = s->buffer->pos; p < s->buffer->last; p++) { + ch = *p; + + switch (state) { + + /* SIEVE command */ + case sw_start: + if (ch == ' ' || ch == CR || ch == LF) { + + c = s->buffer->start; + + switch (p - c) { + + case 4: + if ((c[0] == 'N' || c[0] == 'n') + && (c[1] == 'O'|| c[1] == 'o') + && (c[2] == 'O'|| c[2] == 'o') + && (c[3] == 'P'|| c[3] == 'p')) + { + s->command = NGX_SIEVE_NOOP; + + } else { + goto invalid; + } + break; + + case 6: + if ((c[0] == 'L'|| c[0] == 'l') + && (c[1] == 'O'|| c[1] == 'o') + && (c[2] == 'G'|| c[2] == 'g') + && (c[3] == 'O'|| c[3] == 'o') + && (c[4] == 'U'|| c[4] == 'u') + && (c[5] == 'T'|| c[5] == 't')) + { + s->command = NGX_SIEVE_LOGOUT; + + } else { + goto invalid; + } + break; + +#if (NGX_MAIL_SSL) + case 8: + if ((c[0] == 'S'|| c[0] == 's') + && (c[1] == 'T'|| c[1] == 't') + && (c[2] == 'A'|| c[2] == 'a') + && (c[3] == 'R'|| c[3] == 'r') + && (c[4] == 'T'|| c[4] == 't') + && (c[5] == 'T'|| c[5] == 't') + && (c[6] == 'L'|| c[6] == 'l') + && (c[7] == 'S'|| c[7] == 's')) + { + s->command = NGX_SIEVE_STARTTLS; + + } else { + goto invalid; + } + break; +#endif + + case 10: + if ((c[0] == 'C'|| c[0] == 'c') + && (c[1] == 'A'|| c[1] == 'a') + && (c[2] == 'P'|| c[2] == 'p') + && (c[3] == 'A'|| c[3] == 'a') + && (c[4] == 'B'|| c[4] == 'b') + && (c[5] == 'I'|| c[5] == 'i') + && (c[6] == 'L'|| c[6] == 'l') + && (c[7] == 'I'|| c[7] == 'i') + && (c[8] == 'T'|| c[8] == 't') + && (c[9] == 'Y'|| c[9] == 'y')) + { + s->command = NGX_SIEVE_CAPABILITY; + + } else { + goto invalid; + } + break; + + case 12: + if ((c[0] == 'A'|| c[0] == 'a') + && (c[1] == 'U'|| c[1] == 'u') + && (c[2] == 'T'|| c[2] == 't') + && (c[3] == 'H'|| c[3] == 'h') + && (c[4] == 'E'|| c[4] == 'e') + && (c[5] == 'N'|| c[5] == 'n') + && (c[6] == 'T'|| c[6] == 't') + && (c[7] == 'I'|| c[7] == 'i') + && (c[8] == 'C'|| c[8] == 'c') + && (c[9] == 'A'|| c[9] == 'a') + && (c[10] == 'T'|| c[10] == 't') + && (c[11] == 'E'|| c[11] == 'e')) + { + s->command = NGX_SIEVE_AUTHENTICATE; + + } else { + goto invalid; + } + break; + + default: + goto invalid; + } + + switch (ch) { + case ' ': + state = sw_spaces_before_argument; + break; + case CR: + state = sw_almost_done; + break; + case LF: + goto done; + } + break; + } + + if ((ch < 'A' || ch > 'Z') && (ch < 'a' || ch > 'z')) { + goto invalid; + } + + break; + + case sw_spaces_before_argument: + switch (ch) { + case ' ': + break; + case CR: + state = sw_almost_done; + s->arg_end = p; + break; + case LF: + s->arg_end = p; + goto done; + case '"': + if (s->args.nelts <= 2) { + s->quoted = 1; + s->arg_start = p + 1; + state = sw_argument; + break; + } + goto invalid; + case '{': + if (s->args.nelts <= 2) { + state = sw_literal; + break; + } + goto invalid; + default: + if (s->args.nelts <= 2) { + s->arg_start = p; + state = sw_argument; + break; + } + goto invalid; + } + break; + + case sw_argument: + if (ch == ' ' && s->quoted) { + break; + } + + switch (ch) { + case '"': + if (!s->quoted) { + break; + } + s->quoted = 0; + /* fall through */ + case ' ': + case CR: + case LF: + arg = ngx_array_push(&s->args); + if (arg == NULL) { + return NGX_ERROR; + } + arg->len = p - s->arg_start; + arg->data = s->arg_start; + s->arg_start = NULL; + + switch (ch) { + case '"': + case ' ': + state = sw_spaces_before_argument; + break; + case CR: + state = sw_almost_done; + break; + case LF: + goto done; + } + break; + case '\\': + if (s->quoted) { + s->backslash = 1; + state = sw_backslash; + } + break; + } + break; + + case sw_backslash: + switch (ch) { + case CR: + case LF: + goto invalid; + default: + state = sw_argument; + } + break; + + case sw_literal: + if (ch >= '0' && ch <= '9') { + s->literal_len = s->literal_len * 10 + (ch - '0'); + break; + } + if (ch == '}') { + state = sw_start_literal_argument; + break; + } + if (ch == '+') { + state = sw_no_sync_literal_argument; + break; + } + goto invalid; + + case sw_no_sync_literal_argument: + if (ch == '}') { + s->no_sync_literal = 1; + state = sw_start_literal_argument; + break; + } + goto invalid; + + case sw_start_literal_argument: + switch (ch) { + case CR: + break; + case LF: + s->buffer->pos = p + 1; + s->arg_start = p + 1; + if (s->no_sync_literal == 0) { + s->state = sw_literal_argument; + return NGX_SIEVE_NEXT; + } + state = sw_literal_argument; + s->no_sync_literal = 0; + break; + default: + goto invalid; + } + break; + + case sw_literal_argument: + if (s->literal_len && --s->literal_len) { + break; + } + + arg = ngx_array_push(&s->args); + if (arg == NULL) { + return NGX_ERROR; + } + arg->len = p + 1 - s->arg_start; + arg->data = s->arg_start; + s->arg_start = NULL; + state = sw_end_literal_argument; + + break; + + case sw_end_literal_argument: + switch (ch) { + case '{': + if (s->args.nelts <= 2) { + state = sw_literal; + break; + } + goto invalid; + case CR: + state = sw_almost_done; + break; + case LF: + goto done; + default: + state = sw_spaces_before_argument; + break; + } + break; + + case sw_almost_done: + switch (ch) { + case LF: + goto done; + default: + goto invalid; + } + } + } + + s->buffer->pos = p; + s->state = state; + + return NGX_AGAIN; + +done: + + s->buffer->pos = p + 1; + + if (s->arg_start) { + arg = ngx_array_push(&s->args); + if (arg == NULL) { + return NGX_ERROR; + } + arg->len = s->arg_end - s->arg_start; + arg->data = s->arg_start; + + s->arg_start = NULL; + s->cmd_start = NULL; + s->quoted = 0; + s->no_sync_literal = 0; + s->literal_len = 0; + } + + s->state = (s->command != NGX_SIEVE_AUTHENTICATE) ? sw_start : sw_spaces_before_argument; + + return NGX_OK; + +invalid: + + s->state = sw_start; + s->quoted = 0; + s->no_sync_literal = 0; + s->literal_len = 0; + + return NGX_MAIL_PARSE_INVALID_COMMAND; +} + + +ngx_int_t ngx_mail_smtp_parse_command(ngx_mail_session_t *s) { u_char ch, *p, *c, c0, c1, c2, c3; diff -r 0cb942c1c1aa -r f1dffaf61968 src/mail/ngx_mail_pop3_handler.c --- a/src/mail/ngx_mail_pop3_handler.c Fri Mar 13 02:12:10 2020 +0300 +++ b/src/mail/ngx_mail_pop3_handler.c Wed Apr 08 20:17:11 2020 +0200 @@ -492,7 +492,7 @@ return NGX_MAIL_PARSE_INVALID_COMMAND; } - if (ngx_mail_auth_cram_md5_salt(s, c, "+ ", 2) == NGX_OK) { + if (ngx_mail_auth_cram_md5_salt(s, c, "+ ", 2, 0) == NGX_OK) { s->mail_state = ngx_pop3_auth_cram_md5; return NGX_OK; } diff -r 0cb942c1c1aa -r f1dffaf61968 src/mail/ngx_mail_proxy_module.c --- a/src/mail/ngx_mail_proxy_module.c Fri Mar 13 02:12:10 2020 +0300 +++ b/src/mail/ngx_mail_proxy_module.c Wed Apr 08 20:17:11 2020 +0200 @@ -24,6 +24,7 @@ static void ngx_mail_proxy_block_read(ngx_event_t *rev); static void ngx_mail_proxy_pop3_handler(ngx_event_t *rev); static void ngx_mail_proxy_imap_handler(ngx_event_t *rev); +static void ngx_mail_proxy_sieve_handler(ngx_event_t *rev); static void ngx_mail_proxy_smtp_handler(ngx_event_t *rev); static void ngx_mail_proxy_dummy_handler(ngx_event_t *ev); static ngx_int_t ngx_mail_proxy_read_response(ngx_mail_session_t *s, @@ -173,6 +174,11 @@ s->mail_state = ngx_imap_start; break; + case NGX_MAIL_SIEVE_PROTOCOL: + p->upstream.connection->read->handler = ngx_mail_proxy_sieve_handler; + s->mail_state = ngx_sieve_start; + break; + default: /* NGX_MAIL_SMTP_PROTOCOL */ p->upstream.connection->read->handler = ngx_mail_proxy_smtp_handler; s->mail_state = ngx_smtp_start; @@ -446,6 +452,144 @@ static void +ngx_mail_proxy_sieve_handler(ngx_event_t *rev) +{ + u_char *p; + ngx_int_t rc; + ngx_str_t line; + ngx_str_t userpass; + ngx_str_t b64_userpass; + ngx_connection_t *c; + ngx_mail_session_t *s; + ngx_mail_proxy_conf_t *pcf; + + ngx_log_debug0(NGX_LOG_DEBUG_MAIL, rev->log, 0, + "mail proxy sieve auth handler"); + + c = rev->data; + s = c->data; + + if (rev->timedout) { + ngx_log_error(NGX_LOG_INFO, c->log, NGX_ETIMEDOUT, + "upstream timed out"); + c->timedout = 1; + ngx_mail_proxy_internal_server_error(s); + return; + } + + rc = ngx_mail_proxy_read_response(s, s->mail_state); + + if (rc == NGX_AGAIN) { + return; + } + + if (rc == NGX_ERROR) { + ngx_mail_proxy_upstream_error(s); + return; + } + + switch (s->mail_state) { + + case ngx_sieve_start: + ngx_log_debug0(NGX_LOG_DEBUG_MAIL, rev->log, 0, + "mail proxy send login"); + + s->connection->log->action = "sending LOGIN command to upstream"; + + line.len = s->tag.len + sizeof("AUTHENTICATE ") - 1 + + 1 + NGX_SIZE_T_LEN + 1 + 2; + line.data = ngx_pnalloc(c->pool, line.len); + if (line.data == NULL) { + ngx_mail_proxy_internal_server_error(s); + return; + } + + userpass.len = s->login.len + s->passwd.len + 2; + userpass.data = ngx_pnalloc(c->pool, userpass.len); + if (userpass.data == NULL) { + ngx_mail_proxy_internal_server_error(s); + return; + } + b64_userpass.len = ngx_base64_encoded_length(userpass.len) + 2; + b64_userpass.data = ngx_pnalloc(c->pool, b64_userpass.len); + if (b64_userpass.data == NULL) { + ngx_mail_proxy_internal_server_error(s); + return; + } + + p = userpass.data; + *p++ = '\0'; + p = ngx_cpymem(p, s->login.data, s->login.len); + *p++ = '\0'; + p = ngx_cpymem(p, s->passwd.data, s->passwd.len); + + ngx_encode_base64(&b64_userpass, &userpass); + line.len = ngx_sprintf(line.data, "AUTHENTICATE \"PLAIN\" \"%V\"" CRLF, + &b64_userpass) + - line.data; + + s->mail_state = ngx_sieve_login; + break; + + case ngx_sieve_login: + ngx_log_debug0(NGX_LOG_DEBUG_MAIL, rev->log, 0, + "mail proxy send capabilities"); + + s->connection->log->action = "sending CAPABILITY command to upstream"; + + line.len = s->tag.len + sizeof("AUTHENTICATE ") - 1 + + 1 + NGX_SIZE_T_LEN + 1 + 2; + line.data = ngx_pnalloc(c->pool, line.len); + if (line.data == NULL) { + ngx_mail_proxy_internal_server_error(s); + return; + } + + line.len = ngx_sprintf(line.data, "CAPABILITY" CRLF) + - line.data; + + s->mail_state = ngx_sieve_login_capabilities; + break; + + case ngx_sieve_login_capabilities: + s->connection->read->handler = ngx_mail_proxy_handler; + s->connection->write->handler = ngx_mail_proxy_handler; + rev->handler = ngx_mail_proxy_handler; + c->write->handler = ngx_mail_proxy_handler; + + pcf = ngx_mail_get_module_srv_conf(s, ngx_mail_proxy_module); + ngx_add_timer(s->connection->read, pcf->timeout); + ngx_del_timer(c->read); + + c->log->action = NULL; + ngx_log_error(NGX_LOG_INFO, c->log, 0, "client logged in"); + + ngx_mail_proxy_handler(s->connection->write); + + return; + + default: +#if (NGX_SUPPRESS_WARN) + ngx_str_null(&line); +#endif + break; + } + + if (c->send(c, line.data, line.len) < (ssize_t) line.len) { + /* + * we treat the incomplete sending as NGX_ERROR + * because it is very strange here + */ + ngx_mail_proxy_internal_server_error(s); + return; + } + + s->proxy->buffer->pos = s->proxy->buffer->start; + s->proxy->buffer->last = s->proxy->buffer->start; +} + + +static void ngx_mail_proxy_smtp_handler(ngx_event_t *rev) { u_char *p; @@ -793,6 +937,42 @@ break; + case NGX_MAIL_SIEVE_PROTOCOL: + if (p[0] == '"') { + m = b->last - (sizeof(CRLF "OK" CRLF) - 1); + + while (m > p) { + if (m[0] == CR && m[1] == LF) { + break; + } + m--; + } + + if (m <= p || m[0] == '"') { + return NGX_AGAIN; + } + p = m + 2; + } + + switch (state) { + + case ngx_sieve_start: + case ngx_sieve_login: + ngx_log_error(NGX_LOG_INFO, s->connection->log, 0, "1 upstream sent: \"%s\"", p); + if (p[0] == 'O' && p[1] == 'K') { + return NGX_OK; + } + break; + case ngx_sieve_login_capabilities: + ngx_log_error(NGX_LOG_INFO, s->connection->log, 0, "2 upstream sent: \"%s\"", p); + if (p[0] == 'O' && p[1] == 'K') { + return NGX_OK; + } + break; + } + + break; + default: /* NGX_MAIL_SMTP_PROTOCOL */ if (p[3] == '-') { diff -r 0cb942c1c1aa -r f1dffaf61968 src/mail/ngx_mail_smtp_handler.c --- a/src/mail/ngx_mail_smtp_handler.c Fri Mar 13 02:12:10 2020 +0300 +++ b/src/mail/ngx_mail_smtp_handler.c Wed Apr 08 20:17:11 2020 +0200 @@ -693,7 +693,7 @@ } } - if (ngx_mail_auth_cram_md5_salt(s, c, "334 ", 4) == NGX_OK) { + if (ngx_mail_auth_cram_md5_salt(s, c, "334 ", 4, 0) == NGX_OK) { s->mail_state = ngx_smtp_auth_cram_md5; return NGX_OK; } diff -r 0cb942c1c1aa -r f1dffaf61968 src/misc/ngx_cpp_test_module.cpp --- a/src/misc/ngx_cpp_test_module.cpp Fri Mar 13 02:12:10 2020 +0300 +++ b/src/misc/ngx_cpp_test_module.cpp Wed Apr 08 20:17:11 2020 +0200 @@ -13,6 +13,7 @@ #include #include #include + #include #include } -------------- next part -------------- An HTML attachment was scrubbed... URL: From sander at hoentjen.eu Wed Apr 8 18:39:06 2020 From: sander at hoentjen.eu (Sander Hoentjen) Date: Wed, 8 Apr 2020 20:39:06 +0200 Subject: [PATCH] Tests: added sieve tests (fixed format) In-Reply-To: <58016a3a-c11d-bb31-2b05-916cb8124598@hoentjen.eu> References: <58016a3a-c11d-bb31-2b05-916cb8124598@hoentjen.eu> Message-ID: # HG changeset patch # User Sander Hoentjen # Date 1586370310 -7200 # Wed Apr 08 20:25:10 2020 +0200 # Node ID 3d35b19abfa72d1d09e23d02917df7fbdee0970c # Parent 9e5d38da765152a20098642e37b0afe56312f794 Tests: added sieve tests diff -r 9e5d38da7651 -r 3d35b19abfa7 lib/Test/Nginx.pm --- a/lib/Test/Nginx.pm Fri Mar 20 16:32:06 2020 +0300 +++ b/lib/Test/Nginx.pm Wed Apr 08 20:25:10 2020 +0200 @@ -165,6 +165,7 @@ cache => '(?s)^(?!.*--without-http-cache)', pop3 => '(?s)^(?!.*--without-mail_pop3_module)', imap => '(?s)^(?!.*--without-mail_imap_module)', + sieve => '(?s)^(?!.*--without-mail_sieve_module)', smtp => '(?s)^(?!.*--without-mail_smtp_module)', pcre => '(?s)^(?!.*--without-pcre)', split_clients diff -r 9e5d38da7651 -r 3d35b19abfa7 lib/Test/Nginx/SIEVE.pm --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/lib/Test/Nginx/SIEVE.pm Wed Apr 08 20:25:10 2020 +0200 @@ -0,0 +1,156 @@ +packageTest::Nginx::SIEVE; + +# (C) Maxim Dounin + +# Module for nginx sieve tests. + +############################################################################### + +use warnings; +use strict; + +useTest::More qw//; +useIO::Select; +useIO::Socket; +use Socket qw/ CRLF /; + +useTest::Nginx; + +sub new { + my $self = {}; + bless $self, shift @_; + + $self->{_socket} =IO::Socket::INET->new( + Proto => "tcp", + PeerAddr => "127.0.0.1:" . port(8200), + @_ + ) + or die "Can't connect to nginx: $!\n"; + + if ({@_}->{'SSL'}) { + requireIO::Socket::SSL; + IO::Socket::SSL->start_SSL($self->{_socket}, @_) + or die $IO::Socket::SSL::SSL_ERROR . "\n"; + } + + $self->{_socket}->autoflush(1); + + return $self; +} + +sub eof { + my $self = shift; + return $self->{_socket}->eof(); +} + +sub print { + my ($self, $cmd) = @_; + log_out($cmd); + $self->{_socket}->print($cmd); +} + +sub send { + my ($self, $cmd) = @_; + #warn "\n>>>$cmd\n"; + log_out($cmd); + $self->{_socket}->print($cmd . CRLF); +} + +sub read { + my ($self) = @_; + my $socket = $self->{_socket}; + eval { + local $SIG{ALRM} = sub { die "timeout\n" }; + alarm(8); + while (<$socket>) { + #warn "\n====\n<<<$_\n===\n"; + log_in($_); + # XXX + last; + } + alarm(0); + }; + alarm(0); + if ($@) { + log_in("died: $@"); + return undef; + } + return $_; +} + +sub read_ok { + my ($self) = @_; + my $socket = $self->{_socket}; + eval { + local $SIG{ALRM} = sub { die "timeout\n" }; + alarm(8); + while (<$socket>) { + #warn "\n====\n<<<$_\n===\n"; + log_in($_); + # XXX + next if m/^"/; + last; + } + alarm(0); + }; + alarm(0); + if ($@) { + log_in("died: $@"); + return undef; + } + return $_; +} + +sub check { + my ($self, $regex, $name) = @_; + Test::More->builder->like($self->read(), $regex, $name); +} + +sub ok { + my $self = shift; + Test::More->builder->like($self->read_ok(), qr/^OK/, @_); +} + +sub can_read { + my ($self, $timo) = @_; + IO::Select->new($self->{_socket})->can_read($timo || 3); +} + +############################################################################### + +sub sieve_test_daemon { + my ($port) = @_; + + my $server =IO::Socket::INET->new( + Proto => 'tcp', + LocalAddr => '127.0.0.1:' . ($port || port(8201)), + Listen => 5, + Reuse => 1 + ) + or die "Can't create listening socket: $!\n"; + + while (my $client = $server->accept()) { + $client->autoflush(1); + print $client "OK fake sieve server ready" . CRLF; + + while (<$client>) { + if (/^logout/i) { + print $client 'OK logout ok' . CRLF; + } elsif (/^AUTHENTICATE /i) { + print $client 'OK login ok' . CRLF; + } elsif (/^CAPABILITY/i) { + print $client 'OK capabilty ok' . CRLF; + } else { + print $client 'NO unknown command ' . $_ . CRLF; + } + } + + close $client; + } +} + +############################################################################### + +1; + +############################################################################### diff -r 9e5d38da7651 -r 3d35b19abfa7 mail_capability.t --- a/mail_capability.t Fri Mar 20 16:32:06 2020 +0300 +++ b/mail_capability.t Wed Apr 08 20:25:10 2020 +0200 @@ -3,7 +3,7 @@ # (C) Sergey Kandaurov # (C) Nginx, Inc. -# Tests for imap/pop3/smtp capabilities. +# Tests for imap/pop3/sieve/smtp capabilities. ############################################################################### @@ -18,6 +18,7 @@ useTest::Nginx; useTest::Nginx::IMAP; useTest::Nginx::POP3; +useTest::Nginx::SIEVE; useTest::Nginx::SMTP; ############################################################################### @@ -25,8 +26,8 @@ select STDERR; $| = 1; select STDOUT; $| = 1; -my $t =Test::Nginx->new()->has(qw/mail mail_ssl imap pop3 smtp/) - ->has_daemon('openssl')->plan(17); +my $t =Test::Nginx->new()->has(qw/mail mail_ssl imap pop3 sieve smtp/) + ->has_daemon('openssl')->plan(25); $t->write_file_expand('nginx.conf', <<'EOF'); @@ -81,6 +82,24 @@ } server { + listen 127.0.0.1:8200; + protocol sieve; + sieve_capabilities '"SEE-THIS"'; + } + + server { + listen 127.0.0.1:8201; + protocol sieve; + starttls on; + } + + server { + listen 127.0.0.1:8202; + protocol sieve; + starttls only; + } + + server { listen 127.0.0.1:8025; protocol smtp; starttls off; @@ -188,6 +207,38 @@ unlike($caps, qr/SASL/, 'pop3 starttls only - no methods'); like($caps, qr/STLS/, 'pop3 startls only - stls'); +# sieve, custom capabilities + +$s =Test::Nginx::SIEVE->new(PeerAddr => '127.0.0.1:' . port(8200)); +$s->ok('sieve connection completed'); + +$s->send('CAPABILITY'); +$s->check(qr/^"SEE-THIS"/, 'sieve capability custom'); +$s->check(qr/^"SASL" "PLAIN"/, 'sieve capability sasl'); +$s->ok('sieve capability completed'); + +# sieve starttls + +$s =Test::Nginx::SIEVE->new(PeerAddr => '127.0.0.1:' . port(8201)); +$s->read(); +$s->read(); +$s->read(); +$s->check(qr/^"SASL" "PLAIN"/, + 'sieve capability starttls has plain'); +$s->check(qr/^"STARTTLS"/, + 'sieve capability starttls'); + +# sieve starttls only + +$s =Test::Nginx::SIEVE->new(PeerAddr => '127.0.0.1:' . port(8202)); +$s->read(); +$s->read(); +$s->read(); +$s->check(qr/^"SASL" ""/, + 'sieve capability starttls only not has plain'); +$s->check(qr/^"STARTTLS"/, + 'sieve capability starttls only'); + # smtp $s =Test::Nginx::SMTP->new(PeerAddr => '127.0.0.1:' . port(8025)); diff -r 9e5d38da7651 -r 3d35b19abfa7 mail_sieve.t --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/mail_sieve.t Wed Apr 08 20:25:10 2020 +0200 @@ -0,0 +1,176 @@ +#!/usr/bin/perl + +# (C) Sander Hoentjen +# (C) Antagonist B.V. + +# Tests for nginx mail sieve module. + +############################################################################### + +use warnings; +use strict; + +useTest::More; + +useMIME::Base64; + +BEGIN { use FindBin; chdir($FindBin::Bin); } + +use lib 'lib'; +useTest::Nginx; +useTest::Nginx::SIEVE; + +############################################################################### + +select STDERR; $| = 1; +select STDOUT; $| = 1; + +local $SIG{PIPE} = 'IGNORE'; + +my $t =Test::Nginx->new()->has(qw/mail sieve http rewrite/) + ->write_file_expand('nginx.conf', <<'EOF'); + +%%TEST_GLOBALS%% + +daemon off; + +events { +} + +mail { + proxy_pass_error_message on; + auth_httphttp://127.0.0.1:8080/mail/auth; + + server { + listen 127.0.0.1:8200; + protocol sieve; + sieve_auth plain cram-md5 external; + } +} + +http { + %%TEST_GLOBALS_HTTP%% + + server { + listen 127.0.0.1:8080; + server_name localhost; + + location = /mail/auth { + set $reply ERROR; + set $passw ""; + + if ($http_auth_smtp_to ~ example.com) { + set $reply OK; + } + + set $userpass "$http_auth_user:$http_auth_pass"; + if ($userpass ~ '^test at example.com:secret$') { + set $reply OK; + } + + set $userpass "$http_auth_user:$http_auth_salt:$http_auth_pass"; + if ($userpass ~ '^test at example.com:<.*@.*>:0{32}$') { + set $reply OK; + set $passw secret; + } + + set $userpass "$http_auth_method:$http_auth_user:$http_auth_pass"; + if ($userpass ~ '^external:test at example.com:$') { + set $reply OK; + set $passw secret; + } + + add_header Auth-Status $reply; + add_header Auth-Server 127.0.0.1; + add_header Auth-Port %%PORT_8201%%; + add_header Auth-Pass $passw; + add_header Auth-Wait 1; + return 204; + } + } +} + +EOF + +$t->run_daemon(\&Test::Nginx::SIEVE::sieve_test_daemon); +$t->run()->plan(15); + +$t->waitforsocket('127.0.0.1:' . port(8201)); + +############################################################################### + +my $s =Test::Nginx::SIEVE->new(); +$s->ok('greeting'); + +# bad auth + +$s->send('AUTHENTICATE'); +$s->check(qr/^NO/, 'auth without arguments'); + +# auth plain + +$s->send('AUTHENTICATE "PLAIN" "' . encode_base64("\0test\@example.com\0bad", '') . '"'); +$s->check(qr/^BYE/, 'auth plain with bad password'); + +$s =Test::Nginx::SIEVE->new(); +$s->ok('greeting'); +$s->send('AUTHENTICATE "PLAIN" "' . encode_base64("\0test\@example.com\0secret", '') . '"'); +#$s->check(qr/auth plain/, 'blaat'); +$s->ok('auth plain'); + +# auth login simple + +$s =Test::Nginx::SIEVE->new(); +$s->read_ok(); + +$s->send('AUTHENTICATE "LOGIN"'); +$s->check(qr/"VXNlcm5hbWU6"/, 'auth login username challenge'); + +$s->send('"' . encode_base64('test at example.com', '') . '"'); +$s->check(qr/"UGFzc3dvcmQ6"/, 'auth login password challenge'); + +$s->send('"' . encode_base64('secret', '') . '"'); +$s->ok('auth login simple'); + +# auth login with username + +$s =Test::Nginx::SIEVE->new(); +$s->read_ok(); + +$s->send('AUTHENTICATE "LOGIN" "' . encode_base64('test at example.com', '') . '"'); +$s->check(qr/"UGFzc3dvcmQ6"/, 'auth login with username password challenge'); + +$s->send('"' . encode_base64('secret', '') . '"'); +$s->ok('auth login with username'); + +# auth cram-md5 + +$s =Test::Nginx::SIEVE->new(); +$s->read_ok(); + +$s->send('AUTHENTICATE "CRAM-MD5"'); +$s->check(qr/"/, 'auth cram-md5 challenge'); + +$s->send('"' . encode_base64('test at example.com ' . ('0' x 32), '') . '"'); +$s->ok('auth cram-md5'); + +# auth external + +$s =Test::Nginx::SIEVE->new(); +$s->read_ok(); + +$s->send('AUTHENTICATE "EXTERNAL"'); +$s->check(qr/"VXNlcm5hbWU6"/, 'auth external challenge'); + +$s->send('"' . encode_base64('test at example.com', '') . '"'); +$s->ok('auth external'); + +# auth external with username + +$s =Test::Nginx::SIEVE->new(); +$s->read_ok(); + +$s->send('AUTHENTICATE "EXTERNAL" "' . encode_base64('test at example.com', '') . '"'); +$s->ok('auth external with username'); + +############################################################################### -------------- next part -------------- An HTML attachment was scrubbed... URL: From maciej.grochowski at pm.me Thu Apr 9 06:07:50 2020 From: maciej.grochowski at pm.me (Maciej) Date: Thu, 09 Apr 2020 06:07:50 +0000 Subject: [PATCH] MSAN use-of-uninitialized-value inside ngx_regex_module_init Message-ID: Hi Nginx devlist! This patch fix use-of-uninitialized-value inside ngx_regex_module_init that come after Nginx is run with Clang MSAN SUMMARY: MemorySanitizer: use-of-uninitialized-value /workspace/nginx/src/core/ngx_regex.c:343:13 in ngx_regex_module_init ==45705==WARNING: MemorySanitizer: use-of-uninitialized-value #0 0x13cc4df in ngx_regex_module_init /workspace/nginx/src/core/ngx_regex.c:343:13 #1 0x126187b in ngx_init_modules /workspace/nginx/src/core/ngx_module.c:72:17 #2 0x122d30e in ngx_init_cycle /workspace/nginx/src/core/ngx_cycle.c:625:9 #3 0x10f2189 in main /workspace/nginx/src/core/nginx.c:291:13 #4 0x109d10a in _start /usr/src/lib/csu/amd64/crt1.c:76:7 Apologizes for lack of HG style patch. Hopefully standard diff will be good enough for such a small change. diff --git a/src/core/ngx_regex.c b/src/core/ngx_regex.c index 52169f65..35cf8fd5 100644 --- a/src/core/ngx_regex.c +++ b/src/core/ngx_regex.c @@ -293,6 +293,7 @@ ngx_regex_module_init(ngx_cycle_t *cycle) ngx_regex_elt_t *elts; opt = 0; + errstr = NULL; #if (NGX_HAVE_PCRE_JIT) { -- 2.24.1 Thanks Maciej -------------- next part -------------- An HTML attachment was scrubbed... URL: From mdounin at mdounin.ru Thu Apr 9 16:36:57 2020 From: mdounin at mdounin.ru (Maxim Dounin) Date: Thu, 9 Apr 2020 19:36:57 +0300 Subject: [PATCH] MSAN use-of-uninitialized-value inside ngx_regex_module_init In-Reply-To: References: Message-ID: <20200409163657.GA20357@mdounin.ru> Hello! On Thu, Apr 09, 2020 at 06:07:50AM +0000, Maciej wrote: > Hi Nginx devlist! > > This patch fix use-of-uninitialized-value inside ngx_regex_module_init that come after Nginx is run with Clang MSAN > > SUMMARY: MemorySanitizer: use-of-uninitialized-value /workspace/nginx/src/core/ngx_regex.c:343:13 in ngx_regex_module_init > ==45705==WARNING: MemorySanitizer: use-of-uninitialized-value > #0 0x13cc4df in ngx_regex_module_init /workspace/nginx/src/core/ngx_regex.c:343:13 > #1 0x126187b in ngx_init_modules /workspace/nginx/src/core/ngx_module.c:72:17 > #2 0x122d30e in ngx_init_cycle /workspace/nginx/src/core/ngx_cycle.c:625:9 > #3 0x10f2189 in main /workspace/nginx/src/core/nginx.c:291:13 > #4 0x109d10a in _start /usr/src/lib/csu/amd64/crt1.c:76:7 Quoting MemorySanitizer docs (https://clang.llvm.org/docs/MemorySanitizer.html#handling-external-code): : MemorySanitizer requires that all program code is instrumented. : This also includes any libraries that the program depends on, even : libc. Failing to achieve this may result in false reports. For the : same reason you may need to replace all inline assembly code that : writes to memory with a pure C/C++ code. Have you tried compiling PCRE with memory sanitizer as well? -- Maxim Dounin http://mdounin.ru/ From xeioex at nginx.com Fri Apr 10 11:16:01 2020 From: xeioex at nginx.com (Dmitry Volyntsev) Date: Fri, 10 Apr 2020 11:16:01 +0000 Subject: [njs] Fixed potential heap-buffer-overflow in njs_vm_value(). Message-ID: details: https://hg.nginx.org/njs/rev/5f4adb155dcf branches: changeset: 1368:5f4adb155dcf user: Dmitry Volyntsev date: Fri Apr 10 11:15:12 2020 +0000 description: Fixed potential heap-buffer-overflow in njs_vm_value(). The issue was introduced in 7ccb8b32cc02. diffstat: src/njs_vm.c | 2 +- src/test/njs_unit_test.c | 14 ++++++++++++-- 2 files changed, 13 insertions(+), 3 deletions(-) diffs (43 lines): diff -r 7ccb8b32cc02 -r 5f4adb155dcf src/njs_vm.c --- a/src/njs_vm.c Wed Apr 08 13:15:02 2020 +0000 +++ b/src/njs_vm.c Fri Apr 10 11:15:12 2020 +0000 @@ -593,7 +593,7 @@ njs_vm_value(njs_vm_t *vm, const njs_str njs_set_object(&value, &vm->global_object); for ( ;; ) { - p = njs_strchr(start, '.'); + p = njs_strlchr(start, end, '.'); size = ((p != NULL) ? p : end) - start; if (njs_slow_path(size == 0)) { diff -r 7ccb8b32cc02 -r 5f4adb155dcf src/test/njs_unit_test.c --- a/src/test/njs_unit_test.c Wed Apr 08 13:15:02 2020 +0000 +++ b/src/test/njs_unit_test.c Fri Apr 10 11:15:12 2020 +0000 @@ -17472,7 +17472,7 @@ njs_vm_value_test(njs_opts_t *opts, njs_ { njs_vm_t *vm; njs_int_t ret; - njs_str_t s, *script; + njs_str_t s, *script, path; njs_uint_t i; njs_bool_t success; njs_stat_t prev; @@ -17564,7 +17564,17 @@ njs_vm_value_test(njs_opts_t *opts, njs_ goto done; } - ret = njs_vm_value(vm, &tests[i].path, &vm->retval); + path = tests[i].path; + + path.start = njs_mp_alloc(vm->mem_pool, path.length); + if (path.start == NULL) { + njs_printf("njs_mp_alloc() failed\n"); + goto done; + } + + memcpy(path.start, tests[i].path.start, path.length); + + ret = njs_vm_value(vm, &path, &vm->retval); if (njs_vm_retval_string(vm, &s) != NJS_OK) { njs_printf("njs_vm_retval_string() failed\n"); From xeioex at nginx.com Sun Apr 12 12:39:39 2020 From: xeioex at nginx.com (Dmitry Volyntsev) Date: Sun, 12 Apr 2020 12:39:39 +0000 Subject: [njs] Introduced njs_vm_opt_init(). Message-ID: details: https://hg.nginx.org/njs/rev/089a13ecbe83 branches: changeset: 1369:089a13ecbe83 user: Dmitry Volyntsev date: Sun Apr 12 10:01:40 2020 +0000 description: Introduced njs_vm_opt_init(). diffstat: nginx/ngx_http_js_module.c | 2 +- nginx/ngx_stream_js_module.c | 2 +- src/njs.h | 1 + src/njs_shell.c | 4 ++-- src/njs_vm.c | 7 +++++++ src/test/njs_benchmark.c | 4 ++-- src/test/njs_interactive_test.c | 2 +- src/test/njs_unit_test.c | 6 +++--- 8 files changed, 18 insertions(+), 10 deletions(-) diffs (136 lines): diff -r 5f4adb155dcf -r 089a13ecbe83 nginx/ngx_http_js_module.c --- a/nginx/ngx_http_js_module.c Fri Apr 10 11:15:12 2020 +0000 +++ b/nginx/ngx_http_js_module.c Sun Apr 12 10:01:40 2020 +0000 @@ -2404,7 +2404,7 @@ ngx_http_js_include(ngx_conf_t *cf, ngx_ end = start + size; - ngx_memzero(&options, sizeof(njs_vm_opt_t)); + njs_vm_opt_init(&options); options.backtrace = 1; options.ops = &ngx_http_js_ops; diff -r 5f4adb155dcf -r 089a13ecbe83 nginx/ngx_stream_js_module.c --- a/nginx/ngx_stream_js_module.c Fri Apr 10 11:15:12 2020 +0000 +++ b/nginx/ngx_stream_js_module.c Sun Apr 12 10:01:40 2020 +0000 @@ -1412,7 +1412,7 @@ ngx_stream_js_include(ngx_conf_t *cf, ng end = start + size; - ngx_memzero(&options, sizeof(njs_vm_opt_t)); + njs_vm_opt_init(&options); options.backtrace = 1; options.ops = &ngx_stream_js_ops; diff -r 5f4adb155dcf -r 089a13ecbe83 src/njs.h --- a/src/njs.h Fri Apr 10 11:15:12 2020 +0000 +++ b/src/njs.h Sun Apr 12 10:01:40 2020 +0000 @@ -220,6 +220,7 @@ typedef struct { } njs_vm_opt_t; +NJS_EXPORT void njs_vm_opt_init(njs_vm_opt_t *options); NJS_EXPORT njs_vm_t *njs_vm_create(njs_vm_opt_t *options); NJS_EXPORT void njs_vm_destroy(njs_vm_t *vm); diff -r 5f4adb155dcf -r 089a13ecbe83 src/njs_shell.c --- a/src/njs_shell.c Fri Apr 10 11:15:12 2020 +0000 +++ b/src/njs_shell.c Sun Apr 12 10:01:40 2020 +0000 @@ -230,7 +230,7 @@ main(int argc, char **argv) njs_mm_denormals(opts.denormals); - njs_memzero(&vm_options, sizeof(njs_vm_opt_t)); + njs_vm_opt_init(&vm_options); if (opts.file == NULL) { p = getcwd(path, sizeof(path)); @@ -573,7 +573,7 @@ LLVMFuzzerTestOneInput(const uint8_t* da opts.silent = 1; - njs_memzero(&vm_options, sizeof(njs_vm_opt_t)); + njs_vm_opt_init(&vm_options); vm_options.init = 1; vm_options.backtrace = 0; diff -r 5f4adb155dcf -r 089a13ecbe83 src/njs_vm.c --- a/src/njs_vm.c Fri Apr 10 11:15:12 2020 +0000 +++ b/src/njs_vm.c Sun Apr 12 10:01:40 2020 +0000 @@ -19,6 +19,13 @@ const njs_str_t njs_entry_unknown = const njs_str_t njs_entry_anonymous = njs_str("anonymous"); +void +njs_vm_opt_init(njs_vm_opt_t *options) +{ + njs_memzero(options, sizeof(njs_vm_opt_t)); +} + + njs_vm_t * njs_vm_create(njs_vm_opt_t *options) { diff -r 5f4adb155dcf -r 089a13ecbe83 src/test/njs_benchmark.c --- a/src/test/njs_benchmark.c Fri Apr 10 11:15:12 2020 +0000 +++ b/src/test/njs_benchmark.c Sun Apr 12 10:01:40 2020 +0000 @@ -48,7 +48,7 @@ njs_benchmark_test(njs_vm_t *parent, njs static const njs_value_t usec_key = njs_string("usec"); static const njs_value_t times_key = njs_string("times"); - njs_memzero(&options, sizeof(njs_vm_opt_t)); + njs_vm_opt_init(&options); vm = NULL; nvm = NULL; @@ -375,7 +375,7 @@ main(int argc, char **argv) } } - njs_memzero(&options, sizeof(njs_vm_opt_t)); + njs_vm_opt_init(&options); options.init = 1; options.argv = argv; options.argc = argc; diff -r 5f4adb155dcf -r 089a13ecbe83 src/test/njs_interactive_test.c --- a/src/test/njs_interactive_test.c Fri Apr 10 11:15:12 2020 +0000 +++ b/src/test/njs_interactive_test.c Sun Apr 12 10:01:40 2020 +0000 @@ -316,7 +316,7 @@ njs_interactive_test(njs_bool_t verbose) njs_printf("\"%V\"\n", &test->script); } - njs_memzero(&options, sizeof(njs_vm_opt_t)); + njs_vm_opt_init(&options); options.init = 1; options.accumulative = 1; diff -r 5f4adb155dcf -r 089a13ecbe83 src/test/njs_unit_test.c --- a/src/test/njs_unit_test.c Fri Apr 10 11:15:12 2020 +0000 +++ b/src/test/njs_unit_test.c Sun Apr 12 10:01:40 2020 +0000 @@ -17146,7 +17146,7 @@ njs_unit_test(njs_unit_test_t tests[], s njs_printf("\"%V\"\n", &tests[i].script); } - njs_memzero(&options, sizeof(njs_vm_opt_t)); + njs_vm_opt_init(&options); options.module = opts->module; options.unsafe = opts->unsafe; @@ -17378,7 +17378,7 @@ njs_vm_json_test(njs_opts_t *opts, njs_s for (i = 0; i < njs_nitems(tests); i++) { - memset(&options, 0, sizeof(njs_vm_opt_t)); + njs_vm_opt_init(&options); options.init = 1; vm = njs_vm_create(&options); @@ -18007,7 +18007,7 @@ njs_api_test(njs_opts_t *opts, njs_stat_ }; vm = NULL; - njs_memzero(&options, sizeof(njs_vm_opt_t)); + njs_vm_opt_init(&options); prev = *stat; From alexander at smirn0v.ru Sun Apr 12 19:12:48 2020 From: alexander at smirn0v.ru (Alexander Smirnov) Date: Sun, 12 Apr 2020 22:12:48 +0300 Subject: 'session_tickets off' option for TLS 1.3 Message-ID: Hello, I have found that in TLS 1.3 mode nginx doesn't fully disable session tickets even with session_tickets off; According to https://www.openssl.org/docs/man1.1.1/man3/SSL_get_options.html SSL_CTX_set_options(conf->ssl.ctx, SSL_OP_NO_TICKET); is not enough to disable session tickets. It only disables stateless tickets but preserves stateful ones. It can be easily verified with openssl s_client -connect localhost:443 Nginx still returns session tickets. To fully disable tickets SSL_CTX_set_num_tickets(conf->ssl.ctx, 0); should also be called. I am not sure on changes. Not sure if I fully understand your intentions on this nginx behaviour. Could you please review the proposed patch ? -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: session_tickets_off_tls_1_3.patch Type: application/octet-stream Size: 2032 bytes Desc: not available URL: From mdounin at mdounin.ru Mon Apr 13 00:39:29 2020 From: mdounin at mdounin.ru (Maxim Dounin) Date: Mon, 13 Apr 2020 03:39:29 +0300 Subject: 'session_tickets off' option for TLS 1.3 In-Reply-To: References: Message-ID: <20200413003929.GE20357@mdounin.ru> Hello! On Sun, Apr 12, 2020 at 10:12:48PM +0300, Alexander Smirnov wrote: > I have found that in TLS 1.3 mode nginx doesn't fully disable session > tickets even with > > session_tickets off; > > According to https://www.openssl.org/docs/man1.1.1/man3/SSL_get_options.html > > > SSL_CTX_set_options(conf->ssl.ctx, SSL_OP_NO_TICKET); > > is not enough to disable session tickets. It only disables stateless > tickets but preserves stateful ones. > > It can be easily verified with > > openssl s_client -connect localhost:443 > > Nginx still returns session tickets. > > To fully disable tickets > > SSL_CTX_set_num_tickets(conf->ssl.ctx, 0); > > should also be called. > > I am not sure on changes. Not sure if I fully understand your intentions on > this nginx behaviour. Could you please review the proposed patch ? In TLS 1.3, a separate field for session identifiers was removed, and TLS session tickets are used instead. "Stateful" tickets are essentially what is called "sessions" in TLS 1.2 and before. Since most configuration with disabled tickets rely on session resumption to work via session identifiers, disabling session tickets completely in TLS 1.3 due to "ssl_session_tickets off;" in the configuration is not a good idea. And this is essentially why SSL_OP_NO_TICKET does not disable tickets completely as well - instead, it only disables stateless tickets, which is what was used to be known as "tickets" in TLS 1.2 and before. If for some reason you really want to disable session resumption in TLS 1.3 completely, you may do so in the same way it can be done for previous protocols: by using "ssl_session_cache off; ssl_session_tickets off;". This results in SSL_CTX_set_session_cache_mode(ctx, SSL_SESS_CACHE_OFF); SSL_CTX_set_options(ctx, SSL_OP_NO_TICKET); on the SSL context, so it should be enough for the OpenSSL library to understand that neither stateful nor stateless tickets should be used. Currently it still sends meaningless stateless tickets, not sure why, but probably this is something to be addressed in OpenSSL, not nginx. -- Maxim Dounin http://mdounin.ru/ From alexander at smirn0v.ru Mon Apr 13 18:29:05 2020 From: alexander at smirn0v.ru (Alexander Smirnov) Date: Mon, 13 Apr 2020 21:29:05 +0300 Subject: 'session_tickets off' option for TLS 1.3 Message-ID: Hello, Thank you for your response. So there are two options: * Fix from the nginx side. But with respect not only to 'session_tickets off' but 'session_cache off'. * Fix from OpenSSL side. Do I understand right that you won't accept a fix from nginx side and I should file an issue to OpenSSL ? -------------- next part -------------- An HTML attachment was scrubbed... URL: From mdounin at mdounin.ru Mon Apr 13 19:35:09 2020 From: mdounin at mdounin.ru (Maxim Dounin) Date: Mon, 13 Apr 2020 22:35:09 +0300 Subject: 'session_tickets off' option for TLS 1.3 In-Reply-To: References: Message-ID: <20200413193509.GG20357@mdounin.ru> Hello! On Mon, Apr 13, 2020 at 09:29:05PM +0300, Alexander Smirnov wrote: > Hello, > > Thank you for your response. > > So there are two options: > > * Fix from the nginx side. But with respect not only to 'session_tickets > off' but 'session_cache off'. > * Fix from OpenSSL side. > > Do I understand right that you won't accept a fix from nginx side and I > should file an issue to OpenSSL ? First of all, you may want to elaborate on what are you trying to fix, and why. Disabling session reuse completely is a strange thing to do in the first place, and trying to make this very uncommon mode slightly more efficient might not worth the effort, regardless of where you are going to do this. -- Maxim Dounin http://mdounin.ru/ From alexander at smirn0v.ru Mon Apr 13 20:40:18 2020 From: alexander at smirn0v.ru (Alexander Smirnov) Date: Mon, 13 Apr 2020 23:40:18 +0300 Subject: 'session_tickets off' option for TLS 1.3 In-Reply-To: <20200413193509.GG20357@mdounin.ru> References: <20200413193509.GG20357@mdounin.ru> Message-ID: I'm working on a project that is completely unrelated to nginx. Just found a bug as I thought. Regardless of how common some configuration mode is ? it should work correctly. I don't think that fix for this will somehow significantly change my experience with nginx, but I have time to fix it and it doesn't look too complex. Would be glad to help on this. On Mon, Apr 13, 2020 at 10:35 PM Maxim Dounin wrote: > Hello! > > On Mon, Apr 13, 2020 at 09:29:05PM +0300, Alexander Smirnov wrote: > > > Hello, > > > > Thank you for your response. > > > > So there are two options: > > > > * Fix from the nginx side. But with respect not only to 'session_tickets > > off' but 'session_cache off'. > > * Fix from OpenSSL side. > > > > Do I understand right that you won't accept a fix from nginx side and I > > should file an issue to OpenSSL ? > > First of all, you may want to elaborate on what are you trying to > fix, and why. Disabling session reuse completely is a strange > thing to do in the first place, and trying to make this very > uncommon mode slightly more efficient might not worth the effort, > regardless of where you are going to do this. > > -- > Maxim Dounin > http://mdounin.ru/ > _______________________________________________ > nginx-devel mailing list > nginx-devel at nginx.org > http://mailman.nginx.org/mailman/listinfo/nginx-devel > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mdounin at mdounin.ru Mon Apr 13 21:52:53 2020 From: mdounin at mdounin.ru (Maxim Dounin) Date: Tue, 14 Apr 2020 00:52:53 +0300 Subject: 'session_tickets off' option for TLS 1.3 In-Reply-To: References: <20200413193509.GG20357@mdounin.ru> Message-ID: <20200413215253.GH20357@mdounin.ru> Hello! On Mon, Apr 13, 2020 at 11:40:18PM +0300, Alexander Smirnov wrote: > I'm working on a project that is completely unrelated to nginx. > Just found a bug as I thought. Regardless of how common some configuration > mode is ? it should work correctly. > I don't think that fix for this will somehow significantly change my > experience with nginx, but I have time to fix it and it doesn't look too > complex. > Would be glad to help on this. As already explained, from nginx point of view there is no bug here: "ssl_session_tickets off;" disables stateless tickets, and "ssl_session_cache off;" disables session resumption. Everything works correctly. The fact that in a very specific configuration some meaningless SSL messages are sent - is hardly a bug unless there are other reasons to think it is, hence I don't think it's a bug from OpenSSL point of view either. -- Maxim Dounin http://mdounin.ru/ From xeioex at nginx.com Tue Apr 14 12:41:32 2020 From: xeioex at nginx.com (Dmitry Volyntsev) Date: Tue, 14 Apr 2020 12:41:32 +0000 Subject: [njs] Added "lineNumber" and "fileName" to SyntaxError. Message-ID: details: https://hg.nginx.org/njs/rev/a46c221089c0 branches: changeset: 1370:a46c221089c0 user: Dmitry Volyntsev date: Sun Apr 12 12:56:25 2020 +0000 description: Added "lineNumber" and "fileName" to SyntaxError. diffstat: src/njs_parser.c | 25 +++++++++++++++++++++---- 1 files changed, 21 insertions(+), 4 deletions(-) diffs (42 lines): diff -r 089a13ecbe83 -r a46c221089c0 src/njs_parser.c --- a/src/njs_parser.c Sun Apr 12 10:01:40 2020 +0000 +++ b/src/njs_parser.c Sun Apr 12 12:56:25 2020 +0000 @@ -2320,10 +2320,15 @@ static void njs_parser_scope_error(njs_vm_t *vm, njs_parser_scope_t *scope, njs_object_type_t type, uint32_t line, const char *fmt, va_list args) { - size_t width; - u_char msg[NJS_MAX_ERROR_STR]; - u_char *p, *end; - njs_str_t *file; + size_t width; + u_char msg[NJS_MAX_ERROR_STR]; + u_char *p, *end; + njs_str_t *file; + njs_int_t ret; + njs_value_t value; + + static const njs_value_t file_name = njs_string("fileName"); + static const njs_value_t line_number = njs_string("lineNumber"); file = &scope->file; @@ -2346,6 +2351,18 @@ njs_parser_scope_error(njs_vm_t *vm, njs } njs_error_new(vm, &vm->retval, type, msg, p - msg); + + njs_set_number(&value, line); + njs_value_property_set(vm, &vm->retval, njs_value_arg(&line_number), + &value); + + if (file->length != 0) { + ret = njs_string_set(vm, &value, file->start, file->length); + if (ret == NJS_OK) { + njs_value_property_set(vm, &vm->retval, njs_value_arg(&file_name), + &value); + } + } } From xeioex at nginx.com Tue Apr 14 12:41:34 2020 From: xeioex at nginx.com (Dmitry Volyntsev) Date: Tue, 14 Apr 2020 12:41:34 +0000 Subject: [njs] Modules: added js_import directive. Message-ID: details: https://hg.nginx.org/njs/rev/777ed1eb1918 branches: changeset: 1371:777ed1eb1918 user: Dmitry Volyntsev date: Tue Apr 14 12:18:25 2020 +0000 description: Modules: added js_import directive. diffstat: nginx/ngx_http_js_module.c | 352 +++++++++++++++++++++++++++++++++------- nginx/ngx_stream_js_module.c | 361 ++++++++++++++++++++++++++++++++++-------- 2 files changed, 573 insertions(+), 140 deletions(-) diffs (truncated from 1010 to 1000 lines): diff -r a46c221089c0 -r 777ed1eb1918 nginx/ngx_http_js_module.c --- a/nginx/ngx_http_js_module.c Sun Apr 12 12:56:25 2020 +0000 +++ b/nginx/ngx_http_js_module.c Tue Apr 14 12:18:25 2020 +0000 @@ -1,6 +1,7 @@ /* * Copyright (C) Roman Arutyunyan + * Copyright (C) Dmitry Volyntsev * Copyright (C) NGINX, Inc. */ @@ -14,6 +15,10 @@ typedef struct { njs_vm_t *vm; + ngx_str_t include; + u_char *file; + ngx_uint_t line; + ngx_array_t *imports; ngx_array_t *paths; njs_external_proto_t req_proto; } ngx_http_js_main_conf_t; @@ -25,6 +30,14 @@ typedef struct { typedef struct { + ngx_str_t name; + ngx_str_t path; + u_char *file; + ngx_uint_t line; +} ngx_http_js_import_t; + + +typedef struct { njs_vm_t *vm; ngx_log_t *log; ngx_uint_t done; @@ -131,10 +144,13 @@ static njs_int_t ngx_http_js_string(njs_ static char *ngx_http_js_include(ngx_conf_t *cf, ngx_command_t *cmd, void *conf); +static char *ngx_http_js_import(ngx_conf_t *cf, ngx_command_t *cmd, + void *conf); static char *ngx_http_js_set(ngx_conf_t *cf, ngx_command_t *cmd, void *conf); static char *ngx_http_js_content(ngx_conf_t *cf, ngx_command_t *cmd, void *conf); static void *ngx_http_js_create_main_conf(ngx_conf_t *cf); +static char *ngx_http_js_init_main_conf(ngx_conf_t *cf, void *conf); static void *ngx_http_js_create_loc_conf(ngx_conf_t *cf); static char *ngx_http_js_merge_loc_conf(ngx_conf_t *cf, void *parent, void *child); @@ -146,6 +162,13 @@ static ngx_command_t ngx_http_js_comman NGX_HTTP_MAIN_CONF|NGX_CONF_TAKE1, ngx_http_js_include, NGX_HTTP_MAIN_CONF_OFFSET, + offsetof(ngx_http_js_main_conf_t, include), + NULL }, + + { ngx_string("js_import"), + NGX_HTTP_MAIN_CONF|NGX_CONF_TAKE13, + ngx_http_js_import, + NGX_HTTP_MAIN_CONF_OFFSET, 0, NULL }, @@ -179,7 +202,7 @@ static ngx_http_module_t ngx_http_js_mo NULL, /* postconfiguration */ ngx_http_js_create_main_conf, /* create main configuration */ - NULL, /* init main configuration */ + ngx_http_js_init_main_conf, /* init main configuration */ NULL, /* create server configuration */ NULL, /* merge server configuration */ @@ -2328,82 +2351,113 @@ ngx_http_js_string(njs_vm_t *vm, njs_val static char * -ngx_http_js_include(ngx_conf_t *cf, ngx_command_t *cmd, void *conf) +ngx_http_js_init_main_conf(ngx_conf_t *cf, void *conf) { ngx_http_js_main_conf_t *jmcf = conf; size_t size; - u_char *start, *end; + u_char *start, *end, *p; ssize_t n; ngx_fd_t fd; - ngx_str_t *m, *value, file; + ngx_str_t *m, file; njs_int_t rc; njs_str_t text, path; ngx_uint_t i; + njs_value_t *value; njs_vm_opt_t options; ngx_file_info_t fi; ngx_pool_cleanup_t *cln; + njs_opaque_value_t lvalue, exception; njs_external_proto_t proto; - - if (jmcf->vm) { - return "is duplicate"; - } - - value = cf->args->elts; - file = value[1]; - - if (ngx_conf_full_name(cf->cycle, &file, 1) != NGX_OK) { - return NGX_CONF_ERROR; + ngx_http_js_import_t *import; + + static const njs_str_t line_number_key = njs_str("lineNumber"); + static const njs_str_t file_name_key = njs_str("fileName"); + + if (jmcf->include.len == 0 && jmcf->imports == NGX_CONF_UNSET_PTR) { + return NGX_CONF_OK; } - fd = ngx_open_file(file.data, NGX_FILE_RDONLY, NGX_FILE_OPEN, 0); - if (fd == NGX_INVALID_FILE) { - ngx_conf_log_error(NGX_LOG_EMERG, cf, ngx_errno, - ngx_open_file_n " \"%s\" failed", file.data); - return NGX_CONF_ERROR; + size = 0; + fd = NGX_INVALID_FILE; + + if (jmcf->include.len != 0) { + file = jmcf->include; + + if (ngx_conf_full_name(cf->cycle, &file, 1) != NGX_OK) { + return NGX_CONF_ERROR; + } + + fd = ngx_open_file(file.data, NGX_FILE_RDONLY, NGX_FILE_OPEN, 0); + if (fd == NGX_INVALID_FILE) { + ngx_log_error(NGX_LOG_EMERG, cf->log, ngx_errno, + ngx_open_file_n " \"%s\" failed", file.data); + return NGX_CONF_ERROR; + } + + if (ngx_fd_info(fd, &fi) == NGX_FILE_ERROR) { + ngx_log_error(NGX_LOG_EMERG, cf->log, ngx_errno, + ngx_fd_info_n " \"%s\" failed", file.data); + (void) ngx_close_file(fd); + return NGX_CONF_ERROR; + } + + size = ngx_file_size(&fi); + + } else { + import = jmcf->imports->elts; + for (i = 0; i < jmcf->imports->nelts; i++) { + size += sizeof("import from '';\n") - 1 + import[i].name.len + + import[i].path.len; + } } - if (ngx_fd_info(fd, &fi) == NGX_FILE_ERROR) { - ngx_conf_log_error(NGX_LOG_EMERG, cf, ngx_errno, - ngx_fd_info_n " \"%s\" failed", file.data); - (void) ngx_close_file(fd); - return NGX_CONF_ERROR; - } - - size = ngx_file_size(&fi); - start = ngx_pnalloc(cf->pool, size); if (start == NULL) { - (void) ngx_close_file(fd); - return NGX_CONF_ERROR; - } - - n = ngx_read_fd(fd, start, size); - - if (n == -1) { - ngx_conf_log_error(NGX_LOG_EMERG, cf, ngx_errno, - ngx_read_fd_n " \"%s\" failed", file.data); - - (void) ngx_close_file(fd); + if (fd != NGX_INVALID_FILE) { + (void) ngx_close_file(fd); + } + return NGX_CONF_ERROR; } - if ((size_t) n != size) { - ngx_conf_log_error(NGX_LOG_EMERG, cf, 0, - ngx_read_fd_n " has read only %z of %O from \"%s\"", - n, size, file.data); - - (void) ngx_close_file(fd); - return NGX_CONF_ERROR; + if (jmcf->include.len != 0) { + n = ngx_read_fd(fd, start, size); + + if (n == -1) { + ngx_log_error(NGX_LOG_EMERG, cf->log, ngx_errno, + ngx_read_fd_n " \"%s\" failed", file.data); + + (void) ngx_close_file(fd); + return NGX_CONF_ERROR; + } + + if ((size_t) n != size) { + ngx_log_error(NGX_LOG_EMERG, cf->log, 0, + ngx_read_fd_n " has read only %z " + "of %O from \"%s\"", n, size, file.data); + + (void) ngx_close_file(fd); + return NGX_CONF_ERROR; + } + + if (ngx_close_file(fd) == NGX_FILE_ERROR) { + ngx_log_error(NGX_LOG_EMERG, cf->log, ngx_errno, + ngx_close_file_n " %s failed", file.data); + } + + } else { + p = start; + import = jmcf->imports->elts; + for (i = 0; i < jmcf->imports->nelts; i++) { + p = ngx_cpymem(p, "import ", sizeof("import ") - 1); + p = ngx_cpymem(p, import[i].name.data, import[i].name.len); + p = ngx_cpymem(p, " from '", sizeof(" from '") - 1); + p = ngx_cpymem(p, import[i].path.data, import[i].path.len); + p = ngx_cpymem(p, "';\n", sizeof("';\n") - 1); + } } - if (ngx_close_file(fd) == NGX_FILE_ERROR) { - ngx_conf_log_error(NGX_LOG_EMERG, cf, ngx_errno, - ngx_close_file_n " %s failed", file.data); - } - - end = start + size; - njs_vm_opt_init(&options); options.backtrace = 1; @@ -2411,13 +2465,19 @@ ngx_http_js_include(ngx_conf_t *cf, ngx_ options.argv = ngx_argv; options.argc = ngx_argc; - file = value[1]; + if (jmcf->include.len != 0) { + file = jmcf->include; + + } else { + file = ngx_cycle->conf_prefix; + } + options.file.start = file.data; options.file.length = file.len; jmcf->vm = njs_vm_create(&options); if (jmcf->vm == NULL) { - ngx_conf_log_error(NGX_LOG_EMERG, cf, 0, "failed to create JS VM"); + ngx_log_error(NGX_LOG_EMERG, cf->log, 0, "failed to create js VM"); return NGX_CONF_ERROR; } @@ -2429,12 +2489,12 @@ ngx_http_js_include(ngx_conf_t *cf, ngx_ cln->handler = ngx_http_js_cleanup_vm; cln->data = jmcf->vm; - path.start = ngx_cycle->prefix.data; - path.length = ngx_cycle->prefix.len; + path.start = ngx_cycle->conf_prefix.data; + path.length = ngx_cycle->conf_prefix.len; rc = njs_vm_add_path(jmcf->vm, &path); if (rc != NJS_OK) { - ngx_conf_log_error(NGX_LOG_EMERG, cf, 0, "failed to add path"); + ngx_log_error(NGX_LOG_EMERG, cf->log, 0, "failed to add \"js_path\""); return NGX_CONF_ERROR; } @@ -2442,7 +2502,7 @@ ngx_http_js_include(ngx_conf_t *cf, ngx_ m = jmcf->paths->elts; for (i = 0; i < jmcf->paths->nelts; i++) { - if (ngx_conf_full_name(cf->cycle, &m[i], 0) != NGX_OK) { + if (ngx_conf_full_name(cf->cycle, &m[i], 1) != NGX_OK) { return NGX_CONF_ERROR; } @@ -2451,7 +2511,8 @@ ngx_http_js_include(ngx_conf_t *cf, ngx_ rc = njs_vm_add_path(jmcf->vm, &path); if (rc != NJS_OK) { - ngx_conf_log_error(NGX_LOG_EMERG, cf, 0, "failed to add path"); + ngx_log_error(NGX_LOG_EMERG, cf->log, 0, + "failed to add \"js_path\""); return NGX_CONF_ERROR; } } @@ -2460,27 +2521,54 @@ ngx_http_js_include(ngx_conf_t *cf, ngx_ proto = njs_vm_external_prototype(jmcf->vm, ngx_http_js_ext_request, njs_nitems(ngx_http_js_ext_request)); if (proto == NULL) { - ngx_conf_log_error(NGX_LOG_EMERG, cf, 0, "failed to add request proto"); + ngx_log_error(NGX_LOG_EMERG, cf->log, 0, + "failed to add js request proto"); return NGX_CONF_ERROR; } jmcf->req_proto = proto; + end = start + size; rc = njs_vm_compile(jmcf->vm, &start, end); if (rc != NJS_OK) { + njs_value_assign(&exception, njs_vm_retval(jmcf->vm)); njs_vm_retval_string(jmcf->vm, &text); - ngx_conf_log_error(NGX_LOG_EMERG, cf, 0, - "%*s, included", - text.length, text.start); + if (jmcf->include.len != 0) { + ngx_log_error(NGX_LOG_EMERG, cf->log, 0, "%*s, included in %s:%ui", + text.length, text.start, jmcf->file, jmcf->line); + return NGX_CONF_ERROR; + } + + value = njs_vm_object_prop(jmcf->vm, njs_value_arg(&exception), + &file_name_key, &lvalue); + if (value == NULL) { + value = njs_vm_object_prop(jmcf->vm, njs_value_arg(&exception), + &line_number_key, &lvalue); + + if (value != NULL) { + i = njs_value_number(value) - 1; + + if (i < jmcf->imports->nelts) { + import = jmcf->imports->elts; + ngx_log_error(NGX_LOG_EMERG, cf->log, 0, + "%*s, included in %s:%ui", text.length, + text.start, import[i].file, import[i].line); + return NGX_CONF_ERROR; + } + } + } + + ngx_log_error(NGX_LOG_EMERG, cf->log, 0, "%*s", text.length, + text.start); return NGX_CONF_ERROR; } if (start != end) { - ngx_conf_log_error(NGX_LOG_EMERG, cf, 0, - "extra characters in js script: \"%*s\", included", - end - start, start); + ngx_log_error(NGX_LOG_EMERG, cf->log, 0, + "extra characters in js script: \"%*s\"", + end - start, start); return NGX_CONF_ERROR; } @@ -2489,6 +2577,132 @@ ngx_http_js_include(ngx_conf_t *cf, ngx_ static char * +ngx_http_js_include(ngx_conf_t *cf, ngx_command_t *cmd, void *conf) +{ + ngx_http_js_main_conf_t *jmcf = conf; + + if (jmcf->imports != NGX_CONF_UNSET_PTR) { + return "is incompatible with \"js_import\""; + } + + jmcf->file = cf->conf_file->file.name.data; + jmcf->line = cf->conf_file->line; + + return ngx_conf_set_str_slot(cf, cmd, conf); +} + + +static char * +ngx_http_js_import(ngx_conf_t *cf, ngx_command_t *cmd, void *conf) +{ + ngx_http_js_main_conf_t *jmcf = conf; + + u_char *p, *end, c; + ngx_int_t from; + ngx_str_t *value, name, path; + ngx_http_js_import_t *import; + + if (jmcf->include.len != 0) { + return "is incompatible with \"js_include\""; + } + + value = cf->args->elts; + from = (cf->args->nelts == 4); + + if (from) { + if (ngx_strcmp(value[2].data, "from") != 0) { + ngx_conf_log_error(NGX_LOG_EMERG, cf, 0, + "invalid parameter \"%V\"", &value[2]); + return NGX_CONF_ERROR; + } + } + + name = value[1]; + path = (from ? value[3] : value[1]); + + if (!from) { + end = name.data + name.len; + + for (p = end - 1; p >= name.data; p--) { + if (*p == '/') { + break; + } + } + + name.data = p + 1; + name.len = end - p - 1; + + if (name.len < 3 + || ngx_memcmp(&name.data[name.len - 3], ".js", 3) != 0) + { + ngx_conf_log_error(NGX_LOG_EMERG, cf, 0, + "cannot extract export name from file path " + "\"%V\", use extended \"from\" syntax", &path); + return NGX_CONF_ERROR; + } + + name.len -= 3; + } + + if (name.len == 0) { + ngx_conf_log_error(NGX_LOG_EMERG, cf, 0, "empty export name"); + return NGX_CONF_ERROR; + } + + p = name.data; + end = name.data + name.len; + + while (p < end) { + c = ngx_tolower(*p); + + if (*p != '_' && (c < 'a' || c > 'z')) { + if (p == name.data) { + ngx_conf_log_error(NGX_LOG_EMERG, cf, 0, "cannot start " + "with \"%c\" in export name \"%V\"", *p, + &name); + return NGX_CONF_ERROR; + } + + if (*p < '0' || *p > '9') { + ngx_conf_log_error(NGX_LOG_EMERG, cf, 0, "invalid character " + "\"%c\" in export name \"%V\"", *p, + &name); + return NGX_CONF_ERROR; + } + } + + p++; + } + + if (ngx_strchr(path.data, '\'') != NULL) { + ngx_conf_log_error(NGX_LOG_EMERG, cf, 0, "invalid character \"'\" " + "in file path \"%V\"", &path); + return NGX_CONF_ERROR; + } + + if (jmcf->imports == NGX_CONF_UNSET_PTR) { + jmcf->imports = ngx_array_create(cf->pool, 4, + sizeof(ngx_http_js_import_t)); + if (jmcf->imports == NULL) { + return NGX_CONF_ERROR; + } + } + + import = ngx_array_push(jmcf->imports); + if (import == NULL) { + return NGX_CONF_ERROR; + } + + import->name = name; + import->path = path; + import->file = cf->conf_file->file.name.data; + import->line = cf->conf_file->line; + + return NGX_CONF_OK; +} + + +static char * ngx_http_js_set(ngx_conf_t *cf, ngx_command_t *cmd, void *conf) { ngx_str_t *value, *fname; @@ -2560,10 +2774,14 @@ ngx_http_js_create_main_conf(ngx_conf_t * set by ngx_pcalloc(): * * conf->vm = NULL; + * conf->include = { 0, NULL }; + * conf->file = NULL; + * conf->line = 0; * conf->req_proto = NULL; */ conf->paths = NGX_CONF_UNSET_PTR; + conf->imports = NGX_CONF_UNSET_PTR; return conf; } diff -r a46c221089c0 -r 777ed1eb1918 nginx/ngx_stream_js_module.c --- a/nginx/ngx_stream_js_module.c Sun Apr 12 12:56:25 2020 +0000 +++ b/nginx/ngx_stream_js_module.c Tue Apr 14 12:18:25 2020 +0000 @@ -15,12 +15,24 @@ typedef struct { njs_vm_t *vm; + ngx_str_t include; + u_char *file; + ngx_uint_t line; + ngx_array_t *imports; ngx_array_t *paths; njs_external_proto_t proto; } ngx_stream_js_main_conf_t; typedef struct { + ngx_str_t name; + ngx_str_t path; + u_char *file; + ngx_uint_t line; +} ngx_stream_js_import_t; + + +typedef struct { ngx_str_t access; ngx_str_t preread; ngx_str_t filter; @@ -103,9 +115,12 @@ static njs_int_t ngx_stream_js_string(nj static char *ngx_stream_js_include(ngx_conf_t *cf, ngx_command_t *cmd, void *conf); +static char *ngx_stream_js_import(ngx_conf_t *cf, ngx_command_t *cmd, + void *conf); static char *ngx_stream_js_set(ngx_conf_t *cf, ngx_command_t *cmd, void *conf); static void *ngx_stream_js_create_main_conf(ngx_conf_t *cf); +static char *ngx_stream_js_init_main_conf(ngx_conf_t *cf, void *conf); static void *ngx_stream_js_create_srv_conf(ngx_conf_t *cf); static char *ngx_stream_js_merge_srv_conf(ngx_conf_t *cf, void *parent, void *child); @@ -118,6 +133,13 @@ static ngx_command_t ngx_stream_js_comm NGX_STREAM_MAIN_CONF|NGX_CONF_TAKE1, ngx_stream_js_include, NGX_STREAM_MAIN_CONF_OFFSET, + offsetof(ngx_stream_js_main_conf_t, include), + NULL }, + + { ngx_string("js_import"), + NGX_STREAM_MAIN_CONF|NGX_CONF_TAKE13, + ngx_stream_js_import, + NGX_STREAM_MAIN_CONF_OFFSET, 0, NULL }, @@ -165,7 +187,7 @@ static ngx_stream_module_t ngx_stream_j ngx_stream_js_init, /* postconfiguration */ ngx_stream_js_create_main_conf, /* create main configuration */ - NULL, /* init main configuration */ + ngx_stream_js_init_main_conf, /* init main configuration */ ngx_stream_js_create_srv_conf, /* create server configuration */ ngx_stream_js_merge_srv_conf, /* merge server configuration */ @@ -1336,82 +1358,113 @@ ngx_stream_js_string(njs_vm_t *vm, njs_v static char * -ngx_stream_js_include(ngx_conf_t *cf, ngx_command_t *cmd, void *conf) +ngx_stream_js_init_main_conf(ngx_conf_t *cf, void *conf) { ngx_stream_js_main_conf_t *jmcf = conf; - size_t size; - u_char *start, *end; - ssize_t n; - ngx_fd_t fd; - ngx_str_t *m, *value, file; - njs_int_t rc; - njs_str_t text, path; - ngx_uint_t i; - njs_vm_opt_t options; - ngx_file_info_t fi; - ngx_pool_cleanup_t *cln; - njs_external_proto_t proto; + size_t size; + u_char *start, *end, *p; + ssize_t n; + ngx_fd_t fd; + ngx_str_t *m, file; + njs_int_t rc; + njs_str_t text, path; + ngx_uint_t i; + njs_value_t *value; + njs_vm_opt_t options; + ngx_file_info_t fi; + ngx_pool_cleanup_t *cln; + njs_opaque_value_t lvalue, exception; + njs_external_proto_t proto; + ngx_stream_js_import_t *import; - if (jmcf->vm) { - return "is duplicate"; + static const njs_str_t line_number_key = njs_str("lineNumber"); + static const njs_str_t file_name_key = njs_str("fileName"); + + if (jmcf->include.len == 0 && jmcf->imports == NGX_CONF_UNSET_PTR) { + return NGX_CONF_OK; } - value = cf->args->elts; - file = value[1]; + size = 0; + fd = NGX_INVALID_FILE; + + if (jmcf->include.len != 0) { + file = jmcf->include; + + if (ngx_conf_full_name(cf->cycle, &file, 1) != NGX_OK) { + return NGX_CONF_ERROR; + } - if (ngx_conf_full_name(cf->cycle, &file, 1) != NGX_OK) { - return NGX_CONF_ERROR; + fd = ngx_open_file(file.data, NGX_FILE_RDONLY, NGX_FILE_OPEN, 0); + if (fd == NGX_INVALID_FILE) { + ngx_log_error(NGX_LOG_EMERG, cf->log, ngx_errno, + ngx_open_file_n " \"%s\" failed", file.data); + return NGX_CONF_ERROR; + } + + if (ngx_fd_info(fd, &fi) == NGX_FILE_ERROR) { + ngx_log_error(NGX_LOG_EMERG, cf->log, ngx_errno, + ngx_fd_info_n " \"%s\" failed", file.data); + (void) ngx_close_file(fd); + return NGX_CONF_ERROR; + } + + size = ngx_file_size(&fi); + + } else { + import = jmcf->imports->elts; + for (i = 0; i < jmcf->imports->nelts; i++) { + size += sizeof("import from '';\n") - 1 + import[i].name.len + + import[i].path.len; + } } - fd = ngx_open_file(file.data, NGX_FILE_RDONLY, NGX_FILE_OPEN, 0); - if (fd == NGX_INVALID_FILE) { - ngx_conf_log_error(NGX_LOG_EMERG, cf, ngx_errno, - ngx_open_file_n " \"%s\" failed", file.data); + start = ngx_pnalloc(cf->pool, size); + if (start == NULL) { + if (fd != NGX_INVALID_FILE) { + (void) ngx_close_file(fd); + } + return NGX_CONF_ERROR; } - if (ngx_fd_info(fd, &fi) == NGX_FILE_ERROR) { - ngx_conf_log_error(NGX_LOG_EMERG, cf, ngx_errno, - ngx_fd_info_n " \"%s\" failed", file.data); - (void) ngx_close_file(fd); - return NGX_CONF_ERROR; - } + if (jmcf->include.len != 0) { + n = ngx_read_fd(fd, start, size); + + if (n == -1) { + ngx_log_error(NGX_LOG_EMERG, cf->log, ngx_errno, + ngx_read_fd_n " \"%s\" failed", file.data); - size = ngx_file_size(&fi); + (void) ngx_close_file(fd); + return NGX_CONF_ERROR; + } - start = ngx_pnalloc(cf->pool, size); - if (start == NULL) { - (void) ngx_close_file(fd); - return NGX_CONF_ERROR; - } - - n = ngx_read_fd(fd, start, size); + if ((size_t) n != size) { + ngx_log_error(NGX_LOG_EMERG, cf->log, 0, + ngx_read_fd_n " has read only %z " + "of %O from \"%s\"", n, size, file.data); - if (n == -1) { - ngx_conf_log_error(NGX_LOG_EMERG, cf, ngx_errno, - ngx_read_fd_n " \"%s\" failed", file.data); + (void) ngx_close_file(fd); + return NGX_CONF_ERROR; + } - (void) ngx_close_file(fd); - return NGX_CONF_ERROR; - } + if (ngx_close_file(fd) == NGX_FILE_ERROR) { + ngx_log_error(NGX_LOG_EMERG, cf->log, ngx_errno, + ngx_close_file_n " %s failed", file.data); + } - if ((size_t) n != size) { - ngx_conf_log_error(NGX_LOG_EMERG, cf, 0, - ngx_read_fd_n " has read only %z of %uz from \"%s\"", - n, size, file.data); - - (void) ngx_close_file(fd); - return NGX_CONF_ERROR; + } else { + p = start; + import = jmcf->imports->elts; + for (i = 0; i < jmcf->imports->nelts; i++) { + p = ngx_cpymem(p, "import ", sizeof("import ") - 1); + p = ngx_cpymem(p, import[i].name.data, import[i].name.len); + p = ngx_cpymem(p, " from '", sizeof(" from '") - 1); + p = ngx_cpymem(p, import[i].path.data, import[i].path.len); + p = ngx_cpymem(p, "';\n", sizeof("';\n") - 1); + } } - if (ngx_close_file(fd) == NGX_FILE_ERROR) { - ngx_conf_log_error(NGX_LOG_EMERG, cf, ngx_errno, - ngx_close_file_n " %s failed", file.data); - } - - end = start + size; - njs_vm_opt_init(&options); options.backtrace = 1; @@ -1419,13 +1472,19 @@ ngx_stream_js_include(ngx_conf_t *cf, ng options.argv = ngx_argv; options.argc = ngx_argc; - file = value[1]; + if (jmcf->include.len != 0) { + file = jmcf->include; + + } else { + file = ngx_cycle->conf_prefix; + } + options.file.start = file.data; options.file.length = file.len; jmcf->vm = njs_vm_create(&options); if (jmcf->vm == NULL) { - ngx_conf_log_error(NGX_LOG_EMERG, cf, 0, "failed to create JS VM"); + ngx_log_error(NGX_LOG_EMERG, cf->log, 0, "failed to create js VM"); return NGX_CONF_ERROR; } @@ -1437,12 +1496,12 @@ ngx_stream_js_include(ngx_conf_t *cf, ng cln->handler = ngx_stream_js_cleanup_vm; cln->data = jmcf->vm; - path.start = ngx_cycle->prefix.data; - path.length = ngx_cycle->prefix.len; + path.start = ngx_cycle->conf_prefix.data; + path.length = ngx_cycle->conf_prefix.len; rc = njs_vm_add_path(jmcf->vm, &path); if (rc != NJS_OK) { - ngx_conf_log_error(NGX_LOG_EMERG, cf, 0, "failed to add path"); + ngx_log_error(NGX_LOG_EMERG, cf->log, 0, "failed to add \"js_path\""); return NGX_CONF_ERROR; } @@ -1450,7 +1509,7 @@ ngx_stream_js_include(ngx_conf_t *cf, ng m = jmcf->paths->elts; for (i = 0; i < jmcf->paths->nelts; i++) { - if (ngx_conf_full_name(cf->cycle, &m[i], 0) != NGX_OK) { + if (ngx_conf_full_name(cf->cycle, &m[i], 1) != NGX_OK) { return NGX_CONF_ERROR; } @@ -1459,7 +1518,8 @@ ngx_stream_js_include(ngx_conf_t *cf, ng rc = njs_vm_add_path(jmcf->vm, &path); if (rc != NJS_OK) { - ngx_conf_log_error(NGX_LOG_EMERG, cf, 0, "failed to add path"); + ngx_log_error(NGX_LOG_EMERG, cf->log, 0, + "failed to add \"js_path\""); return NGX_CONF_ERROR; } } @@ -1467,29 +1527,55 @@ ngx_stream_js_include(ngx_conf_t *cf, ng proto = njs_vm_external_prototype(jmcf->vm, ngx_stream_js_ext_session, njs_nitems(ngx_stream_js_ext_session)); - if (proto == NULL) { - ngx_conf_log_error(NGX_LOG_EMERG, cf, 0, "failed to add stream proto"); + ngx_log_error(NGX_LOG_EMERG, cf->log, 0, + "failed to add js request proto"); return NGX_CONF_ERROR; } jmcf->proto = proto; + end = start + size; rc = njs_vm_compile(jmcf->vm, &start, end); if (rc != NJS_OK) { + njs_value_assign(&exception, njs_vm_retval(jmcf->vm)); njs_vm_retval_string(jmcf->vm, &text); - ngx_conf_log_error(NGX_LOG_EMERG, cf, 0, - "%*s, included", - text.length, text.start); + if (jmcf->include.len != 0) { + ngx_log_error(NGX_LOG_EMERG, cf->log, 0, "%*s, included in %s:%ui", + text.length, text.start, jmcf->file, jmcf->line); + return NGX_CONF_ERROR; + } + + value = njs_vm_object_prop(jmcf->vm, njs_value_arg(&exception), + &file_name_key, &lvalue); + if (value == NULL) { + value = njs_vm_object_prop(jmcf->vm, njs_value_arg(&exception), + &line_number_key, &lvalue); + + if (value != NULL) { + i = njs_value_number(value) - 1; + + if (i < jmcf->imports->nelts) { + import = jmcf->imports->elts; + ngx_log_error(NGX_LOG_EMERG, cf->log, 0, + "%*s, included in %s:%ui", text.length, + text.start, import[i].file, import[i].line); + return NGX_CONF_ERROR; + } + } + } + + ngx_log_error(NGX_LOG_EMERG, cf->log, 0, "%*s", text.length, + text.start); return NGX_CONF_ERROR; } if (start != end) { - ngx_conf_log_error(NGX_LOG_EMERG, cf, 0, - "extra characters in js script: \"%*s\", included", - end - start, start); + ngx_log_error(NGX_LOG_EMERG, cf->log, 0, + "extra characters in js script: \"%*s\"", + end - start, start); return NGX_CONF_ERROR; } @@ -1498,6 +1584,131 @@ ngx_stream_js_include(ngx_conf_t *cf, ng static char * +ngx_stream_js_include(ngx_conf_t *cf, ngx_command_t *cmd, void *conf) +{ + ngx_stream_js_main_conf_t *jmcf = conf; + + if (jmcf->imports != NGX_CONF_UNSET_PTR) { + return "is incompatible with \"js_import\""; + } + + jmcf->file = cf->conf_file->file.name.data; + jmcf->line = cf->conf_file->line; + + return ngx_conf_set_str_slot(cf, cmd, conf); +} + + +static char * +ngx_stream_js_import(ngx_conf_t *cf, ngx_command_t *cmd, void *conf) +{ + ngx_stream_js_main_conf_t *jmcf = conf; + + u_char *p, *end, c; + ngx_int_t from; + ngx_str_t *value, name, path; + ngx_stream_js_import_t *import; + + if (jmcf->include.len != 0) { + return "is incompatible with \"js_include\""; + } + + value = cf->args->elts; + from = (cf->args->nelts == 4); + + if (from) { + if (ngx_strcmp(value[2].data, "from") != 0) { + ngx_conf_log_error(NGX_LOG_EMERG, cf, 0, + "invalid parameter \"%V\"", &value[2]); + return NGX_CONF_ERROR; + } + } + + name = value[1]; + path = (from ? value[3] : value[1]); + + if (!from) { + end = name.data + name.len; + + for (p = end - 1; p >= name.data; p--) { + if (*p == '/') { + break; + } + } + + name.data = p + 1; + name.len = end - p - 1; + + if (name.len < 3 + || ngx_memcmp(&name.data[name.len - 3], ".js", 3) != 0) + { + ngx_conf_log_error(NGX_LOG_EMERG, cf, 0, + "cannot extract export name from file path " + "\"%V\", use extended \"from\" syntax", &path); + return NGX_CONF_ERROR; + } + + name.len -= 3; + } + + if (name.len == 0) { + ngx_conf_log_error(NGX_LOG_EMERG, cf, 0, "empty \"name\" parameter"); + return NGX_CONF_ERROR; + } + + p = name.data; + end = name.data + name.len; + + while (p < end) { + c = ngx_tolower(*p); + + if (*p != '_' && (c < 'a' || c > 'z')) { + if (p == name.data) { + ngx_conf_log_error(NGX_LOG_EMERG, cf, 0, "cannot start " + "with \"%c\" in export name \"%V\"", *p, + &name); + return NGX_CONF_ERROR; + } + + if (*p < '0' || *p > '9') { + ngx_conf_log_error(NGX_LOG_EMERG, cf, 0, "invalid character " + "\"%c\" in export name \"%V\"", *p, &name); + return NGX_CONF_ERROR; + } + } + + p++; + } + + if (ngx_strchr(path.data, '\'') != NULL) { + ngx_conf_log_error(NGX_LOG_EMERG, cf, 0, "invalid character \"'\" " + "in file path \"%V\"", &path); + return NGX_CONF_ERROR; + } + + if (jmcf->imports == NGX_CONF_UNSET_PTR) { + jmcf->imports = ngx_array_create(cf->pool, 4, + sizeof(ngx_stream_js_import_t)); + if (jmcf->imports == NULL) { + return NGX_CONF_ERROR; + } + } + + import = ngx_array_push(jmcf->imports); + if (import == NULL) { + return NGX_CONF_ERROR; + } + + import->name = name; + import->path = path; + import->file = cf->conf_file->file.name.data; + import->line = cf->conf_file->line; + + return NGX_CONF_OK; +} + + +static char * ngx_stream_js_set(ngx_conf_t *cf, ngx_command_t *cmd, void *conf) { ngx_str_t *value, *fname; @@ -1547,10 +1758,14 @@ ngx_stream_js_create_main_conf(ngx_conf_ * set by ngx_pcalloc(): * * conf->vm = NULL; + * conf->include = { 0, NULL }; From xeioex at nginx.com Tue Apr 14 12:48:27 2020 From: xeioex at nginx.com (Dmitry Volyntsev) Date: Tue, 14 Apr 2020 12:48:27 +0000 Subject: [njs] Fixed RegExp() constructor with empty pattern and non-empty flags. Message-ID: details: https://hg.nginx.org/njs/rev/c6f1ae6d8fac branches: changeset: 1372:c6f1ae6d8fac user: Dmitry Volyntsev date: Tue Apr 14 12:42:05 2020 +0000 description: Fixed RegExp() constructor with empty pattern and non-empty flags. diffstat: src/njs_regexp.c | 54 +++++++++++++++++++---------------------------- src/test/njs_unit_test.c | 9 ++++++++ 2 files changed, 31 insertions(+), 32 deletions(-) diffs (108 lines): diff -r 777ed1eb1918 -r c6f1ae6d8fac src/njs_regexp.c --- a/src/njs_regexp.c Tue Apr 14 12:18:25 2020 +0000 +++ b/src/njs_regexp.c Tue Apr 14 12:42:05 2020 +0000 @@ -104,29 +104,11 @@ njs_regexp_constructor(njs_vm_t *vm, njs u_char *start; njs_int_t ret; njs_str_t string; - njs_value_t source, flags_string, *pattern, *flags; + njs_value_t source, *pattern, *flags; njs_regexp_flags_t re_flags; pattern = njs_arg(args, nargs, 1); - if (!njs_is_regexp(pattern) && !njs_is_primitive(pattern)) { - ret = njs_value_to_string(vm, &args[1], &args[1]); - if (ret != NJS_OK) { - return ret; - } - } - - flags = njs_arg(args, nargs, 2); - - if (!njs_is_primitive(flags)) { - ret = njs_value_to_string(vm, &args[2], &args[2]); - if (ret != NJS_OK) { - return ret; - } - } - - re_flags = 0; - if (njs_is_regexp(pattern)) { ret = njs_regexp_prototype_source(vm, NULL, pattern, NULL, &source); if (njs_slow_path(ret != NJS_OK)) { @@ -138,25 +120,28 @@ njs_regexp_constructor(njs_vm_t *vm, njs pattern = &source; } else { - if (njs_is_undefined(pattern)) { + if (njs_is_defined(pattern)) { + ret = njs_value_to_string(vm, pattern, pattern); + if (njs_slow_path(ret != NJS_OK)) { + return ret; + } + + } else { pattern = njs_value_arg(&njs_string_empty); } - ret = njs_primitive_value_to_string(vm, &source, pattern); + re_flags = 0; + } + + flags = njs_arg(args, nargs, 2); + + if (njs_is_defined(flags)) { + ret = njs_value_to_string(vm, flags, flags); if (njs_slow_path(ret != NJS_OK)) { return ret; } - pattern = &source; - } - - if (njs_is_defined(flags)) { - ret = njs_primitive_value_to_string(vm, &flags_string, flags); - if (njs_slow_path(ret != NJS_OK)) { - return ret; - } - - njs_string_get(&flags_string, &string); + njs_string_get(flags, &string); start = string.start; @@ -181,7 +166,12 @@ njs_regexp_create(njs_vm_t *vm, njs_valu njs_regexp_t *regexp; njs_regexp_pattern_t *pattern; - if (length != 0) { + if (length != 0 || flags != 0) { + if (length == 0) { + start = (u_char *) "(?:)"; + length = njs_length("(?:)"); + } + pattern = njs_regexp_pattern_create(vm, start, length, flags); if (njs_slow_path(pattern == NULL)) { return NJS_ERROR; diff -r 777ed1eb1918 -r c6f1ae6d8fac src/test/njs_unit_test.c --- a/src/test/njs_unit_test.c Tue Apr 14 12:18:25 2020 +0000 +++ b/src/test/njs_unit_test.c Tue Apr 14 12:42:05 2020 +0000 @@ -7391,6 +7391,15 @@ static njs_unit_test_t njs_test[] = { njs_str("RegExp('\\\\0').source[1]"), njs_str("0") }, + { njs_str("RegExp(undefined, 'g').global"), + njs_str("true") }, + + { njs_str("RegExp('', 'g').global"), + njs_str("true") }, + + { njs_str("var x; RegExp(x, 'g')"), + njs_str("/(?:)/g") }, + { njs_str("']'.match(/]/)"), njs_str("]") }, From xeioex at nginx.com Tue Apr 14 12:48:29 2020 From: xeioex at nginx.com (Dmitry Volyntsev) Date: Tue, 14 Apr 2020 12:48:29 +0000 Subject: [njs] Fixed String.prototype.replace() when function returns non-string. Message-ID: details: https://hg.nginx.org/njs/rev/3a650363913a branches: changeset: 1373:3a650363913a user: Dmitry Volyntsev date: Tue Apr 14 12:43:09 2020 +0000 description: Fixed String.prototype.replace() when function returns non-string. This closes #303 issue on Github. diffstat: src/njs_string.c | 47 ++++++++++++++++++++++++----------------------- src/test/njs_unit_test.c | 6 ++++++ 2 files changed, 30 insertions(+), 23 deletions(-) diffs (78 lines): diff -r c6f1ae6d8fac -r 3a650363913a src/njs_string.c --- a/src/njs_string.c Tue Apr 14 12:42:05 2020 +0000 +++ b/src/njs_string.c Tue Apr 14 12:43:09 2020 +0000 @@ -3718,34 +3718,35 @@ njs_string_replace_regexp_function(njs_v r->part[0].size = captures[0]; ret = njs_function_apply(vm, r->function, arguments, n + 3, &r->retval); - if (njs_slow_path(ret != NJS_OK)) { - return ret; - } - - (void) njs_string_prop(&string, this); - - if (njs_is_string(&r->retval)) { - njs_string_replacement_copy(&r->part[r->empty ? 0 : 1], &r->retval); - - if (njs_regexp_pattern(regex)->global) { - r->part += 2; - - if (r->part[0].start > (string.start + string.size)) { - return njs_string_replace_regexp_join(vm, r); - } - - return njs_string_replace_regexp(vm, this, regex, r); + goto exception; + } + + if (njs_slow_path(!njs_is_string(&r->retval))) { + ret = njs_value_to_string(vm, &r->retval, &r->retval); + if (njs_slow_path(ret != NJS_OK)) { + goto exception; } - - return njs_string_replace_regexp_join(vm, r); - } + } + + njs_string_replacement_copy(&r->part[r->empty ? 0 : 1], &r->retval); + + if (njs_regexp_pattern(regex)->global) { + r->part += 2; + + if (r->part[0].start > (string.start + string.size)) { + return njs_string_replace_regexp_join(vm, r); + } + + return njs_string_replace_regexp(vm, this, regex, r); + } + + return njs_string_replace_regexp_join(vm, r); + +exception: njs_regex_match_data_free(r->match_data, vm->regex_context); - njs_internal_error(vm, "unexpected retval type:%s", - njs_type_string(r->retval.type)); - return NJS_ERROR; } diff -r c6f1ae6d8fac -r 3a650363913a src/test/njs_unit_test.c --- a/src/test/njs_unit_test.c Tue Apr 14 12:42:05 2020 +0000 +++ b/src/test/njs_unit_test.c Tue Apr 14 12:43:09 2020 +0000 @@ -7231,6 +7231,12 @@ static njs_unit_test_t njs_test[] = "{ return '|'+s+'|'+o+'|'+m+'|'+p+'|' })"), njs_str("abc|abcdefghdijklm|3|d|d|efghdijklm") }, + { njs_str("'abc'.replace(/b/, ()=>1)"), + njs_str("a1c") }, + + { njs_str("var n = 0; 'abbbc'.replace(/b/g, function() {return ++n;})"), + njs_str("a123c") }, + { njs_str("'abcdefghdijklm'.replace(/x/, 'X')"), njs_str("abcdefghdijklm") }, From mdounin at mdounin.ru Tue Apr 14 13:00:53 2020 From: mdounin at mdounin.ru (Maxim Dounin) Date: Tue, 14 Apr 2020 13:00:53 +0000 Subject: [nginx] Updated OpenSSL used for win32 builds. Message-ID: details: https://hg.nginx.org/nginx/rev/3726663b83cc branches: changeset: 7639:3726663b83cc user: Maxim Dounin date: Tue Apr 14 15:15:16 2020 +0300 description: Updated OpenSSL used for win32 builds. diffstat: misc/GNUmakefile | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diffs (12 lines): diff -r 681b78a98a52 -r 3726663b83cc misc/GNUmakefile --- a/misc/GNUmakefile Wed Apr 08 01:02:17 2020 +0300 +++ b/misc/GNUmakefile Tue Apr 14 15:15:16 2020 +0300 @@ -6,7 +6,7 @@ TEMP = tmp CC = cl OBJS = objs.msvc8 -OPENSSL = openssl-1.1.1d +OPENSSL = openssl-1.1.1f ZLIB = zlib-1.2.11 PCRE = pcre-8.44 From mdounin at mdounin.ru Tue Apr 14 14:21:22 2020 From: mdounin at mdounin.ru (Maxim Dounin) Date: Tue, 14 Apr 2020 14:21:22 +0000 Subject: [nginx] nginx-1.17.10-RELEASE Message-ID: details: https://hg.nginx.org/nginx/rev/c44970de0147 branches: changeset: 7640:c44970de0147 user: Maxim Dounin date: Tue Apr 14 17:19:26 2020 +0300 description: nginx-1.17.10-RELEASE diffstat: docs/xml/nginx/changes.xml | 14 ++++++++++++++ 1 files changed, 14 insertions(+), 0 deletions(-) diffs (24 lines): diff -r 3726663b83cc -r c44970de0147 docs/xml/nginx/changes.xml --- a/docs/xml/nginx/changes.xml Tue Apr 14 15:15:16 2020 +0300 +++ b/docs/xml/nginx/changes.xml Tue Apr 14 17:19:26 2020 +0300 @@ -5,6 +5,20 @@ + + + + +????????? auth_delay. + + +the "auth_delay" directive. + + + + + + From mdounin at mdounin.ru Tue Apr 14 14:21:25 2020 From: mdounin at mdounin.ru (Maxim Dounin) Date: Tue, 14 Apr 2020 14:21:25 +0000 Subject: [nginx] release-1.17.10 tag Message-ID: details: https://hg.nginx.org/nginx/rev/3a860f22c879 branches: changeset: 7641:3a860f22c879 user: Maxim Dounin date: Tue Apr 14 17:19:26 2020 +0300 description: release-1.17.10 tag diffstat: .hgtags | 1 + 1 files changed, 1 insertions(+), 0 deletions(-) diffs (8 lines): diff -r c44970de0147 -r 3a860f22c879 .hgtags --- a/.hgtags Tue Apr 14 17:19:26 2020 +0300 +++ b/.hgtags Tue Apr 14 17:19:26 2020 +0300 @@ -448,3 +448,4 @@ de68d0d94320cbf033599c6f3ca37e5335c67fd7 e56295fe0ea76bf53b06bffa77a2d3a9a335cb8c release-1.17.7 fdacd273711ddf20f778c1fb91529ab53979a454 release-1.17.8 5e8d52bca714d4b85284ddb649d1ba4a3ca978a8 release-1.17.9 +c44970de01474f6f3e01b0adea85ec1d03e3a5f2 release-1.17.10 From jose.r.r at metztli.com Tue Apr 14 14:45:50 2020 From: jose.r.r at metztli.com (Jose R R) Date: Tue, 14 Apr 2020 07:45:50 -0700 Subject: [nginx] nginx-1.17.10-RELEASE In-Reply-To: References: Message-ID: ?????, ???????. On Tue, Apr 14, 2020 at 7:21 AM Maxim Dounin wrote: > > details: https://hg.nginx.org/nginx/rev/c44970de0147 > branches: > changeset: 7640:c44970de0147 > user: Maxim Dounin > date: Tue Apr 14 17:19:26 2020 +0300 > description: > nginx-1.17.10-RELEASE > > diffstat: > > docs/xml/nginx/changes.xml | 14 ++++++++++++++ > 1 files changed, 14 insertions(+), 0 deletions(-) > > diffs (24 lines): > > diff -r 3726663b83cc -r c44970de0147 docs/xml/nginx/changes.xml > --- a/docs/xml/nginx/changes.xml Tue Apr 14 15:15:16 2020 +0300 > +++ b/docs/xml/nginx/changes.xml Tue Apr 14 17:19:26 2020 +0300 > @@ -5,6 +5,20 @@ > > > > + > + > + > + > +????????? auth_delay. > + > + > +the "auth_delay" directive. > + > + > + > + > + > + > > > > _______________________________________________ > nginx-devel mailing list > nginx-devel at nginx.org > http://mailman.nginx.org/mailman/listinfo/nginx-devel -- Jose R R http://metztli.it --------------------------------------------------------------------------------------------- Download Metztli Reiser4: Debian Buster w/ Linux 5.5.13 AMD64 --------------------------------------------------------------------------------------------- feats ZSTD compression https://sf.net/projects/metztli-reiser4/ ------------------------------------------------------------------------------------------- Official current Reiser4 resources: https://reiser4.wiki.kernel.org/ From sander at hoentjen.eu Tue Apr 14 18:27:15 2020 From: sander at hoentjen.eu (Sander Hoentjen) Date: Tue, 14 Apr 2020 20:27:15 +0200 Subject: [PATCH] Added support for proxying managesieve protocol In-Reply-To: References: <58016a3a-c11d-bb31-2b05-916cb8124598@hoentjen.eu> Message-ID: <89afc309-4185-e0a1-d6d3-e9f25f65235a@hoentjen.eu> Hello list, Since the Nginx development procedure is unknown to me: Did i do the right things to get my submission to be considered? What are the next steps? Will somebody review this, or reject it? Or is it possible that it just won't get any attention, and that this will mean it will not be considered? I hope I will at least get some feedback, even if it is a rejection :) Kind regards, Sander On 4/8/20 8:33 PM, Sander Hoentjen wrote: > Hello list, > > This is my attempt at adding support for the managesieve protocol. I > hope this is something that you would consider to add. Comments on the > code are very welcome! Also, I hope I submitted this the right way. If > I need to change anything, please let me know. > > Kind regards, > > Sander Hoentjen > > On 4/8/20 8:26 PM, Sander Hoentjen wrote: >> # HG changeset patch >> # User Sander Hoentjen >> # Date 1586369831 -7200 >> # Wed Apr 08 20:17:11 2020 +0200 >> # Node ID f1dffaf619688aaab90caf31781ebe27c3f79598 >> # Parent 0cb942c1c1aa98118076e72e0b89940e85e6291c >> Added support for proxying managesieve protocol >> >> Sieve is a widely used protocol to implement server-side filtering. Some >> servers and clients that support this are listed at >> . This >> commit is tested with Dovecot as the server and Roundcube as the >> client, but I >> don't see a readon why it not would work with others. It is a nice >> addition to >> nginx because it adds the possibility to add server side filtering to >> setups >> that use IMAP behind an nginx proxy. >> >> Fixes https://trac.nginx.org/nginx/ticket/1697 >> >> diff -r 0cb942c1c1aa -r f1dffaf61968 auto/modules >> --- a/auto/modules Fri Mar 13 02:12:10 2020 +0300 >> +++ b/auto/modules Wed Apr 08 20:17:11 2020 +0200 >> @@ -965,6 +965,15 @@ >> . auto/module >> fi >> + if [ $MAIL_SIEVE = YES ]; then >> + ngx_module_name=ngx_mail_sieve_module >> + ngx_module_deps=src/mail/ngx_mail_sieve_module.h >> + ngx_module_srcs="src/mail/ngx_mail_sieve_module.c \ >> + src/mail/ngx_mail_sieve_handler.c" >> + >> + . auto/module >> + fi >> + >> if [ $MAIL_SMTP = YES ]; then >> ngx_module_name=ngx_mail_smtp_module >> ngx_module_deps=src/mail/ngx_mail_smtp_module.h >> diff -r 0cb942c1c1aa -r f1dffaf61968 auto/options >> --- a/auto/options Fri Mar 13 02:12:10 2020 +0300 >> +++ b/auto/options Wed Apr 08 20:17:11 2020 +0200 >> @@ -112,6 +112,7 @@ >> MAIL_SSL=NO >> MAIL_POP3=YES >> MAIL_IMAP=YES >> +MAIL_SIEVE=YES >> MAIL_SMTP=YES >> STREAM=NO >> @@ -305,6 +306,8 @@ >> ;; >> --without-mail_pop3_module) MAIL_POP3=NO ;; >> --without-mail_imap_module) MAIL_IMAP=NO ;; >> + --without-mail_sieve_module) >> + MAIL_SIEVE=NO ;; >> --without-mail_smtp_module) MAIL_SMTP=NO ;; >> --with-stream) STREAM=YES ;; >> @@ -517,11 +520,12 @@ >> --without-http disable HTTP server >> --without-http-cache disable HTTP cache >> - --with-mail enable POP3/IMAP4/SMTP proxy module >> - --with-mail=dynamic enable dynamic POP3/IMAP4/SMTP proxy module >> + --with-mail enable POP3/IMAP4/SIEVE/SMTP proxy module >> + --with-mail=dynamic enable dynamic POP3/IMAP4/SIEVE/SMTP proxy module >> --with-mail_ssl_module enable ngx_mail_ssl_module >> --without-mail_pop3_module disable ngx_mail_pop3_module >> --without-mail_imap_module disable ngx_mail_imap_module >> + --without-mail_sieve_module disable ngx_mail_sieve_module >> --without-mail_smtp_module disable ngx_mail_smtp_module >> --with-stream enable TCP/UDP proxy module >> diff -r 0cb942c1c1aa -r f1dffaf61968 contrib/vim/syntax/nginx.vim >> --- a/contrib/vim/syntax/nginx.vim Fri Mar 13 02:12:10 2020 +0300 >> +++ b/contrib/vim/syntax/nginx.vim Wed Apr 08 20:17:11 2020 +0200 >> @@ -571,6 +571,9 @@ >> syn keyword ngxDirective contained session_log_format >> syn keyword ngxDirective contained session_log_zone >> syn keyword ngxDirective contained set_real_ip_from >> +syn keyword ngxDirective contained sieve_auth >> +syn keyword ngxDirective contained sieve_capabilities >> +syn keyword ngxDirective contained sieve_client_buffer >> syn keyword ngxDirective contained slice >> syn keyword ngxDirective contained smtp_auth >> syn keyword ngxDirective contained smtp_capabilities >> diff -r 0cb942c1c1aa -r f1dffaf61968 src/mail/ngx_mail.h >> --- a/src/mail/ngx_mail.h Fri Mar 13 02:12:10 2020 +0300 >> +++ b/src/mail/ngx_mail.h Wed Apr 08 20:17:11 2020 +0200 >> @@ -102,6 +102,7 @@ >> #define NGX_MAIL_POP3_PROTOCOL 0 >> #define NGX_MAIL_IMAP_PROTOCOL 1 >> #define NGX_MAIL_SMTP_PROTOCOL 2 >> +#define NGX_MAIL_SIEVE_PROTOCOL 3 >> typedef struct ngx_mail_protocol_s ngx_mail_protocol_t; >> @@ -154,6 +155,21 @@ >> typedef enum { >> + ngx_sieve_start = 0, >> + ngx_sieve_starttls, >> + ngx_sieve_auth_login_username, >> + ngx_sieve_auth_login_password, >> + ngx_sieve_auth_plain, >> + ngx_sieve_auth_cram_md5, >> + ngx_sieve_auth_external, >> + ngx_sieve_login, >> + ngx_sieve_login_capabilities, >> + ngx_sieve_user, >> + ngx_sieve_passwd >> +} ngx_sieve_state_e; >> + >> + >> +typedef enum { >> ngx_smtp_start = 0, >> ngx_smtp_auth_login_username, >> ngx_smtp_auth_login_password, >> @@ -227,7 +243,7 @@ >> ngx_uint_t login_attempt; >> - /* used to parse POP3/IMAP/SMTP command */ >> + /* used to parse POP3/IMAP/SIEVE/SMTP command */ >> ngx_uint_t state; >> u_char *cmd_start; >> @@ -271,6 +287,14 @@ >> #define NGX_IMAP_AUTHENTICATE 7 >> +#define NGX_SIEVE_LOGOUT 1 >> +#define NGX_SIEVE_CAPABILITY 2 >> +#define NGX_SIEVE_NOOP 3 >> +#define NGX_SIEVE_STARTTLS 4 >> +#define NGX_SIEVE_NEXT 5 >> +#define NGX_SIEVE_AUTHENTICATE 6 >> + >> + >> #define NGX_SMTP_HELO 1 >> #define NGX_SMTP_EHLO 2 >> #define NGX_SMTP_AUTH 3 >> @@ -383,7 +407,7 @@ >> ngx_int_t ngx_mail_auth_login_password(ngx_mail_session_t *s, >> ngx_connection_t *c); >> ngx_int_t ngx_mail_auth_cram_md5_salt(ngx_mail_session_t *s, >> - ngx_connection_t *c, char *prefix, size_t len); >> + ngx_connection_t *c, char *prefix, size_t len, ngx_uint_t quote); >> ngx_int_t ngx_mail_auth_cram_md5(ngx_mail_session_t *s, >> ngx_connection_t *c); >> ngx_int_t ngx_mail_auth_external(ngx_mail_session_t *s, >> ngx_connection_t *c, >> ngx_uint_t n); >> diff -r 0cb942c1c1aa -r f1dffaf61968 >> src/mail/ngx_mail_auth_http_module.c >> --- a/src/mail/ngx_mail_auth_http_module.c Fri Mar 13 02:12:10 2020 >> +0300 >> +++ b/src/mail/ngx_mail_auth_http_module.c Wed Apr 08 20:17:11 2020 >> +0200 >> @@ -533,6 +533,11 @@ >> + sizeof(CRLF) - 1; >> break; >> + case NGX_MAIL_SIEVE_PROTOCOL: >> + size = sizeof("BYE \"\"") - 1 + len >> + + sizeof(CRLF) - 1; >> + break; >> + >> default: /* NGX_MAIL_SMTP_PROTOCOL */ >> ctx->err = ctx->errmsg; >> continue; >> @@ -559,11 +564,27 @@ >> *p++ = 'N'; *p++ = 'O'; *p++ = ' '; >> break; >> + case NGX_MAIL_SIEVE_PROTOCOL: >> + p = ngx_cpymem(p, s->tag.data, s->tag.len); >> + *p++ = 'B'; *p++ = 'Y'; *p++ = 'E'; *p++ = ' '; *p++ = '"'; >> + break; >> + >> default: /* NGX_MAIL_SMTP_PROTOCOL */ >> break; >> } >> p = ngx_cpymem(p, ctx->header_start, len); >> + >> + switch (s->protocol) { >> + >> + case NGX_MAIL_SIEVE_PROTOCOL: >> + *p++ = '"'; >> + break; >> + >> + default: >> + break; >> + } >> + >> *p++ = CR; *p++ = LF; >> ctx->err.len = p - ctx->err.data; >> diff -r 0cb942c1c1aa -r f1dffaf61968 src/mail/ngx_mail_handler.c >> --- a/src/mail/ngx_mail_handler.c Fri Mar 13 02:12:10 2020 +0300 >> +++ b/src/mail/ngx_mail_handler.c Wed Apr 08 20:17:11 2020 +0200 >> @@ -520,28 +520,39 @@ >> ngx_int_t >> ngx_mail_auth_cram_md5_salt(ngx_mail_session_t *s, ngx_connection_t *c, >> - char *prefix, size_t len) >> + char *prefix, size_t len, ngx_uint_t quote) >> { >> u_char *p; >> ngx_str_t salt; >> - ngx_uint_t n; >> + ngx_uint_t n = 0; >> + if (quote) { >> + len += 2; >> + } >> p = ngx_pnalloc(c->pool, len + ngx_base64_encoded_length(s->salt.len) >> + 2); >> if (p == NULL) { >> return NGX_ERROR; >> } >> - salt.data = ngx_cpymem(p, prefix, len); >> + if (quote) { >> + len -= 2; >> + salt.data = ngx_cpymem(p, "\"", 1); >> + n++; >> + } >> + salt.data = ngx_cpymem(p+ n, prefix, len); >> s->salt.len -= 2; >> ngx_encode_base64(&salt, &s->salt); >> s->salt.len += 2; >> - n = len + salt.len; >> + n += len + salt.len; >> + if (quote) { >> + p[n++] = '"'; >> + } >> p[n++] = CR; p[n++] = LF; >> s->out.len = n; >> - s->out.data = p; >> + s->out.data = p--; >> return NGX_OK; >> } >> diff -r 0cb942c1c1aa -r f1dffaf61968 src/mail/ngx_mail_imap_handler.c >> --- a/src/mail/ngx_mail_imap_handler.c Fri Mar 13 02:12:10 2020 +0300 >> +++ b/src/mail/ngx_mail_imap_handler.c Wed Apr 08 20:17:11 2020 +0200 >> @@ -397,7 +397,7 @@ >> } >> } >> - if (ngx_mail_auth_cram_md5_salt(s, c, "+ ", 2) == NGX_OK) { >> + if (ngx_mail_auth_cram_md5_salt(s, c, "+ ", 2, 0) == NGX_OK) { >> s->mail_state = ngx_imap_auth_cram_md5; >> return NGX_OK; >> } >> diff -r 0cb942c1c1aa -r f1dffaf61968 src/mail/ngx_mail_parse.c >> --- a/src/mail/ngx_mail_parse.c Fri Mar 13 02:12:10 2020 +0300 >> +++ b/src/mail/ngx_mail_parse.c Wed Apr 08 20:17:11 2020 +0200 >> @@ -11,6 +11,7 @@ >> #include >> #include >> #include >> +#include >> #include >> @@ -620,6 +621,364 @@ >> ngx_int_t >> +ngx_mail_sieve_parse_command(ngx_mail_session_t *s) >> +{ >> + u_char ch, *p, *c; >> + ngx_str_t *arg; >> + enum { >> + sw_start = 0, >> + sw_spaces_before_argument, >> + sw_argument, >> + sw_backslash, >> + sw_literal, >> + sw_no_sync_literal_argument, >> + sw_start_literal_argument, >> + sw_literal_argument, >> + sw_end_literal_argument, >> + sw_almost_done >> + } state; >> + >> + state = s->state; >> + >> + for (p = s->buffer->pos; p < s->buffer->last; p++) { >> + ch = *p; >> + >> + switch (state) { >> + >> + /* SIEVE command */ >> + case sw_start: >> + if (ch == ' ' || ch == CR || ch == LF) { >> + >> + c = s->buffer->start; >> + >> + switch (p - c) { >> + >> + case 4: >> + if ((c[0] == 'N' || c[0] == 'n') >> + && (c[1] == 'O'|| c[1] == 'o') >> + && (c[2] == 'O'|| c[2] == 'o') >> + && (c[3] == 'P'|| c[3] == 'p')) >> + { >> + s->command = NGX_SIEVE_NOOP; >> + >> + } else { >> + goto invalid; >> + } >> + break; >> + >> + case 6: >> + if ((c[0] == 'L'|| c[0] == 'l') >> + && (c[1] == 'O'|| c[1] == 'o') >> + && (c[2] == 'G'|| c[2] == 'g') >> + && (c[3] == 'O'|| c[3] == 'o') >> + && (c[4] == 'U'|| c[4] == 'u') >> + && (c[5] == 'T'|| c[5] == 't')) >> + { >> + s->command = NGX_SIEVE_LOGOUT; >> + >> + } else { >> + goto invalid; >> + } >> + break; >> + >> +#if (NGX_MAIL_SSL) >> + case 8: >> + if ((c[0] == 'S'|| c[0] == 's') >> + && (c[1] == 'T'|| c[1] == 't') >> + && (c[2] == 'A'|| c[2] == 'a') >> + && (c[3] == 'R'|| c[3] == 'r') >> + && (c[4] == 'T'|| c[4] == 't') >> + && (c[5] == 'T'|| c[5] == 't') >> + && (c[6] == 'L'|| c[6] == 'l') >> + && (c[7] == 'S'|| c[7] == 's')) >> + { >> + s->command = NGX_SIEVE_STARTTLS; >> + >> + } else { >> + goto invalid; >> + } >> + break; >> +#endif >> + >> + case 10: >> + if ((c[0] == 'C'|| c[0] == 'c') >> + && (c[1] == 'A'|| c[1] == 'a') >> + && (c[2] == 'P'|| c[2] == 'p') >> + && (c[3] == 'A'|| c[3] == 'a') >> + && (c[4] == 'B'|| c[4] == 'b') >> + && (c[5] == 'I'|| c[5] == 'i') >> + && (c[6] == 'L'|| c[6] == 'l') >> + && (c[7] == 'I'|| c[7] == 'i') >> + && (c[8] == 'T'|| c[8] == 't') >> + && (c[9] == 'Y'|| c[9] == 'y')) >> + { >> + s->command = NGX_SIEVE_CAPABILITY; >> + >> + } else { >> + goto invalid; >> + } >> + break; >> + >> + case 12: >> + if ((c[0] == 'A'|| c[0] == 'a') >> + && (c[1] == 'U'|| c[1] == 'u') >> + && (c[2] == 'T'|| c[2] == 't') >> + && (c[3] == 'H'|| c[3] == 'h') >> + && (c[4] == 'E'|| c[4] == 'e') >> + && (c[5] == 'N'|| c[5] == 'n') >> + && (c[6] == 'T'|| c[6] == 't') >> + && (c[7] == 'I'|| c[7] == 'i') >> + && (c[8] == 'C'|| c[8] == 'c') >> + && (c[9] == 'A'|| c[9] == 'a') >> + && (c[10] == 'T'|| c[10] == 't') >> + && (c[11] == 'E'|| c[11] == 'e')) >> + { >> + s->command = NGX_SIEVE_AUTHENTICATE; >> + >> + } else { >> + goto invalid; >> + } >> + break; >> + >> + default: >> + goto invalid; >> + } >> + >> + switch (ch) { >> + case ' ': >> + state = sw_spaces_before_argument; >> + break; >> + case CR: >> + state = sw_almost_done; >> + break; >> + case LF: >> + goto done; >> + } >> + break; >> + } >> + >> + if ((ch < 'A' || ch > 'Z') && (ch < 'a' || ch > 'z')) { >> + goto invalid; >> + } >> + >> + break; >> + >> + case sw_spaces_before_argument: >> + switch (ch) { >> + case ' ': >> + break; >> + case CR: >> + state = sw_almost_done; >> + s->arg_end = p; >> + break; >> + case LF: >> + s->arg_end = p; >> + goto done; >> + case '"': >> + if (s->args.nelts <= 2) { >> + s->quoted = 1; >> + s->arg_start = p + 1; >> + state = sw_argument; >> + break; >> + } >> + goto invalid; >> + case '{': >> + if (s->args.nelts <= 2) { >> + state = sw_literal; >> + break; >> + } >> + goto invalid; >> + default: >> + if (s->args.nelts <= 2) { >> + s->arg_start = p; >> + state = sw_argument; >> + break; >> + } >> + goto invalid; >> + } >> + break; >> + >> + case sw_argument: >> + if (ch == ' ' && s->quoted) { >> + break; >> + } >> + >> + switch (ch) { >> + case '"': >> + if (!s->quoted) { >> + break; >> + } >> + s->quoted = 0; >> + /* fall through */ >> + case ' ': >> + case CR: >> + case LF: >> + arg = ngx_array_push(&s->args); >> + if (arg == NULL) { >> + return NGX_ERROR; >> + } >> + arg->len = p - s->arg_start; >> + arg->data = s->arg_start; >> + s->arg_start = NULL; >> + >> + switch (ch) { >> + case '"': >> + case ' ': >> + state = sw_spaces_before_argument; >> + break; >> + case CR: >> + state = sw_almost_done; >> + break; >> + case LF: >> + goto done; >> + } >> + break; >> + case '\\': >> + if (s->quoted) { >> + s->backslash = 1; >> + state = sw_backslash; >> + } >> + break; >> + } >> + break; >> + >> + case sw_backslash: >> + switch (ch) { >> + case CR: >> + case LF: >> + goto invalid; >> + default: >> + state = sw_argument; >> + } >> + break; >> + >> + case sw_literal: >> + if (ch >= '0' && ch <= '9') { >> + s->literal_len = s->literal_len * 10 + (ch - '0'); >> + break; >> + } >> + if (ch == '}') { >> + state = sw_start_literal_argument; >> + break; >> + } >> + if (ch == '+') { >> + state = sw_no_sync_literal_argument; >> + break; >> + } >> + goto invalid; >> + >> + case sw_no_sync_literal_argument: >> + if (ch == '}') { >> + s->no_sync_literal = 1; >> + state = sw_start_literal_argument; >> + break; >> + } >> + goto invalid; >> + >> + case sw_start_literal_argument: >> + switch (ch) { >> + case CR: >> + break; >> + case LF: >> + s->buffer->pos = p + 1; >> + s->arg_start = p + 1; >> + if (s->no_sync_literal == 0) { >> + s->state = sw_literal_argument; >> + return NGX_SIEVE_NEXT; >> + } >> + state = sw_literal_argument; >> + s->no_sync_literal = 0; >> + break; >> + default: >> + goto invalid; >> + } >> + break; >> + >> + case sw_literal_argument: >> + if (s->literal_len && --s->literal_len) { >> + break; >> + } >> + >> + arg = ngx_array_push(&s->args); >> + if (arg == NULL) { >> + return NGX_ERROR; >> + } >> + arg->len = p + 1 - s->arg_start; >> + arg->data = s->arg_start; >> + s->arg_start = NULL; >> + state = sw_end_literal_argument; >> + >> + break; >> + >> + case sw_end_literal_argument: >> + switch (ch) { >> + case '{': >> + if (s->args.nelts <= 2) { >> + state = sw_literal; >> + break; >> + } >> + goto invalid; >> + case CR: >> + state = sw_almost_done; >> + break; >> + case LF: >> + goto done; >> + default: >> + state = sw_spaces_before_argument; >> + break; >> + } >> + break; >> + >> + case sw_almost_done: >> + switch (ch) { >> + case LF: >> + goto done; >> + default: >> + goto invalid; >> + } >> + } >> + } >> + >> + s->buffer->pos = p; >> + s->state = state; >> + >> + return NGX_AGAIN; >> + >> +done: >> + >> + s->buffer->pos = p + 1; >> + >> + if (s->arg_start) { >> + arg = ngx_array_push(&s->args); >> + if (arg == NULL) { >> + return NGX_ERROR; >> + } >> + arg->len = s->arg_end - s->arg_start; >> + arg->data = s->arg_start; >> + >> + s->arg_start = NULL; >> + s->cmd_start = NULL; >> + s->quoted = 0; >> + s->no_sync_literal = 0; >> + s->literal_len = 0; >> + } >> + >> + s->state = (s->command != NGX_SIEVE_AUTHENTICATE) ? sw_start : >> sw_spaces_before_argument; >> + >> + return NGX_OK; >> + >> +invalid: >> + >> + s->state = sw_start; >> + s->quoted = 0; >> + s->no_sync_literal = 0; >> + s->literal_len = 0; >> + >> + return NGX_MAIL_PARSE_INVALID_COMMAND; >> +} >> + >> + >> +ngx_int_t >> ngx_mail_smtp_parse_command(ngx_mail_session_t *s) >> { >> u_char ch, *p, *c, c0, c1, c2, c3; >> diff -r 0cb942c1c1aa -r f1dffaf61968 src/mail/ngx_mail_pop3_handler.c >> --- a/src/mail/ngx_mail_pop3_handler.c Fri Mar 13 02:12:10 2020 +0300 >> +++ b/src/mail/ngx_mail_pop3_handler.c Wed Apr 08 20:17:11 2020 +0200 >> @@ -492,7 +492,7 @@ >> return NGX_MAIL_PARSE_INVALID_COMMAND; >> } >> - if (ngx_mail_auth_cram_md5_salt(s, c, "+ ", 2) == NGX_OK) { >> + if (ngx_mail_auth_cram_md5_salt(s, c, "+ ", 2, 0) == NGX_OK) { >> s->mail_state = ngx_pop3_auth_cram_md5; >> return NGX_OK; >> } >> diff -r 0cb942c1c1aa -r f1dffaf61968 src/mail/ngx_mail_proxy_module.c >> --- a/src/mail/ngx_mail_proxy_module.c Fri Mar 13 02:12:10 2020 +0300 >> +++ b/src/mail/ngx_mail_proxy_module.c Wed Apr 08 20:17:11 2020 +0200 >> @@ -24,6 +24,7 @@ >> static void ngx_mail_proxy_block_read(ngx_event_t *rev); >> static void ngx_mail_proxy_pop3_handler(ngx_event_t *rev); >> static void ngx_mail_proxy_imap_handler(ngx_event_t *rev); >> +static void ngx_mail_proxy_sieve_handler(ngx_event_t *rev); >> static void ngx_mail_proxy_smtp_handler(ngx_event_t *rev); >> static void ngx_mail_proxy_dummy_handler(ngx_event_t *ev); >> static ngx_int_t ngx_mail_proxy_read_response(ngx_mail_session_t *s, >> @@ -173,6 +174,11 @@ >> s->mail_state = ngx_imap_start; >> break; >> + case NGX_MAIL_SIEVE_PROTOCOL: >> + p->upstream.connection->read->handler = ngx_mail_proxy_sieve_handler; >> + s->mail_state = ngx_sieve_start; >> + break; >> + >> default: /* NGX_MAIL_SMTP_PROTOCOL */ >> p->upstream.connection->read->handler = ngx_mail_proxy_smtp_handler; >> s->mail_state = ngx_smtp_start; >> @@ -446,6 +452,144 @@ >> static void >> +ngx_mail_proxy_sieve_handler(ngx_event_t *rev) >> +{ >> + u_char *p; >> + ngx_int_t rc; >> + ngx_str_t line; >> + ngx_str_t userpass; >> + ngx_str_t b64_userpass; >> + ngx_connection_t *c; >> + ngx_mail_session_t *s; >> + ngx_mail_proxy_conf_t *pcf; >> + >> + ngx_log_debug0(NGX_LOG_DEBUG_MAIL, rev->log, 0, >> + "mail proxy sieve auth handler"); >> + >> + c = rev->data; >> + s = c->data; >> + >> + if (rev->timedout) { >> + ngx_log_error(NGX_LOG_INFO, c->log, NGX_ETIMEDOUT, >> + "upstream timed out"); >> + c->timedout = 1; >> + ngx_mail_proxy_internal_server_error(s); >> + return; >> + } >> + >> + rc = ngx_mail_proxy_read_response(s, s->mail_state); >> + >> + if (rc == NGX_AGAIN) { >> + return; >> + } >> + >> + if (rc == NGX_ERROR) { >> + ngx_mail_proxy_upstream_error(s); >> + return; >> + } >> + >> + switch (s->mail_state) { >> + >> + case ngx_sieve_start: >> + ngx_log_debug0(NGX_LOG_DEBUG_MAIL, rev->log, 0, >> + "mail proxy send login"); >> + >> + s->connection->log->action = "sending LOGIN command to upstream"; >> + >> + line.len = s->tag.len + sizeof("AUTHENTICATE ") - 1 >> + + 1 + NGX_SIZE_T_LEN + 1 + 2; >> + line.data = ngx_pnalloc(c->pool, line.len); >> + if (line.data == NULL) { >> + ngx_mail_proxy_internal_server_error(s); >> + return; >> + } >> + >> + userpass.len = s->login.len + s->passwd.len + 2; >> + userpass.data = ngx_pnalloc(c->pool, userpass.len); >> + if (userpass.data == NULL) { >> + ngx_mail_proxy_internal_server_error(s); >> + return; >> + } >> + b64_userpass.len = ngx_base64_encoded_length(userpass.len) + 2; >> + b64_userpass.data = ngx_pnalloc(c->pool, b64_userpass.len); >> + if (b64_userpass.data == NULL) { >> + ngx_mail_proxy_internal_server_error(s); >> + return; >> + } >> + >> + p = userpass.data; >> + *p++ = '\0'; >> + p = ngx_cpymem(p, s->login.data, s->login.len); >> + *p++ = '\0'; >> + p = ngx_cpymem(p, s->passwd.data, s->passwd.len); >> + >> + ngx_encode_base64(&b64_userpass, &userpass); >> + line.len = ngx_sprintf(line.data, "AUTHENTICATE \"PLAIN\" \"%V\"" >> CRLF, >> + &b64_userpass) >> + - line.data; >> + >> + s->mail_state = ngx_sieve_login; >> + break; >> + >> + case ngx_sieve_login: >> + ngx_log_debug0(NGX_LOG_DEBUG_MAIL, rev->log, 0, >> + "mail proxy send capabilities"); >> + >> + s->connection->log->action = "sending CAPABILITY command to upstream"; >> + >> + line.len = s->tag.len + sizeof("AUTHENTICATE ") - 1 >> + + 1 + NGX_SIZE_T_LEN + 1 + 2; >> + line.data = ngx_pnalloc(c->pool, line.len); >> + if (line.data == NULL) { >> + ngx_mail_proxy_internal_server_error(s); >> + return; >> + } >> + >> + line.len = ngx_sprintf(line.data, "CAPABILITY" CRLF) >> + - line.data; >> + >> + s->mail_state = ngx_sieve_login_capabilities; >> + break; >> + >> + case ngx_sieve_login_capabilities: >> + s->connection->read->handler = ngx_mail_proxy_handler; >> + s->connection->write->handler = ngx_mail_proxy_handler; >> + rev->handler = ngx_mail_proxy_handler; >> + c->write->handler = ngx_mail_proxy_handler; >> + >> + pcf = ngx_mail_get_module_srv_conf(s, ngx_mail_proxy_module); >> + ngx_add_timer(s->connection->read, pcf->timeout); >> + ngx_del_timer(c->read); >> + >> + c->log->action = NULL; >> + ngx_log_error(NGX_LOG_INFO, c->log, 0, "client logged in"); >> + >> + ngx_mail_proxy_handler(s->connection->write); >> + >> + return; >> + >> + default: >> +#if (NGX_SUPPRESS_WARN) >> + ngx_str_null(&line); >> +#endif >> + break; >> + } >> + >> + if (c->send(c, line.data, line.len) < (ssize_t) line.len) { >> + /* >> + * we treat the incomplete sending as NGX_ERROR >> + * because it is very strange here >> + */ >> + ngx_mail_proxy_internal_server_error(s); >> + return; >> + } >> + >> + s->proxy->buffer->pos = s->proxy->buffer->start; >> + s->proxy->buffer->last = s->proxy->buffer->start; >> +} >> + >> + >> +static void >> ngx_mail_proxy_smtp_handler(ngx_event_t *rev) >> { >> u_char *p; >> @@ -793,6 +937,42 @@ >> break; >> + case NGX_MAIL_SIEVE_PROTOCOL: >> + if (p[0] == '"') { >> + m = b->last - (sizeof(CRLF "OK" CRLF) - 1); >> + >> + while (m > p) { >> + if (m[0] == CR && m[1] == LF) { >> + break; >> + } >> + m--; >> + } >> + >> + if (m <= p || m[0] == '"') { >> + return NGX_AGAIN; >> + } >> + p = m + 2; >> + } >> + >> + switch (state) { >> + >> + case ngx_sieve_start: >> + case ngx_sieve_login: >> + ngx_log_error(NGX_LOG_INFO, s->connection->log, 0, "1 upstream >> sent: \"%s\"", p); >> + if (p[0] == 'O' && p[1] == 'K') { >> + return NGX_OK; >> + } >> + break; >> + case ngx_sieve_login_capabilities: >> + ngx_log_error(NGX_LOG_INFO, s->connection->log, 0, "2 upstream >> sent: \"%s\"", p); >> + if (p[0] == 'O' && p[1] == 'K') { >> + return NGX_OK; >> + } >> + break; >> + } >> + >> + break; >> + >> default: /* NGX_MAIL_SMTP_PROTOCOL */ >> if (p[3] == '-') { >> diff -r 0cb942c1c1aa -r f1dffaf61968 src/mail/ngx_mail_smtp_handler.c >> --- a/src/mail/ngx_mail_smtp_handler.c Fri Mar 13 02:12:10 2020 +0300 >> +++ b/src/mail/ngx_mail_smtp_handler.c Wed Apr 08 20:17:11 2020 +0200 >> @@ -693,7 +693,7 @@ >> } >> } >> - if (ngx_mail_auth_cram_md5_salt(s, c, "334 ", 4) == NGX_OK) { >> + if (ngx_mail_auth_cram_md5_salt(s, c, "334 ", 4, 0) == NGX_OK) { >> s->mail_state = ngx_smtp_auth_cram_md5; >> return NGX_OK; >> } >> diff -r 0cb942c1c1aa -r f1dffaf61968 src/misc/ngx_cpp_test_module.cpp >> --- a/src/misc/ngx_cpp_test_module.cpp Fri Mar 13 02:12:10 2020 +0300 >> +++ b/src/misc/ngx_cpp_test_module.cpp Wed Apr 08 20:17:11 2020 +0200 >> @@ -13,6 +13,7 @@ >> #include >> #include >> #include >> + #include >> #include >> } >> >> >> _______________________________________________ >> nginx-devel mailing list >> nginx-devel at nginx.org >> http://mailman.nginx.org/mailman/listinfo/nginx-devel >> > _______________________________________________ > nginx-devel mailing list > nginx-devel at nginx.org > http://mailman.nginx.org/mailman/listinfo/nginx-devel > From maxim at nginx.com Wed Apr 15 18:56:34 2020 From: maxim at nginx.com (Maxim Konovalov) Date: Wed, 15 Apr 2020 21:56:34 +0300 Subject: [PATCH] Added support for proxying managesieve protocol In-Reply-To: <89afc309-4185-e0a1-d6d3-e9f25f65235a@hoentjen.eu> References: <58016a3a-c11d-bb31-2b05-916cb8124598@hoentjen.eu> <89afc309-4185-e0a1-d6d3-e9f25f65235a@hoentjen.eu> Message-ID: <911a2e90-5002-3e1c-eb9a-6be3f4e329fc@nginx.com> Hi Sander, First of all, thanks for your code contribution and your work done on it. Among with other tasks we are now in a preparation for a new development branch 1.19 and don't have much developers resources to make a proper review of the code. I'd suggest to put the module somewhere on external repo for now, e.g. on github. I hope we'll be able to return to this topic later in May. Thanks, Maxim On 14.04.2020 21:27, Sander Hoentjen wrote: > Hello list, > > Since the Nginx development procedure is unknown to me: Did i do the > right things to get my submission to be considered? What are the next > steps? Will somebody review this, or reject it? Or is it possible that > it just won't get any attention, and that this will mean it will not be > considered? I hope I will at least get some feedback, even if it is a > rejection :) > > Kind regards, > > Sander > > On 4/8/20 8:33 PM, Sander Hoentjen wrote: >> Hello list, >> >> This is my attempt at adding support for the managesieve protocol. I >> hope this is something that you would consider to add. Comments on the >> code are very welcome! Also, I hope I submitted this the right way. If >> I need to change anything, please let me know. >> >> Kind regards, >> >> Sander Hoentjen >> >> On 4/8/20 8:26 PM, Sander Hoentjen wrote: >>> # HG changeset patch >>> # User Sander Hoentjen >>> # Date 1586369831 -7200 >>> # Wed Apr 08 20:17:11 2020 +0200 >>> # Node ID f1dffaf619688aaab90caf31781ebe27c3f79598 >>> # Parent 0cb942c1c1aa98118076e72e0b89940e85e6291c >>> Added support for proxying managesieve protocol >>> [...] -- Maxim Konovalov From sander at hoentjen.eu Thu Apr 16 17:49:58 2020 From: sander at hoentjen.eu (Sander Hoentjen) Date: Thu, 16 Apr 2020 19:49:58 +0200 Subject: [PATCH] Added support for proxying managesieve protocol In-Reply-To: <911a2e90-5002-3e1c-eb9a-6be3f4e329fc@nginx.com> References: <58016a3a-c11d-bb31-2b05-916cb8124598@hoentjen.eu> <89afc309-4185-e0a1-d6d3-e9f25f65235a@hoentjen.eu> <911a2e90-5002-3e1c-eb9a-6be3f4e329fc@nginx.com> Message-ID: Hi Maxim, Thanks for your response! I will wait till you have time to properly review my code. That is more important than getting it in fast. The code is available at https://github.com/AntagonistHQ/nginx/tree/sieve_v2, so at least it is out in the public for anyone interested. For now, I'll just wait and see what will happen in May. Thank you, Sander On 4/15/20 8:56 PM, Maxim Konovalov wrote: > Hi Sander, > > First of all, thanks for your code contribution and your work done on it. > > Among with other tasks we are now in a preparation for a new development > branch 1.19 and don't have much developers resources to make a proper > review of the code. > > I'd suggest to put the module somewhere on external repo for now, e.g. > on github. I hope we'll be able to return to this topic later in May. > > Thanks, > > Maxim > > On 14.04.2020 21:27, Sander Hoentjen wrote: >> Hello list, >> >> Since the Nginx development procedure is unknown to me: Did i do the >> right things to get my submission to be considered? What are the next >> steps? Will somebody review this, or reject it? Or is it possible that >> it just won't get any attention, and that this will mean it will not be >> considered? I hope I will at least get some feedback, even if it is a >> rejection :) >> >> Kind regards, >> >> Sander >> >> On 4/8/20 8:33 PM, Sander Hoentjen wrote: >>> Hello list, >>> >>> This is my attempt at adding support for the managesieve protocol. I >>> hope this is something that you would consider to add. Comments on the >>> code are very welcome! Also, I hope I submitted this the right way. If >>> I need to change anything, please let me know. >>> >>> Kind regards, >>> >>> Sander Hoentjen >>> >>> On 4/8/20 8:26 PM, Sander Hoentjen wrote: >>>> # HG changeset patch >>>> # User Sander Hoentjen >>>> # Date 1586369831 -7200 >>>> # Wed Apr 08 20:17:11 2020 +0200 >>>> # Node ID f1dffaf619688aaab90caf31781ebe27c3f79598 >>>> # Parent 0cb942c1c1aa98118076e72e0b89940e85e6291c >>>> Added support for proxying managesieve protocol >>>> > [...] > > From thibaultcha at fastmail.com Thu Apr 16 23:51:04 2020 From: thibaultcha at fastmail.com (Thibault Charbonnier) Date: Thu, 16 Apr 2020 16:51:04 -0700 Subject: [PATCH] Ensured SIGQUIT deletes listening UNIX socket files. In-Reply-To: <20200303142811.GO12894@mdounin.ru> References: <4b6bc48b-6c7a-7c76-51ae-1038421ed36c@fastmail.com> <20200227152429.GI12894@mdounin.ru> <2c06ad76-c309-d387-2c40-f6e60aacf4f8@fastmail.com> <20200303142811.GO12894@mdounin.ru> Message-ID: <22c4b64a-eb4e-a8a8-7428-34ab3225fcf5@fastmail.com> On 3/3/20 6:28 AM, Maxim Dounin wrote: > Checking for the oldpid file does not look like a reliable > approach to me. Hi Maxim, For this patch to cover TEST 3 in my previously attached test suite (cancel binary upgrade via SIGQUIT on the new binary), we need a way for a new binary to know whether an old binary is still running or not. Do you have another suggestion on how to achieve this? Checking for the existence of nginx.oldpid seems acceptable to me in this case. Best, Thibault From xeioex at nginx.com Fri Apr 17 17:04:41 2020 From: xeioex at nginx.com (Dmitry Volyntsev) Date: Fri, 17 Apr 2020 17:04:41 +0000 Subject: [njs] Shell: introduced scripts arguments support. Message-ID: details: https://hg.nginx.org/njs/rev/e077780227e0 branches: changeset: 1374:e077780227e0 user: Dmitry Volyntsev date: Fri Apr 17 17:03:59 2020 +0000 description: Shell: introduced scripts arguments support. diffstat: src/njs_shell.c | 34 +++++++++++++++++++++++++++------- test/njs_expect_test.exp | 6 +++++- test/script_args.js | 2 ++ 3 files changed, 34 insertions(+), 8 deletions(-) diffs (119 lines): diff -r 3a650363913a -r e077780227e0 src/njs_shell.c --- a/src/njs_shell.c Tue Apr 14 12:43:09 2020 +0000 +++ b/src/njs_shell.c Fri Apr 17 17:03:59 2020 +0000 @@ -39,6 +39,8 @@ typedef struct { char *command; size_t n_paths; char **paths; + char **argv; + njs_uint_t argc; } njs_opts_t; @@ -264,8 +266,8 @@ main(int argc, char **argv) vm_options.ops = &njs_console_ops; vm_options.external = &njs_console; - vm_options.argv = argv; - vm_options.argc = argc; + vm_options.argv = opts.argv; + vm_options.argc = opts.argc; if (opts.interactive) { ret = njs_interactive_shell(&opts, &vm_options); @@ -295,12 +297,15 @@ done: static njs_int_t njs_get_options(njs_opts_t *opts, int argc, char **argv) { - char *p, **paths; - njs_int_t i, ret; + char *p, **paths; + njs_int_t i, ret; + njs_uint_t n; static const char help[] = "Interactive njs shell.\n" "\n" + "njs [options] [-c string | script.js | -] [script args]" + "\n" "Options:\n" " -c specify the command to execute.\n" " -d print disassembled code.\n" @@ -311,7 +316,7 @@ njs_get_options(njs_opts_t *opts, int ar " -t script|module source code type (script is default).\n" " -v print njs version and exit.\n" " -u disable \"unsafe\" mode.\n" - " | - run code from a file or stdin.\n"; + " script.js | - run code from a file or stdin.\n"; ret = NJS_DONE; @@ -324,7 +329,7 @@ njs_get_options(njs_opts_t *opts, int ar if (p[0] != '-' || (p[0] == '-' && p[1] == '\0')) { opts->interactive = 0; opts->file = argv[i]; - continue; + goto done; } p++; @@ -340,7 +345,7 @@ njs_get_options(njs_opts_t *opts, int ar if (++i < argc) { opts->command = argv[i]; - break; + goto done; } njs_stderror("option \"-c\" requires argument\n"); @@ -418,6 +423,21 @@ njs_get_options(njs_opts_t *opts, int ar } } +done: + + opts->argc = njs_max(argc - i + 1, 2); + opts->argv = malloc(sizeof(char*) * opts->argc); + if (opts->argv == NULL) { + njs_stderror("failed to alloc argv\n"); + return NJS_ERROR; + } + + opts->argv[0] = argv[0]; + opts->argv[1] = (opts->file != NULL) ? opts->file : (char *) ""; + for (n = 2; n < opts->argc; n++) { + opts->argv[n] = argv[i + n - 1]; + } + return NJS_OK; } diff -r 3a650363913a -r e077780227e0 test/njs_expect_test.exp --- a/test/njs_expect_test.exp Tue Apr 14 12:43:09 2020 +0000 +++ b/test/njs_expect_test.exp Fri Apr 17 17:03:59 2020 +0000 @@ -801,7 +801,7 @@ njs_run {"-c" "console.log("} "SyntaxErr # process njs_run {"-c" "console.log(typeof process.argv)"} "object" -njs_run {"-c" "console.log(process.argv[3])" "AAA"} "AAA" +njs_run {"-c" "console.log(process.argv.slice(2))" "AAA"} "AAA" njs_run {"-c" "console.log(typeof process.env)"} "object" njs_run {"-c" "console.log(process.env.HOME != undefined)"} "true" @@ -812,6 +812,10 @@ njs_run {"-c" "console.log(process.pid)" njs_run {"-c" "console.log(process.ppid)"} "\\d+" +# script args + +njs_run {"test/script_args.js" "A" "B"} "AB" + # disassemble njs_test { diff -r 3a650363913a -r e077780227e0 test/script_args.js --- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/test/script_args.js Fri Apr 17 17:03:59 2020 +0000 @@ -0,0 +1,2 @@ +var argv = process.argv.slice(2); +console.log(argv[0] + argv[1]) From xeioex at nginx.com Fri Apr 17 17:04:43 2020 From: xeioex at nginx.com (Dmitry Volyntsev) Date: Fri, 17 Apr 2020 17:04:43 +0000 Subject: [njs] Improved reading of pseudofiles. Message-ID: details: https://hg.nginx.org/njs/rev/f68270171435 branches: changeset: 1375:f68270171435 user: Dmitry Volyntsev date: Fri Apr 17 17:04:18 2020 +0000 description: Improved reading of pseudofiles. diffstat: src/njs_fs.c | 6 ++++++ 1 files changed, 6 insertions(+), 0 deletions(-) diffs (16 lines): diff -r e077780227e0 -r f68270171435 src/njs_fs.c --- a/src/njs_fs.c Fri Apr 17 17:03:59 2020 +0000 +++ b/src/njs_fs.c Fri Apr 17 17:04:18 2020 +0000 @@ -233,6 +233,12 @@ njs_fs_read_file(njs_vm_t *vm, njs_value goto done; } + if (njs_slow_path(data.length < size)) { + /* Pseudo-files may return less data than declared by st_size. */ + njs_string_truncate(&retval, data.length); + } + + size = data.length; start = data.start; } else { From xeioex at nginx.com Tue Apr 21 12:37:26 2020 From: xeioex at nginx.com (Dmitry Volyntsev) Date: Tue, 21 Apr 2020 12:37:26 +0000 Subject: [njs] Improved njs_vm_value() in case of a value is not found. Message-ID: details: https://hg.nginx.org/njs/rev/ca4ab0a7ec76 branches: changeset: 1376:ca4ab0a7ec76 user: Dmitry Volyntsev date: Tue Apr 21 11:56:37 2020 +0000 description: Improved njs_vm_value() in case of a value is not found. diffstat: src/njs_vm.c | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) diffs (14 lines): diff -r f68270171435 -r ca4ab0a7ec76 src/njs_vm.c --- a/src/njs_vm.c Fri Apr 17 17:04:18 2020 +0000 +++ b/src/njs_vm.c Tue Apr 21 11:56:37 2020 +0000 @@ -614,8 +614,8 @@ njs_vm_value(njs_vm_t *vm, const njs_str } ret = njs_value_property(vm, &value, &key, njs_value_arg(retval)); - if (njs_slow_path(ret == NJS_ERROR)) { - return NJS_ERROR; + if (njs_slow_path(ret != NJS_OK)) { + return ret; } if (p == NULL) { From xeioex at nginx.com Tue Apr 21 12:37:28 2020 From: xeioex at nginx.com (Dmitry Volyntsev) Date: Tue, 21 Apr 2020 12:37:28 +0000 Subject: [njs] HTTP: improved iteration over r.headersOut with special headers. Message-ID: details: https://hg.nginx.org/njs/rev/9c3d692d9f4c branches: changeset: 1377:9c3d692d9f4c user: Dmitry Volyntsev date: Tue Apr 21 11:56:46 2020 +0000 description: HTTP: improved iteration over r.headersOut with special headers. In f6234460852a getting of special headers "Content-Type" and "Content-Length" was introduced. Still these headers were invisible during iteration over r.headersOut object. diffstat: nginx/ngx_http_js_module.c | 76 +++++++++++++++++++++++++++++++++++---------- 1 files changed, 58 insertions(+), 18 deletions(-) diffs (114 lines): diff -r ca4ab0a7ec76 -r 9c3d692d9f4c nginx/ngx_http_js_module.c --- a/nginx/ngx_http_js_module.c Tue Apr 21 11:56:37 2020 +0000 +++ b/nginx/ngx_http_js_module.c Tue Apr 21 11:56:46 2020 +0000 @@ -72,7 +72,7 @@ static njs_int_t ngx_http_js_ext_get_str njs_object_prop_t *prop, njs_value_t *value, njs_value_t *setval, njs_value_t *retval); static njs_int_t ngx_http_js_ext_keys_header(njs_vm_t *vm, njs_value_t *value, - njs_value_t *keys, uintptr_t data); + njs_value_t *keys, ngx_list_t *headers); static ngx_table_elt_t *ngx_http_js_get_header(ngx_list_part_t *part, u_char *data, size_t len); static njs_int_t ngx_http_js_ext_header_out(njs_vm_t *vm, @@ -799,26 +799,13 @@ ngx_http_js_ext_get_string(njs_vm_t *vm, static njs_int_t ngx_http_js_ext_keys_header(njs_vm_t *vm, njs_value_t *value, njs_value_t *keys, - uintptr_t data) + ngx_list_t *headers) { - char *p; njs_int_t rc, cookie, x_for; ngx_uint_t item; - ngx_list_t *headers; ngx_list_part_t *part; ngx_table_elt_t *header, *h; - rc = njs_vm_array_alloc(vm, keys, 8); - if (rc != NJS_OK) { - return NJS_ERROR; - } - - p = njs_vm_external(vm, value); - if (p == NULL) { - return NJS_OK; - } - - headers = (ngx_list_t *) (p + data); part = &headers->part; item = 0; @@ -1081,8 +1068,49 @@ static njs_int_t ngx_http_js_ext_keys_header_out(njs_vm_t *vm, njs_value_t *value, njs_value_t *keys) { + njs_int_t rc; + ngx_http_request_t *r; + + rc = njs_vm_array_alloc(vm, keys, 8); + if (rc != NJS_OK) { + return NJS_ERROR; + } + + r = njs_vm_external(vm, value); + if (r == NULL) { + return NJS_OK; + } + + if (r->headers_out.content_type.len) { + value = njs_vm_array_push(vm, keys); + if (value == NULL) { + return NJS_ERROR; + } + + rc = njs_vm_value_string_set(vm, value, (u_char *) "Content-Type", + njs_length("Content-Type")); + if (rc != NJS_OK) { + return NJS_ERROR; + } + } + + if (r->headers_out.content_length == NULL + && r->headers_out.content_length_n >= 0) + { + value = njs_vm_array_push(vm, keys); + if (value == NULL) { + return NJS_ERROR; + } + + rc = njs_vm_value_string_set(vm, value, (u_char *) "Content-Length", + njs_length("Content-Length")); + if (rc != NJS_OK) { + return NJS_ERROR; + } + } + return ngx_http_js_ext_keys_header(vm, value, keys, - offsetof(ngx_http_request_t, headers_out.headers)); + &r->headers_out.headers); } @@ -1616,8 +1644,20 @@ static njs_int_t ngx_http_js_ext_keys_header_in(njs_vm_t *vm, njs_value_t *value, njs_value_t *keys) { - return ngx_http_js_ext_keys_header(vm, value, keys, - offsetof(ngx_http_request_t, headers_in.headers)); + njs_int_t rc; + ngx_http_request_t *r; + + rc = njs_vm_array_alloc(vm, keys, 8); + if (rc != NJS_OK) { + return NJS_ERROR; + } + + r = njs_vm_external(vm, value); + if (r == NULL) { + return NJS_OK; + } + + return ngx_http_js_ext_keys_header(vm, value, keys, &r->headers_in.headers); } static njs_int_t From xeioex at nginx.com Tue Apr 21 12:37:30 2020 From: xeioex at nginx.com (Dmitry Volyntsev) Date: Tue, 21 Apr 2020 12:37:30 +0000 Subject: [njs] HTTP: improved iteration over header objects with duplicates. Message-ID: details: https://hg.nginx.org/njs/rev/55dd0c0acb66 branches: changeset: 1378:55dd0c0acb66 user: Dmitry Volyntsev date: Tue Apr 21 11:57:29 2020 +0000 description: HTTP: improved iteration over header objects with duplicates. In 9e327cd3a33e duplicates were filtered out only for Cookie and X-Forwarded-For. diffstat: nginx/ngx_http_js_module.c | 54 ++++++++++++++++++++------------------------- src/njs.h | 2 + src/njs_vm.c | 7 +++++ 3 files changed, 33 insertions(+), 30 deletions(-) diffs (113 lines): diff -r 9c3d692d9f4c -r 55dd0c0acb66 nginx/ngx_http_js_module.c --- a/nginx/ngx_http_js_module.c Tue Apr 21 11:56:46 2020 +0000 +++ b/nginx/ngx_http_js_module.c Tue Apr 21 11:57:29 2020 +0000 @@ -801,19 +801,19 @@ static njs_int_t ngx_http_js_ext_keys_header(njs_vm_t *vm, njs_value_t *value, njs_value_t *keys, ngx_list_t *headers) { - njs_int_t rc, cookie, x_for; + int64_t i, length; + njs_int_t rc; + njs_str_t hdr; ngx_uint_t item; + njs_value_t *start; ngx_list_part_t *part; ngx_table_elt_t *header, *h; part = &headers->part; item = 0; - - cookie = 0; - x_for = 0; + length = 0; while (part) { - if (item >= part->nelts) { part = part->next; item = 0; @@ -827,36 +827,30 @@ ngx_http_js_ext_keys_header(njs_vm_t *vm continue; } - if (h->key.len == njs_length("Cookie") - && ngx_strncasecmp(h->key.data, (u_char *) "Cookie", - h->key.len) == 0) - { - if (cookie) { - continue; + start = njs_vm_array_start(vm, keys); + + for (i = 0; i < length; i++) { + njs_value_string_get(njs_argument(start, i), &hdr); + + if (h->key.len == hdr.length + && ngx_strncasecmp(h->key.data, hdr.start, hdr.length) == 0) + { + break; } - - cookie = 1; } - if (h->key.len == njs_length("X-Forwarded-For") - && ngx_strncasecmp(h->key.data, (u_char *) "X-Forwarded-For", - h->key.len) == 0) - { - if (x_for) { - continue; + if (i == length) { + value = njs_vm_array_push(vm, keys); + if (value == NULL) { + return NJS_ERROR; } - x_for = 1; - } - - value = njs_vm_array_push(vm, keys); - if (value == NULL) { - return NJS_ERROR; - } - - rc = njs_vm_value_string_set(vm, value, h->key.data, h->key.len); - if (rc != NJS_OK) { - return NJS_ERROR; + rc = njs_vm_value_string_set(vm, value, h->key.data, h->key.len); + if (rc != NJS_OK) { + return NJS_ERROR; + } + + length++; } } diff -r 9c3d692d9f4c -r 55dd0c0acb66 src/njs.h --- a/src/njs.h Tue Apr 21 11:56:46 2020 +0000 +++ b/src/njs.h Tue Apr 21 11:57:29 2020 +0000 @@ -302,6 +302,8 @@ NJS_EXPORT njs_function_t *njs_vm_functi NJS_EXPORT njs_value_t *njs_vm_retval(njs_vm_t *vm); NJS_EXPORT void njs_vm_retval_set(njs_vm_t *vm, const njs_value_t *value); +/* Gets string value, no copy. */ +NJS_EXPORT void njs_value_string_get(njs_value_t *value, njs_str_t *dst); /* * Sets a byte string value. * start data is not copied and should not be freed. diff -r 9c3d692d9f4c -r 55dd0c0acb66 src/njs_vm.c --- a/src/njs_vm.c Tue Apr 21 11:56:46 2020 +0000 +++ b/src/njs_vm.c Tue Apr 21 11:57:29 2020 +0000 @@ -670,6 +670,13 @@ njs_vm_bind(njs_vm_t *vm, const njs_str_ } +void +njs_value_string_get(njs_value_t *value, njs_str_t *dst) +{ + njs_string_get(value, dst); +} + + njs_int_t njs_vm_value_string_set(njs_vm_t *vm, njs_value_t *value, const u_char *start, uint32_t size) From xeioex at nginx.com Tue Apr 21 12:37:32 2020 From: xeioex at nginx.com (Dmitry Volyntsev) Date: Tue, 21 Apr 2020 12:37:32 +0000 Subject: [njs] HTTP: added support for multi-valued headers in r.headersOut. Message-ID: details: https://hg.nginx.org/njs/rev/3df09cf5345c branches: changeset: 1379:3df09cf5345c user: Dmitry Volyntsev date: Tue Apr 21 12:37:00 2020 +0000 description: HTTP: added support for multi-valued headers in r.headersOut. 1) Added support for an array of values in assignments: r.headersOut['Set-Cookie'] = ['a', '', 'b'] will result in Set-Cookie: a Set-Cookie: b headers in output. All previous Set-Cookie are deleted. Only the last element in the table will take effect for standard headers such as Content-Type that only accept a single value. r.headersOut.foo = [] is the same as delete r.headersOut.foo 2) Improved getting of special arrays: Set-Cookie is always returned as an array. Duplicates of Age, Content-Length, Content-Type, ETag, Expires, Last-Modified, Location, Retry-After are ignored. All other duplicate header values are joined together with ','. This closes #266 issue on Github. diffstat: nginx/ngx_http_js_module.c | 516 ++++++++++++++++++++++++++++++++++++++------ 1 files changed, 443 insertions(+), 73 deletions(-) diffs (595 lines): diff -r 55dd0c0acb66 -r 3df09cf5345c nginx/ngx_http_js_module.c --- a/nginx/ngx_http_js_module.c Tue Apr 21 11:57:29 2020 +0000 +++ b/nginx/ngx_http_js_module.c Tue Apr 21 12:37:00 2020 +0000 @@ -57,6 +57,15 @@ typedef struct { } ngx_http_js_event_t; +typedef struct { + njs_str_t name; + njs_int_t (*handler)(njs_vm_t *vm, ngx_http_request_t *r, + njs_str_t *name, njs_value_t *setval, + njs_value_t *retval); + +} ngx_http_js_header_t; + + static ngx_int_t ngx_http_js_content_handler(ngx_http_request_t *r); static void ngx_http_js_content_event_handler(ngx_http_request_t *r); static void ngx_http_js_content_write_event_handler(ngx_http_request_t *r); @@ -78,6 +87,25 @@ static ngx_table_elt_t *ngx_http_js_get_ static njs_int_t ngx_http_js_ext_header_out(njs_vm_t *vm, njs_object_prop_t *prop, njs_value_t *value, njs_value_t *setval, njs_value_t *retval); +static njs_int_t ngx_http_js_header_out_single(njs_vm_t *vm, + ngx_http_request_t *r, njs_str_t *v, njs_value_t *setval, + njs_value_t *retval); +static njs_int_t ngx_http_js_header_out_special(njs_vm_t *vm, + ngx_http_request_t *r, njs_str_t *v, njs_value_t *setval, + njs_value_t *retval, ngx_table_elt_t **hh); +static njs_int_t ngx_http_js_header_out_array(njs_vm_t *vm, + ngx_http_request_t *r, njs_str_t *v, njs_value_t *setval, + njs_value_t *retval); +static njs_int_t ngx_http_js_header_out_generic(njs_vm_t *vm, + ngx_http_request_t *r, njs_str_t *v, njs_value_t *setval, + njs_value_t *retval); +static njs_int_t ngx_http_js_content_length(njs_vm_t *vm, ngx_http_request_t *r, + njs_str_t *name, njs_value_t *setval, njs_value_t *retval); +static njs_int_t ngx_http_js_content_encoding(njs_vm_t *vm, + ngx_http_request_t *r, njs_str_t *name, njs_value_t *setval, + njs_value_t *retval); +static njs_int_t ngx_http_js_content_type(njs_vm_t *vm, ngx_http_request_t *r, + njs_str_t *name, njs_value_t *setval, njs_value_t *retval); static njs_int_t ngx_http_js_ext_keys_header_out(njs_vm_t *vm, njs_value_t *value, njs_value_t *keys); static njs_int_t ngx_http_js_ext_status(njs_vm_t *vm, njs_object_prop_t *prop, @@ -897,14 +925,24 @@ static njs_int_t ngx_http_js_ext_header_out(njs_vm_t *vm, njs_object_prop_t *prop, njs_value_t *value, njs_value_t *setval, njs_value_t *retval) { - u_char *p, *start; - njs_int_t rc; - ngx_int_t n; - njs_str_t *v, s, name; - ngx_str_t *hdr; - ngx_table_elt_t *h; - ngx_http_request_t *r; - u_char content_len[NGX_OFF_T_LEN]; + njs_int_t rc; + njs_str_t name; + ngx_http_request_t *r; + ngx_http_js_header_t *h; + + static ngx_http_js_header_t headers_out[] = { + { njs_str("Age"), ngx_http_js_header_out_single }, + { njs_str("Content-Type"), ngx_http_js_content_type }, + { njs_str("Content-Length"), ngx_http_js_content_length }, + { njs_str("Content-Encoding"), ngx_http_js_content_encoding }, + { njs_str("Etag"), ngx_http_js_header_out_single }, + { njs_str("Expires"), ngx_http_js_header_out_single }, + { njs_str("Last-Modified"), ngx_http_js_header_out_single }, + { njs_str("Location"), ngx_http_js_header_out_single }, + { njs_str("Set-Cookie"), ngx_http_js_header_out_array }, + { njs_str("Retry-After"), ngx_http_js_header_out_single }, + { njs_str(""), ngx_http_js_header_out_generic }, + }; r = njs_vm_external(vm, value); if (r == NULL) { @@ -924,73 +962,76 @@ ngx_http_js_ext_header_out(njs_vm_t *vm, return NJS_DECLINED; } - v = &name; + for (h = headers_out; h->name.length > 0; h++) { + if (h->name.length == name.length + && ngx_strncasecmp(h->name.start, name.start, name.length) == 0) + { + break; + } + } + + return h->handler(vm, r, &name, setval, retval); +} + + +static njs_int_t +ngx_http_js_header_out_single(njs_vm_t *vm, ngx_http_request_t *r, + njs_str_t *name, njs_value_t *setval, njs_value_t *retval) +{ + if (retval != NULL && setval == NULL) { + return ngx_http_js_header_out_special(vm, r, name, setval, retval, + NULL); + } + + return ngx_http_js_header_out_generic(vm, r, name, setval, retval); +} + + +static njs_int_t +ngx_http_js_header_out_special(njs_vm_t *vm, ngx_http_request_t *r, + njs_str_t *v, njs_value_t *setval, njs_value_t *retval, + ngx_table_elt_t **hh) +{ + u_char *p; + int64_t length; + njs_int_t rc; + njs_str_t s; + ngx_list_t *headers; + ngx_table_elt_t *h; + njs_opaque_value_t lvalue; + + headers = &r->headers_out.headers; if (retval != NULL && setval == NULL) { - if (v->length == njs_length("Content-Type") - && ngx_strncasecmp(v->start, (u_char *) "Content-Type", - v->length) == 0) - { - hdr = &r->headers_out.content_type; - return njs_vm_value_string_set(vm, retval, hdr->data, hdr->len); - } - - if (v->length == njs_length("Content-Length") - && ngx_strncasecmp(v->start, (u_char *) "Content-Length", - v->length) == 0) - { - if (r->headers_out.content_length == NULL - && r->headers_out.content_length_n >= 0) - { - p = ngx_sprintf(content_len, "%O", - r->headers_out.content_length_n); - - start = njs_vm_value_string_alloc(vm, retval, p - content_len); - if (start == NULL) { - return NJS_ERROR; - } - - ngx_memcpy(start, content_len, p - content_len); - - return NJS_OK; - } - } - - h = ngx_http_js_get_header(&r->headers_out.headers.part, v->start, - v->length); + h = ngx_http_js_get_header(&headers->part, v->start, v->length); if (h == NULL) { njs_value_undefined_set(retval); return NJS_DECLINED; } - return njs_vm_value_string_set(vm, retval, h->value.data, h->value.len); - } - - if (setval != NULL) { - rc = ngx_http_js_string(vm, setval, &s); + rc = njs_vm_value_string_set(vm, retval, h->value.data, h->value.len); if (rc != NJS_OK) { return NJS_ERROR; } - } else { - s.length = 0; - s.start = NULL; - } - - if (v->length == njs_length("Content-Type") - && ngx_strncasecmp(v->start, (u_char *) "Content-Type", - v->length) == 0) - { - r->headers_out.content_type.len = s.length; - r->headers_out.content_type_len = r->headers_out.content_type.len; - r->headers_out.content_type.data = s.start; - r->headers_out.content_type_lowcase = NULL; - return NJS_OK; } - h = ngx_http_js_get_header(&r->headers_out.headers.part, v->start, - v->length); + if (setval != NULL && njs_value_is_array(setval)) { + rc = njs_vm_array_length(vm, setval, &length); + if (rc != NJS_OK) { + return NJS_ERROR; + } + + setval = njs_vm_array_prop(vm, setval, length - 1, &lvalue); + } + + rc = ngx_http_js_string(vm, setval, &s); + if (rc != NJS_OK) { + return NJS_ERROR; + } + + h = ngx_http_js_get_header(&headers->part, v->start, v->length); if (h != NULL && s.length == 0) { h->hash = 0; @@ -998,7 +1039,7 @@ ngx_http_js_ext_header_out(njs_vm_t *vm, } if (h == NULL && s.length != 0) { - h = ngx_list_push(&r->headers_out.headers); + h = ngx_list_push(headers); if (h == NULL) { return NJS_ERROR; } @@ -1027,22 +1068,313 @@ ngx_http_js_ext_header_out(njs_vm_t *vm, h->hash = 1; } - if (v->length == njs_length("Content-Encoding") - && ngx_strncasecmp(v->start, (u_char *) "Content-Encoding", - v->length) == 0) - { + if (hh != NULL) { + *hh = h; + } + + return NJS_OK; +} + + +static njs_int_t +ngx_http_js_header_out_array(njs_vm_t *vm, ngx_http_request_t *r, + njs_str_t *name, njs_value_t *setval, njs_value_t *retval) +{ + size_t len; + u_char *data; + njs_int_t rc; + ngx_uint_t i; + ngx_list_t *headers; + njs_value_t *value; + ngx_list_part_t *part; + ngx_table_elt_t *header, *h; + + headers = &r->headers_out.headers; + + if (retval != NULL && setval == NULL) { + rc = njs_vm_array_alloc(vm, retval, 4); + if (rc != NJS_OK) { + return NJS_ERROR; + } + + len = name->length; + data = name->start; + + part = &headers->part; + header = part->elts; + + for (i = 0; /* void */ ; i++) { + + if (i >= part->nelts) { + if (part->next == NULL) { + break; + } + + part = part->next; + header = part->elts; + i = 0; + } + + h = &header[i]; + + if (h->hash == 0 + || h->key.len != len + || ngx_strncasecmp(h->key.data, data, len) != 0) + { + continue; + } + + value = njs_vm_array_push(vm, retval); + if (value == NULL) { + return NJS_ERROR; + } + + rc = njs_vm_value_string_set(vm, value, h->value.data, + h->value.len); + if (rc != NJS_OK) { + return NJS_ERROR; + } + } + + return NJS_OK; + } + + return ngx_http_js_header_out_generic(vm, r, name, setval, retval); +} + + +static njs_int_t +ngx_http_js_header_out_generic(njs_vm_t *vm, ngx_http_request_t *r, + njs_str_t *name, njs_value_t *setval, njs_value_t *retval) +{ + size_t len; + u_char *data, *p, *start, *end; + int64_t length; + njs_value_t *array; + njs_int_t rc; + njs_str_t s; + ngx_list_t *headers; + ngx_uint_t i; + ngx_list_part_t *part; + ngx_table_elt_t *header, *h; + njs_opaque_value_t lvalue; + + headers = &r->headers_out.headers; + part = &headers->part; + + if (retval != NULL && setval == NULL) { + header = part->elts; + + p = NULL; + start = NULL; + end = NULL; + + for (i = 0; /* void */ ; i++) { + + if (i >= part->nelts) { + if (part->next == NULL) { + break; + } + + part = part->next; + header = part->elts; + i = 0; + } + + h = &header[i]; + + if (h->hash == 0 + || h->key.len != name->length + || ngx_strncasecmp(h->key.data, name->start, name->length) != 0) + { + continue; + } + + if (p == NULL) { + start = h->value.data; + end = h->value.data + h->value.len; + p = end; + continue; + } + + if (p + h->value.len + 1 > end) { + len = njs_max(p + h->value.len + 1 - start, 2 * (end - start)); + + data = ngx_pnalloc(r->pool, len); + if (data == NULL) { + return NJS_ERROR; + } + + p = ngx_cpymem(data, start, p - start); + start = data; + end = data + len; + } + + *p++ = ','; + p = ngx_cpymem(p, h->value.data, h->value.len); + } + + if (p == NULL) { + njs_value_undefined_set(retval); + return NJS_DECLINED; + } + + return njs_vm_value_string_set(vm, retval, start, p - start); + } + + header = part->elts; + + for (i = 0; /* void */ ; i++) { + + if (i >= part->nelts) { + if (part->next == NULL) { + break; + } + + part = part->next; + header = part->elts; + i = 0; + } + + h = &header[i]; + + if (h->hash == 0 + || h->key.len != name->length + || ngx_strncasecmp(h->key.data, name->start, name->length) != 0) + { + continue; + } + + h->hash = 0; + } + + if (retval == NULL) { + return NJS_OK; + } + + if (njs_value_is_array(setval)) { + array = setval; + + rc = njs_vm_array_length(vm, array, &length); + if (rc != NJS_OK) { + return NJS_ERROR; + } + + if (length == 0) { + return NJS_OK; + } + + } else { + array = NULL; + length = 1; + } + + i = 0; + + for (i = 0; i < (ngx_uint_t) length; i++) { + if (array != NULL) { + setval = njs_vm_array_prop(vm, array, i, &lvalue); + } + + rc = ngx_http_js_string(vm, setval, &s); + if (rc != NJS_OK) { + return NJS_ERROR; + } + + if (s.length == 0) { + continue; + } + + h = ngx_list_push(headers); + if (h == NULL) { + return NJS_ERROR; + } + + p = ngx_pnalloc(r->pool, name->length); + if (p == NULL) { + return NJS_ERROR; + } + + ngx_memcpy(p, name->start, name->length); + + h->key.data = p; + h->key.len = name->length; + + p = ngx_pnalloc(r->pool, s.length); + if (p == NULL) { + return NJS_ERROR; + } + + ngx_memcpy(p, s.start, s.length); + + h->value.data = p; + h->value.len = s.length; + h->hash = 1; + } + + return NJS_OK; +} + + +static njs_int_t +ngx_http_js_content_encoding(njs_vm_t *vm, ngx_http_request_t *r, njs_str_t *v, + njs_value_t *setval, njs_value_t *retval) +{ + njs_int_t rc; + ngx_table_elt_t *h; + + rc = ngx_http_js_header_out_special(vm, r, v, setval, retval, &h); + if (rc != NJS_OK) { + return NJS_ERROR; + } + + if (setval != NULL || retval == NULL) { r->headers_out.content_encoding = h; } - if (v->length == njs_length("Content-Length") - && ngx_strncasecmp(v->start, (u_char *) "Content-Length", - v->length) == 0) - { + return NJS_OK; +} + + +static njs_int_t +ngx_http_js_content_length(njs_vm_t *vm, ngx_http_request_t *r, njs_str_t *v, + njs_value_t *setval, njs_value_t *retval) +{ + u_char *p, *start; + njs_int_t rc; + ngx_int_t n; + ngx_table_elt_t *h; + u_char content_len[NGX_OFF_T_LEN]; + + if (retval != NULL && setval == NULL) { + if (r->headers_out.content_length == NULL + && r->headers_out.content_length_n >= 0) + { + p = ngx_sprintf(content_len, "%O", r->headers_out.content_length_n); + + start = njs_vm_value_string_alloc(vm, retval, p - content_len); + if (start == NULL) { + return NJS_ERROR; + } + + ngx_memcpy(start, content_len, p - content_len); + + return NJS_OK; + } + } + + rc = ngx_http_js_header_out_special(vm, r, v, setval, retval, &h); + if (rc != NJS_OK) { + return NJS_ERROR; + } + + if (setval != NULL || retval == NULL) { if (h != NULL) { - n = ngx_atoi(s.start, s.length); + n = ngx_atoi(h->value.data, h->value.len); if (n == NGX_ERROR) { h->hash = 0; - njs_vm_error(vm, "failed converting argument to integer"); + njs_vm_error(vm, "failed converting argument " + "to positive integer"); return NJS_ERROR; } @@ -1059,6 +1391,44 @@ ngx_http_js_ext_header_out(njs_vm_t *vm, static njs_int_t +ngx_http_js_content_type(njs_vm_t *vm, ngx_http_request_t *r, njs_str_t *v, + njs_value_t *setval, njs_value_t *retval) +{ + int64_t length; + njs_int_t rc; + njs_str_t s; + ngx_str_t *hdr; + njs_opaque_value_t lvalue; + + if (retval != NULL && setval == NULL) { + hdr = &r->headers_out.content_type; + return njs_vm_value_string_set(vm, retval, hdr->data, hdr->len); + } + + if (setval != NULL && njs_value_is_array(setval)) { + rc = njs_vm_array_length(vm, setval, &length); + if (rc != NJS_OK) { + return NJS_ERROR; + } + + setval = njs_vm_array_prop(vm, setval, length - 1, &lvalue); + } + + rc = ngx_http_js_string(vm, setval, &s); + if (rc != NJS_OK) { + return NJS_ERROR; + } + + r->headers_out.content_type.len = s.length; + r->headers_out.content_type_len = r->headers_out.content_type.len; + r->headers_out.content_type.data = s.start; + r->headers_out.content_type_lowcase = NULL; + + return NJS_OK; +} + + +static njs_int_t ngx_http_js_ext_keys_header_out(njs_vm_t *vm, njs_value_t *value, njs_value_t *keys) { @@ -2370,7 +2740,7 @@ ngx_http_js_handle_event(ngx_http_reques static njs_int_t ngx_http_js_string(njs_vm_t *vm, njs_value_t *value, njs_str_t *str) { - if (!njs_value_is_null_or_undefined(value)) { + if (value != NULL && !njs_value_is_null_or_undefined(value)) { if (njs_vm_value_to_string(vm, str, value) == NJS_ERROR) { return NJS_ERROR; } From mdounin at mdounin.ru Tue Apr 21 14:32:32 2020 From: mdounin at mdounin.ru (Maxim Dounin) Date: Tue, 21 Apr 2020 14:32:32 +0000 Subject: [nginx] Stable branch. Message-ID: details: https://hg.nginx.org/nginx/rev/7927f8fb36dd branches: stable-1.18 changeset: 7642:7927f8fb36dd user: Maxim Dounin date: Tue Apr 21 15:38:06 2020 +0300 description: Stable branch. diffstat: src/core/nginx.h | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) diffs (14 lines): diff -r 3a860f22c879 -r 7927f8fb36dd src/core/nginx.h --- a/src/core/nginx.h Tue Apr 14 17:19:26 2020 +0300 +++ b/src/core/nginx.h Tue Apr 21 15:38:06 2020 +0300 @@ -9,8 +9,8 @@ #define _NGINX_H_INCLUDED_ -#define nginx_version 1017010 -#define NGINX_VERSION "1.17.10" +#define nginx_version 1018000 +#define NGINX_VERSION "1.18.0" #define NGINX_VER "nginx/" NGINX_VERSION #ifdef NGX_BUILD From mdounin at mdounin.ru Tue Apr 21 14:32:35 2020 From: mdounin at mdounin.ru (Maxim Dounin) Date: Tue, 21 Apr 2020 14:32:35 +0000 Subject: [nginx] nginx-1.18.0-RELEASE Message-ID: details: https://hg.nginx.org/nginx/rev/f8052414dbe9 branches: stable-1.18 changeset: 7643:f8052414dbe9 user: Maxim Dounin date: Tue Apr 21 17:09:01 2020 +0300 description: nginx-1.18.0-RELEASE diffstat: docs/xml/nginx/changes.xml | 14 ++++++++++++++ 1 files changed, 14 insertions(+), 0 deletions(-) diffs (24 lines): diff -r 7927f8fb36dd -r f8052414dbe9 docs/xml/nginx/changes.xml --- a/docs/xml/nginx/changes.xml Tue Apr 21 15:38:06 2020 +0300 +++ b/docs/xml/nginx/changes.xml Tue Apr 21 17:09:01 2020 +0300 @@ -5,6 +5,20 @@ + + + + +?????????? ????? 1.18.x. + + +1.18.x stable branch. + + + + + + From mdounin at mdounin.ru Tue Apr 21 14:32:37 2020 From: mdounin at mdounin.ru (Maxim Dounin) Date: Tue, 21 Apr 2020 14:32:37 +0000 Subject: [nginx] release-1.18.0 tag Message-ID: details: https://hg.nginx.org/nginx/rev/2a0a77b5fa3a branches: stable-1.18 changeset: 7644:2a0a77b5fa3a user: Maxim Dounin date: Tue Apr 21 17:09:01 2020 +0300 description: release-1.18.0 tag diffstat: .hgtags | 1 + 1 files changed, 1 insertions(+), 0 deletions(-) diffs (8 lines): diff -r f8052414dbe9 -r 2a0a77b5fa3a .hgtags --- a/.hgtags Tue Apr 21 17:09:01 2020 +0300 +++ b/.hgtags Tue Apr 21 17:09:01 2020 +0300 @@ -449,3 +449,4 @@ e56295fe0ea76bf53b06bffa77a2d3a9a335cb8c fdacd273711ddf20f778c1fb91529ab53979a454 release-1.17.8 5e8d52bca714d4b85284ddb649d1ba4a3ca978a8 release-1.17.9 c44970de01474f6f3e01b0adea85ec1d03e3a5f2 release-1.17.10 +f8052414dbe9453578a9f6e753eb507be709c35a release-1.18.0 From xeioex at nginx.com Thu Apr 23 11:30:56 2020 From: xeioex at nginx.com (Dmitry Volyntsev) Date: Thu, 23 Apr 2020 11:30:56 +0000 Subject: [njs] Version 0.4.0. Message-ID: details: https://hg.nginx.org/njs/rev/6144aafa1472 branches: changeset: 1380:6144aafa1472 user: Dmitry Volyntsev date: Thu Apr 23 10:46:54 2020 +0000 description: Version 0.4.0. diffstat: CHANGES | 34 ++++++++++++++++++++++++++++++++++ 1 files changed, 34 insertions(+), 0 deletions(-) diffs (41 lines): diff -r 3df09cf5345c -r 6144aafa1472 CHANGES --- a/CHANGES Tue Apr 21 12:37:00 2020 +0000 +++ b/CHANGES Thu Apr 23 10:46:54 2020 +0000 @@ -1,3 +1,37 @@ + +Changes with njs 0.4.0 23 Apr 2020 + + nginx modules: + + *) Feature: added js_import directive. + + *) Feature: added support for multi-value headers in r.headersOut. + + *) Improvement: iteration over r.headersOut with special headers. + + *) Improvement: iteration over r.headersOut with duplicates. + + *) Change: r.responseBody property handler now returns "undefined" + instead of throwing an exception if response body is not available. + + Core: + + *) Feature: added script arguments support in CLI. + + *) Feature: converting externals values to native js objects. + + *) Bugfix: fixed NULL-pointer dereference in "__proto__" property + handler. + + *) Bugfix: fixed handling of no-newline at the end of the script. + + *) Bugfix: fixed RegExp() constructor with empty pattern and + non-empty flags. + + *) Bugfix: fixed String.prototype.replace() when function + returns non-string. + + *) Bugfix: fixed reading of pseudofiles in "fs". Changes with njs 0.3.9 03 Mar 2020 From xeioex at nginx.com Thu Apr 23 11:30:58 2020 From: xeioex at nginx.com (Dmitry Volyntsev) Date: Thu, 23 Apr 2020 11:30:58 +0000 Subject: [njs] Added tag 0.4.0 for changeset 6144aafa1472 Message-ID: details: https://hg.nginx.org/njs/rev/ee191da6c973 branches: changeset: 1381:ee191da6c973 user: Dmitry Volyntsev date: Thu Apr 23 11:30:25 2020 +0000 description: Added tag 0.4.0 for changeset 6144aafa1472 diffstat: .hgtags | 1 + 1 files changed, 1 insertions(+), 0 deletions(-) diffs (8 lines): diff -r 6144aafa1472 -r ee191da6c973 .hgtags --- a/.hgtags Thu Apr 23 10:46:54 2020 +0000 +++ b/.hgtags Thu Apr 23 11:30:25 2020 +0000 @@ -33,3 +33,4 @@ 7b302775917b72537e1abdbd5dc9d04e55a7d582 9e5ef927c7eaed003de3e5e4a16fa3eab08de7f7 0.3.7 1abb97e9d9dc07dcff2616d4c75132e6d189a6aa 0.3.8 fc4eeaaf0dfe7dc7d41232f1b643bd364f1efe82 0.3.9 +6144aafa1472fbdf79bc9d1f858555938ee08452 0.4.0 From ru at nginx.com Thu Apr 23 12:10:59 2020 From: ru at nginx.com (Ruslan Ermilov) Date: Thu, 23 Apr 2020 12:10:59 +0000 Subject: [nginx] Version bump. Message-ID: details: https://hg.nginx.org/nginx/rev/ed3a10cf88e8 branches: changeset: 7645:ed3a10cf88e8 user: Ruslan Ermilov date: Thu Apr 23 15:10:21 2020 +0300 description: Version bump. diffstat: src/core/nginx.h | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) diffs (14 lines): diff -r 3a860f22c879 -r ed3a10cf88e8 src/core/nginx.h --- a/src/core/nginx.h Tue Apr 14 17:19:26 2020 +0300 +++ b/src/core/nginx.h Thu Apr 23 15:10:21 2020 +0300 @@ -9,8 +9,8 @@ #define _NGINX_H_INCLUDED_ -#define nginx_version 1017010 -#define NGINX_VERSION "1.17.10" +#define nginx_version 1019000 +#define NGINX_VERSION "1.19.0" #define NGINX_VER "nginx/" NGINX_VERSION #ifdef NGX_BUILD From ru at nginx.com Thu Apr 23 12:11:02 2020 From: ru at nginx.com (Ruslan Ermilov) Date: Thu, 23 Apr 2020 12:11:02 +0000 Subject: [nginx] gRPC: RST_STREAM(NO_ERROR) handling (ticket #1792). Message-ID: details: https://hg.nginx.org/nginx/rev/2096b21fcd10 branches: changeset: 7646:2096b21fcd10 user: Ruslan Ermilov date: Thu Apr 23 15:10:24 2020 +0300 description: gRPC: RST_STREAM(NO_ERROR) handling (ticket #1792). As per https://tools.ietf.org/html/rfc7540#section-8.1, : A server can send a complete response prior to the client : sending an entire request if the response does not depend on : any portion of the request that has not been sent and : received. When this is true, a server MAY request that the : client abort transmission of a request without error by : sending a RST_STREAM with an error code of NO_ERROR after : sending a complete response (i.e., a frame with the : END_STREAM flag). Clients MUST NOT discard responses as a : result of receiving such a RST_STREAM, though clients can : always discard responses at their discretion for other : reasons. Previously, RST_STREAM(NO_ERROR) received from upstream after a frame with the END_STREAM flag was incorrectly treated as an error. Now, a single RST_STREAM(NO_ERROR) is properly handled. This fixes problems observed with modern grpc-c [1], as well as with the Go gRPC module. [1] https://github.com/grpc/grpc/pull/1661 diffstat: src/http/modules/ngx_http_grpc_module.c | 26 ++++++++++++++++++++------ 1 files changed, 20 insertions(+), 6 deletions(-) diffs (57 lines): diff -r ed3a10cf88e8 -r 2096b21fcd10 src/http/modules/ngx_http_grpc_module.c --- a/src/http/modules/ngx_http_grpc_module.c Thu Apr 23 15:10:21 2020 +0300 +++ b/src/http/modules/ngx_http_grpc_module.c Thu Apr 23 15:10:24 2020 +0300 @@ -120,6 +120,7 @@ typedef struct { unsigned end_stream:1; unsigned done:1; unsigned status:1; + unsigned rst:1; ngx_http_request_t *request; @@ -1205,6 +1206,7 @@ ngx_http_grpc_reinit_request(ngx_http_re ctx->end_stream = 0; ctx->done = 0; ctx->status = 0; + ctx->rst = 0; ctx->connection = NULL; return NGX_OK; @@ -2088,7 +2090,9 @@ ngx_http_grpc_filter(void *data, ssize_t return NGX_ERROR; } - if (ctx->stream_id && ctx->done) { + if (ctx->stream_id && ctx->done + && ctx->type != NGX_HTTP_V2_RST_STREAM_FRAME) + { ngx_log_error(NGX_LOG_ERR, r->connection->log, 0, "upstream sent frame for closed stream %ui", ctx->stream_id); @@ -2131,11 +2135,21 @@ ngx_http_grpc_filter(void *data, ssize_t return NGX_ERROR; } - ngx_log_error(NGX_LOG_ERR, r->connection->log, 0, - "upstream rejected request with error %ui", - ctx->error); - - return NGX_ERROR; + if (ctx->error || !ctx->done) { + ngx_log_error(NGX_LOG_ERR, r->connection->log, 0, + "upstream rejected request with error %ui", + ctx->error); + return NGX_ERROR; + } + + if (ctx->rst) { + ngx_log_error(NGX_LOG_ERR, r->connection->log, 0, + "upstream sent frame for closed stream %ui", + ctx->stream_id); + return NGX_ERROR; + } + + ctx->rst = 1; } if (ctx->type == NGX_HTTP_V2_GOAWAY_FRAME) { From ru at nginx.com Thu Apr 23 12:11:05 2020 From: ru at nginx.com (Ruslan Ermilov) Date: Thu, 23 Apr 2020 12:11:05 +0000 Subject: [nginx] gRPC: WINDOW_UPDATE after END_STREAM handling (ticket #1797). Message-ID: details: https://hg.nginx.org/nginx/rev/716eddd74bc2 branches: changeset: 7647:716eddd74bc2 user: Ruslan Ermilov date: Thu Apr 23 15:10:26 2020 +0300 description: gRPC: WINDOW_UPDATE after END_STREAM handling (ticket #1797). As per https://tools.ietf.org/html/rfc7540#section-6.9, WINDOW_UPDATE received after a frame with the END_STREAM flag should be handled and not treated as an error. diffstat: src/http/modules/ngx_http_grpc_module.c | 3 ++- 1 files changed, 2 insertions(+), 1 deletions(-) diffs (13 lines): diff -r 2096b21fcd10 -r 716eddd74bc2 src/http/modules/ngx_http_grpc_module.c --- a/src/http/modules/ngx_http_grpc_module.c Thu Apr 23 15:10:24 2020 +0300 +++ b/src/http/modules/ngx_http_grpc_module.c Thu Apr 23 15:10:26 2020 +0300 @@ -2091,7 +2091,8 @@ ngx_http_grpc_filter(void *data, ssize_t } if (ctx->stream_id && ctx->done - && ctx->type != NGX_HTTP_V2_RST_STREAM_FRAME) + && ctx->type != NGX_HTTP_V2_RST_STREAM_FRAME + && ctx->type != NGX_HTTP_V2_WINDOW_UPDATE_FRAME) { ngx_log_error(NGX_LOG_ERR, r->connection->log, 0, "upstream sent frame for closed stream %ui", From garret at trailofbits.com Fri Apr 24 15:17:46 2020 From: garret at trailofbits.com (Garret Reece) Date: Fri, 24 Apr 2020 10:17:46 -0500 Subject: [PATCH] Expose additional SSL variables Message-ID: # HG changeset patch # User Garret Reece # Date 1587691836 18000 # Thu Apr 23 20:30:36 2020 -0500 # Node ID 86d2f46807f597249fa59072b920a389f8c082ee # Parent 716eddd74bc2831537f5b3f7ecd16ad3e516d043 Expose additional SSL variables. Expose the ssl extensions and elliptic curve point formats provided by client. This enables ja3 fingerprinting of TLS connections. diff -r 716eddd74bc2 -r 86d2f46807f5 src/event/ngx_event_openssl.c --- a/src/event/ngx_event_openssl.c Thu Apr 23 15:10:26 2020 +0300 +++ b/src/event/ngx_event_openssl.c Thu Apr 23 20:30:36 2020 -0500 @@ -1588,6 +1588,100 @@ return NGX_OK; } +#if OPENSSL_VERSION_NUMBER >= 0x10101000L + +void +ngx_SSL_client_features(ngx_connection_t *c) +{ + unsigned short *ciphers_out = NULL; + int *curves_out = NULL; + int *point_formats_out = NULL; + size_t len = 0; + SSL *s = NULL; + + if (c == NULL) { + return; + } + s = c->ssl->connection; + + /* Cipher suites */ + c->ssl->ciphers = NULL; + c->ssl->ciphers_sz = SSL_get0_raw_cipherlist(s, &ciphers_out); + c->ssl->ciphers_sz /= 2; + + if (c->ssl->ciphers_sz && ciphers_out) { + len = c->ssl->ciphers_sz * sizeof(unsigned short); + c->ssl->ciphers = ngx_pnalloc(c->pool, len); + ngx_memcpy(c->ssl->ciphers, ciphers_out, len); + } + + /* Elliptic curve points */ + c->ssl->curves_sz = SSL_get1_curves(s, NULL); + if (c->ssl->curves_sz) { + curves_out = OPENSSL_malloc(c->ssl->curves_sz * sizeof(int)); + if (curves_out != NULL) { + SSL_get1_curves(s, curves_out); + len = c->ssl->curves_sz * sizeof(unsigned short); + c->ssl->curves = ngx_pnalloc(c->pool, len); + if (c->ssl->curves != NULL) { + for (size_t i = 0; i < c->ssl->curves_sz; i++) { + c->ssl->curves[i] = curves_out[i]; + } + } + OPENSSL_free(curves_out); + } + } + + /* Elliptic curve point formats */ + c->ssl->point_formats_sz = SSL_get0_ec_point_formats(s, + &point_formats_out); + if (c->ssl->point_formats_sz && point_formats_out != NULL) { + len = c->ssl->point_formats_sz * sizeof(unsigned char); + c->ssl->point_formats = ngx_pnalloc(c->pool, len); + if (c->ssl->point_formats != NULL) { + ngx_memcpy(c->ssl->point_formats, point_formats_out, len); + } + } +} + +int +ngx_SSL_early_cb_fn(SSL *s, int *al, void *arg) +{ + int got_extensions; + int *ext_out; + size_t ext_len; + ngx_connection_t *c; + + c = arg; + + if (c == NULL) { + return 1; + } + + if (c->ssl == NULL) { + return 1; + } + + c->ssl->extensions_size = 0; + c->ssl->extensions = NULL; + got_extensions = SSL_client_hello_get1_extensions_present(s, + &ext_out, + &ext_len); + if (got_extensions) { + if (ext_out && ext_len) { + c->ssl->extensions = + ngx_palloc(c->pool, sizeof(int) * ext_len); + if (c->ssl->extensions != NULL) { + c->ssl->extensions_size = ext_len; + ngx_memcpy(c->ssl->extensions, ext_out, sizeof(int) * ext_len); + OPENSSL_free(ext_out); + } + } + } + + return 1; +} +#endif ngx_int_t ngx_ssl_handshake(ngx_connection_t *c) @@ -1603,6 +1697,10 @@ ngx_ssl_clear_error(c->log); +#if OPENSSL_VERSION_NUMBER >= 0x10101000L + SSL_CTX_set_client_hello_cb(c->ssl->session_ctx, ngx_SSL_early_cb_fn, c); +#endif + n = SSL_do_handshake(c->ssl->connection); ngx_log_debug1(NGX_LOG_DEBUG_EVENT, c->log, 0, "SSL_do_handshake: %d", n); @@ -1623,6 +1721,10 @@ c->ssl->handshaked = 1; +#if OPENSSL_VERSION_NUMBER >= 0x10101000L + ngx_SSL_client_features(c); +#endif + c->recv = ngx_ssl_recv; c->send = ngx_ssl_write; c->recv_chain = ngx_ssl_recv_chain; @@ -5044,6 +5146,86 @@ return NGX_OK; } +#if OPENSSL_VERSION_NUMBER >= 0x10101000L + +ngx_int_t +ngx_ssl_get_extensions(ngx_connection_t *c, ngx_pool_t *pool, ngx_str_t *s) +{ + size_t len; + u_char *p; + + len = 0; + s->len = 0; + + if (c->ssl->extensions_size && c->ssl->extensions) { + for (int n = c->ssl->extensions[0]; n > 9; n /= 10) { + len += 1; + } + len += 1; + for (size_t i = 1; i < c->ssl->extensions_size; ++i) { + len += 1; /* for the '-' separator */ + for (int n = c->ssl->extensions[i]; n > 9; n /= 10) { + len += 1; + } + len += 1; + } + + s->data = ngx_pnalloc(pool, len+1); + if (s->data == NULL) { + return NGX_ERROR; + } + s->len = len; + + p = ngx_sprintf(s->data, "%d", c->ssl->extensions[0]); + for (size_t i = 1; i < c->ssl->extensions_size; ++i) { + p = ngx_sprintf(p, "-%d", c->ssl->extensions[i]); + } + + } + return NGX_OK; +} + + +ngx_int_t +ngx_ssl_get_ec_point_formats(ngx_connection_t *c, ngx_pool_t *pool, + ngx_str_t *s) +{ + size_t len; + u_char *p; + + len = 0; + s->len = 0; + + if (c->ssl->point_formats_sz && c->ssl->point_formats) { + for (unsigned char n = c->ssl->point_formats[0]; n > 9; n /= 10) { + len += 1; + } + len += 1; + for (size_t i = 1; i < c->ssl->point_formats_sz; ++i) { + len += 1; /* for the '-' separator */ + for (unsigned char n = c->ssl->point_formats[i]; n > 9; n /= 10) { + len += 1; + } + len += 1; + } + + s->data = ngx_pnalloc(pool, len+1); + if (s->data == NULL) { + return NGX_ERROR; + } + s->len = len; + + p = ngx_sprintf(s->data, "%d", c->ssl->point_formats[0]); + for (size_t i = 1; i < c->ssl->point_formats_sz; ++i) { + p = ngx_sprintf(p, "-%d", c->ssl->point_formats[i]); + } + } + + return NGX_OK; +} + +#endif + static time_t ngx_ssl_parse_time( diff -r 716eddd74bc2 -r 86d2f46807f5 src/event/ngx_event_openssl.h --- a/src/event/ngx_event_openssl.h Thu Apr 23 15:10:26 2020 +0300 +++ b/src/event/ngx_event_openssl.h Thu Apr 23 20:30:36 2020 -0500 @@ -99,6 +99,21 @@ unsigned in_early:1; unsigned early_preread:1; unsigned write_blocked:1; + +#if OPENSSL_VERSION_NUMBER >= 0x10101000L + + size_t ciphers_sz; + unsigned short *ciphers; + + size_t extensions_size; + int *extensions; + + size_t curves_sz; + unsigned short *curves; + + size_t point_formats_sz; + unsigned char *point_formats; +#endif }; @@ -263,6 +278,14 @@ ngx_int_t ngx_ssl_get_client_v_remain(ngx_connection_t *c, ngx_pool_t *pool, ngx_str_t *s); +#if OPENSSL_VERSION_NUMBER >= 0x10101000L + +ngx_int_t ngx_ssl_get_extensions(ngx_connection_t *c, ngx_pool_t *pool, + ngx_str_t *s); +ngx_int_t ngx_ssl_get_ec_point_formats(ngx_connection_t *c, ngx_pool_t *pool, + ngx_str_t *s); + +#endif ngx_int_t ngx_ssl_handshake(ngx_connection_t *c); ssize_t ngx_ssl_recv(ngx_connection_t *c, u_char *buf, size_t size); diff -r 716eddd74bc2 -r 86d2f46807f5 src/http/modules/ngx_http_ssl_module.c --- a/src/http/modules/ngx_http_ssl_module.c Thu Apr 23 15:10:26 2020 +0300 +++ b/src/http/modules/ngx_http_ssl_module.c Thu Apr 23 20:30:36 2020 -0500 @@ -352,6 +352,17 @@ { ngx_string("ssl_client_v_remain"), NULL, ngx_http_ssl_variable, (uintptr_t) ngx_ssl_get_client_v_remain, NGX_HTTP_VAR_CHANGEABLE, 0 }, +#if OPENSSL_VERSION_NUMBER >= 0x10101000L + + { ngx_string("ssl_extensions"), NULL, ngx_http_ssl_variable, + (uintptr_t) ngx_ssl_get_extensions, NGX_HTTP_VAR_CHANGEABLE, 0 }, + + { ngx_string("ssl_elliptic_curve_point_formats"), NULL, + ngx_http_ssl_variable, (uintptr_t) ngx_ssl_get_ec_point_formats, + NGX_HTTP_VAR_CHANGEABLE, 0 }, + +#endif + ngx_http_null_variable }; From fooinha at gmail.com Fri Apr 24 16:17:28 2020 From: fooinha at gmail.com (Paulo Pacheco) Date: Fri, 24 Apr 2020 17:17:28 +0100 Subject: [PATCH] Expose additional SSL variables In-Reply-To: References: Message-ID: Hello, this looks very similar with what I've done here. https://github.com/fooinha/nginx-ssl-ja3/blob/master/patches/nginx.1.17.1.ssl.extensions.patch Is this the same code? Thanx. > On 24 Apr 2020, at 16:17, Garret Reece wrote: > > # HG changeset patch > # User Garret Reece > # Date 1587691836 18000 > # Thu Apr 23 20:30:36 2020 -0500 > # Node ID 86d2f46807f597249fa59072b920a389f8c082ee > # Parent 716eddd74bc2831537f5b3f7ecd16ad3e516d043 > Expose additional SSL variables. > > Expose the ssl extensions and elliptic curve point formats provided by client. > This enables ja3 fingerprinting of TLS connections. > > diff -r 716eddd74bc2 -r 86d2f46807f5 src/event/ngx_event_openssl.c > --- a/src/event/ngx_event_openssl.c Thu Apr 23 15:10:26 2020 +0300 > +++ b/src/event/ngx_event_openssl.c Thu Apr 23 20:30:36 2020 -0500 > @@ -1588,6 +1588,100 @@ > return NGX_OK; > } > > +#if OPENSSL_VERSION_NUMBER >= 0x10101000L > + > +void > +ngx_SSL_client_features(ngx_connection_t *c) > +{ > + unsigned short *ciphers_out = NULL; > + int *curves_out = NULL; > + int *point_formats_out = NULL; > + size_t len = 0; > + SSL *s = NULL; > + > + if (c == NULL) { > + return; > + } > + s = c->ssl->connection; > + > + /* Cipher suites */ > + c->ssl->ciphers = NULL; > + c->ssl->ciphers_sz = SSL_get0_raw_cipherlist(s, &ciphers_out); > + c->ssl->ciphers_sz /= 2; > + > + if (c->ssl->ciphers_sz && ciphers_out) { > + len = c->ssl->ciphers_sz * sizeof(unsigned short); > + c->ssl->ciphers = ngx_pnalloc(c->pool, len); > + ngx_memcpy(c->ssl->ciphers, ciphers_out, len); > + } > + > + /* Elliptic curve points */ > + c->ssl->curves_sz = SSL_get1_curves(s, NULL); > + if (c->ssl->curves_sz) { > + curves_out = OPENSSL_malloc(c->ssl->curves_sz * sizeof(int)); > + if (curves_out != NULL) { > + SSL_get1_curves(s, curves_out); > + len = c->ssl->curves_sz * sizeof(unsigned short); > + c->ssl->curves = ngx_pnalloc(c->pool, len); > + if (c->ssl->curves != NULL) { > + for (size_t i = 0; i < c->ssl->curves_sz; i++) { > + c->ssl->curves[i] = curves_out[i]; > + } > + } > + OPENSSL_free(curves_out); > + } > + } > + > + /* Elliptic curve point formats */ > + c->ssl->point_formats_sz = SSL_get0_ec_point_formats(s, > + &point_formats_out); > + if (c->ssl->point_formats_sz && point_formats_out != NULL) { > + len = c->ssl->point_formats_sz * sizeof(unsigned char); > + c->ssl->point_formats = ngx_pnalloc(c->pool, len); > + if (c->ssl->point_formats != NULL) { > + ngx_memcpy(c->ssl->point_formats, point_formats_out, len); > + } > + } > +} > + > +int > +ngx_SSL_early_cb_fn(SSL *s, int *al, void *arg) > +{ > + int got_extensions; > + int *ext_out; > + size_t ext_len; > + ngx_connection_t *c; > + > + c = arg; > + > + if (c == NULL) { > + return 1; > + } > + > + if (c->ssl == NULL) { > + return 1; > + } > + > + c->ssl->extensions_size = 0; > + c->ssl->extensions = NULL; > + got_extensions = SSL_client_hello_get1_extensions_present(s, > + &ext_out, > + &ext_len); > + if (got_extensions) { > + if (ext_out && ext_len) { > + c->ssl->extensions = > + ngx_palloc(c->pool, sizeof(int) * ext_len); > + if (c->ssl->extensions != NULL) { > + c->ssl->extensions_size = ext_len; > + ngx_memcpy(c->ssl->extensions, ext_out, sizeof(int) * ext_len); > + OPENSSL_free(ext_out); > + } > + } > + } > + > + return 1; > +} > +#endif > > ngx_int_t > ngx_ssl_handshake(ngx_connection_t *c) > @@ -1603,6 +1697,10 @@ > > ngx_ssl_clear_error(c->log); > > +#if OPENSSL_VERSION_NUMBER >= 0x10101000L > + SSL_CTX_set_client_hello_cb(c->ssl->session_ctx, ngx_SSL_early_cb_fn, c); > +#endif > + > n = SSL_do_handshake(c->ssl->connection); > > ngx_log_debug1(NGX_LOG_DEBUG_EVENT, c->log, 0, "SSL_do_handshake: %d", n); > @@ -1623,6 +1721,10 @@ > > c->ssl->handshaked = 1; > > +#if OPENSSL_VERSION_NUMBER >= 0x10101000L > + ngx_SSL_client_features(c); > +#endif > + > c->recv = ngx_ssl_recv; > c->send = ngx_ssl_write; > c->recv_chain = ngx_ssl_recv_chain; > @@ -5044,6 +5146,86 @@ > return NGX_OK; > } > > +#if OPENSSL_VERSION_NUMBER >= 0x10101000L > + > +ngx_int_t > +ngx_ssl_get_extensions(ngx_connection_t *c, ngx_pool_t *pool, ngx_str_t *s) > +{ > + size_t len; > + u_char *p; > + > + len = 0; > + s->len = 0; > + > + if (c->ssl->extensions_size && c->ssl->extensions) { > + for (int n = c->ssl->extensions[0]; n > 9; n /= 10) { > + len += 1; > + } > + len += 1; > + for (size_t i = 1; i < c->ssl->extensions_size; ++i) { > + len += 1; /* for the '-' separator */ > + for (int n = c->ssl->extensions[i]; n > 9; n /= 10) { > + len += 1; > + } > + len += 1; > + } > + > + s->data = ngx_pnalloc(pool, len+1); > + if (s->data == NULL) { > + return NGX_ERROR; > + } > + s->len = len; > + > + p = ngx_sprintf(s->data, "%d", c->ssl->extensions[0]); > + for (size_t i = 1; i < c->ssl->extensions_size; ++i) { > + p = ngx_sprintf(p, "-%d", c->ssl->extensions[i]); > + } > + > + } > + return NGX_OK; > +} > + > + > +ngx_int_t > +ngx_ssl_get_ec_point_formats(ngx_connection_t *c, ngx_pool_t *pool, > + ngx_str_t *s) > +{ > + size_t len; > + u_char *p; > + > + len = 0; > + s->len = 0; > + > + if (c->ssl->point_formats_sz && c->ssl->point_formats) { > + for (unsigned char n = c->ssl->point_formats[0]; n > 9; n /= 10) { > + len += 1; > + } > + len += 1; > + for (size_t i = 1; i < c->ssl->point_formats_sz; ++i) { > + len += 1; /* for the '-' separator */ > + for (unsigned char n = c->ssl->point_formats[i]; n > 9; n /= 10) { > + len += 1; > + } > + len += 1; > + } > + > + s->data = ngx_pnalloc(pool, len+1); > + if (s->data == NULL) { > + return NGX_ERROR; > + } > + s->len = len; > + > + p = ngx_sprintf(s->data, "%d", c->ssl->point_formats[0]); > + for (size_t i = 1; i < c->ssl->point_formats_sz; ++i) { > + p = ngx_sprintf(p, "-%d", c->ssl->point_formats[i]); > + } > + } > + > + return NGX_OK; > +} > + > +#endif > + > > static time_t > ngx_ssl_parse_time( > diff -r 716eddd74bc2 -r 86d2f46807f5 src/event/ngx_event_openssl.h > --- a/src/event/ngx_event_openssl.h Thu Apr 23 15:10:26 2020 +0300 > +++ b/src/event/ngx_event_openssl.h Thu Apr 23 20:30:36 2020 -0500 > @@ -99,6 +99,21 @@ > unsigned in_early:1; > unsigned early_preread:1; > unsigned write_blocked:1; > + > +#if OPENSSL_VERSION_NUMBER >= 0x10101000L > + > + size_t ciphers_sz; > + unsigned short *ciphers; > + > + size_t extensions_size; > + int *extensions; > + > + size_t curves_sz; > + unsigned short *curves; > + > + size_t point_formats_sz; > + unsigned char *point_formats; > +#endif > }; > > > @@ -263,6 +278,14 @@ > ngx_int_t ngx_ssl_get_client_v_remain(ngx_connection_t *c, ngx_pool_t *pool, > ngx_str_t *s); > > +#if OPENSSL_VERSION_NUMBER >= 0x10101000L > + > +ngx_int_t ngx_ssl_get_extensions(ngx_connection_t *c, ngx_pool_t *pool, > + ngx_str_t *s); > +ngx_int_t ngx_ssl_get_ec_point_formats(ngx_connection_t *c, ngx_pool_t *pool, > + ngx_str_t *s); > + > +#endif > > ngx_int_t ngx_ssl_handshake(ngx_connection_t *c); > ssize_t ngx_ssl_recv(ngx_connection_t *c, u_char *buf, size_t size); > diff -r 716eddd74bc2 -r 86d2f46807f5 src/http/modules/ngx_http_ssl_module.c > --- a/src/http/modules/ngx_http_ssl_module.c Thu Apr 23 15:10:26 2020 +0300 > +++ b/src/http/modules/ngx_http_ssl_module.c Thu Apr 23 20:30:36 2020 -0500 > @@ -352,6 +352,17 @@ > { ngx_string("ssl_client_v_remain"), NULL, ngx_http_ssl_variable, > (uintptr_t) ngx_ssl_get_client_v_remain, NGX_HTTP_VAR_CHANGEABLE, 0 }, > > +#if OPENSSL_VERSION_NUMBER >= 0x10101000L > + > + { ngx_string("ssl_extensions"), NULL, ngx_http_ssl_variable, > + (uintptr_t) ngx_ssl_get_extensions, NGX_HTTP_VAR_CHANGEABLE, 0 }, > + > + { ngx_string("ssl_elliptic_curve_point_formats"), NULL, > + ngx_http_ssl_variable, (uintptr_t) ngx_ssl_get_ec_point_formats, > + NGX_HTTP_VAR_CHANGEABLE, 0 }, > + > +#endif > + > ngx_http_null_variable > }; > _______________________________________________ > nginx-devel mailing list > nginx-devel at nginx.org > http://mailman.nginx.org/mailman/listinfo/nginx-devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From garret at trailofbits.com Fri Apr 24 16:25:26 2020 From: garret at trailofbits.com (Garret Reece) Date: Fri, 24 Apr 2020 11:25:26 -0500 Subject: [PATCH] Expose additional SSL variables In-Reply-To: References: Message-ID: More or less, yes--we had a request from a customer to take just the extensions and the elliptic curve format points data from your patch and make them available in the main package On Fri, Apr 24, 2020 at 11:17 AM Paulo Pacheco wrote: > > Hello, this looks very similar with what I've done here. > > https://github.com/fooinha/nginx-ssl-ja3/blob/master/patches/nginx.1.17.1.ssl.extensions.patch > > Is this the same code? > > > Thanx. > > > > > > > On 24 Apr 2020, at 16:17, Garret Reece wrote: > > # HG changeset patch > # User Garret Reece > # Date 1587691836 18000 > # Thu Apr 23 20:30:36 2020 -0500 > # Node ID 86d2f46807f597249fa59072b920a389f8c082ee > # Parent 716eddd74bc2831537f5b3f7ecd16ad3e516d043 > Expose additional SSL variables. > > Expose the ssl extensions and elliptic curve point formats provided by client. > This enables ja3 fingerprinting of TLS connections. > > diff -r 716eddd74bc2 -r 86d2f46807f5 src/event/ngx_event_openssl.c > --- a/src/event/ngx_event_openssl.c Thu Apr 23 15:10:26 2020 +0300 > +++ b/src/event/ngx_event_openssl.c Thu Apr 23 20:30:36 2020 -0500 > @@ -1588,6 +1588,100 @@ > return NGX_OK; > } > > +#if OPENSSL_VERSION_NUMBER >= 0x10101000L > + > +void > +ngx_SSL_client_features(ngx_connection_t *c) > +{ > + unsigned short *ciphers_out = NULL; > + int *curves_out = NULL; > + int *point_formats_out = NULL; > + size_t len = 0; > + SSL *s = NULL; > + > + if (c == NULL) { > + return; > + } > + s = c->ssl->connection; > + > + /* Cipher suites */ > + c->ssl->ciphers = NULL; > + c->ssl->ciphers_sz = SSL_get0_raw_cipherlist(s, &ciphers_out); > + c->ssl->ciphers_sz /= 2; > + > + if (c->ssl->ciphers_sz && ciphers_out) { > + len = c->ssl->ciphers_sz * sizeof(unsigned short); > + c->ssl->ciphers = ngx_pnalloc(c->pool, len); > + ngx_memcpy(c->ssl->ciphers, ciphers_out, len); > + } > + > + /* Elliptic curve points */ > + c->ssl->curves_sz = SSL_get1_curves(s, NULL); > + if (c->ssl->curves_sz) { > + curves_out = OPENSSL_malloc(c->ssl->curves_sz * sizeof(int)); > + if (curves_out != NULL) { > + SSL_get1_curves(s, curves_out); > + len = c->ssl->curves_sz * sizeof(unsigned short); > + c->ssl->curves = ngx_pnalloc(c->pool, len); > + if (c->ssl->curves != NULL) { > + for (size_t i = 0; i < c->ssl->curves_sz; i++) { > + c->ssl->curves[i] = curves_out[i]; > + } > + } > + OPENSSL_free(curves_out); > + } > + } > + > + /* Elliptic curve point formats */ > + c->ssl->point_formats_sz = SSL_get0_ec_point_formats(s, > + &point_formats_out); > + if (c->ssl->point_formats_sz && point_formats_out != NULL) { > + len = c->ssl->point_formats_sz * sizeof(unsigned char); > + c->ssl->point_formats = ngx_pnalloc(c->pool, len); > + if (c->ssl->point_formats != NULL) { > + ngx_memcpy(c->ssl->point_formats, point_formats_out, len); > + } > + } > +} > + > +int > +ngx_SSL_early_cb_fn(SSL *s, int *al, void *arg) > +{ > + int got_extensions; > + int *ext_out; > + size_t ext_len; > + ngx_connection_t *c; > + > + c = arg; > + > + if (c == NULL) { > + return 1; > + } > + > + if (c->ssl == NULL) { > + return 1; > + } > + > + c->ssl->extensions_size = 0; > + c->ssl->extensions = NULL; > + got_extensions = SSL_client_hello_get1_extensions_present(s, > + &ext_out, > + &ext_len); > + if (got_extensions) { > + if (ext_out && ext_len) { > + c->ssl->extensions = > + ngx_palloc(c->pool, sizeof(int) * ext_len); > + if (c->ssl->extensions != NULL) { > + c->ssl->extensions_size = ext_len; > + ngx_memcpy(c->ssl->extensions, ext_out, sizeof(int) * ext_len); > + OPENSSL_free(ext_out); > + } > + } > + } > + > + return 1; > +} > +#endif > > ngx_int_t > ngx_ssl_handshake(ngx_connection_t *c) > @@ -1603,6 +1697,10 @@ > > ngx_ssl_clear_error(c->log); > > +#if OPENSSL_VERSION_NUMBER >= 0x10101000L > + SSL_CTX_set_client_hello_cb(c->ssl->session_ctx, ngx_SSL_early_cb_fn, c); > +#endif > + > n = SSL_do_handshake(c->ssl->connection); > > ngx_log_debug1(NGX_LOG_DEBUG_EVENT, c->log, 0, "SSL_do_handshake: %d", n); > @@ -1623,6 +1721,10 @@ > > c->ssl->handshaked = 1; > > +#if OPENSSL_VERSION_NUMBER >= 0x10101000L > + ngx_SSL_client_features(c); > +#endif > + > c->recv = ngx_ssl_recv; > c->send = ngx_ssl_write; > c->recv_chain = ngx_ssl_recv_chain; > @@ -5044,6 +5146,86 @@ > return NGX_OK; > } > > +#if OPENSSL_VERSION_NUMBER >= 0x10101000L > + > +ngx_int_t > +ngx_ssl_get_extensions(ngx_connection_t *c, ngx_pool_t *pool, ngx_str_t *s) > +{ > + size_t len; > + u_char *p; > + > + len = 0; > + s->len = 0; > + > + if (c->ssl->extensions_size && c->ssl->extensions) { > + for (int n = c->ssl->extensions[0]; n > 9; n /= 10) { > + len += 1; > + } > + len += 1; > + for (size_t i = 1; i < c->ssl->extensions_size; ++i) { > + len += 1; /* for the '-' separator */ > + for (int n = c->ssl->extensions[i]; n > 9; n /= 10) { > + len += 1; > + } > + len += 1; > + } > + > + s->data = ngx_pnalloc(pool, len+1); > + if (s->data == NULL) { > + return NGX_ERROR; > + } > + s->len = len; > + > + p = ngx_sprintf(s->data, "%d", c->ssl->extensions[0]); > + for (size_t i = 1; i < c->ssl->extensions_size; ++i) { > + p = ngx_sprintf(p, "-%d", c->ssl->extensions[i]); > + } > + > + } > + return NGX_OK; > +} > + > + > +ngx_int_t > +ngx_ssl_get_ec_point_formats(ngx_connection_t *c, ngx_pool_t *pool, > + ngx_str_t *s) > +{ > + size_t len; > + u_char *p; > + > + len = 0; > + s->len = 0; > + > + if (c->ssl->point_formats_sz && c->ssl->point_formats) { > + for (unsigned char n = c->ssl->point_formats[0]; n > 9; n /= 10) { > + len += 1; > + } > + len += 1; > + for (size_t i = 1; i < c->ssl->point_formats_sz; ++i) { > + len += 1; /* for the '-' separator */ > + for (unsigned char n = c->ssl->point_formats[i]; n > 9; n /= 10) { > + len += 1; > + } > + len += 1; > + } > + > + s->data = ngx_pnalloc(pool, len+1); > + if (s->data == NULL) { > + return NGX_ERROR; > + } > + s->len = len; > + > + p = ngx_sprintf(s->data, "%d", c->ssl->point_formats[0]); > + for (size_t i = 1; i < c->ssl->point_formats_sz; ++i) { > + p = ngx_sprintf(p, "-%d", c->ssl->point_formats[i]); > + } > + } > + > + return NGX_OK; > +} > + > +#endif > + > > static time_t > ngx_ssl_parse_time( > diff -r 716eddd74bc2 -r 86d2f46807f5 src/event/ngx_event_openssl.h > --- a/src/event/ngx_event_openssl.h Thu Apr 23 15:10:26 2020 +0300 > +++ b/src/event/ngx_event_openssl.h Thu Apr 23 20:30:36 2020 -0500 > @@ -99,6 +99,21 @@ > unsigned in_early:1; > unsigned early_preread:1; > unsigned write_blocked:1; > + > +#if OPENSSL_VERSION_NUMBER >= 0x10101000L > + > + size_t ciphers_sz; > + unsigned short *ciphers; > + > + size_t extensions_size; > + int *extensions; > + > + size_t curves_sz; > + unsigned short *curves; > + > + size_t point_formats_sz; > + unsigned char *point_formats; > +#endif > }; > > > @@ -263,6 +278,14 @@ > ngx_int_t ngx_ssl_get_client_v_remain(ngx_connection_t *c, ngx_pool_t *pool, > ngx_str_t *s); > > +#if OPENSSL_VERSION_NUMBER >= 0x10101000L > + > +ngx_int_t ngx_ssl_get_extensions(ngx_connection_t *c, ngx_pool_t *pool, > + ngx_str_t *s); > +ngx_int_t ngx_ssl_get_ec_point_formats(ngx_connection_t *c, ngx_pool_t *pool, > + ngx_str_t *s); > + > +#endif > > ngx_int_t ngx_ssl_handshake(ngx_connection_t *c); > ssize_t ngx_ssl_recv(ngx_connection_t *c, u_char *buf, size_t size); > diff -r 716eddd74bc2 -r 86d2f46807f5 src/http/modules/ngx_http_ssl_module.c > --- a/src/http/modules/ngx_http_ssl_module.c Thu Apr 23 15:10:26 2020 +0300 > +++ b/src/http/modules/ngx_http_ssl_module.c Thu Apr 23 20:30:36 2020 -0500 > @@ -352,6 +352,17 @@ > { ngx_string("ssl_client_v_remain"), NULL, ngx_http_ssl_variable, > (uintptr_t) ngx_ssl_get_client_v_remain, NGX_HTTP_VAR_CHANGEABLE, 0 }, > > +#if OPENSSL_VERSION_NUMBER >= 0x10101000L > + > + { ngx_string("ssl_extensions"), NULL, ngx_http_ssl_variable, > + (uintptr_t) ngx_ssl_get_extensions, NGX_HTTP_VAR_CHANGEABLE, 0 }, > + > + { ngx_string("ssl_elliptic_curve_point_formats"), NULL, > + ngx_http_ssl_variable, (uintptr_t) ngx_ssl_get_ec_point_formats, > + NGX_HTTP_VAR_CHANGEABLE, 0 }, > + > +#endif > + > ngx_http_null_variable > }; > _______________________________________________ > nginx-devel mailing list > nginx-devel at nginx.org > http://mailman.nginx.org/mailman/listinfo/nginx-devel > > From fooinha at gmail.com Fri Apr 24 16:32:08 2020 From: fooinha at gmail.com (Paulo Pacheco) Date: Fri, 24 Apr 2020 17:32:08 +0100 Subject: [PATCH] Expose additional SSL variables In-Reply-To: References: Message-ID: The very first try to submit the early development of this patch here, got this response: http://mailman.nginx.org/pipermail/nginx-devel/2017-August/010426.html Thanx > On 24 Apr 2020, at 17:25, Garret Reece wrote: > > More or less, yes--we had a request from a customer to take just the > extensions and the elliptic curve format points data from your patch > and make them available in the main package > > On Fri, Apr 24, 2020 at 11:17 AM Paulo Pacheco wrote: >> >> Hello, this looks very similar with what I've done here. >> >> https://github.com/fooinha/nginx-ssl-ja3/blob/master/patches/nginx.1.17.1.ssl.extensions.patch >> >> Is this the same code? >> >> >> Thanx. >> >> >> >> >> >> >> On 24 Apr 2020, at 16:17, Garret Reece wrote: >> >> # HG changeset patch >> # User Garret Reece >> # Date 1587691836 18000 >> # Thu Apr 23 20:30:36 2020 -0500 >> # Node ID 86d2f46807f597249fa59072b920a389f8c082ee >> # Parent 716eddd74bc2831537f5b3f7ecd16ad3e516d043 >> Expose additional SSL variables. >> >> Expose the ssl extensions and elliptic curve point formats provided by client. >> This enables ja3 fingerprinting of TLS connections. >> >> diff -r 716eddd74bc2 -r 86d2f46807f5 src/event/ngx_event_openssl.c >> --- a/src/event/ngx_event_openssl.c Thu Apr 23 15:10:26 2020 +0300 >> +++ b/src/event/ngx_event_openssl.c Thu Apr 23 20:30:36 2020 -0500 >> @@ -1588,6 +1588,100 @@ >> return NGX_OK; >> } >> >> +#if OPENSSL_VERSION_NUMBER >= 0x10101000L >> + >> +void >> +ngx_SSL_client_features(ngx_connection_t *c) >> +{ >> + unsigned short *ciphers_out = NULL; >> + int *curves_out = NULL; >> + int *point_formats_out = NULL; >> + size_t len = 0; >> + SSL *s = NULL; >> + >> + if (c == NULL) { >> + return; >> + } >> + s = c->ssl->connection; >> + >> + /* Cipher suites */ >> + c->ssl->ciphers = NULL; >> + c->ssl->ciphers_sz = SSL_get0_raw_cipherlist(s, &ciphers_out); >> + c->ssl->ciphers_sz /= 2; >> + >> + if (c->ssl->ciphers_sz && ciphers_out) { >> + len = c->ssl->ciphers_sz * sizeof(unsigned short); >> + c->ssl->ciphers = ngx_pnalloc(c->pool, len); >> + ngx_memcpy(c->ssl->ciphers, ciphers_out, len); >> + } >> + >> + /* Elliptic curve points */ >> + c->ssl->curves_sz = SSL_get1_curves(s, NULL); >> + if (c->ssl->curves_sz) { >> + curves_out = OPENSSL_malloc(c->ssl->curves_sz * sizeof(int)); >> + if (curves_out != NULL) { >> + SSL_get1_curves(s, curves_out); >> + len = c->ssl->curves_sz * sizeof(unsigned short); >> + c->ssl->curves = ngx_pnalloc(c->pool, len); >> + if (c->ssl->curves != NULL) { >> + for (size_t i = 0; i < c->ssl->curves_sz; i++) { >> + c->ssl->curves[i] = curves_out[i]; >> + } >> + } >> + OPENSSL_free(curves_out); >> + } >> + } >> + >> + /* Elliptic curve point formats */ >> + c->ssl->point_formats_sz = SSL_get0_ec_point_formats(s, >> + &point_formats_out); >> + if (c->ssl->point_formats_sz && point_formats_out != NULL) { >> + len = c->ssl->point_formats_sz * sizeof(unsigned char); >> + c->ssl->point_formats = ngx_pnalloc(c->pool, len); >> + if (c->ssl->point_formats != NULL) { >> + ngx_memcpy(c->ssl->point_formats, point_formats_out, len); >> + } >> + } >> +} >> + >> +int >> +ngx_SSL_early_cb_fn(SSL *s, int *al, void *arg) >> +{ >> + int got_extensions; >> + int *ext_out; >> + size_t ext_len; >> + ngx_connection_t *c; >> + >> + c = arg; >> + >> + if (c == NULL) { >> + return 1; >> + } >> + >> + if (c->ssl == NULL) { >> + return 1; >> + } >> + >> + c->ssl->extensions_size = 0; >> + c->ssl->extensions = NULL; >> + got_extensions = SSL_client_hello_get1_extensions_present(s, >> + &ext_out, >> + &ext_len); >> + if (got_extensions) { >> + if (ext_out && ext_len) { >> + c->ssl->extensions = >> + ngx_palloc(c->pool, sizeof(int) * ext_len); >> + if (c->ssl->extensions != NULL) { >> + c->ssl->extensions_size = ext_len; >> + ngx_memcpy(c->ssl->extensions, ext_out, sizeof(int) * ext_len); >> + OPENSSL_free(ext_out); >> + } >> + } >> + } >> + >> + return 1; >> +} >> +#endif >> >> ngx_int_t >> ngx_ssl_handshake(ngx_connection_t *c) >> @@ -1603,6 +1697,10 @@ >> >> ngx_ssl_clear_error(c->log); >> >> +#if OPENSSL_VERSION_NUMBER >= 0x10101000L >> + SSL_CTX_set_client_hello_cb(c->ssl->session_ctx, ngx_SSL_early_cb_fn, c); >> +#endif >> + >> n = SSL_do_handshake(c->ssl->connection); >> >> ngx_log_debug1(NGX_LOG_DEBUG_EVENT, c->log, 0, "SSL_do_handshake: %d", n); >> @@ -1623,6 +1721,10 @@ >> >> c->ssl->handshaked = 1; >> >> +#if OPENSSL_VERSION_NUMBER >= 0x10101000L >> + ngx_SSL_client_features(c); >> +#endif >> + >> c->recv = ngx_ssl_recv; >> c->send = ngx_ssl_write; >> c->recv_chain = ngx_ssl_recv_chain; >> @@ -5044,6 +5146,86 @@ >> return NGX_OK; >> } >> >> +#if OPENSSL_VERSION_NUMBER >= 0x10101000L >> + >> +ngx_int_t >> +ngx_ssl_get_extensions(ngx_connection_t *c, ngx_pool_t *pool, ngx_str_t *s) >> +{ >> + size_t len; >> + u_char *p; >> + >> + len = 0; >> + s->len = 0; >> + >> + if (c->ssl->extensions_size && c->ssl->extensions) { >> + for (int n = c->ssl->extensions[0]; n > 9; n /= 10) { >> + len += 1; >> + } >> + len += 1; >> + for (size_t i = 1; i < c->ssl->extensions_size; ++i) { >> + len += 1; /* for the '-' separator */ >> + for (int n = c->ssl->extensions[i]; n > 9; n /= 10) { >> + len += 1; >> + } >> + len += 1; >> + } >> + >> + s->data = ngx_pnalloc(pool, len+1); >> + if (s->data == NULL) { >> + return NGX_ERROR; >> + } >> + s->len = len; >> + >> + p = ngx_sprintf(s->data, "%d", c->ssl->extensions[0]); >> + for (size_t i = 1; i < c->ssl->extensions_size; ++i) { >> + p = ngx_sprintf(p, "-%d", c->ssl->extensions[i]); >> + } >> + >> + } >> + return NGX_OK; >> +} >> + >> + >> +ngx_int_t >> +ngx_ssl_get_ec_point_formats(ngx_connection_t *c, ngx_pool_t *pool, >> + ngx_str_t *s) >> +{ >> + size_t len; >> + u_char *p; >> + >> + len = 0; >> + s->len = 0; >> + >> + if (c->ssl->point_formats_sz && c->ssl->point_formats) { >> + for (unsigned char n = c->ssl->point_formats[0]; n > 9; n /= 10) { >> + len += 1; >> + } >> + len += 1; >> + for (size_t i = 1; i < c->ssl->point_formats_sz; ++i) { >> + len += 1; /* for the '-' separator */ >> + for (unsigned char n = c->ssl->point_formats[i]; n > 9; n /= 10) { >> + len += 1; >> + } >> + len += 1; >> + } >> + >> + s->data = ngx_pnalloc(pool, len+1); >> + if (s->data == NULL) { >> + return NGX_ERROR; >> + } >> + s->len = len; >> + >> + p = ngx_sprintf(s->data, "%d", c->ssl->point_formats[0]); >> + for (size_t i = 1; i < c->ssl->point_formats_sz; ++i) { >> + p = ngx_sprintf(p, "-%d", c->ssl->point_formats[i]); >> + } >> + } >> + >> + return NGX_OK; >> +} >> + >> +#endif >> + >> >> static time_t >> ngx_ssl_parse_time( >> diff -r 716eddd74bc2 -r 86d2f46807f5 src/event/ngx_event_openssl.h >> --- a/src/event/ngx_event_openssl.h Thu Apr 23 15:10:26 2020 +0300 >> +++ b/src/event/ngx_event_openssl.h Thu Apr 23 20:30:36 2020 -0500 >> @@ -99,6 +99,21 @@ >> unsigned in_early:1; >> unsigned early_preread:1; >> unsigned write_blocked:1; >> + >> +#if OPENSSL_VERSION_NUMBER >= 0x10101000L >> + >> + size_t ciphers_sz; >> + unsigned short *ciphers; >> + >> + size_t extensions_size; >> + int *extensions; >> + >> + size_t curves_sz; >> + unsigned short *curves; >> + >> + size_t point_formats_sz; >> + unsigned char *point_formats; >> +#endif >> }; >> >> >> @@ -263,6 +278,14 @@ >> ngx_int_t ngx_ssl_get_client_v_remain(ngx_connection_t *c, ngx_pool_t *pool, >> ngx_str_t *s); >> >> +#if OPENSSL_VERSION_NUMBER >= 0x10101000L >> + >> +ngx_int_t ngx_ssl_get_extensions(ngx_connection_t *c, ngx_pool_t *pool, >> + ngx_str_t *s); >> +ngx_int_t ngx_ssl_get_ec_point_formats(ngx_connection_t *c, ngx_pool_t *pool, >> + ngx_str_t *s); >> + >> +#endif >> >> ngx_int_t ngx_ssl_handshake(ngx_connection_t *c); >> ssize_t ngx_ssl_recv(ngx_connection_t *c, u_char *buf, size_t size); >> diff -r 716eddd74bc2 -r 86d2f46807f5 src/http/modules/ngx_http_ssl_module.c >> --- a/src/http/modules/ngx_http_ssl_module.c Thu Apr 23 15:10:26 2020 +0300 >> +++ b/src/http/modules/ngx_http_ssl_module.c Thu Apr 23 20:30:36 2020 -0500 >> @@ -352,6 +352,17 @@ >> { ngx_string("ssl_client_v_remain"), NULL, ngx_http_ssl_variable, >> (uintptr_t) ngx_ssl_get_client_v_remain, NGX_HTTP_VAR_CHANGEABLE, 0 }, >> >> +#if OPENSSL_VERSION_NUMBER >= 0x10101000L >> + >> + { ngx_string("ssl_extensions"), NULL, ngx_http_ssl_variable, >> + (uintptr_t) ngx_ssl_get_extensions, NGX_HTTP_VAR_CHANGEABLE, 0 }, >> + >> + { ngx_string("ssl_elliptic_curve_point_formats"), NULL, >> + ngx_http_ssl_variable, (uintptr_t) ngx_ssl_get_ec_point_formats, >> + NGX_HTTP_VAR_CHANGEABLE, 0 }, >> + >> +#endif >> + >> ngx_http_null_variable >> }; >> _______________________________________________ >> nginx-devel mailing list >> nginx-devel at nginx.org >> http://mailman.nginx.org/mailman/listinfo/nginx-devel >> >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From dnj0496 at gmail.com Sat Apr 25 22:26:47 2020 From: dnj0496 at gmail.com (Dk Jack) Date: Sat, 25 Apr 2020 15:26:47 -0700 Subject: http2. Message-ID: Hi, I am trying to understand how the modules work when dealing with http2 traffic? Trying to find any documentation or any other info that will tells me how a module should behave when handling http2 traffic. I am assuming the actual http2 protocol parsing is done by the http2 module. How can a module access the headers, params, body and other data in the request/response? Any info is greatly appreciated. Thanks. Dk. -------------- next part -------------- An HTML attachment was scrubbed... URL: From mdounin at mdounin.ru Sun Apr 26 01:12:40 2020 From: mdounin at mdounin.ru (Maxim Dounin) Date: Sun, 26 Apr 2020 04:12:40 +0300 Subject: [PATCH] Ensured SIGQUIT deletes listening UNIX socket files. In-Reply-To: <22c4b64a-eb4e-a8a8-7428-34ab3225fcf5@fastmail.com> References: <4b6bc48b-6c7a-7c76-51ae-1038421ed36c@fastmail.com> <20200227152429.GI12894@mdounin.ru> <2c06ad76-c309-d387-2c40-f6e60aacf4f8@fastmail.com> <20200303142811.GO12894@mdounin.ru> <22c4b64a-eb4e-a8a8-7428-34ab3225fcf5@fastmail.com> Message-ID: <20200426011240.GL20357@mdounin.ru> Hello! On Thu, Apr 16, 2020 at 04:51:04PM -0700, Thibault Charbonnier wrote: > On 3/3/20 6:28 AM, Maxim Dounin wrote: > > Checking for the oldpid file does not look like a reliable > > approach to me. > > Hi Maxim, > > For this patch to cover TEST 3 in my previously attached test suite > (cancel binary upgrade via SIGQUIT on the new binary), we need a way for > a new binary to know whether an old binary is still running or not. Sure. The problem is that checking this via a filesystem which can be arbitrary modified by external means is not reliable. Further, if the pid file was changed in the configuration along with binary upgrade, there will be no oldpid file available from the very start of the new binary. > Do you have another suggestion on how to achieve this? Checking for the > existence of nginx.oldpid seems acceptable to me in this case. A better approach might be to check parent's pid instead, much like we do when handling the changebin signal on unix (see src/os/unix/ngx_process.c). -- Maxim Dounin http://mdounin.ru/ From amdeich at gmail.com Mon Apr 27 11:16:37 2020 From: amdeich at gmail.com (Andrey Kulikov) Date: Mon, 27 Apr 2020 14:16:37 +0300 Subject: [PATCH] Add missing check for r->cache pointer validity in ngx_http_upstream_send_response() Message-ID: Hello, In ngx_http_upstream_send_response() function there are a lot of checks for r->cache pointer validity. But it is absent for if (u->cacheable){}, which starts with dereferencing valid = r->cache->valid_sec; straight away. If this considered to be an issue - patch attempting to fix it is attached. -- Best wishes, Andrey -------------- next part -------------- A non-text attachment was scrubbed... Name: 0001-Add-missing-check-for-r-cache-pointer-validity-in-ng.patch Type: application/octet-stream Size: 776 bytes Desc: not available URL: From luhliari at redhat.com Mon Apr 27 11:53:17 2020 From: luhliari at redhat.com (Lubos Uhliarik) Date: Mon, 27 Apr 2020 07:53:17 -0400 (EDT) Subject: [PATCH] Support loading server certificate from HW token Message-ID: <1045205432.24749495.1587988397159.JavaMail.zimbra@redhat.com> # HG changeset patch # User Lubos Uhliarik # Date 1587988141 -7200 # Mon Apr 27 13:49:01 2020 +0200 # Node ID 8fe8445769f77165f793a4fd016a134aa1ad373c # Parent 716eddd74bc2831537f5b3f7ecd16ad3e516d043 Support loading server certificate from HW token Nginx supports loading private key from HW token, but does not support loading certificate. This patch adds functionality which allows to load server certificate with a specified id from OpenSSL engine. diff -r 716eddd74bc2 -r 8fe8445769f7 src/event/ngx_event_openssl.c --- a/src/event/ngx_event_openssl.c Thu Apr 23 15:10:26 2020 +0300 +++ b/src/event/ngx_event_openssl.c Mon Apr 27 13:49:01 2020 +0200 @@ -609,6 +609,71 @@ X509 *x509, *temp; u_long n; + if (ngx_strncmp(cert->data, "engine:", sizeof("engine:") - 1) == 0) { + +#ifndef OPENSSL_NO_ENGINE + + u_char *p, *last; + ENGINE *engine; + + p = cert->data + sizeof("engine:") - 1; + last = (u_char *) ngx_strchr(p, ':'); + + if (last == NULL) { + *err = "invalid syntax"; + return NULL; + } + + *last = '\0'; + + engine = ENGINE_by_id((char *) p); + + if (engine == NULL) { + *err = "ENGINE_by_id() failed"; + return NULL; + } + + if (!ENGINE_init(engine)) { + *err = "ENGINE_init() failed"; + ENGINE_free(engine); + return NULL; + } + + *last++ = ':'; + + struct { + const char *cert_id; + X509 *cert; + } params = { (char *) last, NULL }; + + if (!ENGINE_ctrl_cmd(engine, "LOAD_CERT_CTRL", 0, ¶ms, NULL, 1)) { + *err = "ENGINE_ctrl_cmd() failed - Unable to get the certificate"; + ENGINE_free(engine); + return NULL; + } + + ENGINE_finish(engine); + ENGINE_free(engine); + + /* set chain to null */ + + *chain = sk_X509_new_null(); + if (*chain == NULL) { + *err = "sk_X509_new_null() failed"; + X509_free(params.cert); + return NULL; + } + + return params.cert; + +#else + + *err = "loading \"engine:...\" certificate is not supported"; + return NULL; + +#endif + } + if (ngx_strncmp(cert->data, "data:", sizeof("data:") - 1) == 0) { bio = BIO_new_mem_buf(cert->data + sizeof("data:") - 1, From mdounin at mdounin.ru Mon Apr 27 14:32:06 2020 From: mdounin at mdounin.ru (Maxim Dounin) Date: Mon, 27 Apr 2020 17:32:06 +0300 Subject: [PATCH] Add missing check for r->cache pointer validity in ngx_http_upstream_send_response() In-Reply-To: References: Message-ID: <20200427143206.GP20357@mdounin.ru> Hello! On Mon, Apr 27, 2020 at 02:16:37PM +0300, Andrey Kulikov wrote: > In ngx_http_upstream_send_response() function there are a lot of > checks for r->cache pointer validity. > But it is absent for if (u->cacheable){}, which starts with dereferencing > valid = r->cache->valid_sec; > straight away. > > If this considered to be an issue - patch attempting to fix it is attached. The u->cacheable is only expected to be set with r->cache, see ngx_http_upstream_cache(). If u->cacheable is set but r->cache is null, this indicates a bug elsewhere. -- Maxim Dounin http://mdounin.ru/ From thibaultcha at fastmail.com Mon Apr 27 23:26:31 2020 From: thibaultcha at fastmail.com (Thibault Charbonnier) Date: Mon, 27 Apr 2020 16:26:31 -0700 Subject: [PATCH] Ensured SIGQUIT deletes listening UNIX socket files. In-Reply-To: <20200426011240.GL20357@mdounin.ru> References: <4b6bc48b-6c7a-7c76-51ae-1038421ed36c@fastmail.com> <20200227152429.GI12894@mdounin.ru> <2c06ad76-c309-d387-2c40-f6e60aacf4f8@fastmail.com> <20200303142811.GO12894@mdounin.ru> <22c4b64a-eb4e-a8a8-7428-34ab3225fcf5@fastmail.com> <20200426011240.GL20357@mdounin.ru> Message-ID: <6d8edc45-6133-de52-8220-05c962a4a48c@fastmail.com> On 4/25/20 6:12 PM, Maxim Dounin wrote: > A better approach might be to check parent's pid instead, much > like we do when handling the changebin signal on unix (see > src/os/unix/ngx_process.c). Great! Thanks for the suggestion. Below is a revised approach for the patch (also attached to this email) which passes all of the test cases listed in my previous test file at the start of this thread: # HG changeset patch # User Thibault Charbonnier # Date 1582764433 28800 # Wed Feb 26 16:47:13 2020 -0800 # Node ID 8d781bac6c4feebb2d1ea3f4e6df76d71f74e43b # Parent 4f18393a1d51bce6103ea2f1b2587900f349ba3d Ensured SIGQUIT deletes listening UNIX socket files. Prior to this patch, the SIGQUIT signal handling (graceful shutdown) did not remove UNIX socket files since ngx_master_process_cycle reimplemented listening socket closings in lieu of using ngx_close_listening_sockets. Since ngx_master_process_exit will call the aforementioned ngx_close_listening_sockets, we can remove the custom implementation and now expect listening sockets to be closed properly by ngx_close_listening_sockets instead. This fixes the trac issue #753 (https://trac.nginx.org/nginx/ticket/753). diff -r 4f18393a1d51 -r 8d781bac6c4f src/core/ngx_connection.c --- a/src/core/ngx_connection.c Thu Feb 20 16:51:07 2020 +0300 +++ b/src/core/ngx_connection.c Wed Feb 26 16:47:13 2020 -0800 @@ -1070,7 +1070,8 @@ if (ls[i].sockaddr->sa_family == AF_UNIX && ngx_process <= NGX_PROCESS_MASTER - && ngx_new_binary == 0) + && ngx_new_binary == 0 + && ngx_getppid() != ngx_parent) { u_char *name = ls[i].addr_text.data + sizeof("unix:") - 1; diff -r 4f18393a1d51 -r 8d781bac6c4f src/os/unix/ngx_process_cycle.c --- a/src/os/unix/ngx_process_cycle.c Thu Feb 20 16:51:07 2020 +0300 +++ b/src/os/unix/ngx_process_cycle.c Wed Feb 26 16:47:13 2020 -0800 @@ -77,12 +77,11 @@ u_char *p; size_t size; ngx_int_t i; - ngx_uint_t n, sigio; + ngx_uint_t sigio; sigset_t set; struct itimerval itv; ngx_uint_t live; ngx_msec_t delay; - ngx_listening_t *ls; ngx_core_conf_t *ccf; sigemptyset(&set); @@ -205,16 +204,6 @@ ngx_signal_worker_processes(cycle, ngx_signal_value(NGX_SHUTDOWN_SIGNAL)); - ls = cycle->listening.elts; - for (n = 0; n < cycle->listening.nelts; n++) { - if (ngx_close_socket(ls[n].fd) == -1) { - ngx_log_error(NGX_LOG_EMERG, cycle->log, ngx_socket_errno, - ngx_close_socket_n " %V failed", - &ls[n].addr_text); - } - } - cycle->listening.nelts = 0; - continue; } -------------- next part -------------- A non-text attachment was scrubbed... Name: del-unix-sockets-on-sigquit.patch Type: text/x-patch Size: 2526 bytes Desc: not available URL: From amdeich at gmail.com Thu Apr 30 04:59:41 2020 From: amdeich at gmail.com (Andrey Kulikov) Date: Thu, 30 Apr 2020 07:59:41 +0300 Subject: Feature suggestion: Additional check for SSL misconfiguration in stream proxy. Message-ID: Hello, Consider following configuration: stream { server { listen 5443; proxy_pass my-tls-upstream:443; proxy_ssl_verify on; proxy_ssl_server_name on; proxy_ssl_trusted_certificate trusted_root_CAs.cer; } } # end stream It is perfectly Ok for nginx, though it doesn't do what one would expect it to - data being send to upstream server in plain text. This is due to the fact that proxy_ssl if off by default. So all proxy_ssl_* directives being ignored. This looks kind of error-prone, as unlike in HTTP-proxy module, we can't specify schema for upstream connections. Thus, one could expect nginx to complain about misconfiguration (using proxy_ssl_* without specifying proxy_ssl on; first), rather than silently send data in cleartext. If patch with additional checks implementation for stream-proxy module will be submitted, are there any chances it could be considered for merging into upstream? -- Best wishes, Andrey. From xeioex at nginx.com Thu Apr 30 09:43:03 2020 From: xeioex at nginx.com (Dmitry Volyntsev) Date: Thu, 30 Apr 2020 09:43:03 +0000 Subject: [njs] Version bump. Message-ID: details: https://hg.nginx.org/njs/rev/95fba9d4686f branches: changeset: 1382:95fba9d4686f user: Dmitry Volyntsev date: Wed Apr 29 11:57:10 2020 +0000 description: Version bump. diffstat: src/njs.h | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diffs (12 lines): diff -r ee191da6c973 -r 95fba9d4686f src/njs.h --- a/src/njs.h Thu Apr 23 11:30:25 2020 +0000 +++ b/src/njs.h Wed Apr 29 11:57:10 2020 +0000 @@ -11,7 +11,7 @@ #include -#define NJS_VERSION "0.4.0" +#define NJS_VERSION "0.4.1" #include /* STDOUT_FILENO, STDERR_FILENO */ From xeioex at nginx.com Thu Apr 30 09:43:05 2020 From: xeioex at nginx.com (Dmitry Volyntsev) Date: Thu, 30 Apr 2020 09:43:05 +0000 Subject: [njs] HTTP: improved getting of multi-valued headers in r.headersIn. Message-ID: details: https://hg.nginx.org/njs/rev/f35440bf511f branches: changeset: 1383:f35440bf511f user: Dmitry Volyntsev date: Wed Apr 29 11:57:11 2020 +0000 description: HTTP: improved getting of multi-valued headers in r.headersIn. 1) Duplicates of Content-Type, ETag, From, Max-Forwards, Referer, Proxy-Authorization, User-Agent are ignored. 2) Duplicate Cookie headers are joined together with ';'. 3) All other duplicate header values are joined together with ','. diffstat: nginx/ngx_http_js_module.c | 284 ++++++++++++++++++++++++++++---------------- 1 files changed, 180 insertions(+), 104 deletions(-) diffs (363 lines): diff -r 95fba9d4686f -r f35440bf511f nginx/ngx_http_js_module.c --- a/nginx/ngx_http_js_module.c Wed Apr 29 11:57:10 2020 +0000 +++ b/nginx/ngx_http_js_module.c Wed Apr 29 11:57:11 2020 +0000 @@ -136,6 +136,22 @@ static njs_int_t ngx_http_js_ext_get_req static njs_int_t ngx_http_js_ext_get_header_in(njs_vm_t *vm, njs_object_prop_t *prop, njs_value_t *value, njs_value_t *setval, njs_value_t *retval); +static njs_int_t ngx_http_js_header_in_cookie(njs_vm_t *vm, + ngx_http_request_t *r, njs_str_t *name, njs_value_t *setval, + njs_value_t *retval); +#if (NGX_HTTP_X_FORWARDED_FOR) +static njs_int_t ngx_http_js_header_in_x_forwarded_for(njs_vm_t *vm, + ngx_http_request_t *r, njs_str_t *name, njs_value_t *setval, + njs_value_t *retval); +#endif +static njs_int_t ngx_http_js_header_in_array(njs_vm_t *vm, + ngx_http_request_t *r, ngx_array_t *array, u_char sep, njs_value_t *retval); +static njs_int_t ngx_http_js_header_in_single(njs_vm_t *vm, + ngx_http_request_t *r, njs_str_t *name, njs_value_t *setval, + njs_value_t *retval); +static njs_int_t ngx_http_js_header_in_generic(njs_vm_t *vm, + ngx_http_request_t *r, njs_str_t *name, njs_value_t *setval, + njs_value_t *retval); static njs_int_t ngx_http_js_ext_keys_header_in(njs_vm_t *vm, njs_value_t *value, njs_value_t *keys); static njs_int_t ngx_http_js_ext_get_arg(njs_vm_t *vm, @@ -922,6 +938,75 @@ ngx_http_js_get_header(ngx_list_part_t * static njs_int_t +ngx_http_js_return_header(njs_vm_t *vm, ngx_http_request_t *r, + ngx_list_part_t *part, njs_str_t *name, njs_value_t *retval) +{ + size_t len; + u_char *data, *p, *start, *end; + ngx_uint_t i; + ngx_table_elt_t *header, *h; + + header = part->elts; + + p = NULL; + start = NULL; + end = NULL; + + for (i = 0; /* void */ ; i++) { + + if (i >= part->nelts) { + if (part->next == NULL) { + break; + } + + part = part->next; + header = part->elts; + i = 0; + } + + h = &header[i]; + + if (h->hash == 0 + || h->key.len != name->length + || ngx_strncasecmp(h->key.data, name->start, name->length) != 0) + { + continue; + } + + if (p == NULL) { + start = h->value.data; + end = h->value.data + h->value.len; + p = end; + continue; + } + + if (p + h->value.len + 1 > end) { + len = njs_max(p + h->value.len + 1 - start, 2 * (end - start)); + + data = ngx_pnalloc(r->pool, len); + if (data == NULL) { + return NJS_ERROR; + } + + p = ngx_cpymem(data, start, p - start); + start = data; + end = data + len; + } + + *p++ = ','; + p = ngx_cpymem(p, h->value.data, h->value.len); + } + + if (p == NULL) { + njs_value_undefined_set(retval); + return NJS_DECLINED; + } + + return njs_vm_value_string_set(vm, retval, start, p - start); +} + + +static njs_int_t ngx_http_js_ext_header_out(njs_vm_t *vm, njs_object_prop_t *prop, njs_value_t *value, njs_value_t *setval, njs_value_t *retval) { @@ -1147,8 +1232,7 @@ static njs_int_t ngx_http_js_header_out_generic(njs_vm_t *vm, ngx_http_request_t *r, njs_str_t *name, njs_value_t *setval, njs_value_t *retval) { - size_t len; - u_char *data, *p, *start, *end; + u_char *p; int64_t length; njs_value_t *array; njs_int_t rc; @@ -1163,63 +1247,7 @@ ngx_http_js_header_out_generic(njs_vm_t part = &headers->part; if (retval != NULL && setval == NULL) { - header = part->elts; - - p = NULL; - start = NULL; - end = NULL; - - for (i = 0; /* void */ ; i++) { - - if (i >= part->nelts) { - if (part->next == NULL) { - break; - } - - part = part->next; - header = part->elts; - i = 0; - } - - h = &header[i]; - - if (h->hash == 0 - || h->key.len != name->length - || ngx_strncasecmp(h->key.data, name->start, name->length) != 0) - { - continue; - } - - if (p == NULL) { - start = h->value.data; - end = h->value.data + h->value.len; - p = end; - continue; - } - - if (p + h->value.len + 1 > end) { - len = njs_max(p + h->value.len + 1 - start, 2 * (end - start)); - - data = ngx_pnalloc(r->pool, len); - if (data == NULL) { - return NJS_ERROR; - } - - p = ngx_cpymem(data, start, p - start); - start = data; - end = data + len; - } - - *p++ = ','; - p = ngx_cpymem(p, h->value.data, h->value.len); - } - - if (p == NULL) { - njs_value_undefined_set(retval); - return NJS_DECLINED; - } - - return njs_vm_value_string_set(vm, retval, start, p - start); + return ngx_http_js_return_header(vm, r, part, name, retval); } header = part->elts; @@ -1903,69 +1931,92 @@ static njs_int_t ngx_http_js_ext_get_header_in(njs_vm_t *vm, njs_object_prop_t *prop, njs_value_t *value, njs_value_t *setval, njs_value_t *retval) { - u_char *p, *end, sep; - size_t len; - njs_int_t rc; - njs_str_t *v, name; - ngx_uint_t i, n; - ngx_array_t *a; - ngx_table_elt_t *h, **hh; - ngx_http_request_t *r; + njs_int_t rc; + njs_str_t name; + ngx_http_request_t *r; + ngx_http_js_header_t *h; + + static ngx_http_js_header_t headers_in[] = { + { njs_str("Content-Type"), ngx_http_js_header_in_single }, + { njs_str("Cookie"), ngx_http_js_header_in_cookie }, + { njs_str("ETag"), ngx_http_js_header_in_single }, + { njs_str("From"), ngx_http_js_header_in_single }, + { njs_str("Max-Forwards"), ngx_http_js_header_in_single }, + { njs_str("Referer"), ngx_http_js_header_in_single }, + { njs_str("Proxy-Authorization"), ngx_http_js_header_in_single }, + { njs_str("User-Agent"), ngx_http_js_header_in_single }, +#if (NGX_HTTP_X_FORWARDED_FOR) + { njs_str("X-Forwarded-For"), ngx_http_js_header_in_x_forwarded_for }, +#endif + { njs_str(""), ngx_http_js_header_in_generic }, + }; r = njs_vm_external(vm, value); if (r == NULL) { - njs_value_undefined_set(retval); + if (retval != NULL) { + njs_value_undefined_set(retval); + } + return NJS_DECLINED; } rc = njs_vm_prop_name(vm, prop, &name); if (rc != NJS_OK) { - njs_value_undefined_set(retval); + if (retval != NULL) { + njs_value_undefined_set(retval); + } + return NJS_DECLINED; } - v = &name; - - if (v->length == njs_length("Cookie") - && ngx_strncasecmp(v->start, (u_char *) "Cookie", - v->length) == 0) - { - sep = ';'; - a = &r->headers_in.cookies; - goto multi; + for (h = headers_in; h->name.length > 0; h++) { + if (h->name.length == name.length + && ngx_strncasecmp(h->name.start, name.start, name.length) == 0) + { + break; + } } + return h->handler(vm, r, &name, setval, retval); +} + + +static njs_int_t +ngx_http_js_header_in_cookie(njs_vm_t *vm, ngx_http_request_t *r, + njs_str_t *name, njs_value_t *setval, njs_value_t *retval) +{ + return ngx_http_js_header_in_array(vm, r, &r->headers_in.cookies, + ';', retval); +} + + #if (NGX_HTTP_X_FORWARDED_FOR) - if (v->length == njs_length("X-Forwarded-For") - && ngx_strncasecmp(v->start, (u_char *) "X-Forwarded-For", - v->length) == 0) - { - sep = ','; - a = &r->headers_in.x_forwarded_for; - goto multi; - } +static njs_int_t +ngx_http_js_header_in_x_forwarded_for(njs_vm_t *vm, ngx_http_request_t *r, + njs_str_t *name, njs_value_t *setval, njs_value_t *retval) +{ + return ngx_http_js_header_in_array(vm, r, &r->headers_in.x_forwarded_for, + ',', retval); +} #endif - h = ngx_http_js_get_header(&r->headers_in.headers.part, v->start, - v->length); - if (h == NULL) { - njs_value_undefined_set(retval); - return NJS_DECLINED; - } - - return njs_vm_value_string_set(vm, retval, h->value.data, h->value.len); - -multi: - - /* Cookie, X-Forwarded-For */ - - n = a->nelts; - hh = a->elts; + +static njs_int_t +ngx_http_js_header_in_array(njs_vm_t *vm, ngx_http_request_t *r, + ngx_array_t *array, u_char sep, njs_value_t *retval) +{ + u_char *p, *end; + size_t len; + ngx_uint_t i, n; + ngx_table_elt_t **hh; + + n = array->nelts; + hh = array->elts; len = 0; for (i = 0; i < n; i++) { - len += hh[i]->value.len + 2; + len += hh[i]->value.len + 1; } if (len == 0) { @@ -1973,7 +2024,7 @@ multi: return NJS_DECLINED; } - len -= 2; + len -= 1; if (n == 1) { return njs_vm_value_string_set(vm, retval, (*hh)->value.data, @@ -1997,7 +2048,6 @@ multi: } *p++ = sep; - *p++ = ' '; } return NJS_OK; @@ -2005,6 +2055,32 @@ multi: static njs_int_t +ngx_http_js_header_in_single(njs_vm_t *vm, ngx_http_request_t *r, + njs_str_t *name, njs_value_t *setval, njs_value_t *retval) +{ + ngx_table_elt_t *h; + + h = ngx_http_js_get_header(&r->headers_in.headers.part, name->start, + name->length); + if (h == NULL) { + njs_value_undefined_set(retval); + return NJS_DECLINED; + } + + return njs_vm_value_string_set(vm, retval, h->value.data, h->value.len); +} + + +static njs_int_t +ngx_http_js_header_in_generic(njs_vm_t *vm, ngx_http_request_t *r, + njs_str_t *name, njs_value_t *setval, njs_value_t *retval) +{ + return ngx_http_js_return_header(vm, r, &r->headers_in.headers.part, name, + retval); +} + + +static njs_int_t ngx_http_js_ext_keys_header_in(njs_vm_t *vm, njs_value_t *value, njs_value_t *keys) { From xeioex at nginx.com Thu Apr 30 10:12:08 2020 From: xeioex at nginx.com (Dmitry Volyntsev) Date: Thu, 30 Apr 2020 10:12:08 +0000 Subject: [njs] HTTP: simplifed headers handling. Message-ID: details: https://hg.nginx.org/njs/rev/9e1caca2ee71 branches: changeset: 1384:9e1caca2ee71 user: Dmitry Volyntsev date: Thu Apr 30 09:44:00 2020 +0000 description: HTTP: simplifed headers handling. diffstat: nginx/ngx_http_js_module.c | 328 +++++++++++++++++++------------------------- 1 files changed, 144 insertions(+), 184 deletions(-) diffs (512 lines): diff -r f35440bf511f -r 9e1caca2ee71 nginx/ngx_http_js_module.c --- a/nginx/ngx_http_js_module.c Wed Apr 29 11:57:11 2020 +0000 +++ b/nginx/ngx_http_js_module.c Thu Apr 30 09:44:00 2020 +0000 @@ -60,8 +60,8 @@ typedef struct { typedef struct { njs_str_t name; njs_int_t (*handler)(njs_vm_t *vm, ngx_http_request_t *r, - njs_str_t *name, njs_value_t *setval, - njs_value_t *retval); + ngx_list_t *headers, njs_str_t *name, + njs_value_t *setval, njs_value_t *retval); } ngx_http_js_header_t; @@ -87,25 +87,27 @@ static ngx_table_elt_t *ngx_http_js_get_ static njs_int_t ngx_http_js_ext_header_out(njs_vm_t *vm, njs_object_prop_t *prop, njs_value_t *value, njs_value_t *setval, njs_value_t *retval); -static njs_int_t ngx_http_js_header_out_single(njs_vm_t *vm, - ngx_http_request_t *r, njs_str_t *v, njs_value_t *setval, - njs_value_t *retval); +static njs_int_t ngx_http_js_header_single(njs_vm_t *vm, + ngx_http_request_t *r, ngx_list_t *headers, njs_str_t *name, + njs_value_t *setval, njs_value_t *retval); static njs_int_t ngx_http_js_header_out_special(njs_vm_t *vm, ngx_http_request_t *r, njs_str_t *v, njs_value_t *setval, njs_value_t *retval, ngx_table_elt_t **hh); -static njs_int_t ngx_http_js_header_out_array(njs_vm_t *vm, - ngx_http_request_t *r, njs_str_t *v, njs_value_t *setval, - njs_value_t *retval); -static njs_int_t ngx_http_js_header_out_generic(njs_vm_t *vm, - ngx_http_request_t *r, njs_str_t *v, njs_value_t *setval, - njs_value_t *retval); +static njs_int_t ngx_http_js_header_array(njs_vm_t *vm, + ngx_http_request_t *r, ngx_list_t *headers, njs_str_t *name, + njs_value_t *setval, njs_value_t *retval); +static njs_int_t ngx_http_js_header_generic(njs_vm_t *vm, + ngx_http_request_t *r, ngx_list_t *headers, njs_str_t *name, + njs_value_t *setval, njs_value_t *retval); static njs_int_t ngx_http_js_content_length(njs_vm_t *vm, ngx_http_request_t *r, - njs_str_t *name, njs_value_t *setval, njs_value_t *retval); + ngx_list_t *headers, njs_str_t *name, njs_value_t *setval, + njs_value_t *retval); static njs_int_t ngx_http_js_content_encoding(njs_vm_t *vm, - ngx_http_request_t *r, njs_str_t *name, njs_value_t *setval, + ngx_http_request_t *r, ngx_list_t *headers, njs_str_t *name, + njs_value_t *setval, njs_value_t *retval); +static njs_int_t ngx_http_js_content_type(njs_vm_t *vm, ngx_http_request_t *r, + ngx_list_t *headers, njs_str_t *name, njs_value_t *setval, njs_value_t *retval); -static njs_int_t ngx_http_js_content_type(njs_vm_t *vm, ngx_http_request_t *r, - njs_str_t *name, njs_value_t *setval, njs_value_t *retval); static njs_int_t ngx_http_js_ext_keys_header_out(njs_vm_t *vm, njs_value_t *value, njs_value_t *keys); static njs_int_t ngx_http_js_ext_status(njs_vm_t *vm, njs_object_prop_t *prop, @@ -133,25 +135,19 @@ static njs_int_t ngx_http_js_ext_get_rem static njs_int_t ngx_http_js_ext_get_request_body(njs_vm_t *vm, njs_object_prop_t *prop, njs_value_t *value, njs_value_t *setval, njs_value_t *retval); -static njs_int_t ngx_http_js_ext_get_header_in(njs_vm_t *vm, +static njs_int_t ngx_http_js_ext_header_in(njs_vm_t *vm, njs_object_prop_t *prop, njs_value_t *value, njs_value_t *setval, njs_value_t *retval); -static njs_int_t ngx_http_js_header_in_cookie(njs_vm_t *vm, - ngx_http_request_t *r, njs_str_t *name, njs_value_t *setval, - njs_value_t *retval); +static njs_int_t ngx_http_js_header_cookie(njs_vm_t *vm, + ngx_http_request_t *r, ngx_list_t *headers, njs_str_t *name, + njs_value_t *setval, njs_value_t *retval); #if (NGX_HTTP_X_FORWARDED_FOR) -static njs_int_t ngx_http_js_header_in_x_forwarded_for(njs_vm_t *vm, - ngx_http_request_t *r, njs_str_t *name, njs_value_t *setval, - njs_value_t *retval); +static njs_int_t ngx_http_js_header_x_forwarded_for(njs_vm_t *vm, + ngx_http_request_t *r, ngx_list_t *headers, njs_str_t *name, + njs_value_t *setval, njs_value_t *retval); #endif static njs_int_t ngx_http_js_header_in_array(njs_vm_t *vm, ngx_http_request_t *r, ngx_array_t *array, u_char sep, njs_value_t *retval); -static njs_int_t ngx_http_js_header_in_single(njs_vm_t *vm, - ngx_http_request_t *r, njs_str_t *name, njs_value_t *setval, - njs_value_t *retval); -static njs_int_t ngx_http_js_header_in_generic(njs_vm_t *vm, - ngx_http_request_t *r, njs_str_t *name, njs_value_t *setval, - njs_value_t *retval); static njs_int_t ngx_http_js_ext_keys_header_in(njs_vm_t *vm, njs_value_t *value, njs_value_t *keys); static njs_int_t ngx_http_js_ext_get_arg(njs_vm_t *vm, @@ -352,7 +348,7 @@ static njs_external_t ngx_http_js_ext_r .enumerable = 1, .u.object = { .enumerable = 1, - .prop_handler = ngx_http_js_ext_get_header_in, + .prop_handler = ngx_http_js_ext_header_in, .keys = ngx_http_js_ext_keys_header_in, } }, @@ -938,75 +934,6 @@ ngx_http_js_get_header(ngx_list_part_t * static njs_int_t -ngx_http_js_return_header(njs_vm_t *vm, ngx_http_request_t *r, - ngx_list_part_t *part, njs_str_t *name, njs_value_t *retval) -{ - size_t len; - u_char *data, *p, *start, *end; - ngx_uint_t i; - ngx_table_elt_t *header, *h; - - header = part->elts; - - p = NULL; - start = NULL; - end = NULL; - - for (i = 0; /* void */ ; i++) { - - if (i >= part->nelts) { - if (part->next == NULL) { - break; - } - - part = part->next; - header = part->elts; - i = 0; - } - - h = &header[i]; - - if (h->hash == 0 - || h->key.len != name->length - || ngx_strncasecmp(h->key.data, name->start, name->length) != 0) - { - continue; - } - - if (p == NULL) { - start = h->value.data; - end = h->value.data + h->value.len; - p = end; - continue; - } - - if (p + h->value.len + 1 > end) { - len = njs_max(p + h->value.len + 1 - start, 2 * (end - start)); - - data = ngx_pnalloc(r->pool, len); - if (data == NULL) { - return NJS_ERROR; - } - - p = ngx_cpymem(data, start, p - start); - start = data; - end = data + len; - } - - *p++ = ','; - p = ngx_cpymem(p, h->value.data, h->value.len); - } - - if (p == NULL) { - njs_value_undefined_set(retval); - return NJS_DECLINED; - } - - return njs_vm_value_string_set(vm, retval, start, p - start); -} - - -static njs_int_t ngx_http_js_ext_header_out(njs_vm_t *vm, njs_object_prop_t *prop, njs_value_t *value, njs_value_t *setval, njs_value_t *retval) { @@ -1016,17 +943,17 @@ ngx_http_js_ext_header_out(njs_vm_t *vm, ngx_http_js_header_t *h; static ngx_http_js_header_t headers_out[] = { - { njs_str("Age"), ngx_http_js_header_out_single }, + { njs_str("Age"), ngx_http_js_header_single }, { njs_str("Content-Type"), ngx_http_js_content_type }, { njs_str("Content-Length"), ngx_http_js_content_length }, { njs_str("Content-Encoding"), ngx_http_js_content_encoding }, - { njs_str("Etag"), ngx_http_js_header_out_single }, - { njs_str("Expires"), ngx_http_js_header_out_single }, - { njs_str("Last-Modified"), ngx_http_js_header_out_single }, - { njs_str("Location"), ngx_http_js_header_out_single }, - { njs_str("Set-Cookie"), ngx_http_js_header_out_array }, - { njs_str("Retry-After"), ngx_http_js_header_out_single }, - { njs_str(""), ngx_http_js_header_out_generic }, + { njs_str("Etag"), ngx_http_js_header_single }, + { njs_str("Expires"), ngx_http_js_header_single }, + { njs_str("Last-Modified"), ngx_http_js_header_single }, + { njs_str("Location"), ngx_http_js_header_single }, + { njs_str("Set-Cookie"), ngx_http_js_header_array }, + { njs_str("Retry-After"), ngx_http_js_header_single }, + { njs_str(""), ngx_http_js_header_generic }, }; r = njs_vm_external(vm, value); @@ -1055,20 +982,34 @@ ngx_http_js_ext_header_out(njs_vm_t *vm, } } - return h->handler(vm, r, &name, setval, retval); + return h->handler(vm, r, &r->headers_out.headers, &name, setval, retval); } static njs_int_t -ngx_http_js_header_out_single(njs_vm_t *vm, ngx_http_request_t *r, - njs_str_t *name, njs_value_t *setval, njs_value_t *retval) +ngx_http_js_header_single(njs_vm_t *vm, ngx_http_request_t *r, + ngx_list_t *headers, njs_str_t *name, njs_value_t *setval, + njs_value_t *retval) { + njs_int_t rc; + ngx_table_elt_t *h; + if (retval != NULL && setval == NULL) { - return ngx_http_js_header_out_special(vm, r, name, setval, retval, - NULL); + h = ngx_http_js_get_header(&headers->part, name->start, name->length); + if (h == NULL) { + njs_value_undefined_set(retval); + return NJS_DECLINED; + } + + rc = njs_vm_value_string_set(vm, retval, h->value.data, h->value.len); + if (rc != NJS_OK) { + return NJS_ERROR; + } + + return NJS_OK; } - return ngx_http_js_header_out_generic(vm, r, name, setval, retval); + return ngx_http_js_header_generic(vm, r, headers, name, setval, retval); } @@ -1088,18 +1029,7 @@ ngx_http_js_header_out_special(njs_vm_t headers = &r->headers_out.headers; if (retval != NULL && setval == NULL) { - h = ngx_http_js_get_header(&headers->part, v->start, v->length); - if (h == NULL) { - njs_value_undefined_set(retval); - return NJS_DECLINED; - } - - rc = njs_vm_value_string_set(vm, retval, h->value.data, h->value.len); - if (rc != NJS_OK) { - return NJS_ERROR; - } - - return NJS_OK; + return ngx_http_js_header_single(vm, r, headers, v, setval, retval); } if (setval != NULL && njs_value_is_array(setval)) { @@ -1162,20 +1092,18 @@ ngx_http_js_header_out_special(njs_vm_t static njs_int_t -ngx_http_js_header_out_array(njs_vm_t *vm, ngx_http_request_t *r, - njs_str_t *name, njs_value_t *setval, njs_value_t *retval) +ngx_http_js_header_array(njs_vm_t *vm, ngx_http_request_t *r, + ngx_list_t *headers, njs_str_t *name, njs_value_t *setval, + njs_value_t *retval) { size_t len; u_char *data; njs_int_t rc; ngx_uint_t i; - ngx_list_t *headers; njs_value_t *value; ngx_list_part_t *part; ngx_table_elt_t *header, *h; - headers = &r->headers_out.headers; - if (retval != NULL && setval == NULL) { rc = njs_vm_array_alloc(vm, retval, 4); if (rc != NJS_OK) { @@ -1224,30 +1152,86 @@ ngx_http_js_header_out_array(njs_vm_t *v return NJS_OK; } - return ngx_http_js_header_out_generic(vm, r, name, setval, retval); + return ngx_http_js_header_generic(vm, r, headers, name, setval, retval); } static njs_int_t -ngx_http_js_header_out_generic(njs_vm_t *vm, ngx_http_request_t *r, - njs_str_t *name, njs_value_t *setval, njs_value_t *retval) +ngx_http_js_header_generic(njs_vm_t *vm, ngx_http_request_t *r, + ngx_list_t *headers, njs_str_t *name, njs_value_t *setval, + njs_value_t *retval) { - u_char *p; + u_char *data, *p, *start, *end; + size_t len; int64_t length; njs_value_t *array; njs_int_t rc; njs_str_t s; - ngx_list_t *headers; ngx_uint_t i; ngx_list_part_t *part; ngx_table_elt_t *header, *h; njs_opaque_value_t lvalue; - headers = &r->headers_out.headers; part = &headers->part; if (retval != NULL && setval == NULL) { - return ngx_http_js_return_header(vm, r, part, name, retval); + header = part->elts; + + p = NULL; + start = NULL; + end = NULL; + + for (i = 0; /* void */ ; i++) { + + if (i >= part->nelts) { + if (part->next == NULL) { + break; + } + + part = part->next; + header = part->elts; + i = 0; + } + + h = &header[i]; + + if (h->hash == 0 + || h->key.len != name->length + || ngx_strncasecmp(h->key.data, name->start, name->length) != 0) + { + continue; + } + + if (p == NULL) { + start = h->value.data; + end = h->value.data + h->value.len; + p = end; + continue; + } + + if (p + h->value.len + 1 > end) { + len = njs_max(p + h->value.len + 1 - start, 2 * (end - start)); + + data = ngx_pnalloc(r->pool, len); + if (data == NULL) { + return NJS_ERROR; + } + + p = ngx_cpymem(data, start, p - start); + start = data; + end = data + len; + } + + *p++ = ','; + p = ngx_cpymem(p, h->value.data, h->value.len); + } + + if (p == NULL) { + njs_value_undefined_set(retval); + return NJS_DECLINED; + } + + return njs_vm_value_string_set(vm, retval, start, p - start); } header = part->elts; @@ -1345,8 +1329,8 @@ ngx_http_js_header_out_generic(njs_vm_t static njs_int_t -ngx_http_js_content_encoding(njs_vm_t *vm, ngx_http_request_t *r, njs_str_t *v, - njs_value_t *setval, njs_value_t *retval) +ngx_http_js_content_encoding(njs_vm_t *vm, ngx_http_request_t *r, + ngx_list_t *headers, njs_str_t *v, njs_value_t *setval, njs_value_t *retval) { njs_int_t rc; ngx_table_elt_t *h; @@ -1365,8 +1349,8 @@ ngx_http_js_content_encoding(njs_vm_t *v static njs_int_t -ngx_http_js_content_length(njs_vm_t *vm, ngx_http_request_t *r, njs_str_t *v, - njs_value_t *setval, njs_value_t *retval) +ngx_http_js_content_length(njs_vm_t *vm, ngx_http_request_t *r, + ngx_list_t *headers, njs_str_t *v, njs_value_t *setval, njs_value_t *retval) { u_char *p, *start; njs_int_t rc; @@ -1419,8 +1403,8 @@ ngx_http_js_content_length(njs_vm_t *vm, static njs_int_t -ngx_http_js_content_type(njs_vm_t *vm, ngx_http_request_t *r, njs_str_t *v, - njs_value_t *setval, njs_value_t *retval) +ngx_http_js_content_type(njs_vm_t *vm, ngx_http_request_t *r, + ngx_list_t *headers, njs_str_t *v, njs_value_t *setval, njs_value_t *retval) { int64_t length; njs_int_t rc; @@ -1928,7 +1912,7 @@ done: static njs_int_t -ngx_http_js_ext_get_header_in(njs_vm_t *vm, njs_object_prop_t *prop, +ngx_http_js_ext_header_in(njs_vm_t *vm, njs_object_prop_t *prop, njs_value_t *value, njs_value_t *setval, njs_value_t *retval) { njs_int_t rc; @@ -1937,18 +1921,18 @@ ngx_http_js_ext_get_header_in(njs_vm_t * ngx_http_js_header_t *h; static ngx_http_js_header_t headers_in[] = { - { njs_str("Content-Type"), ngx_http_js_header_in_single }, - { njs_str("Cookie"), ngx_http_js_header_in_cookie }, - { njs_str("ETag"), ngx_http_js_header_in_single }, - { njs_str("From"), ngx_http_js_header_in_single }, - { njs_str("Max-Forwards"), ngx_http_js_header_in_single }, - { njs_str("Referer"), ngx_http_js_header_in_single }, - { njs_str("Proxy-Authorization"), ngx_http_js_header_in_single }, - { njs_str("User-Agent"), ngx_http_js_header_in_single }, + { njs_str("Content-Type"), ngx_http_js_header_single }, + { njs_str("Cookie"), ngx_http_js_header_cookie }, + { njs_str("ETag"), ngx_http_js_header_single }, + { njs_str("From"), ngx_http_js_header_single }, + { njs_str("Max-Forwards"), ngx_http_js_header_single }, + { njs_str("Referer"), ngx_http_js_header_single }, + { njs_str("Proxy-Authorization"), ngx_http_js_header_single }, + { njs_str("User-Agent"), ngx_http_js_header_single }, #if (NGX_HTTP_X_FORWARDED_FOR) - { njs_str("X-Forwarded-For"), ngx_http_js_header_in_x_forwarded_for }, + { njs_str("X-Forwarded-For"), ngx_http_js_header_x_forwarded_for }, #endif - { njs_str(""), ngx_http_js_header_in_generic }, + { njs_str(""), ngx_http_js_header_generic }, }; r = njs_vm_external(vm, value); @@ -1977,13 +1961,14 @@ ngx_http_js_ext_get_header_in(njs_vm_t * } } - return h->handler(vm, r, &name, setval, retval); + return h->handler(vm, r, &r->headers_in.headers, &name, setval, retval); } static njs_int_t -ngx_http_js_header_in_cookie(njs_vm_t *vm, ngx_http_request_t *r, - njs_str_t *name, njs_value_t *setval, njs_value_t *retval) +ngx_http_js_header_cookie(njs_vm_t *vm, ngx_http_request_t *r, + ngx_list_t *headers, njs_str_t *name, njs_value_t *setval, + njs_value_t *retval) { return ngx_http_js_header_in_array(vm, r, &r->headers_in.cookies, ';', retval); @@ -1992,8 +1977,9 @@ ngx_http_js_header_in_cookie(njs_vm_t *v #if (NGX_HTTP_X_FORWARDED_FOR) static njs_int_t -ngx_http_js_header_in_x_forwarded_for(njs_vm_t *vm, ngx_http_request_t *r, - njs_str_t *name, njs_value_t *setval, njs_value_t *retval) +ngx_http_js_header_x_forwarded_for(njs_vm_t *vm, ngx_http_request_t *r, + ngx_list_t *headers, njs_str_t *name, njs_value_t *setval, + njs_value_t *retval) { return ngx_http_js_header_in_array(vm, r, &r->headers_in.x_forwarded_for, ',', retval); @@ -2055,32 +2041,6 @@ ngx_http_js_header_in_array(njs_vm_t *vm static njs_int_t -ngx_http_js_header_in_single(njs_vm_t *vm, ngx_http_request_t *r, - njs_str_t *name, njs_value_t *setval, njs_value_t *retval) -{ - ngx_table_elt_t *h; - - h = ngx_http_js_get_header(&r->headers_in.headers.part, name->start, - name->length); - if (h == NULL) { - njs_value_undefined_set(retval); - return NJS_DECLINED; - } - - return njs_vm_value_string_set(vm, retval, h->value.data, h->value.len); -} - - -static njs_int_t -ngx_http_js_header_in_generic(njs_vm_t *vm, ngx_http_request_t *r, - njs_str_t *name, njs_value_t *setval, njs_value_t *retval) -{ - return ngx_http_js_return_header(vm, r, &r->headers_in.headers.part, name, - retval); -} - - -static njs_int_t ngx_http_js_ext_keys_header_in(njs_vm_t *vm, njs_value_t *value, njs_value_t *keys) { From xeioex at nginx.com Thu Apr 30 10:12:11 2020 From: xeioex at nginx.com (Dmitry Volyntsev) Date: Thu, 30 Apr 2020 10:12:11 +0000 Subject: [njs] Modules: setting retval to undefined for functions returning nothing. Message-ID: details: https://hg.nginx.org/njs/rev/d47d99d15d2b branches: changeset: 1385:d47d99d15d2b user: Dmitry Volyntsev date: Thu Apr 30 10:11:33 2020 +0000 description: Modules: setting retval to undefined for functions returning nothing. This fixes #305 issue on Github. diffstat: nginx/ngx_http_js_module.c | 12 ++++++++++++ nginx/ngx_stream_js_module.c | 10 ++++++++++ 2 files changed, 22 insertions(+), 0 deletions(-) diffs (105 lines): diff -r 9e1caca2ee71 -r d47d99d15d2b nginx/ngx_http_js_module.c --- a/nginx/ngx_http_js_module.c Thu Apr 30 09:44:00 2020 +0000 +++ b/nginx/ngx_http_js_module.c Thu Apr 30 10:11:33 2020 +0000 @@ -1546,6 +1546,8 @@ ngx_http_js_ext_send_header(njs_vm_t *vm return NJS_ERROR; } + njs_value_undefined_set(njs_vm_retval(vm)); + return NJS_OK; } @@ -1621,6 +1623,8 @@ ngx_http_js_ext_send(njs_vm_t *vm, njs_v return NJS_ERROR; } + njs_value_undefined_set(njs_vm_retval(vm)); + return NJS_OK; } @@ -1646,6 +1650,8 @@ ngx_http_js_ext_finish(njs_vm_t *vm, njs ctx->status = NGX_OK; + njs_value_undefined_set(njs_vm_retval(vm)); + return NJS_OK; } @@ -1704,6 +1710,8 @@ ngx_http_js_ext_return(njs_vm_t *vm, njs ctx->status = status; } + njs_value_undefined_set(njs_vm_retval(vm)); + return NJS_OK; } @@ -1739,6 +1747,8 @@ ngx_http_js_ext_internal_redirect(njs_vm ctx->status = NGX_DONE; + njs_value_undefined_set(njs_vm_retval(vm)); + return NJS_OK; } @@ -1772,6 +1782,8 @@ ngx_http_js_ext_log(njs_vm_t *vm, njs_va c->log->handler = handler; + njs_value_undefined_set(njs_vm_retval(vm)); + return NJS_OK; } diff -r 9e1caca2ee71 -r d47d99d15d2b nginx/ngx_stream_js_module.c --- a/nginx/ngx_stream_js_module.c Thu Apr 30 09:44:00 2020 +0000 +++ b/nginx/ngx_stream_js_module.c Thu Apr 30 10:11:33 2020 +0000 @@ -956,6 +956,8 @@ ngx_stream_js_ext_done(njs_vm_t *vm, njs ctx->download_event = NULL; } + njs_value_undefined_set(njs_vm_retval(vm)); + return NJS_OK; } @@ -990,6 +992,8 @@ ngx_stream_js_ext_log(njs_vm_t *vm, njs_ c->log->handler = handler; + njs_value_undefined_set(njs_vm_retval(vm)); + return NJS_OK; } @@ -1038,6 +1042,8 @@ ngx_stream_js_ext_on(njs_vm_t *vm, njs_v return NJS_ERROR; } + njs_value_undefined_set(njs_vm_retval(vm)); + return NJS_OK; } @@ -1072,6 +1078,8 @@ ngx_stream_js_ext_off(njs_vm_t *vm, njs_ *event = NULL; + njs_value_undefined_set(njs_vm_retval(vm)); + return NJS_OK; } @@ -1153,6 +1161,8 @@ ngx_stream_js_ext_send(njs_vm_t *vm, njs *ctx->last_out = cl; ctx->last_out = &cl->next; + njs_value_undefined_set(njs_vm_retval(vm)); + return NJS_OK; } From luhliari at redhat.com Thu Apr 30 11:55:54 2020 From: luhliari at redhat.com (Lubos Uhliarik) Date: Thu, 30 Apr 2020 07:55:54 -0400 (EDT) Subject: [PATCH] Support loading server certificate from HW token In-Reply-To: <1045205432.24749495.1587988397159.JavaMail.zimbra@redhat.com> References: <1045205432.24749495.1587988397159.JavaMail.zimbra@redhat.com> Message-ID: <170259234.25242098.1588247754550.JavaMail.zimbra@redhat.com> Hello everyone, do you see any change getting this patch applied to 1.19? Best, ----- Original Message ----- > From: "Lubos Uhliarik" > To: nginx-devel at nginx.org > Sent: Monday, April 27, 2020 1:53:17 PM > Subject: [PATCH] Support loading server certificate from HW token > > # HG changeset patch > # User Lubos Uhliarik > # Date 1587988141 -7200 > # Mon Apr 27 13:49:01 2020 +0200 > # Node ID 8fe8445769f77165f793a4fd016a134aa1ad373c > # Parent 716eddd74bc2831537f5b3f7ecd16ad3e516d043 > Support loading server certificate from HW token > > Nginx supports loading private key from HW token, but does not support > loading certificate. This patch adds functionality which allows to load > server certificate with a specified id from OpenSSL engine. > > diff -r 716eddd74bc2 -r 8fe8445769f7 src/event/ngx_event_openssl.c > --- a/src/event/ngx_event_openssl.c Thu Apr 23 15:10:26 2020 +0300 > +++ b/src/event/ngx_event_openssl.c Mon Apr 27 13:49:01 2020 +0200 > @@ -609,6 +609,71 @@ > X509 *x509, *temp; > u_long n; > > + if (ngx_strncmp(cert->data, "engine:", sizeof("engine:") - 1) == 0) { > + > +#ifndef OPENSSL_NO_ENGINE > + > + u_char *p, *last; > + ENGINE *engine; > + > + p = cert->data + sizeof("engine:") - 1; > + last = (u_char *) ngx_strchr(p, ':'); > + > + if (last == NULL) { > + *err = "invalid syntax"; > + return NULL; > + } > + > + *last = '\0'; > + > + engine = ENGINE_by_id((char *) p); > + > + if (engine == NULL) { > + *err = "ENGINE_by_id() failed"; > + return NULL; > + } > + > + if (!ENGINE_init(engine)) { > + *err = "ENGINE_init() failed"; > + ENGINE_free(engine); > + return NULL; > + } > + > + *last++ = ':'; > + > + struct { > + const char *cert_id; > + X509 *cert; > + } params = { (char *) last, NULL }; > + > + if (!ENGINE_ctrl_cmd(engine, "LOAD_CERT_CTRL", 0, ¶ms, NULL, 1)) > { > + *err = "ENGINE_ctrl_cmd() failed - Unable to get the > certificate"; > + ENGINE_free(engine); > + return NULL; > + } > + > + ENGINE_finish(engine); > + ENGINE_free(engine); > + > + /* set chain to null */ > + > + *chain = sk_X509_new_null(); > + if (*chain == NULL) { > + *err = "sk_X509_new_null() failed"; > + X509_free(params.cert); > + return NULL; > + } > + > + return params.cert; > + > +#else > + > + *err = "loading \"engine:...\" certificate is not supported"; > + return NULL; > + > +#endif > + } > + > if (ngx_strncmp(cert->data, "data:", sizeof("data:") - 1) == 0) { > > bio = BIO_new_mem_buf(cert->data + sizeof("data:") - 1, > > _______________________________________________ > nginx-devel mailing list > nginx-devel at nginx.org > http://mailman.nginx.org/mailman/listinfo/nginx-devel > > -- Lubos Uhliarik Software Engineer - EMEA ENG Developer Experience RH - Brno - TPB-C - 1D221 IRC: zero_byte at irc.freenode.net RED HAT | TRIED. TESTED. TRUSTED. Every airline in the Fortune 500 relies on Red Hat. Find out why at http://www.redhat.com/en/about/trusted Red Hat Inc. http://cz.redhat.com