nginx-devel June 2014

nginx-devel@nginx.org

25 participants
54 discussions

[PATCH] Mail: send starttls flag value to auth script

by Filipe da Silva

# HG changeset patch # User Filipe da Silva <fdasilvayy(a)gmail.com> # Date 1394099468 -3600 # Thu Mar 06 10:51:08 2014 +0100 # Node ID 51fd90f96449c23af0076a19efbfdb1f88702125 # Parent 24df9fa5868957c1fb9a2d1569271e0958327dad Mail: send starttls flag value to auth script. Allow to do logging (if logging takes place in the auth script) and or force some users to use STARTTLS while others can use unencrypted connection. diff -r 24df9fa58689 -r 51fd90f96449 src/mail/ngx_mail_auth_http_module.c --- a/src/mail/ngx_mail_auth_http_module.c Thu Mar 06 10:51:08 2014 +0100 +++ b/src/mail/ngx_mail_auth_http_module.c Thu Mar 06 10:51:08 2014 +0100 @@ -1165,6 +1165,9 @@ ngx_mail_auth_http_create_request(ngx_ma + sizeof("Auth-Salt: ") - 1 + s->salt.len + sizeof("Auth-Protocol: ") - 1 + cscf->protocol->name.len + sizeof(CRLF) - 1 +#if (NGX_MAIL_SSL) + + sizeof("Auth-STARTTLS: ") - 1 + 1 + sizeof(CRLF) - 1 +#endif + sizeof("Auth-Login-Attempt: ") - 1 + NGX_INT_T_LEN + sizeof(CRLF) - 1 + sizeof("Client-IP: ") - 1 + s->connection->addr_text.len @@ -1219,6 +1222,13 @@ ngx_mail_auth_http_create_request(ngx_ma cscf->protocol->name.len); *b->last++ = CR; *b->last++ = LF; +#if (NGX_MAIL_SSL) + b->last = ngx_cpymem(b->last, "Auth-STARTTLS: ", + sizeof("Auth-STARTTLS: ") - 1); + *b->last++ = s->starttls ? '1' : '0' ; + *b->last++ = CR; *b->last++ = LF; +#endif + b->last = ngx_sprintf(b->last, "Auth-Login-Attempt: %ui" CRLF, s->login_attempt);

7 years, 8 months

handle NGX_AGAIN properly

by Julien Zefi

hi, i understand that NGX_AGAIN is returned when a chain could not be send because more data cannot be buffered on that socket. I need to understand the following: in my case, when i receive a request, i start a timer every 10ms and send out some data, then i create a new timer every10ms until i decide to finish sending out data (video frames). But if in some triggered callback by the timer the ngx_http_output_filter(..) returns NGX_AGAIN *i assume* NginX will send that chain as soon as the socket becomes available again. But after that happens, how can i restore my timer cycle ? thnks. J.Z.

7 years, 11 months

[PATCH] Request hang when cache_lock is used in subrequests

by Yichun Zhang (agentzh)

Hello! Akos Gyimesi reported a request hang (downstream connections stuck in the CLOSE_WAIT state forever) regarding use of proxy_cache_lock in subrequests. The issue is that when proxy_cache_lock_timeout is reached, ngx_http_file_cache_lock_wait_handler calls r->connection->write->handler() directly, but r->connection->write->handler is (usually) just ngx_http_request_handler, which simply picks up r->connection->data, which is *not* necessarily the current (sub)request, so the current subrequest may never be continued nor finalized, leading to an infinite request hang. The following patch fixes this issue for me. Comments welcome! Thanks! -agentzh --- nginx-1.4.3/src/http/ngx_http_file_cache.c 2013-10-08 05:07:14.000000000 -0700 +++ nginx-1.4.3-patched/src/http/ngx_http_file_cache.c 2013-10-26 14:47:56.184041728 -0700 @@ -432,6 +432,7 @@ ngx_http_file_cache_lock_wait_handler(ng ngx_uint_t wait; ngx_msec_t timer; ngx_http_cache_t *c; + ngx_connection_t *conn; ngx_http_request_t *r; ngx_http_file_cache_t *cache; @@ -471,7 +472,10 @@ wakeup: c->waiting = 0; r->main->blocked--; - r->connection->write->handler(r->connection->write); + + conn = r->connection; + r->write_event_handler(r); + ngx_http_run_posted_requests(conn); }

7 years, 12 months

[nginx] Entity tags: downgrade strong etags to weak ones as needed.

by Maxim Dounin

details: http://hg.nginx.org/nginx/rev/e491b26fa5a1 branches: changeset: 5733:e491b26fa5a1 user: Maxim Dounin <mdounin(a)mdounin.ru> date: Thu Jun 26 02:21:01 2014 +0400 description: Entity tags: downgrade strong etags to weak ones as needed. See http://mailman.nginx.org/pipermail/nginx-devel/2013-November/004523.html. diffstat: src/http/modules/ngx_http_addition_filter_module.c | 2 +- src/http/modules/ngx_http_gunzip_filter_module.c | 2 +- src/http/modules/ngx_http_gzip_filter_module.c | 2 +- src/http/modules/ngx_http_ssi_filter_module.c | 5 ++- src/http/modules/ngx_http_sub_filter_module.c | 5 ++- src/http/modules/ngx_http_xslt_filter_module.c | 6 ++- src/http/ngx_http_core_module.c | 40 ++++++++++++++++++++++ src/http/ngx_http_core_module.h | 1 + 8 files changed, 56 insertions(+), 7 deletions(-) diffs (153 lines): diff --git a/src/http/modules/ngx_http_addition_filter_module.c b/src/http/modules/ngx_http_addition_filter_module.c --- a/src/http/modules/ngx_http_addition_filter_module.c +++ b/src/http/modules/ngx_http_addition_filter_module.c @@ -121,7 +121,7 @@ ngx_http_addition_header_filter(ngx_http ngx_http_clear_content_length(r); ngx_http_clear_accept_ranges(r); - ngx_http_clear_etag(r); + ngx_http_weak_etag(r); return ngx_http_next_header_filter(r); } diff --git a/src/http/modules/ngx_http_gunzip_filter_module.c b/src/http/modules/ngx_http_gunzip_filter_module.c --- a/src/http/modules/ngx_http_gunzip_filter_module.c +++ b/src/http/modules/ngx_http_gunzip_filter_module.c @@ -165,7 +165,7 @@ ngx_http_gunzip_header_filter(ngx_http_r ngx_http_clear_content_length(r); ngx_http_clear_accept_ranges(r); - ngx_http_clear_etag(r); + ngx_http_weak_etag(r); return ngx_http_next_header_filter(r); } diff --git a/src/http/modules/ngx_http_gzip_filter_module.c b/src/http/modules/ngx_http_gzip_filter_module.c --- a/src/http/modules/ngx_http_gzip_filter_module.c +++ b/src/http/modules/ngx_http_gzip_filter_module.c @@ -306,7 +306,7 @@ ngx_http_gzip_header_filter(ngx_http_req ngx_http_clear_content_length(r); ngx_http_clear_accept_ranges(r); - ngx_http_clear_etag(r); + ngx_http_weak_etag(r); return ngx_http_next_header_filter(r); } diff --git a/src/http/modules/ngx_http_ssi_filter_module.c b/src/http/modules/ngx_http_ssi_filter_module.c --- a/src/http/modules/ngx_http_ssi_filter_module.c +++ b/src/http/modules/ngx_http_ssi_filter_module.c @@ -369,10 +369,13 @@ ngx_http_ssi_header_filter(ngx_http_requ if (r == r->main) { ngx_http_clear_content_length(r); ngx_http_clear_accept_ranges(r); - ngx_http_clear_etag(r); if (!slcf->last_modified) { ngx_http_clear_last_modified(r); + ngx_http_clear_etag(r); + + } else { + ngx_http_weak_etag(r); } } diff --git a/src/http/modules/ngx_http_sub_filter_module.c b/src/http/modules/ngx_http_sub_filter_module.c --- a/src/http/modules/ngx_http_sub_filter_module.c +++ b/src/http/modules/ngx_http_sub_filter_module.c @@ -175,10 +175,13 @@ ngx_http_sub_header_filter(ngx_http_requ if (r == r->main) { ngx_http_clear_content_length(r); - ngx_http_clear_etag(r); if (!slcf->last_modified) { ngx_http_clear_last_modified(r); + ngx_http_clear_etag(r); + + } else { + ngx_http_weak_etag(r); } } diff --git a/src/http/modules/ngx_http_xslt_filter_module.c b/src/http/modules/ngx_http_xslt_filter_module.c --- a/src/http/modules/ngx_http_xslt_filter_module.c +++ b/src/http/modules/ngx_http_xslt_filter_module.c @@ -337,12 +337,14 @@ ngx_http_xslt_send(ngx_http_request_t *r r->headers_out.content_length = NULL; } - ngx_http_clear_etag(r); - conf = ngx_http_get_module_loc_conf(r, ngx_http_xslt_filter_module); if (!conf->last_modified) { ngx_http_clear_last_modified(r); + ngx_http_clear_etag(r); + + } else { + ngx_http_weak_etag(r); } } diff --git a/src/http/ngx_http_core_module.c b/src/http/ngx_http_core_module.c --- a/src/http/ngx_http_core_module.c +++ b/src/http/ngx_http_core_module.c @@ -1851,6 +1851,46 @@ ngx_http_set_etag(ngx_http_request_t *r) } +void +ngx_http_weak_etag(ngx_http_request_t *r) +{ + size_t len; + u_char *p; + ngx_table_elt_t *etag; + + etag = r->headers_out.etag; + + if (etag == NULL) { + return; + } + + if (etag->value.len > 2 + && etag->value.data[0] == 'W' + && etag->value.data[1] == '/') + { + return; + } + + if (etag->value.len < 1 || etag->value.data[0] != '"') { + r->headers_out.etag->hash = 0; + r->headers_out.etag = NULL; + return; + } + + p = ngx_pnalloc(r->pool, etag->value.len + 2); + if (p == NULL) { + r->headers_out.etag->hash = 0; + r->headers_out.etag = NULL; + return; + } + + len = ngx_sprintf(p, "W/%V", &etag->value) - p; + + etag->value.data = p; + etag->value.len = len; +} + + ngx_int_t ngx_http_send_response(ngx_http_request_t *r, ngx_uint_t status, ngx_str_t *ct, ngx_http_complex_value_t *cv) diff --git a/src/http/ngx_http_core_module.h b/src/http/ngx_http_core_module.h --- a/src/http/ngx_http_core_module.h +++ b/src/http/ngx_http_core_module.h @@ -501,6 +501,7 @@ void *ngx_http_test_content_type(ngx_htt ngx_int_t ngx_http_set_content_type(ngx_http_request_t *r); void ngx_http_set_exten(ngx_http_request_t *r); ngx_int_t ngx_http_set_etag(ngx_http_request_t *r); +void ngx_http_weak_etag(ngx_http_request_t *r); ngx_int_t ngx_http_send_response(ngx_http_request_t *r, ngx_uint_t status, ngx_str_t *ct, ngx_http_complex_value_t *cv); u_char *ngx_http_map_uri_to_path(ngx_http_request_t *r, ngx_str_t *name,

8 years, 3 months

[PATCH] Perl: NULL-terminate argument list

by Piotr Sikora

# HG changeset patch # User Piotr Sikora <piotr(a)cloudflare.com> # Date 1403176596 25200 # Thu Jun 19 04:16:36 2014 -0700 # Node ID 290f3fcb9cf552c235b9807cf0af3830b5add5af # Parent 675bda8dcfdbf66e4a17017839f39ed6c8cbb9f5 Perl: NULL-terminate argument list. perl_parse() function expects argv/argc-style argument list, which according to the C standard must be NULL-terminated, that is: argv[argc] == NULL. This change fixes a crash (SIGSEGV) that could happen because of the buffer overrun during perl module initialization. Signed-off-by: Piotr Sikora <piotr(a)cloudflare.com> diff -r 675bda8dcfdb -r 290f3fcb9cf5 src/http/modules/perl/ngx_http_perl_module.c --- a/src/http/modules/perl/ngx_http_perl_module.c Thu Jun 19 13:55:59 2014 +0400 +++ b/src/http/modules/perl/ngx_http_perl_module.c Thu Jun 19 04:16:36 2014 -0700 @@ -577,7 +577,7 @@ ngx_http_perl_create_interpreter(ngx_con n = (pmcf->modules != NGX_CONF_UNSET_PTR) ? pmcf->modules->nelts * 2 : 0; - embedding = ngx_palloc(cf->pool, (4 + n) * sizeof(char *)); + embedding = ngx_palloc(cf->pool, (5 + n) * sizeof(char *)); if (embedding == NULL) { goto fail; } @@ -595,6 +595,7 @@ ngx_http_perl_create_interpreter(ngx_con embedding[n++] = "-Mnginx"; embedding[n++] = "-e"; embedding[n++] = "0"; + embedding[n] = NULL; n = perl_parse(perl, ngx_http_perl_xs_init, n, embedding, NULL);

8 years, 3 months

[PATCH] Core: bugfix for the ngx_slab_max_size case

by Jianjun Zheng

At present, alloting memory with size of ngx_slab_max_size causes 1) an internal fragmentation, size of ngx_slab_max_size, comes into being 2) the slot with index of (ngx_pagesize_shift - pool->min_shift - 1) is the right slot for this size. # HG changeset patch # User Jianjun Zheng <codeeply(a)gmail.com> # Date 1403080799 -28800 # Wed Jun 18 16:39:59 2014 +0800 # Node ID 1704335dd810e2e2abb2b393b4f7b7c9004c6012 # Parent ec919574cc14f7781c0ca212cffec586f88eec40 Core: bugfix for the ngx_slab_max_size case diff -r ec919574cc14 -r 1704335dd810 src/core/ngx_slab.c --- a/src/core/ngx_slab.c Tue Jun 17 16:51:25 2014 +0400 +++ b/src/core/ngx_slab.c Wed Jun 18 16:39:59 2014 +0800 @@ -160,7 +160,7 @@ ngx_uint_t i, slot, shift, map; ngx_slab_page_t *page, *prev, *slots; - if (size >= ngx_slab_max_size) { + if (size > ngx_slab_max_size) { ngx_log_debug1(NGX_LOG_DEBUG_ALLOC, ngx_cycle->log, 0, "slab alloc: %uz", size);

8 years, 4 months

Patch: Refactor ngx_http_write_request_body into a filter

by grrm grrm

Hello, This patch removes some redundant ngx_http_request_body_filter calls, simplifies the ngx_http_do_read_client_request_body and ngx_http_read_client_request_body functions and removes some duplication of code. body.t and body_chunked.t test in nginx-tests are passing. Please kindly consider it. Thank you.

8 years, 4 months

[nginx] Entity tags: explicit flag to skip not modified filter.

by Maxim Dounin

details: http://hg.nginx.org/nginx/rev/5fb1e57c758a branches: changeset: 5735:5fb1e57c758a user: Maxim Dounin <mdounin(a)mdounin.ru> date: Thu Jun 26 02:27:11 2014 +0400 description: Entity tags: explicit flag to skip not modified filter. Previously, last_modified_time was tested against -1 to check if the not modified filter should be skipped. Notably, this prevented nginx from additional If-Modified-Since (et al.) checks on proxied responses. Such behaviour is suboptimal in some cases though, as checks are always skipped on responses from a cache with ETag only (without Last-Modified), resulting in If-None-Match being ignored in such cases. Additionally, it was not possible to return 412 from the If-Unmodified-Since if last modification time was not known for some reason. This change introduces explicit r->disable_not_modified flag instead, which is set by ngx_http_upstream_process_headers(). diffstat: src/http/modules/ngx_http_not_modified_filter_module.c | 10 +++++++++- src/http/ngx_http_request.h | 1 + src/http/ngx_http_upstream.c | 2 ++ 3 files changed, 12 insertions(+), 1 deletions(-) diffs (57 lines): diff --git a/src/http/modules/ngx_http_not_modified_filter_module.c b/src/http/modules/ngx_http_not_modified_filter_module.c --- a/src/http/modules/ngx_http_not_modified_filter_module.c +++ b/src/http/modules/ngx_http_not_modified_filter_module.c @@ -56,7 +56,7 @@ ngx_http_not_modified_header_filter(ngx_ { if (r->headers_out.status != NGX_HTTP_OK || r != r->main - || r->headers_out.last_modified_time == -1) + || r->disable_not_modified) { return ngx_http_next_header_filter(r); } @@ -114,6 +114,10 @@ ngx_http_test_if_unmodified(ngx_http_req { time_t iums; + if (r->headers_out.last_modified_time == (time_t) -1) { + return 0; + } + iums = ngx_http_parse_time(r->headers_in.if_unmodified_since->value.data, r->headers_in.if_unmodified_since->value.len); @@ -134,6 +138,10 @@ ngx_http_test_if_modified(ngx_http_reque time_t ims; ngx_http_core_loc_conf_t *clcf; + if (r->headers_out.last_modified_time == (time_t) -1) { + return 1; + } + clcf = ngx_http_get_module_loc_conf(r, ngx_http_core_module); if (clcf->if_modified_since == NGX_HTTP_IMS_OFF) { diff --git a/src/http/ngx_http_request.h b/src/http/ngx_http_request.h --- a/src/http/ngx_http_request.h +++ b/src/http/ngx_http_request.h @@ -528,6 +528,7 @@ struct ngx_http_request_s { unsigned filter_need_temporary:1; unsigned allow_ranges:1; unsigned single_range:1; + unsigned disable_not_modified:1; #if (NGX_STAT_STUB) unsigned stat_reading:1; diff --git a/src/http/ngx_http_upstream.c b/src/http/ngx_http_upstream.c --- a/src/http/ngx_http_upstream.c +++ b/src/http/ngx_http_upstream.c @@ -2238,6 +2238,8 @@ ngx_http_upstream_process_headers(ngx_ht r->headers_out.content_length_n = u->headers_in.content_length_n; + r->disable_not_modified = !u->cacheable; + u->length = -1; return NGX_OK;

8 years, 4 months

Development

by David Carlier

HI All, I am working as C/C++ developer for a company which makes nginx modules and would like to know if I can contribute a bit. Kind regards. David CARLIER dotMobi / Afilias Technologies DUBLIN

8 years, 5 months

[PATCH] Core: use uppercase hexadecimal digits for percent-encoding

by Piotr Sikora

# HG changeset patch # User Piotr Sikora <piotr(a)cloudflare.com> # Date 1403851163 25200 # Thu Jun 26 23:39:23 2014 -0700 # Node ID 177382006b7d7a421688831d5793b2e417074b48 # Parent 42114bf12da0cf3d428d0e695139f5366cbd0513 Core: use uppercase hexadecimal digits for percent-encoding. RFC3986 says that, for consistency, URI producers and normalizers should use uppercase hexadecimal digits for all percent-encodings. This is also what modern web browsers and other tools use. Using lowercase hexadecimal digits makes it harder to interact with those tools in case when use of the percent-encoded URI is required, for example when $request_uri is part of the cache key. Signed-off-by: Piotr Sikora <piotr(a)cloudflare.com> diff -r 42114bf12da0 -r 177382006b7d src/core/ngx_string.c --- a/src/core/ngx_string.c Mon Jun 16 19:43:25 2014 +0400 +++ b/src/core/ngx_string.c Thu Jun 26 23:39:23 2014 -0700 @@ -1407,7 +1407,7 @@ ngx_escape_uri(u_char *dst, u_char *src, { ngx_uint_t n; uint32_t *escape; - static u_char hex[] = "0123456789abcdef"; + static u_char hex[] = "0123456789ABCDEF"; /* " ", "#", "%", "?", %00-%1F, %7F-%FF */

8 years, 5 months

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

nginx-devel June 2014