nginx-devel December 2014

nginx-devel@nginx.org

17 participants
57 discussions

[PATCH] Mail: send starttls flag value to auth script

by Filipe da Silva

# HG changeset patch # User Filipe da Silva <fdasilvayy(a)gmail.com> # Date 1394099468 -3600 # Thu Mar 06 10:51:08 2014 +0100 # Node ID 51fd90f96449c23af0076a19efbfdb1f88702125 # Parent 24df9fa5868957c1fb9a2d1569271e0958327dad Mail: send starttls flag value to auth script. Allow to do logging (if logging takes place in the auth script) and or force some users to use STARTTLS while others can use unencrypted connection. diff -r 24df9fa58689 -r 51fd90f96449 src/mail/ngx_mail_auth_http_module.c --- a/src/mail/ngx_mail_auth_http_module.c Thu Mar 06 10:51:08 2014 +0100 +++ b/src/mail/ngx_mail_auth_http_module.c Thu Mar 06 10:51:08 2014 +0100 @@ -1165,6 +1165,9 @@ ngx_mail_auth_http_create_request(ngx_ma + sizeof("Auth-Salt: ") - 1 + s->salt.len + sizeof("Auth-Protocol: ") - 1 + cscf->protocol->name.len + sizeof(CRLF) - 1 +#if (NGX_MAIL_SSL) + + sizeof("Auth-STARTTLS: ") - 1 + 1 + sizeof(CRLF) - 1 +#endif + sizeof("Auth-Login-Attempt: ") - 1 + NGX_INT_T_LEN + sizeof(CRLF) - 1 + sizeof("Client-IP: ") - 1 + s->connection->addr_text.len @@ -1219,6 +1222,13 @@ ngx_mail_auth_http_create_request(ngx_ma cscf->protocol->name.len); *b->last++ = CR; *b->last++ = LF; +#if (NGX_MAIL_SSL) + b->last = ngx_cpymem(b->last, "Auth-STARTTLS: ", + sizeof("Auth-STARTTLS: ") - 1); + *b->last++ = s->starttls ? '1' : '0' ; + *b->last++ = CR; *b->last++ = LF; +#endif + b->last = ngx_sprintf(b->last, "Auth-Login-Attempt: %ui" CRLF, s->login_attempt);

7 years, 8 months

nginx + DSCP (Differentied services)

by Dani Bento

Hello, We are using nginx-1.6 in a production environment for caching data from multiple remote origins. To improve our data distribution over the network we thought that using IP DiffServ/ToS to traffic shape by backend would be useful. We developed a small patch for the nginx core to mark packets. Because the nginx workers don't seem to have CAP_NET_ADMIN privileges we are using the TOS field instead of socket marks. This patch only marks client packets and not upstream/backend packets. Besides that, we added an X-Accel-ClassID header so that the upstream/backend itself can choose the class id to use. Do you find this an interesting feature to add to the nginx core? How can we proceed and submit a patch? Thanks, Dani -- Dani Bento Direção de Internet e Tecnologia DTS/DVS tlm: +351 91 429 72 81 dani(a)telecom.pt

7 years, 9 months

[PATCH] Add strict Host validation

by Piotr Sikora

# HG changeset patch # User Piotr Sikora <piotr(a)cloudflare.com> # Date 1418870862 28800 # Wed Dec 17 18:47:42 2014 -0800 # Node ID ab0442e232ce098438943a77422d8a04cc5b6790 # Parent 99751fe3bc3b285801b434f7f707d87fa42b093e Add strict Host validation. According to RFC3986, Host is a sequence of printable ASCII characters, with the exception of: space, ", #, /, <, >, ?, @, \, ^, `, {, | and }. Signed-off-by: Piotr Sikora <piotr(a)cloudflare.com> diff -r 99751fe3bc3b -r ab0442e232ce src/http/ngx_http_request.c --- a/src/http/ngx_http_request.c Fri Dec 12 20:25:42 2014 +0300 +++ b/src/http/ngx_http_request.c Wed Dec 17 18:47:42 2014 -0800 @@ -1955,12 +1955,25 @@ ngx_http_validate_host(ngx_str_t *host, } break; - case '\0': + case ' ': + case '"': + case '#': + case '/': + case '<': + case '>': + case '?': + case '@': + case '\\': + case '^': + case '`': + case '{': + case '|': + case '}': return NGX_DECLINED; default: - if (ngx_path_separator(ch)) { + if (ch < 0x20 || ch > 0x7e) { return NGX_DECLINED; }

7 years, 10 months

[nginx] Core: added prefix-based temporary files.

by Valentin Bartenev

details: http://hg.nginx.org/nginx/rev/a9138c35120d branches: changeset: 5958:a9138c35120d user: Valentin Bartenev <vbart(a)nginx.com> date: Fri Dec 26 16:22:54 2014 +0300 description: Core: added prefix-based temporary files. Now, if the "path" parameter is NULL, ngx_create_temp_file() will use file->name as a predefined file path prefix. diffstat: src/core/ngx_file.c | 28 ++++++++++++++++++++-------- src/core/ngx_file.h | 1 + 2 files changed, 21 insertions(+), 8 deletions(-) diffs (83 lines): diff -r 570677a06dce -r a9138c35120d src/core/ngx_file.c --- a/src/core/ngx_file.c Fri Dec 26 16:22:50 2014 +0300 +++ b/src/core/ngx_file.c Fri Dec 26 16:22:54 2014 +0300 @@ -111,8 +111,9 @@ ngx_write_chain_to_temp_file(ngx_temp_fi ngx_int_t rc; if (tf->file.fd == NGX_INVALID_FILE) { - rc = ngx_create_temp_file(&tf->file, tf->path, tf->pool, - tf->persistent, tf->clean, tf->access); + rc = ngx_create_temp_file(&tf->file, tf->prefix ? NULL : tf->path, + tf->pool, tf->persistent, tf->clean, + tf->access); if (rc != NGX_OK) { return rc; @@ -132,12 +133,15 @@ ngx_int_t ngx_create_temp_file(ngx_file_t *file, ngx_path_t *path, ngx_pool_t *pool, ngx_uint_t persistent, ngx_uint_t clean, ngx_uint_t access) { + u_char *p; uint32_t n; ngx_err_t err; + ngx_str_t prefix; ngx_pool_cleanup_t *cln; ngx_pool_cleanup_file_t *clnf; - file->name.len = path->name.len + 1 + path->len + 10; + prefix = path ? path->name : file->name; + file->name.len = prefix.len + 1 + (path ? path->len : 0) + 10; file->name.data = ngx_pnalloc(pool, file->name.len + 1); if (file->name.data == NULL) { @@ -150,7 +154,14 @@ ngx_create_temp_file(ngx_file_t *file, n } #endif - ngx_memcpy(file->name.data, path->name.data, path->name.len); + p = ngx_cpymem(file->name.data, prefix.data, prefix.len); + + if (path) { + p += 1 + path->len; + + } else { + *p++ = '.'; + } n = (uint32_t) ngx_next_temp_number(0); @@ -160,10 +171,11 @@ ngx_create_temp_file(ngx_file_t *file, n } for ( ;; ) { - (void) ngx_sprintf(file->name.data + path->name.len + 1 + path->len, - "%010uD%Z", n); + (void) ngx_sprintf(p, "%010uD%Z", n); - ngx_create_hashed_filename(path, file->name.data, file->name.len); + if (path) { + ngx_create_hashed_filename(path, file->name.data, file->name.len); + } ngx_log_debug1(NGX_LOG_DEBUG_CORE, file->log, 0, "hashed path: %s", file->name.data); @@ -192,7 +204,7 @@ ngx_create_temp_file(ngx_file_t *file, n continue; } - if ((path->level[0] == 0) || (err != NGX_ENOPATH)) { + if ((path == NULL) || (path->level[0] == 0) || (err != NGX_ENOPATH)) { ngx_log_error(NGX_LOG_CRIT, file->log, err, ngx_open_tempfile_n " \"%s\" failed", file->name.data); diff -r 570677a06dce -r a9138c35120d src/core/ngx_file.h --- a/src/core/ngx_file.h Fri Dec 26 16:22:50 2014 +0300 +++ b/src/core/ngx_file.h Fri Dec 26 16:22:54 2014 +0300 @@ -71,6 +71,7 @@ typedef struct { unsigned log_level:8; unsigned persistent:1; unsigned clean:1; + unsigned prefix:1; } ngx_temp_file_t;

7 years, 10 months

handle NGX_AGAIN properly

by Julien Zefi

hi, i understand that NGX_AGAIN is returned when a chain could not be send because more data cannot be buffered on that socket. I need to understand the following: in my case, when i receive a request, i start a timer every 10ms and send out some data, then i create a new timer every10ms until i decide to finish sending out data (video frames). But if in some triggered callback by the timer the ngx_http_output_filter(..) returns NGX_AGAIN *i assume* NginX will send that chain as soon as the socket becomes available again. But after that happens, how can i restore my timer cycle ? thnks. J.Z.

7 years, 11 months

[nginx] Upstream: use_temp_path parameter of proxy_cache_path an...

by Valentin Bartenev

details: http://hg.nginx.org/nginx/rev/e9effef98874 branches: changeset: 5960:e9effef98874 user: Valentin Bartenev <vbart(a)nginx.com> date: Fri Dec 26 16:22:59 2014 +0300 description: Upstream: use_temp_path parameter of proxy_cache_path and friends. When set to "off", temporary files for cacheable responses will be stored inside cache directory. diffstat: src/http/ngx_http_cache.h | 3 +++ src/http/ngx_http_file_cache.c | 36 +++++++++++++++++++++++++++++++++++- src/http/ngx_http_upstream.c | 8 ++++++++ 3 files changed, 46 insertions(+), 1 deletions(-) diffs (105 lines): diff -r f7584d7c0ccb -r e9effef98874 src/http/ngx_http_cache.h --- a/src/http/ngx_http_cache.h Fri Dec 26 16:22:56 2014 +0300 +++ b/src/http/ngx_http_cache.h Fri Dec 26 16:22:59 2014 +0300 @@ -155,6 +155,9 @@ struct ngx_http_file_cache_s { ngx_msec_t loader_threshold; ngx_shm_zone_t *shm_zone; + + ngx_uint_t use_temp_path; + /* unsigned use_temp_path:1 */ }; diff -r f7584d7c0ccb -r e9effef98874 src/http/ngx_http_file_cache.c --- a/src/http/ngx_http_file_cache.c Fri Dec 26 16:22:56 2014 +0300 +++ b/src/http/ngx_http_file_cache.c Fri Dec 26 16:22:59 2014 +0300 @@ -1935,6 +1935,17 @@ ngx_http_file_cache_add_file(ngx_tree_ct return NGX_ERROR; } + /* + * Temporary files in cache have a suffix consisting of a dot + * followed by 10 digits. + */ + + if (name->len >= 2 * NGX_HTTP_CACHE_KEY_LEN + 1 + 10 + && name->data[name->len - 10 - 1] == '.') + { + return NGX_OK; + } + if (ctx->size < (off_t) sizeof(ngx_http_file_cache_header_t)) { ngx_log_error(NGX_LOG_CRIT, ctx->log, 0, "cache file \"%s\" is too small", name->data); @@ -2063,7 +2074,7 @@ ngx_http_file_cache_set_slot(ngx_conf_t ngx_str_t s, name, *value; ngx_int_t loader_files; ngx_msec_t loader_sleep, loader_threshold; - ngx_uint_t i, n; + ngx_uint_t i, n, use_temp_path; ngx_array_t *caches; ngx_http_file_cache_t *cache, **ce; @@ -2077,6 +2088,8 @@ ngx_http_file_cache_set_slot(ngx_conf_t return NGX_CONF_ERROR; } + use_temp_path = 1; + inactive = 600; loader_files = 100; loader_sleep = 50; @@ -2137,6 +2150,25 @@ ngx_http_file_cache_set_slot(ngx_conf_t return NGX_CONF_ERROR; } + if (ngx_strncmp(value[i].data, "use_temp_path=", 14) == 0) { + + if (ngx_strcmp(&value[i].data[14], "on") == 0) { + use_temp_path = 1; + + } else if (ngx_strcmp(&value[i].data[14], "off") == 0) { + use_temp_path = 0; + + } else { + ngx_conf_log_error(NGX_LOG_EMERG, cf, 0, + "invalid use_temp_path value \"%V\", " + "it must be \"on\" or \"off\"", + &value[i]); + return NGX_CONF_ERROR; + } + + continue; + } + if (ngx_strncmp(value[i].data, "keys_zone=", 10) == 0) { name.data = value[i].data + 10; @@ -2274,6 +2306,8 @@ ngx_http_file_cache_set_slot(ngx_conf_t cache->shm_zone->init = ngx_http_file_cache_init; cache->shm_zone->data = cache; + cache->use_temp_path = use_temp_path; + cache->inactive = inactive; cache->max_size = max_size; diff -r f7584d7c0ccb -r e9effef98874 src/http/ngx_http_upstream.c --- a/src/http/ngx_http_upstream.c Fri Dec 26 16:22:56 2014 +0300 +++ b/src/http/ngx_http_upstream.c Fri Dec 26 16:22:59 2014 +0300 @@ -2677,6 +2677,14 @@ ngx_http_upstream_send_response(ngx_http if (p->cacheable) { p->temp_file->persistent = 1; +#if (NGX_HTTP_CACHE) + if (r->cache && !r->cache->file_cache->use_temp_path) { + p->temp_file->file.name = r->cache->file.name; + p->temp_file->path = r->cache->file_cache->path; + p->temp_file->prefix = 1; + } +#endif + } else { p->temp_file->log_level = NGX_LOG_WARN; p->temp_file->warn = "an upstream response is buffered "

7 years, 11 months

[nginx] Cache: update variant while setting header.

by Valentin Bartenev

details: http://hg.nginx.org/nginx/rev/f7584d7c0ccb branches: changeset: 5959:f7584d7c0ccb user: Valentin Bartenev <vbart(a)nginx.com> date: Fri Dec 26 16:22:56 2014 +0300 description: Cache: update variant while setting header. Some parts of code related to handling variants of a resource moved into a separate function that is called earlier. This allows to use cache file name as a prefix for temporary file in the following patch. diffstat: src/http/ngx_http_cache.h | 2 +- src/http/ngx_http_file_cache.c | 94 +++++++++++++++++++++++++---------------- src/http/ngx_http_upstream.c | 5 +- 3 files changed, 63 insertions(+), 38 deletions(-) diffs (159 lines): diff -r a9138c35120d -r f7584d7c0ccb src/http/ngx_http_cache.h --- a/src/http/ngx_http_cache.h Fri Dec 26 16:22:54 2014 +0300 +++ b/src/http/ngx_http_cache.h Fri Dec 26 16:22:56 2014 +0300 @@ -162,7 +162,7 @@ ngx_int_t ngx_http_file_cache_new(ngx_ht ngx_int_t ngx_http_file_cache_create(ngx_http_request_t *r); void ngx_http_file_cache_create_key(ngx_http_request_t *r); ngx_int_t ngx_http_file_cache_open(ngx_http_request_t *r); -void ngx_http_file_cache_set_header(ngx_http_request_t *r, u_char *buf); +ngx_int_t ngx_http_file_cache_set_header(ngx_http_request_t *r, u_char *buf); void ngx_http_file_cache_update(ngx_http_request_t *r, ngx_temp_file_t *tf); void ngx_http_file_cache_update_header(ngx_http_request_t *r); ngx_int_t ngx_http_cache_send(ngx_http_request_t *); diff -r a9138c35120d -r f7584d7c0ccb src/http/ngx_http_file_cache.c --- a/src/http/ngx_http_file_cache.c Fri Dec 26 16:22:54 2014 +0300 +++ b/src/http/ngx_http_file_cache.c Fri Dec 26 16:22:56 2014 +0300 @@ -37,6 +37,8 @@ static void ngx_http_file_cache_vary_hea ngx_md5_t *md5, ngx_str_t *name); static ngx_int_t ngx_http_file_cache_reopen(ngx_http_request_t *r, ngx_http_cache_t *c); +static ngx_int_t ngx_http_file_cache_update_variant(ngx_http_request_t *r, + ngx_http_cache_t *c); static void ngx_http_file_cache_cleanup(void *data); static time_t ngx_http_file_cache_forced_expire(ngx_http_file_cache_t *cache); static time_t ngx_http_file_cache_expire(ngx_http_file_cache_t *cache); @@ -1122,7 +1124,7 @@ ngx_http_file_cache_reopen(ngx_http_requ } -void +ngx_int_t ngx_http_file_cache_set_header(ngx_http_request_t *r, u_char *buf) { ngx_http_file_cache_header_t *h = (ngx_http_file_cache_header_t *) buf; @@ -1164,9 +1166,10 @@ ngx_http_file_cache_set_header(ngx_http_ ngx_http_file_cache_vary(r, c->vary.data, c->vary.len, c->variant); ngx_memcpy(h->variant, c->variant, NGX_HTTP_CACHE_KEY_LEN); - - } else { - ngx_memzero(c->variant, NGX_HTTP_CACHE_KEY_LEN); + } + + if (ngx_http_file_cache_update_variant(r, c) != NGX_OK) { + return NGX_ERROR; } p = buf + sizeof(ngx_http_file_cache_header_t); @@ -1179,6 +1182,57 @@ ngx_http_file_cache_set_header(ngx_http_ } *p = LF; + + return NGX_OK; +} + + +static ngx_int_t +ngx_http_file_cache_update_variant(ngx_http_request_t *r, ngx_http_cache_t *c) +{ + ngx_http_file_cache_t *cache; + + if (!c->secondary) { + return NGX_OK; + } + + if (c->vary.len + && ngx_memcmp(c->variant, c->key, NGX_HTTP_CACHE_KEY_LEN) == 0) + { + return NGX_OK; + } + + /* + * if the variant hash doesn't match one we used as a secondary + * cache key, switch back to the original key + */ + + cache = c->file_cache; + + ngx_log_debug0(NGX_LOG_DEBUG_HTTP, r->connection->log, 0, + "http file cache main key"); + + ngx_shmtx_lock(&cache->shpool->mutex); + + c->node->count--; + c->node->updating = 0; + c->node = NULL; + + ngx_shmtx_unlock(&cache->shpool->mutex); + + c->file.name.len = 0; + + ngx_memcpy(c->key, c->main, NGX_HTTP_CACHE_KEY_LEN); + + if (ngx_http_file_cache_exists(cache, c) == NGX_ERROR) { + return NGX_ERROR; + } + + if (ngx_http_file_cache_name(r, cache->path) != NGX_OK) { + return NGX_ERROR; + } + + return NGX_OK; } @@ -1204,38 +1258,6 @@ ngx_http_file_cache_update(ngx_http_requ cache = c->file_cache; - if (c->secondary - && ngx_memcmp(c->variant, c->key, NGX_HTTP_CACHE_KEY_LEN) != 0) - { - /* - * if the variant hash doesn't match one we used as a secondary - * cache key, switch back to the original key - */ - - ngx_log_debug0(NGX_LOG_DEBUG_HTTP, r->connection->log, 0, - "http file cache main key"); - - ngx_shmtx_lock(&cache->shpool->mutex); - - c->node->count--; - c->node->updating = 0; - c->node = NULL; - - ngx_shmtx_unlock(&cache->shpool->mutex); - - c->file.name.len = 0; - - ngx_memcpy(c->key, c->main, NGX_HTTP_CACHE_KEY_LEN); - - if (ngx_http_file_cache_exists(cache, c) == NGX_ERROR) { - return; - } - - if (ngx_http_file_cache_name(r, cache->path) != NGX_OK) { - return; - } - } - c->updated = 1; c->updating = 0; diff -r a9138c35120d -r f7584d7c0ccb src/http/ngx_http_upstream.c --- a/src/http/ngx_http_upstream.c Fri Dec 26 16:22:54 2014 +0300 +++ b/src/http/ngx_http_upstream.c Fri Dec 26 16:22:56 2014 +0300 @@ -2628,7 +2628,10 @@ ngx_http_upstream_send_response(ngx_http } } - ngx_http_file_cache_set_header(r, u->buffer.start); + if (ngx_http_file_cache_set_header(r, u->buffer.start) != NGX_OK) { + ngx_http_upstream_finalize_request(r, u, NGX_ERROR); + return; + } } else { u->cacheable = 0;

7 years, 11 months

[nginx] Unified handling of ngx_create_temp_file() return value.

by Valentin Bartenev

details: http://hg.nginx.org/nginx/rev/570677a06dce branches: changeset: 5957:570677a06dce user: Valentin Bartenev <vbart(a)nginx.com> date: Fri Dec 26 16:22:50 2014 +0300 description: Unified handling of ngx_create_temp_file() return value. The original check for NGX_AGAIN was surplus, since the function returns only NGX_OK or NGX_ERROR. Now it looks similar to other places. No functional changes. diffstat: src/core/ngx_file.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diffs (12 lines): diff -r 4045818c726a -r 570677a06dce src/core/ngx_file.c --- a/src/core/ngx_file.c Wed Dec 24 19:02:58 2014 +0300 +++ b/src/core/ngx_file.c Fri Dec 26 16:22:50 2014 +0300 @@ -114,7 +114,7 @@ ngx_write_chain_to_temp_file(ngx_temp_fi rc = ngx_create_temp_file(&tf->file, tf->path, tf->pool, tf->persistent, tf->clean, tf->access); - if (rc == NGX_ERROR || rc == NGX_AGAIN) { + if (rc != NGX_OK) { return rc; }

7 years, 11 months

[PATCH-REQUEST] : SPDY handling for 304 response

by Shrirang Ballal

304 handling for non-spdy requests is exactly same as that of 204 handling (Reference : ngx_http_header_filter_module.c::ngx_http_header_filter()). SPDY requests too should be handled same way. Handle 204/304 response for SPDY requests in same way as the same way that are handled for non-SPDY requests --- .../src/http/ngx_http_spdy_filter_module.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/service/proxy/nginx-1.6.2/src/http/ngx_http_spdy_filter_module.c b/service/proxy/nginx-1.6.2/src/http/ngx_http_spdy_filter_module.c index a31ad1f..8f975f5 100644 --- a/service/proxy/nginx-1.6.2/src/http/ngx_http_spdy_filter_module.c +++ b/service/proxy/nginx-1.6.2/src/http/ngx_http_spdy_filter_module.c @@ -147,10 +147,10 @@ ngx_http_spdy_header_filter(ngx_http_request_t *r) break; case NGX_HTTP_NOT_MODIFIED: - r->header_only = 1; - break; - case NGX_HTTP_NO_CONTENT: + /* take same action for "NGX_HTTP_NOT_MODIFIED" as that for + "NGX_HTTP_NO_CONTENT" */ r->header_only = 1; ngx_str_null(&r->headers_out.content_type);

7 years, 11 months

[nginx] Trailing space fix.

by Maxim Dounin

details: http://hg.nginx.org/nginx/rev/4045818c726a branches: changeset: 5956:4045818c726a user: Maxim Dounin <mdounin(a)mdounin.ru> date: Wed Dec 24 19:02:58 2014 +0300 description: Trailing space fix. diffstat: docs/xml/nginx/changes.xml | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diffs (12 lines): diff --git a/docs/xml/nginx/changes.xml b/docs/xml/nginx/changes.xml --- a/docs/xml/nginx/changes.xml +++ b/docs/xml/nginx/changes.xml @@ -84,7 +84,7 @@ might not work correctly inside the "if" <change type="bugfix"> <para lang="ru"> -директива proxy_store с параметром "on" игнорировалась, +директива proxy_store с параметром "on" игнорировалась, если на предыдущем уровне использовалась директива proxy_store с явно заданным путём к файлам. </para>

7 years, 11 months

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

nginx-devel December 2014