From agentzh at gmail.com Fri Jun 1 10:14:58 2012 From: agentzh at gmail.com (agentzh) Date: Fri, 1 Jun 2012 18:14:58 +0800 Subject: [PATCH] Fixing memory overflow issues in ngx_resolver's debug logging code Message-ID: Hello! I've noticed a small memory overflow issue in ngx_resolver's debug logging code that was caught by Valgrind/Memcheck on Linux x86_64. Basically, when calling ngx_log_debug6 from within ngx_resolver_process_response, the "%ui" formatter is incorrectly used for int-typed values "(query->nns_hi << 8) + query->nns_lo" and "(query->nar_hi << 8) + query->nar_lo". Below attaches a patch for nginx 1.3.0 :) Hope this helps, -agentzh --- nginx-1.3.0/src/core/ngx_resolver.c 2012-05-14 17:13:45.000000000 +0800 +++ nginx-1.3.0-patched/src/core/ngx_resolver.c 2012-06-01 18:08:06.512047421 +0800 @@ -1035,7 +1035,7 @@ nan = (query->nan_hi << 8) + query->nan_lo; ngx_log_debug6(NGX_LOG_DEBUG_CORE, r->log, 0, - "resolver DNS response %ui fl:%04Xui %ui/%ui/%ui/%ui", + "resolver DNS response %ui fl:%04Xui %ui/%ui/%ud/%ud", ident, flags, nqs, nan, (query->nns_hi << 8) + query->nns_lo, (query->nar_hi << 8) + query->nar_lo); -------------- next part -------------- A non-text attachment was scrubbed... Name: nginx-1.3.0-resolver_debug_log_overflow.patch Type: application/octet-stream Size: 594 bytes Desc: not available URL: From ru at nginx.com Fri Jun 1 11:35:10 2012 From: ru at nginx.com (ru at nginx.com) Date: Fri, 1 Jun 2012 11:35:10 +0000 Subject: [nginx] svn commit: r4653 - trunk/src/core Message-ID: <20120601113511.06A843F9F0F@mail.nginx.com> Author: ru Date: 2012-06-01 11:35:09 +0000 (Fri, 01 Jun 2012) New Revision: 4653 URL: http://trac.nginx.org/nginx/changeset/4653/nginx Log: Code reduction (no functional changes). Modified: trunk/src/core/ngx_inet.c Modified: trunk/src/core/ngx_inet.c =================================================================== --- trunk/src/core/ngx_inet.c 2012-05-30 12:43:27 UTC (rev 4652) +++ trunk/src/core/ngx_inet.c 2012-06-01 11:35:09 UTC (rev 4653) @@ -634,11 +634,8 @@ args = ngx_strlchr(host, last, '?'); if (args) { - if (uri == NULL) { + if (uri == NULL || args < uri) { uri = args; - - } else if (args < uri) { - uri = args; } } @@ -663,11 +660,6 @@ len = last - port; - if (len == 0) { - u->err = "invalid port"; - return NGX_ERROR; - } - n = ngx_atoi(port, len); if (n < 1 || n > 65535) { @@ -774,11 +766,7 @@ return NGX_OK; } - if (ngx_inet_resolve_host(pool, u) != NGX_OK) { - return NGX_ERROR; - } - - return NGX_OK; + return ngx_inet_resolve_host(pool, u); } @@ -827,11 +815,6 @@ len = last - port; - if (len == 0) { - u->err = "invalid port"; - return NGX_ERROR; - } - n = ngx_atoi(port, len); if (n < 1 || n > 65535) { From ru at nginx.com Fri Jun 1 14:59:44 2012 From: ru at nginx.com (Ruslan Ermilov) Date: Fri, 1 Jun 2012 18:59:44 +0400 Subject: [PATCH] Fixing memory overflow issues in ngx_resolver's debug logging code In-Reply-To: References: Message-ID: <20120601145944.GB32532@lo0.su> On Fri, Jun 01, 2012 at 06:14:58PM +0800, agentzh wrote: > I've noticed a small memory overflow issue in ngx_resolver's debug > logging code that was caught by Valgrind/Memcheck on Linux x86_64. Calling it "memory overflow" is misleading -- nginx doesn't write beyond some buffer of memory. Instead, nginx will try to access (read) more bytes of memory than expected (e.g., 8 instead of 4), and in theory it can access unallocated or uninitialized memory. In practice, there's no problem on PC because memory is allocated in pages, allocations on stack are aligned, and stack memory pages are pre-zeroed when allocated. > Basically, when calling ngx_log_debug6 from within > ngx_resolver_process_response, the "%ui" formatter is incorrectly used > for int-typed values "(query->nns_hi << 8) + query->nns_lo" and > "(query->nar_hi << 8) + query->nar_lo". > > Below attaches a patch for nginx 1.3.0 :) > > Hope this helps, > -agentzh > > --- nginx-1.3.0/src/core/ngx_resolver.c 2012-05-14 17:13:45.000000000 +0800 > +++ nginx-1.3.0-patched/src/core/ngx_resolver.c 2012-06-01 > 18:08:06.512047421 +0800 > @@ -1035,7 +1035,7 @@ > nan = (query->nan_hi << 8) + query->nan_lo; > > ngx_log_debug6(NGX_LOG_DEBUG_CORE, r->log, 0, > - "resolver DNS response %ui fl:%04Xui %ui/%ui/%ui/%ui", > + "resolver DNS response %ui fl:%04Xui %ui/%ui/%ud/%ud", > ident, flags, nqs, nan, > (query->nns_hi << 8) + query->nns_lo, > (query->nar_hi << 8) + query->nar_lo); I think a better approach would be to cast the last two expressions to ngx_uint_t, like is done for other expressions (via assignments): %%% Index: src/core/ngx_resolver.c =================================================================== --- src/core/ngx_resolver.c (revision 4653) +++ src/core/ngx_resolver.c (working copy) @@ -1042,8 +1042,8 @@ ngx_resolver_process_response(ngx_resolv ngx_log_debug6(NGX_LOG_DEBUG_CORE, r->log, 0, "resolver DNS response %ui fl:%04Xui %ui/%ui/%ui/%ui", ident, flags, nqs, nan, - (query->nns_hi << 8) + query->nns_lo, - (query->nar_hi << 8) + query->nar_lo); + (ngx_uint_t) ((query->nns_hi << 8) + query->nns_lo), + (ngx_uint_t) ((query->nar_hi << 8) + query->nar_lo)); if (!(flags & 0x8000)) { ngx_log_error(r->log_level, r->log, 0, %%% From mdounin at mdounin.ru Fri Jun 1 16:27:55 2012 From: mdounin at mdounin.ru (Maxim Dounin) Date: Fri, 1 Jun 2012 20:27:55 +0400 Subject: [PATCH] Fixing memory overflow issues in ngx_resolver's debug logging code In-Reply-To: <20120601145944.GB32532@lo0.su> References: <20120601145944.GB32532@lo0.su> Message-ID: <20120601162755.GE31671@mdounin.ru> Hello! On Fri, Jun 01, 2012 at 06:59:44PM +0400, Ruslan Ermilov wrote: > > Basically, when calling ngx_log_debug6 from within > > ngx_resolver_process_response, the "%ui" formatter is incorrectly used > > for int-typed values "(query->nns_hi << 8) + query->nns_lo" and > > "(query->nar_hi << 8) + query->nar_lo". > > > > Below attaches a patch for nginx 1.3.0 :) > > > > Hope this helps, > > -agentzh > > > > --- nginx-1.3.0/src/core/ngx_resolver.c 2012-05-14 17:13:45.000000000 +0800 > > +++ nginx-1.3.0-patched/src/core/ngx_resolver.c 2012-06-01 > > 18:08:06.512047421 +0800 > > @@ -1035,7 +1035,7 @@ > > nan = (query->nan_hi << 8) + query->nan_lo; > > > > ngx_log_debug6(NGX_LOG_DEBUG_CORE, r->log, 0, > > - "resolver DNS response %ui fl:%04Xui %ui/%ui/%ui/%ui", > > + "resolver DNS response %ui fl:%04Xui %ui/%ui/%ud/%ud", > > ident, flags, nqs, nan, > > (query->nns_hi << 8) + query->nns_lo, > > (query->nar_hi << 8) + query->nar_lo); > > I think a better approach would be to cast the last two expressions > to ngx_uint_t, like is done for other expressions (via assignments): I don't really see a reason for explicit cast, using correct format is enough. It's highly unlikely we'll ever use nns/nar, so the probability that they'll become variables and %ui format will be appropriate is negligible. Maxim Dounin From mdounin at mdounin.ru Sun Jun 3 23:18:25 2012 From: mdounin at mdounin.ru (mdounin at mdounin.ru) Date: Sun, 3 Jun 2012 23:18:25 +0000 Subject: [nginx] svn commit: r4654 - trunk/src/core Message-ID: <20120603231826.215C83F9E76@mail.nginx.com> Author: mdounin Date: 2012-06-03 23:18:24 +0000 (Sun, 03 Jun 2012) New Revision: 4654 URL: http://trac.nginx.org/nginx/changeset/4654/nginx Log: Resolver: fixed format specification. Patch by Yichun Zhang (agentzh). Modified: trunk/src/core/ngx_resolver.c Modified: trunk/src/core/ngx_resolver.c =================================================================== --- trunk/src/core/ngx_resolver.c 2012-06-01 11:35:09 UTC (rev 4653) +++ trunk/src/core/ngx_resolver.c 2012-06-03 23:18:24 UTC (rev 4654) @@ -1040,7 +1040,7 @@ nan = (query->nan_hi << 8) + query->nan_lo; ngx_log_debug6(NGX_LOG_DEBUG_CORE, r->log, 0, - "resolver DNS response %ui fl:%04Xui %ui/%ui/%ui/%ui", + "resolver DNS response %ui fl:%04Xui %ui/%ui/%ud/%ud", ident, flags, nqs, nan, (query->nns_hi << 8) + query->nns_lo, (query->nar_hi << 8) + query->nar_lo); From mdounin at mdounin.ru Sun Jun 3 23:21:26 2012 From: mdounin at mdounin.ru (mdounin at mdounin.ru) Date: Sun, 3 Jun 2012 23:21:26 +0000 Subject: [nginx] svn commit: r4655 - in trunk: auto src/http/modules Message-ID: <20120603232126.BF4963F9E08@mail.nginx.com> Author: mdounin Date: 2012-06-03 23:21:26 +0000 (Sun, 03 Jun 2012) New Revision: 4655 URL: http://trac.nginx.org/nginx/changeset/4655/nginx Log: Upstream: least_conn balancer module. Added: trunk/src/http/modules/ngx_http_upstream_least_conn_module.c Modified: trunk/auto/modules trunk/auto/options trunk/auto/sources Modified: trunk/auto/modules =================================================================== --- trunk/auto/modules 2012-06-03 23:18:24 UTC (rev 4654) +++ trunk/auto/modules 2012-06-03 23:21:26 UTC (rev 4655) @@ -345,6 +345,11 @@ HTTP_SRCS="$HTTP_SRCS $HTTP_UPSTREAM_IP_HASH_SRCS" fi +if [ $HTTP_UPSTREAM_LEAST_CONN = YES ]; then + HTTP_MODULES="$HTTP_MODULES $HTTP_UPSTREAM_LEAST_CONN_MODULE" + HTTP_SRCS="$HTTP_SRCS $HTTP_UPSTREAM_LEAST_CONN_SRCS" +fi + if [ $HTTP_UPSTREAM_KEEPALIVE = YES ]; then HTTP_MODULES="$HTTP_MODULES $HTTP_UPSTREAM_KEEPALIVE_MODULE" HTTP_SRCS="$HTTP_SRCS $HTTP_UPSTREAM_KEEPALIVE_SRCS" Modified: trunk/auto/options =================================================================== --- trunk/auto/options 2012-06-03 23:18:24 UTC (rev 4654) +++ trunk/auto/options 2012-06-03 23:21:26 UTC (rev 4655) @@ -96,6 +96,7 @@ HTTP_MP4=NO HTTP_GZIP_STATIC=NO HTTP_UPSTREAM_IP_HASH=YES +HTTP_UPSTREAM_LEAST_CONN=YES HTTP_UPSTREAM_KEEPALIVE=YES # STUB @@ -243,6 +244,8 @@ --without-http_empty_gif_module) HTTP_EMPTY_GIF=NO ;; --without-http_browser_module) HTTP_BROWSER=NO ;; --without-http_upstream_ip_hash_module) HTTP_UPSTREAM_IP_HASH=NO ;; + --without-http_upstream_least_conn_module) + HTTP_UPSTREAM_LEAST_CONN=NO ;; --without-http_upstream_keepalive_module) HTTP_UPSTREAM_KEEPALIVE=NO ;; --with-http_perl_module) HTTP_PERL=YES ;; Modified: trunk/auto/sources =================================================================== --- trunk/auto/sources 2012-06-03 23:18:24 UTC (rev 4654) +++ trunk/auto/sources 2012-06-03 23:21:26 UTC (rev 4655) @@ -479,6 +479,11 @@ HTTP_UPSTREAM_IP_HASH_SRCS=src/http/modules/ngx_http_upstream_ip_hash_module.c +HTTP_UPSTREAM_LEAST_CONN_MODULE=ngx_http_upstream_least_conn_module +HTTP_UPSTREAM_LEAST_CONN_SRCS=" \ + src/http/modules/ngx_http_upstream_least_conn_module.c" + + HTTP_UPSTREAM_KEEPALIVE_MODULE=ngx_http_upstream_keepalive_module HTTP_UPSTREAM_KEEPALIVE_SRCS=" \ src/http/modules/ngx_http_upstream_keepalive_module.c" Added: trunk/src/http/modules/ngx_http_upstream_least_conn_module.c =================================================================== --- trunk/src/http/modules/ngx_http_upstream_least_conn_module.c (rev 0) +++ trunk/src/http/modules/ngx_http_upstream_least_conn_module.c 2012-06-03 23:21:26 UTC (rev 4655) @@ -0,0 +1,404 @@ + +/* + * Copyright (C) Maxim Dounin + * Copyright (C) Nginx, Inc. + */ + + +#include +#include +#include + + +typedef struct { + ngx_uint_t *conns; +} ngx_http_upstream_least_conn_conf_t; + + +typedef struct { + /* the round robin data must be first */ + ngx_http_upstream_rr_peer_data_t rrp; + + ngx_uint_t *conns; + + ngx_event_get_peer_pt get_rr_peer; + ngx_event_free_peer_pt free_rr_peer; +} ngx_http_upstream_lc_peer_data_t; + + +static ngx_int_t ngx_http_upstream_init_least_conn_peer(ngx_http_request_t *r, + ngx_http_upstream_srv_conf_t *us); +static ngx_int_t ngx_http_upstream_get_least_conn_peer( + ngx_peer_connection_t *pc, void *data); +static void ngx_http_upstream_free_least_conn_peer(ngx_peer_connection_t *pc, + void *data, ngx_uint_t state); +static void *ngx_http_upstream_least_conn_create_conf(ngx_conf_t *cf); +static char *ngx_http_upstream_least_conn(ngx_conf_t *cf, ngx_command_t *cmd, + void *conf); + + +static ngx_command_t ngx_http_upstream_least_conn_commands[] = { + + { ngx_string("least_conn"), + NGX_HTTP_UPS_CONF|NGX_CONF_NOARGS, + ngx_http_upstream_least_conn, + 0, + 0, + NULL }, + + ngx_null_command +}; + + +static ngx_http_module_t ngx_http_upstream_least_conn_module_ctx = { + NULL, /* preconfiguration */ + NULL, /* postconfiguration */ + + NULL, /* create main configuration */ + NULL, /* init main configuration */ + + ngx_http_upstream_least_conn_create_conf, /* create server configuration */ + NULL, /* merge server configuration */ + + NULL, /* create location configuration */ + NULL /* merge location configuration */ +}; + + +ngx_module_t ngx_http_upstream_least_conn_module = { + NGX_MODULE_V1, + &ngx_http_upstream_least_conn_module_ctx, /* module context */ + ngx_http_upstream_least_conn_commands, /* module directives */ + NGX_HTTP_MODULE, /* module type */ + NULL, /* init master */ + NULL, /* init module */ + NULL, /* init process */ + NULL, /* init thread */ + NULL, /* exit thread */ + NULL, /* exit process */ + NULL, /* exit master */ + NGX_MODULE_V1_PADDING +}; + + +ngx_int_t +ngx_http_upstream_init_least_conn(ngx_conf_t *cf, + ngx_http_upstream_srv_conf_t *us) +{ + ngx_uint_t n; + ngx_http_upstream_rr_peers_t *peers; + ngx_http_upstream_least_conn_conf_t *lcf; + + ngx_log_debug0(NGX_LOG_DEBUG_HTTP, cf->log, 0, + "init least conn"); + + if (ngx_http_upstream_init_round_robin(cf, us) != NGX_OK) { + return NGX_ERROR; + } + + peers = us->peer.data; + + n = peers->number; + + if (peers->next) { + n += peers->next->number; + } + + lcf = ngx_http_conf_upstream_srv_conf(us, + ngx_http_upstream_least_conn_module); + + lcf->conns = ngx_pcalloc(cf->pool, sizeof(ngx_uint_t) * n); + if (lcf->conns == NULL) { + return NGX_ERROR; + } + + us->peer.init = ngx_http_upstream_init_least_conn_peer; + + return NGX_OK; +} + + +static ngx_int_t +ngx_http_upstream_init_least_conn_peer(ngx_http_request_t *r, + ngx_http_upstream_srv_conf_t *us) +{ + ngx_int_t rc; + ngx_http_upstream_lc_peer_data_t *lcp; + ngx_http_upstream_least_conn_conf_t *lcf; + + ngx_log_debug0(NGX_LOG_DEBUG_HTTP, r->connection->log, 0, + "init least conn peer"); + + lcf = ngx_http_conf_upstream_srv_conf(us, + ngx_http_upstream_least_conn_module); + + lcp = ngx_palloc(r->pool, sizeof(ngx_http_upstream_lc_peer_data_t)); + if (lcp == NULL) { + return NGX_ERROR; + } + + lcp->conns = lcf->conns; + + r->upstream->peer.data = &lcp->rrp; + + rc = ngx_http_upstream_init_round_robin_peer(r, us); + + if (ngx_http_upstream_init_round_robin_peer(r, us) != NGX_OK) { + return NGX_ERROR; + } + + r->upstream->peer.get = ngx_http_upstream_get_least_conn_peer; + r->upstream->peer.free = ngx_http_upstream_free_least_conn_peer; + + lcp->get_rr_peer = ngx_http_upstream_get_round_robin_peer; + lcp->free_rr_peer = ngx_http_upstream_free_round_robin_peer; + + return NGX_OK; +} + + +static ngx_int_t +ngx_http_upstream_get_least_conn_peer(ngx_peer_connection_t *pc, void *data) +{ + ngx_http_upstream_lc_peer_data_t *lcp = data; + + time_t now; + uintptr_t m; + ngx_int_t rc, total; + ngx_uint_t i, n, p, many; + ngx_http_upstream_rr_peer_t *peer, *best; + ngx_http_upstream_rr_peers_t *peers; + + ngx_log_debug1(NGX_LOG_DEBUG_HTTP, pc->log, 0, + "get least conn peer, try: %ui", pc->tries); + + if (lcp->rrp.peers->single) { + return lcp->get_rr_peer(pc, &lcp->rrp); + } + + pc->cached = 0; + pc->connection = NULL; + + now = ngx_time(); + + peers = lcp->rrp.peers; + + best = NULL; + total = 0; + +#if (NGX_SUPPRESS_WARN) + many = 0; + p = 0; +#endif + + for (i = 0; i < peers->number; i++) { + + n = i / (8 * sizeof(uintptr_t)); + m = (uintptr_t) 1 << i % (8 * sizeof(uintptr_t)); + + if (lcp->rrp.tried[n] & m) { + continue; + } + + peer = &peers->peer[i]; + + if (peer->down) { + continue; + } + + if (peer->max_fails + && peer->fails >= peer->max_fails + && now - peer->checked <= peer->fail_timeout) + { + continue; + } + + /* + * select peer with least number of connections; if there are + * multiple peers with the same number of connections, select + * based on round-robin + */ + + if (best == NULL + || lcp->conns[i] * best->weight < lcp->conns[p] * peer->weight) + { + best = peer; + many = 0; + p = i; + + } else if (lcp->conns[i] * best->weight + == lcp->conns[p] * peer->weight) + { + many = 1; + } + } + + if (best == NULL) { + ngx_log_debug0(NGX_LOG_DEBUG_HTTP, pc->log, 0, + "get least conn peer, no peer found"); + + goto failed; + } + + if (many) { + ngx_log_debug0(NGX_LOG_DEBUG_HTTP, pc->log, 0, + "get least conn peer, many"); + + for (i = p; i < peers->number; i++) { + + n = i / (8 * sizeof(uintptr_t)); + m = (uintptr_t) 1 << i % (8 * sizeof(uintptr_t)); + + if (lcp->rrp.tried[n] & m) { + continue; + } + + peer = &peers->peer[i]; + + if (peer->down) { + continue; + } + + if (lcp->conns[i] * best->weight != lcp->conns[p] * peer->weight) { + continue; + } + + if (peer->max_fails + && peer->fails >= peer->max_fails + && now - peer->checked <= peer->fail_timeout) + { + continue; + } + + peer->current_weight += peer->effective_weight; + total += peer->effective_weight; + + if (peer->effective_weight < peer->weight) { + peer->effective_weight++; + } + + if (peer->current_weight > best->current_weight) { + best = peer; + p = i; + } + } + } + + best->current_weight -= total; + best->checked = now; + + pc->sockaddr = best->sockaddr; + pc->socklen = best->socklen; + pc->name = &best->name; + + lcp->rrp.current = p; + + n = p / (8 * sizeof(uintptr_t)); + m = (uintptr_t) 1 << p % (8 * sizeof(uintptr_t)); + + lcp->rrp.tried[n] |= m; + lcp->conns[p]++; + + if (pc->tries == 1 && peers->next) { + pc->tries += peers->next->number; + } + + return NGX_OK; + +failed: + + if (peers->next) { + ngx_log_debug0(NGX_LOG_DEBUG_HTTP, pc->log, 0, + "get least conn peer, backup servers"); + + lcp->conns += peers->number; + + lcp->rrp.peers = peers->next; + pc->tries = lcp->rrp.peers->number; + + n = lcp->rrp.peers->number / (8 * sizeof(uintptr_t)) + 1; + for (i = 0; i < n; i++) { + lcp->rrp.tried[i] = 0; + } + + rc = ngx_http_upstream_get_least_conn_peer(pc, lcp); + + if (rc != NGX_BUSY) { + return rc; + } + } + + /* all peers failed, mark them as live for quick recovery */ + + for (i = 0; i < peers->number; i++) { + peers->peer[i].fails = 0; + } + + pc->name = peers->name; + + return NGX_BUSY; +} + + +static void +ngx_http_upstream_free_least_conn_peer(ngx_peer_connection_t *pc, + void *data, ngx_uint_t state) +{ + ngx_http_upstream_lc_peer_data_t *lcp = data; + + ngx_log_debug2(NGX_LOG_DEBUG_HTTP, pc->log, 0, + "free least conn peer %ui %ui", pc->tries, state); + + if (lcp->rrp.peers->single) { + return lcp->free_rr_peer(pc, &lcp->rrp, state); + } + + if (state == 0 && pc->tries == 0) { + return; + } + + lcp->conns[lcp->rrp.current]--; + + return lcp->free_rr_peer(pc, &lcp->rrp, state); +} + + +static void * +ngx_http_upstream_least_conn_create_conf(ngx_conf_t *cf) +{ + ngx_http_upstream_least_conn_conf_t *conf; + + conf = ngx_pcalloc(cf->pool, + sizeof(ngx_http_upstream_least_conn_conf_t)); + if (conf == NULL) { + return NULL; + } + + /* + * set by ngx_pcalloc(): + * + * conf->conns = NULL; + */ + + return conf; +} + + +static char * +ngx_http_upstream_least_conn(ngx_conf_t *cf, ngx_command_t *cmd, void *conf) +{ + ngx_http_upstream_srv_conf_t *uscf; + + uscf = ngx_http_conf_get_module_srv_conf(cf, ngx_http_upstream_module); + + uscf->peer.init_upstream = ngx_http_upstream_init_least_conn; + + uscf->flags = NGX_HTTP_UPSTREAM_CREATE + |NGX_HTTP_UPSTREAM_WEIGHT + |NGX_HTTP_UPSTREAM_MAX_FAILS + |NGX_HTTP_UPSTREAM_FAIL_TIMEOUT + |NGX_HTTP_UPSTREAM_DOWN + |NGX_HTTP_UPSTREAM_BACKUP; + + return NGX_CONF_OK; +} From mdounin at mdounin.ru Sun Jun 3 23:22:41 2012 From: mdounin at mdounin.ru (mdounin at mdounin.ru) Date: Sun, 3 Jun 2012 23:22:41 +0000 Subject: [nginx] svn commit: r4656 - in trunk/src/http: . modules Message-ID: <20120603232241.93CF53F9E08@mail.nginx.com> Author: mdounin Date: 2012-06-03 23:22:41 +0000 (Sun, 03 Jun 2012) New Revision: 4656 URL: http://trac.nginx.org/nginx/changeset/4656/nginx Log: Upstream: weights support in ip_hash balancer. Modified: trunk/src/http/modules/ngx_http_upstream_ip_hash_module.c trunk/src/http/ngx_http_upstream_round_robin.c trunk/src/http/ngx_http_upstream_round_robin.h Modified: trunk/src/http/modules/ngx_http_upstream_ip_hash_module.c =================================================================== --- trunk/src/http/modules/ngx_http_upstream_ip_hash_module.c 2012-06-03 23:21:26 UTC (rev 4655) +++ trunk/src/http/modules/ngx_http_upstream_ip_hash_module.c 2012-06-03 23:22:41 UTC (rev 4656) @@ -140,6 +140,7 @@ ngx_http_upstream_ip_hash_peer_data_t *iphp = data; time_t now; + ngx_int_t w; uintptr_t m; ngx_uint_t i, n, p, hash; ngx_http_upstream_rr_peer_t *peer; @@ -166,8 +167,22 @@ hash = (hash * 113 + iphp->addr[i]) % 6271; } - p = hash % iphp->rrp.peers->number; + if (!iphp->rrp.peers->weighted) { + p = hash % iphp->rrp.peers->number; + } else { + w = hash % iphp->rrp.peers->total_weight; + + for (i = 0; i < iphp->rrp.peers->number; i++) { + w -= iphp->rrp.peers->peer[i].weight; + if (w < 0) { + break; + } + } + + p = i; + } + n = p / (8 * sizeof(uintptr_t)); m = (uintptr_t) 1 << p % (8 * sizeof(uintptr_t)); @@ -229,6 +244,7 @@ uscf->peer.init_upstream = ngx_http_upstream_init_ip_hash; uscf->flags = NGX_HTTP_UPSTREAM_CREATE + |NGX_HTTP_UPSTREAM_WEIGHT |NGX_HTTP_UPSTREAM_MAX_FAILS |NGX_HTTP_UPSTREAM_FAIL_TIMEOUT |NGX_HTTP_UPSTREAM_DOWN; Modified: trunk/src/http/ngx_http_upstream_round_robin.c =================================================================== --- trunk/src/http/ngx_http_upstream_round_robin.c 2012-06-03 23:21:26 UTC (rev 4655) +++ trunk/src/http/ngx_http_upstream_round_robin.c 2012-06-03 23:22:41 UTC (rev 4656) @@ -30,7 +30,7 @@ ngx_http_upstream_srv_conf_t *us) { ngx_url_t u; - ngx_uint_t i, j, n; + ngx_uint_t i, j, n, w; ngx_http_upstream_server_t *server; ngx_http_upstream_rr_peers_t *peers, *backup; @@ -40,6 +40,7 @@ server = us->servers->elts; n = 0; + w = 0; for (i = 0; i < us->servers->nelts; i++) { if (server[i].backup) { @@ -47,6 +48,7 @@ } n += server[i].naddrs; + w += server[i].naddrs * server[i].weight; } if (n == 0) { @@ -64,6 +66,8 @@ peers->single = (n == 1); peers->number = n; + peers->weighted = (w != n); + peers->total_weight = w; peers->name = &us->host; n = 0; @@ -96,6 +100,7 @@ /* backup servers */ n = 0; + w = 0; for (i = 0; i < us->servers->nelts; i++) { if (!server[i].backup) { @@ -103,6 +108,7 @@ } n += server[i].naddrs; + w += server[i].naddrs * server[i].weight; } if (n == 0) { @@ -118,6 +124,8 @@ peers->single = 0; backup->single = 0; backup->number = n; + backup->weighted = (w != n); + backup->total_weight = w; backup->name = &us->host; n = 0; @@ -185,6 +193,8 @@ peers->single = (n == 1); peers->number = n; + peers->weighted = 0; + peers->total_weight = n; peers->name = &us->host; for (i = 0; i < u.naddrs; i++) { Modified: trunk/src/http/ngx_http_upstream_round_robin.h =================================================================== --- trunk/src/http/ngx_http_upstream_round_robin.h 2012-06-03 23:21:26 UTC (rev 4655) +++ trunk/src/http/ngx_http_upstream_round_robin.h 2012-06-03 23:22:41 UTC (rev 4656) @@ -41,13 +41,17 @@ typedef struct ngx_http_upstream_rr_peers_s ngx_http_upstream_rr_peers_t; struct ngx_http_upstream_rr_peers_s { - ngx_uint_t single; /* unsigned single:1; */ ngx_uint_t number; ngx_uint_t last_cached; /* ngx_mutex_t *mutex; */ ngx_connection_t **cached; + ngx_uint_t total_weight; + + unsigned single:1; + unsigned weighted:1; + ngx_str_t *name; ngx_http_upstream_rr_peers_t *next; From mdounin at mdounin.ru Sun Jun 3 23:23:32 2012 From: mdounin at mdounin.ru (Maxim Dounin) Date: Mon, 4 Jun 2012 03:23:32 +0400 Subject: [PATCH] Fixing memory overflow issues in ngx_resolver's debug logging code In-Reply-To: References: Message-ID: <20120603232332.GJ31671@mdounin.ru> Hello! On Fri, Jun 01, 2012 at 06:14:58PM +0800, agentzh wrote: > Hello! > > I've noticed a small memory overflow issue in ngx_resolver's debug > logging code that was caught by Valgrind/Memcheck on Linux x86_64. > > Basically, when calling ngx_log_debug6 from within > ngx_resolver_process_response, the "%ui" formatter is incorrectly used > for int-typed values "(query->nns_hi << 8) + query->nns_lo" and > "(query->nar_hi << 8) + query->nar_lo". > > Below attaches a patch for nginx 1.3.0 :) > > Hope this helps, > -agentzh > > --- nginx-1.3.0/src/core/ngx_resolver.c 2012-05-14 17:13:45.000000000 +0800 > +++ nginx-1.3.0-patched/src/core/ngx_resolver.c 2012-06-01 > 18:08:06.512047421 +0800 > @@ -1035,7 +1035,7 @@ > nan = (query->nan_hi << 8) + query->nan_lo; > > ngx_log_debug6(NGX_LOG_DEBUG_CORE, r->log, 0, > - "resolver DNS response %ui fl:%04Xui %ui/%ui/%ui/%ui", > + "resolver DNS response %ui fl:%04Xui %ui/%ui/%ud/%ud", > ident, flags, nqs, nan, > (query->nns_hi << 8) + query->nns_lo, > (query->nar_hi << 8) + query->nar_lo); Committed, thnx. Maxim Dounin From mdounin at mdounin.ru Mon Jun 4 00:00:31 2012 From: mdounin at mdounin.ru (mdounin at mdounin.ru) Date: Mon, 4 Jun 2012 00:00:31 +0000 Subject: [nginx] svn commit: r4657 - trunk/src/http/modules Message-ID: <20120604000032.0B07E3F9C1B@mail.nginx.com> Author: mdounin Date: 2012-06-04 00:00:28 +0000 (Mon, 04 Jun 2012) New Revision: 4657 URL: http://trac.nginx.org/nginx/changeset/4657/nginx Log: Upstream: least_conn compilation fixes. Removed duplicate call of ngx_http_upstream_init_round_robin_peer() overlooked during code changes. Rewritten "return lcp->free_rr_peer(...)" as MSVC doesn't like it. Modified: trunk/src/http/modules/ngx_http_upstream_least_conn_module.c Modified: trunk/src/http/modules/ngx_http_upstream_least_conn_module.c =================================================================== --- trunk/src/http/modules/ngx_http_upstream_least_conn_module.c 2012-06-03 23:22:41 UTC (rev 4656) +++ trunk/src/http/modules/ngx_http_upstream_least_conn_module.c 2012-06-04 00:00:28 UTC (rev 4657) @@ -122,7 +122,6 @@ ngx_http_upstream_init_least_conn_peer(ngx_http_request_t *r, ngx_http_upstream_srv_conf_t *us) { - ngx_int_t rc; ngx_http_upstream_lc_peer_data_t *lcp; ngx_http_upstream_least_conn_conf_t *lcf; @@ -141,8 +140,6 @@ r->upstream->peer.data = &lcp->rrp; - rc = ngx_http_upstream_init_round_robin_peer(r, us); - if (ngx_http_upstream_init_round_robin_peer(r, us) != NGX_OK) { return NGX_ERROR; } @@ -350,7 +347,8 @@ "free least conn peer %ui %ui", pc->tries, state); if (lcp->rrp.peers->single) { - return lcp->free_rr_peer(pc, &lcp->rrp, state); + lcp->free_rr_peer(pc, &lcp->rrp, state); + return; } if (state == 0 && pc->tries == 0) { @@ -359,7 +357,7 @@ lcp->conns[lcp->rrp.current]--; - return lcp->free_rr_peer(pc, &lcp->rrp, state); + lcp->free_rr_peer(pc, &lcp->rrp, state); } From mdounin at mdounin.ru Mon Jun 4 10:00:39 2012 From: mdounin at mdounin.ru (mdounin at mdounin.ru) Date: Mon, 4 Jun 2012 10:00:39 +0000 Subject: [nginx] svn commit: r4658 - in branches/stable-1.2/src: core http/modules/perl Message-ID: <20120604100039.C50413F9CB1@mail.nginx.com> Author: mdounin Date: 2012-06-04 10:00:39 +0000 (Mon, 04 Jun 2012) New Revision: 4658 URL: http://trac.nginx.org/nginx/changeset/4658/nginx Log: Version bump. Modified: branches/stable-1.2/src/core/nginx.h branches/stable-1.2/src/http/modules/perl/nginx.pm Modified: branches/stable-1.2/src/core/nginx.h =================================================================== --- branches/stable-1.2/src/core/nginx.h 2012-06-04 00:00:28 UTC (rev 4657) +++ branches/stable-1.2/src/core/nginx.h 2012-06-04 10:00:39 UTC (rev 4658) @@ -9,8 +9,8 @@ #define _NGINX_H_INCLUDED_ -#define nginx_version 1002000 -#define NGINX_VERSION "1.2.0" +#define nginx_version 1002001 +#define NGINX_VERSION "1.2.1" #define NGINX_VER "nginx/" NGINX_VERSION #define NGINX_VAR "NGINX" Modified: branches/stable-1.2/src/http/modules/perl/nginx.pm =================================================================== --- branches/stable-1.2/src/http/modules/perl/nginx.pm 2012-06-04 00:00:28 UTC (rev 4657) +++ branches/stable-1.2/src/http/modules/perl/nginx.pm 2012-06-04 10:00:39 UTC (rev 4658) @@ -50,7 +50,7 @@ HTTP_INSUFFICIENT_STORAGE ); -our $VERSION = '1.2.0'; +our $VERSION = '1.2.1'; require XSLoader; XSLoader::load('nginx', $VERSION); From mdounin at mdounin.ru Mon Jun 4 10:15:56 2012 From: mdounin at mdounin.ru (mdounin at mdounin.ru) Date: Mon, 4 Jun 2012 10:15:56 +0000 Subject: [nginx] svn commit: r4659 - in branches/stable-1.2: . src/core Message-ID: <20120604101556.305B83F9EA7@mail.nginx.com> Author: mdounin Date: 2012-06-04 10:15:55 +0000 (Mon, 04 Jun 2012) New Revision: 4659 URL: http://trac.nginx.org/nginx/changeset/4659/nginx Log: Merge of r4611, r4620: resolver fixes. *) Fixed segmentation fault in ngx_resolver_create_name_query(). If name passed for resolution was { 0, NULL } (e.g. as a result of name server returning CNAME pointing to ".") pointer wrapped to (void *) -1 resulting in segmentation fault on an attempt to dereference it. Reported by Lanshun Zhou. *) Resolver: protection from duplicate responses. If we already had CNAME in resolver node (i.e. rn->cnlen and rn->u.cname set), and got additional response with A record, it resulted in rn->cnlen set and rn->u.cname overwritten by rn->u.addr (or rn->u.addrs), causing segmentation fault later in ngx_resolver_free_node() on an attempt to free overwritten rn->u.cname. The opposite (i.e. CNAME got after A) might cause similar problems as well. Modified: branches/stable-1.2/ branches/stable-1.2/src/core/ngx_resolver.c Index: branches/stable-1.2 =================================================================== --- branches/stable-1.2 2012-06-04 10:00:39 UTC (rev 4658) +++ branches/stable-1.2 2012-06-04 10:15:55 UTC (rev 4659) Property changes on: branches/stable-1.2 ___________________________________________________________________ Added: svn:mergeinfo ## -0,0 +1 ## +/trunk:4611,4620 \ No newline at end of property Modified: branches/stable-1.2/src/core/ngx_resolver.c =================================================================== --- branches/stable-1.2/src/core/ngx_resolver.c 2012-06-04 10:00:39 UTC (rev 4658) +++ branches/stable-1.2/src/core/ngx_resolver.c 2012-06-04 10:15:55 UTC (rev 4659) @@ -513,8 +513,10 @@ /* lock alloc mutex */ - ngx_resolver_free_locked(r, rn->query); - rn->query = NULL; + if (rn->query) { + ngx_resolver_free_locked(r, rn->query); + rn->query = NULL; + } if (rn->cnlen) { ngx_resolver_free_locked(r, rn->u.cname); @@ -1409,6 +1411,9 @@ ngx_resolver_free(r, addrs); } + ngx_resolver_free(r, rn->query); + rn->query = NULL; + return; } else if (cname) { @@ -1441,6 +1446,9 @@ (void) ngx_resolve_name_locked(r, ctx); } + ngx_resolver_free(r, rn->query); + rn->query = NULL; + return; } @@ -1834,6 +1842,10 @@ p--; *p-- = '\0'; + if (ctx->name.len == 0) { + return NGX_DECLINED; + } + for (s = ctx->name.data + ctx->name.len - 1; s >= ctx->name.data; s--) { if (*s != '.') { *p = *s; From mdounin at mdounin.ru Mon Jun 4 10:27:00 2012 From: mdounin at mdounin.ru (mdounin at mdounin.ru) Date: Mon, 4 Jun 2012 10:27:00 +0000 Subject: [nginx] svn commit: r4660 - in branches/stable-1.2: . src/http/modules Message-ID: <20120604102700.CB72C3F9CB1@mail.nginx.com> Author: mdounin Date: 2012-06-04 10:27:00 +0000 (Mon, 04 Jun 2012) New Revision: 4660 URL: http://trac.nginx.org/nginx/changeset/4660/nginx Log: Merge of r4612: proper subrequest handling in various modules. Modified: branches/stable-1.2/ branches/stable-1.2/src/http/modules/ngx_http_flv_module.c branches/stable-1.2/src/http/modules/ngx_http_gzip_static_module.c branches/stable-1.2/src/http/modules/ngx_http_mp4_module.c branches/stable-1.2/src/http/modules/ngx_http_stub_status_module.c Index: branches/stable-1.2 =================================================================== --- branches/stable-1.2 2012-06-04 10:15:55 UTC (rev 4659) +++ branches/stable-1.2 2012-06-04 10:27:00 UTC (rev 4660) Property changes on: branches/stable-1.2 ___________________________________________________________________ Modified: svn:mergeinfo ## -1 +1 ## -/trunk:4611,4620 +/trunk:4611-4612,4620 \ No newline at end of property Modified: branches/stable-1.2/src/http/modules/ngx_http_flv_module.c =================================================================== --- branches/stable-1.2/src/http/modules/ngx_http_flv_module.c 2012-06-04 10:15:55 UTC (rev 4659) +++ branches/stable-1.2/src/http/modules/ngx_http_flv_module.c 2012-06-04 10:27:00 UTC (rev 4660) @@ -235,7 +235,7 @@ b->file_last = of.size; b->in_file = b->file_last ? 1: 0; - b->last_buf = 1; + b->last_buf = (r == r->main) ? 1 : 0; b->last_in_chain = 1; b->file->fd = of.fd; Modified: branches/stable-1.2/src/http/modules/ngx_http_gzip_static_module.c =================================================================== --- branches/stable-1.2/src/http/modules/ngx_http_gzip_static_module.c 2012-06-04 10:15:55 UTC (rev 4659) +++ branches/stable-1.2/src/http/modules/ngx_http_gzip_static_module.c 2012-06-04 10:27:00 UTC (rev 4660) @@ -245,7 +245,7 @@ b->file_last = of.size; b->in_file = b->file_last ? 1 : 0; - b->last_buf = 1; + b->last_buf = (r == r->main) ? 1 : 0; b->last_in_chain = 1; b->file->fd = of.fd; Modified: branches/stable-1.2/src/http/modules/ngx_http_mp4_module.c =================================================================== --- branches/stable-1.2/src/http/modules/ngx_http_mp4_module.c 2012-06-04 10:15:55 UTC (rev 4659) +++ branches/stable-1.2/src/http/modules/ngx_http_mp4_module.c 2012-06-04 10:27:00 UTC (rev 4660) @@ -616,7 +616,7 @@ b->file_last = of.size; b->in_file = b->file_last ? 1 : 0; - b->last_buf = 1; + b->last_buf = (r == r->main) ? 1 : 0; b->last_in_chain = 1; b->file->fd = of.fd; Modified: branches/stable-1.2/src/http/modules/ngx_http_stub_status_module.c =================================================================== --- branches/stable-1.2/src/http/modules/ngx_http_stub_status_module.c 2012-06-04 10:15:55 UTC (rev 4659) +++ branches/stable-1.2/src/http/modules/ngx_http_stub_status_module.c 2012-06-04 10:27:00 UTC (rev 4660) @@ -121,7 +121,7 @@ r->headers_out.status = NGX_HTTP_OK; r->headers_out.content_length_n = b->last - b->pos; - b->last_buf = 1; + b->last_buf = (r == r->main) ? 1 : 0; rc = ngx_http_send_header(r); From mdounin at mdounin.ru Mon Jun 4 10:33:39 2012 From: mdounin at mdounin.ru (mdounin at mdounin.ru) Date: Mon, 4 Jun 2012 10:33:39 +0000 Subject: [nginx] svn commit: r4661 - in branches/stable-1.2: . src/http Message-ID: <20120604103339.B83FC3F9E94@mail.nginx.com> Author: mdounin Date: 2012-06-04 10:33:38 +0000 (Mon, 04 Jun 2012) New Revision: 4661 URL: http://trac.nginx.org/nginx/changeset/4661/nginx Log: Merge of r4613: removed surplus condition. Modified: branches/stable-1.2/ branches/stable-1.2/src/http/ngx_http_request.c Index: branches/stable-1.2 =================================================================== --- branches/stable-1.2 2012-06-04 10:27:00 UTC (rev 4660) +++ branches/stable-1.2 2012-06-04 10:33:38 UTC (rev 4661) Property changes on: branches/stable-1.2 ___________________________________________________________________ Modified: svn:mergeinfo ## -1 +1 ## -/trunk:4611-4612,4620 +/trunk:4611-4613,4620 \ No newline at end of property Modified: branches/stable-1.2/src/http/ngx_http_request.c =================================================================== --- branches/stable-1.2/src/http/ngx_http_request.c 2012-06-04 10:27:00 UTC (rev 4660) +++ branches/stable-1.2/src/http/ngx_http_request.c 2012-06-04 10:33:38 UTC (rev 4661) @@ -2001,14 +2001,6 @@ return; } -#if (NGX_DEBUG) - if (r != c->data) { - ngx_log_debug2(NGX_LOG_DEBUG_HTTP, c->log, 0, - "http finalize non-active request: \"%V?%V\"", - &r->uri, &r->args); - } -#endif - pr = r->parent; if (r == c->data) { @@ -2042,6 +2034,10 @@ } else { + ngx_log_debug2(NGX_LOG_DEBUG_HTTP, c->log, 0, + "http finalize non-active request: \"%V?%V\"", + &r->uri, &r->args); + r->write_event_handler = ngx_http_request_finalizer; if (r->waited) { From mdounin at mdounin.ru Mon Jun 4 10:52:44 2012 From: mdounin at mdounin.ru (mdounin at mdounin.ru) Date: Mon, 4 Jun 2012 10:52:44 +0000 Subject: [nginx] svn commit: r4662 - in branches/stable-1.2: . src/http Message-ID: <20120604105244.5EC703F9E76@mail.nginx.com> Author: mdounin Date: 2012-06-04 10:52:43 +0000 (Mon, 04 Jun 2012) New Revision: 4662 URL: http://trac.nginx.org/nginx/changeset/4662/nginx Log: Merge of r4615: write handler reset in ngx_http_named_location(). On internal redirects this happens via ngx_http_handler() call, which is not called on named location redirect. As a result incorrect write handler remained (if previously set) and this might cause incorrect behaviour (likely request hang). Patch by Yichun Zhang (agentzh). Modified: branches/stable-1.2/ branches/stable-1.2/src/http/ngx_http_core_module.c Index: branches/stable-1.2 =================================================================== --- branches/stable-1.2 2012-06-04 10:33:38 UTC (rev 4661) +++ branches/stable-1.2 2012-06-04 10:52:43 UTC (rev 4662) Property changes on: branches/stable-1.2 ___________________________________________________________________ Modified: svn:mergeinfo ## -1 +1 ## -/trunk:4611-4613,4620 +/trunk:4611-4613,4615,4620 \ No newline at end of property Modified: branches/stable-1.2/src/http/ngx_http_core_module.c =================================================================== --- branches/stable-1.2/src/http/ngx_http_core_module.c 2012-06-04 10:33:38 UTC (rev 4661) +++ branches/stable-1.2/src/http/ngx_http_core_module.c 2012-06-04 10:52:43 UTC (rev 4662) @@ -2599,6 +2599,7 @@ r->phase_handler = cmcf->phase_engine.location_rewrite_index; + r->write_event_handler = ngx_http_core_run_phases; ngx_http_core_run_phases(r); return NGX_DONE; From mdounin at mdounin.ru Mon Jun 4 10:54:49 2012 From: mdounin at mdounin.ru (mdounin at mdounin.ru) Date: Mon, 4 Jun 2012 10:54:49 +0000 Subject: [nginx] svn commit: r4663 - in branches/stable-1.2: . src/http/modules Message-ID: <20120604105451.A1C8E3F9E76@mail.nginx.com> Author: mdounin Date: 2012-06-04 10:54:48 +0000 (Mon, 04 Jun 2012) New Revision: 4663 URL: http://trac.nginx.org/nginx/changeset/4663/nginx Log: Merge of r4616: r->state fix. Added r->state reset on fastcgi/scgi/uwsgi request start. Failing to do so results in problems if 400 or 414 requests are redirected to fastcgi/scgi/uwsgi upstream, as well as after invalid headers got from upstream. This was already fixed for proxy in r3478, but fastcgi (the only affected protocol at that time) was missed. Reported by Matthieu Tourne. Modified: branches/stable-1.2/ branches/stable-1.2/src/http/modules/ngx_http_fastcgi_module.c branches/stable-1.2/src/http/modules/ngx_http_scgi_module.c branches/stable-1.2/src/http/modules/ngx_http_uwsgi_module.c Index: branches/stable-1.2 =================================================================== --- branches/stable-1.2 2012-06-04 10:52:43 UTC (rev 4662) +++ branches/stable-1.2 2012-06-04 10:54:48 UTC (rev 4663) Property changes on: branches/stable-1.2 ___________________________________________________________________ Modified: svn:mergeinfo ## -1 +1 ## -/trunk:4611-4613,4615,4620 +/trunk:4611-4613,4615-4616,4620 \ No newline at end of property Modified: branches/stable-1.2/src/http/modules/ngx_http_fastcgi_module.c =================================================================== --- branches/stable-1.2/src/http/modules/ngx_http_fastcgi_module.c 2012-06-04 10:52:43 UTC (rev 4662) +++ branches/stable-1.2/src/http/modules/ngx_http_fastcgi_module.c 2012-06-04 10:54:48 UTC (rev 4663) @@ -619,6 +619,7 @@ u->process_header = ngx_http_fastcgi_process_header; u->abort_request = ngx_http_fastcgi_abort_request; u->finalize_request = ngx_http_fastcgi_finalize_request; + r->state = 0; u->buffering = 1; @@ -1194,6 +1195,8 @@ f->fastcgi_stdout = 0; f->large_stderr = 0; + r->state = 0; + return NGX_OK; } Modified: branches/stable-1.2/src/http/modules/ngx_http_scgi_module.c =================================================================== --- branches/stable-1.2/src/http/modules/ngx_http_scgi_module.c 2012-06-04 10:52:43 UTC (rev 4662) +++ branches/stable-1.2/src/http/modules/ngx_http_scgi_module.c 2012-06-04 10:54:48 UTC (rev 4663) @@ -434,6 +434,7 @@ u->process_header = ngx_http_scgi_process_status_line; u->abort_request = ngx_http_scgi_abort_request; u->finalize_request = ngx_http_scgi_finalize_request; + r->state = 0; u->buffering = scf->upstream.buffering; @@ -843,6 +844,7 @@ status->end = NULL; r->upstream->process_header = ngx_http_scgi_process_status_line; + r->state = 0; return NGX_OK; } Modified: branches/stable-1.2/src/http/modules/ngx_http_uwsgi_module.c =================================================================== --- branches/stable-1.2/src/http/modules/ngx_http_uwsgi_module.c 2012-06-04 10:52:43 UTC (rev 4662) +++ branches/stable-1.2/src/http/modules/ngx_http_uwsgi_module.c 2012-06-04 10:54:48 UTC (rev 4663) @@ -467,6 +467,7 @@ u->process_header = ngx_http_uwsgi_process_status_line; u->abort_request = ngx_http_uwsgi_abort_request; u->finalize_request = ngx_http_uwsgi_finalize_request; + r->state = 0; u->buffering = uwcf->upstream.buffering; @@ -883,6 +884,7 @@ status->end = NULL; r->upstream->process_header = ngx_http_uwsgi_process_status_line; + r->state = 0; return NGX_OK; } From mdounin at mdounin.ru Mon Jun 4 11:00:34 2012 From: mdounin at mdounin.ru (mdounin at mdounin.ru) Date: Mon, 4 Jun 2012 11:00:34 +0000 Subject: [nginx] svn commit: r4664 - in branches/stable-1.2: . src/http/modules Message-ID: <20120604110034.E92DD3F9E7A@mail.nginx.com> Author: mdounin Date: 2012-06-04 11:00:34 +0000 (Mon, 04 Jun 2012) New Revision: 4664 URL: http://trac.nginx.org/nginx/changeset/4664/nginx Log: Merge of r4617: fastcgi padding fix. Fastcgi: fixed padding handling on fixed-size records. Padding was incorrectly ignored on end request, empty stdout and stderr fastcgi records. This resulted in protocol desynchronization if fastcgi application used these records with padding for some reason. Reported by Ilia Vinokurov. Modified: branches/stable-1.2/ branches/stable-1.2/src/http/modules/ngx_http_fastcgi_module.c Index: branches/stable-1.2 =================================================================== --- branches/stable-1.2 2012-06-04 10:54:48 UTC (rev 4663) +++ branches/stable-1.2 2012-06-04 11:00:34 UTC (rev 4664) Property changes on: branches/stable-1.2 ___________________________________________________________________ Modified: svn:mergeinfo ## -1 +1 ## -/trunk:4611-4613,4615-4616,4620 +/trunk:4611-4613,4615-4617,4620 \ No newline at end of property Modified: branches/stable-1.2/src/http/modules/ngx_http_fastcgi_module.c =================================================================== --- branches/stable-1.2/src/http/modules/ngx_http_fastcgi_module.c 2012-06-04 10:54:48 UTC (rev 4663) +++ branches/stable-1.2/src/http/modules/ngx_http_fastcgi_module.c 2012-06-04 11:00:34 UTC (rev 4664) @@ -1356,7 +1356,11 @@ } } else { - f->state = ngx_http_fastcgi_st_version; + if (f->padding) { + f->state = ngx_http_fastcgi_st_padding; + } else { + f->state = ngx_http_fastcgi_st_version; + } } continue; @@ -1689,8 +1693,13 @@ } if (f->type == NGX_HTTP_FASTCGI_STDOUT && f->length == 0) { - f->state = ngx_http_fastcgi_st_version; + if (f->padding) { + f->state = ngx_http_fastcgi_st_padding; + } else { + f->state = ngx_http_fastcgi_st_version; + } + if (!flcf->keep_conn) { p->upstream_done = 1; } @@ -1702,7 +1711,13 @@ } if (f->type == NGX_HTTP_FASTCGI_END_REQUEST) { - f->state = ngx_http_fastcgi_st_version; + + if (f->padding) { + f->state = ngx_http_fastcgi_st_padding; + } else { + f->state = ngx_http_fastcgi_st_version; + } + p->upstream_done = 1; if (flcf->keep_conn) { @@ -1775,7 +1790,11 @@ } } else { - f->state = ngx_http_fastcgi_st_version; + if (f->padding) { + f->state = ngx_http_fastcgi_st_padding; + } else { + f->state = ngx_http_fastcgi_st_version; + } } continue; From mdounin at mdounin.ru Mon Jun 4 11:07:20 2012 From: mdounin at mdounin.ru (mdounin at mdounin.ru) Date: Mon, 4 Jun 2012 11:07:20 +0000 Subject: [nginx] svn commit: r4665 - in branches/stable-1.2: . src/http Message-ID: <20120604110720.7373C3F9EC2@mail.nginx.com> Author: mdounin Date: 2012-06-04 11:07:19 +0000 (Mon, 04 Jun 2012) New Revision: 4665 URL: http://trac.nginx.org/nginx/changeset/4665/nginx Log: Merge of r4618: rewrite escaping fix (ticket #162). The following code resulted in incorrect escaping of uri and possible segfault: location / { rewrite ^(.*) $1?c=$1; return 200 "$uri"; } If there were arguments in a rewrite's replacement string, and length was actually calculated (due to duplicate captures as in the example above, or variables present), the is_args flag was set and incorrectly copied after length calculation. This resulted in escaping applied to the uri part of the replacement, resulting in incorrect escaping. Additionally, buffer was allocated without escaping expected, thus this also resulted in buffer overrun and possible segfault. Modified: branches/stable-1.2/ branches/stable-1.2/src/http/ngx_http_script.c Index: branches/stable-1.2 =================================================================== --- branches/stable-1.2 2012-06-04 11:00:34 UTC (rev 4664) +++ branches/stable-1.2 2012-06-04 11:07:19 UTC (rev 4665) Property changes on: branches/stable-1.2 ___________________________________________________________________ Modified: svn:mergeinfo ## -1 +1 ## -/trunk:4611-4613,4615-4617,4620 +/trunk:4611-4613,4615-4618,4620 \ No newline at end of property Modified: branches/stable-1.2/src/http/ngx_http_script.c =================================================================== --- branches/stable-1.2/src/http/ngx_http_script.c 2012-06-04 11:00:34 UTC (rev 4664) +++ branches/stable-1.2/src/http/ngx_http_script.c 2012-06-04 11:07:19 UTC (rev 4665) @@ -1043,7 +1043,6 @@ } e->buf.len = len; - e->is_args = le.is_args; } if (code->add_args && r->args.len) { From mdounin at mdounin.ru Mon Jun 4 11:10:36 2012 From: mdounin at mdounin.ru (mdounin at mdounin.ru) Date: Mon, 4 Jun 2012 11:10:36 +0000 Subject: [nginx] svn commit: r4666 - in branches/stable-1.2: . src/event src/os/unix src/os/win32 Message-ID: <20120604111037.058F03F9F49@mail.nginx.com> Author: mdounin Date: 2012-06-04 11:10:36 +0000 (Mon, 04 Jun 2012) New Revision: 4666 URL: http://trac.nginx.org/nginx/changeset/4666/nginx Log: Merge of r4619: accept moderation on EMFILE/ENFILE. In case of EMFILE/ENFILE returned from accept() we disable accept events, and (in case of no accept mutex used) arm timer to re-enable them later. With accept mutex we just drop it, and rely on normal accept mutex handling to re-enable accept events once it's acquired again. As we now handle errors in question, logging level was changed to "crit" (instead of "alert" used for unknown errors). Note: the code might call ngx_enable_accept_events() multiple times if there are many listen sockets. The ngx_enable_accept_events() function was modified to check if connection is already active (via c->read->active) and skip it then, thus making multiple calls safe. Modified: branches/stable-1.2/ branches/stable-1.2/src/event/ngx_event_accept.c branches/stable-1.2/src/os/unix/ngx_errno.h branches/stable-1.2/src/os/win32/ngx_errno.h Index: branches/stable-1.2 =================================================================== --- branches/stable-1.2 2012-06-04 11:07:19 UTC (rev 4665) +++ branches/stable-1.2 2012-06-04 11:10:36 UTC (rev 4666) Property changes on: branches/stable-1.2 ___________________________________________________________________ Modified: svn:mergeinfo ## -1 +1 ## -/trunk:4611-4613,4615-4618,4620 +/trunk:4611-4613,4615-4620 \ No newline at end of property Modified: branches/stable-1.2/src/event/ngx_event_accept.c =================================================================== --- branches/stable-1.2/src/event/ngx_event_accept.c 2012-06-04 11:07:19 UTC (rev 4665) +++ branches/stable-1.2/src/event/ngx_event_accept.c 2012-06-04 11:10:36 UTC (rev 4666) @@ -21,6 +21,7 @@ socklen_t socklen; ngx_err_t err; ngx_log_t *log; + ngx_uint_t level; ngx_socket_t s; ngx_event_t *rev, *wev; ngx_listening_t *ls; @@ -31,6 +32,14 @@ static ngx_uint_t use_accept4 = 1; #endif + if (ev->timedout) { + if (ngx_enable_accept_events((ngx_cycle_t *) ngx_cycle) != NGX_OK) { + return; + } + + ev->timedout = 0; + } + ecf = ngx_event_get_conf(ngx_cycle->conf_ctx, ngx_event_core_module); if (ngx_event_flags & NGX_USE_RTSIG_EVENT) { @@ -70,10 +79,17 @@ return; } + level = NGX_LOG_ALERT; + + if (err == NGX_ECONNABORTED) { + level = NGX_LOG_ERR; + + } else if (err == NGX_EMFILE || err == NGX_ENFILE) { + level = NGX_LOG_CRIT; + } + #if (NGX_HAVE_ACCEPT4) - ngx_log_error((ngx_uint_t) ((err == NGX_ECONNABORTED) ? - NGX_LOG_ERR : NGX_LOG_ALERT), - ev->log, err, + ngx_log_error(level, ev->log, err, use_accept4 ? "accept4() failed" : "accept() failed"); if (use_accept4 && err == NGX_ENOSYS) { @@ -82,9 +98,7 @@ continue; } #else - ngx_log_error((ngx_uint_t) ((err == NGX_ECONNABORTED) ? - NGX_LOG_ERR : NGX_LOG_ALERT), - ev->log, err, "accept() failed"); + ngx_log_error(level, ev->log, err, "accept() failed"); #endif if (err == NGX_ECONNABORTED) { @@ -97,6 +111,26 @@ } } + if (err == NGX_EMFILE || err == NGX_ENFILE) { + if (ngx_disable_accept_events((ngx_cycle_t *) ngx_cycle) + != NGX_OK) + { + return; + } + + if (ngx_use_accept_mutex) { + if (ngx_accept_mutex_held) { + ngx_shmtx_unlock(&ngx_accept_mutex); + ngx_accept_mutex_held = 0; + } + + ngx_accept_disabled = 1; + + } else { + ngx_add_timer(ev, ecf->accept_mutex_delay); + } + } + return; } @@ -344,6 +378,10 @@ c = ls[i].connection; + if (c->read->active) { + continue; + } + if (ngx_event_flags & NGX_USE_RTSIG_EVENT) { if (ngx_add_conn(c) == NGX_ERROR) { Modified: branches/stable-1.2/src/os/unix/ngx_errno.h =================================================================== --- branches/stable-1.2/src/os/unix/ngx_errno.h 2012-06-04 11:07:19 UTC (rev 4665) +++ branches/stable-1.2/src/os/unix/ngx_errno.h 2012-06-04 11:10:36 UTC (rev 4666) @@ -29,6 +29,8 @@ #define NGX_ENOTDIR ENOTDIR #define NGX_EISDIR EISDIR #define NGX_EINVAL EINVAL +#define NGX_ENFILE ENFILE +#define NGX_EMFILE EMFILE #define NGX_ENOSPC ENOSPC #define NGX_EPIPE EPIPE #define NGX_EINPROGRESS EINPROGRESS Modified: branches/stable-1.2/src/os/win32/ngx_errno.h =================================================================== --- branches/stable-1.2/src/os/win32/ngx_errno.h 2012-06-04 11:07:19 UTC (rev 4665) +++ branches/stable-1.2/src/os/win32/ngx_errno.h 2012-06-04 11:10:36 UTC (rev 4666) @@ -54,6 +54,8 @@ #define NGX_EALREADY WSAEALREADY #define NGX_EINVAL WSAEINVAL +#define NGX_EMFILE WSAEMFILE +#define NGX_ENFILE WSAEMFILE u_char *ngx_strerror(ngx_err_t err, u_char *errstr, size_t size); From mdounin at mdounin.ru Mon Jun 4 11:15:47 2012 From: mdounin at mdounin.ru (mdounin at mdounin.ru) Date: Mon, 4 Jun 2012 11:15:47 +0000 Subject: [nginx] svn commit: r4667 - in branches/stable-1.2: . src/http Message-ID: <20120604111547.DDF3D3F9C18@mail.nginx.com> Author: mdounin Date: 2012-06-04 11:15:46 +0000 (Mon, 04 Jun 2012) New Revision: 4667 URL: http://trac.nginx.org/nginx/changeset/4667/nginx Log: Merge of r4621, r4641: filter_finalize fixes. *) Fixed possible request hang with filter finalization. With r->filter_finalize set the ngx_http_finalize_connection() wasn't called from ngx_http_finalize_request() called with NGX_OK, resulting in r->main->count not being decremented, thus causing request hang in some rare situations. Patch by Yichun Zhang (agentzh). *) Fixed segfault with filter_finalize introduced in r4621 (1.3.0). See the following thread for more details: http://mailman.nginx.org/pipermail/nginx-devel/2012-May/002190.html Modified: branches/stable-1.2/ branches/stable-1.2/src/http/ngx_http_request.c Index: branches/stable-1.2 =================================================================== --- branches/stable-1.2 2012-06-04 11:10:36 UTC (rev 4666) +++ branches/stable-1.2 2012-06-04 11:15:46 UTC (rev 4667) Property changes on: branches/stable-1.2 ___________________________________________________________________ Modified: svn:mergeinfo ## -1 +1 ## -/trunk:4611-4613,4615-4620 +/trunk:4611-4613,4615-4621,4641 \ No newline at end of property Modified: branches/stable-1.2/src/http/ngx_http_request.c =================================================================== --- branches/stable-1.2/src/http/ngx_http_request.c 2012-06-04 11:10:36 UTC (rev 4666) +++ branches/stable-1.2/src/http/ngx_http_request.c 2012-06-04 11:15:46 UTC (rev 4667) @@ -1933,7 +1933,6 @@ if (rc == NGX_OK && r->filter_finalize) { c->error = 1; - return; } if (rc == NGX_DECLINED) { From mdounin at mdounin.ru Mon Jun 4 11:22:00 2012 From: mdounin at mdounin.ru (mdounin at mdounin.ru) Date: Mon, 4 Jun 2012 11:22:00 +0000 Subject: [nginx] svn commit: r4668 - in branches/stable-1.2: . src/http Message-ID: <20120604112200.258373F9EFF@mail.nginx.com> Author: mdounin Date: 2012-06-04 11:21:58 +0000 (Mon, 04 Jun 2012) New Revision: 4668 URL: http://trac.nginx.org/nginx/changeset/4668/nginx Log: Merge of r4622, r4623: balancing changes. *) Upstream: smooth weighted round-robin balancing. For edge case weights like { 5, 1, 1 } we now produce { a, a, b, a, c, a, a } sequence instead of { c, b, a, a, a, a, a } produced previously. Algorithm is as follows: on each peer selection we increase current_weight of each eligible peer by its weight, select peer with greatest current_weight and reduce its current_weight by total number of weight points distributed among peers. In case of { 5, 1, 1 } weights this gives the following sequence of current_weight's: a b c 0 0 0 (initial state) 5 1 1 (a selected) -2 1 1 3 2 2 (a selected) -4 2 2 1 3 3 (b selected) 1 -4 3 6 -3 4 (a selected) -1 -3 4 4 -2 5 (c selected) 4 -2 -2 9 -1 -1 (a selected) 2 -1 -1 7 0 0 (a selected) 0 0 0 To preserve weight reduction in case of failures the effective_weight variable was introduced, which usually matches peer's weight, but is reduced temporarily on peer failures. This change also fixes loop with backup servers and proxy_next_upstream http_404 (ticket #47), and skipping alive upstreams in some cases if there are multiple dead ones (ticket #64). *) Upstream: fixed ip_hash rebalancing with the "down" flag. Due to weight being set to 0 for down peers, order of peers after sorting wasn't the same as without the "down" flag (with down peers at the end), resulting in client rebalancing for clients on other servers. The only rebalancing which should happen after adding "down" to a server is one for clients on the server. The problem was introduced in r1377 (which fixed endless loop by setting weight to 0 for down servers). The loop is no longer possible with new smooth algorithm, so preserving original weight is safe. Modified: branches/stable-1.2/ branches/stable-1.2/src/http/ngx_http_upstream_round_robin.c branches/stable-1.2/src/http/ngx_http_upstream_round_robin.h Index: branches/stable-1.2 =================================================================== --- branches/stable-1.2 2012-06-04 11:15:46 UTC (rev 4667) +++ branches/stable-1.2 2012-06-04 11:21:58 UTC (rev 4668) Property changes on: branches/stable-1.2 ___________________________________________________________________ Modified: svn:mergeinfo ## -1 +1 ## -/trunk:4611-4613,4615-4621,4641 +/trunk:4611-4613,4615-4623,4641 \ No newline at end of property Modified: branches/stable-1.2/src/http/ngx_http_upstream_round_robin.c =================================================================== --- branches/stable-1.2/src/http/ngx_http_upstream_round_robin.c 2012-06-04 11:15:46 UTC (rev 4667) +++ branches/stable-1.2/src/http/ngx_http_upstream_round_robin.c 2012-06-04 11:21:58 UTC (rev 4668) @@ -12,8 +12,8 @@ static ngx_int_t ngx_http_upstream_cmp_servers(const void *one, const void *two); -static ngx_uint_t -ngx_http_upstream_get_peer(ngx_http_upstream_rr_peers_t *peers); +static ngx_http_upstream_rr_peer_t *ngx_http_upstream_get_peer( + ngx_http_upstream_rr_peer_data_t *rrp); #if (NGX_HTTP_SSL) @@ -80,8 +80,9 @@ peers->peer[n].max_fails = server[i].max_fails; peers->peer[n].fail_timeout = server[i].fail_timeout; peers->peer[n].down = server[i].down; - peers->peer[n].weight = server[i].down ? 0 : server[i].weight; - peers->peer[n].current_weight = peers->peer[n].weight; + peers->peer[n].weight = server[i].weight; + peers->peer[n].effective_weight = server[i].weight; + peers->peer[n].current_weight = 0; n++; } } @@ -131,7 +132,8 @@ backup->peer[n].socklen = server[i].addrs[j].socklen; backup->peer[n].name = server[i].addrs[j].name; backup->peer[n].weight = server[i].weight; - backup->peer[n].current_weight = server[i].weight; + backup->peer[n].effective_weight = server[i].weight; + backup->peer[n].current_weight = 0; backup->peer[n].max_fails = server[i].max_fails; backup->peer[n].fail_timeout = server[i].fail_timeout; backup->peer[n].down = server[i].down; @@ -190,7 +192,8 @@ peers->peer[i].socklen = u.addrs[i].socklen; peers->peer[i].name = u.addrs[i].name; peers->peer[i].weight = 1; - peers->peer[i].current_weight = 1; + peers->peer[i].effective_weight = 1; + peers->peer[i].current_weight = 0; peers->peer[i].max_fails = 1; peers->peer[i].fail_timeout = 10; } @@ -306,7 +309,8 @@ peers->peer[0].socklen = ur->socklen; peers->peer[0].name = ur->host; peers->peer[0].weight = 1; - peers->peer[0].current_weight = 1; + peers->peer[0].effective_weight = 1; + peers->peer[0].current_weight = 0; peers->peer[0].max_fails = 1; peers->peer[0].fail_timeout = 10; @@ -338,7 +342,8 @@ peers->peer[i].name.len = len; peers->peer[i].name.data = p; peers->peer[i].weight = 1; - peers->peer[i].current_weight = 1; + peers->peer[i].effective_weight = 1; + peers->peer[i].current_weight = 0; peers->peer[i].max_fails = 1; peers->peer[i].fail_timeout = 10; } @@ -378,8 +383,6 @@ { ngx_http_upstream_rr_peer_data_t *rrp = data; - time_t now; - uintptr_t m; ngx_int_t rc; ngx_uint_t i, n; ngx_connection_t *c; @@ -389,8 +392,6 @@ ngx_log_debug1(NGX_LOG_DEBUG_HTTP, pc->log, 0, "get rr peer, try: %ui", pc->tries); - now = ngx_time(); - /* ngx_lock_mutex(rrp->peers->mutex); */ if (rrp->peers->last_cached) { @@ -423,118 +424,15 @@ /* there are several peers */ - if (pc->tries == rrp->peers->number) { + peer = ngx_http_upstream_get_peer(rrp); - /* it's a first try - get a current peer */ - - i = pc->tries; - - for ( ;; ) { - rrp->current = ngx_http_upstream_get_peer(rrp->peers); - - ngx_log_debug2(NGX_LOG_DEBUG_HTTP, pc->log, 0, - "get rr peer, current: %ui %i", - rrp->current, - rrp->peers->peer[rrp->current].current_weight); - - n = rrp->current / (8 * sizeof(uintptr_t)); - m = (uintptr_t) 1 << rrp->current % (8 * sizeof(uintptr_t)); - - if (!(rrp->tried[n] & m)) { - peer = &rrp->peers->peer[rrp->current]; - - if (!peer->down) { - - if (peer->max_fails == 0 - || peer->fails < peer->max_fails) - { - break; - } - - if (now - peer->checked > peer->fail_timeout) { - peer->checked = now; - break; - } - - peer->current_weight = 0; - - } else { - rrp->tried[n] |= m; - } - - pc->tries--; - } - - if (pc->tries == 0) { - goto failed; - } - - if (--i == 0) { - ngx_log_error(NGX_LOG_ALERT, pc->log, 0, - "round robin upstream stuck on %ui tries", - pc->tries); - goto failed; - } - } - - peer->current_weight--; - - } else { - - i = pc->tries; - - for ( ;; ) { - n = rrp->current / (8 * sizeof(uintptr_t)); - m = (uintptr_t) 1 << rrp->current % (8 * sizeof(uintptr_t)); - - if (!(rrp->tried[n] & m)) { - - peer = &rrp->peers->peer[rrp->current]; - - if (!peer->down) { - - if (peer->max_fails == 0 - || peer->fails < peer->max_fails) - { - break; - } - - if (now - peer->checked > peer->fail_timeout) { - peer->checked = now; - break; - } - - peer->current_weight = 0; - - } else { - rrp->tried[n] |= m; - } - - pc->tries--; - } - - rrp->current++; - - if (rrp->current >= rrp->peers->number) { - rrp->current = 0; - } - - if (pc->tries == 0) { - goto failed; - } - - if (--i == 0) { - ngx_log_error(NGX_LOG_ALERT, pc->log, 0, - "round robin upstream stuck on %ui tries", - pc->tries); - goto failed; - } - } - - peer->current_weight--; + if (peer == NULL) { + goto failed; } - rrp->tried[n] |= m; + ngx_log_debug2(NGX_LOG_DEBUG_HTTP, pc->log, 0, + "get rr peer, current: %ui %i", + rrp->current, peer->current_weight); } pc->sockaddr = peer->sockaddr; @@ -545,11 +443,6 @@ if (pc->tries == 1 && rrp->peers->next) { pc->tries += rrp->peers->next->number; - - n = rrp->peers->next->number / (8 * sizeof(uintptr_t)) + 1; - for (i = 0; i < n; i++) { - rrp->tried[i] = 0; - } } return NGX_OK; @@ -595,56 +488,71 @@ } -static ngx_uint_t -ngx_http_upstream_get_peer(ngx_http_upstream_rr_peers_t *peers) +static ngx_http_upstream_rr_peer_t * +ngx_http_upstream_get_peer(ngx_http_upstream_rr_peer_data_t *rrp) { - ngx_uint_t i, n, reset = 0; - ngx_http_upstream_rr_peer_t *peer; + time_t now; + uintptr_t m; + ngx_int_t total; + ngx_uint_t i, n; + ngx_http_upstream_rr_peer_t *peer, *best; - peer = &peers->peer[0]; + now = ngx_time(); - for ( ;; ) { + best = NULL; + total = 0; - for (i = 0; i < peers->number; i++) { + for (i = 0; i < rrp->peers->number; i++) { - if (peer[i].current_weight <= 0) { - continue; - } + n = i / (8 * sizeof(uintptr_t)); + m = (uintptr_t) 1 << i % (8 * sizeof(uintptr_t)); - n = i; + if (rrp->tried[n] & m) { + continue; + } - while (i < peers->number - 1) { + peer = &rrp->peers->peer[i]; - i++; + if (peer->down) { + continue; + } - if (peer[i].current_weight <= 0) { - continue; - } + if (peer->max_fails + && peer->fails >= peer->max_fails + && now - peer->checked <= peer->fail_timeout) + { + continue; + } - if (peer[n].current_weight * 1000 / peer[i].current_weight - > peer[n].weight * 1000 / peer[i].weight) - { - return n; - } + peer->current_weight += peer->effective_weight; + total += peer->effective_weight; - n = i; - } - - if (peer[i].current_weight > 0) { - n = i; - } - - return n; + if (peer->effective_weight < peer->weight) { + peer->effective_weight++; } - if (reset++) { - return 0; + if (best == NULL || peer->current_weight > best->current_weight) { + best = peer; } + } - for (i = 0; i < peers->number; i++) { - peer[i].current_weight = peer[i].weight; - } + if (best == NULL) { + return NULL; } + + i = best - &rrp->peers->peer[0]; + + rrp->current = i; + + n = i / (8 * sizeof(uintptr_t)); + m = (uintptr_t) 1 << i % (8 * sizeof(uintptr_t)); + + rrp->tried[n] |= m; + + best->current_weight -= total; + best->checked = now; + + return best; } @@ -683,15 +591,15 @@ peer->checked = now; if (peer->max_fails) { - peer->current_weight -= peer->weight / peer->max_fails; + peer->effective_weight -= peer->weight / peer->max_fails; } ngx_log_debug2(NGX_LOG_DEBUG_HTTP, pc->log, 0, "free rr peer failed: %ui %i", - rrp->current, peer->current_weight); + rrp->current, peer->effective_weight); - if (peer->current_weight < 0) { - peer->current_weight = 0; + if (peer->effective_weight < 0) { + peer->effective_weight = 0; } /* ngx_unlock_mutex(rrp->peers->mutex); */ @@ -705,12 +613,6 @@ } } - rrp->current++; - - if (rrp->current >= rrp->peers->number) { - rrp->current = 0; - } - if (pc->tries) { pc->tries--; } Modified: branches/stable-1.2/src/http/ngx_http_upstream_round_robin.h =================================================================== --- branches/stable-1.2/src/http/ngx_http_upstream_round_robin.h 2012-06-04 11:15:46 UTC (rev 4667) +++ branches/stable-1.2/src/http/ngx_http_upstream_round_robin.h 2012-06-04 11:21:58 UTC (rev 4668) @@ -20,6 +20,7 @@ ngx_str_t name; ngx_int_t current_weight; + ngx_int_t effective_weight; ngx_int_t weight; ngx_uint_t fails; From mdounin at mdounin.ru Mon Jun 4 11:58:12 2012 From: mdounin at mdounin.ru (mdounin at mdounin.ru) Date: Mon, 4 Jun 2012 11:58:12 +0000 Subject: [nginx] svn commit: r4669 - in branches/stable-1.2: . src/event src/http src/http/modules Message-ID: <20120604115813.A8D483F9F2C@mail.nginx.com> Author: mdounin Date: 2012-06-04 11:58:12 +0000 (Mon, 04 Jun 2012) New Revision: 4669 URL: http://trac.nginx.org/nginx/changeset/4669/nginx Log: Merge of r4614, r4624-r4629, r4631: proxy recursive changes. *) Added IPv6 and UNIX-domain socket support in "debug_connection" directive. *) New function ngx_http_get_forwarded_addr() to look up real client address. On input it takes an original address, string in the X-Forwarded-For format and its length, list of trusted proxies, and a flag indicating to perform the recursive search. On output it returns NGX_OK and the "deepest" valid address in a chain, or NGX_DECLINED. It supports AF_INET and AF_INET6. Additionally, original address and/or proxy may be specified as AF_UNIX. *) Realip: chains of trusted proxies and IPv6 support. The module now supports recursive search of client address through the chain of trusted proxies, controlled by the "real_ip_recursive" directive (closes #2). It also gets full IPv6 support (closes #44) and canonical value of the $client_addr variable on address change. Example: real_ip_header X-Forwarded-For; set_real_ip_from 127.0.0.0/8; set_real_ip_from ::1; set_real_ip_from unix:; real_ip_recursive on; *) Geo: chains of trusted proxies and partial IPv6 support. The module now supports recursive search of client address through the chain of trusted proxies, controlled by the "proxy_recursive" directive in the "geo" block. It also gets partial IPv6 support: now proxies may be specified with IPv6 addresses. Example: geo $test { ... proxy 127.0.0.1; proxy ::1; proxy_recursive; } There's also a slight change in behavior. When original client address (as specified by the "geo" directive) is one of the trusted proxies, and the value of the X-Forwarded-For request header cannot not be parsed as a valid address, an original client address will be used for lookup. Previously, 255.255.255.255 was used in this case. *) Geoip: trusted proxies support and partial IPv6 support. The module now supports recursive search of client address through the chain of trusted proxies (closes #100), in the same scope as the geo module. Proxies are listed by the "geoip_proxy" directive, recursive search is enabled by the "geoip_proxy_recursive" directive. IPv6 is partially supported: proxies may be specified with IPv6 addresses. Example: geoip_country .../GeoIP.dat; geoip_proxy 127.0.0.1; geoip_proxy ::1; geoip_proxy 10.0.0.0/8; geoip_proxy_recursive on; Modified: branches/stable-1.2/ branches/stable-1.2/src/event/ngx_event.c branches/stable-1.2/src/event/ngx_event.h branches/stable-1.2/src/event/ngx_event_accept.c branches/stable-1.2/src/http/modules/ngx_http_geo_module.c branches/stable-1.2/src/http/modules/ngx_http_geoip_module.c branches/stable-1.2/src/http/modules/ngx_http_realip_module.c branches/stable-1.2/src/http/ngx_http_core_module.c branches/stable-1.2/src/http/ngx_http_core_module.h Index: branches/stable-1.2 =================================================================== --- branches/stable-1.2 2012-06-04 11:21:58 UTC (rev 4668) +++ branches/stable-1.2 2012-06-04 11:58:12 UTC (rev 4669) Property changes on: branches/stable-1.2 ___________________________________________________________________ Modified: svn:mergeinfo ## -1 +1 ## -/trunk:4611-4613,4615-4623,4641 +/trunk:4611-4629,4631,4641 \ No newline at end of property Modified: branches/stable-1.2/src/event/ngx_event.c =================================================================== --- branches/stable-1.2/src/event/ngx_event.c 2012-06-04 11:21:58 UTC (rev 4668) +++ branches/stable-1.2/src/event/ngx_event.c 2012-06-04 11:58:12 UTC (rev 4669) @@ -1064,38 +1064,34 @@ ngx_int_t rc; ngx_str_t *value; - ngx_event_debug_t *dc; struct hostent *h; - ngx_cidr_t cidr; + ngx_cidr_t *cidr; value = cf->args->elts; - dc = ngx_array_push(&ecf->debug_connection); - if (dc == NULL) { + cidr = ngx_array_push(&ecf->debug_connection); + if (cidr == NULL) { return NGX_CONF_ERROR; } - rc = ngx_ptocidr(&value[1], &cidr); +#if (NGX_HAVE_UNIX_DOMAIN) + if (ngx_strcmp(value[1].data, "unix:") == 0) { + cidr->family = AF_UNIX; + return NGX_CONF_OK; + } + +#endif + + rc = ngx_ptocidr(&value[1], cidr); + if (rc == NGX_DONE) { ngx_conf_log_error(NGX_LOG_WARN, cf, 0, "low address bits of %V are meaningless", &value[1]); - rc = NGX_OK; + return NGX_CONF_OK; } if (rc == NGX_OK) { - - /* AF_INET only */ - - if (cidr.family != AF_INET) { - ngx_conf_log_error(NGX_LOG_EMERG, cf, 0, - "\"debug_connection\" supports IPv4 only"); - return NGX_CONF_ERROR; - } - - dc->mask = cidr.u.in.mask; - dc->addr = cidr.u.in.addr; - return NGX_CONF_OK; } @@ -1107,8 +1103,9 @@ return NGX_CONF_ERROR; } - dc->mask = 0xffffffff; - dc->addr = *(in_addr_t *)(h->h_addr_list[0]); + cidr->family = AF_INET; + cidr->u.in.mask = 0xffffffff; + cidr->u.in.addr = *(in_addr_t *)(h->h_addr_list[0]); #else @@ -1142,7 +1139,7 @@ #if (NGX_DEBUG) if (ngx_array_init(&ecf->debug_connection, cycle->pool, 4, - sizeof(ngx_event_debug_t)) == NGX_ERROR) + sizeof(ngx_cidr_t)) == NGX_ERROR) { return NULL; } Modified: branches/stable-1.2/src/event/ngx_event.h =================================================================== --- branches/stable-1.2/src/event/ngx_event.h 2012-06-04 11:21:58 UTC (rev 4668) +++ branches/stable-1.2/src/event/ngx_event.h 2012-06-04 11:58:12 UTC (rev 4669) @@ -222,12 +222,6 @@ typedef struct { - in_addr_t mask; - in_addr_t addr; -} ngx_event_debug_t; - - -typedef struct { ngx_int_t (*add)(ngx_event_t *ev, ngx_int_t event, ngx_uint_t flags); ngx_int_t (*del)(ngx_event_t *ev, ngx_int_t event, ngx_uint_t flags); Modified: branches/stable-1.2/src/event/ngx_event_accept.c =================================================================== --- branches/stable-1.2/src/event/ngx_event_accept.c 2012-06-04 11:21:58 UTC (rev 4668) +++ branches/stable-1.2/src/event/ngx_event_accept.c 2012-06-04 11:58:12 UTC (rev 4669) @@ -286,17 +286,56 @@ #if (NGX_DEBUG) { - in_addr_t i; - ngx_event_debug_t *dc; - struct sockaddr_in *sin; + struct sockaddr_in *sin; + ngx_cidr_t *cidr; + ngx_uint_t i; +#if (NGX_HAVE_INET6) + struct sockaddr_in6 *sin6; + ngx_uint_t n; +#endif - sin = (struct sockaddr_in *) sa; - dc = ecf->debug_connection.elts; + cidr = ecf->debug_connection.elts; for (i = 0; i < ecf->debug_connection.nelts; i++) { - if ((sin->sin_addr.s_addr & dc[i].mask) == dc[i].addr) { - log->log_level = NGX_LOG_DEBUG_CONNECTION|NGX_LOG_DEBUG_ALL; + if (cidr[i].family != c->sockaddr->sa_family) { + goto next; + } + + switch (cidr[i].family) { + +#if (NGX_HAVE_INET6) + case AF_INET6: + sin6 = (struct sockaddr_in6 *) c->sockaddr; + for (n = 0; n < 16; n++) { + if ((sin6->sin6_addr.s6_addr[n] + & cidr[i].u.in6.mask.s6_addr[n]) + != cidr[i].u.in6.addr.s6_addr[n]) + { + goto next; + } + } break; +#endif + +#if (NGX_HAVE_UNIX_DOMAIN) + case AF_UNIX: + break; +#endif + + default: /* AF_INET */ + sin = (struct sockaddr_in *) c->sockaddr; + if ((sin->sin_addr.s_addr & cidr[i].u.in.mask) + != cidr[i].u.in.addr) + { + goto next; + } + break; } + + log->log_level = NGX_LOG_DEBUG_CONNECTION|NGX_LOG_DEBUG_ALL; + break; + + next: + continue; } } Modified: branches/stable-1.2/src/http/modules/ngx_http_geo_module.c =================================================================== --- branches/stable-1.2/src/http/modules/ngx_http_geo_module.c 2012-06-04 11:21:58 UTC (rev 4668) +++ branches/stable-1.2/src/http/modules/ngx_http_geo_module.c 2012-06-04 11:58:12 UTC (rev 4669) @@ -51,6 +51,7 @@ unsigned outside_entries:1; unsigned allow_binary_include:1; unsigned binary_include:1; + unsigned proxy_recursive:1; } ngx_http_geo_conf_ctx_t; @@ -61,6 +62,7 @@ } u; ngx_array_t *proxies; + unsigned proxy_recursive:1; ngx_int_t index; } ngx_http_geo_ctx_t; @@ -68,8 +70,8 @@ static in_addr_t ngx_http_geo_addr(ngx_http_request_t *r, ngx_http_geo_ctx_t *ctx); -static in_addr_t ngx_http_geo_real_addr(ngx_http_request_t *r, - ngx_http_geo_ctx_t *ctx); +static ngx_int_t ngx_http_geo_real_addr(ngx_http_request_t *r, + ngx_http_geo_ctx_t *ctx, ngx_addr_t *addr); static char *ngx_http_geo_block(ngx_conf_t *cf, ngx_command_t *cmd, void *conf); static char *ngx_http_geo(ngx_conf_t *cf, ngx_command_t *dummy, void *conf); static char *ngx_http_geo_range(ngx_conf_t *cf, ngx_http_geo_conf_ctx_t *ctx, @@ -212,87 +214,60 @@ static in_addr_t ngx_http_geo_addr(ngx_http_request_t *r, ngx_http_geo_ctx_t *ctx) { - u_char *p, *ip; - size_t len; - in_addr_t addr; - ngx_uint_t i, n; - ngx_in_cidr_t *proxies; - ngx_table_elt_t *xfwd; + ngx_addr_t addr; + ngx_table_elt_t *xfwd; + struct sockaddr_in *sin; - addr = ngx_http_geo_real_addr(r, ctx); + if (ngx_http_geo_real_addr(r, ctx, &addr) != NGX_OK) { + return INADDR_NONE; + } xfwd = r->headers_in.x_forwarded_for; - if (xfwd == NULL || ctx->proxies == NULL) { - return addr; + if (xfwd != NULL && ctx->proxies != NULL) { + (void) ngx_http_get_forwarded_addr(r, &addr, xfwd->value.data, + xfwd->value.len, ctx->proxies, + ctx->proxy_recursive); } - proxies = ctx->proxies->elts; - n = ctx->proxies->nelts; +#if (NGX_HAVE_INET6) - for (i = 0; i < n; i++) { - if ((addr & proxies[i].mask) == proxies[i].addr) { + if (addr.sockaddr->sa_family == AF_INET6) { + struct in6_addr *inaddr6; - len = xfwd->value.len; - ip = xfwd->value.data; + inaddr6 = &((struct sockaddr_in6 *) addr.sockaddr)->sin6_addr; - for (p = ip + len - 1; p > ip; p--) { - if (*p == ' ' || *p == ',') { - p++; - len -= p - ip; - ip = p; - break; - } - } - - return ntohl(ngx_inet_addr(ip, len)); + if (IN6_IS_ADDR_V4MAPPED(inaddr6)) { + return ntohl(*(in_addr_t *) &inaddr6->s6_addr[12]); } } - return addr; +#endif + + if (addr.sockaddr->sa_family != AF_INET) { + return INADDR_NONE; + } + + sin = (struct sockaddr_in *) addr.sockaddr; + return ntohl(sin->sin_addr.s_addr); } -static in_addr_t -ngx_http_geo_real_addr(ngx_http_request_t *r, ngx_http_geo_ctx_t *ctx) +static ngx_int_t +ngx_http_geo_real_addr(ngx_http_request_t *r, ngx_http_geo_ctx_t *ctx, + ngx_addr_t *addr) { - struct sockaddr_in *sin; ngx_http_variable_value_t *v; -#if (NGX_HAVE_INET6) - u_char *p; - in_addr_t addr; - struct sockaddr_in6 *sin6; -#endif if (ctx->index == -1) { ngx_log_debug1(NGX_LOG_DEBUG_HTTP, r->connection->log, 0, "http geo started: %V", &r->connection->addr_text); - switch (r->connection->sockaddr->sa_family) { + addr->sockaddr = r->connection->sockaddr; + addr->socklen = r->connection->socklen; + /* addr->name = r->connection->addr_text; */ - case AF_INET: - sin = (struct sockaddr_in *) r->connection->sockaddr; - return ntohl(sin->sin_addr.s_addr); - -#if (NGX_HAVE_INET6) - - case AF_INET6: - sin6 = (struct sockaddr_in6 *) r->connection->sockaddr; - - if (IN6_IS_ADDR_V4MAPPED(&sin6->sin6_addr)) { - p = sin6->sin6_addr.s6_addr; - addr = p[12] << 24; - addr += p[13] << 16; - addr += p[14] << 8; - addr += p[15]; - - return addr; - } - -#endif - } - - return INADDR_NONE; + return NGX_OK; } v = ngx_http_get_flushed_variable(r, ctx->index); @@ -301,13 +276,17 @@ ngx_log_debug0(NGX_LOG_DEBUG_HTTP, r->connection->log, 0, "http geo not found"); - return 0; + return NGX_ERROR; } ngx_log_debug1(NGX_LOG_DEBUG_HTTP, r->connection->log, 0, "http geo started: %v", v); - return ntohl(ngx_inet_addr(v->data, v->len)); + if (ngx_parse_addr(r->pool, addr, v->data, v->len) == NGX_OK) { + return NGX_OK; + } + + return NGX_ERROR; } @@ -388,6 +367,7 @@ *cf = save; geo->proxies = ctx.proxies; + geo->proxy_recursive = ctx.proxy_recursive; if (ctx.high.low) { @@ -493,6 +473,12 @@ goto done; } + + else if (ngx_strcmp(value[0].data, "proxy_recursive") == 0) { + ctx->proxy_recursive = 1; + rv = NGX_CONF_OK; + goto done; + } } if (cf->args->nelts != 2) { @@ -926,6 +912,7 @@ } if (ngx_strcmp(value[0].data, "default") == 0) { + /* cidr.family = AF_INET; */ cidr.u.in.addr = 0; cidr.u.in.mask = 0; net = &value[0]; @@ -944,6 +931,15 @@ return NGX_CONF_ERROR; } + if (cidr.family != AF_INET) { + ngx_conf_log_error(NGX_LOG_EMERG, cf, 0, + "\"geo\" supports IPv4 only"); + return NGX_CONF_ERROR; + } + + cidr.u.in.addr = ntohl(cidr.u.in.addr); + cidr.u.in.mask = ntohl(cidr.u.in.mask); + if (del) { if (ngx_radix32tree_delete(ctx->tree, cidr.u.in.addr, cidr.u.in.mask) @@ -1052,10 +1048,10 @@ ngx_http_geo_add_proxy(ngx_conf_t *cf, ngx_http_geo_conf_ctx_t *ctx, ngx_cidr_t *cidr) { - ngx_in_cidr_t *c; + ngx_cidr_t *c; if (ctx->proxies == NULL) { - ctx->proxies = ngx_array_create(ctx->pool, 4, sizeof(ngx_in_cidr_t)); + ctx->proxies = ngx_array_create(ctx->pool, 4, sizeof(ngx_cidr_t)); if (ctx->proxies == NULL) { return NGX_CONF_ERROR; } @@ -1066,8 +1062,7 @@ return NGX_CONF_ERROR; } - c->addr = cidr->u.in.addr; - c->mask = cidr->u.in.mask; + *c = *cidr; return NGX_CONF_OK; } @@ -1079,6 +1074,7 @@ ngx_int_t rc; if (ngx_strcmp(net->data, "255.255.255.255") == 0) { + cidr->family = AF_INET; cidr->u.in.addr = 0xffffffff; cidr->u.in.mask = 0xffffffff; @@ -1092,19 +1088,11 @@ return NGX_ERROR; } - if (cidr->family != AF_INET) { - ngx_conf_log_error(NGX_LOG_EMERG, cf, 0, "\"geo\" supports IPv4 only"); - return NGX_ERROR; - } - if (rc == NGX_DONE) { ngx_conf_log_error(NGX_LOG_WARN, cf, 0, "low address bits of %V are meaningless", net); } - cidr->u.in.addr = ntohl(cidr->u.in.addr); - cidr->u.in.mask = ntohl(cidr->u.in.mask); - return NGX_OK; } Modified: branches/stable-1.2/src/http/modules/ngx_http_geoip_module.c =================================================================== --- branches/stable-1.2/src/http/modules/ngx_http_geoip_module.c 2012-06-04 11:21:58 UTC (rev 4668) +++ branches/stable-1.2/src/http/modules/ngx_http_geoip_module.c 2012-06-04 11:58:12 UTC (rev 4669) @@ -14,20 +14,24 @@ typedef struct { - GeoIP *country; - GeoIP *org; - GeoIP *city; + GeoIP *country; + GeoIP *org; + GeoIP *city; + ngx_array_t *proxies; /* array of ngx_cidr_t */ + ngx_flag_t proxy_recursive; } ngx_http_geoip_conf_t; typedef struct { - ngx_str_t *name; - uintptr_t data; + ngx_str_t *name; + uintptr_t data; } ngx_http_geoip_var_t; typedef const char *(*ngx_http_geoip_variable_handler_pt)(GeoIP *, u_long addr); +static u_long ngx_http_geoip_addr(ngx_http_request_t *r, + ngx_http_geoip_conf_t *gcf); static ngx_int_t ngx_http_geoip_country_variable(ngx_http_request_t *r, ngx_http_variable_value_t *v, uintptr_t data); static ngx_int_t ngx_http_geoip_org_variable(ngx_http_request_t *r, @@ -44,12 +48,17 @@ static ngx_int_t ngx_http_geoip_add_variables(ngx_conf_t *cf); static void *ngx_http_geoip_create_conf(ngx_conf_t *cf); +static char *ngx_http_geoip_init_conf(ngx_conf_t *cf, void *conf); static char *ngx_http_geoip_country(ngx_conf_t *cf, ngx_command_t *cmd, void *conf); static char *ngx_http_geoip_org(ngx_conf_t *cf, ngx_command_t *cmd, void *conf); static char *ngx_http_geoip_city(ngx_conf_t *cf, ngx_command_t *cmd, void *conf); +static char *ngx_http_geoip_proxy(ngx_conf_t *cf, ngx_command_t *cmd, + void *conf); +static ngx_int_t ngx_http_geoip_cidr_value(ngx_conf_t *cf, ngx_str_t *net, + ngx_cidr_t *cidr); static void ngx_http_geoip_cleanup(void *data); @@ -76,6 +85,20 @@ 0, NULL }, + { ngx_string("geoip_proxy"), + NGX_HTTP_MAIN_CONF|NGX_CONF_TAKE1, + ngx_http_geoip_proxy, + NGX_HTTP_MAIN_CONF_OFFSET, + 0, + NULL }, + + { ngx_string("geoip_proxy_recursive"), + NGX_HTTP_MAIN_CONF|NGX_CONF_FLAG, + ngx_conf_set_flag_slot, + NGX_HTTP_MAIN_CONF_OFFSET, + offsetof(ngx_http_geoip_conf_t, proxy_recursive), + NULL }, + ngx_null_command }; @@ -85,7 +108,7 @@ NULL, /* postconfiguration */ ngx_http_geoip_create_conf, /* create main configuration */ - NULL, /* init main configuration */ + ngx_http_geoip_init_conf, /* init main configuration */ NULL, /* create server configuration */ NULL, /* merge server configuration */ @@ -182,40 +205,44 @@ static u_long -ngx_http_geoip_addr(ngx_http_request_t *r) +ngx_http_geoip_addr(ngx_http_request_t *r, ngx_http_geoip_conf_t *gcf) { - struct sockaddr_in *sin; -#if (NGX_HAVE_INET6) - u_char *p; - u_long addr; - struct sockaddr_in6 *sin6; -#endif + ngx_addr_t addr; + ngx_table_elt_t *xfwd; + struct sockaddr_in *sin; - switch (r->connection->sockaddr->sa_family) { + addr.sockaddr = r->connection->sockaddr; + addr.socklen = r->connection->socklen; + /* addr.name = r->connection->addr_text; */ - case AF_INET: - sin = (struct sockaddr_in *) r->connection->sockaddr; - return ntohl(sin->sin_addr.s_addr); + xfwd = r->headers_in.x_forwarded_for; + if (xfwd != NULL && gcf->proxies != NULL) { + (void) ngx_http_get_forwarded_addr(r, &addr, xfwd->value.data, + xfwd->value.len, gcf->proxies, + gcf->proxy_recursive); + } + #if (NGX_HAVE_INET6) - case AF_INET6: - sin6 = (struct sockaddr_in6 *) r->connection->sockaddr; + if (addr.sockaddr->sa_family == AF_INET6) { + struct in6_addr *inaddr6; - if (IN6_IS_ADDR_V4MAPPED(&sin6->sin6_addr)) { - p = sin6->sin6_addr.s6_addr; - addr = p[12] << 24; - addr += p[13] << 16; - addr += p[14] << 8; - addr += p[15]; + inaddr6 = &((struct sockaddr_in6 *) addr.sockaddr)->sin6_addr; - return addr; + if (IN6_IS_ADDR_V4MAPPED(inaddr6)) { + return ntohl(*(in_addr_t *) &inaddr6->s6_addr[12]); } + } #endif + + if (addr.sockaddr->sa_family != AF_INET) { + return INADDR_NONE; } - return INADDR_NONE; + sin = (struct sockaddr_in *) addr.sockaddr; + return ntohl(sin->sin_addr.s_addr); } @@ -235,7 +262,7 @@ goto not_found; } - val = handler(gcf->country, ngx_http_geoip_addr(r)); + val = handler(gcf->country, ngx_http_geoip_addr(r, gcf)); if (val == NULL) { goto not_found; @@ -273,7 +300,7 @@ goto not_found; } - val = handler(gcf->org, ngx_http_geoip_addr(r)); + val = handler(gcf->org, ngx_http_geoip_addr(r, gcf)); if (val == NULL) { goto not_found; @@ -453,7 +480,7 @@ gcf = ngx_http_get_module_main_conf(r, ngx_http_geoip_module); if (gcf->city) { - return GeoIP_record_by_ipnum(gcf->city, ngx_http_geoip_addr(r)); + return GeoIP_record_by_ipnum(gcf->city, ngx_http_geoip_addr(r, gcf)); } return NULL; @@ -490,6 +517,8 @@ return NULL; } + conf->proxy_recursive = NGX_CONF_UNSET; + cln = ngx_pool_cleanup_add(cf->pool, 0); if (cln == NULL) { return NULL; @@ -503,6 +532,17 @@ static char * +ngx_http_geoip_init_conf(ngx_conf_t *cf, void *conf) +{ + ngx_http_geoip_conf_t *gcf = conf; + + ngx_conf_init_value(gcf->proxy_recursive, 0); + + return NGX_CONF_OK; +} + + +static char * ngx_http_geoip_country(ngx_conf_t *cf, ngx_command_t *cmd, void *conf) { ngx_http_geoip_conf_t *gcf = conf; @@ -652,6 +692,66 @@ } +static char * +ngx_http_geoip_proxy(ngx_conf_t *cf, ngx_command_t *cmd, void *conf) +{ + ngx_http_geoip_conf_t *gcf = conf; + + ngx_str_t *value; + ngx_cidr_t cidr, *c; + + value = cf->args->elts; + + if (ngx_http_geoip_cidr_value(cf, &value[1], &cidr) != NGX_OK) { + return NGX_CONF_ERROR; + } + + if (gcf->proxies == NULL) { + gcf->proxies = ngx_array_create(cf->pool, 4, sizeof(ngx_cidr_t)); + if (gcf->proxies == NULL) { + return NGX_CONF_ERROR; + } + } + + c = ngx_array_push(gcf->proxies); + if (c == NULL) { + return NGX_CONF_ERROR; + } + + *c = cidr; + + return NGX_CONF_OK; +} + +static ngx_int_t +ngx_http_geoip_cidr_value(ngx_conf_t *cf, ngx_str_t *net, ngx_cidr_t *cidr) +{ + ngx_int_t rc; + + if (ngx_strcmp(net->data, "255.255.255.255") == 0) { + cidr->family = AF_INET; + cidr->u.in.addr = 0xffffffff; + cidr->u.in.mask = 0xffffffff; + + return NGX_OK; + } + + rc = ngx_ptocidr(net, cidr); + + if (rc == NGX_ERROR) { + ngx_conf_log_error(NGX_LOG_EMERG, cf, 0, "invalid network \"%V\"", net); + return NGX_ERROR; + } + + if (rc == NGX_DONE) { + ngx_conf_log_error(NGX_LOG_WARN, cf, 0, + "low address bits of %V are meaningless", net); + } + + return NGX_OK; +} + + static void ngx_http_geoip_cleanup(void *data) { Modified: branches/stable-1.2/src/http/modules/ngx_http_realip_module.c =================================================================== --- branches/stable-1.2/src/http/modules/ngx_http_realip_module.c 2012-06-04 11:21:58 UTC (rev 4668) +++ branches/stable-1.2/src/http/modules/ngx_http_realip_module.c 2012-06-04 11:58:12 UTC (rev 4669) @@ -16,13 +16,11 @@ typedef struct { - ngx_array_t *from; /* array of ngx_in_cidr_t */ + ngx_array_t *from; /* array of ngx_cidr_t */ ngx_uint_t type; ngx_uint_t hash; ngx_str_t header; -#if (NGX_HAVE_UNIX_DOMAIN) - ngx_uint_t unixsock; /* unsigned unixsock:2; */ -#endif + ngx_flag_t recursive; } ngx_http_realip_loc_conf_t; @@ -35,8 +33,8 @@ static ngx_int_t ngx_http_realip_handler(ngx_http_request_t *r); -static ngx_int_t ngx_http_realip_set_addr(ngx_http_request_t *r, u_char *ip, - size_t len); +static ngx_int_t ngx_http_realip_set_addr(ngx_http_request_t *r, + ngx_addr_t *addr); static void ngx_http_realip_cleanup(void *data); static char *ngx_http_realip_from(ngx_conf_t *cf, ngx_command_t *cmd, void *conf); @@ -63,6 +61,13 @@ 0, NULL }, + { ngx_string("real_ip_recursive"), + NGX_HTTP_MAIN_CONF|NGX_HTTP_SRV_CONF|NGX_HTTP_LOC_CONF|NGX_CONF_FLAG, + ngx_conf_set_flag_slot, + NGX_HTTP_LOC_CONF_OFFSET, + offsetof(ngx_http_realip_loc_conf_t, recursive), + NULL }, + ngx_null_command }; @@ -105,10 +110,9 @@ u_char *ip, *p; size_t len; ngx_uint_t i, hash; + ngx_addr_t addr; ngx_list_part_t *part; ngx_table_elt_t *header; - struct sockaddr_in *sin; - ngx_in_cidr_t *from; ngx_connection_t *c; ngx_http_realip_ctx_t *ctx; ngx_http_realip_loc_conf_t *rlcf; @@ -121,12 +125,7 @@ rlcf = ngx_http_get_module_loc_conf(r, ngx_http_realip_module); - if (rlcf->from == NULL -#if (NGX_HAVE_UNIX_DOMAIN) - && !rlcf->unixsock -#endif - ) - { + if (rlcf->from == NULL) { return NGX_DECLINED; } @@ -152,15 +151,6 @@ len = r->headers_in.x_forwarded_for->value.len; ip = r->headers_in.x_forwarded_for->value.data; - for (p = ip + len - 1; p > ip; p--) { - if (*p == ' ' || *p == ',') { - p++; - len -= p - ip; - ip = p; - break; - } - } - break; default: /* NGX_HTTP_REALIP_HEADER */ @@ -204,42 +194,27 @@ ngx_log_debug1(NGX_LOG_DEBUG_HTTP, c->log, 0, "realip: \"%s\"", ip); - /* AF_INET only */ + addr.sockaddr = c->sockaddr; + addr.socklen = c->socklen; + /* addr.name = c->addr_text; */ - if (c->sockaddr->sa_family == AF_INET) { - sin = (struct sockaddr_in *) c->sockaddr; - - from = rlcf->from->elts; - for (i = 0; i < rlcf->from->nelts; i++) { - - ngx_log_debug3(NGX_LOG_DEBUG_HTTP, c->log, 0, - "realip: %08XD %08XD %08XD", - sin->sin_addr.s_addr, from[i].mask, from[i].addr); - - if ((sin->sin_addr.s_addr & from[i].mask) == from[i].addr) { - return ngx_http_realip_set_addr(r, ip, len); - } - } + if (ngx_http_get_forwarded_addr(r, &addr, ip, len, rlcf->from, + rlcf->recursive) + == NGX_OK) + { + return ngx_http_realip_set_addr(r, &addr); } -#if (NGX_HAVE_UNIX_DOMAIN) - - if (c->sockaddr->sa_family == AF_UNIX && rlcf->unixsock) { - return ngx_http_realip_set_addr(r, ip, len); - } - -#endif - return NGX_DECLINED; } static ngx_int_t -ngx_http_realip_set_addr(ngx_http_request_t *r, u_char *ip, size_t len) +ngx_http_realip_set_addr(ngx_http_request_t *r, ngx_addr_t *addr) { + size_t len; u_char *p; - ngx_int_t rc; - ngx_addr_t addr; + u_char text[NGX_SOCKADDR_STRLEN]; ngx_connection_t *c; ngx_pool_cleanup_t *cln; ngx_http_realip_ctx_t *ctx; @@ -254,15 +229,9 @@ c = r->connection; - rc = ngx_parse_addr(c->pool, &addr, ip, len); - - switch (rc) { - case NGX_DECLINED: - return NGX_DECLINED; - case NGX_ERROR: + len = ngx_sock_ntop(addr->sockaddr, text, NGX_SOCKADDR_STRLEN, 0); + if (len == 0) { return NGX_HTTP_INTERNAL_SERVER_ERROR; - default: /* NGX_OK */ - break; } p = ngx_pnalloc(c->pool, len); @@ -270,7 +239,7 @@ return NGX_HTTP_INTERNAL_SERVER_ERROR; } - ngx_memcpy(p, ip, len); + ngx_memcpy(p, text, len); cln->handler = ngx_http_realip_cleanup; @@ -279,8 +248,8 @@ ctx->socklen = c->socklen; ctx->addr_text = c->addr_text; - c->sockaddr = addr.sockaddr; - c->socklen = addr.socklen; + c->sockaddr = addr->sockaddr; + c->socklen = addr->socklen; c->addr_text.len = len; c->addr_text.data = p; @@ -310,55 +279,45 @@ ngx_int_t rc; ngx_str_t *value; - ngx_cidr_t cidr; - ngx_in_cidr_t *from; + ngx_cidr_t *cidr; value = cf->args->elts; -#if (NGX_HAVE_UNIX_DOMAIN) - - if (ngx_strcmp(value[1].data, "unix:") == 0) { - rlcf->unixsock = 1; - return NGX_CONF_OK; - } - -#endif - if (rlcf->from == NULL) { rlcf->from = ngx_array_create(cf->pool, 2, - sizeof(ngx_in_cidr_t)); + sizeof(ngx_cidr_t)); if (rlcf->from == NULL) { return NGX_CONF_ERROR; } } - from = ngx_array_push(rlcf->from); - if (from == NULL) { + cidr = ngx_array_push(rlcf->from); + if (cidr == NULL) { return NGX_CONF_ERROR; } - rc = ngx_ptocidr(&value[1], &cidr); +#if (NGX_HAVE_UNIX_DOMAIN) + if (ngx_strcmp(value[1].data, "unix:") == 0) { + cidr->family = AF_UNIX; + return NGX_CONF_OK; + } + +#endif + + rc = ngx_ptocidr(&value[1], cidr); + if (rc == NGX_ERROR) { ngx_conf_log_error(NGX_LOG_EMERG, cf, 0, "invalid parameter \"%V\"", &value[1]); return NGX_CONF_ERROR; } - if (cidr.family != AF_INET) { - ngx_conf_log_error(NGX_LOG_EMERG, cf, 0, - "\"set_real_ip_from\" supports IPv4 only"); - return NGX_CONF_ERROR; - } - if (rc == NGX_DONE) { ngx_conf_log_error(NGX_LOG_WARN, cf, 0, "low address bits of %V are meaningless", &value[1]); } - from->mask = cidr.u.in.mask; - from->addr = cidr.u.in.addr; - return NGX_CONF_OK; } @@ -409,9 +368,7 @@ */ conf->type = NGX_CONF_UNSET_UINT; -#if (NGX_HAVE_UNIX_DOMAIN) - conf->unixsock = 2; -#endif + conf->recursive = NGX_CONF_UNSET; return conf; } @@ -427,13 +384,8 @@ conf->from = prev->from; } -#if (NGX_HAVE_UNIX_DOMAIN) - if (conf->unixsock == 2) { - conf->unixsock = (prev->unixsock == 2) ? 0 : prev->unixsock; - } -#endif - ngx_conf_merge_uint_value(conf->type, prev->type, NGX_HTTP_REALIP_XREALIP); + ngx_conf_merge_value(conf->recursive, prev->recursive, 0); if (conf->header.len == 0) { conf->hash = prev->hash; Modified: branches/stable-1.2/src/http/ngx_http_core_module.c =================================================================== --- branches/stable-1.2/src/http/ngx_http_core_module.c 2012-06-04 11:21:58 UTC (rev 4668) +++ branches/stable-1.2/src/http/ngx_http_core_module.c 2012-06-04 11:58:12 UTC (rev 4669) @@ -2699,6 +2699,109 @@ } +ngx_int_t +ngx_http_get_forwarded_addr(ngx_http_request_t *r, ngx_addr_t *addr, + u_char *xff, size_t xfflen, ngx_array_t *proxies, int recursive) +{ + u_char *p; + in_addr_t inaddr; + ngx_addr_t paddr; + ngx_cidr_t *cidr; + ngx_uint_t family, i; +#if (NGX_HAVE_INET6) + ngx_uint_t n; + struct in6_addr *inaddr6; +#endif + +#if (NGX_SUPPRESS_WARN) + inaddr = 0; +#if (NGX_HAVE_INET6) + inaddr6 = NULL; +#endif +#endif + + family = addr->sockaddr->sa_family; + + if (family == AF_INET) { + inaddr = ((struct sockaddr_in *) addr->sockaddr)->sin_addr.s_addr; + } + +#if (NGX_HAVE_INET6) + else if (family == AF_INET6) { + inaddr6 = &((struct sockaddr_in6 *) addr->sockaddr)->sin6_addr; + + if (IN6_IS_ADDR_V4MAPPED(inaddr6)) { + family = AF_INET; + inaddr = *(in_addr_t *) &inaddr6->s6_addr[12]; + } + } +#endif + + for (cidr = proxies->elts, i = 0; i < proxies->nelts; i++) { + if (cidr[i].family != family) { + goto next; + } + + switch (family) { + +#if (NGX_HAVE_INET6) + case AF_INET6: + for (n = 0; n < 16; n++) { + if ((inaddr6->s6_addr[n] & cidr[i].u.in6.mask.s6_addr[n]) + != cidr[i].u.in6.addr.s6_addr[n]) + { + goto next; + } + } + break; +#endif + +#if (NGX_HAVE_UNIX_DOMAIN) + case AF_UNIX: + break; +#endif + + default: /* AF_INET */ + if ((inaddr & cidr[i].u.in.mask) != cidr[i].u.in.addr) { + goto next; + } + break; + } + + for (p = xff + xfflen - 1; p > xff; p--, xfflen--) { + if (*p != ' ' && *p != ',') { + break; + } + } + + for ( /* void */ ; p > xff; p--) { + if (*p == ' ' || *p == ',') { + p++; + break; + } + } + + if (ngx_parse_addr(r->pool, &paddr, p, xfflen - (p - xff)) != NGX_OK) { + return NGX_DECLINED; + } + + *addr = paddr; + + if (recursive && p > xff) { + (void) ngx_http_get_forwarded_addr(r, addr, xff, p - 1 - xff, + proxies, 1); + } + + return NGX_OK; + + next: + continue; + } + + return NGX_DECLINED; +} + + static char * ngx_http_core_server(ngx_conf_t *cf, ngx_command_t *cmd, void *dummy) { Modified: branches/stable-1.2/src/http/ngx_http_core_module.h =================================================================== --- branches/stable-1.2/src/http/ngx_http_core_module.h 2012-06-04 11:21:58 UTC (rev 4668) +++ branches/stable-1.2/src/http/ngx_http_core_module.h 2012-06-04 11:58:12 UTC (rev 4669) @@ -513,7 +513,10 @@ ngx_int_t ngx_http_set_disable_symlinks(ngx_http_request_t *r, ngx_http_core_loc_conf_t *clcf, ngx_str_t *path, ngx_open_file_info_t *of); +ngx_int_t ngx_http_get_forwarded_addr(ngx_http_request_t *r, ngx_addr_t *addr, + u_char *xff, size_t xfflen, ngx_array_t *proxies, int recursive); + extern ngx_module_t ngx_http_core_module; extern ngx_uint_t ngx_http_max_module; From mdounin at mdounin.ru Mon Jun 4 12:00:38 2012 From: mdounin at mdounin.ru (mdounin at mdounin.ru) Date: Mon, 4 Jun 2012 12:00:38 +0000 Subject: [nginx] svn commit: r4670 - in branches/stable-1.2: . src/event Message-ID: <20120604120038.C98C63F9F00@mail.nginx.com> Author: mdounin Date: 2012-06-04 12:00:38 +0000 (Mon, 04 Jun 2012) New Revision: 4670 URL: http://trac.nginx.org/nginx/changeset/4670/nginx Log: Merge of r4630: fixed c->sent with unbuffered ssl. Update c->sent in ngx_ssl_send_chain() even if SSL buffer is not used. Modified: branches/stable-1.2/ branches/stable-1.2/src/event/ngx_event_openssl.c Index: branches/stable-1.2 =================================================================== --- branches/stable-1.2 2012-06-04 11:58:12 UTC (rev 4669) +++ branches/stable-1.2 2012-06-04 12:00:38 UTC (rev 4670) Property changes on: branches/stable-1.2 ___________________________________________________________________ Modified: svn:mergeinfo ## -1 +1 ## -/trunk:4611-4629,4631,4641 +/trunk:4611-4631,4641 \ No newline at end of property Modified: branches/stable-1.2/src/event/ngx_event_openssl.c =================================================================== --- branches/stable-1.2/src/event/ngx_event_openssl.c 2012-06-04 11:58:12 UTC (rev 4669) +++ branches/stable-1.2/src/event/ngx_event_openssl.c 2012-06-04 12:00:38 UTC (rev 4670) @@ -995,6 +995,7 @@ } in->buf->pos += n; + c->sent += n; if (in->buf->pos == in->buf->last) { in = in->next; From mdounin at mdounin.ru Mon Jun 4 14:07:34 2012 From: mdounin at mdounin.ru (mdounin at mdounin.ru) Date: Mon, 4 Jun 2012 14:07:34 +0000 Subject: [nginx] svn commit: r4671 - trunk/src/core Message-ID: <20120604140734.97D0E3F9F3E@mail.nginx.com> Author: mdounin Date: 2012-06-04 14:07:34 +0000 (Mon, 04 Jun 2012) New Revision: 4671 URL: http://trac.nginx.org/nginx/changeset/4671/nginx Log: Support for IPv6 literals in proxy_pass and so on. Modified: trunk/src/core/ngx_inet.c Modified: trunk/src/core/ngx_inet.c =================================================================== --- trunk/src/core/ngx_inet.c 2012-06-04 12:00:38 UTC (rev 4670) +++ trunk/src/core/ngx_inet.c 2012-06-04 14:07:34 UTC (rev 4671) @@ -808,6 +808,8 @@ u->uri.len = last - uri; u->uri.data = uri; + + last = uri; } if (*port == ':') { @@ -840,8 +842,8 @@ return NGX_ERROR; } - u->host.len = len; - u->host.data = host; + u->host.len = len + 2; + u->host.data = host - 1; if (ngx_inet6_addr(host, len, sin6->sin6_addr.s6_addr) != NGX_OK) { u->err = "invalid IPv6 address"; @@ -852,17 +854,38 @@ u->wildcard = 1; } + if (u->no_port) { + u->port = u->default_port; + sin6->sin6_port = htons(u->default_port); + } + u->family = AF_INET6; + u->naddrs = 1; - if (u->no_resolve) { - return NGX_OK; + u->addrs = ngx_pcalloc(pool, sizeof(ngx_addr_t)); + if (u->addrs == NULL) { + return NGX_ERROR; } - if (u->no_port) { - u->port = u->default_port; - sin6->sin6_port = htons(u->default_port); + sin6 = ngx_pcalloc(pool, sizeof(struct sockaddr_in6)); + if (sin6 == NULL) { + return NGX_ERROR; } + ngx_memcpy(sin6, u->sockaddr, sizeof(struct sockaddr_in6)); + + u->addrs[0].sockaddr = (struct sockaddr *) sin6; + u->addrs[0].socklen = sizeof(struct sockaddr_in6); + + p = ngx_pnalloc(pool, u->host.len + sizeof(":65535") - 1); + if (p == NULL) { + return NGX_ERROR; + } + + u->addrs[0].name.len = ngx_sprintf(p, "%V:%d", + &u->host, u->port) - p; + u->addrs[0].name.data = p; + return NGX_OK; #else From ru at nginx.com Mon Jun 4 14:23:28 2012 From: ru at nginx.com (ru at nginx.com) Date: Mon, 4 Jun 2012 14:23:28 +0000 Subject: [nginx] svn commit: r4672 - trunk/src/core Message-ID: <20120604142328.35E533F9C18@mail.nginx.com> Author: ru Date: 2012-06-04 14:23:27 +0000 (Mon, 04 Jun 2012) New Revision: 4672 URL: http://trac.nginx.org/nginx/changeset/4672/nginx Log: Support for IPv6 literals and an optional port in resolver. Modified: trunk/src/core/ngx_resolver.c Modified: trunk/src/core/ngx_resolver.c =================================================================== --- trunk/src/core/ngx_resolver.c 2012-06-04 14:07:34 UTC (rev 4671) +++ trunk/src/core/ngx_resolver.c 2012-06-04 14:23:27 UTC (rev 4672) @@ -171,14 +171,14 @@ ngx_memzero(&u, sizeof(ngx_url_t)); - u.host = names[i]; - u.port = 53; + u.url = names[i]; + u.default_port = 53; - if (ngx_inet_resolve_host(cf->pool, &u) != NGX_OK) { + if (ngx_parse_url(cf->pool, &u) != NGX_OK) { if (u.err) { ngx_conf_log_error(NGX_LOG_EMERG, cf, 0, "%s in resolver \"%V\"", - u.err, &u.host); + u.err, &u.url); } return NULL; @@ -2189,7 +2189,7 @@ ngx_socket_t s; ngx_connection_t *c; - s = ngx_socket(AF_INET, SOCK_DGRAM, 0); + s = ngx_socket(uc->sockaddr->sa_family, SOCK_DGRAM, 0); ngx_log_debug1(NGX_LOG_DEBUG_EVENT, &uc->log, 0, "UDP socket %d", s); From pass86 at gmail.com Tue Jun 5 10:01:15 2012 From: pass86 at gmail.com (l.jay Yuan) Date: Tue, 5 Jun 2012 18:01:15 +0800 Subject: when to call ngx_handle_write_event?i am not really understanded. Message-ID: when to call ngx_handle_write_event?i am not really understanded. who can explain me? From pass86 at gmail.com Tue Jun 5 10:28:10 2012 From: pass86 at gmail.com (l.jay Yuan) Date: Tue, 5 Jun 2012 18:28:10 +0800 Subject: when to call ngx_handle_write_event?i am not really understanded. In-Reply-To: References: Message-ID: rc = WSASend(c->fd, &wsabuf, 1, (DWORD *) &n, 0, ovlp, NULL); err = ngx_socket_errno; if (rc == 0) { if (ovlp != NULL) { wev->ovlp.posted_zero_byte = 1; wev->ready = 0; return NGX_AGAIN; } #if 0 // is test code? if ((size_t) n < size) { wev->ready = 0; } #endif 2012/6/5 l.jay Yuan : > when to call ngx_handle_write_event?i am not really understanded. > who can explain me? From vshebordaev at mail.ru Tue Jun 5 10:48:10 2012 From: vshebordaev at mail.ru (Vladimir Shebordaev) Date: Tue, 5 Jun 2012 14:48:10 +0400 Subject: when to call ngx_handle_write_event?i am not really understanded. In-Reply-To: References: Message-ID: Hi! ngx_handle_write_event() is the event processiong state machine interface function that is to proper update event and connection states after write event is fired and the data has been probably written to the corresponding descriptor. It can optionally update send_lowat option on the connection socket. -- Regards, Vladimir 2012/6/5 l.jay Yuan : > when to call ngx_handle_write_event?i am not really understanded. > who can explain me? > > _______________________________________________ > nginx-devel mailing list > nginx-devel at nginx.org > http://mailman.nginx.org/mailman/listinfo/nginx-devel > From vshebordaev at mail.ru Tue Jun 5 11:15:04 2012 From: vshebordaev at mail.ru (Vladimir Shebordaev) Date: Tue, 5 Jun 2012 15:15:04 +0400 Subject: when to call ngx_handle_write_event?i am not really understanded. In-Reply-To: References: Message-ID: 2012/6/5 l.jay Yuan : > ? ?rc = WSASend(c->fd, &wsabuf, 1, (DWORD *) &n, 0, ovlp, NULL); > You'd better use platform-independent connection inteface here, i.e ngx_send(). ngx_handle_write_event() is also platform-independent and is expected to be invoked right after you've done with the current write event processing in your module and are about to await for the next one. > ? ?err = ngx_socket_errno; > > ? ?if (rc == 0) { > ? ? ? ?if (ovlp != NULL) { > ? ? ? ? ? ?wev->ovlp.posted_zero_byte = 1; > ? ? ? ? ? ?wev->ready = 0; > ? ? ? ? ? ?return NGX_AGAIN; > ? ? ? ?} > > #if 0 // is test code? > ? ? ? ?if ((size_t) n < size) { > ? ? ? ? ? ?wev->ready = 0; > ? ? ? ?} > #endif > > 2012/6/5 l.jay Yuan : >> when to call ngx_handle_write_event?i am not really understanded. >> who can explain me? > > _______________________________________________ > nginx-devel mailing list > nginx-devel at nginx.org > http://mailman.nginx.org/mailman/listinfo/nginx-devel > From pass86 at gmail.com Tue Jun 5 11:16:00 2012 From: pass86 at gmail.com (l.jay Yuan) Date: Tue, 5 Jun 2012 19:16:00 +0800 Subject: when to call ngx_handle_write_event?i am not really understanded. In-Reply-To: References: Message-ID: if i do not call ngx_handle_write_event the ngx_connection_t 's ngx_event_t *write will never be handler. it means after c->send(c, buf, n) then call c->send(c, buf, n)? 2012/6/5 Vladimir Shebordaev : > Hi! > > ngx_handle_write_event() is the event processiong state machine > interface function that is to proper update event and connection > states after write event is fired and the data has been probably > written to the corresponding descriptor. It can optionally update > send_lowat option on the connection socket. > > -- > Regards, > Vladimir > > > 2012/6/5 l.jay Yuan : >> when to call ngx_handle_write_event?i am not really understanded. >> who can explain me? >> >> _______________________________________________ >> nginx-devel mailing list >> nginx-devel at nginx.org >> http://mailman.nginx.org/mailman/listinfo/nginx-devel >> > > _______________________________________________ > nginx-devel mailing list > nginx-devel at nginx.org > http://mailman.nginx.org/mailman/listinfo/nginx-devel From pass86 at gmail.com Tue Jun 5 11:17:34 2012 From: pass86 at gmail.com (l.jay Yuan) Date: Tue, 5 Jun 2012 19:17:34 +0800 Subject: when to call ngx_handle_write_event?i am not really understanded. In-Reply-To: References: Message-ID: sorry, fix: call ngx_handle_write_event? 2012/6/5 l.jay Yuan : > if i do not call ngx_handle_write_event ?the ngx_connection_t 's > ngx_event_t *write will never be handler. > it means after c->send(c, buf, n) then call c->send(c, buf, n)? > > 2012/6/5 Vladimir Shebordaev : >> Hi! >> >> ngx_handle_write_event() is the event processiong state machine >> interface function that is to proper update event and connection >> states after write event is fired and the data has been probably >> written to the corresponding descriptor. It can optionally update >> send_lowat option on the connection socket. >> >> -- >> Regards, >> Vladimir >> >> >> 2012/6/5 l.jay Yuan : >>> when to call ngx_handle_write_event?i am not really understanded. >>> who can explain me? >>> >>> _______________________________________________ >>> nginx-devel mailing list >>> nginx-devel at nginx.org >>> http://mailman.nginx.org/mailman/listinfo/nginx-devel >>> >> >> _______________________________________________ >> nginx-devel mailing list >> nginx-devel at nginx.org >> http://mailman.nginx.org/mailman/listinfo/nginx-devel From vshebordaev at mail.ru Tue Jun 5 11:23:38 2012 From: vshebordaev at mail.ru (Vladimir Shebordaev) Date: Tue, 5 Jun 2012 15:23:38 +0400 Subject: when to call ngx_handle_write_event?i am not really understanded. In-Reply-To: References: Message-ID: 2012/6/5 l.jay Yuan : > sorry, fix: call ngx_handle_write_event? > > 2012/6/5 l.jay Yuan : >> if i do not call ngx_handle_write_event ?the ngx_connection_t 's >> ngx_event_t *write will never be handler. >> it means after c->send(c, buf, n) then call c->send(c, buf, n)? >> Yes, you should firstly process the data then update the connection state for the next event, i.e. call ngx_handle_write_event() >> 2012/6/5 Vladimir Shebordaev : >>> Hi! >>> >>> ngx_handle_write_event() is the event processiong state machine >>> interface function that is to proper update event and connection >>> states after write event is fired and the data has been probably >>> written to the corresponding descriptor. It can optionally update >>> send_lowat option on the connection socket. >>> >>> -- >>> Regards, >>> Vladimir >>> >>> >>> 2012/6/5 l.jay Yuan : >>>> when to call ngx_handle_write_event?i am not really understanded. >>>> who can explain me? >>>> From pass86 at gmail.com Tue Jun 5 11:29:25 2012 From: pass86 at gmail.com (l.jay Yuan) Date: Tue, 5 Jun 2012 19:29:25 +0800 Subject: when to call ngx_handle_write_event?i am not really understanded. In-Reply-To: References: Message-ID: okay, 3Q. another question, in ngx_wsasend.c 's ngx_wsasend function #if 0 // is test code?or really should do it? if ((size_t) n < size) { wev->ready = 0; } #endif 2012/6/5 Vladimir Shebordaev : > 2012/6/5 l.jay Yuan : >> sorry, fix: call ngx_handle_write_event? >> >> 2012/6/5 l.jay Yuan : >>> if i do not call ngx_handle_write_event ?the ngx_connection_t 's >>> ngx_event_t *write will never be handler. >>> it means after c->send(c, buf, n) then call c->send(c, buf, n)? >>> > > Yes, you should firstly process the data then update the connection > state for the next event, i.e. call ngx_handle_write_event() > >>> 2012/6/5 Vladimir Shebordaev : >>>> Hi! >>>> >>>> ngx_handle_write_event() is the event processiong state machine >>>> interface function that is to proper update event and connection >>>> states after write event is fired and the data has been probably >>>> written to the corresponding descriptor. It can optionally update >>>> send_lowat option on the connection socket. >>>> >>>> -- >>>> Regards, >>>> Vladimir >>>> >>>> >>>> 2012/6/5 l.jay Yuan : >>>>> when to call ngx_handle_write_event?i am not really understanded. >>>>> who can explain me? >>>>> > > _______________________________________________ > nginx-devel mailing list > nginx-devel at nginx.org > http://mailman.nginx.org/mailman/listinfo/nginx-devel From vshebordaev at mail.ru Tue Jun 5 11:45:45 2012 From: vshebordaev at mail.ru (Vladimir Shebordaev) Date: Tue, 5 Jun 2012 15:45:45 +0400 Subject: when to call ngx_handle_write_event?i am not really understanded. In-Reply-To: References: Message-ID: 2012/6/5 l.jay Yuan : > okay, 3Q. > another question, in ngx_wsasend.c 's ngx_wsasend function > > #if 0 // is test code?or really should do it? > ? ? ? if ((size_t) n < size) { > ? ? ? ? ? wev->ready = 0; > ? ? ? } > #endif > As of the current release 'ready' flag is reset there. You should update nginx sources. ngx_handle_write_event() belongs to upper layer interface and is anyways expected to be called. > 2012/6/5 Vladimir Shebordaev : >> 2012/6/5 l.jay Yuan : >>> sorry, fix: call ngx_handle_write_event? >>> >>> 2012/6/5 l.jay Yuan : >>>> if i do not call ngx_handle_write_event ?the ngx_connection_t 's >>>> ngx_event_t *write will never be handler. >>>> it means after c->send(c, buf, n) then call c->send(c, buf, n)? >>>> >> >> Yes, you should firstly process the data then update the connection >> state for the next event, i.e. call ngx_handle_write_event() >> >>>> 2012/6/5 Vladimir Shebordaev : >>>>> Hi! >>>>> >>>>> ngx_handle_write_event() is the event processiong state machine >>>>> interface function that is to proper update event and connection >>>>> states after write event is fired and the data has been probably >>>>> written to the corresponding descriptor. It can optionally update >>>>> send_lowat option on the connection socket. >>>>> >>>>> -- >>>>> Regards, >>>>> Vladimir >>>>> >>>>> >>>>> 2012/6/5 l.jay Yuan : >>>>>> when to call ngx_handle_write_event?i am not really understanded. >>>>>> who can explain me? >>>>>> >> >> _______________________________________________ >> nginx-devel mailing list >> nginx-devel at nginx.org >> http://mailman.nginx.org/mailman/listinfo/nginx-devel > > _______________________________________________ > nginx-devel mailing list > nginx-devel at nginx.org > http://mailman.nginx.org/mailman/listinfo/nginx-devel > From pass86 at gmail.com Tue Jun 5 11:54:03 2012 From: pass86 at gmail.com (l.jay Yuan) Date: Tue, 5 Jun 2012 19:54:03 +0800 Subject: when to call ngx_handle_write_event?i am not really understanded. In-Reply-To: References: Message-ID: soga, my src\os\win32\ is download from https://github.com/hehaiqiang/nginx, where is the official source for win32? 2012/6/5 Vladimir Shebordaev : > 2012/6/5 l.jay Yuan : >> okay, 3Q. >> another question, in ngx_wsasend.c 's ngx_wsasend function >> >> #if 0 // is test code?or really should do it? >> ? ? ? if ((size_t) n < size) { >> ? ? ? ? ? wev->ready = 0; >> ? ? ? } >> #endif >> > > As of the current release 'ready' flag is reset there. You should > update nginx sources. > > ngx_handle_write_event() belongs to upper layer interface and is > anyways expected to be called. > >> 2012/6/5 Vladimir Shebordaev : >>> 2012/6/5 l.jay Yuan : >>>> sorry, fix: call ngx_handle_write_event? >>>> >>>> 2012/6/5 l.jay Yuan : >>>>> if i do not call ngx_handle_write_event ?the ngx_connection_t 's >>>>> ngx_event_t *write will never be handler. >>>>> it means after c->send(c, buf, n) then call c->send(c, buf, n)? >>>>> >>> >>> Yes, you should firstly process the data then update the connection >>> state for the next event, i.e. call ngx_handle_write_event() >>> >>>>> 2012/6/5 Vladimir Shebordaev : >>>>>> Hi! >>>>>> >>>>>> ngx_handle_write_event() is the event processiong state machine >>>>>> interface function that is to proper update event and connection >>>>>> states after write event is fired and the data has been probably >>>>>> written to the corresponding descriptor. It can optionally update >>>>>> send_lowat option on the connection socket. >>>>>> >>>>>> -- >>>>>> Regards, >>>>>> Vladimir >>>>>> >>>>>> >>>>>> 2012/6/5 l.jay Yuan : >>>>>>> when to call ngx_handle_write_event?i am not really understanded. >>>>>>> who can explain me? >>>>>>> >>> >>> _______________________________________________ >>> nginx-devel mailing list >>> nginx-devel at nginx.org >>> http://mailman.nginx.org/mailman/listinfo/nginx-devel >> >> _______________________________________________ >> nginx-devel mailing list >> nginx-devel at nginx.org >> http://mailman.nginx.org/mailman/listinfo/nginx-devel >> > > _______________________________________________ > nginx-devel mailing list > nginx-devel at nginx.org > http://mailman.nginx.org/mailman/listinfo/nginx-devel From vshebordaev at mail.ru Tue Jun 5 12:03:02 2012 From: vshebordaev at mail.ru (Vladimir Shebordaev) Date: Tue, 5 Jun 2012 16:03:02 +0400 Subject: when to call ngx_handle_write_event?i am not really understanded. In-Reply-To: References: Message-ID: 2012/6/5 l.jay Yuan : > soga, my src\os\win32\ is download from > https://github.com/hehaiqiang/nginx, where is the official source for > win32? Not sure about win32 specific sources, but you can get stock nginx 1.3.1 from svn://svn.nginx.org/nginx/trunk > > 2012/6/5 Vladimir Shebordaev : >> 2012/6/5 l.jay Yuan : >>> okay, 3Q. >>> another question, in ngx_wsasend.c 's ngx_wsasend function >>> >>> #if 0 // is test code?or really should do it? >>> ? ? ? if ((size_t) n < size) { >>> ? ? ? ? ? wev->ready = 0; >>> ? ? ? } >>> #endif >>> >> >> As of the current release 'ready' flag is reset there. You should >> update nginx sources. >> >> ngx_handle_write_event() belongs to upper layer interface and is >> anyways expected to be called. >> >>> 2012/6/5 Vladimir Shebordaev : >>>> 2012/6/5 l.jay Yuan : >>>>> sorry, fix: call ngx_handle_write_event? >>>>> >>>>> 2012/6/5 l.jay Yuan : >>>>>> if i do not call ngx_handle_write_event ?the ngx_connection_t 's >>>>>> ngx_event_t *write will never be handler. >>>>>> it means after c->send(c, buf, n) then call c->send(c, buf, n)? >>>>>> >>>> >>>> Yes, you should firstly process the data then update the connection >>>> state for the next event, i.e. call ngx_handle_write_event() >>>> >>>>>> 2012/6/5 Vladimir Shebordaev : >>>>>>> Hi! >>>>>>> >>>>>>> ngx_handle_write_event() is the event processiong state machine >>>>>>> interface function that is to proper update event and connection >>>>>>> states after write event is fired and the data has been probably >>>>>>> written to the corresponding descriptor. It can optionally update >>>>>>> send_lowat option on the connection socket. >>>>>>> >>>>>>> -- >>>>>>> Regards, >>>>>>> Vladimir >>>>>>> >>>>>>> >>>>>>> 2012/6/5 l.jay Yuan : >>>>>>>> when to call ngx_handle_write_event?i am not really understanded. >>>>>>>> who can explain me? >>>>>>>> >>>> >>>> _______________________________________________ >>>> nginx-devel mailing list >>>> nginx-devel at nginx.org >>>> http://mailman.nginx.org/mailman/listinfo/nginx-devel >>> >>> _______________________________________________ >>> nginx-devel mailing list >>> nginx-devel at nginx.org >>> http://mailman.nginx.org/mailman/listinfo/nginx-devel >>> >> >> _______________________________________________ >> nginx-devel mailing list >> nginx-devel at nginx.org >> http://mailman.nginx.org/mailman/listinfo/nginx-devel > > _______________________________________________ > nginx-devel mailing list > nginx-devel at nginx.org > http://mailman.nginx.org/mailman/listinfo/nginx-devel > From pass86 at gmail.com Tue Jun 5 12:12:10 2012 From: pass86 at gmail.com (l.jay Yuan) Date: Tue, 5 Jun 2012 20:12:10 +0800 Subject: when to call ngx_handle_write_event?i am not really understanded. In-Reply-To: References: Message-ID: 3Q again, :D 2012/6/5 Vladimir Shebordaev : > 2012/6/5 l.jay Yuan : >> soga, my src\os\win32\ is download from >> https://github.com/hehaiqiang/nginx, where is the official source for >> win32? > > Not sure about win32 specific sources, but you can get stock nginx 1.3.1 from > > svn://svn.nginx.org/nginx/trunk > >> >> 2012/6/5 Vladimir Shebordaev : >>> 2012/6/5 l.jay Yuan : >>>> okay, 3Q. >>>> another question, in ngx_wsasend.c 's ngx_wsasend function >>>> >>>> #if 0 // is test code?or really should do it? >>>> ? ? ? if ((size_t) n < size) { >>>> ? ? ? ? ? wev->ready = 0; >>>> ? ? ? } >>>> #endif >>>> >>> >>> As of the current release 'ready' flag is reset there. You should >>> update nginx sources. >>> >>> ngx_handle_write_event() belongs to upper layer interface and is >>> anyways expected to be called. >>> >>>> 2012/6/5 Vladimir Shebordaev : >>>>> 2012/6/5 l.jay Yuan : >>>>>> sorry, fix: call ngx_handle_write_event? >>>>>> >>>>>> 2012/6/5 l.jay Yuan : >>>>>>> if i do not call ngx_handle_write_event ?the ngx_connection_t 's >>>>>>> ngx_event_t *write will never be handler. >>>>>>> it means after c->send(c, buf, n) then call c->send(c, buf, n)? >>>>>>> >>>>> >>>>> Yes, you should firstly process the data then update the connection >>>>> state for the next event, i.e. call ngx_handle_write_event() >>>>> >>>>>>> 2012/6/5 Vladimir Shebordaev : >>>>>>>> Hi! >>>>>>>> >>>>>>>> ngx_handle_write_event() is the event processiong state machine >>>>>>>> interface function that is to proper update event and connection >>>>>>>> states after write event is fired and the data has been probably >>>>>>>> written to the corresponding descriptor. It can optionally update >>>>>>>> send_lowat option on the connection socket. >>>>>>>> >>>>>>>> -- >>>>>>>> Regards, >>>>>>>> Vladimir >>>>>>>> >>>>>>>> >>>>>>>> 2012/6/5 l.jay Yuan : >>>>>>>>> when to call ngx_handle_write_event?i am not really understanded. >>>>>>>>> who can explain me? >>>>>>>>> >>>>> >>>>> _______________________________________________ >>>>> nginx-devel mailing list >>>>> nginx-devel at nginx.org >>>>> http://mailman.nginx.org/mailman/listinfo/nginx-devel >>>> >>>> _______________________________________________ >>>> nginx-devel mailing list >>>> nginx-devel at nginx.org >>>> http://mailman.nginx.org/mailman/listinfo/nginx-devel >>>> >>> >>> _______________________________________________ >>> nginx-devel mailing list >>> nginx-devel at nginx.org >>> http://mailman.nginx.org/mailman/listinfo/nginx-devel >> >> _______________________________________________ >> nginx-devel mailing list >> nginx-devel at nginx.org >> http://mailman.nginx.org/mailman/listinfo/nginx-devel >> > > _______________________________________________ > nginx-devel mailing list > nginx-devel at nginx.org > http://mailman.nginx.org/mailman/listinfo/nginx-devel From mdounin at mdounin.ru Tue Jun 5 13:17:06 2012 From: mdounin at mdounin.ru (mdounin at mdounin.ru) Date: Tue, 5 Jun 2012 13:17:06 +0000 Subject: [nginx] svn commit: r4673 - in branches/stable-1.2: . misc Message-ID: <20120605131706.B89A73F9F3E@mail.nginx.com> Author: mdounin Date: 2012-06-05 13:17:05 +0000 (Tue, 05 Jun 2012) New Revision: 4673 URL: http://trac.nginx.org/nginx/changeset/4673/nginx Log: Merge of r4632: updated openssl used for win32 builds. Modified: branches/stable-1.2/ branches/stable-1.2/misc/GNUmakefile Index: branches/stable-1.2 =================================================================== --- branches/stable-1.2 2012-06-04 14:23:27 UTC (rev 4672) +++ branches/stable-1.2 2012-06-05 13:17:05 UTC (rev 4673) Property changes on: branches/stable-1.2 ___________________________________________________________________ Modified: svn:mergeinfo ## -1 +1 ## -/trunk:4611-4631,4641 +/trunk:4611-4632,4641 \ No newline at end of property Modified: branches/stable-1.2/misc/GNUmakefile =================================================================== --- branches/stable-1.2/misc/GNUmakefile 2012-06-04 14:23:27 UTC (rev 4672) +++ branches/stable-1.2/misc/GNUmakefile 2012-06-05 13:17:05 UTC (rev 4673) @@ -6,7 +6,7 @@ REPO = $(shell svn info | sed -n 's/^Repository Root: //p') OBJS = objs.msvc8 -OPENSSL = openssl-1.0.0i +OPENSSL = openssl-1.0.1c ZLIB = zlib-1.2.5 PCRE = pcre-8.30 From mdounin at mdounin.ru Tue Jun 5 13:36:09 2012 From: mdounin at mdounin.ru (mdounin at mdounin.ru) Date: Tue, 5 Jun 2012 13:36:09 +0000 Subject: [nginx] svn commit: r4674 - trunk/src/os/win32 Message-ID: <20120605133610.0C3803F9EC2@mail.nginx.com> Author: mdounin Date: 2012-06-05 13:36:09 +0000 (Tue, 05 Jun 2012) New Revision: 4674 URL: http://trac.nginx.org/nginx/changeset/4674/nginx Log: Win32: disallowed access to various non-canonical name variants. This includes trailings dots and spaces, NTFS streams (and short names, as previously checked). The checks are now also done in ngx_file_info(), thus allowing to use the "try_files" directive to protect external scripts. Modified: trunk/src/os/win32/ngx_files.c Modified: trunk/src/os/win32/ngx_files.c =================================================================== --- trunk/src/os/win32/ngx_files.c 2012-06-05 13:17:05 UTC (rev 4673) +++ trunk/src/os/win32/ngx_files.c 2012-06-05 13:36:09 UTC (rev 4674) @@ -11,6 +11,8 @@ #define NGX_UTF16_BUFLEN 256 +static ngx_int_t ngx_win32_check_filename(u_char *name, u_short *u, + size_t len); static u_short *ngx_utf8_to_utf16(u_short *utf16, u_char *utf8, size_t *len); @@ -20,8 +22,7 @@ ngx_open_file(u_char *name, u_long mode, u_long create, u_long access) { size_t len; - u_long n; - u_short *u, *lu; + u_short *u; ngx_fd_t fd; ngx_err_t err; u_short utf16[NGX_UTF16_BUFLEN]; @@ -34,25 +35,11 @@ } fd = INVALID_HANDLE_VALUE; - lu = NULL; - if (create == NGX_FILE_OPEN) { - - lu = malloc(len * 2); - if (lu == NULL) { - goto failed; - } - - n = GetLongPathNameW(u, lu, len); - - if (n == 0) { - goto failed; - } - - if (n != len - 1 || _wcsicmp(u, lu) != 0) { - ngx_set_errno(NGX_ENOENT); - goto failed; - } + if (create == NGX_FILE_OPEN + && ngx_win32_check_filename(name, u, len) != NGX_OK) + { + goto failed; } fd = CreateFileW(u, mode, @@ -61,18 +48,12 @@ failed: - err = ngx_errno; - - if (lu) { - ngx_free(lu); - } - if (u != utf16) { + err = ngx_errno; ngx_free(u); + ngx_set_errno(err); } - ngx_set_errno(err); - return fd; } @@ -294,14 +275,14 @@ return NGX_FILE_ERROR; } - rc = GetFileAttributesExW(u, GetFileExInfoStandard, &fa); + rc = NGX_FILE_ERROR; - if (u != utf16) { - err = ngx_errno; - ngx_free(u); - ngx_set_errno(err); + if (ngx_win32_check_filename(file, u, len) != NGX_OK) { + goto failed; } + rc = GetFileAttributesExW(u, GetFileExInfoStandard, &fa); + sb->dwFileAttributes = fa.dwFileAttributes; sb->ftCreationTime = fa.ftCreationTime; sb->ftLastAccessTime = fa.ftLastAccessTime; @@ -309,6 +290,14 @@ sb->nFileSizeHigh = fa.nFileSizeHigh; sb->nFileSizeLow = fa.nFileSizeLow; +failed: + + if (u != utf16) { + err = ngx_errno; + ngx_free(u); + ngx_set_errno(err); + } + return rc; } @@ -640,6 +629,148 @@ } +static ngx_int_t +ngx_win32_check_filename(u_char *name, u_short *u, size_t len) +{ + u_char *p, ch; + u_long n; + u_short *lu; + ngx_err_t err; + enum { + sw_start = 0, + sw_normal, + sw_after_slash, + sw_after_colon, + sw_after_dot + } state; + + /* check for NTFS streams (":"), trailing dots and spaces */ + + lu = NULL; + state = sw_start; + + for (p = name; *p; p++) { + ch = *p; + + switch (state) { + + case sw_start: + + /* + * skip till first "/" to allow paths starting with drive and + * relative path, like "c:html/" + */ + + if (ch == '/' || ch == '\\') { + state = sw_after_slash; + } + + break; + + case sw_normal: + + if (ch == ':') { + state = sw_after_colon; + break; + } + + if (ch == '.' || ch == ' ') { + state = sw_after_dot; + break; + } + + if (ch == '/' || ch == '\\') { + state = sw_after_slash; + break; + } + + break; + + case sw_after_slash: + + if (ch == '/' || ch == '\\') { + break; + } + + if (ch == '.') { + break; + } + + if (ch == ':') { + state = sw_after_colon; + break; + } + + state = sw_normal; + break; + + case sw_after_colon: + + if (ch == '/' || ch == '\\') { + state = sw_after_slash; + break; + } + + goto invalid; + + case sw_after_dot: + + if (ch == '/' || ch == '\\') { + goto invalid; + } + + if (ch == ':') { + goto invalid; + } + + if (ch == '.' || ch == ' ') { + break; + } + + state = sw_normal; + break; + } + } + + if (state == sw_after_dot) { + goto invalid; + } + + /* check if long name match */ + + lu = malloc(len * 2); + if (lu == NULL) { + return NGX_ERROR; + } + + n = GetLongPathNameW(u, lu, len); + + if (n == 0) { + goto failed; + } + + if (n != len - 1 || _wcsicmp(u, lu) != 0) { + goto invalid; + } + + return NGX_OK; + +invalid: + + ngx_set_errno(NGX_ENOENT); + +failed: + + if (lu) { + err = ngx_errno; + ngx_free(lu); + ngx_set_errno(err); + } + + return NGX_ERROR; +} + + static u_short * ngx_utf8_to_utf16(u_short *utf16, u_char *utf8, size_t *len) { From mdounin at mdounin.ru Tue Jun 5 13:37:30 2012 From: mdounin at mdounin.ru (mdounin at mdounin.ru) Date: Tue, 5 Jun 2012 13:37:30 +0000 Subject: [nginx] svn commit: r4675 - trunk/src/http Message-ID: <20120605133730.5F2CB3F9EC2@mail.nginx.com> Author: mdounin Date: 2012-06-05 13:37:29 +0000 (Tue, 05 Jun 2012) New Revision: 4675 URL: http://trac.nginx.org/nginx/changeset/4675/nginx Log: Win32: normalization of trailing dot inside uri. Windows treats "/directory./" identical to "/directory/". Do the same when working on Windows. Note that the behaviour is different from one with last path component (where multiple spaces and dots are ignored by Windows). Modified: trunk/src/http/ngx_http_parse.c Modified: trunk/src/http/ngx_http_parse.c =================================================================== --- trunk/src/http/ngx_http_parse.c 2012-06-05 13:36:09 UTC (rev 4674) +++ trunk/src/http/ngx_http_parse.c 2012-06-05 13:37:29 UTC (rev 4675) @@ -543,6 +543,13 @@ switch (ch) { case '/': +#if (NGX_WIN32) + if (r->uri_ext == p) { + r->complex_uri = 1; + state = sw_uri; + break; + } +#endif r->uri_ext = NULL; state = sw_after_slash_in_uri; break; @@ -1117,6 +1124,12 @@ switch(ch) { #if (NGX_WIN32) case '\\': + if (u - 2 >= r->uri.data + && *(u - 1) == '.' && *(u - 2) != '.') + { + u--; + } + r->uri_ext = NULL; if (p == r->uri_start + r->uri.len) { @@ -1134,6 +1147,13 @@ break; #endif case '/': +#if (NGX_WIN32) + if (u - 2 >= r->uri.data + && *(u - 1) == '.' && *(u - 2) != '.') + { + u--; + } +#endif r->uri_ext = NULL; state = sw_slash; *u++ = ch; From mdounin at mdounin.ru Tue Jun 5 13:38:28 2012 From: mdounin at mdounin.ru (mdounin at mdounin.ru) Date: Tue, 5 Jun 2012 13:38:28 +0000 Subject: [nginx] svn commit: r4676 - trunk/src/http Message-ID: <20120605133828.532973F9F3E@mail.nginx.com> Author: mdounin Date: 2012-06-05 13:38:27 +0000 (Tue, 05 Jun 2012) New Revision: 4676 URL: http://trac.nginx.org/nginx/changeset/4676/nginx Log: Win32: uris with ":$" are now rejected. There are too many problems with special NTFS streams, notably "::$data", "::$index_allocation" and ":$i30:$index_allocation". For now we don't reject all URIs with ":" like Apache does as there are no good reasons seen yet, and there are multiple programs using it in URLs (e.g. MediaWiki). Modified: trunk/src/http/ngx_http_request.c Modified: trunk/src/http/ngx_http_request.c =================================================================== --- trunk/src/http/ngx_http_request.c 2012-06-05 13:37:29 UTC (rev 4675) +++ trunk/src/http/ngx_http_request.c 2012-06-05 13:38:27 UTC (rev 4676) @@ -812,8 +812,29 @@ #if (NGX_WIN32) { - u_char *p; + u_char *p, *last; + p = r->uri.data; + last = r->uri.data + r->uri.len; + + while (p < last) { + + if (*p++ == ':') { + + /* + * this check covers "::$data", "::$index_allocation" and + * ":$i30:$index_allocation" + */ + + if (p < last && *p == '$') { + ngx_log_error(NGX_LOG_INFO, c->log, 0, + "client sent unsafe win32 URI"); + ngx_http_finalize_request(r, NGX_HTTP_BAD_REQUEST); + return; + } + } + } + p = r->uri.data + r->uri.len - 1; while (p > r->uri.data) { @@ -828,11 +849,6 @@ continue; } - if (ngx_strncasecmp(p - 6, (u_char *) "::$data", 7) == 0) { - p -= 7; - continue; - } - break; } From mdounin at mdounin.ru Tue Jun 5 13:47:30 2012 From: mdounin at mdounin.ru (mdounin at mdounin.ru) Date: Tue, 5 Jun 2012 13:47:30 +0000 Subject: [nginx] svn commit: r4677 - trunk/docs/xml/nginx Message-ID: <20120605134730.D84143F9F70@mail.nginx.com> Author: mdounin Date: 2012-06-05 13:47:29 +0000 (Tue, 05 Jun 2012) New Revision: 4677 URL: http://trac.nginx.org/nginx/changeset/4677/nginx Log: nginx-1.3.1-RELEASE Modified: trunk/docs/xml/nginx/changes.xml Modified: trunk/docs/xml/nginx/changes.xml =================================================================== --- trunk/docs/xml/nginx/changes.xml 2012-06-05 13:38:27 UTC (rev 4676) +++ trunk/docs/xml/nginx/changes.xml 2012-06-05 13:47:29 UTC (rev 4677) @@ -9,6 +9,136 @@ nginx changelog + + + + +?????? nginx/Windows ?????????? ????? ? ????? ?????????? URI +? ?? ????????? URI, ?????????? ?????????????????? ":$".
+??????? ????????? ?????????, Positive Research Center. + + +now nginx/Windows ignores trailing dot in URI path component, and +does not allow URIs with ":$" in it.
+Thanks to Vladimir Kochetkov, Positive Research Center. + + + + + +????????? proxy_pass, fastcgi_pass, scgi_pass, uwsgi_pass ? +????????? server ? ????? upstream +?????? ???????????? IPv6-??????. + + +the "proxy_pass", "fastcgi_pass", "scgi_pass", "uwsgi_pass" directives, and +the "server" directive inside the "upstream" block, +now support IPv6 addresses. + + + + + +? ????????? resolver ?????? ????? ????????? ???? ? +???????? IPv6-?????? DNS-????????. + + +the "resolver" directive now support IPv6 addresses and +an optional port specification. + + + + + +????????? least_conn ? ????? upstream. + + +the "least_conn" directive inside the "upstream" block. + + + + + +??? ????????????? ????????? ip_hash +?????? ????? ???????? ???? ????????. + + +it is now possible to specify a weight for servers +while using the "ip_hash" directive. + + + + + +? ??????? ???????? ??? ????????? segmentation fault, +???? ?????????????? ????????? image_filter; +?????? ????????? ? 1.3.0. + + +a segmentation fault might occur in a worker process +if the "image_filter" directive was used; +the bug had appeared in 1.3.0. + + + + + +nginx ?? ????????? ? ??????? ngx_cpp_test_module; +?????? ????????? ? 1.1.12. + + +nginx could not be built with ngx_cpp_test_module; +the bug had appeared in 1.1.12. + + + + + +?????? ? ?????????? ?? SSI ? ??????????? ????? ??? ?? ???????? ????? +????????????????.
+??????? Yichun Zhang. + + +access to variables from SSI and embedded perl module might not work after +reconfiguration.
+Thanks to Yichun Zhang. + + + + + +? ?????? ngx_http_xslt_filter_module.
+??????? Kuramoto Eiji. + + +in the ngx_http_xslt_filter_module.
+Thanks to Kuramoto Eiji. + + + + + +?????? ?????? ??? ????????????? ?????????? $geoip_org.
+??????? ?????? ????????. + + +memory leak if $geoip_org variable was used.
+Thanks to Denis F. Latypoff. + + + + + +? ?????????? proxy_cookie_domain ? proxy_cookie_path. + + +in the "proxy_cookie_domain" and "proxy_cookie_path" directives. + + + + + + From mdounin at mdounin.ru Tue Jun 5 13:47:50 2012 From: mdounin at mdounin.ru (mdounin at mdounin.ru) Date: Tue, 5 Jun 2012 13:47:50 +0000 Subject: [nginx] svn commit: r4678 - tags Message-ID: <20120605134751.885D33F9FA6@mail.nginx.com> Author: mdounin Date: 2012-06-05 13:47:50 +0000 (Tue, 05 Jun 2012) New Revision: 4678 URL: http://trac.nginx.org/nginx/changeset/4678/nginx Log: release-1.3.1 tag Added: tags/release-1.3.1/ From mdounin at mdounin.ru Tue Jun 5 13:52:37 2012 From: mdounin at mdounin.ru (mdounin at mdounin.ru) Date: Tue, 5 Jun 2012 13:52:37 +0000 Subject: [nginx] svn commit: r4679 - in branches/stable-1.2: . src/http src/os/win32 Message-ID: <20120605135237.F3D803F9F70@mail.nginx.com> Author: mdounin Date: 2012-06-05 13:52:37 +0000 (Tue, 05 Jun 2012) New Revision: 4679 URL: http://trac.nginx.org/nginx/changeset/4679/nginx Log: Merge of r4674, r4675, r4676: win32 fixes. *) Win32: disallowed access to various non-canonical name variants. This includes trailings dots and spaces, NTFS streams (and short names, as previously checked). The checks are now also done in ngx_file_info(), thus allowing to use the "try_files" directive to protect external scripts. *) Win32: normalization of trailing dot inside uri. Windows treats "/directory./" identical to "/directory/". Do the same when working on Windows. Note that the behaviour is different from one with last path component (where multiple spaces and dots are ignored by Windows). *) Win32: uris with ":$" are now rejected. There are too many problems with special NTFS streams, notably "::$data", "::$index_allocation" and ":$i30:$index_allocation". For now we don't reject all URIs with ":" like Apache does as there are no good reasons seen yet, and there are multiple programs using it in URLs (e.g. MediaWiki). Modified: branches/stable-1.2/ branches/stable-1.2/src/http/ngx_http_parse.c branches/stable-1.2/src/http/ngx_http_request.c branches/stable-1.2/src/os/win32/ngx_files.c Index: branches/stable-1.2 =================================================================== --- branches/stable-1.2 2012-06-05 13:47:50 UTC (rev 4678) +++ branches/stable-1.2 2012-06-05 13:52:37 UTC (rev 4679) Property changes on: branches/stable-1.2 ___________________________________________________________________ Modified: svn:mergeinfo ## -1 +1 ## -/trunk:4611-4632,4641 +/trunk:4611-4632,4641,4674-4676 \ No newline at end of property Modified: branches/stable-1.2/src/http/ngx_http_parse.c =================================================================== --- branches/stable-1.2/src/http/ngx_http_parse.c 2012-06-05 13:47:50 UTC (rev 4678) +++ branches/stable-1.2/src/http/ngx_http_parse.c 2012-06-05 13:52:37 UTC (rev 4679) @@ -543,6 +543,13 @@ switch (ch) { case '/': +#if (NGX_WIN32) + if (r->uri_ext == p) { + r->complex_uri = 1; + state = sw_uri; + break; + } +#endif r->uri_ext = NULL; state = sw_after_slash_in_uri; break; @@ -1117,6 +1124,12 @@ switch(ch) { #if (NGX_WIN32) case '\\': + if (u - 2 >= r->uri.data + && *(u - 1) == '.' && *(u - 2) != '.') + { + u--; + } + r->uri_ext = NULL; if (p == r->uri_start + r->uri.len) { @@ -1134,6 +1147,13 @@ break; #endif case '/': +#if (NGX_WIN32) + if (u - 2 >= r->uri.data + && *(u - 1) == '.' && *(u - 2) != '.') + { + u--; + } +#endif r->uri_ext = NULL; state = sw_slash; *u++ = ch; Modified: branches/stable-1.2/src/http/ngx_http_request.c =================================================================== --- branches/stable-1.2/src/http/ngx_http_request.c 2012-06-05 13:47:50 UTC (rev 4678) +++ branches/stable-1.2/src/http/ngx_http_request.c 2012-06-05 13:52:37 UTC (rev 4679) @@ -812,8 +812,29 @@ #if (NGX_WIN32) { - u_char *p; + u_char *p, *last; + p = r->uri.data; + last = r->uri.data + r->uri.len; + + while (p < last) { + + if (*p++ == ':') { + + /* + * this check covers "::$data", "::$index_allocation" and + * ":$i30:$index_allocation" + */ + + if (p < last && *p == '$') { + ngx_log_error(NGX_LOG_INFO, c->log, 0, + "client sent unsafe win32 URI"); + ngx_http_finalize_request(r, NGX_HTTP_BAD_REQUEST); + return; + } + } + } + p = r->uri.data + r->uri.len - 1; while (p > r->uri.data) { @@ -828,11 +849,6 @@ continue; } - if (ngx_strncasecmp(p - 6, (u_char *) "::$data", 7) == 0) { - p -= 7; - continue; - } - break; } Modified: branches/stable-1.2/src/os/win32/ngx_files.c =================================================================== --- branches/stable-1.2/src/os/win32/ngx_files.c 2012-06-05 13:47:50 UTC (rev 4678) +++ branches/stable-1.2/src/os/win32/ngx_files.c 2012-06-05 13:52:37 UTC (rev 4679) @@ -11,6 +11,8 @@ #define NGX_UTF16_BUFLEN 256 +static ngx_int_t ngx_win32_check_filename(u_char *name, u_short *u, + size_t len); static u_short *ngx_utf8_to_utf16(u_short *utf16, u_char *utf8, size_t *len); @@ -20,8 +22,7 @@ ngx_open_file(u_char *name, u_long mode, u_long create, u_long access) { size_t len; - u_long n; - u_short *u, *lu; + u_short *u; ngx_fd_t fd; ngx_err_t err; u_short utf16[NGX_UTF16_BUFLEN]; @@ -34,25 +35,11 @@ } fd = INVALID_HANDLE_VALUE; - lu = NULL; - if (create == NGX_FILE_OPEN) { - - lu = malloc(len * 2); - if (lu == NULL) { - goto failed; - } - - n = GetLongPathNameW(u, lu, len); - - if (n == 0) { - goto failed; - } - - if (n != len - 1 || _wcsicmp(u, lu) != 0) { - ngx_set_errno(NGX_ENOENT); - goto failed; - } + if (create == NGX_FILE_OPEN + && ngx_win32_check_filename(name, u, len) != NGX_OK) + { + goto failed; } fd = CreateFileW(u, mode, @@ -61,18 +48,12 @@ failed: - err = ngx_errno; - - if (lu) { - ngx_free(lu); - } - if (u != utf16) { + err = ngx_errno; ngx_free(u); + ngx_set_errno(err); } - ngx_set_errno(err); - return fd; } @@ -294,14 +275,14 @@ return NGX_FILE_ERROR; } - rc = GetFileAttributesExW(u, GetFileExInfoStandard, &fa); + rc = NGX_FILE_ERROR; - if (u != utf16) { - err = ngx_errno; - ngx_free(u); - ngx_set_errno(err); + if (ngx_win32_check_filename(file, u, len) != NGX_OK) { + goto failed; } + rc = GetFileAttributesExW(u, GetFileExInfoStandard, &fa); + sb->dwFileAttributes = fa.dwFileAttributes; sb->ftCreationTime = fa.ftCreationTime; sb->ftLastAccessTime = fa.ftLastAccessTime; @@ -309,6 +290,14 @@ sb->nFileSizeHigh = fa.nFileSizeHigh; sb->nFileSizeLow = fa.nFileSizeLow; +failed: + + if (u != utf16) { + err = ngx_errno; + ngx_free(u); + ngx_set_errno(err); + } + return rc; } @@ -640,6 +629,148 @@ } +static ngx_int_t +ngx_win32_check_filename(u_char *name, u_short *u, size_t len) +{ + u_char *p, ch; + u_long n; + u_short *lu; + ngx_err_t err; + enum { + sw_start = 0, + sw_normal, + sw_after_slash, + sw_after_colon, + sw_after_dot + } state; + + /* check for NTFS streams (":"), trailing dots and spaces */ + + lu = NULL; + state = sw_start; + + for (p = name; *p; p++) { + ch = *p; + + switch (state) { + + case sw_start: + + /* + * skip till first "/" to allow paths starting with drive and + * relative path, like "c:html/" + */ + + if (ch == '/' || ch == '\\') { + state = sw_after_slash; + } + + break; + + case sw_normal: + + if (ch == ':') { + state = sw_after_colon; + break; + } + + if (ch == '.' || ch == ' ') { + state = sw_after_dot; + break; + } + + if (ch == '/' || ch == '\\') { + state = sw_after_slash; + break; + } + + break; + + case sw_after_slash: + + if (ch == '/' || ch == '\\') { + break; + } + + if (ch == '.') { + break; + } + + if (ch == ':') { + state = sw_after_colon; + break; + } + + state = sw_normal; + break; + + case sw_after_colon: + + if (ch == '/' || ch == '\\') { + state = sw_after_slash; + break; + } + + goto invalid; + + case sw_after_dot: + + if (ch == '/' || ch == '\\') { + goto invalid; + } + + if (ch == ':') { + goto invalid; + } + + if (ch == '.' || ch == ' ') { + break; + } + + state = sw_normal; + break; + } + } + + if (state == sw_after_dot) { + goto invalid; + } + + /* check if long name match */ + + lu = malloc(len * 2); + if (lu == NULL) { + return NGX_ERROR; + } + + n = GetLongPathNameW(u, lu, len); + + if (n == 0) { + goto failed; + } + + if (n != len - 1 || _wcsicmp(u, lu) != 0) { + goto invalid; + } + + return NGX_OK; + +invalid: + + ngx_set_errno(NGX_ENOENT); + +failed: + + if (lu) { + err = ngx_errno; + ngx_free(lu); + ngx_set_errno(err); + } + + return NGX_ERROR; +} + + static u_short * ngx_utf8_to_utf16(u_short *utf16, u_char *utf8, size_t *len) { From mdounin at mdounin.ru Tue Jun 5 14:01:45 2012 From: mdounin at mdounin.ru (mdounin at mdounin.ru) Date: Tue, 5 Jun 2012 14:01:45 +0000 Subject: [nginx] svn commit: r4680 - branches/stable-1.2/docs/xml/nginx Message-ID: <20120605140145.91C713F9EC2@mail.nginx.com> Author: mdounin Date: 2012-06-05 14:01:45 +0000 (Tue, 05 Jun 2012) New Revision: 4680 URL: http://trac.nginx.org/nginx/changeset/4680/nginx Log: nginx-1.2.1-RELEASE Modified: branches/stable-1.2/docs/xml/nginx/changes.xml Modified: branches/stable-1.2/docs/xml/nginx/changes.xml =================================================================== --- branches/stable-1.2/docs/xml/nginx/changes.xml 2012-06-05 13:52:37 UTC (rev 4679) +++ branches/stable-1.2/docs/xml/nginx/changes.xml 2012-06-05 14:01:45 UTC (rev 4680) @@ -9,6 +9,158 @@ nginx changelog + + + + +?????? nginx/Windows ?????????? ????? ? ????? ?????????? URI +? ?? ????????? URI, ?????????? ?????????????????? ":$".
+??????? ????????? ?????????, Positive Research Center. + + +now nginx/Windows ignores trailing dot in URI path component, and +does not allow URIs with ":$" in it.
+Thanks to Vladimir Kochetkov, Positive Research Center. + + + + + +????????? debug_connection ?????? ???????????? IPv6-?????? +? ???????? "unix:". + + +the "debug_connection" directive now supports IPv6 addresses +and the "unix:" parameter. + + + + + +????????? set_real_ip_from ? ???????? proxy +????????? geo ?????? ???????????? IPv6-??????. + + +the "set_real_ip_from" directive and the "proxy" parameter +of the "geo" directive now support IPv6 addresses. + + + + + +????????? real_ip_recursive, geoip_proxy ? geoip_proxy_recursive. + + +the "real_ip_recursive", "geoip_proxy", and "geoip_proxy_recursive" directives. + + + + + +???????? proxy_recursive ????????? geo. + + +the "proxy_recursive" parameter of the "geo" directive. + + + + + +? ??????? ???????? ??? ????????? segmentation fault, +???? ?????????????? ????????? resolver. + + +a segmentation fault might occur in a worker process +if the "resolver" directive was used. + + + + + +? ??????? ???????? ??? ????????? segmentation fault, +???? ?????????????? ????????? fastcgi_pass, scgi_pass ??? uwsgi_pass +? ?????? ????????? ???????????? ?????. + + +a segmentation fault might occur in a worker process +if the "fastcgi_pass", "scgi_pass", or "uwsgi_pass" directives were used +and backend returned incorrect response. + + + + + +? ??????? ???????? ??? ????????? segmentation fault, +???? ?????????????? ????????? rewrite ? ? ????? ?????????? ??????? ? ?????? +?????? ?????????????? ??????????. + + +a segmentation fault might occur in a worker process +if the "rewrite" directive was used and new request arguments +in a replacement used variables. + + + + + +nginx ??? ????????? ?????????, +???? ???? ?????????? ??????????? ?? ?????????? ???????? ??????. + + +nginx might hog CPU +if the open file resource limit was reached. + + + + + +??? ????????????? ????????? proxy_next_upstream ? ?????????? http_404 +nginx ??? ?????????? ?????????? ???????, ???? ? ????? upstream ??? +???? ?? ???? ?????? ? ?????? backup. + + +nginx might loop infinitely over backends +if the "proxy_next_upstream" directive with the "http_404" parameter was used +and there were backup servers specified in an upstream block. + + + + + +??? ????????????? ????????? ip_hash +????????? ????????? down ????????? server +????? ????????? ? ????????? ????????????????? ???????? ????? ?????????. + + +adding the "down" parameter of the "server" directive +might cause unneeded client redistribution among backend servers +if the "ip_hash" directive was used. + + + + + +?????? ???????.
+??????? Yichun Zhang. + + +socket leak.
+Thanks to Yichun Zhang. + + + + + +? ?????? ngx_http_fastcgi_module. + + +in the ngx_http_fastcgi_module. + + + + + + From mdounin at mdounin.ru Tue Jun 5 14:02:03 2012 From: mdounin at mdounin.ru (mdounin at mdounin.ru) Date: Tue, 5 Jun 2012 14:02:03 +0000 Subject: [nginx] svn commit: r4681 - tags Message-ID: <20120605140203.93B8A3F9EC2@mail.nginx.com> Author: mdounin Date: 2012-06-05 14:02:03 +0000 (Tue, 05 Jun 2012) New Revision: 4681 URL: http://trac.nginx.org/nginx/changeset/4681/nginx Log: release-1.2.1 tag Added: tags/release-1.2.1/ From me at prudhvi.net Thu Jun 7 01:42:57 2012 From: me at prudhvi.net (Prudhvi Krishna Surapaneni) Date: Wed, 6 Jun 2012 18:42:57 -0700 Subject: Parsing Variables to access them in a custom module Message-ID: <20120607014257.GA5478@prudhvi.net> Hi All, I am trying to write a module that needs to consume the regex matched variables like $1 and $2 and pass them as values to the module. Here is what i am trying to do /* -- CONFIG -- */ location ~ /foo/([a-zA-Z0-9]+)/bar/([0-9]+) { my_module; my_module_first $1; my_module_second $2; } /* -- CONFIG -- */ Inside my module i have defined them as follows. /* -- CODE -- */ typedef stuct { ngx_str_t my_module_first; ngx_str_t my_module_second; } ngx_http_my_module_loc_conf_t /* -- CODE -- */ When trying to access the values from those variables i get the value $1 and $2 instead of the substitution. What am i missing?. Are these variables only availble for the rewrite module?. Thanks, Prudhvi Krishna Surapaneni. -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 490 bytes Desc: Digital signature URL: From vshebordaev at mail.ru Thu Jun 7 05:02:44 2012 From: vshebordaev at mail.ru (Vladimir Shebordaev) Date: Thu, 7 Jun 2012 09:02:44 +0400 Subject: Parsing Variables to access them in a custom module In-Reply-To: <20120607014257.GA5478@prudhvi.net> References: <20120607014257.GA5478@prudhvi.net> Message-ID: 2012/6/7 Prudhvi Krishna Surapaneni : > Hi All, > ? ? ? ?I am trying to write a module that needs to consume the regex matched > ? ? ? ?variables like $1 and $2 and pass them as values to the module. > > ? ? ? ?Here is what i am trying to do > > ? ? ? ?/* -- CONFIG -- */ > > ? ? ? ? ? ? ? ? ? ? ? ?location ~ /foo/([a-zA-Z0-9]+)/bar/([0-9]+) { > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?my_module; > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?my_module_first $1; > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?my_module_second $2; > ? ? ? ? ? ? ? ? ? ? ? ?} > > ? ? ? ?/* -- CONFIG -- */ > > ? ? ? ?Inside my module i have defined them as follows. > > ? ? ? ?/* -- CODE -- */ > > ? ? ? ? ? ? ? ?typedef stuct { > ? ? ? ? ? ? ? ? ? ? ? ?ngx_str_t my_module_first; > ? ? ? ? ? ? ? ? ? ? ? ?ngx_str_t my_module_second; > ? ? ? ? ? ? ? ?} ngx_http_my_module_loc_conf_t > > ? ?/* -- CODE -- */ > > ? ? ? ?When trying to access the values from those variables i get the > ? ? ? ?value $1 and $2 instead of the substitution. What am i missing?. > > ? ? ? ?Are these variables only availble for the rewrite module?. Not particularly. But you should use the script engine to get captures evaluated at run-time. See, e.g. proxy module for an example. > > Thanks, > Prudhvi Krishna Surapaneni. > From agentzh at gmail.com Thu Jun 7 09:36:36 2012 From: agentzh at gmail.com (agentzh) Date: Thu, 7 Jun 2012 17:36:36 +0800 Subject: [PATCH] Fixing segfaults in ngx_poll_del_event at worker exit Message-ID: Hello! I observed consistent segmentation faults when the Nginx worker process exits due to HUP signal reload. The Nginx server has both resolver and ngx_poll_module configured. The backtrace looks like this: Process terminating with default action of signal 11 (SIGSEGV) Access not within mapped region at address 0x50 at 0x437AF8: ngx_poll_del_event (ngx_poll_module.c:208) by 0x4220C0: ngx_close_connection (ngx_connection.c:872) by 0x4267D1: ngx_resolver_cleanup (ngx_resolver.c:223) by 0x4176D0: ngx_destroy_pool (ngx_palloc.c:55) by 0x43317C: ngx_worker_process_exit (ngx_process_cycle.c:1059) by 0x43327D: ngx_worker_process_cycle (ngx_process_cycle.c:800) by 0x431AAE: ngx_spawn_process (ngx_process.c:198) by 0x4328C9: ngx_start_worker_processes (ngx_process_cycle.c:365) by 0x4342ED: ngx_master_process_cycle (ngx_process_cycle.c:250) by 0x416653: main (nginx.c:410) At this point of failure, ngx_cycle->files was a NULL pointer (due to the assignment of ngx_exit_cycle to ngx_cycle in ngx_worker_process_exit) but ngx_poll_del_event tragically accessed this array by an index, which was an invalid read. Below attaches a patch for nginx 1.0.15 that fixes this segfault. It can also be cleanly applied to at least nginx 1.2.1. Comments will be highly appreciated as usual :) Thanks! -agentzh --- nginx-1.0.15/src/event/modules/ngx_poll_module.c 2012-02-06 04:02:59.000000000 +0800 +++ nginx-1.0.15-patched/src/event/modules/ngx_poll_module.c 2012-06-07 17:22:43.538168219 +0800 @@ -205,19 +205,21 @@ event_list[ev->index] = event_list[nevents]; - c = ngx_cycle->files[event_list[nevents].fd]; + if (ngx_cycle->files) { + c = ngx_cycle->files[event_list[nevents].fd]; - if (c->fd == -1) { - ngx_log_error(NGX_LOG_ALERT, ev->log, 0, - "unexpected last event"); - - } else { - if (c->read->index == (ngx_uint_t) nevents) { - c->read->index = ev->index; - } - - if (c->write->index == (ngx_uint_t) nevents) { - c->write->index = ev->index; + if (c->fd == -1) { + ngx_log_error(NGX_LOG_ALERT, ev->log, 0, + "unexpected last event"); + + } else { + if (c->read->index == (ngx_uint_t) nevents) { + c->read->index = ev->index; + } + + if (c->write->index == (ngx_uint_t) nevents) { + c->write->index = ev->index; + } } } } -------------- next part -------------- A non-text attachment was scrubbed... Name: nginx-1.0.15-poll_del_event_at_exit.patch Type: application/octet-stream Size: 1370 bytes Desc: not available URL: From mdounin at mdounin.ru Thu Jun 7 10:36:54 2012 From: mdounin at mdounin.ru (Maxim Dounin) Date: Thu, 7 Jun 2012 14:36:54 +0400 Subject: Parsing Variables to access them in a custom module In-Reply-To: References: <20120607014257.GA5478@prudhvi.net> Message-ID: <20120607103654.GA31671@mdounin.ru> Hello! On Thu, Jun 07, 2012 at 09:02:44AM +0400, Vladimir Shebordaev wrote: > 2012/6/7 Prudhvi Krishna Surapaneni : > > Hi All, > > ? ? ? ?I am trying to write a module that needs to consume the regex matched > > ? ? ? ?variables like $1 and $2 and pass them as values to the module. > > > > ? ? ? ?Here is what i am trying to do > > > > ? ? ? ?/* -- CONFIG -- */ > > > > ? ? ? ? ? ? ? ? ? ? ? ?location ~ /foo/([a-zA-Z0-9]+)/bar/([0-9]+) { > > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?my_module; > > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?my_module_first $1; > > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?my_module_second $2; > > ? ? ? ? ? ? ? ? ? ? ? ?} > > > > ? ? ? ?/* -- CONFIG -- */ > > > > ? ? ? ?Inside my module i have defined them as follows. > > > > ? ? ? ?/* -- CODE -- */ > > > > ? ? ? ? ? ? ? ?typedef stuct { > > ? ? ? ? ? ? ? ? ? ? ? ?ngx_str_t my_module_first; > > ? ? ? ? ? ? ? ? ? ? ? ?ngx_str_t my_module_second; > > ? ? ? ? ? ? ? ?} ngx_http_my_module_loc_conf_t > > > > ? ?/* -- CODE -- */ > > > > ? ? ? ?When trying to access the values from those variables i get the > > ? ? ? ?value $1 and $2 instead of the substitution. What am i missing?. > > > > ? ? ? ?Are these variables only availble for the rewrite module?. > > Not particularly. But you should use the script engine to get captures > evaluated at run-time. See, e.g. proxy module for an example. I would rather recommend using complex values here, as it's simpliest way to use variables. Basically all it needs is using ngx_http_set_complex_value_slot instead of ngx_conf_set_str_slot in configuration directive definition, and an additional ngx_http_complex_value() call before use. Complete example can be found in secure link module. Maxim Dounin From ru at nginx.com Fri Jun 8 09:41:55 2012 From: ru at nginx.com (ru at nginx.com) Date: Fri, 8 Jun 2012 09:41:55 +0000 Subject: [nginx] svn commit: r4682 - in trunk: auto src/http/modules Message-ID: <20120608094155.C2C083F9F49@mail.nginx.com> Author: ru Date: 2012-06-08 09:41:55 +0000 (Fri, 08 Jun 2012) New Revision: 4682 URL: http://trac.nginx.org/nginx/changeset/4682/nginx Log: Fixed spelling of "endianness", and called it "byte ordering" in the user visible part. Added: trunk/auto/endianness Removed: trunk/auto/endianess Modified: trunk/auto/unix trunk/src/http/modules/ngx_http_geo_module.c Deleted: trunk/auto/endianess =================================================================== --- trunk/auto/endianess 2012-06-05 14:02:03 UTC (rev 4681) +++ trunk/auto/endianess 2012-06-08 09:41:55 UTC (rev 4682) @@ -1,45 +0,0 @@ - -# Copyright (C) Igor Sysoev -# Copyright (C) Nginx, Inc. - - -echo $ngx_n "checking for system endianess ...$ngx_c" -echo >> $NGX_ERR -echo "checking for system endianess" >> $NGX_ERR - - -cat << END > $NGX_AUTOTEST.c - -int main() { - int i = 0x11223344; - char *p; - - p = (char *) &i; - if (*p == 0x44) return 0; - return 1; -} - -END - -ngx_test="$CC $CC_TEST_FLAGS $CC_AUX_FLAGS \ - -o $NGX_AUTOTEST $NGX_AUTOTEST.c $NGX_LD_OPT $ngx_feature_libs" - -eval "$ngx_test >> $NGX_AUTOCONF_ERR 2>&1" - -if [ -x $NGX_AUTOTEST ]; then - if $NGX_AUTOTEST >/dev/null 2>&1; then - echo " little endianess" - have=NGX_HAVE_LITTLE_ENDIAN . auto/have - else - echo " big endianess" - fi - - rm $NGX_AUTOTEST* - -else - rm $NGX_AUTOTEST* - - echo - echo "$0: error: can not detect system endianess" - exit 1 -fi Copied: trunk/auto/endianness (from rev 4681, trunk/auto/endianess) =================================================================== --- trunk/auto/endianness (rev 0) +++ trunk/auto/endianness 2012-06-08 09:41:55 UTC (rev 4682) @@ -0,0 +1,45 @@ + +# Copyright (C) Igor Sysoev +# Copyright (C) Nginx, Inc. + + +echo $ngx_n "checking for system byte ordering ...$ngx_c" +echo >> $NGX_ERR +echo "checking for system byte ordering" >> $NGX_ERR + + +cat << END > $NGX_AUTOTEST.c + +int main() { + int i = 0x11223344; + char *p; + + p = (char *) &i; + if (*p == 0x44) return 0; + return 1; +} + +END + +ngx_test="$CC $CC_TEST_FLAGS $CC_AUX_FLAGS \ + -o $NGX_AUTOTEST $NGX_AUTOTEST.c $NGX_LD_OPT $ngx_feature_libs" + +eval "$ngx_test >> $NGX_AUTOCONF_ERR 2>&1" + +if [ -x $NGX_AUTOTEST ]; then + if $NGX_AUTOTEST >/dev/null 2>&1; then + echo " little endian" + have=NGX_HAVE_LITTLE_ENDIAN . auto/have + else + echo " big endian" + fi + + rm $NGX_AUTOTEST* + +else + rm $NGX_AUTOTEST* + + echo + echo "$0: error: cannot detect system byte ordering" + exit 1 +fi Modified: trunk/auto/unix =================================================================== --- trunk/auto/unix 2012-06-05 14:02:03 UTC (rev 4681) +++ trunk/auto/unix 2012-06-08 09:41:55 UTC (rev 4682) @@ -464,7 +464,7 @@ . auto/types/uintptr_t -. auto/endianess +. auto/endianness ngx_type="size_t"; . auto/types/sizeof ngx_param=NGX_MAX_SIZE_T_VALUE; ngx_value=$ngx_max_value; . auto/types/value Modified: trunk/src/http/modules/ngx_http_geo_module.c =================================================================== --- trunk/src/http/modules/ngx_http_geo_module.c 2012-06-05 14:02:03 UTC (rev 4681) +++ trunk/src/http/modules/ngx_http_geo_module.c 2012-06-08 09:41:55 UTC (rev 4682) @@ -145,7 +145,7 @@ u_char GEORNG[6]; u_char version; u_char ptr_size; - uint32_t endianess; + uint32_t endianness; uint32_t crc32; } ngx_http_geo_header_t; From goelvivek2011 at gmail.com Sat Jun 9 12:57:03 2012 From: goelvivek2011 at gmail.com (vivek goel) Date: Sat, 9 Jun 2012 18:27:03 +0530 Subject: How to identify correct requirement of number of worker process ? Message-ID: I want to identify the correct setting for nginx worker count. on my server I am serving big static files and io read is slow. Is there a way I can identify that all nginx worker process are busy in doing work and I should increase number of nginx worker? regards Vivek Goel -------------- next part -------------- An HTML attachment was scrubbed... URL: From vshebordaev at mail.ru Sat Jun 9 14:09:34 2012 From: vshebordaev at mail.ru (Vladimir Shebordaev) Date: Sat, 9 Jun 2012 18:09:34 +0400 Subject: How to identify correct requirement of number of worker process ? In-Reply-To: References: Message-ID: Hi, 2012/6/9 vivek goel : > I want to identify the correct setting for nginx worker count. > on my server I am serving big static files and io read is slow. > Is there a way I can identify that all nginx worker process are busy in > doing work and I should increase number of nginx worker? > Would you please elaborate on your setup? Do you make use of async/direct IO, cached files or whatever? If you do and still experience slow io, you'd better upgrade your hardware. Anyways, increasing number of worker threads over recommended number of CPUs is not an option, since you are definitely io-bound. > > regards > Vivek Goel > From sunyxing at gmail.com Tue Jun 12 11:29:56 2012 From: sunyxing at gmail.com (Thomas) Date: Tue, 12 Jun 2012 19:29:56 +0800 Subject: Is there any way to redirect request to another server conf? Message-ID: Hi All, I'm trying to use nginx as a proxy server which can judge server name by url. e.g. server { listen 80 default_server; my_rewrite_directive; } server { server_name a.com; root html/a; } server { server_name b.com; root html/b; } then GET /a.com/index.html will be redirected to server a.com; I tried to write a module to accomplish this by modifying members in http request structure, but it seems that r->srv_conf has already been set before running module phases. I tried internal redirect and it didn't help. so my question is, Is there any way to redirect request to another server conf without modifying the nginx core? Thanks in advance. -------------- next part -------------- An HTML attachment was scrubbed... URL: From appa at perusio.net Tue Jun 12 11:40:09 2012 From: appa at perusio.net (=?UTF-8?B?QW50w7NuaW8=?= P. P. Almeida) Date: Tue, 12 Jun 2012 13:40:09 +0200 Subject: Is there any way to redirect request to another server conf? In-Reply-To: References: Message-ID: <87vciwg3mu.wl%appa@perusio.net> On 12 Jun 2012 13h29 CEST, sunyxing at gmail.com wrote: > Hi All, > > I'm trying to use nginx as a proxy server which can judge server > name by url. > > e.g. > > server { > listen 80 default_server; > my_rewrite_directive; > } > > server { > server_name a.com; > root html/a; > } > > server { > server_name b.com; > root html/b; > } > > then GET /a.com/index.html will be redirected to server a.com; Doesn't this do what you need? server { listen 80 default_server; location /a.com/index.html { return 302 http://a.com$request_uri; } location /b.com/index.html { return 302 http://b.com$request_uri; } } --- appa From sunyxing at gmail.com Tue Jun 12 13:22:29 2012 From: sunyxing at gmail.com (Thomas) Date: Tue, 12 Jun 2012 21:22:29 +0800 Subject: nginx-devel Digest, Vol 32, Issue 13 In-Reply-To: References: Message-ID: On Tue, Jun 12, 2012 at 8:00 PM, wrote: > > Send nginx-devel mailing list submissions to > ? ? ? ?nginx-devel at nginx.org > > To subscribe or unsubscribe via the World Wide Web, visit > ? ? ? ?http://mailman.nginx.org/mailman/listinfo/nginx-devel > or, via email, send a message with subject or body 'help' to > ? ? ? ?nginx-devel-request at nginx.org > > You can reach the person managing the list at > ? ? ? ?nginx-devel-owner at nginx.org > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of nginx-devel digest..." > > > Today's Topics: > > ? 1. Is there any way to redirect request to another server conf? > ? ? ?(Thomas) > ? 2. Re: Is there any way to redirect request to another server > ? ? ?conf? (Ant?nio P. P. Almeida) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Tue, 12 Jun 2012 19:29:56 +0800 > From: Thomas > To: nginx-devel at nginx.org > Subject: Is there any way to redirect request to another server conf? > Message-ID: > > ? > Content-Type: text/plain; charset="iso-8859-1" > > Hi All, > > ? ?I'm trying to use nginx as a proxy server which can judge server name > by url. > > ? e.g. > > ? server { > ? ? ?listen 80 default_server; > ? ? ?my_rewrite_directive; > ? } > > ? server { > ? ? ?server_name a.com; > ? ? ?root html/a; > ? } > > ? server { > ? ? ?server_name b.com; > ? ? ?root html/b; > ? } > > ?then ?GET /a.com/index.html ?will be redirected to server a.com; > > I tried to write a module to accomplish this by modifying members in http > request structure, but it seems that r->srv_conf has already been set > before running module phases. I tried internal redirect and it didn't > help. > so my question is, > > Is there any way to redirect request to another server conf without > modifying the nginx core? > > Thanks in advance. > -------------- next part -------------- > An HTML attachment was scrubbed... > URL: > > > ------------------------------ > > Message: 2 > Date: Tue, 12 Jun 2012 13:40:09 +0200 > From: Ant?nio P. P. Almeida > To: nginx-devel at nginx.org > Subject: Re: Is there any way to redirect request to another server > ? ? ? ?conf? > Message-ID: <87vciwg3mu.wl%appa at perusio.net> > Content-Type: text/plain; charset=US-ASCII > > On 12 Jun 2012 13h29 CEST, sunyxing at gmail.com wrote: > > > Hi All, > > > > I'm trying to use nginx as a proxy server which can judge server > > name by url. > > > > e.g. > > > > server { > > listen 80 default_server; > > my_rewrite_directive; > > } > > > > server { > > server_name a.com; > > root html/a; > > } > > > > server { > > server_name b.com; > > root html/b; > > } > > > > then ?GET /a.com/index.html ?will be redirected to server a.com; > > Doesn't this do what you need? > > server { > ? ?listen 80 default_server; > > ? ?location /a.com/index.html { > ? ? ? ?return 302 http://a.com$request_uri; > ? ?} > > ? ?location /b.com/index.html { > ? ? ? ?return 302 http://b.com$request_uri; > ? ?} > } > > --- appa > Thanks for advice. Since my servers are supposed to handle the request directly and independently, 302/301 are not acceptable in this situation. From sunyxing at gmail.com Tue Jun 12 13:39:23 2012 From: sunyxing at gmail.com (Thomas) Date: Tue, 12 Jun 2012 21:39:23 +0800 Subject: Is there any way to redirect request to another server conf? Message-ID: > Message: 2 > Date: Tue, 12 Jun 2012 13:40:09 +0200 > From: Ant?nio P. P. Almeida > To: nginx-devel at nginx.org > Subject: Re: Is there any way to redirect request to another server > conf? > Message-ID: <87vciwg3mu.wl%appa at perusio.net> > Content-Type: text/plain; charset=US-ASCII > > On 12 Jun 2012 13h29 CEST, sunyxing at gmail.com wrote: > > > Hi All, > > > > I'm trying to use nginx as a proxy server which can judge server > > name by url. > > > > e.g. > > > > server { > > listen 80 default_server; > > my_rewrite_directive; > > } > > > > server { > > server_name a.com; > > root html/a; > > } > > > > server { > > server_name b.com; > > root html/b; > > } > > > > then GET /a.com/index.html will be redirected to server a.com; > > Doesn't this do what you need? > > server { > listen 80 default_server; > > location /a.com/index.html { > return 302 http://a.com$request_uri; > } > > location /b.com/index.html { > return 302 http://b.com$request_uri; > } > } > > --- appa > Thanks for advice. Since my servers are supposed to handle the request directly and independently, 302/301 are not acceptable in this situation. SORRY to send the last mail in a wrong way, hope I'm not bothering anybody. From appa at perusio.net Tue Jun 12 13:47:29 2012 From: appa at perusio.net (=?UTF-8?B?QW50w7NuaW8=?= P. P. Almeida) Date: Tue, 12 Jun 2012 15:47:29 +0200 Subject: Is there any way to redirect request to another server conf? In-Reply-To: References: Message-ID: <87sje0fxqm.wl%appa@perusio.net> On 12 Jun 2012 15h39 CEST, sunyxing at gmail.com wrote: >> Message: 2 >> Date: Tue, 12 Jun 2012 13:40:09 +0200 >> From: Ant?nio P. P. Almeida >> To: nginx-devel at nginx.org >> Subject: Re: Is there any way to redirect request to another server >> conf? >> Message-ID: <87vciwg3mu.wl%appa at perusio.net> >> Content-Type: text/plain; charset=US-ASCII >> >> On 12 Jun 2012 13h29 CEST, sunyxing at gmail.com wrote: >> >>> Hi All, >>> >>> I'm trying to use nginx as a proxy server which can judge server >>> name by url. >>> >>> e.g. >>> >>> server { >>> listen 80 default_server; >>> my_rewrite_directive; >>> } >>> >>> server { >>> server_name a.com; >>> root html/a; >>> } >>> >>> server { >>> server_name b.com; >>> root html/b; >>> } >>> >>> then GET /a.com/index.html will be redirected to server a.com; >> >> Doesn't this do what you need? >> >> server { >> listen 80 default_server; >> >> location /a.com/index.html { >> return 302 http://a.com$request_uri; >> } >> >> location /b.com/index.html { >> return 302 http://b.com$request_uri; >> } >> } Then use Nginx as a load balancer and do consistent hashing. You can do it in Lua or Perl. https://github.com/chaoslawful/lua-nginx-module https://github.com/zzzcpan/nginx-perl AFAIK you'll need to code a new module for doing it more specifically. --- appa From vshebordaev at mail.ru Tue Jun 12 15:00:59 2012 From: vshebordaev at mail.ru (Vladimir Shebordaev) Date: Tue, 12 Jun 2012 19:00:59 +0400 Subject: Is there any way to redirect request to another server conf? In-Reply-To: References: Message-ID: Hi! 2012/6/12 Thomas : >> On 12 Jun 2012 13h29 CEST, sunyxing at gmail.com wrote: >> >> > Hi All, >> > >> > I'm trying to use nginx as a proxy server which can judge server >> > name by url. >> > >> > e.g. >> > >> > server { >> > listen 80 default_server; >> > my_rewrite_directive; >> > } >> > >> > server { >> > server_name a.com; >> > root html/a; >> > } >> > >> > server { >> > server_name b.com; >> > root html/b; >> > } >> > >> > then ?GET /a.com/index.html ?will be redirected to server a.com; >> >> Doesn't this do what you need? >> >> server { >> ? ?listen 80 default_server; >> >> ? ?location /a.com/index.html { >> ? ? ? ?return 302 http://a.com$request_uri; >> ? ?} >> >> ? ?location /b.com/index.html { >> ? ? ? ?return 302 http://b.com$request_uri; >> ? ?} >> } >> >> --- appa >> > > Thanks for advice. Since my servers are supposed to handle the request > ?directly and independently, ?302/301 are not acceptable in this > situation. > It seems, you need some kind of reverse proxy setup where nginx is running as a frontend server. If you do I would suggest to add something like this to your nginx frontend configuration location ~ ^/([ab]\.com)/(.*)$ { proxy_pass http://$1/$2; proxy_redirect http://$proxy_host:$proxy_port/ /$1/; } or more elaborated one based on your backend locations. You'll probably also need to add certain rules to handle cookies and other headers also depending on your particular backends. Please refer to proxy module documentation for details. In the hope it helps. Regards, Vladimir From goelvivek2011 at gmail.com Wed Jun 13 11:34:01 2012 From: goelvivek2011 at gmail.com (vivek goel) Date: Wed, 13 Jun 2012 17:04:01 +0530 Subject: Nginx module example with callback Message-ID: Hi, I am using a non-blocking library which reads from file-system and calls a callback when event has accrued. I want to integrate it with nginx module. >From where I can find an example where I can send response from third party library callback? What I am planning 1. Read complete request body of the client. 2. Register callback with nginx request object. 3. When callback is fired send response to the client. Is there a basic example I can non-blocking wait for this callback inside location handler ? regards Vivek Goel -------------- next part -------------- An HTML attachment was scrubbed... URL: From agentzh at gmail.com Wed Jun 13 15:06:31 2012 From: agentzh at gmail.com (agentzh) Date: Wed, 13 Jun 2012 23:06:31 +0800 Subject: Nginx module example with callback In-Reply-To: References: Message-ID: Hello! On Wed, Jun 13, 2012 at 7:34 PM, vivek goel wrote: > I am using a non-blocking library which reads from file-system and calls a > callback when event has accrued. I want to integrate it with nginx module. > From where I can find an example where I can send response from third party > library callback? > The key here is to let nginx monitor the events for your library here because there has to be one single event loop in the nginx worker process. So your library cannot take the control and run its own event loop because it will certainly block the nginx worker. You can take a look at the ngx_drizzle and ngx_postgres modules: http://wiki.nginx.org/HttpDrizzleModule https://github.com/FRiCKLE/ngx_postgres Both of them integrate nonblocking libraries (libdrizzle and libpq) into the nginx event model. And for ngx_drizzle, it bypasses the "poll" calls in libdrizzle to prevent conflicts. Best regards, -agentzh From me at prudhvi.net Wed Jun 13 21:33:18 2012 From: me at prudhvi.net (Prudhvi Krishna Surapaneni) Date: Wed, 13 Jun 2012 14:33:18 -0700 Subject: Parsing Variables to access them in a custom module In-Reply-To: <20120607103654.GA31671@mdounin.ru> References: <20120607014257.GA5478@prudhvi.net> <20120607103654.GA31671@mdounin.ru> Message-ID: <20120613213318.GC5478@prudhvi.net> On Thu, Jun 07, 2012 at 02:36:54PM +0400, Maxim Dounin wrote: > Hello! > > On Thu, Jun 07, 2012 at 09:02:44AM +0400, Vladimir Shebordaev wrote: > > > 2012/6/7 Prudhvi Krishna Surapaneni : > > > Hi All, > > > ? ? ? ?I am trying to write a module that needs to consume the regex matched > > > ? ? ? ?variables like $1 and $2 and pass them as values to the module. > > > > > > ? ? ? ?Here is what i am trying to do > > > > > > ? ? ? ?/* -- CONFIG -- */ > > > > > > ? ? ? ? ? ? ? ? ? ? ? ?location ~ /foo/([a-zA-Z0-9]+)/bar/([0-9]+) { > > > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?my_module; > > > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?my_module_first $1; > > > ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?my_module_second $2; > > > ? ? ? ? ? ? ? ? ? ? ? ?} > > > > > > ? ? ? ?/* -- CONFIG -- */ > > > > > > ? ? ? ?Inside my module i have defined them as follows. > > > > > > ? ? ? ?/* -- CODE -- */ > > > > > > ? ? ? ? ? ? ? ?typedef stuct { > > > ? ? ? ? ? ? ? ? ? ? ? ?ngx_str_t my_module_first; > > > ? ? ? ? ? ? ? ? ? ? ? ?ngx_str_t my_module_second; > > > ? ? ? ? ? ? ? ?} ngx_http_my_module_loc_conf_t > > > > > > ? ?/* -- CODE -- */ > > > > > > ? ? ? ?When trying to access the values from those variables i get the > > > ? ? ? ?value $1 and $2 instead of the substitution. What am i missing?. > > > > > > ? ? ? ?Are these variables only availble for the rewrite module?. > > > > Not particularly. But you should use the script engine to get captures > > evaluated at run-time. See, e.g. proxy module for an example. > > I would rather recommend using complex values here, as it's > simpliest way to use variables. > > Basically all it needs is using ngx_http_set_complex_value_slot > instead of ngx_conf_set_str_slot in configuration directive > definition, and an additional ngx_http_complex_value() call before > use. > > Complete example can be found in secure link module. > > Maxim Dounin > Hello Maxim, This is exactly what i wanted. Thank you very much. -Prudhvi Krishna Surapaneni > _______________________________________________ > nginx-devel mailing list > nginx-devel at nginx.org > http://mailman.nginx.org/mailman/listinfo/nginx-devel -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 490 bytes Desc: Digital signature URL: From mdounin at mdounin.ru Thu Jun 14 14:28:28 2012 From: mdounin at mdounin.ru (Maxim Dounin) Date: Thu, 14 Jun 2012 18:28:28 +0400 Subject: [PATCH] Fixing segfaults in ngx_poll_del_event at worker exit In-Reply-To: References: Message-ID: <20120614142828.GH31671@mdounin.ru> Hello! On Thu, Jun 07, 2012 at 05:36:36PM +0800, agentzh wrote: > Hello! > > I observed consistent segmentation faults when the Nginx worker > process exits due to HUP signal reload. The Nginx server has both > resolver and ngx_poll_module configured. The backtrace looks like > this: > > Process terminating with default action of signal 11 (SIGSEGV) > Access not within mapped region at address 0x50 > at 0x437AF8: ngx_poll_del_event (ngx_poll_module.c:208) > by 0x4220C0: ngx_close_connection (ngx_connection.c:872) > by 0x4267D1: ngx_resolver_cleanup (ngx_resolver.c:223) > by 0x4176D0: ngx_destroy_pool (ngx_palloc.c:55) > by 0x43317C: ngx_worker_process_exit (ngx_process_cycle.c:1059) > by 0x43327D: ngx_worker_process_cycle (ngx_process_cycle.c:800) > by 0x431AAE: ngx_spawn_process (ngx_process.c:198) > by 0x4328C9: ngx_start_worker_processes (ngx_process_cycle.c:365) > by 0x4342ED: ngx_master_process_cycle (ngx_process_cycle.c:250) > by 0x416653: main (nginx.c:410) > > At this point of failure, ngx_cycle->files was a NULL pointer (due to > the assignment of ngx_exit_cycle to ngx_cycle in > ngx_worker_process_exit) but ngx_poll_del_event tragically accessed > this array by an index, which was an invalid read. > > Below attaches a patch for nginx 1.0.15 that fixes this segfault. It > can also be cleanly applied to at least nginx 1.2.1. > > Comments will be highly appreciated as usual :) > > Thanks! > -agentzh > > --- nginx-1.0.15/src/event/modules/ngx_poll_module.c 2012-02-06 > 04:02:59.000000000 +0800 > +++ nginx-1.0.15-patched/src/event/modules/ngx_poll_module.c 2012-06-07 > 17:22:43.538168219 +0800 > @@ -205,19 +205,21 @@ > > event_list[ev->index] = event_list[nevents]; > > - c = ngx_cycle->files[event_list[nevents].fd]; > + if (ngx_cycle->files) { > + c = ngx_cycle->files[event_list[nevents].fd]; [...] While the patch obviously fixes the segfault, I don't really like it as it leaves incorrect event index set for a copied event. It's unlikely this will cause any problems on exit, but it's still incorrect. Something like this should be better way to solve the problem: diff --git a/src/os/unix/ngx_process_cycle.c b/src/os/unix/ngx_process_cycle.c --- a/src/os/unix/ngx_process_cycle.c +++ b/src/os/unix/ngx_process_cycle.c @@ -711,6 +711,8 @@ ngx_master_process_exit(ngx_cycle_t *cyc ngx_exit_log.file = &ngx_exit_log_file; ngx_exit_cycle.log = &ngx_exit_log; + ngx_exit_cycle.files = ngx_cycle->files; + ngx_exit_cycle.files_n = ngx_cycle->files_n; ngx_cycle = &ngx_exit_cycle; ngx_destroy_pool(cycle->pool); @@ -1054,6 +1056,8 @@ ngx_worker_process_exit(ngx_cycle_t *cyc ngx_exit_log.file = &ngx_exit_log_file; ngx_exit_cycle.log = &ngx_exit_log; + ngx_exit_cycle.files = ngx_cycle->files; + ngx_exit_cycle.files_n = ngx_cycle->files_n; ngx_cycle = &ngx_exit_cycle; ngx_destroy_pool(cycle->pool); Maxim Dounin From ne at vbart.ru Fri Jun 15 12:40:57 2012 From: ne at vbart.ru (Valentin V. Bartenev) Date: Fri, 15 Jun 2012 16:40:57 +0400 Subject: Announcing SPDY draft 2 implementation in nginx Message-ID: <201206151640.57505.ne@vbart.ru> We are pleased to announce the first beta version of SPDY draft 2 module for nginx. It is currently distributed as a patch for nginx 1.3.x development version. For more information about SPDY protocol specification please check http://www.chromium.org/spdy/spdy-protocol Known problems and limitations of this revision: - server push is not supported; - post_action directive is not supported for SPDY connections; - rate limiting is not supported for SPDY connections; - SSL buffer is switched off. We will be working on improving SPDY support during the next few months with the goal of eventually integrating it fully into the main nginx code. Expect frequent updates about this development, and please report your experiences, and send the feedback to nginx development mailing list nginx-devel at nginx.org (check http://mailman.nginx.org/mailman/listinfo/nginx-devel). Configuration and installation instructions are below. How to build nginx/spdy binary: 1. Install OpenSSL 1.0.1, it's required because SPDY module uses Next Protocol Negotiation TLS extension. 2. Download nginx 1.3.x tar-gzip package (check http://nginx.org/en/download.html) $ wget http://nginx.org/download/nginx-1.3.1.tar.gz 3. Unpack nginx-1.3 $ tar xvfz nginx-1.3.1.tar.gz $ cd nginx-1.3.1 4. Download and apply SPDY module patch $ wget http://nginx.org/patches/spdy/patch.spdy-34.txt $ patch -p0 < patch.spdy-34.txt 5. Configure nginx build $ ./configure --with-http_ssl_module Use --with-openssl=/path/to/openssl-1.0.1, when building OpenSSL separately and statically linking. Use --with-cc-opt and --with-ld-opt accordingly, if OpenSSL is installed as an optional library, e.g. on Mac OS X $ ./configure --with-http_ssl_module \ --with-cc-opt="-I/opt/local/include" \ --with-ld-opt="-L/opt/local/lib" 6. Build nginx $ make To enable SPDY in nginx configuration, just add "spdy" and "ssl" parameters to the listen directive: server { listen 443 ssl spdy default_server; ssl_certificate server.crt; ssl_certificate_key server.key; ... } Optional SPDY configuration directives: - spdy_recv_buffer_size - specifies the size of input buffer (per worker), 1MB by default; - spdy_max_concurrent_streams - maximum number of concurrent SPDY streams in a single connection, 100 by default; - spdy_streams_index_size - size of SPDY stream ID index, should be power of 2, default is 32; - spdy_recv_timeout - timeout when expecting more data from the client, default is 30s; - spdy_keepalive_timeout - inactivity timeout after which connection is closed, default is 3m; - spdy_headers_comp - header compression level (0 = no comp, 9 = max comp), default is 1; - spdy_headers_comp_window - size of LZ77 compression window, default is 4KB. The default values are more or less optimized for generic use, there's normally no need to tweak them. SPDY variables: - $spdy - version of SPDY protocol if the request came via SPDY (currently "2"), or empty value; - $spdy_request_priority - priority of the stream, if the request came via SPDY. Disclaimer: use this code at your own risk, it is distributed under the 2-clause BSD-like license, and at nginx we are not responsible for any negative impact or effects that the usage of this code might cause. -- NGINX, Inc., http://nginx.com From agentzh at gmail.com Sat Jun 16 03:28:34 2012 From: agentzh at gmail.com (agentzh) Date: Sat, 16 Jun 2012 11:28:34 +0800 Subject: [PATCH] Fixing segfaults in ngx_poll_del_event at worker exit In-Reply-To: <20120614142828.GH31671@mdounin.ru> References: <20120614142828.GH31671@mdounin.ru> Message-ID: Hello! On Thu, Jun 14, 2012 at 10:28 PM, Maxim Dounin wrote: > While the patch obviously fixes the segfault, I don't really like > it as it leaves incorrect event index set for a copied event. > It's unlikely this will cause any problems on exit, but it's still > incorrect. > Yeah, it makes sense :) > Something like this should be better way to solve the problem: > > diff --git a/src/os/unix/ngx_process_cycle.c b/src/os/unix/ngx_process_cycle.c > --- a/src/os/unix/ngx_process_cycle.c > +++ b/src/os/unix/ngx_process_cycle.c > @@ -711,6 +711,8 @@ ngx_master_process_exit(ngx_cycle_t *cyc > ? ? ngx_exit_log.file = &ngx_exit_log_file; > > ? ? ngx_exit_cycle.log = &ngx_exit_log; > + ? ?ngx_exit_cycle.files = ngx_cycle->files; > + ? ?ngx_exit_cycle.files_n = ngx_cycle->files_n; > ? ? ngx_cycle = &ngx_exit_cycle; > > ? ? ngx_destroy_pool(cycle->pool); > @@ -1054,6 +1056,8 @@ ngx_worker_process_exit(ngx_cycle_t *cyc > ? ? ngx_exit_log.file = &ngx_exit_log_file; > > ? ? ngx_exit_cycle.log = &ngx_exit_log; > + ? ?ngx_exit_cycle.files = ngx_cycle->files; > + ? ?ngx_exit_cycle.files_n = ngx_cycle->files_n; > ? ? ngx_cycle = &ngx_exit_cycle; > > ? ? ngx_destroy_pool(cycle->pool); > This patch works for me :) Thanks! -agentzh From ru at nginx.com Mon Jun 18 11:07:45 2012 From: ru at nginx.com (ru at nginx.com) Date: Mon, 18 Jun 2012 11:07:45 +0000 Subject: [nginx] svn commit: r4683 - in trunk/src: core http/modules/perl Message-ID: <20120618110745.9FF753FA0B8@mail.nginx.com> Author: ru Date: 2012-06-18 11:07:44 +0000 (Mon, 18 Jun 2012) New Revision: 4683 URL: http://trac.nginx.org/nginx/changeset/4683/nginx Log: Version bump. Modified: trunk/src/core/nginx.h trunk/src/http/modules/perl/nginx.pm Modified: trunk/src/core/nginx.h =================================================================== --- trunk/src/core/nginx.h 2012-06-08 09:41:55 UTC (rev 4682) +++ trunk/src/core/nginx.h 2012-06-18 11:07:44 UTC (rev 4683) @@ -9,8 +9,8 @@ #define _NGINX_H_INCLUDED_ -#define nginx_version 1003001 -#define NGINX_VERSION "1.3.1" +#define nginx_version 1003002 +#define NGINX_VERSION "1.3.2" #define NGINX_VER "nginx/" NGINX_VERSION #define NGINX_VAR "NGINX" Modified: trunk/src/http/modules/perl/nginx.pm =================================================================== --- trunk/src/http/modules/perl/nginx.pm 2012-06-08 09:41:55 UTC (rev 4682) +++ trunk/src/http/modules/perl/nginx.pm 2012-06-18 11:07:44 UTC (rev 4683) @@ -50,7 +50,7 @@ HTTP_INSUFFICIENT_STORAGE ); -our $VERSION = '1.3.1'; +our $VERSION = '1.3.2'; require XSLoader; XSLoader::load('nginx', $VERSION); From ru at nginx.com Mon Jun 18 12:30:48 2012 From: ru at nginx.com (ru at nginx.com) Date: Mon, 18 Jun 2012 12:30:48 +0000 Subject: [nginx] svn commit: r4684 - trunk/src/core Message-ID: <20120618123048.3E7863F9F14@mail.nginx.com> Author: ru Date: 2012-06-18 12:30:45 +0000 (Mon, 18 Jun 2012) New Revision: 4684 URL: http://trac.nginx.org/nginx/changeset/4684/nginx Log: Fixed crash in ngx_resolver_cleanup_tree(). If sending a DNS request fails with an error (e.g., when mistakenly trying to send it to a local IP broadcast), such a request is not deleted if there are clients waiting on it. However, it was still erroneously removed from the queue. Later ngx_resolver_cleanup_tree() attempted to remove it from the queue again that resulted in a NULL pointer dereference. Modified: trunk/src/core/ngx_resolver.c Modified: trunk/src/core/ngx_resolver.c =================================================================== --- trunk/src/core/ngx_resolver.c 2012-06-18 11:07:44 UTC (rev 4683) +++ trunk/src/core/ngx_resolver.c 2012-06-18 12:30:45 UTC (rev 4684) @@ -977,12 +977,11 @@ if (rn->waiting) { - if (ngx_resolver_send_query(r, rn) == NGX_OK) { + (void) ngx_resolver_send_query(r, rn); - rn->expire = now + r->resend_timeout; + rn->expire = now + r->resend_timeout; - ngx_queue_insert_head(queue, &rn->queue); - } + ngx_queue_insert_head(queue, q); continue; } From ru at nginx.com Mon Jun 18 12:46:06 2012 From: ru at nginx.com (ru at nginx.com) Date: Mon, 18 Jun 2012 12:46:06 +0000 Subject: [nginx] svn commit: r4685 - trunk/src/core Message-ID: <20120618124606.4AA893FA69D@mail.nginx.com> Author: ru Date: 2012-06-18 12:46:05 +0000 (Mon, 18 Jun 2012) New Revision: 4685 URL: http://trac.nginx.org/nginx/changeset/4685/nginx Log: When "resolver" is configured with a domain name, only the first resolved address was used. Now all addresses will be used. Modified: trunk/src/core/ngx_resolver.c Modified: trunk/src/core/ngx_resolver.c =================================================================== --- trunk/src/core/ngx_resolver.c 2012-06-18 12:30:45 UTC (rev 4684) +++ trunk/src/core/ngx_resolver.c 2012-06-18 12:46:05 UTC (rev 4685) @@ -96,7 +96,7 @@ { ngx_str_t s; ngx_url_t u; - ngx_uint_t i; + ngx_uint_t i, j; ngx_resolver_t *r; ngx_pool_cleanup_t *cln; ngx_udp_connection_t *uc; @@ -184,16 +184,18 @@ return NULL; } - uc = ngx_array_push(&r->udp_connections); + uc = ngx_array_push_n(&r->udp_connections, u.naddrs); if (uc == NULL) { return NULL; } - ngx_memzero(uc, sizeof(ngx_udp_connection_t)); + ngx_memzero(uc, u.naddrs * sizeof(ngx_udp_connection_t)); - uc->sockaddr = u.addrs->sockaddr; - uc->socklen = u.addrs->socklen; - uc->server = u.addrs->name; + for (j = 0; j < u.naddrs; j++) { + uc[j].sockaddr = u.addrs[j].sockaddr; + uc[j].socklen = u.addrs[j].socklen; + uc[j].server = u.addrs[j].name; + } } return r; From defan at nginx.com Mon Jun 18 13:43:45 2012 From: defan at nginx.com (defan at nginx.com) Date: Mon, 18 Jun 2012 13:43:45 +0000 Subject: [nginx] svn commit: r4686 - in trunk/src/http: . modules Message-ID: <20120618134345.519013F9F35@mail.nginx.com> Author: defan Date: 2012-06-18 13:43:44 +0000 (Mon, 18 Jun 2012) New Revision: 4686 URL: http://trac.nginx.org/nginx/changeset/4686/nginx Log: New core variable: $status. Contains response status code as a 3-digit integer (with leading zeroes if necessary), or one of the following values: 000 - response status code has not yet been assigned 009 - HTTP/0.9 request is being processed Modified: trunk/src/http/modules/ngx_http_log_module.c trunk/src/http/ngx_http_variables.c Modified: trunk/src/http/modules/ngx_http_log_module.c =================================================================== --- trunk/src/http/modules/ngx_http_log_module.c 2012-06-18 12:46:05 UTC (rev 4685) +++ trunk/src/http/modules/ngx_http_log_module.c 2012-06-18 13:43:44 UTC (rev 4686) @@ -584,10 +584,7 @@ status = r->headers_out.status; } else if (r->http_version == NGX_HTTP_VERSION_9) { - *buf++ = '0'; - *buf++ = '0'; - *buf++ = '9'; - return buf; + status = 9; } else { status = 0; Modified: trunk/src/http/ngx_http_variables.c =================================================================== --- trunk/src/http/ngx_http_variables.c 2012-06-18 12:46:05 UTC (rev 4685) +++ trunk/src/http/ngx_http_variables.c 2012-06-18 13:43:44 UTC (rev 4686) @@ -78,6 +78,8 @@ static ngx_int_t ngx_http_variable_request_body_file(ngx_http_request_t *r, ngx_http_variable_value_t *v, uintptr_t data); +static ngx_int_t ngx_http_variable_status(ngx_http_request_t *r, + ngx_http_variable_value_t *v, uintptr_t data); static ngx_int_t ngx_http_variable_sent_content_type(ngx_http_request_t *r, ngx_http_variable_value_t *v, uintptr_t data); static ngx_int_t ngx_http_variable_sent_content_length(ngx_http_request_t *r, @@ -225,6 +227,10 @@ ngx_http_variable_request_body_file, 0, 0, 0 }, + { ngx_string("status"), NULL, + ngx_http_variable_status, 0, + NGX_HTTP_VAR_NOCACHEABLE, 0 }, + { ngx_string("sent_http_content_type"), NULL, ngx_http_variable_sent_content_type, 0, 0, 0 }, @@ -1456,6 +1462,39 @@ static ngx_int_t +ngx_http_variable_status(ngx_http_request_t *r, + ngx_http_variable_value_t *v, uintptr_t data) +{ + ngx_uint_t status; + + v->data = ngx_pnalloc(r->pool, NGX_INT_T_LEN); + if (v->data == NULL) { + return NGX_ERROR; + } + + if (r->err_status) { + status = r->err_status; + + } else if (r->headers_out.status) { + status = r->headers_out.status; + + } else if (r->http_version == NGX_HTTP_VERSION_9) { + status = 9; + + } else { + status = 0; + } + + v->len = ngx_sprintf(v->data, "%03ui", status) - v->data; + v->valid = 1; + v->no_cacheable = 0; + v->not_found = 0; + + return NGX_OK; +} + + +static ngx_int_t ngx_http_variable_sent_content_type(ngx_http_request_t *r, ngx_http_variable_value_t *v, uintptr_t data) { From defan at nginx.com Mon Jun 18 13:51:21 2012 From: defan at nginx.com (defan at nginx.com) Date: Mon, 18 Jun 2012 13:51:21 +0000 Subject: [nginx] svn commit: r4687 - trunk/src/http Message-ID: <20120618135121.329ED3F9C1B@mail.nginx.com> Author: defan Date: 2012-06-18 13:51:20 +0000 (Mon, 18 Jun 2012) New Revision: 4687 URL: http://trac.nginx.org/nginx/changeset/4687/nginx Log: Style fix. Modified: trunk/src/http/ngx_http_variables.c Modified: trunk/src/http/ngx_http_variables.c =================================================================== --- trunk/src/http/ngx_http_variables.c 2012-06-18 13:43:44 UTC (rev 4686) +++ trunk/src/http/ngx_http_variables.c 2012-06-18 13:51:20 UTC (rev 4687) @@ -77,9 +77,9 @@ ngx_http_variable_value_t *v, uintptr_t data); static ngx_int_t ngx_http_variable_request_body_file(ngx_http_request_t *r, ngx_http_variable_value_t *v, uintptr_t data); - static ngx_int_t ngx_http_variable_status(ngx_http_request_t *r, ngx_http_variable_value_t *v, uintptr_t data); + static ngx_int_t ngx_http_variable_sent_content_type(ngx_http_request_t *r, ngx_http_variable_value_t *v, uintptr_t data); static ngx_int_t ngx_http_variable_sent_content_length(ngx_http_request_t *r, From mdounin at mdounin.ru Mon Jun 18 14:01:18 2012 From: mdounin at mdounin.ru (mdounin at mdounin.ru) Date: Mon, 18 Jun 2012 14:01:18 +0000 Subject: [nginx] svn commit: r4688 - trunk/src/http/modules Message-ID: <20120618140119.08A7D3F9C1E@mail.nginx.com> Author: mdounin Date: 2012-06-18 14:01:18 +0000 (Mon, 18 Jun 2012) New Revision: 4688 URL: http://trac.nginx.org/nginx/changeset/4688/nginx Log: Mp4: fixed non-keyframe seeks in some cases (ticket #175). Number of entries in stsc atom was wrong if we've added an entry to split a chunk. Additionally, there is no need to add an entry if we are going to split last chunk in an entry, it's enough to update the entry we already have. Previously new entry was added and old one was left as is, resulting in incorrect entry with zero chunks which might confuse some software. Modified: trunk/src/http/modules/ngx_http_mp4_module.c Modified: trunk/src/http/modules/ngx_http_mp4_module.c =================================================================== --- trunk/src/http/modules/ngx_http_mp4_module.c 2012-06-18 13:51:20 UTC (rev 4687) +++ trunk/src/http/modules/ngx_http_mp4_module.c 2012-06-18 14:01:18 UTC (rev 4688) @@ -2488,8 +2488,14 @@ ngx_mp4_set_32value(entry->chunk, 1); - if (trak->chunk_samples) { + if (trak->chunk_samples && next_chunk - trak->start_chunk == 2) { + /* last chunk in the entry */ + + ngx_mp4_set_32value(entry->samples, samples - trak->chunk_samples); + + } else if (trak->chunk_samples) { + first = &trak->stsc_chunk_entry; ngx_mp4_set_32value(first->chunk, 1); ngx_mp4_set_32value(first->samples, samples - trak->chunk_samples); @@ -2504,6 +2510,7 @@ ngx_mp4_set_32value(entry->chunk, 2); + entries++; atom_size += sizeof(ngx_mp4_stsc_entry_t); } From mdounin at mdounin.ru Mon Jun 18 14:02:20 2012 From: mdounin at mdounin.ru (mdounin at mdounin.ru) Date: Mon, 18 Jun 2012 14:02:20 +0000 Subject: [nginx] svn commit: r4689 - trunk/src/http/modules Message-ID: <20120618140220.F170A3F9E7A@mail.nginx.com> Author: mdounin Date: 2012-06-18 14:02:20 +0000 (Mon, 18 Jun 2012) New Revision: 4689 URL: http://trac.nginx.org/nginx/changeset/4689/nginx Log: Mp4: fixed streaming if moov atom is at buffer edge. Modified: trunk/src/http/modules/ngx_http_mp4_module.c Modified: trunk/src/http/modules/ngx_http_mp4_module.c =================================================================== --- trunk/src/http/modules/ngx_http_mp4_module.c 2012-06-18 14:01:18 UTC (rev 4688) +++ trunk/src/http/modules/ngx_http_mp4_module.c 2012-06-18 14:02:20 UTC (rev 4689) @@ -1024,6 +1024,10 @@ + NGX_HTTP_MP4_MOOV_BUFFER_EXCESS * no_mdat; } + if (ngx_http_mp4_read(mp4, atom_data_size) != NGX_OK) { + return NGX_ERROR; + } + mp4->trak.elts = &mp4->traks; mp4->trak.size = sizeof(ngx_http_mp4_trak_t); mp4->trak.nalloc = 2; @@ -1044,6 +1048,12 @@ mp4->buffer_start = mp4->buffer_pos; mp4->buffer_size = NGX_HTTP_MP4_MOOV_BUFFER_EXCESS; + if (mp4->buffer_start + mp4->buffer_size > mp4->buffer_end) { + mp4->buffer = NULL; + mp4->buffer_pos = NULL; + mp4->buffer_end = NULL; + } + } else { /* skip atoms after moov atom */ mp4->offset = mp4->end; From mdounin at mdounin.ru Mon Jun 18 14:06:01 2012 From: mdounin at mdounin.ru (mdounin at mdounin.ru) Date: Mon, 18 Jun 2012 14:06:01 +0000 Subject: [nginx] svn commit: r4690 - trunk/src/core Message-ID: <20120618140602.03D283FA8FA@mail.nginx.com> Author: mdounin Date: 2012-06-18 14:06:00 +0000 (Mon, 18 Jun 2012) New Revision: 4690 URL: http://trac.nginx.org/nginx/changeset/4690/nginx Log: Fixed handling of conflicting wildcard server names. With previous code wildcard names were added to hash even if conflict was detected. This resulted in identical names in hash and segfault later in ngx_hash_wildcard_init(). Modified: trunk/src/core/ngx_hash.c Modified: trunk/src/core/ngx_hash.c =================================================================== --- trunk/src/core/ngx_hash.c 2012-06-18 14:02:20 UTC (rev 4689) +++ trunk/src/core/ngx_hash.c 2012-06-18 14:06:00 UTC (rev 4690) @@ -924,17 +924,6 @@ } - hk = ngx_array_push(hwc); - if (hk == NULL) { - return NGX_ERROR; - } - - hk->key.len = last - 1; - hk->key.data = p; - hk->key_hash = 0; - hk->value = value; - - /* check conflicts in wildcard hash */ name = keys->elts; @@ -972,5 +961,18 @@ ngx_memcpy(name->data, key->data + skip, name->len); + + /* add to wildcard hash */ + + hk = ngx_array_push(hwc); + if (hk == NULL) { + return NGX_ERROR; + } + + hk->key.len = last - 1; + hk->key.data = p; + hk->key_hash = 0; + hk->value = value; + return NGX_OK; } From mdounin at mdounin.ru Mon Jun 18 14:09:55 2012 From: mdounin at mdounin.ru (mdounin at mdounin.ru) Date: Mon, 18 Jun 2012 14:09:55 +0000 Subject: [nginx] svn commit: r4691 - trunk/auto/os Message-ID: <20120618140955.4CDB53F9C5F@mail.nginx.com> Author: mdounin Date: 2012-06-18 14:09:54 +0000 (Mon, 18 Jun 2012) New Revision: 4691 URL: http://trac.nginx.org/nginx/changeset/4691/nginx Log: Changed default alignment to 16. This fixes alignment problems observerd on ARMs, and likely also needed for MIPSes. Unless we know alignment is not required just assume we need 16, which appears to be safe default for all architectures. See here for details: http://mailman.nginx.org/pipermail/nginx/2012-June/034139.html Modified: trunk/auto/os/conf Modified: trunk/auto/os/conf =================================================================== --- trunk/auto/os/conf 2012-06-18 14:06:00 UTC (rev 4690) +++ trunk/auto/os/conf 2012-06-18 14:09:54 UTC (rev 4691) @@ -93,6 +93,7 @@ ;; *) + have=NGX_ALIGNMENT value=16 . auto/define NGX_MACH_CACHE_LINE=32 ;; From mdounin at mdounin.ru Mon Jun 18 14:10:50 2012 From: mdounin at mdounin.ru (mdounin at mdounin.ru) Date: Mon, 18 Jun 2012 14:10:50 +0000 Subject: [nginx] svn commit: r4692 - trunk/src/os/unix Message-ID: <20120618141050.D33A13F9C5F@mail.nginx.com> Author: mdounin Date: 2012-06-18 14:10:50 +0000 (Mon, 18 Jun 2012) New Revision: 4692 URL: http://trac.nginx.org/nginx/changeset/4692/nginx Log: Fixed segfault with poll and resolver used. Poll event method needs ngx_cycle->files to work, and use of ngx_exit_cycle without files set caused null pointer dereference in resolver's cleanup on udp socket close. Modified: trunk/src/os/unix/ngx_process_cycle.c Modified: trunk/src/os/unix/ngx_process_cycle.c =================================================================== --- trunk/src/os/unix/ngx_process_cycle.c 2012-06-18 14:09:54 UTC (rev 4691) +++ trunk/src/os/unix/ngx_process_cycle.c 2012-06-18 14:10:50 UTC (rev 4692) @@ -711,6 +711,8 @@ ngx_exit_log.file = &ngx_exit_log_file; ngx_exit_cycle.log = &ngx_exit_log; + ngx_exit_cycle.files = ngx_cycle->files; + ngx_exit_cycle.files_n = ngx_cycle->files_n; ngx_cycle = &ngx_exit_cycle; ngx_destroy_pool(cycle->pool); @@ -1054,6 +1056,8 @@ ngx_exit_log.file = &ngx_exit_log_file; ngx_exit_cycle.log = &ngx_exit_log; + ngx_exit_cycle.files = ngx_cycle->files; + ngx_exit_cycle.files_n = ngx_cycle->files_n; ngx_cycle = &ngx_exit_cycle; ngx_destroy_pool(cycle->pool); From mdounin at mdounin.ru Mon Jun 18 14:11:29 2012 From: mdounin at mdounin.ru (mdounin at mdounin.ru) Date: Mon, 18 Jun 2012 14:11:29 +0000 Subject: [nginx] svn commit: r4693 - in trunk: auto/os src/os/unix Message-ID: <20120618141129.6C53D3FA0D1@mail.nginx.com> Author: mdounin Date: 2012-06-18 14:11:29 +0000 (Mon, 18 Jun 2012) New Revision: 4693 URL: http://trac.nginx.org/nginx/changeset/4693/nginx Log: Fixed "sendmsg() failed" alerts on HP-UX. HP-UX needs _HPUX_ALT_XOPEN_SOCKET_API to be defined to be able to use various POSIX versions of networking functions. Notably sendmsg() resulted in "sendmsg() failed (9: Bad file number)" alerts without it. See xopen_networking(7) for more details. Modified: trunk/auto/os/conf trunk/src/os/unix/ngx_posix_config.h Modified: trunk/auto/os/conf =================================================================== --- trunk/auto/os/conf 2012-06-18 14:10:50 UTC (rev 4692) +++ trunk/auto/os/conf 2012-06-18 14:11:29 UTC (rev 4693) @@ -48,6 +48,7 @@ CORE_DEPS="$UNIX_DEPS $POSIX_DEPS" CORE_SRCS="$UNIX_SRCS" CC_AUX_FLAGS="$CC_AUX_FLAGS -D_XOPEN_SOURCE -D_XOPEN_SOURCE_EXTENDED=1" + CC_AUX_FLAGS="$CC_AUX_FLAGS -D_HPUX_ALT_XOPEN_SOCKET_API" ;; OSF1:*) Modified: trunk/src/os/unix/ngx_posix_config.h =================================================================== --- trunk/src/os/unix/ngx_posix_config.h 2012-06-18 14:10:50 UTC (rev 4692) +++ trunk/src/os/unix/ngx_posix_config.h 2012-06-18 14:11:29 UTC (rev 4693) @@ -12,6 +12,7 @@ #if (NGX_HPUX) #define _XOPEN_SOURCE #define _XOPEN_SOURCE_EXTENDED 1 +#define _HPUX_ALT_XOPEN_SOCKET_API #endif From mdounin at mdounin.ru Mon Jun 18 14:12:03 2012 From: mdounin at mdounin.ru (mdounin at mdounin.ru) Date: Mon, 18 Jun 2012 14:12:03 +0000 Subject: [nginx] svn commit: r4694 - in trunk/src/os: unix win32 Message-ID: <20120618141203.7F9EF3FA6B2@mail.nginx.com> Author: mdounin Date: 2012-06-18 14:12:03 +0000 (Mon, 18 Jun 2012) New Revision: 4694 URL: http://trac.nginx.org/nginx/changeset/4694/nginx Log: Fixed return type of ngx_strerror_init(). Modified: trunk/src/os/unix/ngx_errno.c trunk/src/os/unix/ngx_errno.h trunk/src/os/win32/ngx_errno.c trunk/src/os/win32/ngx_errno.h Modified: trunk/src/os/unix/ngx_errno.c =================================================================== --- trunk/src/os/unix/ngx_errno.c 2012-06-18 14:11:29 UTC (rev 4693) +++ trunk/src/os/unix/ngx_errno.c 2012-06-18 14:12:03 UTC (rev 4694) @@ -42,7 +42,7 @@ } -ngx_uint_t +ngx_int_t ngx_strerror_init(void) { char *msg; Modified: trunk/src/os/unix/ngx_errno.h =================================================================== --- trunk/src/os/unix/ngx_errno.h 2012-06-18 14:11:29 UTC (rev 4693) +++ trunk/src/os/unix/ngx_errno.h 2012-06-18 14:12:03 UTC (rev 4694) @@ -69,7 +69,7 @@ u_char *ngx_strerror(ngx_err_t err, u_char *errstr, size_t size); -ngx_uint_t ngx_strerror_init(void); +ngx_int_t ngx_strerror_init(void); #endif /* _NGX_ERRNO_H_INCLUDED_ */ Modified: trunk/src/os/win32/ngx_errno.c =================================================================== --- trunk/src/os/win32/ngx_errno.c 2012-06-18 14:11:29 UTC (rev 4693) +++ trunk/src/os/win32/ngx_errno.c 2012-06-18 14:12:03 UTC (rev 4694) @@ -53,7 +53,7 @@ } -ngx_uint_t +ngx_int_t ngx_strerror_init(void) { return NGX_OK; Modified: trunk/src/os/win32/ngx_errno.h =================================================================== --- trunk/src/os/win32/ngx_errno.h 2012-06-18 14:11:29 UTC (rev 4693) +++ trunk/src/os/win32/ngx_errno.h 2012-06-18 14:12:03 UTC (rev 4694) @@ -59,7 +59,7 @@ u_char *ngx_strerror(ngx_err_t err, u_char *errstr, size_t size); -ngx_uint_t ngx_strerror_init(void); +ngx_int_t ngx_strerror_init(void); #endif /* _NGX_ERRNO_H_INCLUDED_ */ From mdounin at mdounin.ru Mon Jun 18 14:23:42 2012 From: mdounin at mdounin.ru (mdounin at mdounin.ru) Date: Mon, 18 Jun 2012 14:23:42 +0000 Subject: [nginx] svn commit: r4695 - trunk/src/http/modules Message-ID: <20120618142342.E379C3FA0CF@mail.nginx.com> Author: mdounin Date: 2012-06-18 14:23:42 +0000 (Mon, 18 Jun 2012) New Revision: 4695 URL: http://trac.nginx.org/nginx/changeset/4695/nginx Log: Upstream keepalive: "single" parameter deprecated. The original idea was to optimize edge cases in case of interchangeable backends, i.e. don't establish a new connection if we have any one cached. This causes more harm than good though, as it screws up underlying balancer's idea about backends used and may result in various unexpected problems. Modified: trunk/src/http/modules/ngx_http_upstream_keepalive_module.c Modified: trunk/src/http/modules/ngx_http_upstream_keepalive_module.c =================================================================== --- trunk/src/http/modules/ngx_http_upstream_keepalive_module.c 2012-06-18 14:12:03 UTC (rev 4694) +++ trunk/src/http/modules/ngx_http_upstream_keepalive_module.c 2012-06-18 14:23:42 UTC (rev 4695) @@ -12,7 +12,6 @@ typedef struct { ngx_uint_t max_cached; - ngx_uint_t single; /* unsigned:1 */ ngx_queue_t cache; ngx_queue_t free; @@ -223,36 +222,11 @@ kp->failed = 0; - /* single pool of cached connections */ + /* ask balancer */ - if (kp->conf->single && !ngx_queue_empty(&kp->conf->cache)) { - - q = ngx_queue_head(&kp->conf->cache); - - item = ngx_queue_data(q, ngx_http_upstream_keepalive_cache_t, queue); - c = item->connection; - - ngx_queue_remove(q); - ngx_queue_insert_head(&kp->conf->free, q); - - ngx_log_debug1(NGX_LOG_DEBUG_HTTP, pc->log, 0, - "get keepalive peer: using connection %p", c); - - c->idle = 0; - c->log = pc->log; - c->read->log = pc->log; - c->write->log = pc->log; - c->pool->log = pc->log; - - pc->connection = c; - pc->cached = 1; - - return NGX_DONE; - } - rc = kp->original_get_peer(pc, kp->data); - if (kp->conf->single || rc != NGX_OK) { + if (rc != NGX_OK) { return rc; } @@ -552,7 +526,8 @@ for (i = 2; i < cf->args->nelts; i++) { if (ngx_strcmp(value[i].data, "single") == 0) { - kcf->single = 1; + ngx_conf_log_error(NGX_LOG_WARN, cf, 0, + "the \"single\" parameter is deprecated"); continue; } From gmm at csdoc.com Mon Jun 18 15:24:53 2012 From: gmm at csdoc.com (Gena Makhomed) Date: Mon, 18 Jun 2012 18:24:53 +0300 Subject: [nginx] svn commit: r4695 - trunk/src/http/modules In-Reply-To: <20120618142342.E379C3FA0CF@mail.nginx.com> References: <20120618142342.E379C3FA0CF@mail.nginx.com> Message-ID: <4FDF4845.4070005@csdoc.com> On 18.06.2012 17:23, mdounin at mdounin.ru wrote: > Upstream keepalive: "single" parameter deprecated. "deprecated" means "it is not recommended to use and will be removed in future versions". parameter "single" now is not deprecated - it is already removed completely. http://en.wikipedia.org/wiki/Deprecation ...deprecation may indicate that the feature will be removed in the future... -- Best regards, Gena From mdounin at mdounin.ru Mon Jun 18 15:59:17 2012 From: mdounin at mdounin.ru (Maxim Dounin) Date: Mon, 18 Jun 2012 19:59:17 +0400 Subject: [nginx] svn commit: r4695 - trunk/src/http/modules In-Reply-To: <4FDF4845.4070005@csdoc.com> References: <20120618142342.E379C3FA0CF@mail.nginx.com> <4FDF4845.4070005@csdoc.com> Message-ID: <20120618155916.GM31671@mdounin.ru> Hello! On Mon, Jun 18, 2012 at 06:24:53PM +0300, Gena Makhomed wrote: > On 18.06.2012 17:23, mdounin at mdounin.ru wrote: > > >Upstream keepalive: "single" parameter deprecated. > > "deprecated" means "it is not recommended to use and will be removed > in future versions". Sure it will be removed. > parameter "single" now is not deprecated - it is already removed completely. The parameter itself is still accepted (though now produce a warning). In this particular case it is enough to provide backward compatibility. Maxim Dounin From ru at nginx.com Tue Jun 19 12:36:54 2012 From: ru at nginx.com (ru at nginx.com) Date: Tue, 19 Jun 2012 12:36:54 +0000 Subject: [nginx] svn commit: r4696 - trunk/src/http/modules Message-ID: <20120619123654.EB4CE3F9ECC@mail.nginx.com> Author: ru Date: 2012-06-19 12:36:54 +0000 (Tue, 19 Jun 2012) New Revision: 4696 URL: http://trac.nginx.org/nginx/changeset/4696/nginx Log: Added IPv6 support to ip_hash. Modified: trunk/src/http/modules/ngx_http_upstream_ip_hash_module.c Modified: trunk/src/http/modules/ngx_http_upstream_ip_hash_module.c =================================================================== --- trunk/src/http/modules/ngx_http_upstream_ip_hash_module.c 2012-06-18 14:23:42 UTC (rev 4695) +++ trunk/src/http/modules/ngx_http_upstream_ip_hash_module.c 2012-06-19 12:36:54 UTC (rev 4696) @@ -16,7 +16,8 @@ ngx_uint_t hash; - u_char addr[3]; + u_char addrlen; + u_char *addr; u_char tries; @@ -76,7 +77,10 @@ }; -ngx_int_t +static u_char ngx_http_upstream_ip_hash_pseudo_addr[3]; + + +static ngx_int_t ngx_http_upstream_init_ip_hash(ngx_conf_t *cf, ngx_http_upstream_srv_conf_t *us) { if (ngx_http_upstream_init_round_robin(cf, us) != NGX_OK) { @@ -93,8 +97,10 @@ ngx_http_upstream_init_ip_hash_peer(ngx_http_request_t *r, ngx_http_upstream_srv_conf_t *us) { - u_char *p; struct sockaddr_in *sin; +#if (NGX_HAVE_INET6) + struct sockaddr_in6 *sin6; +#endif ngx_http_upstream_ip_hash_peer_data_t *iphp; iphp = ngx_palloc(r->pool, sizeof(ngx_http_upstream_ip_hash_peer_data_t)); @@ -110,20 +116,25 @@ r->upstream->peer.get = ngx_http_upstream_get_ip_hash_peer; - /* AF_INET only */ + switch (r->connection->sockaddr->sa_family) { - if (r->connection->sockaddr->sa_family == AF_INET) { - + case AF_INET: sin = (struct sockaddr_in *) r->connection->sockaddr; - p = (u_char *) &sin->sin_addr.s_addr; - iphp->addr[0] = p[0]; - iphp->addr[1] = p[1]; - iphp->addr[2] = p[2]; + iphp->addr = (u_char *) &sin->sin_addr.s_addr; + iphp->addrlen = 3; + break; - } else { - iphp->addr[0] = 0; - iphp->addr[1] = 0; - iphp->addr[2] = 0; +#if (NGX_HAVE_INET6) + case AF_INET6: + sin6 = (struct sockaddr_in6 *) r->connection->sockaddr; + iphp->addr = (u_char *) &sin6->sin6_addr.s6_addr; + iphp->addrlen = 16; + break; +#endif + + default: + iphp->addr = ngx_http_upstream_ip_hash_pseudo_addr; + iphp->addrlen = 3; } iphp->hash = 89; @@ -163,7 +174,7 @@ for ( ;; ) { - for (i = 0; i < 3; i++) { + for (i = 0; i < iphp->addrlen; i++) { hash = (hash * 113 + iphp->addr[i]) % 6271; } From matthieu.tourne at gmail.com Wed Jun 20 01:12:12 2012 From: matthieu.tourne at gmail.com (Matthieu Tourne) Date: Tue, 19 Jun 2012 18:12:12 -0700 Subject: Spdy - Double Status header Message-ID: Hi, I think, I've found an interesting corner case with the recent Nginx SPDY support. I'm using the latest : http://nginx.org/patches/spdy/patch.spdy-37.txt I connect to Nginx using SPDY, and then proxy_pass to another server, with a location similar to this : location /redirect { add_header 'Status' '301 Moved Permanently'; rewrite (.*) / permanent; } This causes Nginx to return 2 Spdy ":status" headers, which I guess is disallowed by the protocol. This is the output from spdycat (https://github.com/tatsuhiro-t/spdylay/) Chrome reports a similar error. [ 0.141] [INVALID; status=PROTOCOL_ERROR] recv SYN_REPLY frame (...) status: 301 status: 301 Moved Permanently Note that this can be fixed using configuration to hide any potential "Status" header returned by the second server, before the Spdy response is generated. But the code should probably handle this ? Matthieu From atribble at amazon.com Wed Jun 20 01:47:56 2012 From: atribble at amazon.com (Tribble, Alex) Date: Tue, 19 Jun 2012 18:47:56 -0700 Subject: [PATCH] Fix for ticket #106: Correctly handle multiple X-Forwarded-For headers Message-ID: <4553D8D1-06EF-4056-B707-68090684E8FA@amazon.com> When nginx gets multiple X-Forwarded-For headers in a single request, it only keeps the last one in r->headers_in (and thus in $http_x_forwarded_for, $proxy_add_x_forwarded_for). Reverse proxies behind an nginx instance sometimes need the entire X-Forwarded-For chain - part of which is discarded in this case. Per RFC 2616, it's equivalent to concatenate each header value (separated by a comma) and send the concatenated value to the upstream: 4.2 -snip- Multiple message-header fields with the same field-name MAY be present in a message if and only if the entire field-value for that header field is defined as a comma-separated list [i.e., #(values)]. It MUST be possible to combine the multiple header fields into one "field-name: field-value" pair, without changing the semantics of the message, by appending each subsequent field-value to the first, each separated by a comma. The order in which header fields with the same field-name are received is therefore significant to the interpretation of the combined field value, and thus a proxy MUST NOT change the order of these field values when a message is forwarded. -snip- Attached is a patch that does exactly this, in the case of multiple headers. Please let me know if you have any comments about this patch - I'm happy to make any changes you suggest. Relevant bug report: http://trac.nginx.org/nginx/ticket/106 Thanks, Alex Tribble [Sorry for the attachment, my MUAs all unanimously decided they hate me] -------------- next part -------------- A non-text attachment was scrubbed... Name: merge-xff.patch Type: application/octet-stream Size: 2342 bytes Desc: merge-xff.patch URL: From vshebordaev at mail.ru Wed Jun 20 07:15:01 2012 From: vshebordaev at mail.ru (Vladimir Shebordaev) Date: Wed, 20 Jun 2012 11:15:01 +0400 Subject: [PATCH] Fix for ticket #106: Correctly handle multiple X-Forwarded-For headers In-Reply-To: <4553D8D1-06EF-4056-B707-68090684E8FA@amazon.com> References: <4553D8D1-06EF-4056-B707-68090684E8FA@amazon.com> Message-ID: Hi. 2012/6/20 Tribble, Alex : > When nginx gets multiple X-Forwarded-For headers in a single request, it > only keeps the last one in r->headers_in (and thus in > $http_x_forwarded_for, $proxy_add_x_forwarded_for). Reverse proxies behind > an nginx instance sometimes need the entire X-Forwarded-For chain - part > of which is discarded in this case. > > Per RFC 2616, it's equivalent to concatenate each header value (separated > by a comma) and send the concatenated value to the upstream: > 4.2 > -snip- > ? Multiple message-header fields with the same field-name MAY be > ? present in a message if and only if the entire field-value for that > ? header field is defined as a comma-separated list [i.e., #(values)]. > ? It MUST be possible to combine the multiple header fields into one > ? "field-name: field-value" pair, without changing the semantics of the > ? message, by appending each subsequent field-value to the first, each > ? separated by a comma. The order in which header fields with the same > ? field-name are received is therefore significant to the > ? interpretation of the combined field value, and thus a proxy MUST NOT > ? change the order of these field values when a message is forwarded. > -snip- > > Attached is a patch that does exactly this, in the case of multiple headers. > Please let me know if you have any comments about this patch - I'm happy > to make any changes you suggest. > Basically, ngx_table_elt_t element structure is to contain an atomic header value for faster access when it is used internally, so I'd suggest to embed an array of ngx_table_elt_t elements into headers_in structure that would stack the headers in order of appearence. If the headers already come in in a comma separated list as per RFC 2616, it would be consistent to split 'em and also push into that array in order. It seems, output header structures and routines should be also changed accordingly. By the way, I guess X-Forwarded-For is not the only header that can stack. > Relevant bug report: > http://trac.nginx.org/nginx/ticket/106 > > Thanks, > Alex Tribble > In the hope it helps. -- Regards, Vladimir From igor at sysoev.ru Wed Jun 20 12:55:29 2012 From: igor at sysoev.ru (igor at sysoev.ru) Date: Wed, 20 Jun 2012 12:55:29 +0000 Subject: [nginx] svn commit: r4697 - trunk/src/event Message-ID: <20120620125529.283AB3F9F5A@mail.nginx.com> Author: is Date: 2012-06-20 12:55:28 +0000 (Wed, 20 Jun 2012) New Revision: 4697 URL: http://trac.nginx.org/nginx/changeset/4697/nginx Log: Disabled gzip compression in OpenSSL prior to 1.0.0 version. This saves about 522K per connection. Modified: trunk/src/event/ngx_event_openssl.c Modified: trunk/src/event/ngx_event_openssl.c =================================================================== --- trunk/src/event/ngx_event_openssl.c 2012-06-19 12:36:54 UTC (rev 4696) +++ trunk/src/event/ngx_event_openssl.c 2012-06-20 12:55:28 UTC (rev 4697) @@ -94,6 +94,24 @@ OpenSSL_add_all_algorithms(); +#ifndef SSL_OP_NO_COMPRESSION + { + /* + * Disable gzip compression in OpenSSL prior to 1.0.0 version, + * this saves about 522K per connection. + */ + int i, n; + STACK_OF(SSL_COMP) *ssl_comp_methods; + + ssl_comp_methods = SSL_COMP_get_compression_methods(); + n = sk_SSL_COMP_num(ssl_comp_methods); + + for (i = 0; i < n; i++) { + (void) sk_SSL_COMP_delete(ssl_comp_methods, i); + } + } +#endif + ngx_ssl_connection_index = SSL_get_ex_new_index(0, NULL, NULL, NULL, NULL); if (ngx_ssl_connection_index == -1) { From ne at vbart.ru Wed Jun 20 15:42:28 2012 From: ne at vbart.ru (Valentin V. Bartenev) Date: Wed, 20 Jun 2012 19:42:28 +0400 Subject: Spdy - Double Status header In-Reply-To: References: Message-ID: <201206201942.28689.ne@vbart.ru> On Wednesday 20 June 2012 05:12:12 Matthieu Tourne wrote: > Hi, > > I think, I've found an interesting corner case with the recent Nginx > SPDY support. > I'm using the latest : http://nginx.org/patches/spdy/patch.spdy-37.txt > > I connect to Nginx using SPDY, and then proxy_pass to another server, > with a location similar to this : > > location /redirect { > add_header 'Status' '301 Moved Permanently'; > rewrite (.*) / permanent; > } > > > This causes Nginx to return 2 Spdy ":status" headers, which I guess is > disallowed by the protocol. > > > This is the output from spdycat (https://github.com/tatsuhiro-t/spdylay/) > Chrome reports a similar error. > > [ 0.141] [INVALID; status=PROTOCOL_ERROR] recv SYN_REPLY frame > > (...) > status: 301 > status: 301 Moved Permanently > > > Note that this can be fixed using configuration to hide any potential > "Status" header returned by the second server, before the Spdy > response is generated. > But the code should probably handle this ? The problem is that spdy draft 2 reserved "status" and "version" header names for its own purpose. This was fixed in draft 3, where all such headers have the colon prefix (":status", ":version"). Personally, I think that since these headers are non-standard for HTTP protocol, then the user is entirely responsible for their usage. However, because the "Status" header is quite often appears somewhere, then hiding of these headers inside the spdy module will cause less headaches. Done in http://nginx.org/patches/spdy/patch.spdy-38.txt Thanks. wbr, Valentin V. Bartenev From matthieu.tourne at gmail.com Wed Jun 20 16:23:54 2012 From: matthieu.tourne at gmail.com (Matthieu Tourne) Date: Wed, 20 Jun 2012 09:23:54 -0700 Subject: Spdy - Double Status header In-Reply-To: <201206201942.28689.ne@vbart.ru> References: <201206201942.28689.ne@vbart.ru> Message-ID: On Jun 20, 2012 8:42 AM, "Valentin V. Bartenev" wrote: > > On Wednesday 20 June 2012 05:12:12 Matthieu Tourne wrote: > > Hi, > > > > I think, I've found an interesting corner case with the recent Nginx > > SPDY support. > > I'm using the latest : http://nginx.org/patches/spdy/patch.spdy-37.txt > > > > I connect to Nginx using SPDY, and then proxy_pass to another server, > > with a location similar to this : > > > > location /redirect { > > add_header 'Status' '301 Moved Permanently'; > > rewrite (.*) / permanent; > > } > > > > > > This causes Nginx to return 2 Spdy ":status" headers, which I guess is > > disallowed by the protocol. > > > > > > This is the output from spdycat (https://github.com/tatsuhiro-t/spdylay/ ) > > Chrome reports a similar error. > > > > [ 0.141] [INVALID; status=PROTOCOL_ERROR] recv SYN_REPLY frame > > > > (...) > > status: 301 > > status: 301 Moved Permanently > > > > > > Note that this can be fixed using configuration to hide any potential > > "Status" header returned by the second server, before the Spdy > > response is generated. > > But the code should probably handle this ? > > The problem is that spdy draft 2 reserved "status" and "version" header > names for its own purpose. This was fixed in draft 3, where all such > headers have the colon prefix (":status", ":version"). > > Personally, I think that since these headers are non-standard for HTTP > protocol, then the user is entirely responsible for their usage. However, > because the "Status" header is quite often appears somewhere, then hiding > of these headers inside the spdy module will cause less headaches. Why not guarantee that all these reserved headers will only appear once in the response? This way the visiting browser never "breaks". Nginx could always take the first header found in the http response for the spdy response. And avoid breaking just because the upstream is badly configured and returns 2 Version headers for example. -------------- next part -------------- An HTML attachment was scrubbed... URL: From ne at vbart.ru Wed Jun 20 17:00:00 2012 From: ne at vbart.ru (Valentin V. Bartenev) Date: Wed, 20 Jun 2012 21:00:00 +0400 Subject: Spdy - Double Status header In-Reply-To: References: <201206201942.28689.ne@vbart.ru> Message-ID: <201206202100.00349.ne@vbart.ru> On Wednesday 20 June 2012 20:23:54 Matthieu Tourne wrote: > On Jun 20, 2012 8:42 AM, "Valentin V. Bartenev" wrote: > > On Wednesday 20 June 2012 05:12:12 Matthieu Tourne wrote: > > > Hi, > > > > > > I think, I've found an interesting corner case with the recent Nginx > > > SPDY support. > > > I'm using the latest : http://nginx.org/patches/spdy/patch.spdy-37.txt > > > > > > I connect to Nginx using SPDY, and then proxy_pass to another server, > > > > > > with a location similar to this : > > > location /redirect { > > > > > > add_header 'Status' '301 Moved Permanently'; > > > rewrite (.*) / permanent; > > > > > > } > > > > > > This causes Nginx to return 2 Spdy ":status" headers, which I guess is > > > disallowed by the protocol. > > > > > > > > > This is the output from spdycat > > > (https://github.com/tatsuhiro-t/spdylay/ > > ) > > > > Chrome reports a similar error. > > > > > > [ 0.141] [INVALID; status=PROTOCOL_ERROR] recv SYN_REPLY frame > > > > > > (...) > > > > > > status: 301 > > > status: 301 Moved Permanently > > > > > > Note that this can be fixed using configuration to hide any potential > > > "Status" header returned by the second server, before the Spdy > > > response is generated. > > > But the code should probably handle this ? > > > > The problem is that spdy draft 2 reserved "status" and "version" header > > names for its own purpose. This was fixed in draft 3, where all such > > headers have the colon prefix (":status", ":version"). > > > > Personally, I think that since these headers are non-standard for HTTP > > protocol, then the user is entirely responsible for their usage. However, > > because the "Status" header is quite often appears somewhere, then hiding > > of these headers inside the spdy module will cause less headaches. > > Why not guarantee that all these reserved headers will only appear once in > the response? > This way the visiting browser never "breaks". > > Nginx could always take the first header found in the http response for the > spdy response. And avoid breaking just because the upstream is badly > configured and returns 2 Version headers for example. The SPDY "status" and "version" headers are special. They are not the same as the "Status:" and "Version:" headers from HTTP response. SPDY "status" is analog of Status-Code from HTTP Status-Line, while the SPDY "version" header stores HTTP-Version from the Status-Line. http://tools.ietf.org/html/rfc2616#section-6.1 wbr, Valentin V. Bartenev From agentzh at gmail.com Thu Jun 21 02:55:49 2012 From: agentzh at gmail.com (agentzh) Date: Thu, 21 Jun 2012 10:55:49 +0800 Subject: [PATCH] Fix logging keepalive related variables bug In-Reply-To: <20120224043334.GH67687@mdounin.ru> References: <20120224043334.GH67687@mdounin.ru> Message-ID: Hello! On Fri, Feb 24, 2012 at 12:33 PM, Maxim Dounin wrote: > On Thu, Feb 23, 2012 at 03:21:05PM +0800, Joshua Zhu wrote: >> On Mon, Feb 13, 2012 at 3:08 PM, Joshua Zhu wrote: >> > Hi, >> > >> > A bug was introduced in revision 3181 that r->keepalive was set to 0 >> > before calling ngx_http_log_reques(), so the $sent_http_connection and >> > $sent_http_keep_alive variables will not work anymore. >> >> Could someone please review this patch? > > It's flagged in my mailbox. ?There is nothing wrong with the > patch, but I tend to think that original r3181 is wrong and should > be just reverted (or redone properly if it tries to address some > valid problem). ?I've tried to ask Igor about it but he doesn't > remember details, so it waits for detailed invesigation. > Any progress on this issue? It seems that it still exists in at least nginx 1.0.15 and 1.2.1 :) Thanks! -agentzh From ru at nginx.com Thu Jun 21 11:02:22 2012 From: ru at nginx.com (ru at nginx.com) Date: Thu, 21 Jun 2012 11:02:22 +0000 Subject: [nginx] svn commit: r4698 - in trunk: auto src/http Message-ID: <20120621110223.0E4A63F9F4E@mail.nginx.com> Author: ru Date: 2012-06-21 11:02:22 +0000 (Thu, 21 Jun 2012) New Revision: 4698 URL: http://trac.nginx.org/nginx/changeset/4698/nginx Log: Fixed compile-time conditionals used to detect if X-Forwarded-For support is needed. Modified: trunk/auto/modules trunk/src/http/ngx_http_request.c trunk/src/http/ngx_http_request.h trunk/src/http/ngx_http_variables.c Modified: trunk/auto/modules =================================================================== --- trunk/auto/modules 2012-06-20 12:55:28 UTC (rev 4697) +++ trunk/auto/modules 2012-06-21 11:02:22 UTC (rev 4698) @@ -223,6 +223,7 @@ if [ $HTTP_REALIP = YES ]; then have=NGX_HTTP_REALIP . auto/have + have=NGX_HTTP_X_FORWARDED_FOR . auto/have HTTP_MODULES="$HTTP_MODULES $HTTP_REALIP_MODULE" HTTP_SRCS="$HTTP_SRCS $HTTP_REALIP_SRCS" fi @@ -233,12 +234,13 @@ fi if [ $HTTP_GEO = YES ]; then - have=NGX_HTTP_GEO . auto/have + have=NGX_HTTP_X_FORWARDED_FOR . auto/have HTTP_MODULES="$HTTP_MODULES $HTTP_GEO_MODULE" HTTP_SRCS="$HTTP_SRCS $HTTP_GEO_SRCS" fi if [ $HTTP_GEOIP = YES ]; then + have=NGX_HTTP_X_FORWARDED_FOR . auto/have HTTP_MODULES="$HTTP_MODULES $HTTP_GEOIP_MODULE" HTTP_SRCS="$HTTP_SRCS $HTTP_GEOIP_SRCS" fi @@ -273,7 +275,7 @@ fi if [ $HTTP_PROXY = YES ]; then - have=NGX_HTTP_PROXY . auto/have + have=NGX_HTTP_X_FORWARDED_FOR . auto/have #USE_MD5=YES HTTP_MODULES="$HTTP_MODULES $HTTP_PROXY_MODULE" HTTP_DEPS="$HTTP_DEPS $HTTP_PROXY_DEPS" Modified: trunk/src/http/ngx_http_request.c =================================================================== --- trunk/src/http/ngx_http_request.c 2012-06-20 12:55:28 UTC (rev 4697) +++ trunk/src/http/ngx_http_request.c 2012-06-21 11:02:22 UTC (rev 4698) @@ -138,7 +138,7 @@ { ngx_string("Keep-Alive"), offsetof(ngx_http_headers_in_t, keep_alive), ngx_http_process_header_line }, -#if (NGX_HTTP_PROXY || NGX_HTTP_REALIP || NGX_HTTP_GEO) +#if (NGX_HTTP_X_FORWARDED_FOR) { ngx_string("X-Forwarded-For"), offsetof(ngx_http_headers_in_t, x_forwarded_for), ngx_http_process_header_line }, Modified: trunk/src/http/ngx_http_request.h =================================================================== --- trunk/src/http/ngx_http_request.h 2012-06-20 12:55:28 UTC (rev 4697) +++ trunk/src/http/ngx_http_request.h 2012-06-21 11:02:22 UTC (rev 4698) @@ -192,7 +192,7 @@ ngx_table_elt_t *keep_alive; -#if (NGX_HTTP_PROXY || NGX_HTTP_REALIP || NGX_HTTP_GEO) +#if (NGX_HTTP_X_FORWARDED_FOR) ngx_table_elt_t *x_forwarded_for; #endif Modified: trunk/src/http/ngx_http_variables.c =================================================================== --- trunk/src/http/ngx_http_variables.c 2012-06-20 12:55:28 UTC (rev 4697) +++ trunk/src/http/ngx_http_variables.c 2012-06-21 11:02:22 UTC (rev 4698) @@ -134,7 +134,7 @@ offsetof(ngx_http_request_t, headers_in.via), 0, 0 }, #endif -#if (NGX_HTTP_PROXY || NGX_HTTP_REALIP) +#if (NGX_HTTP_X_FORWARDED_FOR) { ngx_string("http_x_forwarded_for"), NULL, ngx_http_variable_header, offsetof(ngx_http_request_t, headers_in.x_forwarded_for), 0, 0 }, #endif From rob.stradling at comodo.com Thu Jun 21 14:04:26 2012 From: rob.stradling at comodo.com (Rob Stradling) Date: Thu, 21 Jun 2012 15:04:26 +0100 Subject: Does anyone plan to develop the feature of openssl' OCSP stapling? In-Reply-To: <4F5DCBF2.6020107@comodo.com> References: <201106161430.55434.rob.stradling@comodo.com> <20110616150212.GA38841@sysoev.ru> <4F5DCBF2.6020107@comodo.com> Message-ID: <4FE329EA.1010005@comodo.com> On 12/03/12 10:12, Rob Stradling wrote: > "As to OCSP, I'm going to implement it in the next 2.0 version." > > Igor, is this still your plan? > If so, do you have a (very rough) ETA for nginx v2.0? > (I don't see v2.0 mentioned here: http://trac.nginx.org/nginx/roadmap) > > Or are there any other regular nginx developers who could work on OCSP > Stapling? > > Or is it worth me having a go at writing a patch? Just to update this thread... "GlobalSign, DigiCert and Comodo Collaborate with NGINX to Improve Online Trust through Enhanced Certificate Revocation Checking, sign a Sponsorship Agreement" http://nginx.com/news/nginx-ocsp-stapling.html > Thanks! > > On 16/06/11 16:02, Igor Sysoev wrote: >> On Thu, Jun 16, 2011 at 02:30:55PM +0100, Rob Stradling wrote: >>> On November 25, 2010 04:42AM Weibin Yao wrote: >>>> Hi everyone, >>>> >>>> I think the the feature of OCSP stapling[1] is very useful for the >>>> browser blocked by the OCSP request. And the feature has supported >>>> since >>>> openssl 0.9.8g. Apache's mod_ssl has also added this patch in the >>>> development branch[2]. >>>> >>>> Does anyone have the plan to develop this feature? >>> >>> Hi. The CAs and Browsers represented in the CA/Browser Forum >>> (http://cabforum.org/forum.html) are growing increasingly interested in >>> encouraging wider adoption of OCSP Stapling. >>> >>> Since nobody else has replied to this thread, I presume that OCSP >>> Stapling is >>> not currently a priority for the core nginx developers. So, I've started >>> having a go at writing a patch. I'm basing it heavily on Dr Steve >>> Henson's >>> OCSP Stapling code that was first included in Apache httpd 2.3.3 [3]. >>> I'd like >>> to ask a few questions before I proceed any further: >>> >>> 1. If I am able to complete my patch, are you likely to review/commit >>> it? >>> Or is OCSP Stapling the sort of feature that you'd prefer to only let >>> a core >>> nginx developer work on? >>> >>> 2. I was under the impression that nginx started life as a fork of >>> Apache >>> httpd, but I don't see any messages along the lines of "This product >>> includes >>> software developed by the Apache Group..." in the source code. Is >>> nginx 100% >>> *not* a derivative work of Apache httpd? >>> >>> 3. Steve Henson's code is presumably licensed under ASL 2.0 [4], which >>> presumably means that my patch would be classed as a "Derivative >>> Work" subject >>> to various conditions (see the "4. Redistribution" section in ASL >>> 2.0). Would >>> this prevent you from accepting it? >>> >>> (Since ASL 2.0 says "nothing herein shall supersede or modify the >>> terms of any >>> separate license agreement you may have executed with Licensor >>> regarding such >>> Contributions", perhaps I should ask Steve Henson if he would be >>> willing to >>> contribute the same code to nginx under a different licence). >> >> nginx is not Apache fork. I know well enough Apache 1.3 and I've got some >> ideas from Apache such as memory pools, configuration methods, processing >> phases, etc., but there is no line of Apache code. As to OCSP, I'm going >> to implement it in the next 2.0 version. >> >> > -- Rob Stradling Senior Research & Development Scientist COMODO - Creating Trust Online Office Tel: +44.(0)1274.730505 Office Fax: +44.(0)1274.730909 www.comodo.com COMODO CA Limited, Registered in England No. 04058690 Registered Office: 3rd Floor, 26 Office Village, Exchange Quay, Trafford Road, Salford, Manchester M5 3EQ This e-mail and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom they are addressed. If you have received this email in error please notify the sender by replying to the e-mail containing this attachment. Replies to this email may be monitored by COMODO for operational or business reasons. Whilst every endeavour is taken to ensure that e-mails are free from viruses, no liability can be accepted and the recipient is requested to use their own virus checking software. From mdounin at mdounin.ru Thu Jun 21 17:08:03 2012 From: mdounin at mdounin.ru (Maxim Dounin) Date: Thu, 21 Jun 2012 21:08:03 +0400 Subject: [PATCH] Fix logging keepalive related variables bug In-Reply-To: References: <20120224043334.GH67687@mdounin.ru> Message-ID: <20120621170803.GP31671@mdounin.ru> Hello! On Thu, Jun 21, 2012 at 10:55:49AM +0800, agentzh wrote: > Hello! > > On Fri, Feb 24, 2012 at 12:33 PM, Maxim Dounin wrote: > > On Thu, Feb 23, 2012 at 03:21:05PM +0800, Joshua Zhu wrote: > >> On Mon, Feb 13, 2012 at 3:08 PM, Joshua Zhu wrote: > >> > Hi, > >> > > >> > A bug was introduced in revision 3181 that r->keepalive was set to 0 > >> > before calling ngx_http_log_reques(), so the $sent_http_connection and > >> > $sent_http_keep_alive variables will not work anymore. > >> > >> Could someone please review this patch? > > > > It's flagged in my mailbox. ?There is nothing wrong with the > > patch, but I tend to think that original r3181 is wrong and should > > be just reverted (or redone properly if it tries to address some > > valid problem). ?I've tried to ask Igor about it but he doesn't > > remember details, so it waits for detailed invesigation. > > > > Any progress on this issue? It seems that it still exists in at least > nginx 1.0.15 and 1.2.1 :) No progress yet, ENOTIME. Maxim Dounin From faskiri.devel at gmail.com Sat Jun 23 07:00:16 2012 From: faskiri.devel at gmail.com (Fasih) Date: Sat, 23 Jun 2012 12:30:16 +0530 Subject: Regarding rewrite Message-ID: Hello all I have a usecase for a server rewrite, what I essentially want to do is have a common domain like common.faskiri.com serve some contents for specific domains like zone1.com, zone2.com etc. for some specific url pattern. For instance: common.faskiri.com/zone1/asset should basically be rewritten to zone1/asset. Now zone1 has its own server section with /asset configured. I tried using rewrite module and realized it doesnt seem to handle the usecase I have. I created a plugin with NGX_HTTP_SERVER_REWRITE_PHASE expecting that if I rewrite request.headers_in.server here, nginx would set the srv_conf to zone1 and evaluate location config accordingly. But this doesnt seem to work either, nginx seems to handle these cases with a 301 redirect instead. For now I have configured my system to not use common.faskiri.com and instead have common.zone1.com which uses the same set of location config as zone1.com however what I failed to understand was, why is server rewriting not allowed, or if it is allowed, how do I use it. Thank you for your patience +Fasih From ne at vbart.ru Sat Jun 23 11:59:20 2012 From: ne at vbart.ru (Valentin V. Bartenev) Date: Sat, 23 Jun 2012 15:59:20 +0400 Subject: Regarding rewrite In-Reply-To: References: Message-ID: <201206231559.21116.ne@vbart.ru> On Saturday 23 June 2012 11:00:16 Fasih wrote: > Hello all > > I have a usecase for a server rewrite, what I essentially want to do > is have a common domain like common.faskiri.com serve some contents > for specific domains like zone1.com, zone2.com etc. for some specific > url pattern. > > For instance: > common.faskiri.com/zone1/asset should basically be rewritten to > zone1/asset. Now zone1 has its own server section with /asset > configured. > > I tried using rewrite module and realized it doesnt seem to handle the > usecase I have. > > I created a plugin with NGX_HTTP_SERVER_REWRITE_PHASE expecting that > if I rewrite request.headers_in.server here, nginx would set the > srv_conf to zone1 and evaluate location config accordingly. But this > doesnt seem to work either, nginx seems to handle these cases with a > 301 redirect instead. > > For now I have configured my system to not use common.faskiri.com and > instead have common.zone1.com which uses the same set of location > config as zone1.com however what I failed to understand was, why is > server rewriting not allowed, or if it is allowed, how do I use it. > Why do you need several different "server" sections, instead of something like this: server { server_name common.faskiri.com; location /zone1 { # locations for /zone1 here } location /zone2 { # locations for /zone2 here } } ? wbr, Valentin V. Bartenev From agentzh at gmail.com Sun Jun 24 05:10:13 2012 From: agentzh at gmail.com (agentzh) Date: Sun, 24 Jun 2012 13:10:13 +0800 Subject: [PATCH] fixed an "IfIsEvil" example that ngx_proxy did not inherit plcf->location Message-ID: Hello! The standard ngx_proxy module does not inherit the location config's "location" field for the "location if" block in ngx_http_proxy_merge_loc_conf, making the following config example fail mentioned on the "IfIsEvil" page: # request will be sent to backend without uri changed # to '/' due to if location /proxy-pass-uri { proxy_pass http://127.0.0.1:8080/; set $true 1; if ($true) { # nothing } } Below is a patch for nginx 1.2.1 that attempts to fix this issue. Comments welcome! Thanks! -agentzh --- nginx-1.2.1/src/http/modules/ngx_http_proxy_module.c 2012-04-23 18:40:01.000000000 +0800 +++ nginx-1.2.1-patched/src/http/modules/ngx_http_proxy_module.c 2012-06-24 12:48:57.289834450 +0800 @@ -3023,8 +3023,10 @@ if (conf->upstream.upstream || conf->proxy_lengths) { clcf = ngx_http_conf_get_module_loc_conf(cf, ngx_http_core_module); - if (clcf->handler == NULL && clcf->lmt_excpt) { - clcf->handler = ngx_http_proxy_handler; + if (clcf->handler == NULL) { + if (clcf->lmt_excpt) { + clcf->handler = ngx_http_proxy_handler; + } conf->location = prev->location; } } -------------- next part -------------- A non-text attachment was scrubbed... Name: nginx-1.2.1-location_if_inherits_proxy.patch Type: application/octet-stream Size: 674 bytes Desc: not available URL: From faskiri.devel at gmail.com Sun Jun 24 07:08:58 2012 From: faskiri.devel at gmail.com (Fasih) Date: Sun, 24 Jun 2012 12:38:58 +0530 Subject: Regarding rewrite In-Reply-To: <201206231559.21116.ne@vbart.ru> References: <201206231559.21116.ne@vbart.ru> Message-ID: Hi Thank you for your suggestion, as I said, I can work around with a different configuration. But I need to configure the system with many servers because each of the server is a different virtual host each with its own configuration. I was trying to understand if this is a bug/limitation in the code or something more basic. Best Regards On Sat, Jun 23, 2012 at 5:29 PM, Valentin V. Bartenev wrote: > On Saturday 23 June 2012 11:00:16 Fasih wrote: >> Hello all >> >> I have a usecase for a server rewrite, what I essentially want to do >> is have a common domain like common.faskiri.com serve some contents >> for specific domains like zone1.com, zone2.com etc. for some specific >> url pattern. >> >> For instance: >> common.faskiri.com/zone1/asset should basically be rewritten to >> zone1/asset. Now zone1 has its own server section with /asset >> configured. >> >> I tried using rewrite module and realized it doesnt seem to handle the >> usecase I have. >> >> I created a plugin with NGX_HTTP_SERVER_REWRITE_PHASE expecting that >> if I rewrite request.headers_in.server here, nginx would set the >> srv_conf to zone1 and evaluate location config accordingly. But this >> doesnt seem to work either, nginx seems to handle these cases with a >> 301 redirect instead. >> >> For now I have configured my system to not use common.faskiri.com and >> instead have common.zone1.com which uses the same set of location >> config as zone1.com however what I failed to understand was, why is >> server rewriting not allowed, or if it is allowed, how do I use it. >> > > Why do you need several different "server" sections, instead of > something like this: > > ?server { > ? ? server_name common.faskiri.com; > > ? ? location /zone1 { > ? ? ? ? # locations for /zone1 here > ? ? } > > ? ? location /zone2 { > ? ? ? ? # locations for /zone2 here > ? ? } > ?} > > ? > > ?wbr, Valentin V. Bartenev > > _______________________________________________ > nginx-devel mailing list > nginx-devel at nginx.org > http://mailman.nginx.org/mailman/listinfo/nginx-devel From info at tvdw.eu Sun Jun 24 12:18:23 2012 From: info at tvdw.eu (Tom van der Woerdt) Date: Sun, 24 Jun 2012 14:18:23 +0200 Subject: SPDY bug: gzip_static doesn't work Message-ID: <4FE7058F.7010202@tvdw.eu> Bug summary: Cannot use gzip_static when spdy is enabled. Removing spdy from the configuration restores previous functionality. Steps to reproduce: * Sample location directive: location = /style.css { alias /path/to/style.css; gzip_static on; expires 24h; } * Ensure style.css.gz exists and style.css does not * Navigate to the site and notice that there's no stylesheet (use Firefox) Error log: 2012/06/24 14:03:15 [error] 12823#0: *3127 open() "/path/to/style.css" failed (2: No such file or directory) while SSL handshaking, client: my.ip.goes.here, server: example.com, request: "GET /style.css HTTP/1.1", host: "example.com", referrer: "https://example.com/" Notes: Bug only seems to affect Firefox, not Chrome SPDY spec mentions that all clients MUST support gzip, so there's no reason for this request to fail Request headers: Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8 Accept-Encoding: gzip, deflate Accept-Language: tlh,nl_NL;q=0.8,en-us;q=0.5,en;q=0.3 Cache-Control: max-age=0 Connection: keep-alive Cookie: Host: example.com Referer: https://example.com/style.css User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.7; rv:15.0) Gecko/20120623 Firefox/15.0a2 (Also tested on Firefox/13 but that one doesn't have Firebug) Build info: nginx version: nginx/1.3.1 built by gcc 4.1.2 20080704 (Red Hat 4.1.2-52) TLS SNI support enabled configure arguments: --prefix=/etc/nginx/ --sbin-path=/usr/sbin/nginx --conf-path=/etc/nginx/nginx.conf --error-log-path=/var/log/nginx/error.log --http-log-path=/var/log/nginx/access.log --pid-path=/var/run/nginx.pid --lock-path=/var/run/nginx.lock --http-client-body-temp-path=/var/cache/nginx/client_temp --http-proxy-temp-path=/var/cache/nginx/proxy_temp --http-fastcgi-temp-path=/var/cache/nginx/fastcgi_temp --http-uwsgi-temp-path=/var/cache/nginx/uwsgi_temp --http-scgi-temp-path=/var/cache/nginx/scgi_temp --user=nginx --group=nginx --with-http_ssl_module --with-http_realip_module --with-http_addition_module --with-http_sub_module --with-http_dav_module --with-http_flv_module --with-http_mp4_module --with-http_gzip_static_module --with-http_random_index_module --with-http_secure_link_module --with-http_stub_status_module --with-mail --with-mail_ssl_module --with-file-aio --with-ipv6 --with-cc-opt='-I/home/admin/customroot/include/ -O2 -g -m32 -march=i386 -mtune=generic -fasynchronous-unwind-tables' --with-ld-opt='-L/home/admin/customroot/lib/ -static' --with-openssl=/home/admin/services/openssl-1.0.1-beta3 Using spdy patch version 39 (latest, jun 22) From ne at vbart.ru Sun Jun 24 15:00:08 2012 From: ne at vbart.ru (Valentin V. Bartenev) Date: Sun, 24 Jun 2012 19:00:08 +0400 Subject: SPDY bug: gzip_static doesn't work In-Reply-To: <4FE7058F.7010202@tvdw.eu> References: <4FE7058F.7010202@tvdw.eu> Message-ID: <201206241900.08418.ne@vbart.ru> On Sunday 24 June 2012 16:18:23 Tom van der Woerdt wrote: > Bug summary: > Cannot use gzip_static when spdy is enabled. Removing spdy from the > configuration restores previous functionality. Thanks for the report. Fixed in: http://nginx.org/patches/spdy/patch.spdy-40.txt wbr, Valentin V. Bartenev > Steps to reproduce: > * Sample location directive: > location = /style.css { > alias /path/to/style.css; > gzip_static on; > expires 24h; > } > > * Ensure style.css.gz exists and style.css does not > * Navigate to the site and notice that there's no stylesheet (use > Firefox) > > Error log: > 2012/06/24 14:03:15 [error] 12823#0: *3127 open() > "/path/to/style.css" failed (2: No such file or directory) while SSL > handshaking, client: my.ip.goes.here, server: example.com, request: "GET > /style.css HTTP/1.1", host: "example.com", referrer: "https://example.com/" > > Notes: > Bug only seems to affect Firefox, not Chrome > SPDY spec mentions that all clients MUST support gzip, so there's no > reason for this request to fail > > Request headers: > Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8 > Accept-Encoding: gzip, deflate > Accept-Language: tlh,nl_NL;q=0.8,en-us;q=0.5,en;q=0.3 > Cache-Control: max-age=0 > Connection: keep-alive > Cookie: > Host: example.com > Referer: https://example.com/style.css > User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.7; rv:15.0) > Gecko/20120623 Firefox/15.0a2 > (Also tested on Firefox/13 but that one doesn't have Firebug) > > Build info: > nginx version: nginx/1.3.1 > built by gcc 4.1.2 20080704 (Red Hat 4.1.2-52) > TLS SNI support enabled > configure arguments: --prefix=/etc/nginx/ --sbin-path=/usr/sbin/nginx > --conf-path=/etc/nginx/nginx.conf > --error-log-path=/var/log/nginx/error.log > --http-log-path=/var/log/nginx/access.log --pid-path=/var/run/nginx.pid > --lock-path=/var/run/nginx.lock > --http-client-body-temp-path=/var/cache/nginx/client_temp > --http-proxy-temp-path=/var/cache/nginx/proxy_temp > --http-fastcgi-temp-path=/var/cache/nginx/fastcgi_temp > --http-uwsgi-temp-path=/var/cache/nginx/uwsgi_temp > --http-scgi-temp-path=/var/cache/nginx/scgi_temp --user=nginx > --group=nginx --with-http_ssl_module --with-http_realip_module > --with-http_addition_module --with-http_sub_module > --with-http_dav_module --with-http_flv_module --with-http_mp4_module > --with-http_gzip_static_module --with-http_random_index_module > --with-http_secure_link_module --with-http_stub_status_module > --with-mail --with-mail_ssl_module --with-file-aio --with-ipv6 > --with-cc-opt='-I/home/admin/customroot/include/ -O2 -g -m32 -march=i386 > -mtune=generic -fasynchronous-unwind-tables' > --with-ld-opt='-L/home/admin/customroot/lib/ -static' > --with-openssl=/home/admin/services/openssl-1.0.1-beta3 > > Using spdy patch version 39 (latest, jun 22) > > _______________________________________________ > nginx-devel mailing list > nginx-devel at nginx.org > http://mailman.nginx.org/mailman/listinfo/nginx-devel From info at tvdw.eu Sun Jun 24 16:11:05 2012 From: info at tvdw.eu (Tom van der Woerdt) Date: Sun, 24 Jun 2012 18:11:05 +0200 Subject: SPDY bug: gzip_static doesn't work In-Reply-To: <201206241900.08418.ne@vbart.ru> References: <4FE7058F.7010202@tvdw.eu> <201206241900.08418.ne@vbart.ru> Message-ID: <4FE73C19.6010405@tvdw.eu> Patch works. Thanks! Tom On 6/24/12 5:00 PM, Valentin V. Bartenev wrote: > On Sunday 24 June 2012 16:18:23 Tom van der Woerdt wrote: >> Bug summary: >> Cannot use gzip_static when spdy is enabled. Removing spdy from the >> configuration restores previous functionality. > Thanks for the report. Fixed in: http://nginx.org/patches/spdy/patch.spdy-40.txt > > wbr, Valentin V. Bartenev > > >> Steps to reproduce: >> * Sample location directive: >> location = /style.css { >> alias /path/to/style.css; >> gzip_static on; >> expires 24h; >> } >> >> * Ensure style.css.gz exists and style.css does not >> * Navigate to the site and notice that there's no stylesheet (use >> Firefox) >> >> Error log: >> 2012/06/24 14:03:15 [error] 12823#0: *3127 open() >> "/path/to/style.css" failed (2: No such file or directory) while SSL >> handshaking, client: my.ip.goes.here, server: example.com, request: "GET >> /style.css HTTP/1.1", host: "example.com", referrer: "https://example.com/" >> >> Notes: >> Bug only seems to affect Firefox, not Chrome >> SPDY spec mentions that all clients MUST support gzip, so there's no >> reason for this request to fail >> >> Request headers: >> Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8 >> Accept-Encoding: gzip, deflate >> Accept-Language: tlh,nl_NL;q=0.8,en-us;q=0.5,en;q=0.3 >> Cache-Control: max-age=0 >> Connection: keep-alive >> Cookie: >> Host: example.com >> Referer: https://example.com/style.css >> User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.7; rv:15.0) >> Gecko/20120623 Firefox/15.0a2 >> (Also tested on Firefox/13 but that one doesn't have Firebug) >> >> Build info: >> nginx version: nginx/1.3.1 >> built by gcc 4.1.2 20080704 (Red Hat 4.1.2-52) >> TLS SNI support enabled >> configure arguments: --prefix=/etc/nginx/ --sbin-path=/usr/sbin/nginx >> --conf-path=/etc/nginx/nginx.conf >> --error-log-path=/var/log/nginx/error.log >> --http-log-path=/var/log/nginx/access.log --pid-path=/var/run/nginx.pid >> --lock-path=/var/run/nginx.lock >> --http-client-body-temp-path=/var/cache/nginx/client_temp >> --http-proxy-temp-path=/var/cache/nginx/proxy_temp >> --http-fastcgi-temp-path=/var/cache/nginx/fastcgi_temp >> --http-uwsgi-temp-path=/var/cache/nginx/uwsgi_temp >> --http-scgi-temp-path=/var/cache/nginx/scgi_temp --user=nginx >> --group=nginx --with-http_ssl_module --with-http_realip_module >> --with-http_addition_module --with-http_sub_module >> --with-http_dav_module --with-http_flv_module --with-http_mp4_module >> --with-http_gzip_static_module --with-http_random_index_module >> --with-http_secure_link_module --with-http_stub_status_module >> --with-mail --with-mail_ssl_module --with-file-aio --with-ipv6 >> --with-cc-opt='-I/home/admin/customroot/include/ -O2 -g -m32 -march=i386 >> -mtune=generic -fasynchronous-unwind-tables' >> --with-ld-opt='-L/home/admin/customroot/lib/ -static' >> --with-openssl=/home/admin/services/openssl-1.0.1-beta3 >> >> Using spdy patch version 39 (latest, jun 22) >> >> _______________________________________________ >> nginx-devel mailing list >> nginx-devel at nginx.org >> http://mailman.nginx.org/mailman/listinfo/nginx-devel > _______________________________________________ > nginx-devel mailing list > nginx-devel at nginx.org > http://mailman.nginx.org/mailman/listinfo/nginx-devel From agentzh at gmail.com Mon Jun 25 06:04:53 2012 From: agentzh at gmail.com (agentzh) Date: Mon, 25 Jun 2012 14:04:53 +0800 Subject: [ANN] Test::Nginx 0.20 released In-Reply-To: References: Message-ID: Hello! I've just uploaded Test::Nginx 0.20 to CPAN: ? http://search.cpan.org/perldoc?Test::Nginx It will appear on the CPAN mirror near you in the next few hours or so. Special thanks go to all our contributors and users :) Here's the complete change log for this release (compared to the last CPAN release, 0.19): * feature: now "--- error_log" allows a single regexp object as its value. * feature: made client socket timeout as a test failure. * feature: added support for the environment TEST_NGINX_CHECK_LEAK=1 that enables the most general memory leak check by calling ab/weighttp + ps (for the nginx process's RSS). * feature: when the TEST_NGINX_USE_HUP environment is set to true, then HUP signal will be used to update the nginx config between different test blocks and there will be no HUP reload between every two successive test cases within the same test block when "repeat_each" is set to a number more than 1. * bugfix: fixed the TEST_NGINX_SLEEP environment support. * bugfix: we should always restore user files before each "repeat_each" iteration because some test case may modify the user files. * bugfix: setting environment TEST_NGINX_USE_HUP could not work with TEST_NGINX_USE_VALGRIND=1 properly. * bugfix: print out every test block's name when TEST_NGINX_USE_HUP=1 and TEST_NGINX_USE_VALGRIND=1. * bugfix: the "response body truncated" warning was a false alarm for HEAD requests. thanks Piotr Sikora for reporting this in github issue #1. * bugfix: do not add --gen-suppressions and --suppressions options to the valgrind command line when custom non-number TEST_NGINX_USE_VALGRIND env value is specified. * docs: documented the "--- timeout" section. * docs: added a link to Ant?nio P. P. Almeida's debian package for this module. This Perl module provides a test scaffold based on IO::Socket or LWP for automated testing in Nginx C module or ngx_lua-based Lua library development. This class inherits from Test::Base, thus bringing all its declarative power to the Nginx C module testing practices. Please check out the full documentation on CPAN: ? http://search.cpan.org/perldoc?Test::Nginx::Socket All of our Nginx modules (as well as our lua-resty-* libraries) are using Test::Nginx to drive their test suites. Please note that this module is completely different from the Test::Nginx module created by Maxim Dounin. Enjoy! -agentzh From ru at nginx.com Mon Jun 25 06:23:00 2012 From: ru at nginx.com (Ruslan Ermilov) Date: Mon, 25 Jun 2012 10:23:00 +0400 Subject: [PATCH] Fix for ticket #106: Correctly handle multiple X-Forwarded-For headers In-Reply-To: <4553D8D1-06EF-4056-B707-68090684E8FA@amazon.com> References: <4553D8D1-06EF-4056-B707-68090684E8FA@amazon.com> Message-ID: <20120625062300.GA22404@lo0.su> On Tue, Jun 19, 2012 at 06:47:56PM -0700, Tribble, Alex wrote: > When nginx gets multiple X-Forwarded-For headers in a single request, it > only keeps the last one in r->headers_in (and thus in > $http_x_forwarded_for, $proxy_add_x_forwarded_for). Reverse proxies behind > an nginx instance sometimes need the entire X-Forwarded-For chain - part > of which is discarded in this case. > > Per RFC 2616, it's equivalent to concatenate each header value (separated > by a comma) and send the concatenated value to the upstream: > 4.2 > -snip- > Multiple message-header fields with the same field-name MAY be > present in a message if and only if the entire field-value for that > header field is defined as a comma-separated list [i.e., #(values)]. > It MUST be possible to combine the multiple header fields into one > "field-name: field-value" pair, without changing the semantics of the > message, by appending each subsequent field-value to the first, each > separated by a comma. The order in which header fields with the same > field-name are received is therefore significant to the > interpretation of the combined field value, and thus a proxy MUST NOT > change the order of these field values when a message is forwarded. > -snip- > > Attached is a patch that does exactly this, in the case of multiple headers. > Please let me know if you have any comments about this patch - I'm happy > to make any changes you suggest. > > Relevant bug report: > http://trac.nginx.org/nginx/ticket/106 How's this patch instead? %%% Index: src/http/ngx_http_request.h =================================================================== --- src/http/ngx_http_request.h (revision 4698) +++ src/http/ngx_http_request.h (working copy) @@ -193,7 +193,7 @@ typedef struct { ngx_table_elt_t *keep_alive; #if (NGX_HTTP_X_FORWARDED_FOR) - ngx_table_elt_t *x_forwarded_for; + ngx_array_t x_forwarded_for; #endif #if (NGX_HTTP_REALIP) Index: src/http/ngx_http_request.c =================================================================== --- src/http/ngx_http_request.c (revision 4698) +++ src/http/ngx_http_request.c (working copy) @@ -21,14 +21,14 @@ static ngx_int_t ngx_http_process_header ngx_table_elt_t *h, ngx_uint_t offset); static ngx_int_t ngx_http_process_unique_header_line(ngx_http_request_t *r, ngx_table_elt_t *h, ngx_uint_t offset); +static ngx_int_t ngx_http_process_multi_header_lines(ngx_http_request_t *r, + ngx_table_elt_t *h, ngx_uint_t offset); static ngx_int_t ngx_http_process_host(ngx_http_request_t *r, ngx_table_elt_t *h, ngx_uint_t offset); static ngx_int_t ngx_http_process_connection(ngx_http_request_t *r, ngx_table_elt_t *h, ngx_uint_t offset); static ngx_int_t ngx_http_process_user_agent(ngx_http_request_t *r, ngx_table_elt_t *h, ngx_uint_t offset); -static ngx_int_t ngx_http_process_cookie(ngx_http_request_t *r, - ngx_table_elt_t *h, ngx_uint_t offset); static ngx_int_t ngx_http_process_request_header(ngx_http_request_t *r); static void ngx_http_process_request(ngx_http_request_t *r); @@ -141,7 +141,7 @@ ngx_http_header_t ngx_http_headers_in[] #if (NGX_HTTP_X_FORWARDED_FOR) { ngx_string("X-Forwarded-For"), offsetof(ngx_http_headers_in_t, x_forwarded_for), - ngx_http_process_header_line }, + ngx_http_process_multi_header_lines }, #endif #if (NGX_HTTP_REALIP) @@ -173,7 +173,8 @@ ngx_http_header_t ngx_http_headers_in[] ngx_http_process_header_line }, #endif - { ngx_string("Cookie"), 0, ngx_http_process_cookie }, + { ngx_string("Cookie"), offsetof(ngx_http_headers_in_t, cookies), + ngx_http_process_multi_header_lines }, { ngx_null_string, 0, NULL } }; @@ -917,15 +918,6 @@ ngx_http_process_request_line(ngx_event_ return; } - - if (ngx_array_init(&r->headers_in.cookies, r->pool, 2, - sizeof(ngx_table_elt_t *)) - != NGX_OK) - { - ngx_http_close_request(r, NGX_HTTP_INTERNAL_SERVER_ERROR); - return; - } - c->log->action = "reading client request headers"; rev->handler = ngx_http_process_request_headers; @@ -1526,20 +1518,31 @@ ngx_http_process_user_agent(ngx_http_req static ngx_int_t -ngx_http_process_cookie(ngx_http_request_t *r, ngx_table_elt_t *h, +ngx_http_process_multi_header_lines(ngx_http_request_t *r, ngx_table_elt_t *h, ngx_uint_t offset) { - ngx_table_elt_t **cookie; + ngx_array_t *headers; + ngx_table_elt_t **ph; - cookie = ngx_array_push(&r->headers_in.cookies); - if (cookie) { - *cookie = h; - return NGX_OK; + headers = (ngx_array_t *) ((char *) &r->headers_in + offset); + + if (headers->elts == NULL) { + if (ngx_array_init(headers, r->pool, 1, sizeof(ngx_table_elt_t *)) + != NGX_OK) + { + ngx_http_close_request(r, NGX_HTTP_INTERNAL_SERVER_ERROR); + return NGX_ERROR; + } } - ngx_http_close_request(r, NGX_HTTP_INTERNAL_SERVER_ERROR); + ph = ngx_array_push(headers); + if (ph == NULL) { + ngx_http_close_request(r, NGX_HTTP_INTERNAL_SERVER_ERROR); + return NGX_ERROR; + } - return NGX_ERROR; + *ph = h; + return NGX_OK; } Index: src/http/ngx_http_core_module.h =================================================================== --- src/http/ngx_http_core_module.h (revision 4696) +++ src/http/ngx_http_core_module.h (working copy) @@ -514,7 +514,8 @@ ngx_int_t ngx_http_set_disable_symlinks( ngx_http_core_loc_conf_t *clcf, ngx_str_t *path, ngx_open_file_info_t *of); ngx_int_t ngx_http_get_forwarded_addr(ngx_http_request_t *r, ngx_addr_t *addr, - u_char *xff, size_t xfflen, ngx_array_t *proxies, int recursive); + ngx_array_t *headers, ngx_str_t *hvalue, ngx_array_t *proxies, + int recursive); extern ngx_module_t ngx_http_core_module; Index: src/http/ngx_http_core_module.c =================================================================== --- src/http/ngx_http_core_module.c (revision 4696) +++ src/http/ngx_http_core_module.c (working copy) @@ -2699,12 +2699,13 @@ ngx_http_set_disable_symlinks(ngx_http_r } -ngx_int_t -ngx_http_get_forwarded_addr(ngx_http_request_t *r, ngx_addr_t *addr, +static ngx_int_t +ngx_http_get_forwarded_addr_iter(ngx_http_request_t *r, ngx_addr_t *addr, u_char *xff, size_t xfflen, ngx_array_t *proxies, int recursive) { u_char *p; in_addr_t inaddr; + ngx_int_t rc; ngx_addr_t paddr; ngx_cidr_t *cidr; ngx_uint_t family, i; @@ -2788,8 +2789,15 @@ ngx_http_get_forwarded_addr(ngx_http_req *addr = paddr; if (recursive && p > xff) { - (void) ngx_http_get_forwarded_addr(r, addr, xff, p - 1 - xff, - proxies, 1); + rc = ngx_http_get_forwarded_addr_iter(r, addr, xff, p - 1 - xff, + proxies, 1); + + if (rc == NGX_DECLINED) { + return NGX_DONE; + } + + /* rc == NGX_OK || rc == NGX_DONE */ + return rc; } return NGX_OK; @@ -2802,6 +2810,45 @@ ngx_http_get_forwarded_addr(ngx_http_req } +ngx_int_t +ngx_http_get_forwarded_addr(ngx_http_request_t *r, ngx_addr_t *addr, + ngx_array_t *headers, ngx_str_t *hvalue, ngx_array_t *proxies, + int recursive) +{ + ngx_int_t rc; + ngx_uint_t i; + ngx_table_elt_t **h; + + if (headers == NULL) { + return ngx_http_get_forwarded_addr_iter(r, addr, hvalue->data, + hvalue->len, proxies, + recursive); + } + + i = headers->nelts; + h = headers->elts; + + rc = NGX_DECLINED; + + for ( ;; ) { + if (i == 0) { + break; + } + i--; + + rc = ngx_http_get_forwarded_addr_iter(r, addr, h[i]->value.data, + h[i]->value.len, proxies, + recursive); + + if (!recursive || rc != NGX_OK) { + break; + } + } + + return rc; +} + + static char * ngx_http_core_server(ngx_conf_t *cf, ngx_command_t *cmd, void *dummy) { Index: src/http/ngx_http_variables.c =================================================================== --- src/http/ngx_http_variables.c (revision 4698) +++ src/http/ngx_http_variables.c (working copy) @@ -21,8 +21,14 @@ static void ngx_http_variable_request_se ngx_http_variable_value_t *v, uintptr_t data); static ngx_int_t ngx_http_variable_header(ngx_http_request_t *r, ngx_http_variable_value_t *v, uintptr_t data); -static ngx_int_t ngx_http_variable_headers(ngx_http_request_t *r, +static ngx_int_t ngx_http_variable_headers_sep(ngx_http_request_t *r, + ngx_http_variable_value_t *v, uintptr_t data, u_char sep); +static ngx_int_t ngx_http_variable_headers_semi(ngx_http_request_t *r, ngx_http_variable_value_t *v, uintptr_t data); +#if (NGX_HTTP_X_FORWARDED_FOR) +static ngx_int_t ngx_http_variable_headers_comma(ngx_http_request_t *r, + ngx_http_variable_value_t *v, uintptr_t data); +#endif static ngx_int_t ngx_http_variable_unknown_header_in(ngx_http_request_t *r, ngx_http_variable_value_t *v, uintptr_t data); @@ -112,8 +118,8 @@ static ngx_int_t ngx_http_variable_pid(n */ /* - * the $http_host, $http_user_agent, $http_referer, $http_via, - * and $http_x_forwarded_for variables may be handled by generic + * the $http_host, $http_user_agent, $http_referer, and $http_via + * variables may be handled by generic * ngx_http_variable_unknown_header_in(), but for performance reasons * they are handled using dedicated entries */ @@ -135,11 +141,11 @@ static ngx_http_variable_t ngx_http_cor #endif #if (NGX_HTTP_X_FORWARDED_FOR) - { ngx_string("http_x_forwarded_for"), NULL, ngx_http_variable_header, + { ngx_string("http_x_forwarded_for"), NULL, ngx_http_variable_headers_comma, offsetof(ngx_http_request_t, headers_in.x_forwarded_for), 0, 0 }, #endif - { ngx_string("http_cookie"), NULL, ngx_http_variable_headers, + { ngx_string("http_cookie"), NULL, ngx_http_variable_headers_semi, offsetof(ngx_http_request_t, headers_in.cookies), 0, 0 }, { ngx_string("content_length"), NULL, ngx_http_variable_header, @@ -252,7 +258,8 @@ static ngx_http_variable_t ngx_http_cor { ngx_string("sent_http_transfer_encoding"), NULL, ngx_http_variable_sent_transfer_encoding, 0, 0, 0 }, - { ngx_string("sent_http_cache_control"), NULL, ngx_http_variable_headers, + { ngx_string("sent_http_cache_control"), NULL, + ngx_http_variable_headers_semi, offsetof(ngx_http_request_t, headers_out.cache_control), 0, 0 }, { ngx_string("limit_rate"), ngx_http_variable_request_set_size, @@ -666,8 +673,8 @@ ngx_http_variable_header(ngx_http_reques static ngx_int_t -ngx_http_variable_headers(ngx_http_request_t *r, ngx_http_variable_value_t *v, - uintptr_t data) +ngx_http_variable_headers_sep(ngx_http_request_t *r, + ngx_http_variable_value_t *v, uintptr_t data, u_char sep) { size_t len; u_char *p, *end; @@ -688,7 +695,7 @@ ngx_http_variable_headers(ngx_http_reque continue; } - len += h[i]->value.len + sizeof("; ") - 1; + len += h[i]->value.len + 2; } if (len == 0) { @@ -696,7 +703,7 @@ ngx_http_variable_headers(ngx_http_reque return NGX_OK; } - len -= sizeof("; ") - 1; + len -= 2; v->valid = 1; v->no_cacheable = 0; @@ -731,7 +738,7 @@ ngx_http_variable_headers(ngx_http_reque break; } - *p++ = ';'; *p++ = ' '; + *p++ = sep; *p++ = ' '; } return NGX_OK; @@ -739,6 +746,26 @@ ngx_http_variable_headers(ngx_http_reque static ngx_int_t +ngx_http_variable_headers_semi(ngx_http_request_t *r, + ngx_http_variable_value_t *v, uintptr_t data) +{ + return ngx_http_variable_headers_sep(r, v, data, ';'); +} + + +#if (NGX_HTTP_X_FORWARDED_FOR) + +static ngx_int_t +ngx_http_variable_headers_comma(ngx_http_request_t *r, + ngx_http_variable_value_t *v, uintptr_t data) +{ + return ngx_http_variable_headers_sep(r, v, data, ','); +} + +#endif + + +static ngx_int_t ngx_http_variable_unknown_header_in(ngx_http_request_t *r, ngx_http_variable_value_t *v, uintptr_t data) { Index: src/http/modules/ngx_http_geo_module.c =================================================================== --- src/http/modules/ngx_http_geo_module.c (revision 4696) +++ src/http/modules/ngx_http_geo_module.c (working copy) @@ -215,19 +215,18 @@ static in_addr_t ngx_http_geo_addr(ngx_http_request_t *r, ngx_http_geo_ctx_t *ctx) { ngx_addr_t addr; - ngx_table_elt_t *xfwd; + ngx_array_t *xfwd; struct sockaddr_in *sin; if (ngx_http_geo_real_addr(r, ctx, &addr) != NGX_OK) { return INADDR_NONE; } - xfwd = r->headers_in.x_forwarded_for; + xfwd = &r->headers_in.x_forwarded_for; - if (xfwd != NULL && ctx->proxies != NULL) { - (void) ngx_http_get_forwarded_addr(r, &addr, xfwd->value.data, - xfwd->value.len, ctx->proxies, - ctx->proxy_recursive); + if (xfwd->elts != NULL && ctx->proxies != NULL) { + (void) ngx_http_get_forwarded_addr(r, &addr, xfwd, NULL, + ctx->proxies, ctx->proxy_recursive); } #if (NGX_HAVE_INET6) Index: src/http/modules/ngx_http_geoip_module.c =================================================================== --- src/http/modules/ngx_http_geoip_module.c (revision 4696) +++ src/http/modules/ngx_http_geoip_module.c (working copy) @@ -208,19 +208,18 @@ static u_long ngx_http_geoip_addr(ngx_http_request_t *r, ngx_http_geoip_conf_t *gcf) { ngx_addr_t addr; - ngx_table_elt_t *xfwd; + ngx_array_t *xfwd; struct sockaddr_in *sin; addr.sockaddr = r->connection->sockaddr; addr.socklen = r->connection->socklen; /* addr.name = r->connection->addr_text; */ - xfwd = r->headers_in.x_forwarded_for; + xfwd = &r->headers_in.x_forwarded_for; - if (xfwd != NULL && gcf->proxies != NULL) { - (void) ngx_http_get_forwarded_addr(r, &addr, xfwd->value.data, - xfwd->value.len, gcf->proxies, - gcf->proxy_recursive); + if (xfwd->elts != NULL && gcf->proxies != NULL) { + (void) ngx_http_get_forwarded_addr(r, &addr, xfwd, NULL, + gcf->proxies, gcf->proxy_recursive); } #if (NGX_HAVE_INET6) Index: src/http/modules/ngx_http_proxy_module.c =================================================================== --- src/http/modules/ngx_http_proxy_module.c (revision 4696) +++ src/http/modules/ngx_http_proxy_module.c (working copy) @@ -2260,32 +2260,55 @@ static ngx_int_t ngx_http_proxy_add_x_forwarded_for_variable(ngx_http_request_t *r, ngx_http_variable_value_t *v, uintptr_t data) { - u_char *p; + size_t len; + u_char *p; + ngx_uint_t i, n; + ngx_table_elt_t **h; v->valid = 1; v->no_cacheable = 0; v->not_found = 0; - if (r->headers_in.x_forwarded_for == NULL) { + n = r->headers_in.x_forwarded_for.nelts; + h = r->headers_in.x_forwarded_for.elts; + + len = 0; + + for (i = 0; i < n; i++) { + + if (h[i]->hash == 0) { + continue; + } + + len += h[i]->value.len + sizeof(", ") - 1; + } + + if (len == 0) { v->len = r->connection->addr_text.len; v->data = r->connection->addr_text.data; return NGX_OK; } - v->len = r->headers_in.x_forwarded_for->value.len - + sizeof(", ") - 1 + r->connection->addr_text.len; + len += r->connection->addr_text.len; - p = ngx_pnalloc(r->pool, v->len); + p = ngx_pnalloc(r->pool, len); if (p == NULL) { return NGX_ERROR; } + v->len = len; v->data = p; - p = ngx_copy(p, r->headers_in.x_forwarded_for->value.data, - r->headers_in.x_forwarded_for->value.len); + for (i = 0; i < n; i++) { + + if (h[i]->hash == 0) { + continue; + } + + p = ngx_copy(p, h[i]->value.data, h[i]->value.len); - *p++ = ','; *p++ = ' '; + *p++ = ','; *p++ = ' '; + } ngx_memcpy(p, r->connection->addr_text.data, r->connection->addr_text.len); Index: src/http/modules/ngx_http_realip_module.c =================================================================== --- src/http/modules/ngx_http_realip_module.c (revision 4696) +++ src/http/modules/ngx_http_realip_module.c (working copy) @@ -107,10 +107,12 @@ ngx_module_t ngx_http_realip_module = { static ngx_int_t ngx_http_realip_handler(ngx_http_request_t *r) { - u_char *ip, *p; + u_char *p; size_t len; + ngx_str_t *hvalue; ngx_uint_t i, hash; ngx_addr_t addr; + ngx_array_t *xfwd; ngx_list_part_t *part; ngx_table_elt_t *header; ngx_connection_t *c; @@ -137,19 +139,20 @@ ngx_http_realip_handler(ngx_http_request return NGX_DECLINED; } - len = r->headers_in.x_real_ip->value.len; - ip = r->headers_in.x_real_ip->value.data; + hvalue = &r->headers_in.x_real_ip->value; + xfwd = NULL; break; case NGX_HTTP_REALIP_XFWD: - if (r->headers_in.x_forwarded_for == NULL) { + xfwd = &r->headers_in.x_forwarded_for; + + if (xfwd->elts == NULL) { return NGX_DECLINED; } - len = r->headers_in.x_forwarded_for->value.len; - ip = r->headers_in.x_forwarded_for->value.data; + hvalue = NULL; break; @@ -178,8 +181,8 @@ ngx_http_realip_handler(ngx_http_request && len == header[i].key.len && ngx_strncmp(p, header[i].lowcase_key, len) == 0) { - len = header[i].value.len; - ip = header[i].value.data; + hvalue = &header[i].value; + xfwd = NULL; goto found; } @@ -192,15 +195,13 @@ found: c = r->connection; - ngx_log_debug1(NGX_LOG_DEBUG_HTTP, c->log, 0, "realip: \"%s\"", ip); - addr.sockaddr = c->sockaddr; addr.socklen = c->socklen; /* addr.name = c->addr_text; */ - if (ngx_http_get_forwarded_addr(r, &addr, ip, len, rlcf->from, + if (ngx_http_get_forwarded_addr(r, &addr, xfwd, hvalue, rlcf->from, rlcf->recursive) - == NGX_OK) + != NGX_DECLINED) { return ngx_http_realip_set_addr(r, &addr); } %%% From ru at nginx.com Mon Jun 25 13:08:25 2012 From: ru at nginx.com (ru at nginx.com) Date: Mon, 25 Jun 2012 13:08:25 +0000 Subject: [nginx] svn commit: r4699 - trunk/src/http Message-ID: <20120625130826.078473F9F0F@mail.nginx.com> Author: ru Date: 2012-06-25 13:08:25 +0000 (Mon, 25 Jun 2012) New Revision: 4699 URL: http://trac.nginx.org/nginx/changeset/4699/nginx Log: Fixed a harmless error in spelling of "Connection: close" when computing the response header length. Modified: trunk/src/http/ngx_http_header_filter_module.c Modified: trunk/src/http/ngx_http_header_filter_module.c =================================================================== --- trunk/src/http/ngx_http_header_filter_module.c 2012-06-21 11:02:22 UTC (rev 4698) +++ trunk/src/http/ngx_http_header_filter_module.c 2012-06-25 13:08:25 UTC (rev 4699) @@ -395,7 +395,7 @@ } } else { - len += sizeof("Connection: closed" CRLF) - 1; + len += sizeof("Connection: close" CRLF) - 1; } #if (NGX_HTTP_GZIP) From piotr.sikora at frickle.com Mon Jun 25 14:42:31 2012 From: piotr.sikora at frickle.com (Piotr Sikora) Date: Mon, 25 Jun 2012 16:42:31 +0200 Subject: [PATCH] SPDY: Kill unsafe version of ngx_spdy_frame_write_len macro. Message-ID: <7A03A1954B0C42D29CE719D58E139013@Desktop> commit 13d9f7edd9ad88e2bb1fb32a61b439c332c04076 Author: Piotr Sikora Date: Mon Jun 25 14:32:33 2012 +0000 Kill unsafe version of ngx_spdy_frame_write_len macro. This macro was accessing 4 bytes (via both: cast to uint32_t and read of p[3]), instead of 3 bytes it was supposed to, which could result in reads outside of the allocated memory region and SIGSEGV. Signed-off-by: Piotr Sikora diff --git a/src/http/ngx_http_spdy.h b/src/http/ngx_http_spdy.h index b26e05d..018624b 100644 --- a/src/http/ngx_http_spdy.h +++ b/src/http/ngx_http_spdy.h @@ -112,9 +112,6 @@ void ngx_http_spdy_filter_free_data_frame(ngx_http_spdy_frame_chain_t *frame); #define ngx_spdy_frame_write_uint32(p, s) \ (*(uint32_t *) (p) = htonl(s), (p) + 4) -#define ngx_spdy_frame_write_len(p, s) \ - (*(uint32_t *) (p) = htonl(((s) << 8) | (p)[3]), (p) + 3) - #define ngx_spdy_frame_write_len_unsafe(p, s) \ (*(uint32_t *) (p) = htonl((s) << 8), (p) + 3) @@ -131,13 +128,13 @@ void ngx_http_spdy_filter_free_data_frame(ngx_http_spdy_frame_chain_t *frame); (p)[2] = (u_char) (s) >> 8, \ (p)[3] = (u_char) (s), (p) + 4) +#define ngx_spdy_frame_write_len_unsafe ngx_spdy_frame_write_len + +#endif + #define ngx_spdy_frame_write_len(p, s) \ ((p)[0] = (u_char) ((s) >> 16), \ (p)[1] = (u_char) ((s) >> 8), \ (p)[2] = (u_char) (s), (p) + 3) -#define ngx_spdy_frame_write_len_unsafe ngx_spdy_frame_write_len - -#endif - #endif /* _NGX_HTTP_SPDY_H_INCLUDED_ */ From faskiri.devel at gmail.com Mon Jun 25 15:25:16 2012 From: faskiri.devel at gmail.com (Fasih) Date: Mon, 25 Jun 2012 20:55:16 +0530 Subject: Regarding rewrite In-Reply-To: References: <201206231559.21116.ne@vbart.ru> Message-ID: Bump On Sun, Jun 24, 2012 at 12:38 PM, Fasih wrote: > Hi > > Thank you for your suggestion, as I said, I can work around with a > different configuration. But I need to configure the system with many > servers because each of the server is a different virtual host each > with its own configuration. I was trying to understand if this is a > bug/limitation in the code or something more basic. > > Best Regards > > On Sat, Jun 23, 2012 at 5:29 PM, Valentin V. Bartenev wrote: >> On Saturday 23 June 2012 11:00:16 Fasih wrote: >>> Hello all >>> >>> I have a usecase for a server rewrite, what I essentially want to do >>> is have a common domain like common.faskiri.com serve some contents >>> for specific domains like zone1.com, zone2.com etc. for some specific >>> url pattern. >>> >>> For instance: >>> common.faskiri.com/zone1/asset should basically be rewritten to >>> zone1/asset. Now zone1 has its own server section with /asset >>> configured. >>> >>> I tried using rewrite module and realized it doesnt seem to handle the >>> usecase I have. >>> >>> I created a plugin with NGX_HTTP_SERVER_REWRITE_PHASE expecting that >>> if I rewrite request.headers_in.server here, nginx would set the >>> srv_conf to zone1 and evaluate location config accordingly. But this >>> doesnt seem to work either, nginx seems to handle these cases with a >>> 301 redirect instead. >>> >>> For now I have configured my system to not use common.faskiri.com and >>> instead have common.zone1.com which uses the same set of location >>> config as zone1.com however what I failed to understand was, why is >>> server rewriting not allowed, or if it is allowed, how do I use it. >>> >> >> Why do you need several different "server" sections, instead of >> something like this: >> >> ?server { >> ? ? server_name common.faskiri.com; >> >> ? ? location /zone1 { >> ? ? ? ? # locations for /zone1 here >> ? ? } >> >> ? ? location /zone2 { >> ? ? ? ? # locations for /zone2 here >> ? ? } >> ?} >> >> ? >> >> ?wbr, Valentin V. Bartenev >> >> _______________________________________________ >> nginx-devel mailing list >> nginx-devel at nginx.org >> http://mailman.nginx.org/mailman/listinfo/nginx-devel From mdounin at mdounin.ru Mon Jun 25 15:30:17 2012 From: mdounin at mdounin.ru (mdounin at mdounin.ru) Date: Mon, 25 Jun 2012 15:30:17 +0000 Subject: [nginx] svn commit: r4700 - tags Message-ID: <20120625153017.7EB233F9FEA@mail.nginx.com> Author: mdounin Date: 2012-06-25 15:30:17 +0000 (Mon, 25 Jun 2012) New Revision: 4700 URL: http://trac.nginx.org/nginx/changeset/4700/nginx Log: Incorrect tag release-1.2.1 removed. Changes to misc/GNUmakefile was missed during stable-1.2 branch creation, resulting in tag set on trunk, not on branch. Reported by Marcel Gmuer. Removed: tags/release-1.2.1/ From mdounin at mdounin.ru Mon Jun 25 15:35:27 2012 From: mdounin at mdounin.ru (mdounin at mdounin.ru) Date: Mon, 25 Jun 2012 15:35:27 +0000 Subject: [nginx] svn commit: r4701 - in tags: . release-1.2.1 Message-ID: <20120625153527.B14FF3F9FAF@mail.nginx.com> Author: mdounin Date: 2012-06-25 15:35:27 +0000 (Mon, 25 Jun 2012) New Revision: 4701 URL: http://trac.nginx.org/nginx/changeset/4701/nginx Log: Retagged release-1.2.1 properly. Added: tags/release-1.2.1/ Index: tags/release-1.2.1 =================================================================== --- branches/stable-1.2 2012-06-05 14:01:45 UTC (rev 4680) +++ tags/release-1.2.1 2012-06-25 15:35:27 UTC (rev 4701) Property changes on: tags/release-1.2.1 ___________________________________________________________________ Added: svn:ignore ## -0,0 +1,14 ## +access.log +client_body_temp +fastcgi_temp +proxy_temp +scgi_temp +uwsgi_temp +GNUmakefile +Makefile +makefile +nginx +nginx.conf +nginx-*.tar.gz +objs* +tmp Added: svn:mergeinfo ## -0,0 +1 ## +/trunk:4611-4632,4641,4674-4676 \ No newline at end of property From mdounin at mdounin.ru Mon Jun 25 15:40:30 2012 From: mdounin at mdounin.ru (mdounin at mdounin.ru) Date: Mon, 25 Jun 2012 15:40:30 +0000 Subject: [nginx] svn commit: r4702 - in branches/stable-1.2/src: core http/modules/perl Message-ID: <20120625154030.6013A3F9E2A@mail.nginx.com> Author: mdounin Date: 2012-06-25 15:40:29 +0000 (Mon, 25 Jun 2012) New Revision: 4702 URL: http://trac.nginx.org/nginx/changeset/4702/nginx Log: Version bump. Modified: branches/stable-1.2/src/core/nginx.h branches/stable-1.2/src/http/modules/perl/nginx.pm Modified: branches/stable-1.2/src/core/nginx.h =================================================================== --- branches/stable-1.2/src/core/nginx.h 2012-06-25 15:35:27 UTC (rev 4701) +++ branches/stable-1.2/src/core/nginx.h 2012-06-25 15:40:29 UTC (rev 4702) @@ -9,8 +9,8 @@ #define _NGINX_H_INCLUDED_ -#define nginx_version 1002001 -#define NGINX_VERSION "1.2.1" +#define nginx_version 1002002 +#define NGINX_VERSION "1.2.2" #define NGINX_VER "nginx/" NGINX_VERSION #define NGINX_VAR "NGINX" Modified: branches/stable-1.2/src/http/modules/perl/nginx.pm =================================================================== --- branches/stable-1.2/src/http/modules/perl/nginx.pm 2012-06-25 15:35:27 UTC (rev 4701) +++ branches/stable-1.2/src/http/modules/perl/nginx.pm 2012-06-25 15:40:29 UTC (rev 4702) @@ -50,7 +50,7 @@ HTTP_INSUFFICIENT_STORAGE ); -our $VERSION = '1.2.1'; +our $VERSION = '1.2.2'; require XSLoader; XSLoader::load('nginx', $VERSION); From mdounin at mdounin.ru Mon Jun 25 15:41:17 2012 From: mdounin at mdounin.ru (mdounin at mdounin.ru) Date: Mon, 25 Jun 2012 15:41:17 +0000 Subject: [nginx] svn commit: r4703 - branches/stable-1.2/misc Message-ID: <20120625154117.B5A233F9C18@mail.nginx.com> Author: mdounin Date: 2012-06-25 15:41:17 +0000 (Mon, 25 Jun 2012) New Revision: 4703 URL: http://trac.nginx.org/nginx/changeset/4703/nginx Log: Fixed misc/GNUmakefile to properly tag on stable-1.2 branch. Modified: branches/stable-1.2/misc/GNUmakefile Modified: branches/stable-1.2/misc/GNUmakefile =================================================================== --- branches/stable-1.2/misc/GNUmakefile 2012-06-25 15:40:29 UTC (rev 4702) +++ branches/stable-1.2/misc/GNUmakefile 2012-06-25 15:41:17 UTC (rev 4703) @@ -46,7 +46,8 @@ svn ci -F $(TEMP)/message echo "release-$(VER) tag" > $(TEMP)/message - svn copy $(REPO)/trunk $(REPO)/tags/release-$(VER) \ + svn copy $(REPO)/branches/stable-1.2 \ + $(REPO)/tags/release-$(VER) \ -F $(TEMP)/message svn up From ne at vbart.ru Mon Jun 25 15:47:55 2012 From: ne at vbart.ru (Valentin V. Bartenev) Date: Mon, 25 Jun 2012 19:47:55 +0400 Subject: [PATCH] SPDY: Kill unsafe version of ngx_spdy_frame_write_len macro. In-Reply-To: <7A03A1954B0C42D29CE719D58E139013@Desktop> References: <7A03A1954B0C42D29CE719D58E139013@Desktop> Message-ID: <201206251947.55968.ne@vbart.ru> On Monday 25 June 2012 18:42:31 Piotr Sikora wrote: > commit 13d9f7edd9ad88e2bb1fb32a61b439c332c04076 > Author: Piotr Sikora > Date: Mon Jun 25 14:32:33 2012 +0000 > > Kill unsafe version of ngx_spdy_frame_write_len macro. > > This macro was accessing 4 bytes (via both: cast to uint32_t and > read of p[3]), instead of 3 bytes it was supposed to, which could > result in reads outside of the allocated memory region and SIGSEGV. > > Signed-off-by: Piotr Sikora Patch was accepted. Thanks. wbr, Valentin V. Bartenev > diff --git a/src/http/ngx_http_spdy.h b/src/http/ngx_http_spdy.h > index b26e05d..018624b 100644 > --- a/src/http/ngx_http_spdy.h > +++ b/src/http/ngx_http_spdy.h > @@ -112,9 +112,6 @@ void > ngx_http_spdy_filter_free_data_frame(ngx_http_spdy_frame_chain_t *frame); > #define ngx_spdy_frame_write_uint32(p, s) > \ (*(uint32_t *) (p) = htonl(s), (p) + 4) > > -#define ngx_spdy_frame_write_len(p, s) > \ - (*(uint32_t *) (p) = htonl(((s) << 8) | (p)[3]), (p) + 3) > - > #define ngx_spdy_frame_write_len_unsafe(p, s) > \ (*(uint32_t *) (p) = htonl((s) << 8), (p) + 3) > > @@ -131,13 +128,13 @@ void > ngx_http_spdy_filter_free_data_frame(ngx_http_spdy_frame_chain_t *frame); > (p)[2] = (u_char) (s) >> 8, > \ (p)[3] = (u_char) (s), (p) + 4) > > +#define ngx_spdy_frame_write_len_unsafe ngx_spdy_frame_write_len > + > +#endif > + > #define ngx_spdy_frame_write_len(p, s) > \ ((p)[0] = (u_char) ((s) >> 16), > \ (p)[1] = (u_char) ((s) >> 8), > \ (p)[2] = (u_char) (s), (p) + 3) > > -#define ngx_spdy_frame_write_len_unsafe ngx_spdy_frame_write_len > - > -#endif > - > #endif /* _NGX_HTTP_SPDY_H_INCLUDED_ */ > > _______________________________________________ > nginx-devel mailing list > nginx-devel at nginx.org > http://mailman.nginx.org/mailman/listinfo/nginx-devel From mdounin at mdounin.ru Mon Jun 25 18:09:38 2012 From: mdounin at mdounin.ru (mdounin at mdounin.ru) Date: Mon, 25 Jun 2012 18:09:38 +0000 Subject: [nginx] svn commit: r4704 - trunk/src/os/win32 Message-ID: <20120625180938.C523E3F9FE3@mail.nginx.com> Author: mdounin Date: 2012-06-25 18:09:38 +0000 (Mon, 25 Jun 2012) New Revision: 4704 URL: http://trac.nginx.org/nginx/changeset/4704/nginx Log: Style. Modified: trunk/src/os/win32/ngx_files.c Modified: trunk/src/os/win32/ngx_files.c =================================================================== --- trunk/src/os/win32/ngx_files.c 2012-06-25 15:41:17 UTC (rev 4703) +++ trunk/src/os/win32/ngx_files.c 2012-06-25 18:09:38 UTC (rev 4704) @@ -722,7 +722,7 @@ if (ch == ':') { goto invalid; } - + if (ch == '.' || ch == ' ') { break; } From gert.cuykens at gmail.com Tue Jun 26 07:50:18 2012 From: gert.cuykens at gmail.com (Gert Cuykens) Date: Tue, 26 Jun 2012 09:50:18 +0200 Subject: parse authentication header into $user $password Message-ID: location ~ ^/~(.+?)/(.*)$ { if ($user != $1) {return 403;} alias /home/$1/$2; auth_pam "Restricted"; auth_pam_service_name "nginx"; dav_methods PUT DELETE MKCOL COPY MOVE; dav_access group:rw all:r; create_full_put_path on; } Can this be implemented pleas? From ru at nginx.com Tue Jun 26 08:15:40 2012 From: ru at nginx.com (ru at nginx.com) Date: Tue, 26 Jun 2012 08:15:40 +0000 Subject: [nginx] svn commit: r4705 - trunk/auto/lib/google-perftools Message-ID: <20120626081540.BB65A3F9F6E@mail.nginx.com> Author: ru Date: 2012-06-26 08:15:40 +0000 (Tue, 26 Jun 2012) New Revision: 4705 URL: http://trac.nginx.org/nginx/changeset/4705/nginx Log: Added MacPorts support. Modified: trunk/auto/lib/google-perftools/conf Modified: trunk/auto/lib/google-perftools/conf =================================================================== --- trunk/auto/lib/google-perftools/conf 2012-06-25 18:09:38 UTC (rev 4704) +++ trunk/auto/lib/google-perftools/conf 2012-06-26 08:15:40 UTC (rev 4705) @@ -29,6 +29,22 @@ fi +if [ $ngx_found = no ]; then + + # MacPorts + + ngx_feature="Google perftools in /opt/local/" + + if [ $NGX_RPATH = YES ]; then + ngx_feature_libs="-R/opt/local/lib -L/opt/local/lib -lprofiler" + else + ngx_feature_libs="-L/opt/local/lib -lprofiler" + fi + + . auto/feature +fi + + if [ $ngx_found = yes ]; then CORE_LIBS="$CORE_LIBS $ngx_feature_libs" From mdounin at mdounin.ru Tue Jun 26 09:50:00 2012 From: mdounin at mdounin.ru (Maxim Dounin) Date: Tue, 26 Jun 2012 13:50:00 +0400 Subject: parse authentication header into $user $password In-Reply-To: References: Message-ID: <20120626095000.GY31671@mdounin.ru> Hello! On Tue, Jun 26, 2012 at 09:50:18AM +0200, Gert Cuykens wrote: > location ~ ^/~(.+?)/(.*)$ { > if ($user != $1) {return 403;} > alias /home/$1/$2; > auth_pam "Restricted"; > auth_pam_service_name "nginx"; > dav_methods PUT DELETE MKCOL COPY MOVE; > dav_access group:rw all:r; > create_full_put_path on; > } > > Can this be implemented pleas? Try $remote_user. http://nginx.org/en/docs/http/ngx_http_core_module.html#variables Maxim Dounin From mdounin at mdounin.ru Tue Jun 26 12:31:40 2012 From: mdounin at mdounin.ru (mdounin at mdounin.ru) Date: Tue, 26 Jun 2012 12:31:40 +0000 Subject: [nginx] svn commit: r4706 - trunk/src/http/modules Message-ID: <20120626123140.A71223F9F2F@mail.nginx.com> Author: mdounin Date: 2012-06-26 12:31:40 +0000 (Tue, 26 Jun 2012) New Revision: 4706 URL: http://trac.nginx.org/nginx/changeset/4706/nginx Log: Mp4: fixed build on win32 after r4689. Modified: trunk/src/http/modules/ngx_http_mp4_module.c Modified: trunk/src/http/modules/ngx_http_mp4_module.c =================================================================== --- trunk/src/http/modules/ngx_http_mp4_module.c 2012-06-26 08:15:40 UTC (rev 4705) +++ trunk/src/http/modules/ngx_http_mp4_module.c 2012-06-26 12:31:40 UTC (rev 4706) @@ -1024,7 +1024,7 @@ + NGX_HTTP_MP4_MOOV_BUFFER_EXCESS * no_mdat; } - if (ngx_http_mp4_read(mp4, atom_data_size) != NGX_OK) { + if (ngx_http_mp4_read(mp4, (size_t) atom_data_size) != NGX_OK) { return NGX_ERROR; } From mdounin at mdounin.ru Tue Jun 26 13:46:23 2012 From: mdounin at mdounin.ru (mdounin at mdounin.ru) Date: Tue, 26 Jun 2012 13:46:23 +0000 Subject: [nginx] svn commit: r4707 - trunk/docs/xml/nginx Message-ID: <20120626134624.1C9BB3F9F2F@mail.nginx.com> Author: mdounin Date: 2012-06-26 13:46:23 +0000 (Tue, 26 Jun 2012) New Revision: 4707 URL: http://trac.nginx.org/nginx/changeset/4707/nginx Log: nginx-1.3.2-RELEASE Modified: trunk/docs/xml/nginx/changes.xml Modified: trunk/docs/xml/nginx/changes.xml =================================================================== --- trunk/docs/xml/nginx/changes.xml 2012-06-26 12:31:40 UTC (rev 4706) +++ trunk/docs/xml/nginx/changes.xml 2012-06-26 13:46:23 UTC (rev 4707) @@ -9,6 +9,111 @@ nginx changelog + + + + +???????? single ????????? keepalive ?????? ????????????. + + +the "single" parameter of the "keepalive" directive is now ignored. + + + + + +?????? SSL ?????? ????????? +? ??? ????? ??? ????????????? OpenSSL c????? 1.0.0. + + +SSL compression is now disabled when using all versions of OpenSSL, +including ones prior to 1.0.0. + + + + + +????????? "ip_hash" ?????? ????? ???????????? ??? ???????????? IPv6 ????????. + + +it is now possible to use the "ip_hash" directive to balance IPv6 clients. + + + + + +?????????? $status ?????? ????? ???????????? ?? ?????? ? ????????? log_format. + + +the $status variable can now be used not only in the "log_format" directive. + + + + + +??? ?????????? ???????? ???????? ??? ????????? segmentation fault, +???? ?????????????? ????????? resolver. + + +a segmentation fault might occur in a worker process on shutdown +if the "resolver" directive was used. + + + + + +? ??????? ???????? ??? ????????? segmentation fault, +???? ????????????? ?????? ngx_http_mp4_module. + + +a segmentation fault might occur in a worker process +if the ngx_http_mp4_module was used. + + + + + +? ?????? ngx_http_mp4_module. + + +in the ngx_http_mp4_module. + + + + + +? ??????? ???????? ??? ????????? segmentation fault, +???? ?????????????? ????????????? ????? ???????? ? ???????. + + +a segmentation fault might occur in a worker process +if conflicting wildcard server names were used. + + + + + +?? ????????? ARM nginx ??? ???????? ??????????? ?? ??????? SIGBUS. + + +nginx might be terminated abnormally on a SIGBUS signal on ARM platform. + + + + + +?? ????? ???????????????? ?? HP-UX ? ??? +??????????? alert "sendmsg() failed (9: Bad file number)". + + +an alert "sendmsg() failed (9: Bad file number)" on HP-UX +while reconfiguration. + + + + + + From mdounin at mdounin.ru Tue Jun 26 13:46:53 2012 From: mdounin at mdounin.ru (mdounin at mdounin.ru) Date: Tue, 26 Jun 2012 13:46:53 +0000 Subject: [nginx] svn commit: r4708 - tags Message-ID: <20120626134653.B52483F9FA8@mail.nginx.com> Author: mdounin Date: 2012-06-26 13:46:53 +0000 (Tue, 26 Jun 2012) New Revision: 4708 URL: http://trac.nginx.org/nginx/changeset/4708/nginx Log: release-1.3.2 tag Added: tags/release-1.3.2/ From mdounin at mdounin.ru Tue Jun 26 14:40:23 2012 From: mdounin at mdounin.ru (mdounin at mdounin.ru) Date: Tue, 26 Jun 2012 14:40:23 +0000 Subject: [nginx] svn commit: r4709 - in branches/stable-1.2: . src/http/modules/perl Message-ID: <20120626144023.F30513F9FAE@mail.nginx.com> Author: mdounin Date: 2012-06-26 14:40:23 +0000 (Tue, 26 Jun 2012) New Revision: 4709 URL: http://trac.nginx.org/nginx/changeset/4709/nginx Log: Merge of r4645: fixed warning during nginx.xs compilation. Modified: branches/stable-1.2/ branches/stable-1.2/src/http/modules/perl/nginx.xs Index: branches/stable-1.2 =================================================================== --- branches/stable-1.2 2012-06-26 13:46:53 UTC (rev 4708) +++ branches/stable-1.2 2012-06-26 14:40:23 UTC (rev 4709) Property changes on: branches/stable-1.2 ___________________________________________________________________ Modified: svn:mergeinfo ## -1 +1 ## -/trunk:4611-4632,4641,4674-4676 +/trunk:4611-4632,4641,4645,4674-4676 \ No newline at end of property Modified: branches/stable-1.2/src/http/modules/perl/nginx.xs =================================================================== --- branches/stable-1.2/src/http/modules/perl/nginx.xs 2012-06-26 13:46:53 UTC (rev 4708) +++ branches/stable-1.2/src/http/modules/perl/nginx.xs 2012-06-26 14:40:23 UTC (rev 4709) @@ -476,7 +476,7 @@ } if (header->key.len == sizeof("Content-Encoding") - 1 - && ngx_strncasecmp(header->key.data, "Content-Encoding", + && ngx_strncasecmp(header->key.data, (u_char *) "Content-Encoding", sizeof("Content-Encoding") - 1) == 0) { r->headers_out.content_encoding = header; From atribble at amazon.com Tue Jun 26 17:36:57 2012 From: atribble at amazon.com (Tribble, Alex) Date: Tue, 26 Jun 2012 10:36:57 -0700 Subject: [PATCH] Fix for ticket #106: Correctly handle multiple X-Forwarded-For headers In-Reply-To: <20120625062300.GA22404@lo0.su> Message-ID: It looks good to me - thank you very much for doing this. Attached patch applies to the stable-1.2 branch. On 6/24/12 11:23 PM, "Ruslan Ermilov" wrote: >On Tue, Jun 19, 2012 at 06:47:56PM -0700, Tribble, Alex wrote: >> When nginx gets multiple X-Forwarded-For headers in a single request, it >> only keeps the last one in r->headers_in (and thus in >> $http_x_forwarded_for, $proxy_add_x_forwarded_for). Reverse proxies >>behind >> an nginx instance sometimes need the entire X-Forwarded-For chain - part >> of which is discarded in this case. >> >> Per RFC 2616, it's equivalent to concatenate each header value >>(separated >> by a comma) and send the concatenated value to the upstream: >> 4.2 >> -snip- >> Multiple message-header fields with the same field-name MAY be >> present in a message if and only if the entire field-value for that >> header field is defined as a comma-separated list [i.e., #(values)]. >> It MUST be possible to combine the multiple header fields into one >> "field-name: field-value" pair, without changing the semantics of the >> message, by appending each subsequent field-value to the first, each >> separated by a comma. The order in which header fields with the same >> field-name are received is therefore significant to the >> interpretation of the combined field value, and thus a proxy MUST NOT >> change the order of these field values when a message is forwarded. >> -snip- >> >> Attached is a patch that does exactly this, in the case of multiple >>headers. >> Please let me know if you have any comments about this patch - I'm happy >> to make any changes you suggest. >> >> Relevant bug report: >> http://trac.nginx.org/nginx/ticket/106 > >How's this patch instead? > > -------------- next part -------------- A non-text attachment was scrubbed... Name: multi-xff-1_2.patch Type: application/octet-stream Size: 17334 bytes Desc: multi-xff-1_2.patch URL: From pgnet.dev+nginx at gmail.com Tue Jun 26 18:43:11 2012 From: pgnet.dev+nginx at gmail.com (pgndev) Date: Tue, 26 Jun 2012 11:43:11 -0700 Subject: enabling SPDY (patch 42) +nginx 1.3.2 breaks gzip Message-ID: building nginx 1.3.2 + SPDY patch (http://nginx.org/patches/spdy/patch.spdy-42.txt) with SPDY disabled, response headers show backend's gzip compression headers (1) nginx/1.3.2, SPDY <- OFF + Varnish3 nginx vhost conf: listen 192.168.1.17:443; listen [2001:453:3cd6:7b2::9]:443 ipv6only=on; > Response Headers Accept-Ranges bytes Age 5 Cache-Control public, max-age=300 ==> Content-Encoding gzip Content-Language en Content-Length 4560 Content-Type text/html; charset=utf-8 Date Tue, 26 Jun 2012 16:55:27 GMT Expires Sun, 19 Nov 1978 05:00:00 GMT Last-Modified Tue, 26 Jun 2012 16:55:18 GMT Link ; rel="canonical" ==> Server nginx Vary Accept-Encoding,User-Agent Via 1.1 varnish X-Cache-Hits 1 X-Drupal-Cache MISS X-Varnish 179527312 179527285 X-Varnish-Cache HIT x-ua-compatible IE=Edge,chrome=1 Request Headers Accept text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8 ==> Accept-Encoding gzip, deflate Accept-Language en,en-us;q=0.5 Connection keep-alive Cookie has_js=1 DNT 1 Host test.svr06.loc User-Agent Mozilla/5.0 (X11; Linux x86_64; rv:13.0) Gecko/20100101 Firefox/13.0.1 with SPDY EN-abled, gzip response headers are missing (2) nginx/1.3.2, SPDY <- ON + Varnish3 {{{ nginx vhost conf: listen 192.168.1.17:443 ssl spdy; listen [2001:453:3cd6:7b2::9]:443 ssl spdy ipv6only=on; > Response Headers Age 26 Cache-Control public, max-age=300 Content-Language en Content-Type text/html; charset=utf-8 Date Tue, 26 Jun 2012 16:51:22 GMT Expires Sun, 19 Nov 1978 05:00:00 GMT Last-Modified Tue, 26 Jun 2012 16:50:52 GMT Link ; rel="canonical" ==> Server nginx Vary Accept-Encoding,User-Agent Via 1.1 varnish X-Cache-Hits 1 X-Drupal-Cache MISS ==> X-Firefox-Spdy 1 X-Varnish 851454988 851454961 X-Varnish-Cache HIT x-ua-compatible IE=Edge,chrome=1 > Request Headers Accept text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8 ==> Accept-Encoding gzip, deflate Accept-Language en,en-us;q=0.5 Connection keep-alive Cookie has_js=1 DNT 1 Host test.svr06.loc User-Agent Mozilla/5.0 (X11; Linux x86_64; rv:13.0) Gecko/20100101 Firefox/13.0.1 From gert.cuykens at gmail.com Tue Jun 26 19:11:48 2012 From: gert.cuykens at gmail.com (Gert Cuykens) Date: Tue, 26 Jun 2012 21:11:48 +0200 Subject: parse authentication header into $user $password In-Reply-To: <20120626095000.GY31671@mdounin.ru> References: <20120626095000.GY31671@mdounin.ru> Message-ID: On Tue, Jun 26, 2012 at 11:50 AM, Maxim Dounin wrote: > > On Tue, Jun 26, 2012 at 09:50:18AM +0200, Gert Cuykens wrote: > >> location ~ ^/~(.+?)/(.*)$ { >> ? ? ? ? if ($user != $1) {return 403;} >> ? ? ? ? alias /home/$1/$2; >> ? ? ? ? auth_pam "Restricted"; >> ? ? ? ? auth_pam_service_name "nginx"; >> ? ? ? ? dav_methods PUT DELETE MKCOL COPY MOVE; >> ? ? ? ? dav_access group:rw all:r; >> ? ? ? ? create_full_put_path on; >> ? ? } >> >> Can this be implemented pleas? > > Try $remote_user. > > http://nginx.org/en/docs/http/ngx_http_core_module.html#variables Works, thanks From ne at vbart.ru Tue Jun 26 19:23:39 2012 From: ne at vbart.ru (Valentin V. Bartenev) Date: Tue, 26 Jun 2012 23:23:39 +0400 Subject: enabling SPDY (patch 42) +nginx 1.3.2 breaks gzip In-Reply-To: References: Message-ID: <201206262323.40090.ne@vbart.ru> On Tuesday 26 June 2012 22:43:11 pgndev wrote: > building nginx 1.3.2 + SPDY patch > (http://nginx.org/patches/spdy/patch.spdy-42.txt) > > with SPDY disabled, response headers show backend's gzip compression > headers > > (1) nginx/1.3.2, SPDY <- OFF + Varnish3 > > nginx vhost conf: > listen 192.168.1.17:443; > listen [2001:453:3cd6:7b2::9]:443 ipv6only=on; > > Response Headers > Accept-Ranges bytes > Age 5 > Cache-Control public, max-age=300 > ==> Content-Encoding gzip > Content-Language en > Content-Length 4560 > Content-Type text/html; charset=utf-8 > Date Tue, 26 Jun 2012 16:55:27 GMT > Expires Sun, 19 Nov 1978 05:00:00 GMT > Last-Modified Tue, 26 Jun 2012 16:55:18 GMT > Link ; rel="canonical" > ==> Server nginx > Vary Accept-Encoding,User-Agent > Via 1.1 varnish > X-Cache-Hits 1 > X-Drupal-Cache MISS > X-Varnish 179527312 179527285 > X-Varnish-Cache HIT > x-ua-compatible IE=Edge,chrome=1 > > > Request Headers > Accept text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8 > ==> Accept-Encoding gzip, deflate > Accept-Language en,en-us;q=0.5 > Connection keep-alive > Cookie has_js=1 > DNT 1 > Host test.svr06.loc > User-Agent Mozilla/5.0 (X11; Linux x86_64; rv:13.0) Gecko/20100101 > Firefox/13.0.1 > > > > > > with SPDY EN-abled, gzip response headers are missing > > > (2) nginx/1.3.2, SPDY <- ON + Varnish3 > {{{ > nginx vhost conf: > listen 192.168.1.17:443 ssl spdy; > listen [2001:453:3cd6:7b2::9]:443 ssl spdy ipv6only=on; > > Response Headers > Age 26 > Cache-Control public, max-age=300 > Content-Language en > Content-Type text/html; charset=utf-8 > Date Tue, 26 Jun 2012 16:51:22 GMT > Expires Sun, 19 Nov 1978 05:00:00 GMT > Last-Modified Tue, 26 Jun 2012 16:50:52 GMT > Link ; rel="canonical" > ==> Server nginx > Vary Accept-Encoding,User-Agent > Via 1.1 varnish > X-Cache-Hits 1 > X-Drupal-Cache MISS > ==> X-Firefox-Spdy 1 > X-Varnish 851454988 851454961 > X-Varnish-Cache HIT > x-ua-compatible IE=Edge,chrome=1 > > Request Headers > Accept text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8 > ==> Accept-Encoding gzip, deflate > Accept-Language en,en-us;q=0.5 > Connection keep-alive > Cookie has_js=1 > DNT 1 > Host test.svr06.loc > User-Agent Mozilla/5.0 (X11; Linux x86_64; rv:13.0) Gecko/20100101 > Firefox/13.0.1 > Actually, Firefox doesn't* send the "Accept-Encoding" header over SPDY connection, and Nginx doesn't add it when proxying to backend. * Note, that Firebug isn't accurate about the headers have actually been sent. The issue is clear enough and will be fixed in the future revision. Currently, you can use something like this: map $spdy $spdy_ae { default $http_accept_encoding; 2 "gzip, deflate"; } proxy_set_header Accept-Encoding $spdy_ae; or enable compression in Nginx. wbr, Valentin V. Bartenev From pgnet.dev+nginx at gmail.com Tue Jun 26 19:43:15 2012 From: pgnet.dev+nginx at gmail.com (pgndev) Date: Tue, 26 Jun 2012 12:43:15 -0700 Subject: enabling SPDY (patch 42) +nginx 1.3.2 breaks gzip In-Reply-To: <201206262323.40090.ne@vbart.ru> References: <201206262323.40090.ne@vbart.ru> Message-ID: hi On Tue, Jun 26, 2012 at 12:23 PM, Valentin V. Bartenev wrote: > Actually, Firefox doesn't* send the "Accept-Encoding" header over SPDY I'm not clear as to that statement's implication ... Can you please clarify what's not working with Firefox+SPDY ? Do I need to change/add something to headers in FF config as well? Fwiw, I already have installed this extension: https://github.com/chengsun/moz-spdy-indicator Does that not address this issue? > and Nginx doesn't add it when proxying to backend. ok > * Note, that Firebug isn't accurate about the headers have actually been sent. That's useful to know. Do you have some more detail on this? I can take it up @firebug if it hasn't been already > The issue is clear enough and will be fixed in the future revision. Currently, > you can use something like this: > > ? ?map $spdy $spdy_ae { > ? ? ? ?default ?$http_accept_encoding; > ? ? ? ?2 ? ? ? ?"gzip, deflate"; > ? ?} > > ? ?proxy_set_header Accept-Encoding $spdy_ae; I've added that stanza to my nginx "http{..." conf. Cleared all-caches & restarted server. I'm still seeing no gzip headers: Response Headers Age 42 Cache-Control public, max-age=300 Content-Language en Content-Type text/html; charset=utf-8 Date Tue, 26 Jun 2012 19:35:27 GMT Expires Sun, 19 Nov 1978 05:00:00 GMT Last-Modified Tue, 26 Jun 2012 19:34:41 GMT Link ; rel="canonical" Server nginx Vary Accept-Encoding,User-Agent Via 1.1 varnish X-Cache-Hits 1 X-Drupal-Cache MISS X-Firefox-Spdy 1 X-Varnish 1136383481 1136383476 X-Varnish-Cache HIT x-ua-compatible IE=Edge,chrome=1 > or enable compression in Nginx. Compression is, atm, handled at my stack's Apache 24X backend. From ne at vbart.ru Tue Jun 26 23:53:13 2012 From: ne at vbart.ru (Valentin V. Bartenev) Date: Wed, 27 Jun 2012 03:53:13 +0400 Subject: enabling SPDY (patch 42) +nginx 1.3.2 breaks gzip In-Reply-To: References: <201206262323.40090.ne@vbart.ru> Message-ID: <201206270353.13265.ne@vbart.ru> On Tuesday 26 June 2012 23:43:15 pgndev wrote: > hi > > On Tue, Jun 26, 2012 at 12:23 PM, Valentin V. Bartenev wrote: > > Actually, Firefox doesn't* send the "Accept-Encoding" header over SPDY > > I'm not clear as to that statement's implication ... Can you please > clarify what's not working with Firefox+SPDY ? Firefox doesn't send "Accept-Encoding" header and according to the SPDY specification it's not needed for using gzip compression. So, there's nothing wrong with Firefox. But, since you pass requests over HTTP and compression handled by your upstream servers, thus the lack of this header results in an uncompressed response. > Do I need to change/add something to headers in FF config as well? No, you don't. > Fwiw, I already have installed this extension: > > https://github.com/chengsun/moz-spdy-indicator > > Does that not address this issue? No, it doesn't. It's just an icon in your address bar. > > and Nginx doesn't add it when proxying to backend. > > ok > > > * Note, that Firebug isn't accurate about the headers have actually been > > sent. > > That's useful to know. Do you have some more detail on this? I can > take it up @firebug if it hasn't been already As I already mentioned, Firebug shows "Accept-Encoding gzip, deflate" as a request header, when it actually hasn't been sent. But I don't think that it's a bug. Probably, it's a feature. Firebug just isn't the right tool for low level debugging of browser-server communication. Firebug shows not only real headers but also those "implied". > > The issue is clear enough and will be fixed in the future revision. > > Currently, you can use something like this: > > > > map $spdy $spdy_ae { > > default $http_accept_encoding; > > 2 "gzip, deflate"; > > } > > > > proxy_set_header Accept-Encoding $spdy_ae; > > I've added that stanza to my nginx "http{..." conf. Cleared > all-caches & restarted server. > > I'm still seeing no gzip headers: Since you added the "proxy_set_header" directive to "http" section, you should make sure that it is not overridden by the directives on other levels. http://nginx.org/r/proxy_set_header Please note: "These directives are inherited from the previous level if and only if there are no proxy_set_header directives defined on the current level." wbr, Valentin V. Bartenev From pgnet.dev+nginx at gmail.com Wed Jun 27 00:22:55 2012 From: pgnet.dev+nginx at gmail.com (pgndev) Date: Tue, 26 Jun 2012 17:22:55 -0700 Subject: enabling SPDY (patch 42) +nginx 1.3.2 breaks gzip In-Reply-To: <201206270353.13265.ne@vbart.ru> References: <201206262323.40090.ne@vbart.ru> <201206270353.13265.ne@vbart.ru> Message-ID: hi On Tue, Jun 26, 2012 at 4:53 PM, Valentin V. Bartenev wrote: > Firefox doesn't send "Accept-Encoding" header and according to the SPDY > specification it's not needed for using gzip compression. So, there's > nothing wrong with Firefox. > > But, since you pass requests over HTTP and compression handled by your upstream > servers, thus the lack of this header results in an uncompressed response. > >> Do I need to change/add something to headers in FF config as well? > > No, you don't. understood. > But I don't think that it's a bug. Probably, it's a feature. Firebug just > isn't the right tool for low level debugging of browser-server communication. I didn't realize that. I'll dig for the right tool for myself. Do you recommend anything in particular that's helpful in tracing nginx issues and reporting the results here? > Since you added the "proxy_set_header" directive to "http" section, you should > make sure that it is not overridden by the directives on other levels. > > http://nginx.org/r/proxy_set_header > Please note: "These directives are inherited from the previous level if and only > if there are no proxy_set_header directives defined on the current level." Missed that completely. Consolidating my directives straightened out the header inheritance. Now, I see: Response Headers Accept-Ranges bytes Age 14 Cache-Control public, max-age=300 ==> Content-Encoding gzip Content-Language en Content-Length 4561 Content-Type text/html; charset=utf-8 Date Wed, 27 Jun 2012 00:17:07 GMT Expires Sun, 19 Nov 1978 05:00:00 GMT Last-Modified Wed, 27 Jun 2012 00:16:48 GMT Link ; rel="canonical" ==> Server nginx Vary Accept-Encoding,User-Agent Via 1.1 varnish X-Cache-Hits 1 ==> X-Firefox-Spdy 1 X-Varnish 1691959371 1691959345 X-Varnish-Cache HIT x-ua-compatible IE=Edge,chrome=1 which, IIUC, appears to be correct. thanks! From ne at vbart.ru Wed Jun 27 12:42:46 2012 From: ne at vbart.ru (Valentin V. Bartenev) Date: Wed, 27 Jun 2012 16:42:46 +0400 Subject: enabling SPDY (patch 42) +nginx 1.3.2 breaks gzip In-Reply-To: References: <201206270353.13265.ne@vbart.ru> Message-ID: <201206271642.46713.ne@vbart.ru> On Wednesday 27 June 2012 04:22:55 pgndev wrote: [...] > > But I don't think that it's a bug. Probably, it's a feature. Firebug just > > isn't the right tool for low level debugging of browser-server > > communication. > > I didn't realize that. I'll dig for the right tool for myself. Do > you recommend anything in particular that's helpful in tracing nginx > issues and reporting the results here? > One of the best tool for tracing nginx issues is its debug log, see guide here how to enable it: http://nginx.org/en/docs/debugging_log.html It's also pretty useful tool for identifying problems in the config and for understanding how the things works. For debugging connections you may use low level tools like tcpdump/tcpflow, or wireshark. Wireshark is also good for SSL/TLS. wbr, Valentin V. Bartenev From gert.cuykens at gmail.com Wed Jun 27 16:34:46 2012 From: gert.cuykens at gmail.com (Gert Cuykens) Date: Wed, 27 Jun 2012 18:34:46 +0200 Subject: ngx_http_auth_basic_module.c Message-ID: Can Auth PAM module be merged into ngx_http_auth_basic_module.c pleas Seems Auth PAM is not active anymore and will result into security risks and eventually into a broken module. http://web.iti.upv.es/~sto/nginx/ http://pastebin.com/EZuY1TKM If merging is not a option can you take a look at the source code and tell me if it is possible to make it a extension of ngx_http_auth_basic_module.c So that future updates to ngx_http_auth_basic_module.c will also apply to the PAM module? From magicbearmo at gmail.com Wed Jun 27 21:13:09 2012 From: magicbearmo at gmail.com (MagicBear) Date: Thu, 28 Jun 2012 05:13:09 +0800 Subject: Patch for ngx_purge_cache on nginx-1.3.2 not working Message-ID: --- a/ngx_cache_purge_module.c 2012-06-28 05:08:44.000000000 +0800 +++ b/ngx_cache_purge_module.c 2011-12-20 20:36:20.000000000 +0800 @@ -41,7 +41,7 @@ char *ngx_http_fastcgi_cache_purge ngx_int_t ngx_http_fastcgi_cache_purge_handler(ngx_http_request_t *r); # endif /* NGX_HTTP_FASTCGI */ -# if (NGX_HTTP_PROXY || nginx_version >= 1003002) +# if (NGX_HTTP_PROXY) char *ngx_http_proxy_cache_purge_conf(ngx_conf_t *cf, ngx_command_t *cmd, void *conf); ngx_int_t ngx_http_proxy_cache_purge_handler(ngx_http_request_t *r); @@ -76,7 +76,7 @@ static ngx_command_t ngx_http_cache_pur NULL }, # endif /* NGX_HTTP_FASTCGI */ -# if (NGX_HTTP_PROXY || nginx_version >= 1003002) +# if (NGX_HTTP_PROXY) { ngx_string("proxy_cache_purge"), NGX_HTTP_LOC_CONF|NGX_CONF_TAKE2, ngx_http_proxy_cache_purge_conf, @@ -257,7 +257,7 @@ ngx_http_fastcgi_cache_purge_handler(ngx } # endif /* NGX_HTTP_FASTCGI */ -# if (NGX_HTTP_PROXY || nginx_version >= 1003002) +# if (NGX_HTTP_PROXY) extern ngx_module_t ngx_http_proxy_module; typedef struct { -- MagicBear From magicbearmo at gmail.com Wed Jun 27 21:18:39 2012 From: magicbearmo at gmail.com (MagicBear) Date: Thu, 28 Jun 2012 05:18:39 +0800 Subject: Patch for ngx_purge_cache on nginx-1.3.2 not working In-Reply-To: References: Message-ID: I am sorry, i make patch file opposite, this is the corrent version. --- a/ngx_cache_purge_module.c 2012-06-28 05:08:44.000000000 +0800 +++ b/ngx_cache_purge_module.c 2011-12-20 20:36:20.000000000 +0800 @@ -41,7 +41,7 @@ char *ngx_http_fastcgi_cache_purge ngx_int_t ngx_http_fastcgi_cache_purge_handler(ngx_http_request_t *r); # endif /* NGX_HTTP_FASTCGI */ -# if (NGX_HTTP_PROXY) +# if (NGX_HTTP_PROXY || nginx_version >= 1003002) char *ngx_http_proxy_cache_purge_conf(ngx_conf_t *cf, ngx_command_t *cmd, void *conf); ngx_int_t ngx_http_proxy_cache_purge_handler(ngx_http_request_t *r); @@ -76,7 +76,7 @@ static ngx_command_t ngx_http_cache_pur NULL }, # endif /* NGX_HTTP_FASTCGI */ -# if (NGX_HTTP_PROXY) +# if (NGX_HTTP_PROXY || nginx_version >= 1003002) { ngx_string("proxy_cache_purge"), NGX_HTTP_LOC_CONF|NGX_CONF_TAKE2, ngx_http_proxy_cache_purge_conf, @@ -257,7 +257,7 @@ ngx_http_fastcgi_cache_purge_handler(ngx } # endif /* NGX_HTTP_FASTCGI */ -# if (NGX_HTTP_PROXY) +# if (NGX_HTTP_PROXY || nginx_version >= 1003002) extern ngx_module_t ngx_http_proxy_module; typedef struct { 2012/6/28 MagicBear : > --- a/ngx_cache_purge_module.c ?2012-06-28 05:08:44.000000000 +0800 > +++ b/ngx_cache_purge_module.c ?2011-12-20 20:36:20.000000000 +0800 > @@ -41,7 +41,7 @@ char ? ? ? *ngx_http_fastcgi_cache_purge > ?ngx_int_t ? ngx_http_fastcgi_cache_purge_handler(ngx_http_request_t *r); > ?# endif /* NGX_HTTP_FASTCGI */ > > -# if (NGX_HTTP_PROXY || nginx_version >= 1003002) > +# if (NGX_HTTP_PROXY) > ?char ? ? ? *ngx_http_proxy_cache_purge_conf(ngx_conf_t *cf, > ? ? ? ? ? ? ? ? ngx_command_t *cmd, void *conf); > ?ngx_int_t ? ngx_http_proxy_cache_purge_handler(ngx_http_request_t *r); > @@ -76,7 +76,7 @@ static ngx_command_t ?ngx_http_cache_pur > ? ? ? NULL }, > ?# endif /* NGX_HTTP_FASTCGI */ > > -# if (NGX_HTTP_PROXY || nginx_version >= 1003002) > +# if (NGX_HTTP_PROXY) > ? ? { ngx_string("proxy_cache_purge"), > ? ? ? NGX_HTTP_LOC_CONF|NGX_CONF_TAKE2, > ? ? ? ngx_http_proxy_cache_purge_conf, > @@ -257,7 +257,7 @@ ngx_http_fastcgi_cache_purge_handler(ngx > ?} > ?# endif /* NGX_HTTP_FASTCGI */ > > -# if (NGX_HTTP_PROXY || nginx_version >= 1003002) > +# if (NGX_HTTP_PROXY) > ?extern ngx_module_t ?ngx_http_proxy_module; > > ?typedef struct { > > > -- > MagicBear -- MagicBear From iamzhanghu at gmail.com Thu Jun 28 02:13:20 2012 From: iamzhanghu at gmail.com (=?GB2312?B?1cUgu6I=?=) Date: Thu, 28 Jun 2012 10:13:20 +0800 Subject: Ningx Module: IspIP Message-ID: <19864030-F664-40C7-8E2E-7241FED8283C@gmail.com> HI All, I'm announcing my frist Nginx module: IspIP. It's my replacement of GeoIP module. I developed IspIP module because I want to customize the IP database, but GeoIP is using a closed format IP data file. IspIP API library git clone git://vps3.yun09.com/ispip.git Nginx IspIP module git clone git://vps3.yun09.com/nginx-ispip.git Any comments are appreciated. Tiger From ru at nginx.com Thu Jun 28 10:36:57 2012 From: ru at nginx.com (Ruslan Ermilov) Date: Thu, 28 Jun 2012 14:36:57 +0400 Subject: nginx.org documents in simplified Chinese In-Reply-To: References: <20120530125746.GX31671@mdounin.ru> Message-ID: <20120628103657.GB47269@lo0.su> Hi there, On Thu, May 31, 2012 at 07:54:02PM +0800, chen cw wrote: > I come here with the patch for [1]nginx.org documents in simplified > Chinese, and I'm glad that someone will give it a review. This work is > done by the whole Server Platforms Team at Taobao.com. I've committed your translation, thanks! The only changes I made were: - removed translated version of security_advisories (we only maintain a single version of it in English); - put English and Russian at the top of the language list as these are the only two officially supported translations; - made GNUmakefile diff'able with other languages makefiles. Don't hesitate to send us more patches and keep them updated! From ru at nginx.com Thu Jun 28 13:44:57 2012 From: ru at nginx.com (Ruslan Ermilov) Date: Thu, 28 Jun 2012 17:44:57 +0400 Subject: Patch for ngx_purge_cache on nginx-1.3.2 not working In-Reply-To: References: Message-ID: <20120628134457.GA53777@lo0.su> On Thu, Jun 28, 2012 at 05:18:39AM +0800, MagicBear wrote: > I am sorry, i make patch file opposite, this is the corrent version. > > --- a/ngx_cache_purge_module.c 2012-06-28 05:08:44.000000000 +0800 > +++ b/ngx_cache_purge_module.c 2011-12-20 20:36:20.000000000 +0800 > @@ -41,7 +41,7 @@ char *ngx_http_fastcgi_cache_purge > ngx_int_t ngx_http_fastcgi_cache_purge_handler(ngx_http_request_t *r); > # endif /* NGX_HTTP_FASTCGI */ > > -# if (NGX_HTTP_PROXY) > +# if (NGX_HTTP_PROXY || nginx_version >= 1003002) > char *ngx_http_proxy_cache_purge_conf(ngx_conf_t *cf, > ngx_command_t *cmd, void *conf); > ngx_int_t ngx_http_proxy_cache_purge_handler(ngx_http_request_t *r); > @@ -76,7 +76,7 @@ static ngx_command_t ngx_http_cache_pur > NULL }, > # endif /* NGX_HTTP_FASTCGI */ > > -# if (NGX_HTTP_PROXY) > +# if (NGX_HTTP_PROXY || nginx_version >= 1003002) > { ngx_string("proxy_cache_purge"), > NGX_HTTP_LOC_CONF|NGX_CONF_TAKE2, > ngx_http_proxy_cache_purge_conf, > @@ -257,7 +257,7 @@ ngx_http_fastcgi_cache_purge_handler(ngx > } > # endif /* NGX_HTTP_FASTCGI */ > > -# if (NGX_HTTP_PROXY) > +# if (NGX_HTTP_PROXY || nginx_version >= 1003002) > extern ngx_module_t ngx_http_proxy_module; > > typedef struct { This is a wrong patch. As of 1.3.2, nginx no longer exposes the NGX_HTTP_PROXY macro, and it never exposed any of the macros NGX_HTTP_{FASTCGI,SCGI,UWSGI} that this module is trying to use to detect if the modules were configured to be built. The below patch should be better compatible with both old and changed versions of nginx: %%% --- ngx_cache_purge-1.5/config 2011-12-20 16:36:20.000000000 +0400 +++ ngx_cache_purge-1.5/config 2012-06-28 17:33:42.000000000 +0400 @@ -1,3 +1,7 @@ +if [ "$HTTP_PROXY" = "YES" ]; then + have=NGX_HTTP_PROXY . auto/have +fi + if [ "$HTTP_FASTCGI" = "YES" ]; then have=NGX_HTTP_FASTCGI . auto/have fi %%% From ru at nginx.com Fri Jun 29 11:03:02 2012 From: ru at nginx.com (ru at nginx.com) Date: Fri, 29 Jun 2012 11:03:02 +0000 Subject: [nginx] svn commit: r4710 - in trunk/src: core http/modules/perl Message-ID: <20120629110302.7F71A3F9F74@mail.nginx.com> Author: ru Date: 2012-06-29 11:03:01 +0000 (Fri, 29 Jun 2012) New Revision: 4710 URL: http://trac.nginx.org/nginx/changeset/4710/nginx Log: Version bump. Modified: trunk/src/core/nginx.h trunk/src/http/modules/perl/nginx.pm Modified: trunk/src/core/nginx.h =================================================================== --- trunk/src/core/nginx.h 2012-06-26 14:40:23 UTC (rev 4709) +++ trunk/src/core/nginx.h 2012-06-29 11:03:01 UTC (rev 4710) @@ -9,8 +9,8 @@ #define _NGINX_H_INCLUDED_ -#define nginx_version 1003002 -#define NGINX_VERSION "1.3.2" +#define nginx_version 1003003 +#define NGINX_VERSION "1.3.3" #define NGINX_VER "nginx/" NGINX_VERSION #define NGINX_VAR "NGINX" Modified: trunk/src/http/modules/perl/nginx.pm =================================================================== --- trunk/src/http/modules/perl/nginx.pm 2012-06-26 14:40:23 UTC (rev 4709) +++ trunk/src/http/modules/perl/nginx.pm 2012-06-29 11:03:01 UTC (rev 4710) @@ -50,7 +50,7 @@ HTTP_INSUFFICIENT_STORAGE ); -our $VERSION = '1.3.2'; +our $VERSION = '1.3.3'; require XSLoader; XSLoader::load('nginx', $VERSION); From zhuzhaoyuan at gmail.com Fri Jun 29 15:20:39 2012 From: zhuzhaoyuan at gmail.com (Joshua Zhu) Date: Fri, 29 Jun 2012 23:20:39 +0800 Subject: nginx.org documents in simplified Chinese In-Reply-To: <20120628103657.GB47269@lo0.su> References: <20120530125746.GX31671@mdounin.ru> <20120628103657.GB47269@lo0.su> Message-ID: Hi, On Thu, Jun 28, 2012 at 6:36 PM, Ruslan Ermilov wrote: > Hi there, > > On Thu, May 31, 2012 at 07:54:02PM +0800, chen cw wrote: > > I come here with the patch for [1]nginx.org documents in simplified > > Chinese, and I'm glad that someone will give it a review. This work is > > done by the whole Server Platforms Team at Taobao.com. > > I've committed your translation, thanks! > > The only changes I made were: > > - removed translated version of security_advisories (we only > maintain a single version of it in English); > > - put English and Russian at the top of the language list as > these are the only two officially supported translations; > > - made GNUmakefile diff'able with other languages makefiles. > > Don't hesitate to send us more patches and keep them updated! Thanks a lot! We'll keep on working on it. Regards, -- Joshua Zhu Senior Software Engineer Server Platforms Team at Taobao -------------- next part -------------- An HTML attachment was scrubbed... URL: From mdounin at mdounin.ru Fri Jun 29 17:28:41 2012 From: mdounin at mdounin.ru (mdounin at mdounin.ru) Date: Fri, 29 Jun 2012 17:28:41 +0000 Subject: [nginx] svn commit: r4711 - in branches/stable-1.2: . src/http src/http/modules Message-ID: <20120629172841.BF8703F9F46@mail.nginx.com> Author: mdounin Date: 2012-06-29 17:28:41 +0000 (Fri, 29 Jun 2012) New Revision: 4711 URL: http://trac.nginx.org/nginx/changeset/4711/nginx Log: Merge of r4636, r4637, r4638: config sanity checks. *) Added syntax checking of the second parameter of the "split_clients" directive. *) Capped the status code that may be returned with "return" and "try_files". *) Zero padded the returned and logged HTTP status code, and fixed possible buffer overrun in $status handling. Modified: branches/stable-1.2/ branches/stable-1.2/src/http/modules/ngx_http_log_module.c branches/stable-1.2/src/http/modules/ngx_http_rewrite_module.c branches/stable-1.2/src/http/modules/ngx_http_split_clients_module.c branches/stable-1.2/src/http/ngx_http_core_module.c branches/stable-1.2/src/http/ngx_http_header_filter_module.c Index: branches/stable-1.2 =================================================================== --- branches/stable-1.2 2012-06-29 11:03:01 UTC (rev 4710) +++ branches/stable-1.2 2012-06-29 17:28:41 UTC (rev 4711) Property changes on: branches/stable-1.2 ___________________________________________________________________ Modified: svn:mergeinfo ## -1 +1 ## -/trunk:4611-4632,4641,4645,4674-4676 +/trunk:4611-4632,4636-4638,4641,4645,4674-4676 \ No newline at end of property Modified: branches/stable-1.2/src/http/modules/ngx_http_log_module.c =================================================================== --- branches/stable-1.2/src/http/modules/ngx_http_log_module.c 2012-06-29 11:03:01 UTC (rev 4710) +++ branches/stable-1.2/src/http/modules/ngx_http_log_module.c 2012-06-29 17:28:41 UTC (rev 4711) @@ -205,7 +205,7 @@ { ngx_string("msec"), NGX_TIME_T_LEN + 4, ngx_http_log_msec }, { ngx_string("request_time"), NGX_TIME_T_LEN + 4, ngx_http_log_request_time }, - { ngx_string("status"), 3, ngx_http_log_status }, + { ngx_string("status"), NGX_INT_T_LEN, ngx_http_log_status }, { ngx_string("bytes_sent"), NGX_OFF_T_LEN, ngx_http_log_bytes_sent }, { ngx_string("body_bytes_sent"), NGX_OFF_T_LEN, ngx_http_log_body_bytes_sent }, @@ -593,7 +593,7 @@ status = 0; } - return ngx_sprintf(buf, "%ui", status); + return ngx_sprintf(buf, "%03ui", status); } Modified: branches/stable-1.2/src/http/modules/ngx_http_rewrite_module.c =================================================================== --- branches/stable-1.2/src/http/modules/ngx_http_rewrite_module.c 2012-06-29 11:03:01 UTC (rev 4710) +++ branches/stable-1.2/src/http/modules/ngx_http_rewrite_module.c 2012-06-29 17:28:41 UTC (rev 4711) @@ -485,6 +485,12 @@ } else { + if (ret->status > 999) { + ngx_conf_log_error(NGX_LOG_EMERG, cf, 0, + "invalid return code \"%V\"", &value[1]); + return NGX_CONF_ERROR; + } + if (cf->args->nelts == 2) { return NGX_CONF_OK; } Modified: branches/stable-1.2/src/http/modules/ngx_http_split_clients_module.c =================================================================== --- branches/stable-1.2/src/http/modules/ngx_http_split_clients_module.c 2012-06-29 11:03:01 UTC (rev 4710) +++ branches/stable-1.2/src/http/modules/ngx_http_split_clients_module.c 2012-06-29 17:28:41 UTC (rev 4711) @@ -138,6 +138,13 @@ } name = value[2]; + + if (name.len < 2 || name.data[0] != '$') { + ngx_conf_log_error(NGX_LOG_EMERG, cf, 0, + "invalid variable name \"%V\"", &name); + return NGX_CONF_ERROR; + } + name.len--; name.data++; Modified: branches/stable-1.2/src/http/ngx_http_core_module.c =================================================================== --- branches/stable-1.2/src/http/ngx_http_core_module.c 2012-06-29 11:03:01 UTC (rev 4710) +++ branches/stable-1.2/src/http/ngx_http_core_module.c 2012-06-29 17:28:41 UTC (rev 4711) @@ -4662,7 +4662,7 @@ code = ngx_atoi(tf[i - 1].name.data + 1, tf[i - 1].name.len - 2); - if (code == NGX_ERROR) { + if (code == NGX_ERROR || code > 999) { ngx_conf_log_error(NGX_LOG_EMERG, cf, 0, "invalid code \"%*s\"", tf[i - 1].name.len - 1, tf[i - 1].name.data); Modified: branches/stable-1.2/src/http/ngx_http_header_filter_module.c =================================================================== --- branches/stable-1.2/src/http/ngx_http_header_filter_module.c 2012-06-29 11:03:01 UTC (rev 4710) +++ branches/stable-1.2/src/http/ngx_http_header_filter_module.c 2012-06-29 17:28:41 UTC (rev 4711) @@ -445,7 +445,7 @@ b->last = ngx_copy(b->last, status_line->data, status_line->len); } else { - b->last = ngx_sprintf(b->last, "%ui", status); + b->last = ngx_sprintf(b->last, "%03ui", status); } *b->last++ = CR; *b->last++ = LF; From mdounin at mdounin.ru Fri Jun 29 17:32:44 2012 From: mdounin at mdounin.ru (mdounin at mdounin.ru) Date: Fri, 29 Jun 2012 17:32:44 +0000 Subject: [nginx] svn commit: r4712 - in branches/stable-1.2: . auto src/core Message-ID: <20120629173244.694953F9F47@mail.nginx.com> Author: mdounin Date: 2012-06-29 17:32:43 +0000 (Fri, 29 Jun 2012) New Revision: 4712 URL: http://trac.nginx.org/nginx/changeset/4712/nginx Log: Merge of r4639, r4640: C++ fixes. Fixed the ngx_regex.h header file compatibility with C++. Fixed building --with-cpp_test_module on some systems. Modified: branches/stable-1.2/ branches/stable-1.2/auto/modules branches/stable-1.2/src/core/ngx_regex.c branches/stable-1.2/src/core/ngx_regex.h Index: branches/stable-1.2 =================================================================== --- branches/stable-1.2 2012-06-29 17:28:41 UTC (rev 4711) +++ branches/stable-1.2 2012-06-29 17:32:43 UTC (rev 4712) Property changes on: branches/stable-1.2 ___________________________________________________________________ Modified: svn:mergeinfo ## -1 +1 ## -/trunk:4611-4632,4636-4638,4641,4645,4674-4676 +/trunk:4611-4632,4636-4641,4645,4674-4676 \ No newline at end of property Modified: branches/stable-1.2/auto/modules =================================================================== --- branches/stable-1.2/auto/modules 2012-06-29 17:28:41 UTC (rev 4711) +++ branches/stable-1.2/auto/modules 2012-06-29 17:32:43 UTC (rev 4712) @@ -458,6 +458,7 @@ if [ $NGX_CPP_TEST = YES ]; then NGX_MISC_SRCS="$NGX_MISC_SRCS $NGX_CPP_TEST_SRCS" + CORE_LIBS="$CORE_LIBS -lstdc++" fi Modified: branches/stable-1.2/src/core/ngx_regex.c =================================================================== --- branches/stable-1.2/src/core/ngx_regex.c 2012-06-29 17:28:41 UTC (rev 4711) +++ branches/stable-1.2/src/core/ngx_regex.c 2012-06-29 17:32:43 UTC (rev 4712) @@ -152,7 +152,7 @@ return NGX_ERROR; } - rc->regex->pcre = re; + rc->regex->code = re; /* do not study at runtime */ @@ -367,7 +367,7 @@ i = 0; } - elts[i].regex->extra = pcre_study(elts[i].regex->pcre, opt, &errstr); + elts[i].regex->extra = pcre_study(elts[i].regex->code, opt, &errstr); if (errstr != NULL) { ngx_log_error(NGX_LOG_ALERT, cycle->log, 0, @@ -380,7 +380,7 @@ int jit, n; jit = 0; - n = pcre_fullinfo(elts[i].regex->pcre, elts[i].regex->extra, + n = pcre_fullinfo(elts[i].regex->code, elts[i].regex->extra, PCRE_INFO_JIT, &jit); if (n != 0 || jit != 1) { Modified: branches/stable-1.2/src/core/ngx_regex.h =================================================================== --- branches/stable-1.2/src/core/ngx_regex.h 2012-06-29 17:28:41 UTC (rev 4711) +++ branches/stable-1.2/src/core/ngx_regex.h 2012-06-29 17:32:43 UTC (rev 4712) @@ -21,7 +21,7 @@ typedef struct { - pcre *pcre; + pcre *code; pcre_extra *extra; } ngx_regex_t; @@ -50,7 +50,7 @@ ngx_int_t ngx_regex_compile(ngx_regex_compile_t *rc); #define ngx_regex_exec(re, s, captures, size) \ - pcre_exec(re->pcre, re->extra, (const char *) (s)->data, (s)->len, 0, 0, \ + pcre_exec(re->code, re->extra, (const char *) (s)->data, (s)->len, 0, 0, \ captures, size) #define ngx_regex_exec_n "pcre_exec()" From ru at nginx.com Fri Jun 29 20:33:26 2012 From: ru at nginx.com (ru at nginx.com) Date: Fri, 29 Jun 2012 20:33:26 +0000 Subject: [nginx] svn commit: r4713 - trunk/src/http/modules Message-ID: <20120629203327.042163FA0A0@mail.nginx.com> Author: ru Date: 2012-06-29 20:33:26 +0000 (Fri, 29 Jun 2012) New Revision: 4713 URL: http://trac.nginx.org/nginx/changeset/4713/nginx Log: map: strip final dot before looking up in a map of hostnames. (closes #182) Modified: trunk/src/http/modules/ngx_http_map_module.c Modified: trunk/src/http/modules/ngx_http_map_module.c =================================================================== --- trunk/src/http/modules/ngx_http_map_module.c 2012-06-29 17:32:43 UTC (rev 4712) +++ trunk/src/http/modules/ngx_http_map_module.c 2012-06-29 20:33:26 UTC (rev 4713) @@ -110,7 +110,6 @@ { ngx_http_map_ctx_t *map = (ngx_http_map_ctx_t *) data; - size_t len; ngx_str_t val; ngx_http_variable_value_t *value; @@ -121,10 +120,8 @@ return NGX_ERROR; } - len = val.len; - - if (len && map->hostnames && val.data[len - 1] == '.') { - len--; + if (map->hostnames && val.len > 0 && val.data[val.len - 1] == '.') { + val.len--; } value = ngx_http_map_find(r, &map->map, &val); @@ -281,6 +278,8 @@ map->default_value = ctx.default_value ? ctx.default_value: &ngx_http_variable_null_value; + map->hostnames = ctx.hostnames; + hash.key = ngx_hash_key_lc; hash.max_size = mcf->hash_max_size; hash.bucket_size = mcf->hash_bucket_size; From naquad at gmail.com Sat Jun 30 00:25:51 2012 From: naquad at gmail.com (naquad) Date: Sat, 30 Jun 2012 03:25:51 +0300 Subject: nginx memory management algorithm? Message-ID: Hi. I'm trying to figure out how does nginx manage its memory per request. From what I understand it uses region-based memory management (implemented in src/core/ngx_palloc.{h,c}). But is there any documentation on details of its implementation and algorithm itself? I've tried to just read through sources, but I can't get figure algorithm out of them :( Could somebody please sched some light on this? Thank you. -------------- next part -------------- An HTML attachment was scrubbed... URL: From vshebordaev at mail.ru Sat Jun 30 04:32:04 2012 From: vshebordaev at mail.ru (Vladimir Shebordaev) Date: Sat, 30 Jun 2012 08:32:04 +0400 Subject: nginx memory management algorithm? In-Reply-To: References: Message-ID: Hi! 2012/6/30 naquad : > > I'm trying to figure out how does nginx manage its memory per request. From > what I understand it uses region-based memory management (implemented in > src/core/ngx_palloc.{h,c}). > But is there any documentation on details of its implementation and > algorithm itself? Basically, the pool memory allocator in nginx is not that complicated, so I guess there is no dedicated document. > I've tried to just read through sources, but I can't get figure algorithm > out of them :( > Could somebody please sched some light on this? > As per core/ngx_palloc.h, the pool is nothing but a list of memory chunks named as blocks described by embedded ngx_pool_data_t . Regular allocations get the memory from the current chunk. When the requested amount of memory does fit neither into current chunk nor into other partially full chunks but still has a modest size - the one specified at pool creation time either the system page size, whichever is smaller, - new memory block is requested from the system, put on the list of chunks and installed as the current chunk for regular allocations. When the amount of memory is requested that is larger than p->max, i.e. the one specified at pool creation time either the system page size, whichever is smaller, it is allocated directly from the system with ngx_alloc and put on p->large list. Please notice, the pool allocator in nginx is specifically designed for speed and efficiency, it doesn't store the information about particular allocations neither allocated objects bear such information. So, the only mean to reclaim memory is to destroy the pool. When you allocate from pool the objects having accotiated system resources that are to be released along with the object itself, e.g. file buffers, you can install a cleanup handler that is to be invoked at pool destruction time. > Thank you. > In the hope it helps. Regards, Vladimir From naquad at gmail.com Sat Jun 30 16:00:21 2012 From: naquad at gmail.com (naquad) Date: Sat, 30 Jun 2012 19:00:21 +0300 Subject: nginx memory management algorithm? In-Reply-To: References: Message-ID: Thank you, Vladimir. You've made things clear. Now I've got it. 2012/6/30 Vladimir Shebordaev > Hi! > > 2012/6/30 naquad : > > > > I'm trying to figure out how does nginx manage its memory per request. > From > > what I understand it uses region-based memory management (implemented in > > src/core/ngx_palloc.{h,c}). > > But is there any documentation on details of its implementation and > > algorithm itself? > > Basically, the pool memory allocator in nginx is not that complicated, > so I guess there is no dedicated document. > > > I've tried to just read through sources, but I can't get figure algorithm > > out of them :( > > Could somebody please sched some light on this? > > > > As per core/ngx_palloc.h, the pool is nothing but a list of memory > chunks named as blocks described by embedded ngx_pool_data_t . > > Regular allocations get the memory from the current chunk. > > When the requested amount of memory does fit neither into current > chunk nor into other partially full chunks but still has a modest size > - the one specified at pool creation time either the system page size, > whichever is smaller, - new memory block is requested from the > system, put on the list of chunks and installed as the current chunk > for regular allocations. > > When the amount of memory is requested that is larger than p->max, > i.e. the one specified at pool creation time either the system page > size, whichever is smaller, it is allocated directly from the system > with ngx_alloc and put on p->large list. > > Please notice, the pool allocator in nginx is specifically designed > for speed and efficiency, it doesn't store the information about > particular allocations neither allocated objects bear such > information. So, the only mean to reclaim memory is to destroy the > pool. > > When you allocate from pool the objects having accotiated system > resources that are to be released along with the object itself, e.g. > file buffers, you can install a cleanup handler that is to be invoked > at pool destruction time. > > > Thank you. > > > > In the hope it helps. > > Regards, > Vladimir > > _______________________________________________ > nginx-devel mailing list > nginx-devel at nginx.org > http://mailman.nginx.org/mailman/listinfo/nginx-devel > -------------- next part -------------- An HTML attachment was scrubbed... URL: