From mdounin at mdounin.ru Sat Sep 1 09:24:55 2012 From: mdounin at mdounin.ru (Maxim Dounin) Date: Sat, 1 Sep 2012 13:24:55 +0400 Subject: a bug in limit_request module In-Reply-To: <201208221855.04686.ne@vbart.ru> References: <201208221855.04686.ne@vbart.ru> Message-ID: <20120901092455.GI40452@mdounin.ru> Hello! On Wed, Aug 22, 2012 at 06:55:04PM +0400, Valentin V. Bartenev wrote: > On Wednesday 22 August 2012 17:49:52 chen cw wrote: > > Hi, > > The red-black tree used in limit_request module has two level of > > keys, the top is hash, and the next is the value string itself. However, > > when inserting a new node into the tree, only hash is set, the value string > > is left empty, [...] > > > > Nice catch. Thank you. > > wbr, Valentin V. Bartenev > > > Index: src/http/modules/ngx_http_limit_req_module.c > =================================================================== > --- src/http/modules/ngx_http_limit_req_module.c (revision 4826) > +++ src/http/modules/ngx_http_limit_req_module.c (working copy) > @@ -444,17 +444,17 @@ ngx_http_limit_req_lookup(ngx_http_limit_req_limit > > node->key = hash; > > - ngx_rbtree_insert(&ctx->sh->rbtree, node); > - > lr = (ngx_http_limit_req_node_t *) &node->color; > > - ngx_queue_insert_head(&ctx->sh->queue, &lr->queue); > - > lr->len = (u_char) len; > lr->excess = 0; > > ngx_memcpy(lr->data, data, len); > > + ngx_rbtree_insert(&ctx->sh->rbtree, node); > + > + ngx_queue_insert_head(&ctx->sh->queue, &lr->queue); > + > if (account) { > lr->last = now; > lr->count = 0; Looks fine, please commit. Maxim Dounin From mdounin at mdounin.ru Sat Sep 1 10:50:56 2012 From: mdounin at mdounin.ru (Maxim Dounin) Date: Sat, 1 Sep 2012 14:50:56 +0400 Subject: problem with nginx round robin upstream In-Reply-To: References: Message-ID: <20120901105056.GL40452@mdounin.ru> Hello! On Thu, Aug 30, 2012 at 10:19:11AM +0300, Tsahi Asher wrote: > I have a problem with the upstream round robin mechanism. > in ngx_http_upstream.c ngx_http_upstream_server function the hostname that > is added in the configuration file under upstream directive is added to the > upstream server list. > the function uses ngx_parse_url which in turn uses ngx_parse_inet_url that > calls ngx_inet_resolve_host. > ngx_inet_resolve_host tries to resolve the host name from the configuration > file by calling gethostbyname. for each url gethostbyname return the list > of ip's that are associated with this url and these ip's are added to the > list of upstream servers. > This results in a situation that if there are three servers defined in an > upstream directive, each with an equal weight, but one of those server is > associated with two ip's (for example internal and an external ip) then the > round robin weight will not be equal for each server. The server which has > two ip's will get a double weight. Yes, this is how it works. Hostname which resolves to two ip addresses effectively defines two upstream servers with identical settings. This is even explicitly documented, see http://nginx.org/en/docs/http/ngx_http_upstream_module.html#server: : ... A domain name that resolves to several IP addresses : essentially defines multiple servers. It's understood that this isn't exactly in line with historical meaning of multiple ip addresses as addresses of the same multihomed host in multiple networks. It's much more common now to use multiple ip addresses for the same name to mean multiple servers in a group though, and the handling in the upstream module matches this usage pattern. > i can add a log if required or if my explanation is not good enough. > i saw that a url has a flag called one_addr that if is set to true will get > resolved to a single ip only, but this flag is not used. This flag is a minor optimization used when calling code isn't capable of handling multiple ip addresses. > adding the host ip address is one way to solve this, is there another way? You have to either use ip address or a name which resolves to exactly one ip address (if you want nginx to treat it as single server). Maxim Dounin From mdounin at mdounin.ru Sat Sep 1 10:55:42 2012 From: mdounin at mdounin.ru (Maxim Dounin) Date: Sat, 1 Sep 2012 14:55:42 +0400 Subject: problem with nginx round robin upstream In-Reply-To: <20120830092934.GA75395@lo0.su> References: <20120830092934.GA75395@lo0.su> Message-ID: <20120901105541.GM40452@mdounin.ru> Hello! On Thu, Aug 30, 2012 at 01:29:34PM +0400, Ruslan Ermilov wrote: > On Thu, Aug 30, 2012 at 10:19:11AM +0300, Tsahi Asher wrote: > > Hi > > > > I have a problem with the upstream round robin mechanism. > > in ngx_http_upstream.c ngx_http_upstream_server function the hostname that > > is added in the configuration file under upstream directive is added to > > the upstream server list. > > the function uses ngx_parse_url which in turn uses ngx_parse_inet_url that > > calls ngx_inet_resolve_host. > > ngx_inet_resolve_host tries to resolve the host name from the > > configuration file by calling gethostbyname. for each url gethostbyname > > return the list of ip's that are associated with this url and these ip's > > are added to the list of upstream servers. > > This results in a situation that if there are three servers defined in an > > upstream directive, each with an equal weight, but one of those server is > > associated with two ip's (for example internal and an external ip) then > > the round robin weight will not be equal for each server. The server which > > has two ip's will get a double weight. > > i can add a log if required or if my explanation is not good enough. > > i saw that a url has a flag called one_addr that if is set to true will > > get resolved to a single ip only, but this flag is not used. > > > > adding the host ip address is one way to solve this, is there another way? > > This is expected. Here's the relevant quote from documentation [1]: > > : A domain name that resolves to several IP addresses essentially defines > : multiple servers. > > [1] http://nginx.org/en/docs/http/ngx_http_upstream_module.html#server Oops, sorry, missed your reply. In any case I hope mine wasn't completely redundant and provides some rationale for the behaviour in question. :) Maxim Dounin From gevik at xs4all.nl Sat Sep 1 12:42:14 2012 From: gevik at xs4all.nl (Gevik Babakhani) Date: Sat, 01 Sep 2012 14:42:14 +0200 Subject: nginx standalone, is it possible? Message-ID: <504202A6.1070006@xs4all.nl> Hi, I would like to know if it is possible to compile and distribute a standalone version of nginx. I have a web application, which normally only runs on the localhost and is not accessible from outside. This application is used to create files and folders in a given folder. I want the user to start a standalone server, let's say on port 8080 and be able to work with the app. The main reason for this is to be able to distribute a per-built version on nginx (with PHP and database support), without the need to ask the users to compile and configure the server themselves. The "standalone" server needs to run under the users account to be able to access and create files/folders. Any suggestions is very much appreciated. Thanks From vbart at nginx.com Mon Sep 3 12:55:51 2012 From: vbart at nginx.com (vbart at nginx.com) Date: Mon, 3 Sep 2012 12:55:51 +0000 Subject: [nginx] svn commit: r4833 - trunk/src/http/modules Message-ID: <20120903125551.1FB693F9E2A@mail.nginx.com> Author: vbart Date: 2012-09-03 12:55:50 +0000 (Mon, 03 Sep 2012) New Revision: 4833 URL: http://trac.nginx.org/nginx/changeset/4833/nginx Log: Limit req: fix of rbtree node insertion on hash collisions. The rbtree used in ngx_http_limit_req_module has two level of keys, the top is hash, and the next is the value string itself. However, when inserting a new node, only hash has been set, while the value string has been left empty. The bug was introduced in r4419 (1.1.14). Found by Charles Chen. Modified: trunk/src/http/modules/ngx_http_limit_req_module.c Modified: trunk/src/http/modules/ngx_http_limit_req_module.c =================================================================== --- trunk/src/http/modules/ngx_http_limit_req_module.c 2012-08-30 16:08:13 UTC (rev 4832) +++ trunk/src/http/modules/ngx_http_limit_req_module.c 2012-09-03 12:55:50 UTC (rev 4833) @@ -444,17 +444,17 @@ node->key = hash; - ngx_rbtree_insert(&ctx->sh->rbtree, node); - lr = (ngx_http_limit_req_node_t *) &node->color; - ngx_queue_insert_head(&ctx->sh->queue, &lr->queue); - lr->len = (u_char) len; lr->excess = 0; ngx_memcpy(lr->data, data, len); + ngx_rbtree_insert(&ctx->sh->rbtree, node); + + ngx_queue_insert_head(&ctx->sh->queue, &lr->queue); + if (account) { lr->last = now; lr->count = 0; From ne at vbart.ru Mon Sep 3 13:37:13 2012 From: ne at vbart.ru (Valentin V. Bartenev) Date: Mon, 3 Sep 2012 17:37:13 +0400 Subject: a bug in limit_request module In-Reply-To: References: Message-ID: <201209031737.13943.ne@vbart.ru> On Wednesday 22 August 2012 17:49:52 chen cw wrote: > Hi, > The red-black tree used in limit_request module has two level of > keys, the top is hash, and the next is the value string itself. However, > when inserting a new node into the tree, only hash is set, the value string > is left empty, as such code shows the whole thing: > > 445: node->key = hash; > 446: > 447: ngx_rbtree_insert(&ctx->sh->rbtree, node); > 448: > 449: lr = (ngx_http_limit_req_node_t *) &node->color; > 450: > 451: ngx_queue_insert_head(&ctx->sh->queue, &lr->queue); > 452: > 453: lr->len = (u_char) len; > 454: lr->excess = 0; > 455: > 456: ngx_memcpy(lr->data, data, len); > > So there are chances nginx inserts a node which the value string > is large, into the left sub tree. > > The bugfix is to move line 447 to under line 456. > Thanks, the fix was committed as r4833. wbr, Valentin V. Bartenev From mdounin at mdounin.ru Wed Sep 5 11:14:40 2012 From: mdounin at mdounin.ru (Maxim Dounin) Date: Wed, 05 Sep 2012 15:14:40 +0400 Subject: [PATCH 0 of 4] OCSP stapling Message-ID: Hello! Here are patches for OCSP stapling support. Testing and review appreciated. New directives: ssl_trusted_certificate /path/to/file; Specifies a file with CA certificates in the PEM format used for certificate verification. In contrast to ssl_client_certificate, DNs of these certificates aren't sent to a client in CertificateRequest. ssl_stapling on|off; Activates OCSP stapling. ssl_stapling_file /path/to/file; Use predefined OCSP response for stapling, do not query responder. Assumes OCSP response in DER format as produced by "openssl ocsp". ssl_stapling_responder URL; Use specified OCSP responder instead of one found in AIA certificate extension. Example configuration: server { listen 443 ssl; ssl_certificate /path/to/cert.pem; ssl_certificate_key /path/to/key.pem; ssl_stapling on; ssl_trusted_certificate /path/to/ca.pem; resolver 8.8.8.8; } Known limitations: - Unless externally set OCSP response is used (via the "ssl_stapling_file" directive), stapled response won't be sent in a first connection. This is due to the fact that OCSP responders are currently queried by nginx once it receives connection with certificate_status extension in ClientHello, and due to limitations in OpenSSL API (certificate status callback is blocking). - Cached OCSP responses are currently stored in local process memory (thus each worker process will query OCSP responders independently). This shouldn't be a problem as typical number of worker processes is low, usually set match number of CPUs. - Various timeouts are hardcoded (connect/read/write timeouts are 60s, response is considered to be valid for 1h after loading). Adding configuration directives to control these would be trivial, but it may be a better idea to actually omit them for simplicity. - Only "http://" OCSP responders are recognized. Patch can be found here: http://nginx.org/patches/ocsp-stapling/ Thanks to Comodo, DigiCert and GlobalSign for sponsoring this work. Maxim Dounin From mdounin at mdounin.ru Wed Sep 5 11:14:41 2012 From: mdounin at mdounin.ru (Maxim Dounin) Date: Wed, 05 Sep 2012 15:14:41 +0400 Subject: [PATCH 1 of 4] OCSP stapling: ssl_trusted_certificate directive In-Reply-To: References: Message-ID: # HG changeset patch # User Maxim Dounin # Date 1346840510 -14400 # Node ID ca180646a706288271f411ed65f40ae5a1c26211 # Parent 384fd80c0fe6db6f3f61de47f899567da7b98417 OCSP stapling: ssl_trusted_certificate directive. The directive allows to specify additional trusted Certificate Authority certificates to be used during certificate verification. In contrast to ssl_client_certificate DNs of these cerificates aren't sent to a client during handshake. Trusted certificates are loaded regardless of the fact whether client certificates verification is enabled as the same certificates will be used for OCSP stapling, during construction of an OCSP request and for verification of an OCSP response. The same applies to a CRL (which is now always loaded). diff --git a/src/event/ngx_event_openssl.c b/src/event/ngx_event_openssl.c --- a/src/event/ngx_event_openssl.c +++ b/src/event/ngx_event_openssl.c @@ -295,6 +295,33 @@ ngx_ssl_client_certificate(ngx_conf_t *c ngx_int_t +ngx_ssl_trusted_certificate(ngx_conf_t *cf, ngx_ssl_t *ssl, ngx_str_t *cert, + ngx_int_t depth) +{ + SSL_CTX_set_verify_depth(ssl->ctx, depth); + + if (cert->len == 0) { + return NGX_OK; + } + + if (ngx_conf_full_name(cf->cycle, cert, 1) != NGX_OK) { + return NGX_ERROR; + } + + if (SSL_CTX_load_verify_locations(ssl->ctx, (char *) cert->data, NULL) + == 0) + { + ngx_ssl_error(NGX_LOG_EMERG, ssl->log, 0, + "SSL_CTX_load_verify_locations(\"%s\") failed", + cert->data); + return NGX_ERROR; + } + + return NGX_OK; +} + + +ngx_int_t ngx_ssl_crl(ngx_conf_t *cf, ngx_ssl_t *ssl, ngx_str_t *crl) { X509_STORE *store; diff --git a/src/event/ngx_event_openssl.h b/src/event/ngx_event_openssl.h --- a/src/event/ngx_event_openssl.h +++ b/src/event/ngx_event_openssl.h @@ -101,6 +101,8 @@ ngx_int_t ngx_ssl_certificate(ngx_conf_t ngx_str_t *cert, ngx_str_t *key); ngx_int_t ngx_ssl_client_certificate(ngx_conf_t *cf, ngx_ssl_t *ssl, ngx_str_t *cert, ngx_int_t depth); +ngx_int_t ngx_ssl_trusted_certificate(ngx_conf_t *cf, ngx_ssl_t *ssl, + ngx_str_t *cert, ngx_int_t depth); ngx_int_t ngx_ssl_crl(ngx_conf_t *cf, ngx_ssl_t *ssl, ngx_str_t *crl); RSA *ngx_ssl_rsa512_key_callback(SSL *ssl, int is_export, int key_length); ngx_int_t ngx_ssl_dhparam(ngx_conf_t *cf, ngx_ssl_t *ssl, ngx_str_t *file); diff --git a/src/http/modules/ngx_http_ssl_module.c b/src/http/modules/ngx_http_ssl_module.c --- a/src/http/modules/ngx_http_ssl_module.c +++ b/src/http/modules/ngx_http_ssl_module.c @@ -124,6 +124,13 @@ static ngx_command_t ngx_http_ssl_comma offsetof(ngx_http_ssl_srv_conf_t, client_certificate), NULL }, + { ngx_string("ssl_trusted_certificate"), + NGX_HTTP_MAIN_CONF|NGX_HTTP_SRV_CONF|NGX_CONF_TAKE1, + ngx_conf_set_str_slot, + NGX_HTTP_SRV_CONF_OFFSET, + offsetof(ngx_http_ssl_srv_conf_t, trusted_certificate), + NULL }, + { ngx_string("ssl_prefer_server_ciphers"), NGX_HTTP_MAIN_CONF|NGX_HTTP_SRV_CONF|NGX_CONF_FLAG, ngx_conf_set_flag_slot, @@ -325,6 +332,7 @@ ngx_http_ssl_create_srv_conf(ngx_conf_t * sscf->dhparam = { 0, NULL }; * sscf->ecdh_curve = { 0, NULL }; * sscf->client_certificate = { 0, NULL }; + * sscf->trusted_certificate = { 0, NULL }; * sscf->crl = { 0, NULL }; * sscf->ciphers = { 0, NULL }; * sscf->shm_zone = NULL; @@ -380,6 +388,8 @@ ngx_http_ssl_merge_srv_conf(ngx_conf_t * ngx_conf_merge_str_value(conf->client_certificate, prev->client_certificate, ""); + ngx_conf_merge_str_value(conf->trusted_certificate, + prev->trusted_certificate, ""); ngx_conf_merge_str_value(conf->crl, prev->crl, ""); ngx_conf_merge_str_value(conf->ecdh_curve, prev->ecdh_curve, @@ -479,10 +489,18 @@ ngx_http_ssl_merge_srv_conf(ngx_conf_t * { return NGX_CONF_ERROR; } + } - if (ngx_ssl_crl(cf, &conf->ssl, &conf->crl) != NGX_OK) { - return NGX_CONF_ERROR; - } + if (ngx_ssl_trusted_certificate(cf, &conf->ssl, + &conf->trusted_certificate, + conf->verify_depth) + != NGX_OK) + { + return NGX_CONF_ERROR; + } + + if (ngx_ssl_crl(cf, &conf->ssl, &conf->crl) != NGX_OK) { + return NGX_CONF_ERROR; } if (conf->prefer_server_ciphers) { diff --git a/src/http/modules/ngx_http_ssl_module.h b/src/http/modules/ngx_http_ssl_module.h --- a/src/http/modules/ngx_http_ssl_module.h +++ b/src/http/modules/ngx_http_ssl_module.h @@ -35,6 +35,7 @@ typedef struct { ngx_str_t dhparam; ngx_str_t ecdh_curve; ngx_str_t client_certificate; + ngx_str_t trusted_certificate; ngx_str_t crl; ngx_str_t ciphers; From mdounin at mdounin.ru Wed Sep 5 11:14:42 2012 From: mdounin at mdounin.ru (Maxim Dounin) Date: Wed, 05 Sep 2012 15:14:42 +0400 Subject: [PATCH 2 of 4] OCSP stapling: ssl_stapling_file support In-Reply-To: References: Message-ID: <9fa7f13e92b0287cbef6.1346843682@vm-bsd.mdounin.ru> # HG changeset patch # User Maxim Dounin # Date 1346840511 -14400 # Node ID 9fa7f13e92b0287cbef687cc28a6a282a6d3f4c8 # Parent ca180646a706288271f411ed65f40ae5a1c26211 OCSP stapling: ssl_stapling_file support. Very basic version without any OCSP responder query code, assuming valid DER-encoded OCSP response is present in a ssl_stapling_file configured. Such file might be produced with openssl like this: openssl ocsp -issuer root.crt -cert domain.crt -respout domain.staple \ -url http://ocsp.example.com diff --git a/auto/sources b/auto/sources --- a/auto/sources +++ b/auto/sources @@ -77,7 +77,8 @@ REGEX_SRCS=src/core/ngx_regex.c OPENSSL_MODULE=ngx_openssl_module OPENSSL_DEPS=src/event/ngx_event_openssl.h -OPENSSL_SRCS=src/event/ngx_event_openssl.c +OPENSSL_SRCS="src/event/ngx_event_openssl.c \ + src/event/ngx_event_openssl_stapling.c" EVENT_MODULES="ngx_events_module ngx_event_core_module" diff --git a/src/event/ngx_event_openssl.h b/src/event/ngx_event_openssl.h --- a/src/event/ngx_event_openssl.h +++ b/src/event/ngx_event_openssl.h @@ -17,6 +17,7 @@ #include #include #include +#include #define NGX_SSL_NAME "OpenSSL" @@ -104,6 +105,7 @@ ngx_int_t ngx_ssl_client_certificate(ngx ngx_int_t ngx_ssl_trusted_certificate(ngx_conf_t *cf, ngx_ssl_t *ssl, ngx_str_t *cert, ngx_int_t depth); ngx_int_t ngx_ssl_crl(ngx_conf_t *cf, ngx_ssl_t *ssl, ngx_str_t *crl); +ngx_int_t ngx_ssl_stapling(ngx_conf_t *cf, ngx_ssl_t *ssl, ngx_str_t *file); RSA *ngx_ssl_rsa512_key_callback(SSL *ssl, int is_export, int key_length); ngx_int_t ngx_ssl_dhparam(ngx_conf_t *cf, ngx_ssl_t *ssl, ngx_str_t *file); ngx_int_t ngx_ssl_ecdh_curve(ngx_conf_t *cf, ngx_ssl_t *ssl, ngx_str_t *name); diff --git a/src/event/ngx_event_openssl_stapling.c b/src/event/ngx_event_openssl_stapling.c new file mode 100644 --- /dev/null +++ b/src/event/ngx_event_openssl_stapling.c @@ -0,0 +1,121 @@ + +/* + * Copyright (C) Maxim Dounin + * Copyright (C) Nginx, Inc. + */ + + +#include +#include +#include + + +static int ngx_ssl_certificate_status_callback(ngx_ssl_conn_t *ssl_conn, + void *data); + + +ngx_int_t +ngx_ssl_stapling(ngx_conf_t *cf, ngx_ssl_t *ssl, ngx_str_t *file) +{ + BIO *bio; + int len; + u_char *p, *buf; + ngx_str_t *staple; + OCSP_RESPONSE *response; + + if (file->len == 0) { + return NGX_OK; + } + + if (ngx_conf_full_name(cf->cycle, file, 1) != NGX_OK) { + return NGX_ERROR; + } + + bio = BIO_new_file((char *) file->data, "r"); + if (bio == NULL) { + ngx_ssl_error(NGX_LOG_EMERG, ssl->log, 0, + "BIO_new_file(\"%s\") failed", file->data); + return NGX_ERROR; + } + + response = d2i_OCSP_RESPONSE_bio(bio, NULL); + if (response == NULL) { + ngx_ssl_error(NGX_LOG_EMERG, ssl->log, 0, + "d2i_OCSP_RESPONSE_bio(\"%s\") failed", file->data); + BIO_free(bio); + return NGX_ERROR; + } + + len = i2d_OCSP_RESPONSE(response, NULL); + if (len <= 0) { + ngx_ssl_error(NGX_LOG_EMERG, ssl->log, 0, + "i2d_OCSP_RESPONSE(\"%s\") failed", file->data); + goto failed; + } + + buf = ngx_pnalloc(cf->pool, len); + if (buf == NULL) { + goto failed; + } + + p = buf; + len = i2d_OCSP_RESPONSE(response, &p); + if (len <= 0) { + ngx_ssl_error(NGX_LOG_EMERG, ssl->log, 0, + "i2d_OCSP_RESPONSE(\"%s\") failed", file->data); + goto failed; + } + + OCSP_RESPONSE_free(response); + BIO_free(bio); + + staple = ngx_palloc(cf->pool, sizeof(ngx_str_t)); + if (staple == NULL) { + return NGX_ERROR; + } + + staple->data = buf; + staple->len = len; + + SSL_CTX_set_tlsext_status_cb(ssl->ctx, ngx_ssl_certificate_status_callback); + SSL_CTX_set_tlsext_status_arg(ssl->ctx, staple); + + return NGX_OK; + +failed: + + OCSP_RESPONSE_free(response); + BIO_free(bio); + + return NGX_ERROR; +} + + +static int +ngx_ssl_certificate_status_callback(ngx_ssl_conn_t *ssl_conn, void *data) +{ + u_char *p; + ngx_str_t *staple; + ngx_connection_t *c; + + c = ngx_ssl_get_connection(ssl_conn); + + ngx_log_debug0(NGX_LOG_DEBUG_EVENT, c->log, 0, + "SSL certificate status callback"); + + staple = data; + + /* we have to copy the staple as OpenSSL will free it by itself */ + + p = OPENSSL_malloc(staple->len); + if (p == NULL) { + ngx_ssl_error(NGX_LOG_ALERT, c->log, 0, "OPENSSL_malloc() failed"); + return SSL_TLSEXT_ERR_ALERT_FATAL; + } + + ngx_memcpy(p, staple->data, staple->len); + + SSL_set_tlsext_status_ocsp_resp(ssl_conn, p, staple->len); + + return SSL_TLSEXT_ERR_OK; +} diff --git a/src/http/modules/ngx_http_ssl_module.c b/src/http/modules/ngx_http_ssl_module.c --- a/src/http/modules/ngx_http_ssl_module.c +++ b/src/http/modules/ngx_http_ssl_module.c @@ -159,6 +159,20 @@ static ngx_command_t ngx_http_ssl_comma offsetof(ngx_http_ssl_srv_conf_t, crl), NULL }, + { ngx_string("ssl_stapling"), + NGX_HTTP_MAIN_CONF|NGX_HTTP_SRV_CONF|NGX_CONF_FLAG, + ngx_conf_set_flag_slot, + NGX_HTTP_SRV_CONF_OFFSET, + offsetof(ngx_http_ssl_srv_conf_t, stapling), + NULL }, + + { ngx_string("ssl_stapling_file"), + NGX_HTTP_MAIN_CONF|NGX_HTTP_SRV_CONF|NGX_CONF_TAKE1, + ngx_conf_set_str_slot, + NGX_HTTP_SRV_CONF_OFFSET, + offsetof(ngx_http_ssl_srv_conf_t, stapling_file), + NULL }, + ngx_null_command }; @@ -336,6 +350,7 @@ ngx_http_ssl_create_srv_conf(ngx_conf_t * sscf->crl = { 0, NULL }; * sscf->ciphers = { 0, NULL }; * sscf->shm_zone = NULL; + * sscf->stapling_file = { 0, NULL }; */ sscf->enable = NGX_CONF_UNSET; @@ -344,6 +359,7 @@ ngx_http_ssl_create_srv_conf(ngx_conf_t sscf->verify_depth = NGX_CONF_UNSET_UINT; sscf->builtin_session_cache = NGX_CONF_UNSET; sscf->session_timeout = NGX_CONF_UNSET; + sscf->stapling = NGX_CONF_UNSET; return sscf; } @@ -397,6 +413,8 @@ ngx_http_ssl_merge_srv_conf(ngx_conf_t * ngx_conf_merge_str_value(conf->ciphers, prev->ciphers, NGX_DEFAULT_CIPHERS); + ngx_conf_merge_value(conf->stapling, prev->stapling, 0); + ngx_conf_merge_str_value(conf->stapling_file, prev->stapling_file, ""); conf->ssl.log = cf->log; @@ -533,6 +551,12 @@ ngx_http_ssl_merge_srv_conf(ngx_conf_t * return NGX_CONF_ERROR; } + if (conf->stapling + && ngx_ssl_stapling(cf, &conf->ssl, &conf->stapling_file) != NGX_OK) + { + return NGX_CONF_ERROR; + } + return NGX_CONF_OK; } diff --git a/src/http/modules/ngx_http_ssl_module.h b/src/http/modules/ngx_http_ssl_module.h --- a/src/http/modules/ngx_http_ssl_module.h +++ b/src/http/modules/ngx_http_ssl_module.h @@ -42,6 +42,9 @@ typedef struct { ngx_shm_zone_t *shm_zone; + ngx_flag_t stapling; + ngx_str_t stapling_file; + u_char *file; ngx_uint_t line; } ngx_http_ssl_srv_conf_t; From mdounin at mdounin.ru Wed Sep 5 11:14:43 2012 From: mdounin at mdounin.ru (Maxim Dounin) Date: Wed, 05 Sep 2012 15:14:43 +0400 Subject: [PATCH 3 of 4] OCSP stapling: loading OCSP responses In-Reply-To: References: Message-ID: <73a3c526e072c5f5ed8f.1346843683@vm-bsd.mdounin.ru> # HG changeset patch # User Maxim Dounin # Date 1346843259 -14400 # Node ID 73a3c526e072c5f5ed8f04a75e3498def5cd97e5 # Parent 9fa7f13e92b0287cbef687cc28a6a282a6d3f4c8 OCSP stapling: loading OCSP responses. This includes the ssl_stapling_responder directive (defaults to OCSP responder set in certificate's AIA extension). OCSP response for a given certificate is requested once we get at least one connection with certificate_status extension in ClientHello, and certificate status won't be sent in the connection in question. This due to limitations in the OpenSSL API (certificate status callback is blocking). Note: SSL_CTX_use_certificate_chain_file() was reimplemented as it doesn't allow to access the certificate loaded via SSL_CTX. diff --git a/src/core/ngx_core.h b/src/core/ngx_core.h --- a/src/core/ngx_core.h +++ b/src/core/ngx_core.h @@ -69,12 +69,12 @@ typedef void (*ngx_connection_handler_pt #include #include #include +#include #if (NGX_OPENSSL) #include #endif #include #include -#include #include #include #include diff --git a/src/event/ngx_event_openssl.c b/src/event/ngx_event_openssl.c --- a/src/event/ngx_event_openssl.c +++ b/src/event/ngx_event_openssl.c @@ -82,6 +82,8 @@ ngx_module_t ngx_openssl_module = { int ngx_ssl_connection_index; int ngx_ssl_server_conf_index; int ngx_ssl_session_cache_index; +int ngx_ssl_certificate_index; +int ngx_ssl_stapling_index; ngx_int_t @@ -135,6 +137,22 @@ ngx_ssl_init(ngx_log_t *log) return NGX_ERROR; } + ngx_ssl_certificate_index = SSL_CTX_get_ex_new_index(0, NULL, NULL, NULL, + NULL); + if (ngx_ssl_certificate_index == -1) { + ngx_ssl_error(NGX_LOG_ALERT, log, 0, + "SSL_CTX_get_ex_new_index() failed"); + return NGX_ERROR; + } + + ngx_ssl_stapling_index = SSL_CTX_get_ex_new_index(0, NULL, NULL, NULL, + NULL); + if (ngx_ssl_stapling_index == -1) { + ngx_ssl_error(NGX_LOG_ALERT, log, 0, + "SSL_CTX_get_ex_new_index() failed"); + return NGX_ERROR; + } + return NGX_OK; } @@ -216,19 +234,89 @@ ngx_int_t ngx_ssl_certificate(ngx_conf_t *cf, ngx_ssl_t *ssl, ngx_str_t *cert, ngx_str_t *key) { + BIO *bio; + X509 *x509; + u_long n; + if (ngx_conf_full_name(cf->cycle, cert, 1) != NGX_OK) { return NGX_ERROR; } - if (SSL_CTX_use_certificate_chain_file(ssl->ctx, (char *) cert->data) + /* + * we can't use SSL_CTX_use_certificate_chain_file() as it doesn't + * allow to access certificate later from SSL_CTX, so we reimplement + * it here + */ + + bio = BIO_new_file((char *) cert->data, "r"); + if (bio == NULL) { + ngx_ssl_error(NGX_LOG_EMERG, ssl->log, 0, + "BIO_new_file(\"%s\") failed", cert->data); + return NGX_ERROR; + } + + x509 = PEM_read_bio_X509_AUX(bio, NULL, NULL, NULL); + if (x509 == NULL) { + ngx_ssl_error(NGX_LOG_EMERG, ssl->log, 0, + "PEM_read_bio_X509_AUX(\"%s\") failed", cert->data); + BIO_free(bio); + return NGX_ERROR; + } + + if (SSL_CTX_use_certificate(ssl->ctx, x509) == 0) { + ngx_ssl_error(NGX_LOG_EMERG, ssl->log, 0, + "SSL_CTX_use_certificate(\"%s\") failed", cert->data); + X509_free(x509); + BIO_free(bio); + return NGX_ERROR; + } + + if (SSL_CTX_set_ex_data(ssl->ctx, ngx_ssl_certificate_index, x509) == 0) { ngx_ssl_error(NGX_LOG_EMERG, ssl->log, 0, - "SSL_CTX_use_certificate_chain_file(\"%s\") failed", - cert->data); + "SSL_CTX_set_ex_data() failed"); return NGX_ERROR; } + X509_free(x509); + + /* read rest of the chain */ + + for ( ;; ) { + + x509 = PEM_read_bio_X509(bio, NULL, NULL, NULL); + if (x509 == NULL) { + n = ERR_peek_last_error(); + + if (ERR_GET_LIB(n) == ERR_LIB_PEM + && ERR_GET_REASON(n) == PEM_R_NO_START_LINE) + { + /* end of file */ + ERR_clear_error(); + break; + } + + /* some real error */ + + ngx_ssl_error(NGX_LOG_EMERG, ssl->log, 0, + "PEM_read_bio_X509(\"%s\") failed", cert->data); + BIO_free(bio); + return NGX_ERROR; + } + + if (SSL_CTX_add_extra_chain_cert(ssl->ctx, x509) == 0) { + ngx_ssl_error(NGX_LOG_EMERG, ssl->log, 0, + "SSL_CTX_add_extra_chain_cert(\"%s\") failed", + cert->data); + X509_free(x509); + BIO_free(bio); + return NGX_ERROR; + } + } + + BIO_free(bio); + if (ngx_conf_full_name(cf->cycle, key, 1) != NGX_OK) { return NGX_ERROR; } diff --git a/src/event/ngx_event_openssl.h b/src/event/ngx_event_openssl.h --- a/src/event/ngx_event_openssl.h +++ b/src/event/ngx_event_openssl.h @@ -105,7 +105,10 @@ ngx_int_t ngx_ssl_client_certificate(ngx ngx_int_t ngx_ssl_trusted_certificate(ngx_conf_t *cf, ngx_ssl_t *ssl, ngx_str_t *cert, ngx_int_t depth); ngx_int_t ngx_ssl_crl(ngx_conf_t *cf, ngx_ssl_t *ssl, ngx_str_t *crl); -ngx_int_t ngx_ssl_stapling(ngx_conf_t *cf, ngx_ssl_t *ssl, ngx_str_t *file); +ngx_int_t ngx_ssl_stapling(ngx_conf_t *cf, ngx_ssl_t *ssl, + ngx_str_t *responder, ngx_str_t *file); +ngx_int_t ngx_ssl_stapling_resolver(ngx_conf_t *cf, ngx_ssl_t *ssl, + ngx_resolver_t *resolver, ngx_msec_t resolver_timeout); RSA *ngx_ssl_rsa512_key_callback(SSL *ssl, int is_export, int key_length); ngx_int_t ngx_ssl_dhparam(ngx_conf_t *cf, ngx_ssl_t *ssl, ngx_str_t *file); ngx_int_t ngx_ssl_ecdh_curve(ngx_conf_t *cf, ngx_ssl_t *ssl, ngx_str_t *name); @@ -161,6 +164,8 @@ void ngx_ssl_cleanup_ctx(void *data); extern int ngx_ssl_connection_index; extern int ngx_ssl_server_conf_index; extern int ngx_ssl_session_cache_index; +extern int ngx_ssl_certificate_index; +extern int ngx_ssl_stapling_index; #endif /* _NGX_EVENT_OPENSSL_H_INCLUDED_ */ diff --git a/src/event/ngx_event_openssl_stapling.c b/src/event/ngx_event_openssl_stapling.c --- a/src/event/ngx_event_openssl_stapling.c +++ b/src/event/ngx_event_openssl_stapling.c @@ -8,25 +8,183 @@ #include #include #include +#include +#ifdef SSL_CTRL_SET_TLSEXT_STATUS_REQ_CB + +typedef struct { + ngx_str_t staple; + ngx_msec_t timeout; + + ngx_resolver_t *resolver; + ngx_msec_t resolver_timeout; + + ngx_addr_t *addrs; + ngx_str_t host; + ngx_str_t uri; + in_port_t port; + + SSL_CTX *ssl_ctx; + + X509 *cert; + X509 *issuer; + + time_t valid; + + ngx_uint_t loading; /* unsigned:1 */ +} ngx_ssl_stapling_t; + + +typedef struct ngx_ssl_ocsp_ctx_s ngx_ssl_ocsp_ctx_t; + +struct ngx_ssl_ocsp_ctx_s { + X509 *cert; + X509 *issuer; + + ngx_uint_t naddrs; + + ngx_addr_t *addrs; + ngx_str_t host; + ngx_str_t uri; + in_port_t port; + + ngx_resolver_t *resolver; + ngx_msec_t resolver_timeout; + + ngx_msec_t timeout; + + void (*handler)(ngx_ssl_ocsp_ctx_t *r); + void *data; + + ngx_buf_t *request; + ngx_buf_t *response; + ngx_peer_connection_t peer; + + ngx_int_t (*process)(ngx_ssl_ocsp_ctx_t *r); + + ngx_uint_t state; + + ngx_uint_t code; + ngx_uint_t count; + + ngx_uint_t done; + + u_char *header_name_start; + u_char *header_name_end; + u_char *header_start; + u_char *header_end; + + ngx_pool_t *pool; + ngx_log_t *log; +}; + + +static ngx_int_t ngx_ssl_stapling_file(ngx_conf_t *cf, ngx_ssl_t *ssl, + ngx_str_t *file); +static ngx_int_t ngx_ssl_stapling_responder(ngx_conf_t *cf, ngx_ssl_t *ssl, + ngx_str_t *responder); + static int ngx_ssl_certificate_status_callback(ngx_ssl_conn_t *ssl_conn, void *data); +static void ngx_ssl_stapling_update(ngx_ssl_stapling_t *staple); +static void ngx_ssl_stapling_ocsp_handler(ngx_ssl_ocsp_ctx_t *ctx); + +static X509 *ngx_ssl_stapling_get_issuer(SSL_CTX *ssl_ctx, X509 *cert, + ngx_log_t *log); + +static void ngx_ssl_stapling_cleanup(void *data); + +static ngx_ssl_ocsp_ctx_t *ngx_ssl_ocsp_start(void); +static void ngx_ssl_ocsp_done(ngx_ssl_ocsp_ctx_t *ctx); +static void ngx_ssl_ocsp_request(ngx_ssl_ocsp_ctx_t *ctx); +static void ngx_ssl_ocsp_resolve_handler(ngx_resolver_ctx_t *resolve); +static void ngx_ssl_ocsp_connect(ngx_ssl_ocsp_ctx_t *ctx); +static void ngx_ssl_ocsp_write_handler(ngx_event_t *wev); +static void ngx_ssl_ocsp_read_handler(ngx_event_t *rev); +static void ngx_ssl_ocsp_dummy_handler(ngx_event_t *ev); + +static ngx_int_t ngx_ssl_ocsp_create_request(ngx_ssl_ocsp_ctx_t *ctx); +static ngx_int_t ngx_ssl_ocsp_process_status_line(ngx_ssl_ocsp_ctx_t *ctx); +static ngx_int_t ngx_ssl_ocsp_parse_status_line(ngx_ssl_ocsp_ctx_t *ctx); +static ngx_int_t ngx_ssl_ocsp_process_headers(ngx_ssl_ocsp_ctx_t *ctx); +static ngx_int_t ngx_ssl_ocsp_parse_header_line(ngx_ssl_ocsp_ctx_t *ctx); +static ngx_int_t ngx_ssl_ocsp_process_body(ngx_ssl_ocsp_ctx_t *ctx); + +static u_char *ngx_ssl_ocsp_log_error(ngx_log_t *log, u_char *buf, size_t len); ngx_int_t -ngx_ssl_stapling(ngx_conf_t *cf, ngx_ssl_t *ssl, ngx_str_t *file) +ngx_ssl_stapling(ngx_conf_t *cf, ngx_ssl_t *ssl, ngx_str_t *responder, + ngx_str_t *file) { - BIO *bio; - int len; - u_char *p, *buf; - ngx_str_t *staple; - OCSP_RESPONSE *response; + ngx_pool_cleanup_t *cln; + ngx_ssl_stapling_t *staple; - if (file->len == 0) { + staple = ngx_pcalloc(cf->pool, sizeof(ngx_ssl_stapling_t)); + if (staple == NULL) { + return NGX_ERROR; + } + + cln = ngx_pool_cleanup_add(cf->pool, 0); + if (cln == NULL) { + return NGX_ERROR; + } + + cln->handler = ngx_ssl_stapling_cleanup; + cln->data = staple; + + SSL_CTX_set_tlsext_status_cb(ssl->ctx, ngx_ssl_certificate_status_callback); + SSL_CTX_set_tlsext_status_arg(ssl->ctx, staple); + + if (SSL_CTX_set_ex_data(ssl->ctx, ngx_ssl_stapling_index, staple) + == 0) + { + ngx_ssl_error(NGX_LOG_EMERG, ssl->log, 0, + "SSL_CTX_set_ex_data() failed"); + return NGX_ERROR; + } + + staple->ssl_ctx = ssl->ctx; + staple->timeout = 60000; + + if (file->len) { + /* use OCSP response from the file */ + + if (ngx_ssl_stapling_file(cf, ssl, file) != NGX_OK) { + return NGX_ERROR; + } + return NGX_OK; } + staple->cert = SSL_CTX_get_ex_data(ssl->ctx, ngx_ssl_certificate_index); + staple->issuer = ngx_ssl_stapling_get_issuer(ssl->ctx, staple->cert, + ssl->log); + + if (staple->issuer == NULL) { + return NGX_ERROR; + } + + if (ngx_ssl_stapling_responder(cf, ssl, responder) != NGX_OK) { + return NGX_ERROR; + } + + return NGX_OK; +} + + +static ngx_int_t +ngx_ssl_stapling_file(ngx_conf_t *cf, ngx_ssl_t *ssl, ngx_str_t *file) +{ + BIO *bio; + int len; + u_char *p, *buf; + OCSP_RESPONSE *response; + ngx_ssl_stapling_t *staple; + + staple = SSL_CTX_get_ex_data(ssl->ctx, ngx_ssl_stapling_index); + if (ngx_conf_full_name(cf->cycle, file, 1) != NGX_OK) { return NGX_ERROR; } @@ -53,7 +211,7 @@ ngx_ssl_stapling(ngx_conf_t *cf, ngx_ssl goto failed; } - buf = ngx_pnalloc(cf->pool, len); + buf = ngx_alloc(len, ssl->log); if (buf == NULL) { goto failed; } @@ -63,22 +221,15 @@ ngx_ssl_stapling(ngx_conf_t *cf, ngx_ssl if (len <= 0) { ngx_ssl_error(NGX_LOG_EMERG, ssl->log, 0, "i2d_OCSP_RESPONSE(\"%s\") failed", file->data); + ngx_free(buf); goto failed; } OCSP_RESPONSE_free(response); BIO_free(bio); - staple = ngx_palloc(cf->pool, sizeof(ngx_str_t)); - if (staple == NULL) { - return NGX_ERROR; - } - - staple->data = buf; - staple->len = len; - - SSL_CTX_set_tlsext_status_cb(ssl->ctx, ngx_ssl_certificate_status_callback); - SSL_CTX_set_tlsext_status_arg(ssl->ctx, staple); + staple->staple.data = buf; + staple->staple.len = len; return NGX_OK; @@ -91,12 +242,112 @@ failed: } +static ngx_int_t +ngx_ssl_stapling_responder(ngx_conf_t *cf, ngx_ssl_t *ssl, ngx_str_t *responder) +{ + ngx_url_t u; + char *s; + ngx_ssl_stapling_t *staple; + STACK_OF(OPENSSL_STRING) *aia; + + staple = SSL_CTX_get_ex_data(ssl->ctx, ngx_ssl_stapling_index); + + if (responder->len == 0) { + + /* extract OCSP responder URL from certificate */ + + aia = X509_get1_ocsp(staple->cert); + if (aia == NULL) { + ngx_ssl_error(NGX_LOG_EMERG, ssl->log, 0, + "no OCSP responder URL in the certificate, " + "use the \"ssl_stapling_responder\" directive " + "to set one"); + return NGX_ERROR; + } + + s = sk_OPENSSL_STRING_value(aia, 0); + if (s == NULL) { + ngx_ssl_error(NGX_LOG_EMERG, ssl->log, 0, + "no OCSP responder URL in the certificate, " + "use the \"ssl_stapling_responder\" directive " + "to set one"); + X509_email_free(aia); + return NGX_ERROR; + } + + responder->len = ngx_strlen(s); + responder->data = ngx_palloc(cf->pool, responder->len); + if (responder->data == NULL) { + X509_email_free(aia); + return NGX_ERROR; + } + + ngx_memcpy(responder->data, s, responder->len); + X509_email_free(aia); + } + + ngx_memzero(&u, sizeof(ngx_url_t)); + + u.url = *responder; + u.default_port = 80; + u.uri_part = 1; + + if (u.url.len > 7 + && ngx_strncasecmp(u.url.data, (u_char *) "http://", 7) == 0) + { + u.url.len -= 7; + u.url.data += 7; + + } else { + ngx_log_error(NGX_LOG_ERR, ssl->log, 0, + "invalid URL prefix in \"%V\"", &u.url); + return NGX_ERROR; + } + + if (ngx_parse_url(cf->pool, &u) != NGX_OK) { + if (u.err) { + ngx_log_error(NGX_LOG_ERR, ssl->log, 0, + "%s in OCSP responder \"%V\"", u.err, &u.url); + } + + return NGX_ERROR; + } + + staple->addrs = u.addrs; + staple->host = u.host; + staple->uri = u.uri; + staple->port = u.port; + + if (staple->uri.len == 0) { + ngx_str_set(&staple->uri, "/"); + } + + return NGX_OK; +} + + +ngx_int_t +ngx_ssl_stapling_resolver(ngx_conf_t *cf, ngx_ssl_t *ssl, + ngx_resolver_t *resolver, ngx_msec_t resolver_timeout) +{ + ngx_ssl_stapling_t *staple; + + staple = SSL_CTX_get_ex_data(ssl->ctx, ngx_ssl_stapling_index); + + staple->resolver = resolver; + staple->resolver_timeout = resolver_timeout; + + return NGX_OK; +} + + static int ngx_ssl_certificate_status_callback(ngx_ssl_conn_t *ssl_conn, void *data) { - u_char *p; - ngx_str_t *staple; - ngx_connection_t *c; + int rc; + u_char *p; + ngx_connection_t *c; + ngx_ssl_stapling_t *staple; c = ngx_ssl_get_connection(ssl_conn); @@ -104,18 +355,1338 @@ ngx_ssl_certificate_status_callback(ngx_ "SSL certificate status callback"); staple = data; + rc = SSL_TLSEXT_ERR_NOACK; - /* we have to copy the staple as OpenSSL will free it by itself */ + if (staple->staple.len) { + /* we have to copy ocsp response as OpenSSL will free it by itself */ - p = OPENSSL_malloc(staple->len); - if (p == NULL) { - ngx_ssl_error(NGX_LOG_ALERT, c->log, 0, "OPENSSL_malloc() failed"); - return SSL_TLSEXT_ERR_ALERT_FATAL; + p = OPENSSL_malloc(staple->staple.len); + if (p == NULL) { + ngx_ssl_error(NGX_LOG_ALERT, c->log, 0, "OPENSSL_malloc() failed"); + return SSL_TLSEXT_ERR_NOACK; + } + + ngx_memcpy(p, staple->staple.data, staple->staple.len); + + SSL_set_tlsext_status_ocsp_resp(ssl_conn, p, staple->staple.len); + + rc = SSL_TLSEXT_ERR_OK; } - ngx_memcpy(p, staple->data, staple->len); + ngx_ssl_stapling_update(staple); - SSL_set_tlsext_status_ocsp_resp(ssl_conn, p, staple->len); + return rc; +} - return SSL_TLSEXT_ERR_OK; + +static void +ngx_ssl_stapling_update(ngx_ssl_stapling_t *staple) +{ + ngx_ssl_ocsp_ctx_t *ctx; + + if (staple->host.len == 0 + || staple->loading || staple->valid >= ngx_time()) + { + return; + } + + staple->loading = 1; + + ctx = ngx_ssl_ocsp_start(); + if (ctx == NULL) { + return; + } + + ctx->cert = staple->cert; + ctx->issuer = staple->issuer; + + ctx->addrs = staple->addrs; + ctx->host = staple->host; + ctx->uri = staple->uri; + ctx->port = staple->port; + ctx->timeout = staple->timeout; + + ctx->resolver = staple->resolver; + ctx->resolver_timeout = staple->resolver_timeout; + + ctx->handler = ngx_ssl_stapling_ocsp_handler; + ctx->data = staple; + + ngx_ssl_ocsp_request(ctx); + + return; } + + +static X509 * +ngx_ssl_stapling_get_issuer(SSL_CTX *ssl_ctx, X509 *cert, ngx_log_t *log) +{ + int i, n, rc; + X509 *issuer; + X509_STORE *store; + X509_STORE_CTX *store_ctx; + STACK_OF(X509) *chain; + +#if OPENSSL_VERSION_NUMBER >= 0x10001000L + SSL_CTX_get_extra_chain_certs(ssl_ctx, &chain); +#else + chain = ssl_ctx->extra_certs; +#endif + + n = sk_X509_num(chain); + + ngx_log_debug1(NGX_LOG_DEBUG_EVENT, log, 0, + "SSL get issuer: %d extra certs", n); + + for (i = 0; i < n; i++) { + issuer = sk_X509_value(chain, i); + if (X509_check_issued(issuer, cert) == X509_V_OK) { + CRYPTO_add(&issuer->references, 1, CRYPTO_LOCK_X509); + + ngx_log_debug1(NGX_LOG_DEBUG_EVENT, log, 0, + "SSL get issuer: found %p in extra certs", issuer); + + return issuer; + } + } + + store = SSL_CTX_get_cert_store(ssl_ctx); + if (store == NULL) { + ngx_ssl_error(NGX_LOG_EMERG, log, 0, + "SSL_CTX_get_cert_store() failed"); + return NULL; + } + + store_ctx = X509_STORE_CTX_new(); + if (store_ctx == NULL) { + ngx_ssl_error(NGX_LOG_EMERG, log, 0, + "X509_STORE_CTX_new() failed"); + return NULL; + } + + if (X509_STORE_CTX_init(store_ctx, store, NULL, NULL) == 0) { + ngx_ssl_error(NGX_LOG_EMERG, log, 0, + "X509_STORE_CTX_init() failed"); + return NULL; + } + + rc = X509_STORE_CTX_get1_issuer(&issuer, store_ctx, cert); + + if (rc == -1) { + ngx_ssl_error(NGX_LOG_EMERG, log, 0, + "X509_STORE_CTX_get1_issuer() failed"); + X509_STORE_CTX_free(store_ctx); + return NULL; + } + + if (rc == 0) { + /* TODO: provide better error message */ + ngx_log_error(NGX_LOG_EMERG, log, 0, + "X509_STORE_CTX_get1_issuer() failed, " + "issuer certificate not found"); + X509_STORE_CTX_free(store_ctx); + return NULL; + } + + X509_STORE_CTX_free(store_ctx); + + ngx_log_debug1(NGX_LOG_DEBUG_EVENT, log, 0, + "SSL get issuer: found %p in cert store", issuer); + + return issuer; +} + + +static void +ngx_ssl_stapling_ocsp_handler(ngx_ssl_ocsp_ctx_t *ctx) +{ +#if OPENSSL_VERSION_NUMBER >= 0x0090707fL + const +#endif + u_char *p; + int n; + size_t len; + ngx_str_t response; + X509_STORE *store; + STACK_OF(X509) *chain; + OCSP_CERTID *id; + OCSP_RESPONSE *ocsp; + OCSP_BASICRESP *basic; + ngx_ssl_stapling_t *staple; + ASN1_GENERALIZEDTIME *thisupdate, *nextupdate; + + staple = ctx->data; + ocsp = NULL; + basic = NULL; + id = NULL; + + if (ctx->code != 200) { + goto error; + } + + /* check the response */ + + len = ctx->response->last - ctx->response->pos; + p = ctx->response->pos; + + ocsp = d2i_OCSP_RESPONSE(NULL, &p, len); + if (ocsp == NULL) { + ngx_ssl_error(NGX_LOG_ERR, ctx->log, 0, + "d2i_OCSP_RESPONSE() failed"); + goto error; + } + + n = OCSP_response_status(ocsp); + + if (n != OCSP_RESPONSE_STATUS_SUCCESSFUL) { + ngx_log_error(NGX_LOG_ERR, ctx->log, 0, + "OCSP response not successfull (%d: %s)", + n, OCSP_response_status_str(n)); + goto error; + } + + basic = OCSP_response_get1_basic(ocsp); + if (basic == NULL) { + ngx_ssl_error(NGX_LOG_ERR, ctx->log, 0, + "OCSP_response_get1_basic() failed"); + goto error; + } + + store = SSL_CTX_get_cert_store(staple->ssl_ctx); + if (store == NULL) { + ngx_ssl_error(NGX_LOG_CRIT, ctx->log, 0, + "SSL_CTX_get_cert_store() failed"); + goto error; + } + +#if OPENSSL_VERSION_NUMBER >= 0x10001000L + SSL_CTX_get_extra_chain_certs(staple->ssl_ctx, &chain); +#else + chain = staple->ssl_ctx->extra_certs; +#endif + + if (OCSP_basic_verify(basic, chain, store, 0) != 1) { + ngx_ssl_error(NGX_LOG_ERR, ctx->log, 0, + "OCSP_basic_verify() failed"); + goto error; + } + + id = OCSP_cert_to_id(NULL, ctx->cert, ctx->issuer); + if (id == NULL) { + ngx_ssl_error(NGX_LOG_CRIT, ctx->log, 0, + "OCSP_cert_to_id() failed"); + goto error; + } + + if (OCSP_resp_find_status(basic, id, &n, NULL, NULL, + &thisupdate, &nextupdate) + != 1) + { + ngx_log_error(NGX_LOG_ERR, ctx->log, 0, + "certificate status not found in the OCSP response", + n, OCSP_response_status_str(n)); + goto error; + } + + if (n != V_OCSP_CERTSTATUS_GOOD) { + ngx_log_error(NGX_LOG_ERR, ctx->log, 0, + "certificate status \"%s\" in the OCSP response", + n, OCSP_cert_status_str(n)); + goto error; + } + + if (OCSP_check_validity(thisupdate, nextupdate, 300, -1) != 1) { + ngx_ssl_error(NGX_LOG_ERR, ctx->log, 0, + "OCSP_check_validity() failed"); + goto error; + } + + OCSP_CERTID_free(id); + OCSP_BASICRESP_free(basic); + OCSP_RESPONSE_free(ocsp); + + /* copy the response to memory not in ctx->pool */ + + response.len = len; + response.data = ngx_alloc(response.len, ctx->log); + + if (response.data == NULL) { + goto done; + } + + ngx_memcpy(response.data, ctx->response->pos, response.len); + + ngx_log_debug2(NGX_LOG_DEBUG_EVENT, ctx->log, 0, + "ssl ocsp response, %s, %uz", + OCSP_cert_status_str(n), response.len); + + if (staple->staple.data) { + ngx_free(staple->staple.data); + } + + staple->staple = response; + +done: + + staple->loading = 0; + staple->valid = ngx_time() + 3600; /* ssl_stapling_valid */ + + ngx_ssl_ocsp_done(ctx); + return; + +error: + + staple->loading = 0; + staple->valid = ngx_time() + 300; /* ssl_stapling_err_valid */ + + if (id) { + OCSP_CERTID_free(id); + } + + if (basic) { + OCSP_BASICRESP_free(basic); + } + + if (ocsp) { + OCSP_RESPONSE_free(ocsp); + } + + ngx_ssl_ocsp_done(ctx); +} + + +static void +ngx_ssl_stapling_cleanup(void *data) +{ + ngx_ssl_stapling_t *staple = data; + + if (staple->issuer) { + X509_free(staple->issuer); + } + + if (staple->staple.data) { + ngx_free(staple->staple.data); + } +} + + +static ngx_ssl_ocsp_ctx_t * +ngx_ssl_ocsp_start(void) +{ + ngx_log_t *log; + ngx_pool_t *pool; + ngx_ssl_ocsp_ctx_t *ctx; + + pool = ngx_create_pool(2048, ngx_cycle->log); + if (pool == NULL) { + return NULL; + } + + ctx = ngx_pcalloc(pool, sizeof(ngx_ssl_ocsp_ctx_t)); + if (ctx == NULL) { + ngx_destroy_pool(pool); + return NULL; + } + + log = ngx_palloc(pool, sizeof(ngx_log_t)); + if (log == NULL) { + ngx_destroy_pool(pool); + return NULL; + } + + ctx->pool = pool; + + *log = *ctx->pool->log; + + ctx->pool->log = log; + ctx->log = log; + + log->handler = ngx_ssl_ocsp_log_error; + log->data = ctx; + log->action = "requesting certificate status"; + + return ctx; +} + + +static void +ngx_ssl_ocsp_done(ngx_ssl_ocsp_ctx_t *ctx) +{ + ngx_log_debug0(NGX_LOG_DEBUG_EVENT, ctx->log, 0, + "ssl ocsp done"); + + if (ctx->peer.connection) { + ngx_close_connection(ctx->peer.connection); + } + + ngx_destroy_pool(ctx->pool); +} + + +static void +ngx_ssl_ocsp_error(ngx_ssl_ocsp_ctx_t *ctx) +{ + ngx_log_debug0(NGX_LOG_DEBUG_EVENT, ctx->log, 0, + "ssl ocsp error"); + + ctx->code = 0; + ctx->handler(ctx); +} + + +static void +ngx_ssl_ocsp_request(ngx_ssl_ocsp_ctx_t *ctx) +{ + ngx_resolver_ctx_t *resolve, temp; + + ngx_log_debug0(NGX_LOG_DEBUG_EVENT, ctx->log, 0, + "ssl ocsp request"); + + if (ngx_ssl_ocsp_create_request(ctx) != NGX_OK) { + ngx_ssl_ocsp_error(ctx); + return; + } + + if (ctx->resolver) { + /* resolve OCSP responder hostname */ + + temp.name = ctx->host; + + resolve = ngx_resolve_start(ctx->resolver, &temp); + if (resolve == NULL) { + ngx_ssl_ocsp_error(ctx); + return; + } + + if (resolve == NGX_NO_RESOLVER) { + ngx_log_error(NGX_LOG_ERR, ctx->log, 0, + "no resolver defined to resolve %V", &ctx->host); + ngx_ssl_ocsp_error(ctx); + return; + } + + resolve->name = ctx->host; + resolve->type = NGX_RESOLVE_A; + resolve->handler = ngx_ssl_ocsp_resolve_handler; + resolve->data = ctx; + resolve->timeout = ctx->resolver_timeout; + + if (ngx_resolve_name(resolve) != NGX_OK) { + ngx_ssl_ocsp_error(ctx); + return; + } + + return; + } + + ngx_ssl_ocsp_connect(ctx); +} + + +static void +ngx_ssl_ocsp_resolve_handler(ngx_resolver_ctx_t *resolve) +{ + ngx_ssl_ocsp_ctx_t *ctx = resolve->data; + + u_char *p; + size_t len; + in_port_t port; + ngx_uint_t i; + struct sockaddr_in *sin; + + ngx_log_debug0(NGX_LOG_ALERT, ctx->log, 0, + "ssl ocsp resolve handler"); + + if (resolve->state) { + ngx_log_error(NGX_LOG_ERR, ctx->log, 0, + "%V could not be resolved (%i: %s)", + &resolve->name, resolve->state, + ngx_resolver_strerror(resolve->state)); + goto failed; + } + +#if (NGX_DEBUG) + { + in_addr_t addr; + + for (i = 0; i < resolve->naddrs; i++) { + addr = ntohl(resolve->addrs[i]); + + ngx_log_debug4(NGX_LOG_DEBUG_EVENT, ctx->log, 0, + "name was resolved to %ud.%ud.%ud.%ud", + (addr >> 24) & 0xff, (addr >> 16) & 0xff, + (addr >> 8) & 0xff, addr & 0xff); + } + } +#endif + + ctx->naddrs = resolve->naddrs; + ctx->addrs = ngx_pcalloc(ctx->pool, ctx->naddrs * sizeof(ngx_addr_t)); + + if (ctx->addrs == NULL) { + goto failed; + } + + port = htons(ctx->port); + + for (i = 0; i < resolve->naddrs; i++) { + + sin = ngx_pcalloc(ctx->pool, sizeof(struct sockaddr_in)); + if (sin == NULL) { + goto failed; + } + + sin->sin_family = AF_INET; + sin->sin_port = port; + sin->sin_addr.s_addr = resolve->addrs[i]; + + ctx->addrs[i].sockaddr = (struct sockaddr *) sin; + ctx->addrs[i].socklen = sizeof(struct sockaddr_in); + + len = NGX_INET_ADDRSTRLEN + sizeof(":65535") - 1; + + p = ngx_pnalloc(ctx->pool, len); + if (p == NULL) { + goto failed; + } + + len = ngx_sock_ntop((struct sockaddr *) sin, p, len, 1); + + ctx->addrs[i].name.len = len; + ctx->addrs[i].name.data = p; + } + + ngx_resolve_name_done(resolve); + + ngx_ssl_ocsp_connect(ctx); + return; + +failed: + + ngx_resolve_name_done(resolve); + ngx_ssl_ocsp_error(ctx); +} + + +static void +ngx_ssl_ocsp_connect(ngx_ssl_ocsp_ctx_t *ctx) +{ + ngx_int_t rc; + + ngx_log_debug0(NGX_LOG_DEBUG_EVENT, ctx->log, 0, + "ssl ocsp connect"); + + /* TODO: use all ip addresses */ + + ctx->peer.sockaddr = ctx->addrs[0].sockaddr; + ctx->peer.socklen = ctx->addrs[0].socklen; + ctx->peer.name = &ctx->addrs[0].name; + ctx->peer.get = ngx_event_get_peer; + ctx->peer.log = ctx->log; + ctx->peer.log_error = NGX_ERROR_ERR; + + rc = ngx_event_connect_peer(&ctx->peer); + + ngx_log_debug0(NGX_LOG_DEBUG_EVENT, ctx->log, 0, + "ssl ocsp connect peer done"); + + if (rc == NGX_ERROR || rc == NGX_BUSY || rc == NGX_DECLINED) { + ngx_ssl_ocsp_error(ctx); + return; + } + + ctx->peer.connection->data = ctx; + ctx->peer.connection->pool = ctx->pool; + + ctx->peer.connection->read->handler = ngx_ssl_ocsp_read_handler; + ctx->peer.connection->write->handler = ngx_ssl_ocsp_write_handler; + + ctx->process = ngx_ssl_ocsp_process_status_line; + + ngx_add_timer(ctx->peer.connection->read, ctx->timeout); + ngx_add_timer(ctx->peer.connection->write, ctx->timeout); + + if (rc == NGX_OK) { + ngx_ssl_ocsp_write_handler(ctx->peer.connection->write); + return; + } +} + + +static void +ngx_ssl_ocsp_write_handler(ngx_event_t *wev) +{ + ssize_t n, size; + ngx_connection_t *c; + ngx_ssl_ocsp_ctx_t *ctx; + + c = wev->data; + ctx = c->data; + + ngx_log_debug0(NGX_LOG_DEBUG_EVENT, wev->log, 0, + "ssl ocsp write handler"); + + if (wev->timedout) { + ngx_log_error(NGX_LOG_ERR, wev->log, NGX_ETIMEDOUT, + "OCSP responder timed out"); + ngx_ssl_ocsp_error(ctx); + return; + } + + size = ctx->request->last - ctx->request->pos; + + n = ngx_send(c, ctx->request->pos, size); + + if (n == NGX_ERROR) { + ngx_ssl_ocsp_error(ctx); + return; + } + + if (n > 0) { + ctx->request->pos += n; + + if (n == size) { + wev->handler = ngx_ssl_ocsp_dummy_handler; + + if (wev->timer_set) { + ngx_del_timer(wev); + } + + if (ngx_handle_write_event(wev, 0) != NGX_OK) { + ngx_ssl_ocsp_error(ctx); + } + + return; + } + } + + if (!wev->timer_set) { + ngx_add_timer(wev, ctx->timeout); + } +} + + +static void +ngx_ssl_ocsp_read_handler(ngx_event_t *rev) +{ + ssize_t n, size; + ngx_int_t rc; + ngx_ssl_ocsp_ctx_t *ctx; + ngx_connection_t *c; + + c = rev->data; + ctx = c->data; + + ngx_log_debug0(NGX_LOG_DEBUG_EVENT, rev->log, 0, + "ssl ocsp read handler"); + + if (rev->timedout) { + ngx_log_error(NGX_LOG_ERR, rev->log, NGX_ETIMEDOUT, + "OCSP responder timed out"); + ngx_ssl_ocsp_error(ctx); + return; + } + + if (ctx->response == NULL) { + ctx->response = ngx_create_temp_buf(ctx->pool, 16384); + if (ctx->response == NULL) { + ngx_ssl_ocsp_error(ctx); + return; + } + } + + for ( ;; ) { + + size = ctx->response->end - ctx->response->last; + + n = ngx_recv(c, ctx->response->last, size); + + if (n > 0) { + ctx->response->last += n; + + rc = ctx->process(ctx); + + if (rc == NGX_ERROR) { + ngx_ssl_ocsp_error(ctx); + return; + } + + continue; + } + + if (n == NGX_AGAIN) { + + if (ngx_handle_read_event(rev, 0) != NGX_OK) { + ngx_ssl_ocsp_error(ctx); + } + + return; + } + + break; + } + + ctx->done = 1; + + rc = ctx->process(ctx); + + if (rc == NGX_DONE) { + /* ctx->handler() was called */ + return; + } + + ngx_log_error(NGX_LOG_ERR, ctx->log, 0, + "OCSP responder prematurely closed connection"); + + ngx_ssl_ocsp_error(ctx); +} + + +static void +ngx_ssl_ocsp_dummy_handler(ngx_event_t *ev) +{ + ngx_log_debug0(NGX_LOG_DEBUG_EVENT, ev->log, 0, + "ssl ocsp dummy handler"); +} + + +static ngx_int_t +ngx_ssl_ocsp_create_request(ngx_ssl_ocsp_ctx_t *ctx) +{ + int len; + u_char *p; + uintptr_t escape; + ngx_str_t binary, base64; + ngx_buf_t *b; + OCSP_CERTID *id; + OCSP_REQUEST *ocsp; + + ocsp = OCSP_REQUEST_new(); + if (ocsp == NULL) { + ngx_ssl_error(NGX_LOG_CRIT, ctx->log, 0, + "OCSP_REQUEST_new() failed"); + return NGX_ERROR; + } + + id = OCSP_cert_to_id(NULL, ctx->cert, ctx->issuer); + if (id == NULL) { + ngx_ssl_error(NGX_LOG_CRIT, ctx->log, 0, + "OCSP_cert_to_id() failed"); + goto failed; + } + + if (OCSP_request_add0_id(ocsp, id) == NULL) { + ngx_ssl_error(NGX_LOG_CRIT, ctx->log, 0, + "OCSP_request_add0_id() failed"); + goto failed; + } + + len = i2d_OCSP_REQUEST(ocsp, NULL); + if (len <= 0) { + ngx_ssl_error(NGX_LOG_CRIT, ctx->log, 0, + "i2d_OCSP_REQUEST() failed"); + goto failed; + } + + binary.len = len; + binary.data = ngx_palloc(ctx->pool, len); + if (binary.data == NULL) { + goto failed; + } + + p = binary.data; + len = i2d_OCSP_REQUEST(ocsp, &p); + if (len <= 0) { + ngx_ssl_error(NGX_LOG_EMERG, ctx->log, 0, + "i2d_OCSP_REQUEST() failed"); + goto failed; + } + + base64.len = ngx_base64_encoded_length(binary.len); + base64.data = ngx_palloc(ctx->pool, base64.len); + if (base64.data == NULL) { + goto failed; + } + + ngx_encode_base64(&base64, &binary); + + escape = ngx_escape_uri(NULL, base64.data, base64.len, + NGX_ESCAPE_URI_COMPONENT); + + ngx_log_debug(NGX_LOG_DEBUG_EVENT, ctx->log, 0, + "ssl ocsp request length %z, escape %d", + base64.len, escape); + + len = sizeof("GET ") - 1 + ctx->uri.len + sizeof("/") - 1 + + base64.len + 2 * escape + sizeof(" HTTP/1.0" CRLF) - 1 + + sizeof("Host: ") - 1 + ctx->host.len + sizeof(CRLF) - 1 + + sizeof(CRLF) - 1; + + b = ngx_create_temp_buf(ctx->pool, len); + if (b == NULL) { + goto failed; + } + + p = b->last; + + p = ngx_cpymem(p, "GET ", sizeof("GET ") - 1); + p = ngx_cpymem(p, ctx->uri.data, ctx->uri.len); + + if (ctx->uri.data[ctx->uri.len - 1] != '/') { + *p++ = '/'; + } + + if (escape == 0) { + p = ngx_cpymem(p, base64.data, base64.len); + + } else { + p = (u_char *) ngx_escape_uri(p, base64.data, base64.len, + NGX_ESCAPE_URI_COMPONENT); + } + + p = ngx_cpymem(p, " HTTP/1.0" CRLF, sizeof(" HTTP/1.0" CRLF) - 1); + p = ngx_cpymem(p, "Host: ", sizeof("Host: ") - 1); + p = ngx_cpymem(p, ctx->host.data, ctx->host.len); + *p++ = CR; *p++ = LF; + + /* add "\r\n" at the header end */ + *p++ = CR; *p++ = LF; + + b->last = p; + ctx->request = b; + + return NGX_OK; + +failed: + + OCSP_REQUEST_free(ocsp); + + return NGX_ERROR; +} + + +static ngx_int_t +ngx_ssl_ocsp_process_status_line(ngx_ssl_ocsp_ctx_t *ctx) +{ + ngx_int_t rc; + + rc = ngx_ssl_ocsp_parse_status_line(ctx); + + if (rc == NGX_OK) { +#if 0 + ngx_log_debug2(NGX_LOG_DEBUG_EVENT, ctx->log, 0, + "ssl ocsp status line \"%*s\"", + ctx->response->pos - ctx->response->start, + ctx->response->start); +#endif + + ctx->process = ngx_ssl_ocsp_process_headers; + return ctx->process(ctx); + } + + if (rc == NGX_AGAIN) { + return NGX_AGAIN; + } + + /* rc == NGX_ERROR */ + + ngx_log_error(NGX_LOG_ERR, ctx->log, 0, + "OCSP responder sent invalid response"); + + return NGX_ERROR; +} + + +static ngx_int_t +ngx_ssl_ocsp_parse_status_line(ngx_ssl_ocsp_ctx_t *ctx) +{ + u_char ch; + u_char *p; + ngx_buf_t *b; + enum { + sw_start = 0, + sw_H, + sw_HT, + sw_HTT, + sw_HTTP, + sw_first_major_digit, + sw_major_digit, + sw_first_minor_digit, + sw_minor_digit, + sw_status, + sw_space_after_status, + sw_status_text, + sw_almost_done + } state; + + ngx_log_debug0(NGX_LOG_DEBUG_EVENT, ctx->log, 0, + "ssl ocsp process status line"); + + state = ctx->state; + b = ctx->response; + + for (p = b->pos; p < b->last; p++) { + ch = *p; + + switch (state) { + + /* "HTTP/" */ + case sw_start: + switch (ch) { + case 'H': + state = sw_H; + break; + default: + return NGX_ERROR; + } + break; + + case sw_H: + switch (ch) { + case 'T': + state = sw_HT; + break; + default: + return NGX_ERROR; + } + break; + + case sw_HT: + switch (ch) { + case 'T': + state = sw_HTT; + break; + default: + return NGX_ERROR; + } + break; + + case sw_HTT: + switch (ch) { + case 'P': + state = sw_HTTP; + break; + default: + return NGX_ERROR; + } + break; + + case sw_HTTP: + switch (ch) { + case '/': + state = sw_first_major_digit; + break; + default: + return NGX_ERROR; + } + break; + + /* the first digit of major HTTP version */ + case sw_first_major_digit: + if (ch < '1' || ch > '9') { + return NGX_ERROR; + } + + state = sw_major_digit; + break; + + /* the major HTTP version or dot */ + case sw_major_digit: + if (ch == '.') { + state = sw_first_minor_digit; + break; + } + + if (ch < '0' || ch > '9') { + return NGX_ERROR; + } + + break; + + /* the first digit of minor HTTP version */ + case sw_first_minor_digit: + if (ch < '0' || ch > '9') { + return NGX_ERROR; + } + + state = sw_minor_digit; + break; + + /* the minor HTTP version or the end of the request line */ + case sw_minor_digit: + if (ch == ' ') { + state = sw_status; + break; + } + + if (ch < '0' || ch > '9') { + return NGX_ERROR; + } + + break; + + /* HTTP status code */ + case sw_status: + if (ch == ' ') { + break; + } + + if (ch < '0' || ch > '9') { + return NGX_ERROR; + } + + ctx->code = ctx->code * 10 + ch - '0'; + + if (++ctx->count == 3) { + state = sw_space_after_status; + } + + break; + + /* space or end of line */ + case sw_space_after_status: + switch (ch) { + case ' ': + state = sw_status_text; + break; + case '.': /* IIS may send 403.1, 403.2, etc */ + state = sw_status_text; + break; + case CR: + state = sw_almost_done; + break; + case LF: + goto done; + default: + return NGX_ERROR; + } + break; + + /* any text until end of line */ + case sw_status_text: + switch (ch) { + case CR: + state = sw_almost_done; + break; + case LF: + goto done; + } + break; + + /* end of status line */ + case sw_almost_done: + switch (ch) { + case LF: + goto done; + default: + return NGX_ERROR; + } + } + } + + b->pos = p; + ctx->state = state; + + return NGX_AGAIN; + +done: + + b->pos = p + 1; + ctx->state = sw_start; + + return NGX_OK; +} + + +static ngx_int_t +ngx_ssl_ocsp_process_headers(ngx_ssl_ocsp_ctx_t *ctx) +{ + ngx_int_t rc; + + ngx_log_debug0(NGX_LOG_DEBUG_EVENT, ctx->log, 0, + "ssl ocsp process headers"); + + for ( ;; ) { + rc = ngx_ssl_ocsp_parse_header_line(ctx); + + if (rc == NGX_OK) { + + ngx_log_debug4(NGX_LOG_DEBUG_EVENT, ctx->log, 0, + "ssl ocsp header \"%*s: %*s\"", + ctx->header_name_end - ctx->header_name_start, + ctx->header_name_start, + ctx->header_end - ctx->header_start, + ctx->header_start); + + /* TODO: honor Content-Length */ + + continue; + } + + if (rc == NGX_DONE) { + break; + } + + if (rc == NGX_AGAIN) { + return NGX_AGAIN; + } + + /* rc == NGX_ERROR */ + + ngx_log_error(NGX_LOG_ERR, ctx->log, 0, + "OCSP responder sent invalid response"); + + return NGX_ERROR; + } + + ctx->process = ngx_ssl_ocsp_process_body; + return ctx->process(ctx); +} + +static ngx_int_t +ngx_ssl_ocsp_parse_header_line(ngx_ssl_ocsp_ctx_t *ctx) +{ + u_char c, ch, *p; + enum { + sw_start = 0, + sw_name, + sw_space_before_value, + sw_value, + sw_space_after_value, + sw_almost_done, + sw_header_almost_done + } state; + + state = ctx->state; + + for (p = ctx->response->pos; p < ctx->response->last; p++) { + ch = *p; + +#if 0 + ngx_log_debug3(NGX_LOG_DEBUG_EVENT, ctx->log, 0, + "s:%d in:'%02Xd:%c'", state, ch, ch); +#endif + + switch (state) { + + /* first char */ + case sw_start: + + switch (ch) { + case CR: + ctx->header_end = p; + state = sw_header_almost_done; + break; + case LF: + ctx->header_end = p; + goto header_done; + default: + state = sw_name; + ctx->header_name_start = p; + + c = (u_char) (ch | 0x20); + if (c >= 'a' && c <= 'z') { + break; + } + + if (ch >= '0' && ch <= '9') { + break; + } + + return NGX_ERROR; + } + break; + + /* header name */ + case sw_name: + c = (u_char) (ch | 0x20); + if (c >= 'a' && c <= 'z') { + break; + } + + if (ch == ':') { + ctx->header_name_end = p; + state = sw_space_before_value; + break; + } + + if (ch == '-') { + break; + } + + if (ch >= '0' && ch <= '9') { + break; + } + + if (ch == CR) { + ctx->header_name_end = p; + ctx->header_start = p; + ctx->header_end = p; + state = sw_almost_done; + break; + } + + if (ch == LF) { + ctx->header_name_end = p; + ctx->header_start = p; + ctx->header_end = p; + goto done; + } + + return NGX_ERROR; + + /* space* before header value */ + case sw_space_before_value: + switch (ch) { + case ' ': + break; + case CR: + ctx->header_start = p; + ctx->header_end = p; + state = sw_almost_done; + break; + case LF: + ctx->header_start = p; + ctx->header_end = p; + goto done; + default: + ctx->header_start = p; + state = sw_value; + break; + } + break; + + /* header value */ + case sw_value: + switch (ch) { + case ' ': + ctx->header_end = p; + state = sw_space_after_value; + break; + case CR: + ctx->header_end = p; + state = sw_almost_done; + break; + case LF: + ctx->header_end = p; + goto done; + } + break; + + /* space* before end of header line */ + case sw_space_after_value: + switch (ch) { + case ' ': + break; + case CR: + state = sw_almost_done; + break; + case LF: + goto done; + default: + state = sw_value; + break; + } + break; + + /* end of header line */ + case sw_almost_done: + switch (ch) { + case LF: + goto done; + default: + return NGX_ERROR; + } + + /* end of header */ + case sw_header_almost_done: + switch (ch) { + case LF: + goto header_done; + default: + return NGX_ERROR; + } + } + } + + ctx->response->pos = p; + ctx->state = state; + + return NGX_AGAIN; + +done: + + ctx->response->pos = p + 1; + ctx->state = sw_start; + + return NGX_OK; + +header_done: + + ctx->response->pos = p + 1; + ctx->state = sw_start; + + return NGX_DONE; +} + + +static ngx_int_t +ngx_ssl_ocsp_process_body(ngx_ssl_ocsp_ctx_t *ctx) +{ + ngx_log_debug0(NGX_LOG_DEBUG_EVENT, ctx->log, 0, + "ssl ocsp process body"); + + if (ctx->done) { + ctx->handler(ctx); + return NGX_DONE; + } + + return NGX_AGAIN; +} + + +static u_char * +ngx_ssl_ocsp_log_error(ngx_log_t *log, u_char *buf, size_t len) +{ + u_char *p; + ngx_ssl_ocsp_ctx_t *ctx; + + p = buf; + + if (log->action) { + p = ngx_snprintf(buf, len, " while %s", log->action); + len -= p - buf; + } + + ctx = log->data; + + if (ctx) { + p = ngx_snprintf(p, len, ", responder: %V", &ctx->host); + } + + return p; +} + + +#else + + +ngx_int_t +ngx_ssl_stapling(ngx_conf_t *cf, ngx_ssl_t *ssl, ngx_str_t *responder, + ngx_str_t *file) +{ + ngx_log_error(NGX_LOG_WARN, ssl->log, 0, + "OCSP stapling not supported on this platform, ignored"); + + return NGX_OK; +} + +ngx_int_t +ngx_ssl_stapling_resolver(ngx_conf_t *cf, ngx_ssl_t *ssl, + ngx_resolver_t *resolver, ngx_msec_t resolver_timeout) +{ + return NGX_OK; +} + + +#endif diff --git a/src/http/modules/ngx_http_ssl_module.c b/src/http/modules/ngx_http_ssl_module.c --- a/src/http/modules/ngx_http_ssl_module.c +++ b/src/http/modules/ngx_http_ssl_module.c @@ -33,6 +33,8 @@ static char *ngx_http_ssl_enable(ngx_con static char *ngx_http_ssl_session_cache(ngx_conf_t *cf, ngx_command_t *cmd, void *conf); +static ngx_int_t ngx_http_ssl_init(ngx_conf_t *cf); + static ngx_conf_bitmask_t ngx_http_ssl_protocols[] = { { ngx_string("SSLv2"), NGX_SSL_SSLv2 }, @@ -173,13 +175,20 @@ static ngx_command_t ngx_http_ssl_comma offsetof(ngx_http_ssl_srv_conf_t, stapling_file), NULL }, + { ngx_string("ssl_stapling_responder"), + NGX_HTTP_MAIN_CONF|NGX_HTTP_SRV_CONF|NGX_CONF_TAKE1, + ngx_conf_set_str_slot, + NGX_HTTP_SRV_CONF_OFFSET, + offsetof(ngx_http_ssl_srv_conf_t, stapling_responder), + NULL }, + ngx_null_command }; static ngx_http_module_t ngx_http_ssl_module_ctx = { ngx_http_ssl_add_variables, /* preconfiguration */ - NULL, /* postconfiguration */ + ngx_http_ssl_init, /* postconfiguration */ NULL, /* create main configuration */ NULL, /* init main configuration */ @@ -351,6 +360,7 @@ ngx_http_ssl_create_srv_conf(ngx_conf_t * sscf->ciphers = { 0, NULL }; * sscf->shm_zone = NULL; * sscf->stapling_file = { 0, NULL }; + * sscf->stapling_responder = { 0, NULL }; */ sscf->enable = NGX_CONF_UNSET; @@ -415,6 +425,8 @@ ngx_http_ssl_merge_srv_conf(ngx_conf_t * ngx_conf_merge_value(conf->stapling, prev->stapling, 0); ngx_conf_merge_str_value(conf->stapling_file, prev->stapling_file, ""); + ngx_conf_merge_str_value(conf->stapling_responder, + prev->stapling_responder, ""); conf->ssl.log = cf->log; @@ -551,10 +563,15 @@ ngx_http_ssl_merge_srv_conf(ngx_conf_t * return NGX_CONF_ERROR; } - if (conf->stapling - && ngx_ssl_stapling(cf, &conf->ssl, &conf->stapling_file) != NGX_OK) - { - return NGX_CONF_ERROR; + if (conf->stapling) { + + if (ngx_ssl_stapling(cf, &conf->ssl, &conf->stapling_responder, + &conf->stapling_file) + != NGX_OK) + { + return NGX_CONF_ERROR; + } + } return NGX_CONF_OK; @@ -692,3 +709,37 @@ invalid: return NGX_CONF_ERROR; } + + +static ngx_int_t +ngx_http_ssl_init(ngx_conf_t *cf) +{ + ngx_uint_t s; + ngx_http_ssl_srv_conf_t *sscf; + ngx_http_core_loc_conf_t *clcf; + ngx_http_core_srv_conf_t **cscfp; + ngx_http_core_main_conf_t *cmcf; + + cmcf = ngx_http_conf_get_module_main_conf(cf, ngx_http_core_module); + cscfp = cmcf->servers.elts; + + for (s = 0; s < cmcf->servers.nelts; s++) { + + sscf = cscfp[s]->ctx->srv_conf[ngx_http_ssl_module.ctx_index]; + + if (!sscf->stapling) { + continue; + } + + clcf = cscfp[s]->ctx->loc_conf[ngx_http_core_module.ctx_index]; + + if (ngx_ssl_stapling_resolver(cf, &sscf->ssl, clcf->resolver, + clcf->resolver_timeout) + != NGX_OK) + { + return NGX_ERROR; + } + } + + return NGX_OK; +} diff --git a/src/http/modules/ngx_http_ssl_module.h b/src/http/modules/ngx_http_ssl_module.h --- a/src/http/modules/ngx_http_ssl_module.h +++ b/src/http/modules/ngx_http_ssl_module.h @@ -44,6 +44,7 @@ typedef struct { ngx_flag_t stapling; ngx_str_t stapling_file; + ngx_str_t stapling_responder; u_char *file; ngx_uint_t line; From mdounin at mdounin.ru Wed Sep 5 11:14:44 2012 From: mdounin at mdounin.ru (Maxim Dounin) Date: Wed, 05 Sep 2012 15:14:44 +0400 Subject: [PATCH 4 of 4] OCSP stapling: check Content-Type In-Reply-To: References: Message-ID: <8cb953fed894c2311945.1346843684@vm-bsd.mdounin.ru> # HG changeset patch # User Maxim Dounin # Date 1346843260 -14400 # Node ID 8cb953fed894c231194519b404a382458621e16a # Parent 73a3c526e072c5f5ed8f04a75e3498def5cd97e5 OCSP stapling: check Content-Type. This will result in better error message in case of incorrect response from OCSP responder: ... OCSP responder sent invalid "Content-Type" header: "text/plain" while requesting certificate status, responder: ... vs. ... d2i_OCSP_RESPONSE() failed (SSL: error:0D07209B:asn1 encoding routines:ASN1_get_object:too long error:0D068066:asn1 encoding routines:ASN1_CHECK_TLEN:bad object header error:0D07803A:asn1 encoding routines:ASN1_ITEM_EX_D2I:nested asn1 error) while requesting certificate status, responder: ... diff --git a/src/event/ngx_event_openssl_stapling.c b/src/event/ngx_event_openssl_stapling.c --- a/src/event/ngx_event_openssl_stapling.c +++ b/src/event/ngx_event_openssl_stapling.c @@ -1401,6 +1401,7 @@ done: static ngx_int_t ngx_ssl_ocsp_process_headers(ngx_ssl_ocsp_ctx_t *ctx) { + size_t len; ngx_int_t rc; ngx_log_debug0(NGX_LOG_DEBUG_EVENT, ctx->log, 0, @@ -1418,6 +1419,33 @@ ngx_ssl_ocsp_process_headers(ngx_ssl_ocs ctx->header_end - ctx->header_start, ctx->header_start); + len = ctx->header_name_end - ctx->header_name_start; + + if (len == sizeof("Content-Type") - 1 + && ngx_strncasecmp(ctx->header_name_start, + (u_char *) "Content-Type", + sizeof("Content-Type") - 1) + == 0) + { + len = ctx->header_end - ctx->header_start; + + if (len != sizeof("application/ocsp-response") - 1 + || ngx_strncasecmp(ctx->header_start, + (u_char *) "application/ocsp-response", + sizeof("application/ocsp-response") - 1) + != 0) + { + ngx_log_error(NGX_LOG_ERR, ctx->log, 0, + "OCSP responder sent invalid " + "\"Content-Type\" header: \"%*s\"", + ctx->header_end - ctx->header_start, + ctx->header_start); + return NGX_ERROR; + } + + continue; + } + /* TODO: honor Content-Length */ continue; From mdounin at mdounin.ru Wed Sep 5 15:06:48 2012 From: mdounin at mdounin.ru (mdounin at mdounin.ru) Date: Wed, 5 Sep 2012 15:06:48 +0000 Subject: [nginx] svn commit: r4834 - trunk/src/http Message-ID: <20120905150648.420FB3F9F2A@mail.nginx.com> Author: mdounin Date: 2012-09-05 15:06:47 +0000 (Wed, 05 Sep 2012) New Revision: 4834 URL: http://trac.nginx.org/nginx/changeset/4834/nginx Log: Write filter: replaced unneeded loop with one to free chains. Noted by Gabor Lekeny. Modified: trunk/src/http/ngx_http_write_filter_module.c Modified: trunk/src/http/ngx_http_write_filter_module.c =================================================================== --- trunk/src/http/ngx_http_write_filter_module.c 2012-09-03 12:55:50 UTC (rev 4833) +++ trunk/src/http/ngx_http_write_filter_module.c 2012-09-05 15:06:47 UTC (rev 4834) @@ -185,23 +185,19 @@ } if (size == 0 && !(c->buffered & NGX_LOWLEVEL_BUFFERED)) { - if (last) { + if (last || flush) { + for (cl = r->out; cl; /* void */) { + ln = cl; + cl = cl->next; + ngx_free_chain(r->pool, ln); + } + r->out = NULL; c->buffered &= ~NGX_HTTP_WRITE_BUFFERED; return NGX_OK; } - if (flush) { - do { - r->out = r->out->next; - } while (r->out); - - c->buffered &= ~NGX_HTTP_WRITE_BUFFERED; - - return NGX_OK; - } - ngx_log_error(NGX_LOG_ALERT, c->log, 0, "the http output chain is empty"); From vbart at nginx.com Fri Sep 7 13:24:46 2012 From: vbart at nginx.com (vbart at nginx.com) Date: Fri, 7 Sep 2012 13:24:46 +0000 Subject: [nginx] svn commit: r4835 - trunk/docs/html Message-ID: <20120907132447.06B9F3FA1DB@mail.nginx.com> Author: vbart Date: 2012-09-07 13:24:46 +0000 (Fri, 07 Sep 2012) New Revision: 4835 URL: http://trac.nginx.org/nginx/changeset/4835/nginx Log: Improved welcome page. The feature set of the new page: - HTML5 compliant; - Description of why the user is seeing the page and what his next step should be; - Links to official community and commercial support websites. Modified: trunk/docs/html/index.html Modified: trunk/docs/html/index.html =================================================================== --- trunk/docs/html/index.html 2012-09-05 15:06:47 UTC (rev 4834) +++ trunk/docs/html/index.html 2012-09-07 13:24:46 UTC (rev 4835) @@ -1,8 +1,25 @@ + Welcome to nginx! + - -

Welcome to nginx!

+ +

Welcome to nginx!

+

If you see this page, the nginx web server is successfully installed and +working. Further configuration is required.

+ +

For online documentation and support please refer to +nginx.org.
+Commercial support is available at +nginx.com.

+ +

Thank you for using nginx.

From mdounin at mdounin.ru Fri Sep 7 14:45:00 2012 From: mdounin at mdounin.ru (Maxim Dounin) Date: Fri, 7 Sep 2012 18:45:00 +0400 Subject: [PATCH] Random peer selection for implicit upstream defined by proxy_pass In-Reply-To: References: Message-ID: <20120907144500.GL40452@mdounin.ru> Hello! (sorry for late reply, I was busy with OCSP stapling and slowly catching up with resulting mail backlog now) On Tue, Aug 21, 2012 at 11:36:49AM -0700, Anton Jouline wrote: > hello, Nginx developers, > > submitting this patch for your review. It changes the behaviour of > peer selection > for the case, when the upstream is defined implicitly by proxy_pass > directive like this: > > resolver 172.16.0.23; > set $backend backend.servers.example.com; > proxy_pass http://$backend; [...] > It turns out that the round-robin algorithm actually does not work in > this case, > because of the way things are implemented in src/http/ngx_http_upstream.c. > The peer array is created and initialized on each request, even if a > cached result of > DNS query is returned by resolver. The upstream code has no knowledge of whether > the list of IP addresses changed or not. So what ends up happening is this: > Let's say "backend.servers.example.com" resolves to 6 IP addresses. Then the > first good one will be always used, and the rest will never be even looked at. Surely this is valid problem, and it needs fixing. Though I don't think we have to introduce another balancer module just for this. I think of two possible ways to resolve this: 1. Enforce random order of server in ngx_http_upstream_create_round_robin_peer(). Note it's only used to create upsteams on the fly, so it should be ok (and will lead to the same result as with your patch, though will be simplier). 2. Introduce round-robin in resolver code before we return cached response, much like it will be done by a normal DNS servers. This should also simplify other possible uses of a resolver. Not sure which one would be better though. Maxim Dounin From juce66 at gmail.com Sat Sep 8 08:19:49 2012 From: juce66 at gmail.com (Anton Jouline) Date: Sat, 8 Sep 2012 01:19:49 -0700 Subject: [PATCH] Random peer selection for implicit upstream defined by proxy_pass In-Reply-To: <20120907144500.GL40452@mdounin.ru> References: <20120907144500.GL40452@mdounin.ru> Message-ID: hello, Maxim. Thanks for your response! You are right that my patch looks unnecessarily complicated for what it actually does. Lots of code duplication too... After reading your message I started thinking again of a simpler way to accomplish the same task, and i think i found it. But first, let me provide feedback for your 2 proposals. #1 - Enforcing the random order in ngx_http_upstream_create_round_robin_peer() is part of the solution, but it's not enough, because of how the peer is selected in ngx_upstream_get_peer() function in the same file. So we need to modify that function also, but must be careful there, because it is used by the regular weighted round robin algorithm, when upstream is defined either implicitly by resolved-once proxy_pass, or explicitly by upstream block in the configuration file. We don't want to modify the existing round robin behaviour for those cases. I believe i found a way to do that (see the new patch at the end of the message) #2 - Introduce round-robin in resolver code. I think that would be good, but it does not really solve the issue of poor load distribution, because during the period of time when the DNS lookup is cached, still only 1 ip address would be used out of the list of many. It is true that when TTL expires, the next lookup would return a re-ordered list and perhaps a different ip would be in the first position, but for the duration of TTL there will still be no distribution of requests among the available servers. That's why i think #1 is a better alternative. Ok, and now goes the new version of the patch. It is much shorter and simpler! No new files needed, just a couple of small changes in ngx_http_upstream_round_robin.c. Tested successfully with versions 1.2.2, 1.2.3, and 1.3.5 Thanks. Anton. === diff -u -r nginx-1.2.2/src/http/ngx_http_upstream_round_robin.c nginx-1.2.2-mod/src/http/ngx_http_upstream_round_robin.c --- nginx-1.2.2/src/http/ngx_http_upstream_round_robin.c 2012-07-02 12:41:13.000000000 -0400 +++ nginx-1.2.2-mod/src/http/ngx_http_upstream_round_robin.c 2012-09-08 03:10:17.000000000 -0400 @@ -288,7 +288,7 @@ { u_char *p; size_t len; - ngx_uint_t i, n; + ngx_uint_t i, j, n; struct sockaddr_in *sin; ngx_http_upstream_rr_peers_t *peers; ngx_http_upstream_rr_peer_data_t *rrp; @@ -359,8 +359,11 @@ } } + /* start at random index */ + j = (int) ( (float)(peers->number) * (rand() / (RAND_MAX + 1.0))); + rrp->peers = peers; - rrp->current = 0; + rrp->current = j; if (rrp->peers->number <= 8 * sizeof(uintptr_t)) { rrp->tried = &rrp->data; @@ -504,7 +507,7 @@ time_t now; uintptr_t m; ngx_int_t total; - ngx_uint_t i, n; + ngx_uint_t i, j, n; ngx_http_upstream_rr_peer_t *peer, *best; now = ngx_time(); @@ -514,14 +517,15 @@ for (i = 0; i < rrp->peers->number; i++) { - n = i / (8 * sizeof(uintptr_t)); - m = (uintptr_t) 1 << i % (8 * sizeof(uintptr_t)); + j = (i + rrp->current) % rrp->peers->number; + n = j / (8 * sizeof(uintptr_t)); + m = (uintptr_t) 1 << j % (8 * sizeof(uintptr_t)); if (rrp->tried[n] & m) { continue; } - peer = &rrp->peers->peer[i]; + peer = &rrp->peers->peer[j]; if (peer->down) { continue; === On Fri, Sep 7, 2012 at 7:45 AM, Maxim Dounin wrote: > Hello! > > (sorry for late reply, I was busy with OCSP stapling and slowly > catching up with resulting mail backlog now) > > On Tue, Aug 21, 2012 at 11:36:49AM -0700, Anton Jouline wrote: > >> hello, Nginx developers, >> >> submitting this patch for your review. It changes the behaviour of >> peer selection >> for the case, when the upstream is defined implicitly by proxy_pass >> directive like this: >> >> resolver 172.16.0.23; >> set $backend backend.servers.example.com; >> proxy_pass http://$backend; > > [...] > >> It turns out that the round-robin algorithm actually does not work in >> this case, >> because of the way things are implemented in src/http/ngx_http_upstream.c. >> The peer array is created and initialized on each request, even if a >> cached result of >> DNS query is returned by resolver. The upstream code has no knowledge of whether >> the list of IP addresses changed or not. So what ends up happening is this: >> Let's say "backend.servers.example.com" resolves to 6 IP addresses. Then the >> first good one will be always used, and the rest will never be even looked at. > > Surely this is valid problem, and it needs fixing. Though I don't > think we have to introduce another balancer module just for this. > > I think of two possible ways to resolve this: > > 1. Enforce random order of server in ngx_http_upstream_create_round_robin_peer(). > Note it's only used to create upsteams on the fly, so it should be > ok (and will lead to the same result as with your patch, though > will be simplier). > > 2. Introduce round-robin in resolver code before we return cached > response, much like it will be done by a normal DNS servers. This > should also simplify other possible uses of a resolver. > > Not sure which one would be better though. > > Maxim Dounin > > _______________________________________________ > nginx-devel mailing list > nginx-devel at nginx.org > http://mailman.nginx.org/mailman/listinfo/nginx-devel From bekars at gmail.com Mon Sep 10 02:15:08 2012 From: bekars at gmail.com (bekars) Date: Mon, 10 Sep 2012 10:15:08 +0800 Subject: how to minify gzip format html page when used nginx as a reverse proxy? Message-ID: I want to get the goal: 1) nginx run as a reverse proxy, the web server gzip the html format page, and the client get the gzip page 2) nginx receive server gzip data then unzip it ==> minify html page ==> gzip minify html page to cache 3) client request the page and get the minify gzip page from nginx cache Questions: 1) How do I implement the idea in nginx structure? 2) Is there have any sample path to get the goal? Any help would be greatly appreciated. -------------- next part -------------- An HTML attachment was scrubbed... URL: From umu618 at hotmail.com Mon Sep 10 03:09:05 2012 From: umu618 at hotmail.com (UUM) Date: Mon, 10 Sep 2012 03:09:05 +0000 Subject: What's the plan about supporting google pagespeed? Message-ID: What's the plan about porting google?s mod_pagespeed to nginx, if these?s any? Sent from Windows Mail -------------- next part -------------- An HTML attachment was scrubbed... URL: From agentzh at gmail.com Mon Sep 10 05:03:24 2012 From: agentzh at gmail.com (agentzh) Date: Sun, 9 Sep 2012 22:03:24 -0700 Subject: [PATCH] Make ngx_http_upstream provide a way to expose errors after sending out the response header Message-ID: Hello! According to the current implementation of ngx_http_upstream, there is almost no way for 3rd-party output body filters and "post_subrequest" handlers (in the subrequest context) to know if there's any errors while ngx_http_upstream is processing the upstream response body after the response header is sent out (in both the buffered and non-buffered modes). For example, if 1. a read-timeout happens in the middle of the process of reading the upstream response body, 2. or the upstream connection is closed prematurely in the same situation, then ngx_http_upstream will just happily finalize the current request with the status code 0 (i.e., NGX_OK). This issue already affects at least our ngx_srcache and ngx_lua modules (originally reported by Bryan Alger). Here attaches a patch that makes ngx_http_upstream set r->headers_out.status to a new error status code to notify the outside world if there is a problem. Comments will be highly appreciated as always :) Thanks! -agentzh --- nginx-1.2.3/src/http/ngx_http_upstream.c 2012-08-06 10:34:08.000000000 -0700 +++ nginx-1.2.3-patched/src/http/ngx_http_upstream.c 2012-09-09 21:58:04.727761891 -0700 @@ -2383,7 +2383,7 @@ if (c->read->timedout) { ngx_connection_error(c, NGX_ETIMEDOUT, "upstream timed out"); - ngx_http_upstream_finalize_request(r, u, 0); + ngx_http_upstream_finalize_request(r, u, NGX_HTTP_GATEWAY_TIME_OUT); return; } @@ -2430,13 +2430,17 @@ if (u->busy_bufs == NULL) { if (u->length == 0 - || upstream->read->eof - || upstream->read->error) + || (upstream->read->eof && u->headers_in.content_length_n == -1)) { ngx_http_upstream_finalize_request(r, u, 0); return; } + if (upstream->read->eof || upstream->read->error) { + ngx_http_upstream_finalize_request(r, u, NGX_HTTP_BAD_GATEWAY); + return; + } + b->pos = b->start; b->last = b->start; } @@ -2710,7 +2714,16 @@ #if 0 ngx_http_busy_unlock(u->conf->busy_lock, &u->busy_lock); #endif - ngx_http_upstream_finalize_request(r, u, 0); + + if (p->upstream_done + || (p->upstream_eof && u->headers_in.content_length_n == -1)) + { + ngx_http_upstream_finalize_request(r, u, 0); + + } else { + ngx_http_upstream_finalize_request(r, u, NGX_HTTP_BAD_GATEWAY); + } + return; } } @@ -3073,6 +3086,13 @@ && rc != NGX_HTTP_REQUEST_TIME_OUT && (rc == NGX_ERROR || rc >= NGX_HTTP_SPECIAL_RESPONSE)) { + if (rc == NGX_ERROR) { + r->headers_out.status = NGX_HTTP_INTERNAL_SERVER_ERROR; + + } else { + r->headers_out.status = rc; + } + rc = 0; } -------------- next part -------------- A non-text attachment was scrubbed... Name: nginx-1.2.3-nonbuffered-upstream-truncation.patch Type: application/octet-stream Size: 2021 bytes Desc: not available URL: From mdounin at mdounin.ru Mon Sep 10 10:35:40 2012 From: mdounin at mdounin.ru (Maxim Dounin) Date: Mon, 10 Sep 2012 14:35:40 +0400 Subject: [PATCH] Make ngx_http_upstream provide a way to expose errors after sending out the response header In-Reply-To: References: Message-ID: <20120910103540.GN40452@mdounin.ru> Hello! On Sun, Sep 09, 2012 at 10:03:24PM -0700, agentzh wrote: > Hello! > > According to the current implementation of ngx_http_upstream, there is > almost no way for 3rd-party output body filters and "post_subrequest" > handlers (in the subrequest context) to know if there's any errors > while ngx_http_upstream is processing the upstream response body after > the response header is sent out (in both the buffered and non-buffered > modes). > > For example, if > > 1. a read-timeout happens in the middle of the process of reading the > upstream response body, > 2. or the upstream connection is closed prematurely in the same situation, > > then ngx_http_upstream will just happily finalize the current request > with the status code 0 (i.e., NGX_OK). This issue already affects at > least our ngx_srcache and ngx_lua modules (originally reported by > Bryan Alger). > > Here attaches a patch that makes ngx_http_upstream set > r->headers_out.status to a new error status code to notify the outside > world if there is a problem. Comments will be highly appreciated as > always :) I don't like the idea of changing r->headers_out.status here. It's really incorrect and confusing: r->headers_out.status is a status code of a response, and it's way too late to change it when we've found an error during sending the response body. > > Thanks! > -agentzh > > --- nginx-1.2.3/src/http/ngx_http_upstream.c 2012-08-06 10:34:08.000000000 -0700 > +++ nginx-1.2.3-patched/src/http/ngx_http_upstream.c 2012-09-09 > 21:58:04.727761891 -0700 > @@ -2383,7 +2383,7 @@ Just a side note: please use -p when submitting patches. > > if (c->read->timedout) { > ngx_connection_error(c, NGX_ETIMEDOUT, "upstream timed out"); > - ngx_http_upstream_finalize_request(r, u, 0); > + ngx_http_upstream_finalize_request(r, u, NGX_HTTP_GATEWAY_TIME_OUT); > return; > } > > @@ -2430,13 +2430,17 @@ > if (u->busy_bufs == NULL) { > > if (u->length == 0 > - || upstream->read->eof > - || upstream->read->error) > + || (upstream->read->eof && > u->headers_in.content_length_n == -1)) Note: you can't be sure that eof with u->headers_in.content_length_n == -1 is ok. E.g. with chunked encoding used it's certainly not unless u->length == 0 as well. While content_length_n check is already used in cache and store cases in ngx_http_upstream_process_request(), we might want to get rid of it instead of reintroducing it in various other places. > { > ngx_http_upstream_finalize_request(r, u, 0); > return; > } > > + if (upstream->read->eof || upstream->read->error) { > + ngx_http_upstream_finalize_request(r, u, > NGX_HTTP_BAD_GATEWAY); > + return; > + } > + Just a side note: resulting code looks not very readable for me. > b->pos = b->start; > b->last = b->start; > } > @@ -2710,7 +2714,16 @@ > #if 0 > ngx_http_busy_unlock(u->conf->busy_lock, &u->busy_lock); > #endif > - ngx_http_upstream_finalize_request(r, u, 0); > + > + if (p->upstream_done > + || (p->upstream_eof && u->headers_in.content_length_n == -1)) > + { > + ngx_http_upstream_finalize_request(r, u, 0); > + > + } else { > + ngx_http_upstream_finalize_request(r, u, NGX_HTTP_BAD_GATEWAY); > + } > + Same here (well, two above comments about readability and content_length_n apply). > return; > } > } > @@ -3073,6 +3086,13 @@ > && rc != NGX_HTTP_REQUEST_TIME_OUT > && (rc == NGX_ERROR || rc >= NGX_HTTP_SPECIAL_RESPONSE)) > { > + if (rc == NGX_ERROR) { > + r->headers_out.status = NGX_HTTP_INTERNAL_SERVER_ERROR; > + > + } else { > + r->headers_out.status = rc; > + } > + > rc = 0; > } See above. Additionally, I think major problem here isn't subrequest-in-memory special case, but sending buf with "last" set, which in some situations may result in truncated response being considered full by a downstream. Maxim Dounin From mdounin at mdounin.ru Mon Sep 10 16:43:50 2012 From: mdounin at mdounin.ru (mdounin at mdounin.ru) Date: Mon, 10 Sep 2012 16:43:50 +0000 Subject: [nginx] svn commit: r4836 - trunk/src/http/modules Message-ID: <20120910164350.717443FA9A8@mail.nginx.com> Author: mdounin Date: 2012-09-10 16:43:49 +0000 (Mon, 10 Sep 2012) New Revision: 4836 URL: http://trac.nginx.org/nginx/changeset/4836/nginx Log: Memcached: memcached_gzip_flag directive. This directive allows to test desired flag as returned by memcached and sets Content-Encoding to gzip if one found. This is reimplementation of patch by Tomash Brechko as available on http://openhack.ru/. It should be a bit more correct though (at least I think so). In particular, it doesn't try to detect if we are able to gunzip data, but instead just sets correct Content-Encoding. Modified: trunk/src/http/modules/ngx_http_memcached_module.c Modified: trunk/src/http/modules/ngx_http_memcached_module.c =================================================================== --- trunk/src/http/modules/ngx_http_memcached_module.c 2012-09-07 13:24:46 UTC (rev 4835) +++ trunk/src/http/modules/ngx_http_memcached_module.c 2012-09-10 16:43:49 UTC (rev 4836) @@ -13,6 +13,7 @@ typedef struct { ngx_http_upstream_conf_t upstream; ngx_int_t index; + ngx_uint_t gzip_flag; } ngx_http_memcached_loc_conf_t; @@ -101,6 +102,13 @@ offsetof(ngx_http_memcached_loc_conf_t, upstream.next_upstream), &ngx_http_memcached_next_upstream_masks }, + { ngx_string("memcached_gzip_flag"), + NGX_HTTP_MAIN_CONF|NGX_HTTP_SRV_CONF|NGX_HTTP_LOC_CONF|NGX_CONF_TAKE1, + ngx_conf_set_num_slot, + NGX_HTTP_LOC_CONF_OFFSET, + offsetof(ngx_http_memcached_loc_conf_t, gzip_flag), + NULL }, + ngx_null_command }; @@ -281,10 +289,13 @@ static ngx_int_t ngx_http_memcached_process_header(ngx_http_request_t *r) { - u_char *p, *len; - ngx_str_t line; - ngx_http_upstream_t *u; - ngx_http_memcached_ctx_t *ctx; + u_char *p, *start; + ngx_str_t line; + ngx_uint_t flags; + ngx_table_elt_t *h; + ngx_http_upstream_t *u; + ngx_http_memcached_ctx_t *ctx; + ngx_http_memcached_loc_conf_t *mlcf; u = r->upstream; @@ -309,6 +320,7 @@ p = u->buffer.pos; ctx = ngx_http_get_module_ctx(r, ngx_http_memcached_module); + mlcf = ngx_http_get_module_loc_conf(r, ngx_http_memcached_module); if (ngx_strncmp(p, "VALUE ", sizeof("VALUE ") - 1) == 0) { @@ -329,23 +341,56 @@ goto no_valid; } - /* skip flags */ + /* flags */ + start = p; + while (*p) { if (*p++ == ' ') { - goto length; + if (mlcf->gzip_flag) { + goto flags; + } else { + goto length; + } } } goto no_valid; + flags: + + flags = ngx_atoi(start, p - start - 1); + + if (flags == (ngx_uint_t) NGX_ERROR) { + ngx_log_error(NGX_LOG_ERR, r->connection->log, 0, + "memcached sent invalid flags in response \"%V\" " + "for key \"%V\"", + &line, &ctx->key); + return NGX_HTTP_UPSTREAM_INVALID_HEADER; + } + + if (flags & mlcf->gzip_flag) { + h = ngx_list_push(&r->headers_out.headers); + if (h == NULL) { + return NGX_ERROR; + } + + h->hash = 1; + h->key.len = sizeof("Content-Encoding") - 1; + h->key.data = (u_char *) "Content-Encoding"; + h->value.len = sizeof("gzip") - 1; + h->value.data = (u_char *) "gzip"; + + r->headers_out.content_encoding = h; + } + length: - len = p; + start = p; while (*p && *p++ != CR) { /* void */ } - u->headers_in.content_length_n = ngx_atoof(len, p - len - 1); + u->headers_in.content_length_n = ngx_atoof(start, p - start - 1); if (u->headers_in.content_length_n == -1) { ngx_log_error(NGX_LOG_ERR, r->connection->log, 0, "memcached sent invalid length in response \"%V\" " @@ -550,6 +595,7 @@ conf->upstream.pass_request_body = 0; conf->index = NGX_CONF_UNSET; + conf->gzip_flag = NGX_CONF_UNSET_UINT; return conf; } @@ -593,6 +639,8 @@ conf->index = prev->index; } + ngx_conf_merge_uint_value(conf->gzip_flag, prev->gzip_flag, 0); + return NGX_CONF_OK; } From mdounin at mdounin.ru Mon Sep 10 16:48:25 2012 From: mdounin at mdounin.ru (mdounin at mdounin.ru) Date: Mon, 10 Sep 2012 16:48:25 +0000 Subject: [nginx] svn commit: r4837 - trunk/src/http/modules Message-ID: <20120910164825.786FF3F9C14@mail.nginx.com> Author: mdounin Date: 2012-09-10 16:48:25 +0000 (Mon, 10 Sep 2012) New Revision: 4837 URL: http://trac.nginx.org/nginx/changeset/4837/nginx Log: Gzip static: "always" parameter in "gzip_static" directive. With "always" gzip static returns gzipped content in all cases, without checking if client supports it. It is useful if there are no uncompressed files on disk anyway. Modified: trunk/src/http/modules/ngx_http_gzip_static_module.c Modified: trunk/src/http/modules/ngx_http_gzip_static_module.c =================================================================== --- trunk/src/http/modules/ngx_http_gzip_static_module.c 2012-09-10 16:43:49 UTC (rev 4836) +++ trunk/src/http/modules/ngx_http_gzip_static_module.c 2012-09-10 16:48:25 UTC (rev 4837) @@ -10,8 +10,13 @@ #include +#define NGX_HTTP_GZIP_STATIC_OFF 0 +#define NGX_HTTP_GZIP_STATIC_ON 1 +#define NGX_HTTP_GZIP_STATIC_ALWAYS 2 + + typedef struct { - ngx_flag_t enable; + ngx_uint_t enable; } ngx_http_gzip_static_conf_t; @@ -22,14 +27,22 @@ static ngx_int_t ngx_http_gzip_static_init(ngx_conf_t *cf); +static ngx_conf_enum_t ngx_http_gzip_static[] = { + { ngx_string("off"), NGX_HTTP_GZIP_STATIC_OFF }, + { ngx_string("on"), NGX_HTTP_GZIP_STATIC_ON }, + { ngx_string("always"), NGX_HTTP_GZIP_STATIC_ALWAYS }, + { ngx_null_string, 0 } +}; + + static ngx_command_t ngx_http_gzip_static_commands[] = { { ngx_string("gzip_static"), NGX_HTTP_MAIN_CONF|NGX_HTTP_SRV_CONF|NGX_HTTP_LOC_CONF|NGX_CONF_FLAG, - ngx_conf_set_flag_slot, + ngx_conf_set_enum_slot, NGX_HTTP_LOC_CONF_OFFSET, offsetof(ngx_http_gzip_static_conf_t, enable), - NULL }, + &ngx_http_gzip_static }, ngx_null_command }; @@ -92,12 +105,18 @@ gzcf = ngx_http_get_module_loc_conf(r, ngx_http_gzip_static_module); - if (!gzcf->enable) { + if (gzcf->enable == NGX_HTTP_GZIP_STATIC_OFF) { return NGX_DECLINED; } - rc = ngx_http_gzip_ok(r); + if (gzcf->enable == NGX_HTTP_GZIP_STATIC_ON) { + rc = ngx_http_gzip_ok(r); + } else { + /* always */ + rc = NGX_OK; + } + clcf = ngx_http_get_module_loc_conf(r, ngx_http_core_module); if (!clcf->gzip_vary && rc != NGX_OK) { @@ -169,10 +188,12 @@ return NGX_DECLINED; } - r->gzip_vary = 1; + if (gzcf->enable == NGX_HTTP_GZIP_STATIC_ON) { + r->gzip_vary = 1; - if (rc != NGX_OK) { - return NGX_DECLINED; + if (rc != NGX_OK) { + return NGX_DECLINED; + } } ngx_log_debug1(NGX_LOG_DEBUG_HTTP, log, 0, "http static fd: %d", of.fd); @@ -274,7 +295,7 @@ return NULL; } - conf->enable = NGX_CONF_UNSET; + conf->enable = NGX_CONF_UNSET_UINT; return conf; } @@ -286,7 +307,8 @@ ngx_http_gzip_static_conf_t *prev = parent; ngx_http_gzip_static_conf_t *conf = child; - ngx_conf_merge_value(conf->enable, prev->enable, 0); + ngx_conf_merge_uint_value(conf->enable, prev->enable, + NGX_HTTP_GZIP_STATIC_OFF); return NGX_CONF_OK; } From mdounin at mdounin.ru Mon Sep 10 16:52:48 2012 From: mdounin at mdounin.ru (mdounin at mdounin.ru) Date: Mon, 10 Sep 2012 16:52:48 +0000 Subject: [nginx] svn commit: r4838 - in trunk: auto src/http/modules Message-ID: <20120910165248.82EE13F9C44@mail.nginx.com> Author: mdounin Date: 2012-09-10 16:52:47 +0000 (Mon, 10 Sep 2012) New Revision: 4838 URL: http://trac.nginx.org/nginx/changeset/4838/nginx Log: Gunzip filter import. Added: trunk/src/http/modules/ngx_http_gunzip_filter_module.c Modified: trunk/auto/modules trunk/auto/options trunk/auto/sources Modified: trunk/auto/modules =================================================================== --- trunk/auto/modules 2012-09-10 16:48:25 UTC (rev 4837) +++ trunk/auto/modules 2012-09-10 16:52:47 UTC (rev 4838) @@ -109,6 +109,7 @@ # ngx_http_image_filter # ngx_http_sub_filter # ngx_http_addition_filter +# ngx_http_gunzip_filter # ngx_http_userid_filter # ngx_http_headers_filter # ngx_http_copy_filter @@ -166,6 +167,13 @@ HTTP_SRCS="$HTTP_SRCS $HTTP_ADDITION_SRCS" fi +if [ $HTTP_GUNZIP = YES ]; then + have=NGX_HTTP_GZIP . auto/have + USE_ZLIB=YES + HTTP_FILTER_MODULES="$HTTP_FILTER_MODULES $HTTP_GUNZIP_FILTER_MODULE" + HTTP_SRCS="$HTTP_SRCS $HTTP_GUNZIP_SRCS" +fi + if [ $HTTP_USERID = YES ]; then HTTP_FILTER_MODULES="$HTTP_FILTER_MODULES $HTTP_USERID_FILTER_MODULE" HTTP_SRCS="$HTTP_SRCS $HTTP_USERID_SRCS" Modified: trunk/auto/options =================================================================== --- trunk/auto/options 2012-09-10 16:48:25 UTC (rev 4837) +++ trunk/auto/options 2012-09-10 16:52:47 UTC (rev 4838) @@ -94,6 +94,7 @@ HTTP_DEGRADATION=NO HTTP_FLV=NO HTTP_MP4=NO +HTTP_GUNZIP=NO HTTP_GZIP_STATIC=NO HTTP_UPSTREAM_IP_HASH=YES HTTP_UPSTREAM_LEAST_CONN=YES @@ -210,6 +211,7 @@ --with-http_dav_module) HTTP_DAV=YES ;; --with-http_flv_module) HTTP_FLV=YES ;; --with-http_mp4_module) HTTP_MP4=YES ;; + --with-http_gunzip_module) HTTP_GUNZIP=YES ;; --with-http_gzip_static_module) HTTP_GZIP_STATIC=YES ;; --with-http_random_index_module) HTTP_RANDOM_INDEX=YES ;; --with-http_secure_link_module) HTTP_SECURE_LINK=YES ;; @@ -356,6 +358,7 @@ --with-http_dav_module enable ngx_http_dav_module --with-http_flv_module enable ngx_http_flv_module --with-http_mp4_module enable ngx_http_mp4_module + --with-http_gunzip_module enable ngx_http_gunzip_module --with-http_gzip_static_module enable ngx_http_gzip_static_module --with-http_random_index_module enable ngx_http_random_index_module --with-http_secure_link_module enable ngx_http_secure_link_module Modified: trunk/auto/sources =================================================================== --- trunk/auto/sources 2012-09-10 16:48:25 UTC (rev 4837) +++ trunk/auto/sources 2012-09-10 16:52:47 UTC (rev 4838) @@ -331,6 +331,10 @@ HTTP_GZIP_SRCS=src/http/modules/ngx_http_gzip_filter_module.c +HTTP_GUNZIP_FILTER_MODULE=ngx_http_gunzip_filter_module +HTTP_GUNZIP_SRCS=src/http/modules/ngx_http_gunzip_filter_module.c + + HTTP_SSI_FILTER_MODULE=ngx_http_ssi_filter_module HTTP_SSI_DEPS=src/http/modules/ngx_http_ssi_filter_module.h HTTP_SSI_SRCS=src/http/modules/ngx_http_ssi_filter_module.c Added: trunk/src/http/modules/ngx_http_gunzip_filter_module.c =================================================================== --- trunk/src/http/modules/ngx_http_gunzip_filter_module.c (rev 0) +++ trunk/src/http/modules/ngx_http_gunzip_filter_module.c 2012-09-10 16:52:47 UTC (rev 4838) @@ -0,0 +1,677 @@ + +/* + * Copyright (C) Igor Sysoev + * Copyright (C) Maxim Dounin + * Copyright (C) Nginx, Inc. + */ + + +#include +#include +#include +#include + +#include + + +typedef struct { + ngx_flag_t enable; + ngx_bufs_t bufs; +} ngx_http_gunzip_conf_t; + + +typedef struct { + ngx_chain_t *in; + ngx_chain_t *free; + ngx_chain_t *busy; + ngx_chain_t *out; + ngx_chain_t **last_out; + + ngx_buf_t *in_buf; + ngx_buf_t *out_buf; + ngx_int_t bufs; + + unsigned started:1; + unsigned flush:4; + unsigned redo:1; + unsigned done:1; + unsigned nomem:1; + + z_stream zstream; + ngx_http_request_t *request; +} ngx_http_gunzip_ctx_t; + + +static ngx_int_t ngx_http_gunzip_filter_inflate_start(ngx_http_request_t *r, + ngx_http_gunzip_ctx_t *ctx); +static ngx_int_t ngx_http_gunzip_filter_add_data(ngx_http_request_t *r, + ngx_http_gunzip_ctx_t *ctx); +static ngx_int_t ngx_http_gunzip_filter_get_buf(ngx_http_request_t *r, + ngx_http_gunzip_ctx_t *ctx); +static ngx_int_t ngx_http_gunzip_filter_inflate(ngx_http_request_t *r, + ngx_http_gunzip_ctx_t *ctx); +static ngx_int_t ngx_http_gunzip_filter_inflate_end(ngx_http_request_t *r, + ngx_http_gunzip_ctx_t *ctx); + +static void *ngx_http_gunzip_filter_alloc(void *opaque, u_int items, + u_int size); +static void ngx_http_gunzip_filter_free(void *opaque, void *address); + +static ngx_int_t ngx_http_gunzip_filter_init(ngx_conf_t *cf); +static void *ngx_http_gunzip_create_conf(ngx_conf_t *cf); +static char *ngx_http_gunzip_merge_conf(ngx_conf_t *cf, + void *parent, void *child); + + +static ngx_command_t ngx_http_gunzip_filter_commands[] = { + + { ngx_string("gunzip"), + NGX_HTTP_MAIN_CONF|NGX_HTTP_SRV_CONF|NGX_HTTP_LOC_CONF|NGX_CONF_FLAG, + ngx_conf_set_flag_slot, + NGX_HTTP_LOC_CONF_OFFSET, + offsetof(ngx_http_gunzip_conf_t, enable), + NULL }, + + { ngx_string("gunzip_buffers"), + NGX_HTTP_MAIN_CONF|NGX_HTTP_SRV_CONF|NGX_HTTP_LOC_CONF|NGX_CONF_TAKE2, + ngx_conf_set_bufs_slot, + NGX_HTTP_LOC_CONF_OFFSET, + offsetof(ngx_http_gunzip_conf_t, bufs), + NULL }, + + ngx_null_command +}; + + +static ngx_http_module_t ngx_http_gunzip_filter_module_ctx = { + NULL, /* preconfiguration */ + ngx_http_gunzip_filter_init, /* postconfiguration */ + + NULL, /* create main configuration */ + NULL, /* init main configuration */ + + NULL, /* create server configuration */ + NULL, /* merge server configuration */ + + ngx_http_gunzip_create_conf, /* create location configuration */ + ngx_http_gunzip_merge_conf /* merge location configuration */ +}; + + +ngx_module_t ngx_http_gunzip_filter_module = { + NGX_MODULE_V1, + &ngx_http_gunzip_filter_module_ctx, /* module context */ + ngx_http_gunzip_filter_commands, /* module directives */ + NGX_HTTP_MODULE, /* module type */ + NULL, /* init master */ + NULL, /* init module */ + NULL, /* init process */ + NULL, /* init thread */ + NULL, /* exit thread */ + NULL, /* exit process */ + NULL, /* exit master */ + NGX_MODULE_V1_PADDING +}; + + +static ngx_http_output_header_filter_pt ngx_http_next_header_filter; +static ngx_http_output_body_filter_pt ngx_http_next_body_filter; + + +static ngx_int_t +ngx_http_gunzip_header_filter(ngx_http_request_t *r) +{ + ngx_http_gunzip_ctx_t *ctx; + ngx_http_gunzip_conf_t *conf; + + conf = ngx_http_get_module_loc_conf(r, ngx_http_gunzip_filter_module); + + /* TODO support multiple content-codings */ + /* TODO always gunzip - due to configuration or module request */ + /* TODO ignore content encoding? */ + + if (!conf->enable + || r->headers_out.content_encoding == NULL + || r->headers_out.content_encoding->value.len != 4 + || ngx_strncasecmp(r->headers_out.content_encoding->value.data, + (u_char *) "gzip", 4) != 0) + { + return ngx_http_next_header_filter(r); + } + + r->gzip_vary = 1; + + if (!r->gzip_tested) { + if (ngx_http_gzip_ok(r) == NGX_OK) { + return ngx_http_next_header_filter(r); + } + + } else if (!r->gzip_ok) { + return ngx_http_next_header_filter(r); + } + + ctx = ngx_pcalloc(r->pool, sizeof(ngx_http_gunzip_ctx_t)); + if (ctx == NULL) { + return NGX_ERROR; + } + + ngx_http_set_ctx(r, ctx, ngx_http_gunzip_filter_module); + + ctx->request = r; + + r->filter_need_in_memory = 1; + + r->headers_out.content_encoding->hash = 0; + r->headers_out.content_encoding = NULL; + + ngx_http_clear_content_length(r); + ngx_http_clear_accept_ranges(r); + + return ngx_http_next_header_filter(r); +} + + +static ngx_int_t +ngx_http_gunzip_body_filter(ngx_http_request_t *r, ngx_chain_t *in) +{ + int rc; + ngx_chain_t *cl; + ngx_http_gunzip_ctx_t *ctx; + + ctx = ngx_http_get_module_ctx(r, ngx_http_gunzip_filter_module); + + if (ctx == NULL || ctx->done) { + return ngx_http_next_body_filter(r, in); + } + + ngx_log_debug0(NGX_LOG_DEBUG_HTTP, r->connection->log, 0, + "http gunzip filter"); + + if (!ctx->started) { + if (ngx_http_gunzip_filter_inflate_start(r, ctx) != NGX_OK) { + goto failed; + } + } + + if (in) { + if (ngx_chain_add_copy(r->pool, &ctx->in, in) != NGX_OK) { + goto failed; + } + } + + if (ctx->nomem) { + + /* flush busy buffers */ + + if (ngx_http_next_body_filter(r, NULL) == NGX_ERROR) { + goto failed; + } + + cl = NULL; + + ngx_chain_update_chains(r->pool, &ctx->free, &ctx->busy, &cl, + (ngx_buf_tag_t) &ngx_http_gunzip_filter_module); + ctx->nomem = 0; + } + + for ( ;; ) { + + /* cycle while we can write to a client */ + + for ( ;; ) { + + /* cycle while there is data to feed zlib and ... */ + + rc = ngx_http_gunzip_filter_add_data(r, ctx); + + if (rc == NGX_DECLINED) { + break; + } + + if (rc == NGX_AGAIN) { + continue; + } + + + /* ... there are buffers to write zlib output */ + + rc = ngx_http_gunzip_filter_get_buf(r, ctx); + + if (rc == NGX_DECLINED) { + break; + } + + if (rc == NGX_ERROR) { + goto failed; + } + + rc = ngx_http_gunzip_filter_inflate(r, ctx); + + if (rc == NGX_OK) { + break; + } + + if (rc == NGX_ERROR) { + goto failed; + } + + /* rc == NGX_AGAIN */ + } + + if (ctx->out == NULL) { + return ctx->busy ? NGX_AGAIN : NGX_OK; + } + + rc = ngx_http_next_body_filter(r, ctx->out); + + if (rc == NGX_ERROR) { + goto failed; + } + + ngx_chain_update_chains(r->pool, &ctx->free, &ctx->busy, &ctx->out, + (ngx_buf_tag_t) &ngx_http_gunzip_filter_module); + ctx->last_out = &ctx->out; + + ngx_log_debug1(NGX_LOG_DEBUG_HTTP, r->connection->log, 0, + "gunzip out: %p", ctx->out); + + ctx->nomem = 0; + + if (ctx->done) { + return rc; + } + } + + /* unreachable */ + +failed: + + ctx->done = 1; + + return NGX_ERROR; +} + + +static ngx_int_t +ngx_http_gunzip_filter_inflate_start(ngx_http_request_t *r, + ngx_http_gunzip_ctx_t *ctx) +{ + int rc; + + ctx->zstream.next_in = Z_NULL; + ctx->zstream.avail_in = 0; + + ctx->zstream.zalloc = ngx_http_gunzip_filter_alloc; + ctx->zstream.zfree = ngx_http_gunzip_filter_free; + ctx->zstream.opaque = ctx; + + /* windowBits +16 to decode gzip, zlib 1.2.0.4+ */ + rc = inflateInit2(&ctx->zstream, MAX_WBITS + 16); + + if (rc != Z_OK) { + ngx_log_error(NGX_LOG_ALERT, r->connection->log, 0, + "inflateInit2() failed: %d", rc); + return NGX_ERROR; + } + + ctx->started = 1; + + ctx->last_out = &ctx->out; + ctx->flush = Z_NO_FLUSH; + + return NGX_OK; +} + + +static ngx_int_t +ngx_http_gunzip_filter_add_data(ngx_http_request_t *r, + ngx_http_gunzip_ctx_t *ctx) +{ + if (ctx->zstream.avail_in || ctx->flush != Z_NO_FLUSH || ctx->redo) { + return NGX_OK; + } + + ngx_log_debug1(NGX_LOG_DEBUG_HTTP, r->connection->log, 0, + "gunzip in: %p", ctx->in); + + if (ctx->in == NULL) { + return NGX_DECLINED; + } + + ctx->in_buf = ctx->in->buf; + ctx->in = ctx->in->next; + + ctx->zstream.next_in = ctx->in_buf->pos; + ctx->zstream.avail_in = ctx->in_buf->last - ctx->in_buf->pos; + + ngx_log_debug3(NGX_LOG_DEBUG_HTTP, r->connection->log, 0, + "gunzip in_buf:%p ni:%p ai:%ud", + ctx->in_buf, + ctx->zstream.next_in, ctx->zstream.avail_in); + + if (ctx->in_buf->last_buf || ctx->in_buf->last_in_chain) { + ctx->flush = Z_FINISH; + + } else if (ctx->in_buf->flush) { + ctx->flush = Z_SYNC_FLUSH; + + } else if (ctx->zstream.avail_in == 0) { + /* ctx->flush == Z_NO_FLUSH */ + return NGX_AGAIN; + } + + return NGX_OK; +} + + +static ngx_int_t +ngx_http_gunzip_filter_get_buf(ngx_http_request_t *r, + ngx_http_gunzip_ctx_t *ctx) +{ + ngx_http_gunzip_conf_t *conf; + + if (ctx->zstream.avail_out) { + return NGX_OK; + } + + conf = ngx_http_get_module_loc_conf(r, ngx_http_gunzip_filter_module); + + if (ctx->free) { + ctx->out_buf = ctx->free->buf; + ctx->free = ctx->free->next; + + ctx->out_buf->flush = 0; + + } else if (ctx->bufs < conf->bufs.num) { + + ctx->out_buf = ngx_create_temp_buf(r->pool, conf->bufs.size); + if (ctx->out_buf == NULL) { + return NGX_ERROR; + } + + ctx->out_buf->tag = (ngx_buf_tag_t) &ngx_http_gunzip_filter_module; + ctx->out_buf->recycled = 1; + ctx->bufs++; + + } else { + ctx->nomem = 1; + return NGX_DECLINED; + } + + ctx->zstream.next_out = ctx->out_buf->pos; + ctx->zstream.avail_out = conf->bufs.size; + + return NGX_OK; +} + + +static ngx_int_t +ngx_http_gunzip_filter_inflate(ngx_http_request_t *r, + ngx_http_gunzip_ctx_t *ctx) +{ + int rc; + ngx_buf_t *b; + ngx_chain_t *cl; + + ngx_log_debug6(NGX_LOG_DEBUG_HTTP, r->connection->log, 0, + "inflate in: ni:%p no:%p ai:%ud ao:%ud fl:%d redo:%d", + ctx->zstream.next_in, ctx->zstream.next_out, + ctx->zstream.avail_in, ctx->zstream.avail_out, + ctx->flush, ctx->redo); + + rc = inflate(&ctx->zstream, ctx->flush); + + if (rc != Z_OK && rc != Z_STREAM_END && rc != Z_BUF_ERROR) { + ngx_log_error(NGX_LOG_ALERT, r->connection->log, 0, + "inflate() failed: %d, %d", ctx->flush, rc); + return NGX_ERROR; + } + + ngx_log_debug5(NGX_LOG_DEBUG_HTTP, r->connection->log, 0, + "inflate out: ni:%p no:%p ai:%ud ao:%ud rc:%d", + ctx->zstream.next_in, ctx->zstream.next_out, + ctx->zstream.avail_in, ctx->zstream.avail_out, + rc); + + ngx_log_debug2(NGX_LOG_DEBUG_HTTP, r->connection->log, 0, + "gunzip in_buf:%p pos:%p", + ctx->in_buf, ctx->in_buf->pos); + + if (ctx->zstream.next_in) { + ctx->in_buf->pos = ctx->zstream.next_in; + + if (ctx->zstream.avail_in == 0) { + ctx->zstream.next_in = NULL; + } + } + + ctx->out_buf->last = ctx->zstream.next_out; + + if (ctx->zstream.avail_out == 0) { + + /* zlib wants to output some more data */ + + cl = ngx_alloc_chain_link(r->pool); + if (cl == NULL) { + return NGX_ERROR; + } + + cl->buf = ctx->out_buf; + cl->next = NULL; + *ctx->last_out = cl; + ctx->last_out = &cl->next; + + ctx->redo = 1; + + return NGX_AGAIN; + } + + ctx->redo = 0; + + if (ctx->flush == Z_SYNC_FLUSH) { + + ctx->flush = Z_NO_FLUSH; + + cl = ngx_alloc_chain_link(r->pool); + if (cl == NULL) { + return NGX_ERROR; + } + + b = ctx->out_buf; + + if (ngx_buf_size(b) == 0) { + + b = ngx_calloc_buf(ctx->request->pool); + if (b == NULL) { + return NGX_ERROR; + } + + } else { + ctx->zstream.avail_out = 0; + } + + b->flush = 1; + + cl->buf = b; + cl->next = NULL; + *ctx->last_out = cl; + ctx->last_out = &cl->next; + + return NGX_OK; + } + + if (rc == Z_STREAM_END && ctx->flush == Z_FINISH + && ctx->zstream.avail_in == 0) + { + + if (ngx_http_gunzip_filter_inflate_end(r, ctx) != NGX_OK) { + return NGX_ERROR; + } + + return NGX_OK; + } + + if (rc == Z_STREAM_END && ctx->zstream.avail_in > 0) { + + rc = inflateReset(&ctx->zstream); + + if (rc != Z_OK) { + ngx_log_error(NGX_LOG_ALERT, r->connection->log, 0, + "inflateReset() failed: %d", rc); + return NGX_ERROR; + } + + ctx->redo = 1; + + return NGX_AGAIN; + } + + if (ctx->in == NULL) { + + b = ctx->out_buf; + + if (ngx_buf_size(b) == 0) { + return NGX_OK; + } + + cl = ngx_alloc_chain_link(r->pool); + if (cl == NULL) { + return NGX_ERROR; + } + + ctx->zstream.avail_out = 0; + + cl->buf = b; + cl->next = NULL; + *ctx->last_out = cl; + ctx->last_out = &cl->next; + + return NGX_OK; + } + + return NGX_AGAIN; +} + + +static ngx_int_t +ngx_http_gunzip_filter_inflate_end(ngx_http_request_t *r, + ngx_http_gunzip_ctx_t *ctx) +{ + int rc; + ngx_buf_t *b; + ngx_chain_t *cl; + + ngx_log_debug0(NGX_LOG_DEBUG_HTTP, r->connection->log, 0, + "gunzip inflate end"); + + rc = inflateEnd(&ctx->zstream); + + if (rc != Z_OK) { + ngx_log_error(NGX_LOG_ALERT, r->connection->log, 0, + "inflateEnd() failed: %d", rc); + return NGX_ERROR; + } + + b = ctx->out_buf; + + if (ngx_buf_size(b) == 0) { + + b = ngx_calloc_buf(ctx->request->pool); + if (b == NULL) { + return NGX_ERROR; + } + } + + cl = ngx_alloc_chain_link(r->pool); + if (cl == NULL) { + return NGX_ERROR; + } + + cl->buf = b; + cl->next = NULL; + *ctx->last_out = cl; + ctx->last_out = &cl->next; + + b->last_buf = (r == r->main) ? 1 : 0; + b->last_in_chain = 1; + b->sync = 1; + + ctx->done = 1; + + return NGX_OK; +} + + +static void * +ngx_http_gunzip_filter_alloc(void *opaque, u_int items, u_int size) +{ + ngx_http_gunzip_ctx_t *ctx = opaque; + + ngx_log_debug2(NGX_LOG_DEBUG_HTTP, ctx->request->connection->log, 0, + "gunzip alloc: n:%ud s:%ud", + items, size); + + return ngx_palloc(ctx->request->pool, items * size); +} + + +static void +ngx_http_gunzip_filter_free(void *opaque, void *address) +{ +#if 0 + ngx_http_gunzip_ctx_t *ctx = opaque; + + ngx_log_debug1(NGX_LOG_DEBUG_HTTP, ctx->request->connection->log, 0, + "gunzip free: %p", address); +#endif +} + + +static void * +ngx_http_gunzip_create_conf(ngx_conf_t *cf) +{ + ngx_http_gunzip_conf_t *conf; + + conf = ngx_pcalloc(cf->pool, sizeof(ngx_http_gunzip_conf_t)); + if (conf == NULL) { + return NULL; + } + + /* + * set by ngx_pcalloc(): + * + * conf->bufs.num = 0; + */ + + conf->enable = NGX_CONF_UNSET; + + return conf; +} + + +static char * +ngx_http_gunzip_merge_conf(ngx_conf_t *cf, void *parent, void *child) +{ + ngx_http_gunzip_conf_t *prev = parent; + ngx_http_gunzip_conf_t *conf = child; + + ngx_conf_merge_value(conf->enable, prev->enable, 0); + + ngx_conf_merge_bufs_value(conf->bufs, prev->bufs, + (128 * 1024) / ngx_pagesize, ngx_pagesize); + + return NGX_CONF_OK; +} + + +static ngx_int_t +ngx_http_gunzip_filter_init(ngx_conf_t *cf) +{ + ngx_http_next_header_filter = ngx_http_top_header_filter; + ngx_http_top_header_filter = ngx_http_gunzip_header_filter; + + ngx_http_next_body_filter = ngx_http_top_body_filter; + ngx_http_top_body_filter = ngx_http_gunzip_body_filter; + + return NGX_OK; +} From agentzh at gmail.com Mon Sep 10 18:22:46 2012 From: agentzh at gmail.com (agentzh) Date: Mon, 10 Sep 2012 11:22:46 -0700 Subject: [PATCH] Make ngx_http_upstream provide a way to expose errors after sending out the response header In-Reply-To: <20120910103540.GN40452@mdounin.ru> References: <20120910103540.GN40452@mdounin.ru> Message-ID: Hello! On Mon, Sep 10, 2012 at 3:35 AM, Maxim Dounin wrote: > > I don't like the idea of changing r->headers_out.status here. > It's really incorrect and confusing: r->headers_out.status is a > status code of a response, and it's way too late to change it when > we've found an error during sending the response body. > I don't like it either. Just cannot think of an existing neutral flag that can be made use of. >> --- nginx-1.2.3/src/http/ngx_http_upstream.c 2012-08-06 10:34:08.000000000 -0700 >> +++ nginx-1.2.3-patched/src/http/ngx_http_upstream.c 2012-09-09 >> 21:58:04.727761891 -0700 >> @@ -2383,7 +2383,7 @@ > > Just a side note: please use -p when submitting patches. > Okay, I will at the next time :) >> @@ -2430,13 +2430,17 @@ >> if (u->busy_bufs == NULL) { >> >> if (u->length == 0 >> - || upstream->read->eof >> - || upstream->read->error) >> + || (upstream->read->eof && >> u->headers_in.content_length_n == -1)) > > Note: you can't be sure that eof with u->headers_in.content_length_n == > -1 is ok. E.g. with chunked encoding used it's certainly not > unless u->length == 0 as well. > Yes, I was ware of this hole. Just could not think of a solution for this. Any suggestion? > While content_length_n check is already used in cache and store > cases in ngx_http_upstream_process_request(), we might want to get > rid of it instead of reintroducing it in various other places. > Can I ask how will you get rid of this? Any plans in your head? >> { >> ngx_http_upstream_finalize_request(r, u, 0); >> return; >> } >> >> + if (upstream->read->eof || upstream->read->error) { >> + ngx_http_upstream_finalize_request(r, u, >> NGX_HTTP_BAD_GATEWAY); >> + return; >> + } >> + > > Just a side note: resulting code looks not very readable for me. > Yes, I could have done a little better here :) > > Additionally, I think major problem here isn't > subrequest-in-memory special case, but sending buf with "last" > set, which in some situations may result in truncated response > being considered full by a downstream. > But unfortunately, in the context of subrequests, the "last_buf" flag is already never set anyway. So we cannot use this flag as an indicator for data truncation errors anyway. Best regards, -agentzh From mdounin at mdounin.ru Tue Sep 11 00:23:36 2012 From: mdounin at mdounin.ru (Maxim Dounin) Date: Tue, 11 Sep 2012 04:23:36 +0400 Subject: [PATCH] Make ngx_http_upstream provide a way to expose errors after sending out the response header In-Reply-To: References: <20120910103540.GN40452@mdounin.ru> Message-ID: <20120911002335.GW40452@mdounin.ru> Hello! On Mon, Sep 10, 2012 at 11:22:46AM -0700, agentzh wrote: [...] > > Note: you can't be sure that eof with u->headers_in.content_length_n == > > -1 is ok. E.g. with chunked encoding used it's certainly not > > unless u->length == 0 as well. > > > > Yes, I was ware of this hole. Just could not think of a solution for > this. Any suggestion? > > > While content_length_n check is already used in cache and store > > cases in ngx_http_upstream_process_request(), we might want to get > > rid of it instead of reintroducing it in various other places. > > > > Can I ask how will you get rid of this? Any plans in your head? As of now, probably correct way to handle it is to use u->length == -1 to test if eof is ok (and p->length for a buffered case). Not sure if we want some separate flag for this. [...] > > Additionally, I think major problem here isn't > > subrequest-in-memory special case, but sending buf with "last" > > set, which in some situations may result in truncated response > > being considered full by a downstream. > > > > But unfortunately, in the context of subrequests, the "last_buf" flag > is already never set anyway. So we cannot use this flag as an > indicator for data truncation errors anyway. For subrequests in general there is last_in_chain flag. Maxim Dounin From mdounin at mdounin.ru Tue Sep 11 01:13:23 2012 From: mdounin at mdounin.ru (mdounin at mdounin.ru) Date: Tue, 11 Sep 2012 01:13:23 +0000 Subject: [nginx] svn commit: r4839 - trunk/src/http/modules Message-ID: <20120911011323.E3F9A3FAA08@mail.nginx.com> Author: mdounin Date: 2012-09-11 01:13:23 +0000 (Tue, 11 Sep 2012) New Revision: 4839 URL: http://trac.nginx.org/nginx/changeset/4839/nginx Log: Gunzip: removed nginx.h leftover include. Modified: trunk/src/http/modules/ngx_http_gunzip_filter_module.c Modified: trunk/src/http/modules/ngx_http_gunzip_filter_module.c =================================================================== --- trunk/src/http/modules/ngx_http_gunzip_filter_module.c 2012-09-10 16:52:47 UTC (rev 4838) +++ trunk/src/http/modules/ngx_http_gunzip_filter_module.c 2012-09-11 01:13:23 UTC (rev 4839) @@ -9,7 +9,6 @@ #include #include #include -#include #include From thierry.magnien at sfr.com Tue Sep 11 09:21:26 2012 From: thierry.magnien at sfr.com (MAGNIEN, Thierry) Date: Tue, 11 Sep 2012 09:21:26 +0000 Subject: Issuing an http request to distant server from a timer handler Message-ID: <5D103CE839D50E4CBC62C9FD7B83287C1AD188@EXCN015.encara.local.ads> Hi, I'm writing a module that needs to periodically request a distant HTTP server. I've setup an event handler through ngx_add_timer so that the handler is called every N seconds. Is there a simple way to make this HTTP request from the handler ? I've looked at ngx_http_named_location but this needs a request as argument and I don't have such in my event handler. Can I just create/allocate a new request and pass it to ngx_http_named_location ? Thanks for your help, Thierry From mdounin at mdounin.ru Tue Sep 11 12:35:31 2012 From: mdounin at mdounin.ru (mdounin at mdounin.ru) Date: Tue, 11 Sep 2012 12:35:31 +0000 Subject: [nginx] svn commit: r4840 - trunk/auto/lib/zlib Message-ID: <20120911123531.8AFEC3F9C5F@mail.nginx.com> Author: mdounin Date: 2012-09-11 12:35:30 +0000 (Tue, 11 Sep 2012) New Revision: 4840 URL: http://trac.nginx.org/nginx/changeset/4840/nginx Log: Configure: provide inflate() when building zlib on win32. It is now needed for gunzip filter. Modified: trunk/auto/lib/zlib/makefile.bcc trunk/auto/lib/zlib/makefile.msvc trunk/auto/lib/zlib/makefile.owc Modified: trunk/auto/lib/zlib/makefile.bcc =================================================================== --- trunk/auto/lib/zlib/makefile.bcc 2012-09-11 01:13:23 UTC (rev 4839) +++ trunk/auto/lib/zlib/makefile.bcc 2012-09-11 12:35:30 UTC (rev 4840) @@ -8,8 +8,10 @@ zlib.lib: cd $(ZLIB) - bcc32 -c $(CFLAGS) adler32.c crc32.c deflate.c trees.c zutil.c \ - compress.c + bcc32 -c $(CFLAGS) adler32.c crc32.c deflate.c \ + trees.c zutil.c compress.c \ + inflate.c inffast.c inftrees.c tlib zlib.lib +adler32.obj +crc32.obj +deflate.obj \ - +trees.obj +zutil.obj +compress.obj + +trees.obj +zutil.obj +compress.obj \ + +inflate.obj +inffast.obj +inftrees.obj Modified: trunk/auto/lib/zlib/makefile.msvc =================================================================== --- trunk/auto/lib/zlib/makefile.msvc 2012-09-11 01:13:23 UTC (rev 4839) +++ trunk/auto/lib/zlib/makefile.msvc 2012-09-11 12:35:30 UTC (rev 4840) @@ -8,7 +8,10 @@ zlib.lib: cd $(ZLIB) - cl -c $(CFLAGS) adler32.c crc32.c deflate.c trees.c zutil.c compress.c + cl -c $(CFLAGS) adler32.c crc32.c deflate.c \ + trees.c zutil.c compress.c \ + inflate.c inffast.c inftrees.c link -lib -out:zlib.lib adler32.obj crc32.obj deflate.obj \ - trees.obj zutil.obj compress.obj + trees.obj zutil.obj compress.obj \ + inflate.obj inffast.obj inftrees.obj Modified: trunk/auto/lib/zlib/makefile.owc =================================================================== --- trunk/auto/lib/zlib/makefile.owc 2012-09-11 01:13:23 UTC (rev 4839) +++ trunk/auto/lib/zlib/makefile.owc 2012-09-11 12:35:30 UTC (rev 4840) @@ -9,6 +9,6 @@ cd $(ZLIB) wcl386 -c $(CFLAGS) adler32.c crc32.c deflate.c trees.c zutil.c & - compress.c + compress.c inflate.c inffast.c inftrees.c wlib -n zlib.lib adler32.obj crc32.obj deflate.obj trees.obj & - zutil.obj compress.obj + zutil.obj compress.obj inflate.obj inffast.obj inftrees.obj From vbart at nginx.com Tue Sep 11 12:37:04 2012 From: vbart at nginx.com (vbart at nginx.com) Date: Tue, 11 Sep 2012 12:37:04 +0000 Subject: [nginx] svn commit: r4841 - trunk/docs/html Message-ID: <20120911123704.E7B833F9C5F@mail.nginx.com> Author: vbart Date: 2012-09-11 12:37:04 +0000 (Tue, 11 Sep 2012) New Revision: 4841 URL: http://trac.nginx.org/nginx/changeset/4841/nginx Log: Improved 50x error page. The feature set of the new page: - HTML5 compliant; - Looks similar to the new "Welcome page" (r4835); - Sysadmin hint with a link to the documentation of the "error_log" directive. Modified: trunk/docs/html/50x.html Modified: trunk/docs/html/50x.html =================================================================== --- trunk/docs/html/50x.html 2012-09-11 12:35:30 UTC (rev 4840) +++ trunk/docs/html/50x.html 2012-09-11 12:37:04 UTC (rev 4841) @@ -1,18 +1,21 @@ + -The page is temporarily unavailable +Error - - - - - -
-The page you are looking for is temporarily unavailable.
-Please try again later. -
+ +

An error occurred.

+

Sorry, the page you are looking for is currently unavailable.
+Please try again later.

+

If you are the system administrator of this resource then you should check +the error log for details.

+

Faithfully yours, nginx.

From mdounin at mdounin.ru Tue Sep 11 12:43:59 2012 From: mdounin at mdounin.ru (mdounin at mdounin.ru) Date: Tue, 11 Sep 2012 12:43:59 +0000 Subject: [nginx] svn commit: r4842 - in trunk/auto: . lib/libatomic lib/perl Message-ID: <20120911124359.A4B863F9EA7@mail.nginx.com> Author: mdounin Date: 2012-09-11 12:43:59 +0000 (Tue, 11 Sep 2012) New Revision: 4842 URL: http://trac.nginx.org/nginx/changeset/4842/nginx Log: Configure: fixed make macros to use parentheses instead of braces. Parentheses are more portable, in particular they are understood by nmake while braces aren't. Modified: trunk/auto/install trunk/auto/lib/libatomic/make trunk/auto/lib/perl/make Modified: trunk/auto/install =================================================================== --- trunk/auto/install 2012-09-11 12:37:04 UTC (rev 4841) +++ trunk/auto/install 2012-09-11 12:43:59 UTC (rev 4842) @@ -8,7 +8,7 @@ cat << END >> $NGX_MAKEFILE install_perl_modules: - cd $NGX_OBJS/src/http/modules/perl && \${MAKE} install + cd $NGX_OBJS/src/http/modules/perl && \$(MAKE) install END NGX_INSTALL_PERL_MODULES=install_perl_modules Modified: trunk/auto/lib/libatomic/make =================================================================== --- trunk/auto/lib/libatomic/make 2012-09-11 12:37:04 UTC (rev 4841) +++ trunk/auto/lib/libatomic/make 2012-09-11 12:43:59 UTC (rev 4842) @@ -6,7 +6,7 @@ cat << END >> $NGX_MAKEFILE $NGX_LIBATOMIC/src/libatomic_ops.a: $NGX_LIBATOMIC/Makefile - cd $NGX_LIBATOMIC && \${MAKE} + cd $NGX_LIBATOMIC && \$(MAKE) $NGX_LIBATOMIC/Makefile: $NGX_MAKEFILE cd $NGX_LIBATOMIC && ./configure Modified: trunk/auto/lib/perl/make =================================================================== --- trunk/auto/lib/perl/make 2012-09-11 12:37:04 UTC (rev 4841) +++ trunk/auto/lib/perl/make 2012-09-11 12:43:59 UTC (rev 4842) @@ -12,7 +12,7 @@ $NGX_OBJS/src/http/modules/perl/Makefile cp -p src/http/modules/perl/nginx.* $NGX_OBJS/src/http/modules/perl/ - cd $NGX_OBJS/src/http/modules/perl && \${MAKE} + cd $NGX_OBJS/src/http/modules/perl && \$(MAKE) rm -rf $NGX_OBJS/install_perl From mdounin at mdounin.ru Tue Sep 11 13:17:32 2012 From: mdounin at mdounin.ru (mdounin at mdounin.ru) Date: Tue, 11 Sep 2012 13:17:32 +0000 Subject: [nginx] svn commit: r4843 - trunk/misc Message-ID: <20120911131732.3A89C3F9C46@mail.nginx.com> Author: mdounin Date: 2012-09-11 13:17:31 +0000 (Tue, 11 Sep 2012) New Revision: 4843 URL: http://trac.nginx.org/nginx/changeset/4843/nginx Log: Helper target "win32" to run configure for win32 builds. Modified: trunk/misc/GNUmakefile Modified: trunk/misc/GNUmakefile =================================================================== --- trunk/misc/GNUmakefile 2012-09-11 12:43:59 UTC (rev 4842) +++ trunk/misc/GNUmakefile 2012-09-11 13:17:31 UTC (rev 4843) @@ -82,6 +82,43 @@ tar -c -z -f $(NGINX).tar.gz --directory $(TEMP) $(NGINX) +win32: + ./auto/configure \ + --with-cc=cl \ + --builddir=${OBJS} \ + --with-debug \ + --prefix= \ + --conf-path=conf/nginx.conf \ + --pid-path=logs/nginx.pid \ + --http-log-path=logs/access.log \ + --error-log-path=logs/error.log \ + --sbin-path=nginx.exe \ + --http-client-body-temp-path=temp/client_body_temp \ + --http-proxy-temp-path=temp/proxy_temp \ + --http-fastcgi-temp-path=temp/fastcgi_temp \ + --http-scgi-temp-path=temp/scgi_temp \ + --http-uwsgi-temp-path=temp/uwsgi_temp \ + --with-cc-opt=-DFD_SETSIZE=1024 \ + --with-pcre=${OBJS}/lib/${PCRE} \ + --with-zlib=${OBJS}/lib/${ZLIB} \ + --with-select_module \ + --with-http_realip_module \ + --with-http_addition_module \ + --with-http_sub_module \ + --with-http_dav_module \ + --with-http_stub_status_module \ + --with-http_flv_module \ + --with-http_mp4_module \ + --with-http_gunzip_module \ + --with-http_gzip_static_module \ + --with-http_random_index_module \ + --with-http_secure_link_module \ + --with-mail \ + --with-openssl=${OBJS}/lib/${OPENSSL} \ + --with-openssl-opt=enable-tlsext \ + --with-http_ssl_module \ + --with-mail_ssl_module \ + --with-ipv6 zip: rm -rf $(TEMP) From mdounin at mdounin.ru Tue Sep 11 13:36:34 2012 From: mdounin at mdounin.ru (mdounin at mdounin.ru) Date: Tue, 11 Sep 2012 13:36:34 +0000 Subject: [nginx] svn commit: r4844 - trunk/misc Message-ID: <20120911133634.D6C1C3F9E7F@mail.nginx.com> Author: mdounin Date: 2012-09-11 13:36:34 +0000 (Tue, 11 Sep 2012) New Revision: 4844 URL: http://trac.nginx.org/nginx/changeset/4844/nginx Log: Updated zlib used for win32 builds. Modified: trunk/misc/GNUmakefile Modified: trunk/misc/GNUmakefile =================================================================== --- trunk/misc/GNUmakefile 2012-09-11 13:17:31 UTC (rev 4843) +++ trunk/misc/GNUmakefile 2012-09-11 13:36:34 UTC (rev 4844) @@ -7,7 +7,7 @@ OBJS = objs.msvc8 OPENSSL = openssl-1.0.1c -ZLIB = zlib-1.2.5 +ZLIB = zlib-1.2.7 PCRE = pcre-8.31 From dirk.feytons at gmail.com Wed Sep 12 09:46:52 2012 From: dirk.feytons at gmail.com (Dirk Feytons) Date: Wed, 12 Sep 2012 11:46:52 +0200 Subject: [PATCH] Unix domain sockets in abstract namespace Message-ID: Hi, I was playing around with ngx_lua and its socket API, which is built on top of nginx functionality. The API supports unix domain sockets but not in the abstract namespace, at least not officially. When I tried specifying an abstract address I found it didn't work. I tracked it down to two calls to ngx_cpystrn() in core/ngx_inet.c:ngx_parse_unix_domain_url(). That function gets confused by the leading '\0' in an abstract namespace address. Changing it to ngx_memcpy() fixes this and then everything works. I attached a patch agains nginx 1.2.3. The fix seems trivial but this is my first look into the nginx internals so I don't know whether there are unwanted side effects from this change. Feedback welcome. Regards, Dirk F. -------------- next part -------------- A non-text attachment was scrubbed... Name: nginx-1.2.3-abstract-unix-domain-socket.patch Type: application/octet-stream Size: 911 bytes Desc: not available URL: From mdounin at mdounin.ru Wed Sep 12 10:41:37 2012 From: mdounin at mdounin.ru (mdounin at mdounin.ru) Date: Wed, 12 Sep 2012 10:41:37 +0000 Subject: [nginx] svn commit: r4845 - trunk/docs/xml/nginx Message-ID: <20120912104137.9A1E43F9C44@mail.nginx.com> Author: mdounin Date: 2012-09-12 10:41:36 +0000 (Wed, 12 Sep 2012) New Revision: 4845 URL: http://trac.nginx.org/nginx/changeset/4845/nginx Log: nginx-1.3.6-RELEASE Modified: trunk/docs/xml/nginx/changes.xml Modified: trunk/docs/xml/nginx/changes.xml =================================================================== --- trunk/docs/xml/nginx/changes.xml 2012-09-11 13:36:34 UTC (rev 4844) +++ trunk/docs/xml/nginx/changes.xml 2012-09-12 10:41:36 UTC (rev 4845) @@ -5,6 +5,62 @@ + + + + +?????? ngx_http_gunzip_filter_module. + + +the ngx_http_gunzip_filter_module. + + + + + +????????? memcached_gzip_flag. + + +the "memcached_gzip_flag" directive. + + + + + +???????? always ????????? gzip_static. + + +the "always" parameter of the "gzip_static" directive. + + + + + +? ????????? "limit_req"; +?????? ????????? ? 1.1.14.
+??????? Charles Chen. +
+ +in the "limit_req" directive; +the bug had appeared in 1.1.14.
+Thanks to Charles Chen. +
+
+ + + +nginx ?? ????????? gcc 4.7 ? ???????????? -O2 +???? ????????????? ???????? --with-ipv6. + + +nginx could not be built by gcc 4.7 with -O2 optimization +if the --with-ipv6 option was used. + + + +
+ + From mdounin at mdounin.ru Wed Sep 12 10:41:57 2012 From: mdounin at mdounin.ru (mdounin at mdounin.ru) Date: Wed, 12 Sep 2012 10:41:57 +0000 Subject: [nginx] svn commit: r4846 - tags Message-ID: <20120912104157.179633F9F59@mail.nginx.com> Author: mdounin Date: 2012-09-12 10:41:56 +0000 (Wed, 12 Sep 2012) New Revision: 4846 URL: http://trac.nginx.org/nginx/changeset/4846/nginx Log: release-1.3.6 tag Added: tags/release-1.3.6/ From juce66 at gmail.com Sat Sep 15 16:21:32 2012 From: juce66 at gmail.com (Anton Jouline) Date: Sat, 15 Sep 2012 09:21:32 -0700 Subject: [PATCH] Random peer selection for implicit upstream defined by proxy_pass Message-ID: Done some additional testing and it looks good. Regular round-robin upstream continues to work as before, unaffected by the change. For implicit upstream via proxy_pass - we get the random distribution among available IPs. Anything else you would like added/changed in the new patch? Not sure it can be made much smaller... except maybe the expression to choose a random index could be simplified to use random() or ngx_random() function instead of rand(). Anton. From crk_world at yahoo.com.cn Mon Sep 17 07:11:33 2012 From: crk_world at yahoo.com.cn (chen cw) Date: Mon, 17 Sep 2012 15:11:33 +0800 Subject: a problem in CPU affinity Message-ID: Hi, When a worker is down, the master will spawn a new one. However, in case that we enable CPU affinity, the newly created workers will be bound to the same CPU, as the CPU which was bound to by the latest worker spawn at the startup or during the reload. This problem still exists in the latest version 1.3.6. Sincerely -- Charles Chen Software Engineer Server Platforms Team at Taobao.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From mdounin at mdounin.ru Tue Sep 18 08:07:09 2012 From: mdounin at mdounin.ru (Maxim Dounin) Date: Tue, 18 Sep 2012 12:07:09 +0400 Subject: a problem in CPU affinity In-Reply-To: References: Message-ID: <20120918080709.GR40452@mdounin.ru> Hello! On Mon, Sep 17, 2012 at 03:11:33PM +0800, chen cw wrote: > Hi, > > When a worker is down, the master will spawn a new one. However, in case > that we enable CPU affinity, the newly created workers will be bound to the > same CPU, as the CPU which was bound to by the latest worker spawn at the > startup or during the reload. This problem still exists in the latest > version 1.3.6. Yes, thank you for report. Quick and dirty fix would be: --- a/src/os/unix/ngx_process_cycle.c +++ b/src/os/unix/ngx_process_cycle.c @@ -371,6 +371,8 @@ ngx_start_worker_processes(ngx_cycle_t * ngx_pass_open_channel(cycle, &ch); } + + cpu_affinity = 0; } With this patch workers respawn after crashes won't have cpu affinity set, which is probably better. It's still not ideal, but after all real problem is a worker crash, respawn is just an emergency mechanism. Maxim Dounin From mailtogroe at gmail.com Tue Sep 18 13:55:38 2012 From: mailtogroe at gmail.com (=?ISO-8859-1?Q?Benjamin_Gr=F6ssing?=) Date: Tue, 18 Sep 2012 15:55:38 +0200 Subject: Can not use $bytes_sent in module (complex values) Message-ID: Hello, I am currently working on a nginx module, that parses user defined strings for variables using ngx_http_complex_value_t. That works great with all variables defined in ngx_http_variables.c (i.e. $body_bytes_sent). However, variables as $bytes_sent do not exist here since ngx_http_log_module uses its own variable parsing system. What is the best way to use these extra variables in my module? I have thought of the following options, I am not very sure about what is the best way to go: a) Copy the ngx_http_log_module code in my own module and rename every single type and function to avoid double definitions. Reuse functions and types from there. b) Move the ngx_http_log_module code in my own module directory and compile it only there. Reuse functions and types from there. c) Write own variable parsing system, reuse a few snippets from ngx_http_variables and ngx_http_log_module, whatever fits best. d) Use complex values and try to extend the ngx_http_core_variables array to include $body_bytes_sent and the other missing variables (I don't understand why they are not included here anyway?). e) Any better way to achieve that? I'd really appreciate your help. Regards Benjamin From vshebordaev at mail.ru Tue Sep 18 15:56:32 2012 From: vshebordaev at mail.ru (=?UTF-8?B?VmxhZGltaXIgU2hlYm9yZGFldg==?=) Date: Tue, 18 Sep 2012 19:56:32 +0400 Subject: Can not use $bytes_sent in module (complex values) In-Reply-To: References: Message-ID: <1347983792.885909300@f192.mail.ru> Hi! The ngx_http_log module uses indexed variables just like the rewrite module does. Hope it helps. -- Regards, Vladimir Tue, 18 Sep 2012 15:55:38 +0200 ?? Benjamin Gr?ssing : > > > > >Hello, > > I am currently working on a nginx module, that parses user defined > strings for variables using ngx_http_complex_value_t. That works great > with all variables defined in ngx_http_variables.c (i.e. > $body_bytes_sent). However, variables as $bytes_sent do not exist here > since ngx_http_log_module uses its own variable parsing system. What > is the best way to use these extra variables in my module? I have > thought of the following options, I am not very sure about what is the > best way to go: > a) Copy the ngx_http_log_module code in my own module and rename every > single type and function to avoid double definitions. Reuse functions > and types from there. > b) Move the ngx_http_log_module code in my own module directory and > compile it only there. Reuse functions and types from there. > c) Write own variable parsing system, reuse a few snippets from > ngx_http_variables and ngx_http_log_module, whatever fits best. > d) Use complex values and try to extend the ngx_http_core_variables > array to include $body_bytes_sent and the other missing variables (I > don't understand why they are not included here anyway?). > e) Any better way to achieve that? > > I'd really appreciate your help. > > Regards > Benjamin > > _______________________________________________ > nginx-devel mailing list >nginx-devel at nginx.org >http://mailman.nginx.org/mailman/listinfo/nginx-devel > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mdounin at mdounin.ru Wed Sep 19 00:06:56 2012 From: mdounin at mdounin.ru (Maxim Dounin) Date: Wed, 19 Sep 2012 04:06:56 +0400 Subject: Can not use $bytes_sent in module (complex values) In-Reply-To: References: Message-ID: <20120919000656.GA40452@mdounin.ru> Hello! On Tue, Sep 18, 2012 at 03:55:38PM +0200, Benjamin Gr?ssing wrote: > Hello, > > I am currently working on a nginx module, that parses user defined > strings for variables using ngx_http_complex_value_t. That works great > with all variables defined in ngx_http_variables.c (i.e. > $body_bytes_sent). However, variables as $bytes_sent do not exist here > since ngx_http_log_module uses its own variable parsing system. What > is the best way to use these extra variables in my module? I have > thought of the following options, I am not very sure about what is the > best way to go: > a) Copy the ngx_http_log_module code in my own module and rename every > single type and function to avoid double definitions. Reuse functions > and types from there. > b) Move the ngx_http_log_module code in my own module directory and > compile it only there. Reuse functions and types from there. > c) Write own variable parsing system, reuse a few snippets from > ngx_http_variables and ngx_http_log_module, whatever fits best. > d) Use complex values and try to extend the ngx_http_core_variables > array to include $body_bytes_sent and the other missing variables (I > don't understand why they are not included here anyway?). > e) Any better way to achieve that? > > I'd really appreciate your help. All variables that are currently present only in log module should really be made available as normal variables as well. They aren't yet mostly due to historical reasons (log module appeared before normal variables were implemented) and the fact that most of them are only usable for logging. No idea what would be best for your particular module, but in general it's better to use complex values and what's available via normal variables. Unless you need $bytes_sent right now - it's probably the best aproach, and missing variables will be available once they appear in nginx. (The latter is something you may speedup by submitting a good patch. Most tricky part here would be to follow coding style.) Maxim Dounin From crk_world at yahoo.com.cn Wed Sep 19 01:57:31 2012 From: crk_world at yahoo.com.cn (chen cw) Date: Wed, 19 Sep 2012 09:57:31 +0800 Subject: a problem in CPU affinity In-Reply-To: <20120918080709.GR40452@mdounin.ru> References: <20120918080709.GR40452@mdounin.ru> Message-ID: Yes, I believe it is much better. Thank you On Tue, Sep 18, 2012 at 4:07 PM, Maxim Dounin wrote: > Hello! > > On Mon, Sep 17, 2012 at 03:11:33PM +0800, chen cw wrote: > > > Hi, > > > > When a worker is down, the master will spawn a new one. However, in case > > that we enable CPU affinity, the newly created workers will be bound to > the > > same CPU, as the CPU which was bound to by the latest worker spawn at the > > startup or during the reload. This problem still exists in the latest > > version 1.3.6. > > Yes, thank you for report. Quick and dirty fix would be: > > --- a/src/os/unix/ngx_process_cycle.c > +++ b/src/os/unix/ngx_process_cycle.c > @@ -371,6 +371,8 @@ ngx_start_worker_processes(ngx_cycle_t * > > ngx_pass_open_channel(cycle, &ch); > } > + > + cpu_affinity = 0; > } > > > With this patch workers respawn after crashes won't have cpu > affinity set, which is probably better. It's still not ideal, but > after all real problem is a worker crash, respawn is just an > emergency mechanism. > > Maxim Dounin > > _______________________________________________ > nginx-devel mailing list > nginx-devel at nginx.org > http://mailman.nginx.org/mailman/listinfo/nginx-devel > -- -- Charles Chen Software Engineer Server Platforms Team at Taobao.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From crk_world at yahoo.com.cn Wed Sep 19 03:56:41 2012 From: crk_world at yahoo.com.cn (chen cw) Date: Wed, 19 Sep 2012 11:56:41 +0800 Subject: ngx_http_log_if_module Message-ID: Hi everyone, I hope to control nginx when to log the requests and when not to, so I wrote a simple module "ngx_http_log_if_module". This module works, and I'd like to share. Welcome to use, it's my honor. Moreover, any advise, any requirement is also welcome, it's my honor too. Ah, the codes, I've almost forgotten to give it out. It's here: https://github.com/cfsego/ngx_log_if Sincerely -------------- next part -------------- An HTML attachment was scrubbed... URL: From sb at waeme.net Wed Sep 19 11:17:37 2012 From: sb at waeme.net (Sergey Budnevitch) Date: Wed, 19 Sep 2012 15:17:37 +0400 Subject: Administrivia: envelope sender address and the mailing list Message-ID: Until recently messages to the mailing list from unsubscribed addresses were forwarded directly to the maillist administrator, and sender had no knowledge why message was delayed or dropped. Since the number of messages automatically forwarded to maillist administrator grew with time mostly because of spam, we have added extra sender validation on mail server level. Unfortunately mail server checks envelope sender address and Mailman checks From: address. In most cases they match, but for example, google mail servers set envelope sender to google account address, while From: address may be any you like. In this case sender will receive non-delivery report, although address in From: is correct and subscribed to the mailing list. As a workaround it is possible to subscribe second (envelope sender) address to the mailing list and disable delivery through Mailman web interface. From mdounin at mdounin.ru Wed Sep 19 13:56:09 2012 From: mdounin at mdounin.ru (Maxim Dounin) Date: Wed, 19 Sep 2012 17:56:09 +0400 Subject: [PATCH] Unix domain sockets in abstract namespace In-Reply-To: References: Message-ID: <20120919135608.GH40452@mdounin.ru> Hello! On Wed, Sep 12, 2012 at 11:46:52AM +0200, Dirk Feytons wrote: > Hi, > > I was playing around with ngx_lua and its socket API, which is built > on top of nginx functionality. The API supports unix domain sockets > but not in the abstract namespace, at least not officially. When I > tried specifying an abstract address I found it didn't work. I tracked > it down to two calls to ngx_cpystrn() in > core/ngx_inet.c:ngx_parse_unix_domain_url(). That function gets > confused by the leading '\0' in an abstract namespace address. This is because the abstract namespace is actually linux extension to unix domain sockets, "The abstract socket namespace is a nonportable Linux extension" (from [1]). Strictly speaking, it's not something even permitted by POSIX as socket path ends with first null byte as per POSIX. > Changing it to ngx_memcpy() fixes this and then everything works. > > I attached a patch agains nginx 1.2.3. The fix seems trivial but this > is my first look into the nginx internals so I don't know whether > there are unwanted side effects from this change. Feedback welcome. I suspect there may be some, in particular during logging of errors and probably while parsing untrusted urls. [1] http://www.kernel.org/doc/man-pages/online/pages/man7/unix.7.html Maxim Dounin From mdounin at mdounin.ru Wed Sep 19 15:24:50 2012 From: mdounin at mdounin.ru (Maxim Dounin) Date: Wed, 19 Sep 2012 19:24:50 +0400 Subject: [PATCH] Random peer selection for implicit upstream defined by proxy_pass In-Reply-To: References: <20120907144500.GL40452@mdounin.ru> Message-ID: <20120919152450.GI40452@mdounin.ru> Hello! On Sat, Sep 08, 2012 at 01:19:49AM -0700, Anton Jouline wrote: > hello, Maxim. > > Thanks for your response! > You are right that my patch looks unnecessarily complicated for > what it actually does. Lots of code duplication too... After reading your > message I started thinking again of a simpler way to accomplish the same > task, and i think i found it. > > But first, let me provide feedback for your 2 proposals. (again, sorry for late reply, still too busy) > > #1 - Enforcing the random order in ngx_http_upstream_create_round_robin_peer() > is part of the solution, but it's not enough, because of how the peer is > selected in ngx_upstream_get_peer() function in the same file. So we need > to modify that function also, but must be careful there, because it is used by > the regular weighted round robin algorithm, when upstream is defined > either implicitly by resolved-once proxy_pass, or explicitly by upstream block > in the configuration file. We don't want to modify the existing round robin > behaviour for those cases. I believe i found a way to do that (see the new > patch at the end of the message) Just enforcing random order of peers as originally suggested should be enough here. The code in ngx_http_upstream_get_peer() will then select first one, which will be random one. See below for some comments on your patch. > #2 - Introduce round-robin in resolver code. I think that would be good, > but it does not really solve the issue of poor load distribution, > because during > the period of time when the DNS lookup is cached, still only 1 ip address > would be used out of the list of many. It is true that when TTL expires, > the next lookup would return a re-ordered list and perhaps a different ip > would be in the first position, but for the duration of TTL there will still be > no distribution of requests among the available servers. > That's why i think #1 is a better alternative. No, no, no. I mean - always return names in random order from a resolver, even if DNS response is cached. This should do the same as (1) but will cover other cases as well. > Ok, and now goes the new version of the patch. > It is much shorter and simpler! No new files needed, just a couple > of small changes in ngx_http_upstream_round_robin.c. > Tested successfully with versions 1.2.2, 1.2.3, and 1.3.5 > > Thanks. > > Anton. > > > === > diff -u -r nginx-1.2.2/src/http/ngx_http_upstream_round_robin.c > nginx-1.2.2-mod/src/http/ngx_http_upstream_round_robin.c > --- nginx-1.2.2/src/http/ngx_http_upstream_round_robin.c 2012-07-02 > 12:41:13.000000000 -0400 > +++ nginx-1.2.2-mod/src/http/ngx_http_upstream_round_robin.c 2012-09-08 > 03:10:17.000000000 -0400 > @@ -288,7 +288,7 @@ > { > u_char *p; > size_t len; > - ngx_uint_t i, n; > + ngx_uint_t i, j, n; > struct sockaddr_in *sin; > ngx_http_upstream_rr_peers_t *peers; > ngx_http_upstream_rr_peer_data_t *rrp; > @@ -359,8 +359,11 @@ > } > } > > + /* start at random index */ > + j = (int) ( (float)(peers->number) * (rand() / (RAND_MAX + 1.0))); > + > rrp->peers = peers; > - rrp->current = 0; > + rrp->current = j; This looks overcomplicated. The same thing can be done with rrp->current = ngx_random() % peers->number. which is much more readable. > > if (rrp->peers->number <= 8 * sizeof(uintptr_t)) { > rrp->tried = &rrp->data; > @@ -504,7 +507,7 @@ > time_t now; > uintptr_t m; > ngx_int_t total; > - ngx_uint_t i, n; > + ngx_uint_t i, j, n; > ngx_http_upstream_rr_peer_t *peer, *best; > > now = ngx_time(); > @@ -514,14 +517,15 @@ > > for (i = 0; i < rrp->peers->number; i++) { > > - n = i / (8 * sizeof(uintptr_t)); > - m = (uintptr_t) 1 << i % (8 * sizeof(uintptr_t)); > + j = (i + rrp->current) % rrp->peers->number; > + n = j / (8 * sizeof(uintptr_t)); > + m = (uintptr_t) 1 << j % (8 * sizeof(uintptr_t)); Note this might affect peer selection in some situations. If there are two peers with identical effective weight, peer selection now depends on rrp->current (previous peer tried in a normal case), while with previous code selection was stable and always used first one. BTW, you may find more details about peer selection algorithm currently used in the r4622 commit log, see here: http://trac.nginx.org/nginx/changeset/4622/nginx Maxim Dounin > > if (rrp->tried[n] & m) { > continue; > } > > - peer = &rrp->peers->peer[i]; > + peer = &rrp->peers->peer[j]; > > if (peer->down) { > continue; > === > > > On Fri, Sep 7, 2012 at 7:45 AM, Maxim Dounin wrote: > > Hello! > > > > (sorry for late reply, I was busy with OCSP stapling and slowly > > catching up with resulting mail backlog now) > > > > On Tue, Aug 21, 2012 at 11:36:49AM -0700, Anton Jouline wrote: > > > >> hello, Nginx developers, > >> > >> submitting this patch for your review. It changes the behaviour of > >> peer selection > >> for the case, when the upstream is defined implicitly by proxy_pass > >> directive like this: > >> > >> resolver 172.16.0.23; > >> set $backend backend.servers.example.com; > >> proxy_pass http://$backend; > > > > [...] > > > >> It turns out that the round-robin algorithm actually does not work in > >> this case, > >> because of the way things are implemented in src/http/ngx_http_upstream.c. > >> The peer array is created and initialized on each request, even if a > >> cached result of > >> DNS query is returned by resolver. The upstream code has no knowledge of whether > >> the list of IP addresses changed or not. So what ends up happening is this: > >> Let's say "backend.servers.example.com" resolves to 6 IP addresses. Then the > >> first good one will be always used, and the rest will never be even looked at. > > > > Surely this is valid problem, and it needs fixing. Though I don't > > think we have to introduce another balancer module just for this. > > > > I think of two possible ways to resolve this: > > > > 1. Enforce random order of server in ngx_http_upstream_create_round_robin_peer(). > > Note it's only used to create upsteams on the fly, so it should be > > ok (and will lead to the same result as with your patch, though > > will be simplier). > > > > 2. Introduce round-robin in resolver code before we return cached > > response, much like it will be done by a normal DNS servers. This > > should also simplify other possible uses of a resolver. > > > > Not sure which one would be better though. > > > > Maxim Dounin > > > > _______________________________________________ > > nginx-devel mailing list > > nginx-devel at nginx.org > > http://mailman.nginx.org/mailman/listinfo/nginx-devel > > _______________________________________________ > nginx-devel mailing list > nginx-devel at nginx.org > http://mailman.nginx.org/mailman/listinfo/nginx-devel From mailtogroe at gmail.com Wed Sep 19 21:11:57 2012 From: mailtogroe at gmail.com (=?ISO-8859-1?Q?Benjamin_Gr=F6ssing?=) Date: Wed, 19 Sep 2012 23:11:57 +0200 Subject: Can not use $bytes_sent in module (complex values) In-Reply-To: <20120919000656.GA40452@mdounin.ru> References: <20120919000656.GA40452@mdounin.ru> Message-ID: Hi Vladimir, hi Maxim! Thanks for your answers. I've had a look at the rewrite module - unfortunately it does not support the $bytes_sent variable either as configuring "if ($bytes_sent) { }" throws the error: [emerg] unknown "bytes_sent" variable. Nice to hear that there is a plan to include all variables in the complex value system - I think that would make that part a of nginx a lot more clear! I'd be happy if I could contribute here, but I am not sure about the details. Is there any guide to contributing to nginx available or do I just have to create a patch file and post it here? Can I just start with the nginx 1.2.3 source code from nginx.org? Regards Benjamin 2012/9/19 Maxim Dounin : > Hello! > > On Tue, Sep 18, 2012 at 03:55:38PM +0200, Benjamin Gr?ssing wrote: > >> Hello, >> >> I am currently working on a nginx module, that parses user defined >> strings for variables using ngx_http_complex_value_t. That works great >> with all variables defined in ngx_http_variables.c (i.e. >> $body_bytes_sent). However, variables as $bytes_sent do not exist here >> since ngx_http_log_module uses its own variable parsing system. What >> is the best way to use these extra variables in my module? I have >> thought of the following options, I am not very sure about what is the >> best way to go: >> a) Copy the ngx_http_log_module code in my own module and rename every >> single type and function to avoid double definitions. Reuse functions >> and types from there. >> b) Move the ngx_http_log_module code in my own module directory and >> compile it only there. Reuse functions and types from there. >> c) Write own variable parsing system, reuse a few snippets from >> ngx_http_variables and ngx_http_log_module, whatever fits best. >> d) Use complex values and try to extend the ngx_http_core_variables >> array to include $body_bytes_sent and the other missing variables (I >> don't understand why they are not included here anyway?). >> e) Any better way to achieve that? >> >> I'd really appreciate your help. > > All variables that are currently present only in log module should > really be made available as normal variables as well. They aren't > yet mostly due to historical reasons (log module appeared before > normal variables were implemented) and the fact that most of > them are only usable for logging. > > No idea what would be best for your particular module, but in > general it's better to use complex values and what's available via > normal variables. Unless you need $bytes_sent right now - it's > probably the best aproach, and missing variables will be available > once they appear in nginx. > > (The latter is something you may speedup by submitting a good > patch. Most tricky part here would be to follow coding style.) > > Maxim Dounin > > _______________________________________________ > nginx-devel mailing list > nginx-devel at nginx.org > http://mailman.nginx.org/mailman/listinfo/nginx-devel From vshebordaev at mail.ru Wed Sep 19 23:43:20 2012 From: vshebordaev at mail.ru (=?UTF-8?B?VmxhZGltaXIgU2hlYm9yZGFldg==?=) Date: Thu, 20 Sep 2012 03:43:20 +0400 Subject: Can not use $bytes_sent in module (complex values) In-Reply-To: References: <20120919000656.GA40452@mdounin.ru> Message-ID: <1348098200.250143680@f105.mail.ru> Wed, 19 Sep 2012 23:11:57 +0200 ?? Benjamin Gr?ssing : >2012/9/20 Benjamin Gr?ssing : > >Hi Vladimir, hi Maxim! > >Thanks for your answers. I've had a look at the rewrite module - >unfortunately it does not support the $bytes_sent variable either as >configuring "if ($bytes_sent) { }" throws the error: [emerg] unknown >"bytes_sent" variable. Yes, this is what the indexed variables are. They have no name at?runtime and accessed by the index into the request->variables array by the?script engine that is to evaluate a complex value and if conditionals.?This is faster to do, since it helps to avoid the cmcf->variables_hash?lookup and a memory allocation at evaluation time having the space for?the value reserved at the request initialization time. But the index?value must be known to the rewrite module at the configuration time. >Nice to hear that there is a plan to include all variables in the >complex value system - I think that would make that part a of nginx a >lot more clear! I'd be happy if I could contribute here, but I am not One way to make the log module varaibles known to the rewite module is?to register 'em with ngx_http_add_variable(). But this way they loose?in performance, not to say this would be not quite consistent with?their semantics that is "automatic" wrt the request. Another way is to make the log module use cmcf->variables as the?global registry for the indexed variables just like the rewrite module?does, and only rely on the script engine to evaluate 'em. ?This way?the log module variables would be available to rewrite module?remaining the light-weight indexed ones. This will also help to avoid?the dependency on the module initialization sequence. >sure about the details. Is there any guide to contributing to nginx >available or do I just have to create a patch file and post it here? >Can I just start with the nginx 1.2.3 source code from?nginx.org? > > I guess the patch sent to this list would do as long as it applies to?any svn branch. The code base will change while you are working on the?feature, so your patch will anyways have to be merged. Regards, Vladimir >Regards > Benjamin > > 2012/9/19 Maxim Dounin : > > Hello! > > > > On Tue, Sep 18, 2012 at 03:55:38PM +0200, Benjamin Gr?ssing wrote: > > > >> Hello, > >> > >> I am currently working on a nginx module, that parses user defined > >> strings for variables using ngx_http_complex_value_t. That works great > >> with all variables defined in ngx_http_variables.c (i.e. > >> $body_bytes_sent). However, variables as $bytes_sent do not exist here > >> since ngx_http_log_module uses its own variable parsing system. What > >> is the best way to use these extra variables in my module? I have > >> thought of the following options, I am not very sure about what is the > >> best way to go: > >> a) Copy the ngx_http_log_module code in my own module and rename every > >> single type and function to avoid double definitions. Reuse functions > >> and types from there. > >> b) Move the ngx_http_log_module code in my own module directory and > >> compile it only there. Reuse functions and types from there. > >> c) Write own variable parsing system, reuse a few snippets from > >> ngx_http_variables and ngx_http_log_module, whatever fits best. > >> d) Use complex values and try to extend the ngx_http_core_variables > >> array to include $body_bytes_sent and the other missing variables (I > >> don't understand why they are not included here anyway?). > >> e) Any better way to achieve that? > >> > >> I'd really appreciate your help. > > > > All variables that are currently present only in log module should > > really be made available as normal variables as well. They aren't > > yet mostly due to historical reasons (log module appeared before > > normal variables were implemented) and the fact that most of > > them are only usable for logging. > > > > No idea what would be best for your particular module, but in > > general it's better to use complex values and what's available via > > normal variables. Unless you need $bytes_sent right now - it's > > probably the best aproach, and missing variables will be available > > once they appear in nginx. > > > > (The latter is something you may speedup by submitting a good > > patch. Most tricky part here would be to follow coding style.) > > > > Maxim Dounin > > > > _______________________________________________ > > nginx-devel mailing list > > nginx-devel at nginx.org > > http://mailman.nginx.org/mailman/listinfo/nginx-devel > > _______________________________________________ > nginx-devel mailing list >nginx-devel at nginx.org >http://mailman.nginx.org/mailman/listinfo/nginx-devel > -------------- next part -------------- An HTML attachment was scrubbed... URL: From juce66 at gmail.com Thu Sep 20 05:39:01 2012 From: juce66 at gmail.com (Anton Jouline) Date: Wed, 19 Sep 2012 22:39:01 -0700 Subject: [PATCH] Random peer selection for implicit upstream defined by proxy_pass In-Reply-To: <20120919152450.GI40452@mdounin.ru> References: <20120907144500.GL40452@mdounin.ru> <20120919152450.GI40452@mdounin.ru> Message-ID: Hello, thanks for your response. Please see comments below. On Wed, Sep 19, 2012 at 8:24 AM, Maxim Dounin wrote: > Hello! > > ... > > Just enforcing random order of peers as originally suggested > should be enough here. The code in ngx_http_upstream_get_peer() > will then select first one, which will be random one. That's true, if you actually re-order the addresses in ctx->addrs array. Perhaps, i was trying to solve it the wrong way, by instead initializing the rrp->current to a random index. Because of that, i needed to modify the ngx_http_upstream_get_peer(). If instead ctx->addrs already contains items in random order, then it is not necessary. > No, no, no. I mean - always return names in random order from a > resolver, even if DNS response is cached. This should do the same > as (1) but will cover other cases as well. > ah, ok. Thanks for clarification. Yes, this would work quite well. The latest version of my patch (see below), now implements this approach. >> + /* start at random index */ >> + j = (int) ( (float)(peers->number) * (rand() / (RAND_MAX + 1.0))); >> + >> rrp->peers = peers; >> - rrp->current = 0; >> + rrp->current = j; > > This looks overcomplicated. The same thing can be done with > > rrp->current = ngx_random() % peers->number. > > which is much more readable. Agreed. >> for (i = 0; i < rrp->peers->number; i++) { >> >> - n = i / (8 * sizeof(uintptr_t)); >> - m = (uintptr_t) 1 << i % (8 * sizeof(uintptr_t)); >> + j = (i + rrp->current) % rrp->peers->number; >> + n = j / (8 * sizeof(uintptr_t)); >> + m = (uintptr_t) 1 << j % (8 * sizeof(uintptr_t)); > > Note this might affect peer selection in some situations. If > there are two peers with identical effective weight, peer > selection now depends on rrp->current (previous peer tried in a > normal case), while with previous code selection was stable and > always used first one. Yes, thanks for catching that. Possibly, other scenarios were affected too... Which is not good. Tried to fix that problem, but after various experiments, realized that it all essentially comes down to the need to reorder items in the addrs array in-place. Which is ok..., but again felt like it was the wrong place to do that. So, i decided instead to take a look at ngx_resolver code and it turned out that it's actually very simple to randomize there, since there is code already that copies the array from one place in memory to another. Looked like that was ideal place for changing the order via simple rotation. Here is the new patch: --- a/src/core/ngx_resolver.c +++ b/src/core/ngx_resolver.c @@ -88,6 +88,8 @@ static void *ngx_resolver_calloc(ngx_resolver_t *r, size_t size); static void ngx_resolver_free(ngx_resolver_t *r, void *p); static void ngx_resolver_free_locked(ngx_resolver_t *r, void *p); static void *ngx_resolver_dup(ngx_resolver_t *r, void *src, size_t size); +static in_addr_t *ngx_resolver_dup_rotated(ngx_resolver_t *r, in_addr_t *src, + u_short n); static u_char *ngx_resolver_log_error(ngx_log_t *log, u_char *buf, size_t len); @@ -445,8 +447,8 @@ ngx_resolve_name_locked(ngx_resolver_t *r, ngx_resolver_ctx_t *ctx) if (naddrs != 1) { addr = 0; - addrs = ngx_resolver_dup(r, rn->u.addrs, - naddrs * sizeof(in_addr_t)); + addrs = ngx_resolver_dup_rotated(r, rn->u.addrs, + naddrs); if (addrs == NULL) { return NGX_ERROR; } @@ -1381,8 +1383,7 @@ ngx_resolver_process_a(ngx_resolver_t *r, u_char *buf, size_t last, rn->u.addrs = addrs; - addrs = ngx_resolver_dup(r, rn->u.addrs, - naddrs * sizeof(in_addr_t)); + addrs = ngx_resolver_dup_rotated(r, rn->u.addrs, naddrs); if (addrs == NULL) { return; } @@ -2134,6 +2135,32 @@ ngx_resolver_dup(ngx_resolver_t *r, void *src, size_t size) return dst; } +static in_addr_t * +ngx_resolver_dup_rotated(ngx_resolver_t *r, in_addr_t *src, u_short n) +{ + in_addr_t *dst; + int j; + + dst = ngx_resolver_alloc(r, n * sizeof(in_addr_t)); + + if (dst == NULL) { + return dst; + } + + j = ngx_random() % n; + +#if (NGX_DEBUG) + ngx_log_debug1(NGX_LOG_DEBUG_CORE, ngx_cycle->log, 0, + "random rotation of addrs: %d", j); +#endif + + ngx_memcpy(dst, src + j, (n - j) * sizeof(in_addr_t)); + if (j > 0) { + ngx_memcpy(dst + (n - j), src, j * sizeof(in_addr_t)); + } + + return dst; +} char * ngx_resolver_strerror(ngx_int_t err) From mdounin at mdounin.ru Thu Sep 20 07:52:59 2012 From: mdounin at mdounin.ru (Maxim Dounin) Date: Thu, 20 Sep 2012 11:52:59 +0400 Subject: Can not use $bytes_sent in module (complex values) In-Reply-To: <1348098200.250143680@f105.mail.ru> References: <20120919000656.GA40452@mdounin.ru> <1348098200.250143680@f105.mail.ru> Message-ID: <20120920075259.GM40452@mdounin.ru> Hello! On Thu, Sep 20, 2012 at 03:43:20AM +0400, Vladimir Shebordaev wrote: > Wed, 19 Sep 2012 23:11:57 +0200 ?? Benjamin Gr?ssing : > >2012/9/20 Benjamin Gr?ssing : > > > >Hi Vladimir, hi Maxim! > > > >Thanks for your answers. I've had a look at the rewrite module - > >unfortunately it does not support the $bytes_sent variable either as > >configuring "if ($bytes_sent) { }" throws the error: [emerg] unknown > >"bytes_sent" variable. > Yes, this is what the indexed variables are. They have no name > at?runtime and accessed by the index into the request->variables > array by the?script engine that is to evaluate a complex value > and if conditionals.?This is faster to do, since it helps to > avoid the cmcf->variables_hash?lookup and a memory allocation at > evaluation time having the space for?the value reserved at the > request initialization time. But the index?value must be known > to the rewrite module at the configuration time. No. The $bytes_sent variable isn't actually variable at all: it's only available in log module via it's own machinery, not as a generic variable. It's completely unrelated to indexed variables access. [...] Maxim Dounin From mdounin at mdounin.ru Thu Sep 20 08:04:58 2012 From: mdounin at mdounin.ru (Maxim Dounin) Date: Thu, 20 Sep 2012 12:04:58 +0400 Subject: Can not use $bytes_sent in module (complex values) In-Reply-To: References: <20120919000656.GA40452@mdounin.ru> Message-ID: <20120920080457.GN40452@mdounin.ru> Hello! On Wed, Sep 19, 2012 at 11:11:57PM +0200, Benjamin Gr?ssing wrote: > Hi Vladimir, hi Maxim! > > Thanks for your answers. I've had a look at the rewrite module - > unfortunately it does not support the $bytes_sent variable either as > configuring "if ($bytes_sent) { }" throws the error: [emerg] unknown > "bytes_sent" variable. Yes, this is expected as $bytes_sent isn't something available outside of log module. And that's why we just need to make it normal variable to avoid confusion, much like it was recently done with $status (http://trac.nginx.org/nginx/changeset/4686/nginx). > Nice to hear that there is a plan to include all variables in the > complex value system - I think that would make that part a of nginx a > lot more clear! I'd be happy if I could contribute here, but I am not > sure about the details. Is there any guide to contributing to nginx > available or do I just have to create a patch file and post it here? > Can I just start with the nginx 1.2.3 source code from nginx.org? There is no guide (probably we should write one), but posting patches here is a right way to go. It's usually better to use svn trunk or latest devel version as a base. Maxim Dounin > > Regards > Benjamin > > 2012/9/19 Maxim Dounin : > > Hello! > > > > On Tue, Sep 18, 2012 at 03:55:38PM +0200, Benjamin Gr?ssing wrote: > > > >> Hello, > >> > >> I am currently working on a nginx module, that parses user defined > >> strings for variables using ngx_http_complex_value_t. That works great > >> with all variables defined in ngx_http_variables.c (i.e. > >> $body_bytes_sent). However, variables as $bytes_sent do not exist here > >> since ngx_http_log_module uses its own variable parsing system. What > >> is the best way to use these extra variables in my module? I have > >> thought of the following options, I am not very sure about what is the > >> best way to go: > >> a) Copy the ngx_http_log_module code in my own module and rename every > >> single type and function to avoid double definitions. Reuse functions > >> and types from there. > >> b) Move the ngx_http_log_module code in my own module directory and > >> compile it only there. Reuse functions and types from there. > >> c) Write own variable parsing system, reuse a few snippets from > >> ngx_http_variables and ngx_http_log_module, whatever fits best. > >> d) Use complex values and try to extend the ngx_http_core_variables > >> array to include $body_bytes_sent and the other missing variables (I > >> don't understand why they are not included here anyway?). > >> e) Any better way to achieve that? > >> > >> I'd really appreciate your help. > > > > All variables that are currently present only in log module should > > really be made available as normal variables as well. They aren't > > yet mostly due to historical reasons (log module appeared before > > normal variables were implemented) and the fact that most of > > them are only usable for logging. > > > > No idea what would be best for your particular module, but in > > general it's better to use complex values and what's available via > > normal variables. Unless you need $bytes_sent right now - it's > > probably the best aproach, and missing variables will be available > > once they appear in nginx. > > > > (The latter is something you may speedup by submitting a good > > patch. Most tricky part here would be to follow coding style.) > > > > Maxim Dounin > > > > _______________________________________________ > > nginx-devel mailing list > > nginx-devel at nginx.org > > http://mailman.nginx.org/mailman/listinfo/nginx-devel > > _______________________________________________ > nginx-devel mailing list > nginx-devel at nginx.org > http://mailman.nginx.org/mailman/listinfo/nginx-devel From toli at webforge.bg Thu Sep 20 09:12:58 2012 From: toli at webforge.bg (Anatoli Marinov) Date: Thu, 20 Sep 2012 12:12:58 +0300 Subject: how to redirect user from handler Message-ID: <505ADE1A.4060508@webforge.bg> Hello Colleagues, Which is the right way to redirect user request from handler module. Now I have something like this in my handler: My handler function(r) { header = ngx_list_push(&r->headers_out.headers); header->key.len = sizeof("Location") - 1; header->key.data = (u_char *) "Location"; header->value.len = strlen("http://new_location.com/1.dat"); header->value.data = ngx_pcalloc(r->pool, header->value.len); ngx_snprintf(header->value.data, header->value.len, "%s", "http://new_location.com/1.dat"); r->headers_out.status = NGX_HTTP_MOVED_TEMPORARILY; r->chunked = 0; ngx_http_send_header(r); return NGX_DECLINED; } I think this approach is wrong. It sends 302 to the client but there is a body with 404 return code which is not correct. So please advise me how to redirect from hander. Thanks in advance Anatoli Marinov From toli at webforge.bg Thu Sep 20 09:31:12 2012 From: toli at webforge.bg (Anatoli Marinov) Date: Thu, 20 Sep 2012 12:31:12 +0300 Subject: how to redirect user from handler In-Reply-To: <505ADE1A.4060508@webforge.bg> References: <505ADE1A.4060508@webforge.bg> Message-ID: <505AE260.2090204@webforge.bg> I tried to send my buffer with a message and the code looks like this: static ngx_int_t ngx_http_my_handler(ngx_http_request_t *r) { header = ngx_list_push(&r->headers_out.headers); if (header == NULL) { return NGX_HTTP_INTERNAL_SERVER_ERROR; } header->value.len = strlen("http://new_location.com/1.dat"); header->value.data = ngx_pcalloc(r->pool, header->value.len); ngx_snprintf(header->value.data, header->value.len, "%s", "http://new_location.com/1.dat"); r->headers_out.status = NGX_HTTP_MOVED_TEMPORARILY; r->chunked = 0; ngx_http_send_header(r); b = ngx_pcalloc(r->pool, sizeof(ngx_buf_t)); if (b == NULL) { return NGX_HTTP_INTERNAL_SERVER_ERROR; } b->pos = "Go to new address"; b->last = "Go to new address" + sizeof("Go to new address"); b->memory = 1; b->last_buf = 1; out.buf = b; out.next = NULL; return ngx_http_output_filter(r, &out); } And this seems to work but 302 answer has a body and it is a chunked response? Is it possible to send it without body. I also do not want to be chunked if it is possible. On 09/20/2012 12:12 PM, Anatoli Marinov wrote: > Hello Colleagues, > Which is the right way to redirect user request from handler module. > Now I have something like this in my handler: > > My handler function(r) { > header = ngx_list_push(&r->headers_out.headers); > header->key.len = sizeof("Location") - 1; > header->key.data = (u_char *) "Location"; > > header->value.len = strlen("http://new_location.com/1.dat"); > header->value.data = ngx_pcalloc(r->pool, header->value.len); > > ngx_snprintf(header->value.data, header->value.len, "%s", > "http://new_location.com/1.dat"); > r->headers_out.status = NGX_HTTP_MOVED_TEMPORARILY; > r->chunked = 0; > > ngx_http_send_header(r); > > return NGX_DECLINED; > } > > I think this approach is wrong. It sends 302 to the client but there > is a body with 404 return code which is not correct. > > > So please advise me how to redirect from hander. > > Thanks in advance > Anatoli Marinov > > _______________________________________________ > nginx-devel mailing list > nginx-devel at nginx.org > http://mailman.nginx.org/mailman/listinfo/nginx-devel From mdounin at mdounin.ru Thu Sep 20 13:17:29 2012 From: mdounin at mdounin.ru (Maxim Dounin) Date: Thu, 20 Sep 2012 17:17:29 +0400 Subject: how to redirect user from handler In-Reply-To: <505AE260.2090204@webforge.bg> References: <505ADE1A.4060508@webforge.bg> <505AE260.2090204@webforge.bg> Message-ID: <20120920131729.GT40452@mdounin.ru> Hello! On Thu, Sep 20, 2012 at 12:31:12PM +0300, Anatoli Marinov wrote: [...] > And this seems to work but 302 answer has a body and it is a chunked > response? This is because you said it to. > Is it possible to send it without body. No, it's not allowed by a protocol (unless request was HEAD). Response body must be present. It may be empty though. > I also do not want to be chunked if it is possible. It is. You should set content length, then chunked encoding won't be used. You may take a look at ngx_http_send_response() for a complete code (and the function is actually a usefull helper for simple cases when you need to return some predefined response). But actually to return a 302 redirect from a content handler it's enough to set Location header and return NGX_HTTP_MOVED_TEMPORARILY, i.e. to do something like this: ngx_http_clear_location(r); r->headers_out.location = ngx_palloc(r->pool, sizeof(ngx_table_elt_t)); if (r->headers_out.location == NULL) { return NGX_HTTP_INTERNAL_SERVER_ERROR; } /* * we do not need to set the r->headers_out.location->hash and * r->headers_out.location->key fields */ ngx_str_set(&r->headers_out.location->value, "http://example.com/"); return NGX_HTTP_MOVED_TEMPORARILY; (The code was mostly borrowed from static module, with minor modification.) Maxim Dounin > > > > On 09/20/2012 12:12 PM, Anatoli Marinov wrote: > >Hello Colleagues, > >Which is the right way to redirect user request from handler > >module. Now I have something like this in my handler: > > > >My handler function(r) { > > header = ngx_list_push(&r->headers_out.headers); > > header->key.len = sizeof("Location") - 1; > > header->key.data = (u_char *) "Location"; > > > > header->value.len = strlen("http://new_location.com/1.dat"); > > header->value.data = ngx_pcalloc(r->pool, header->value.len); > > > > ngx_snprintf(header->value.data, header->value.len, "%s", > >"http://new_location.com/1.dat"); > > r->headers_out.status = NGX_HTTP_MOVED_TEMPORARILY; > > r->chunked = 0; > > > > ngx_http_send_header(r); > > > > return NGX_DECLINED; > >} > > > >I think this approach is wrong. It sends 302 to the client but > >there is a body with 404 return code which is not correct. > > > > > >So please advise me how to redirect from hander. > > > >Thanks in advance > >Anatoli Marinov > > > >_______________________________________________ > >nginx-devel mailing list > >nginx-devel at nginx.org > >http://mailman.nginx.org/mailman/listinfo/nginx-devel > > _______________________________________________ > nginx-devel mailing list > nginx-devel at nginx.org > http://mailman.nginx.org/mailman/listinfo/nginx-devel From toli at webforge.bg Thu Sep 20 13:48:38 2012 From: toli at webforge.bg (Anatoli Marinov) Date: Thu, 20 Sep 2012 16:48:38 +0300 Subject: how to redirect user from handler In-Reply-To: <20120920131729.GT40452@mdounin.ru> References: <505ADE1A.4060508@webforge.bg> <505AE260.2090204@webforge.bg> <20120920131729.GT40452@mdounin.ru> Message-ID: <505B1EB6.9030203@webforge.bg> Thank you very much Maxim! On 09/20/2012 04:17 PM, Maxim Dounin wrote: > Hello! > > On Thu, Sep 20, 2012 at 12:31:12PM +0300, Anatoli Marinov wrote: > > [...] > >> And this seems to work but 302 answer has a body and it is a chunked >> response? > This is because you said it to. > >> Is it possible to send it without body. > No, it's not allowed by a protocol (unless request was HEAD). > Response body must be present. It may be empty though. > >> I also do not want to be chunked if it is possible. > It is. You should set content length, then chunked encoding won't > be used. You may take a look at ngx_http_send_response() for a > complete code (and the function is actually a usefull helper for > simple cases when you need to return some predefined response). > > But actually to return a 302 redirect from a content handler it's > enough to set Location header and return NGX_HTTP_MOVED_TEMPORARILY, > i.e. to do something like this: > > ngx_http_clear_location(r); > > r->headers_out.location = ngx_palloc(r->pool, sizeof(ngx_table_elt_t)); > if (r->headers_out.location == NULL) { > return NGX_HTTP_INTERNAL_SERVER_ERROR; > } > > /* > * we do not need to set the r->headers_out.location->hash and > * r->headers_out.location->key fields > */ > > ngx_str_set(&r->headers_out.location->value, "http://example.com/"); > > return NGX_HTTP_MOVED_TEMPORARILY; > > (The code was mostly borrowed from static module, with minor > modification.) > > Maxim Dounin > > >> >> >> On 09/20/2012 12:12 PM, Anatoli Marinov wrote: >>> Hello Colleagues, >>> Which is the right way to redirect user request from handler >>> module. Now I have something like this in my handler: >>> >>> My handler function(r) { >>> header = ngx_list_push(&r->headers_out.headers); >>> header->key.len = sizeof("Location") - 1; >>> header->key.data = (u_char *) "Location"; >>> >>> header->value.len = strlen("http://new_location.com/1.dat"); >>> header->value.data = ngx_pcalloc(r->pool, header->value.len); >>> >>> ngx_snprintf(header->value.data, header->value.len, "%s", >>> "http://new_location.com/1.dat"); >>> r->headers_out.status = NGX_HTTP_MOVED_TEMPORARILY; >>> r->chunked = 0; >>> >>> ngx_http_send_header(r); >>> >>> return NGX_DECLINED; >>> } >>> >>> I think this approach is wrong. It sends 302 to the client but >>> there is a body with 404 return code which is not correct. >>> >>> >>> So please advise me how to redirect from hander. >>> >>> Thanks in advance >>> Anatoli Marinov >>> >>> _______________________________________________ >>> nginx-devel mailing list >>> nginx-devel at nginx.org >>> http://mailman.nginx.org/mailman/listinfo/nginx-devel >> _______________________________________________ >> nginx-devel mailing list >> nginx-devel at nginx.org >> http://mailman.nginx.org/mailman/listinfo/nginx-devel > _______________________________________________ > nginx-devel mailing list > nginx-devel at nginx.org > http://mailman.nginx.org/mailman/listinfo/nginx-devel From zhuzhaoyuan at gmail.com Fri Sep 21 04:31:48 2012 From: zhuzhaoyuan at gmail.com (Joshua Zhu) Date: Fri, 21 Sep 2012 12:31:48 +0800 Subject: [PATCH] Update the help message of configure Message-ID: Hi, It seems the help message of configure was not updated when ngx_http_upstream_least_conn_module and ngx_http_upstream_keepalive_module were imported. Below attaches a patch for this. Hope it helps :) $ svn diff Index: auto/options =================================================================== --- auto/options (revision 4846) +++ auto/options (working copy) @@ -388,6 +388,10 @@ --without-http_browser_module disable ngx_http_browser_module --without-http_upstream_ip_hash_module disable ngx_http_upstream_ip_hash_module + --without-http_upstream_least_conn_module + disable ngx_http_upstream_least_conn_module + --without-http_upstream_keepalive_module + disable ngx_http_upstream_keepalive_module --with-http_perl_module enable ngx_http_perl_module --with-perl_modules_path=PATH set Perl modules path Regards, -- Joshua Zhu Senior Software Engineer Server Platforms Team at Taobao -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: configure_help.patch Type: application/octet-stream Size: 745 bytes Desc: not available URL: From dirk.feytons at gmail.com Fri Sep 21 07:28:16 2012 From: dirk.feytons at gmail.com (Dirk Feytons) Date: Fri, 21 Sep 2012 09:28:16 +0200 Subject: [PATCH] Unix domain sockets in abstract namespace In-Reply-To: <20120919135608.GH40452@mdounin.ru> References: <20120919135608.GH40452@mdounin.ru> Message-ID: On Wed, Sep 19, 2012 at 3:56 PM, Maxim Dounin wrote: > Hello! > > On Wed, Sep 12, 2012 at 11:46:52AM +0200, Dirk Feytons wrote: > >> Hi, >> >> I was playing around with ngx_lua and its socket API, which is built >> on top of nginx functionality. The API supports unix domain sockets >> but not in the abstract namespace, at least not officially. When I >> tried specifying an abstract address I found it didn't work. I tracked >> it down to two calls to ngx_cpystrn() in >> core/ngx_inet.c:ngx_parse_unix_domain_url(). That function gets >> confused by the leading '\0' in an abstract namespace address. > > This is because the abstract namespace is actually linux extension > to unix domain sockets, "The abstract socket namespace is a > nonportable Linux extension" (from [1]). > > Strictly speaking, it's not something even permitted by POSIX as > socket path ends with first null byte as per POSIX. Yeah; I know it's Linux specific. Too bad in my opinion; I find it easier to work with than the regular namespace. >> Changing it to ngx_memcpy() fixes this and then everything works. >> >> I attached a patch agains nginx 1.2.3. The fix seems trivial but this >> is my first look into the nginx internals so I don't know whether >> there are unwanted side effects from this change. Feedback welcome. > > I suspect there may be some, in particular during logging of > errors and probably while parsing untrusted urls. Okay, I'll have a look at the code again. Are you willing to accept a patch to support the abstract namespace? Or does the fact that it's a nonportable Linux extension make it a no-go for inclusion in nginx? > [1] http://www.kernel.org/doc/man-pages/online/pages/man7/unix.7.html Thanks for the feedback. Dirk F. From vshebordaev at mail.ru Fri Sep 21 08:17:18 2012 From: vshebordaev at mail.ru (=?UTF-8?B?VmxhZGltaXIgU2hlYm9yZGFldg==?=) Date: Fri, 21 Sep 2012 12:17:18 +0400 Subject: Can not use $bytes_sent in module (complex values) In-Reply-To: References: <20120919000656.GA40452@mdounin.ru> Message-ID: <1348215438.687512098@f258.mail.ru> Benjamin, it seems?I used not quite neat wording, so?my earlier explanations might look obfuscating. Basically, the indexed variable is any "automatic" variable referenced?thow the request->variables array. The variables of the current log?module do use index slots, so they always have space for the value?preallocated in ngx_http_init_request(). But those variables can be?referenced from a script or by another module neither by name since?they are missing from cmcf->variables_hash, nor by index, ?e.g. by a?complex value, since they have no getter method thus are rejected in?ngx_http_variables_init_vars() as unknown. If you modify the log module to register its variables with?nxg_http_add_variable() and implement correct get handlers, they would surely?be available everywhere like any other variable. For performance?reasons the log module in itself should still use its variables as?indexed ones, since log operations are expensive and rather frequent.?If you implement the generic getter method for the http variable?infrastructure, you might immediatly use it in the log module handler probably using their own machinery. Hope it helps. Regards, Vladimir Wed, 19 Sep 2012 23:11:57 +0200 ?? Benjamin Gr?ssing : > > > > >Hi Vladimir, hi Maxim! > > Thanks for your answers. I've had a look at the rewrite module - > unfortunately it does not support the $bytes_sent variable either as > configuring "if ($bytes_sent) { }" throws the error: [emerg] unknown > "bytes_sent" variable. > > Nice to hear that there is a plan to include all variables in the > complex value system - I think that would make that part a of nginx a > lot more clear! I'd be happy if I could contribute here, but I am not > sure about the details. Is there any guide to contributing to nginx > available or do I just have to create a patch file and post it here? > Can I just start with the nginx 1.2.3 source code from nginx.org? > > Regards > Benjamin > > 2012/9/19 Maxim Dounin : > > Hello! > > > > On Tue, Sep 18, 2012 at 03:55:38PM +0200, Benjamin Gr?ssing wrote: > > > >> Hello, > >> > >> I am currently working on a nginx module, that parses user defined > >> strings for variables using ngx_http_complex_value_t. That works great > >> with all variables defined in ngx_http_variables.c (i.e. > >> $body_bytes_sent). However, variables as $bytes_sent do not exist here > >> since ngx_http_log_module uses its own variable parsing system. What > >> is the best way to use these extra variables in my module? I have > >> thought of the following options, I am not very sure about what is the > >> best way to go: > >> a) Copy the ngx_http_log_module code in my own module and rename every > >> single type and function to avoid double definitions. Reuse functions > >> and types from there. > >> b) Move the ngx_http_log_module code in my own module directory and > >> compile it only there. Reuse functions and types from there. > >> c) Write own variable parsing system, reuse a few snippets from > >> ngx_http_variables and ngx_http_log_module, whatever fits best. > >> d) Use complex values and try to extend the ngx_http_core_variables > >> array to include $body_bytes_sent and the other missing variables (I > >> don't understand why they are not included here anyway?). > >> e) Any better way to achieve that? > >> > >> I'd really appreciate your help. > > > > All variables that are currently present only in log module should > > really be made available as normal variables as well. They aren't > > yet mostly due to historical reasons (log module appeared before > > normal variables were implemented) and the fact that most of > > them are only usable for logging. > > > > No idea what would be best for your particular module, but in > > general it's better to use complex values and what's available via > > normal variables. Unless you need $bytes_sent right now - it's > > probably the best aproach, and missing variables will be available > > once they appear in nginx. > > > > (The latter is something you may speedup by submitting a good > > patch. Most tricky part here would be to follow coding style.) > > > > Maxim Dounin > > > > _______________________________________________ > > nginx-devel mailing list > > nginx-devel at nginx.org > > http://mailman.nginx.org/mailman/listinfo/nginx-devel > > _______________________________________________ > nginx-devel mailing list >nginx-devel at nginx.org >http://mailman.nginx.org/mailman/listinfo/nginx-devel > -------------- next part -------------- An HTML attachment was scrubbed... URL: From fs20063 at outlook.com Fri Sep 21 16:01:54 2012 From: fs20063 at outlook.com (Filipe Silva) Date: Fri, 21 Sep 2012 17:01:54 +0100 Subject: nginx filter to concat text to a response body Message-ID: Hello, I've been trying to produce a nginx filter to concat text to a response body but without success. An important detail is that it works with a nginx configured as a web server but not with a nginx server configured as reverse proxy. Should I be using an handler instead? Thank you, Filipe S -------------- next part -------------- An HTML attachment was scrubbed... URL: From zhuzhaoyuan at gmail.com Fri Sep 21 17:45:40 2012 From: zhuzhaoyuan at gmail.com (Joshua Zhu) Date: Sat, 22 Sep 2012 01:45:40 +0800 Subject: nginx filter to concat text to a response body In-Reply-To: References: Message-ID: Hi Filipe, On Sat, Sep 22, 2012 at 12:01 AM, Filipe Silva wrote: > Hello, > > I've been trying to produce a nginx filter to concat text to a response > body but without success. > > An important detail is that it works with a nginx configured as a web > server but not with a nginx server configured as reverse proxy. > > Should I be using an handler instead? > Is this what you are looking for? https://github.com/taobao/nginx-http-footer-filter Regards, -- Joshua Zhu Senior Software Engineer Server Platforms Team at Taobao -------------- next part -------------- An HTML attachment was scrubbed... URL: From fs20063 at outlook.com Sat Sep 22 05:28:50 2012 From: fs20063 at outlook.com (Filipe Silva) Date: Sat, 22 Sep 2012 06:28:50 +0100 Subject: nginx filter to concat text to a response body In-Reply-To: References: , Message-ID: Hi Zhu, I tried taobao's module and it works like a charm if nginx is configured as a web server: server { listen 80; server_name localhost; location / { footer ""; index index.html; } } The result is a html comment with a gmt date like added to the end of the response body. But if configured like a reverse proxy it does not add the comment. server { listen 80; server_name localhost; location / { proxy_pass http://127.0.0.1:8080/; footer ""; }} Well, this simple nginx location configuration points the traffic to a upstream web server. As expected, on return I don't get the nginx's default index.html, but the upstream html page instead, with the difference that the comment is not added. I'm pretty sure I'm missing something here. I'm starting to believe that if I want to write to a response body served by a upstream server a filter may not be the answer. I've tried evanmiller.org guide and I found the section "3.2. Anatomy of an Upstream (a.k.a Proxy) Handler" (http://www.evanmiller.org/nginx-modules-guide.html#proxying) and I have the impression that this might help me finding the answer, but I'm failing to reproduce the example because at the moment I'm not as familiar with the guts of an handler as I am with a filter. Anyway, I don't even know if an handler will do the trick. So any help is welcome :) Thank you. Date: Sat, 22 Sep 2012 01:45:40 +0800 Subject: Re: nginx filter to concat text to a response body From: zhuzhaoyuan at gmail.com To: nginx-devel at nginx.org Hi Filipe, On Sat, Sep 22, 2012 at 12:01 AM, Filipe Silva wrote: Hello, I've been trying to produce a nginx filter to concat text to a response body but without success. An important detail is that it works with a nginx configured as a web server but not with a nginx server configured as reverse proxy. Should I be using an handler instead? Is this what you are looking for? https://github.com/taobao/nginx-http-footer-filter Regards, -- Joshua Zhu Senior Software Engineer Server Platforms Team at Taobao _______________________________________________ nginx-devel mailing list nginx-devel at nginx.org http://mailman.nginx.org/mailman/listinfo/nginx-devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From zhuzhaoyuan at gmail.com Sat Sep 22 06:21:58 2012 From: zhuzhaoyuan at gmail.com (Joshua Zhu) Date: Sat, 22 Sep 2012 14:21:58 +0800 Subject: nginx filter to concat text to a response body In-Reply-To: References: Message-ID: Hi, On Sat, Sep 22, 2012 at 1:28 PM, Filipe Silva wrote: > Hi Zhu, > > I tried taobao's module and it works like a charm if nginx is configured > as a web server: > > server { > listen 80; > server_name localhost; > > location / { > footer ""; > index index.html; > } > } > > The result is a html comment with a gmt date like > added to the end of the response body. > > But if configured like a reverse proxy it does not add the comment. > > server { > listen 80; > server_name localhost; > > location / { > proxy_pass http://127.0.0.1:8080/; > footer ""; > } > } > > Well, this simple nginx location configuration points the traffic to a > upstream web server. As expected, on return I don't get the nginx's default > index.html, but the upstream html page instead, with the difference that > the comment is not added. > Strange. It should work. What is the 'Content-Type' of the output of your upstream server, by the way? By default the 'footer' module only processes contents in 'text/html'. You can add more content types by using the 'footer_types' directive though. And I tested the following configuration. The footer was correctly added. server { listen 80; server_name localhost; location / { root html; index index.html index.htm; proxy_pass http://127.0.0.1:8080; footer "this is a footer\n"; } } server { listen 8080; server_name hi; location / { root html2; index index.html index.htm; } } > I'm pretty sure I'm missing something here. I'm starting to believe that if > I want to write to a response body served by a upstream server a filter may > not be the answer. > > I've tried evanmiller.org guide and I found the section "3.2. Anatomy of > an Upstream (a.k.a Proxy) Handler" ( > http://www.evanmiller.org/nginx-modules-guide.html#proxying) and I have > the impression that this might help me finding the answer, but I'm failing > to reproduce the example because at the moment I'm not as familiar with the > guts of an handler as I am with a filter. > > Anyway, I don't even know if an handler will do the trick. So any help is > welcome :) > There is no doubt that the module should be a filter, not a handler. The filters will be called no matter whether you are accessing an upstream server or not, since they are _output body_ filters. Regards, -- Joshua Zhu Senior Software Engineer Server Platforms Team at Taobao -------------- next part -------------- An HTML attachment was scrubbed... URL: From igor at sysoev.ru Sat Sep 22 06:35:37 2012 From: igor at sysoev.ru (Igor Sysoev) Date: Sat, 22 Sep 2012 10:35:37 +0400 Subject: nginx filter to concat text to a response body In-Reply-To: References: Message-ID: <20120922063537.GA30907@nginx.com> On Fri, Sep 21, 2012 at 05:01:54PM +0100, Filipe Silva wrote: > Hello, > I've been trying to produce a nginx filter to concat text to a response body but without success. > An important detail is that it works with a nginx configured as a web server but not with a nginx server configured as reverse proxy. > Should I be using an handler instead? > Thank you, > Filipe S http://nginx.org/en/docs/http/ngx_http_addition_module.html -- Igor Sysoev http://nginx.com/support.html From fs20063 at outlook.com Sat Sep 22 08:23:16 2012 From: fs20063 at outlook.com (Filipe Silva) Date: Sat, 22 Sep 2012 09:23:16 +0100 Subject: nginx filter to concat text to a response body In-Reply-To: References: , , , Message-ID: Hi, If I setup to 2 nginx servers, one as the reverse proxy and the other as the web server - exactly what you did on your example - the footer is added to the response body. Well, that works and confirms that filter is the way to go :) However, what I'm using here as web server is not nginx but an apache server. The 'Content-Type' is text/html. The 'Content-Length' grows from 146 (apache web server port 8080) to 163 (nginx reverse proxy port 80) but the footer "this is a footer\n" (17 bytes long) is not added by the module. I'm taking a look at apache default configuration to check if there is something that is making nginx behave differently. Below are the response headers from each case: Response headers from reverse proxy nginx 80 HTTP/1.1 200 OKServer: nginx/1.3.5Date: Sat, 22 Sep 2012 08:13:08 GMTContent-Type: text/htmlContent-Length: 163Connection: keep-aliveLast-Modified: Thu, 06 Sep 2012 13:39:44 GMTEtag: "81333-b1-4c9089fc7c802"Vary: Accept-EncodingContent-Encoding: gzip Response headers from web server apache 8080 HTTP/1.1 200 OKDate: Sat, 22 Sep 2012 08:13:13 GMTServer: Apache/2.2.16 (Ubuntu)Last-Modified: Thu, 06 Sep 2012 13:39:44 GMTEtag: "81333-b1-4c9089fc7c802"Accept-Ranges: bytesVary: Accept-EncodingContent-Encoding: gzipContent-Length: 146Keep-Alive: timeout=15, max=100Connection: Keep-AliveContent-Type: text/html On the meanwhile if you have a clue what is happening I'll be thankful. Thank you, Filipe S Date: Sat, 22 Sep 2012 14:21:58 +0800 Subject: Re: nginx filter to concat text to a response body From: zhuzhaoyuan at gmail.com To: nginx-devel at nginx.org Hi, On Sat, Sep 22, 2012 at 1:28 PM, Filipe Silva wrote: Hi Zhu, I tried taobao's module and it works like a charm if nginx is configured as a web server: server { listen 80; server_name localhost; location / { footer ""; index index.html; } } The result is a html comment with a gmt date like added to the end of the response body. But if configured like a reverse proxy it does not add the comment. server { listen 80; server_name localhost; location / { proxy_pass http://127.0.0.1:8080/; footer ""; }} Well, this simple nginx location configuration points the traffic to a upstream web server. As expected, on return I don't get the nginx's default index.html, but the upstream html page instead, with the difference that the comment is not added. Strange. It should work. What is the 'Content-Type' of the output of your upstream server, by the way? By default the 'footer' module only processes contents in 'text/html'. You can add more content types by using the 'footer_types' directive though. And I tested the following configuration. The footer was correctly added. server { listen 80; server_name localhost; location / { root html; index index.html index.htm; proxy_pass http://127.0.0.1:8080; footer "this is a footer\n"; } } server { listen 8080; server_name hi; location / { root html2; index index.html index.htm; } } I'm pretty sure I'm missing something here. I'm starting to believe that if I want to write to a response body served by a upstream server a filter may not be the answer. I've tried evanmiller.org guide and I found the section "3.2. Anatomy of an Upstream (a.k.a Proxy) Handler" (http://www.evanmiller.org/nginx-modules-guide.html#proxying) and I have the impression that this might help me finding the answer, but I'm failing to reproduce the example because at the moment I'm not as familiar with the guts of an handler as I am with a filter. Anyway, I don't even know if an handler will do the trick. So any help is welcome :) There is no doubt that the module should be a filter, not a handler. The filters will be called no matter whether you are accessing an upstream server or not, since they are _output body_ filters. Regards, -- Joshua Zhu Senior Software Engineer Server Platforms Team at Taobao _______________________________________________ nginx-devel mailing list nginx-devel at nginx.org http://mailman.nginx.org/mailman/listinfo/nginx-devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From zhuzhaoyuan at gmail.com Sat Sep 22 08:55:31 2012 From: zhuzhaoyuan at gmail.com (Joshua Zhu) Date: Sat, 22 Sep 2012 16:55:31 +0800 Subject: nginx filter to concat text to a response body In-Reply-To: References: Message-ID: Hi, On Sat, Sep 22, 2012 at 4:23 PM, Filipe Silva wrote: > Hi, > [...] I'm taking a look at apache default configuration to check if there is > something that is making nginx behave differently. Below are the > response headers from each case: > > Response headers from reverse proxy nginx 80 > > HTTP/1.1 200 OK > Server: nginx/1.3.5 > Date: Sat, 22 Sep 2012 08:13:08 GMT > Content-Type: text/html > Content-Length: 163 > Connection: keep-alive > Last-Modified: Thu, 06 Sep 2012 13:39:44 GMT > Etag: "81333-b1-4c9089fc7c802" > Vary: Accept-Encoding > Content-Encoding: gzip > > Response headers from web server apache 8080 > > HTTP/1.1 200 OK > Date: Sat, 22 Sep 2012 08:13:13 GMT > Server: Apache/2.2.16 (Ubuntu) > Last-Modified: Thu, 06 Sep 2012 13:39:44 GMT > Etag: "81333-b1-4c9089fc7c802" > Accept-Ranges: bytes > Vary: Accept-Encoding > Content-Encoding: gzip > Content-Length: 146 > Keep-Alive: timeout=15, max=100 > Connection: Keep-Alive > Content-Type: text/html > > On the meanwhile if you have a clue what is happening I'll be thankful. > Please turn off gzip on Apache side then try again :) Regards, -- Joshua Zhu Senior Software Engineer Server Platforms Team at Taobao -------------- next part -------------- An HTML attachment was scrubbed... URL: From fs20063 at outlook.com Sat Sep 22 10:03:42 2012 From: fs20063 at outlook.com (Filipe Silva) Date: Sat, 22 Sep 2012 11:03:42 +0100 Subject: nginx filter to concat text to a response body In-Reply-To: References: , , , , , Message-ID: Tunnel vision 1 - Filipe 0 Thanks Zhu ;) Date: Sat, 22 Sep 2012 16:55:31 +0800 Subject: Re: nginx filter to concat text to a response body From: zhuzhaoyuan at gmail.com To: nginx-devel at nginx.org Hi, On Sat, Sep 22, 2012 at 4:23 PM, Filipe Silva wrote: Hi, [...] I'm taking a look at apache default configuration to check if there is something that is making nginx behave differently. Below are the response headers from each case: Response headers from reverse proxy nginx 80 HTTP/1.1 200 OKServer: nginx/1.3.5Date: Sat, 22 Sep 2012 08:13:08 GMT Content-Type: text/htmlContent-Length: 163Connection: keep-alive Last-Modified: Thu, 06 Sep 2012 13:39:44 GMTEtag: "81333-b1-4c9089fc7c802"Vary: Accept-Encoding Content-Encoding: gzip Response headers from web server apache 8080 HTTP/1.1 200 OKDate: Sat, 22 Sep 2012 08:13:13 GMTServer: Apache/2.2.16 (Ubuntu)Last-Modified: Thu, 06 Sep 2012 13:39:44 GMT Etag: "81333-b1-4c9089fc7c802"Accept-Ranges: bytesVary: Accept-EncodingContent-Encoding: gzipContent-Length: 146Keep-Alive: timeout=15, max=100 Connection: Keep-AliveContent-Type: text/html On the meanwhile if you have a clue what is happening I'll be thankful. Please turn off gzip on Apache side then try again :) Regards, -- Joshua Zhu Senior Software Engineer Server Platforms Team at Taobao _______________________________________________ nginx-devel mailing list nginx-devel at nginx.org http://mailman.nginx.org/mailman/listinfo/nginx-devel -------------- next part -------------- An HTML attachment was scrubbed... URL: From mk.fraggod at gmail.com Sat Sep 22 11:09:53 2012 From: mk.fraggod at gmail.com (Mike Kazantsev) Date: Sat, 22 Sep 2012 17:09:53 +0600 Subject: [PATCH] (re-post) Add "optional_no_ca" option to ssl_verify_client to enable app-only CA chain validation In-Reply-To: <20120918074355.GQ40452@mdounin.ru> References: <20120918074355.GQ40452@mdounin.ru> Message-ID: <20120922170953.3f8f50b1@sacrilege> Apologies for a bit late answer, managed to miss (or forgot to check option for) thread update notification from a forum, my bad. On Tue, 18 Sep 2012 11:43:55 +0400 Maxim Dounin wrote: > Hello! > > On Sat, Sep 15, 2012 at 07:52:30AM -0400, mk.fg wrote: > > > Re-post of patch from > > http://forum.nginx.org/read.php?2,228761,229586#msg-229586 ... > > You may want to join discussion here, about the similar patch > submitted: > > http://mailman.nginx.org/pipermail/nginx-devel/2012-August/002643.html > I hope you don't mind if I'll continue it in this thread, adding Cc to Eric. After my latest failure to get forum notification, and because I've managed to find a gmane archive for this list (don't think I've seen it there in the not-so-distant past), I'd prefer to use email, but alas, indicated thread was rotated-out from even latest-300 on gmane, so I can't easily find original message to reply to. > In particular, I would like someone to actually test if the > error_page 495 aproach works instead as suggested here: > > http://mailman.nginx.org/pipermail/nginx-devel/2012-August/002650.html > It doesn't seem to be of much use in current state, problems: - Requires "ssl_client_certificate" to be set to some valid certificate, which then shouldn't actually never be used in this case. - In case of "ssl_verify_client" set to "on" or "optional", setting "error_page 400 495 496 =200 /altpath;" doesn't seem to stop nginx from returning "HTTP/1.1 400 Bad Request" response with "400 The SSL certificate error" in html body. This code in ngx_http_request.c is probable cause of that: if (sscf->verify) { rc = SSL_get_verify_result(c->ssl->connection); if (rc != X509_V_OK) { ngx_log_error(NGX_LOG_INFO, c->log, 0, "client SSL certificate verify error: (%l:%s)", rc, X509_verify_cert_error_string(rc)); ngx_ssl_remove_cached_session(sscf->ssl.ctx, (SSL_get0_session(c->ssl->connection))); ngx_http_finalize_request(r, NGX_HTTPS_CERT_ERROR); return; } I'm still unsure if it's a general bug in ssl module or some error in my configuration above, because docs clearly state that error should be handled by error_page and it doesn't seem to be. - "ssl_verify_client off;" isn't much useful, because it doesn't return clent certificate and doesn't check it in any way. - "ssl_verify_client on;", still gives all-or-nothing check, though I see that it's what might indeed be desirable, as Eric indicated. - I think it's really non-obvious way to do it. > And a quick comment for your patch: I tend to think that > introduction of ngx_http_ssl_variable_get_client_verify() is > misleading. We shouldn't try to claim the certificate was > verified unless it actually was. It might be misleading, indeed, I see your point. Attached patch doesn't try to alter ssl_client_verify result. On a completely unrelated note - nginx fails to build from svn here due to 'Exception in thread "main" java.lang.NoClassDefFoundError: com/pault/StyleSheet' (so I just disabled building changelog for tests). URL for the revised patch: https://gist.github.com/3319062 Inline patch follows. diff --git a/src/event/ngx_event_openssl.h b/src/event/ngx_event_openssl.h index cd6d885..97da051 100644 --- a/src/event/ngx_event_openssl.h +++ b/src/event/ngx_event_openssl.h @@ -141,6 +141,14 @@ ngx_int_t ngx_ssl_get_client_verify(ngx_connection_t *c, ngx_pool_t *pool, ngx_str_t *s); +#define ngx_ssl_verify_error_is_optional(errnum) \ + ((errnum == X509_V_ERR_DEPTH_ZERO_SELF_SIGNED_CERT) \ + || (errnum == X509_V_ERR_SELF_SIGNED_CERT_IN_CHAIN) \ + || (errnum == X509_V_ERR_UNABLE_TO_GET_ISSUER_CERT_LOCALLY) \ + || (errnum == X509_V_ERR_CERT_UNTRUSTED) \ + || (errnum == X509_V_ERR_UNABLE_TO_VERIFY_LEAF_SIGNATURE)) + + ngx_int_t ngx_ssl_handshake(ngx_connection_t *c); ssize_t ngx_ssl_recv(ngx_connection_t *c, u_char *buf, size_t size); ssize_t ngx_ssl_write(ngx_connection_t *c, u_char *data, size_t size); diff --git a/src/http/modules/ngx_http_ssl_module.c b/src/http/modules/ngx_http_ssl_module.c index d759489..ab91670 100644 --- a/src/http/modules/ngx_http_ssl_module.c +++ b/src/http/modules/ngx_http_ssl_module.c @@ -48,6 +48,7 @@ static ngx_conf_enum_t ngx_http_ssl_verify[] = { { ngx_string("off"), 0 }, { ngx_string("on"), 1 }, { ngx_string("optional"), 2 }, + { ngx_string("optional_no_ca"), 3 }, { ngx_null_string, 0 } }; @@ -466,7 +467,7 @@ ngx_http_ssl_merge_srv_conf(ngx_conf_t *cf, void *parent, void *child) if (conf->verify) { - if (conf->client_certificate.len == 0) { + if (conf->verify != 3 && conf->client_certificate.len == 0) { ngx_log_error(NGX_LOG_EMERG, cf->log, 0, "no ssl_client_certificate for ssl_client_verify"); return NGX_CONF_ERROR; diff --git a/src/http/ngx_http_request.c b/src/http/ngx_http_request.c index cb970c5..96cec55 100644 --- a/src/http/ngx_http_request.c +++ b/src/http/ngx_http_request.c @@ -1642,7 +1642,9 @@ ngx_http_process_request(ngx_http_request_t *r) if (sscf->verify) { rc = SSL_get_verify_result(c->ssl->connection); - if (rc != X509_V_OK) { + if ((sscf->verify != 3 && rc != X509_V_OK) + || !(sscf->verify == 3 && ngx_ssl_verify_error_is_optional(rc))) + { ngx_log_error(NGX_LOG_INFO, c->log, 0, "client SSL certificate verify error: (%l:%s)", rc, X509_verify_cert_error_string(rc)); -- 1.7.12 -- Mike Kazantsev // fraggod.net -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 198 bytes Desc: not available URL: From eoconnor at coincident.com Sun Sep 23 01:38:28 2012 From: eoconnor at coincident.com (Eric O'Connor) Date: Sat, 22 Sep 2012 21:38:28 -0400 Subject: [PATCH] (re-post) Add "optional_no_ca" option to ssl_verify_client to enable app-only CA chain validation In-Reply-To: <20120922170953.3f8f50b1@sacrilege> References: <20120918074355.GQ40452@mdounin.ru> <20120922170953.3f8f50b1@sacrilege> Message-ID: Ah, I didn't know this was previously suggested on the forums. Also, thanks for looking into the error 495 issue. I looked at it briefly before deciding that we would rather maintain a 3 line patch than attempt to hijack an HTTP error code. Nice work. I had assumed that anyone using this sort of feature would use an existing SSL library to fully verify the passed certificate, but blocking totally erroneous/expired/revoked certificates can't hurt. Also, I'm happy that there are at least 3 people who think this is useful, and a few technologies that rely on it. ~ Eric O'Connor On Sat, Sep 22, 2012 at 7:09 AM, Mike Kazantsev wrote: > > Apologies for a bit late answer, managed to miss (or forgot to check > option for) thread update notification from a forum, my bad. > > > On Tue, 18 Sep 2012 11:43:55 +0400 > Maxim Dounin wrote: > >> Hello! >> >> On Sat, Sep 15, 2012 at 07:52:30AM -0400, mk.fg wrote: >> >> > Re-post of patch from >> > http://forum.nginx.org/read.php?2,228761,229586#msg-229586 > ... >> >> You may want to join discussion here, about the similar patch >> submitted: >> >> http://mailman.nginx.org/pipermail/nginx-devel/2012-August/002643.html >> > > I hope you don't mind if I'll continue it in this thread, adding Cc to > Eric. > > After my latest failure to get forum notification, and because I've > managed to find a gmane archive for this list (don't think I've seen it > there in the not-so-distant past), I'd prefer to use email, but alas, > indicated thread was rotated-out from even latest-300 on gmane, so I > can't easily find original message to reply to. > > >> In particular, I would like someone to actually test if the >> error_page 495 aproach works instead as suggested here: >> >> http://mailman.nginx.org/pipermail/nginx-devel/2012-August/002650.html >> > > It doesn't seem to be of much use in current state, problems: > > - Requires "ssl_client_certificate" to be set to some valid > certificate, which then shouldn't actually never be used in this > case. > > - In case of "ssl_verify_client" set to "on" or "optional", setting > "error_page 400 495 496 =200 /altpath;" doesn't seem to stop nginx > from returning "HTTP/1.1 400 Bad Request" response with "400 The SSL > certificate error" in html body. > > This code in ngx_http_request.c is probable cause of that: > > if (sscf->verify) { > rc = SSL_get_verify_result(c->ssl->connection); > > if (rc != X509_V_OK) { > ngx_log_error(NGX_LOG_INFO, c->log, 0, > "client SSL certificate verify error: (%l:%s)", > rc, X509_verify_cert_error_string(rc)); > > ngx_ssl_remove_cached_session(sscf->ssl.ctx, > (SSL_get0_session(c->ssl->connection))); > > ngx_http_finalize_request(r, NGX_HTTPS_CERT_ERROR); > return; > } > > I'm still unsure if it's a general bug in ssl module or some error > in my configuration above, because docs clearly state that error > should be handled by error_page and it doesn't seem to be. > > - "ssl_verify_client off;" isn't much useful, because it doesn't return > clent certificate and doesn't check it in any way. > > - "ssl_verify_client on;", still gives all-or-nothing check, though > I see that it's what might indeed be desirable, as Eric indicated. > > - I think it's really non-obvious way to do it. > > >> And a quick comment for your patch: I tend to think that >> introduction of ngx_http_ssl_variable_get_client_verify() is >> misleading. We shouldn't try to claim the certificate was >> verified unless it actually was. > > It might be misleading, indeed, I see your point. > Attached patch doesn't try to alter ssl_client_verify result. > > On a completely unrelated note - nginx fails to build from svn here due > to 'Exception in thread "main" java.lang.NoClassDefFoundError: > com/pault/StyleSheet' (so I just disabled building changelog for tests). > > URL for the revised patch: https://gist.github.com/3319062 > > > Inline patch follows. > > diff --git a/src/event/ngx_event_openssl.h b/src/event/ngx_event_openssl.h > index cd6d885..97da051 100644 > --- a/src/event/ngx_event_openssl.h > +++ b/src/event/ngx_event_openssl.h > @@ -141,6 +141,14 @@ ngx_int_t ngx_ssl_get_client_verify(ngx_connection_t *c, ngx_pool_t *pool, > ngx_str_t *s); > > > +#define ngx_ssl_verify_error_is_optional(errnum) \ > + ((errnum == X509_V_ERR_DEPTH_ZERO_SELF_SIGNED_CERT) \ > + || (errnum == X509_V_ERR_SELF_SIGNED_CERT_IN_CHAIN) \ > + || (errnum == X509_V_ERR_UNABLE_TO_GET_ISSUER_CERT_LOCALLY) \ > + || (errnum == X509_V_ERR_CERT_UNTRUSTED) \ > + || (errnum == X509_V_ERR_UNABLE_TO_VERIFY_LEAF_SIGNATURE)) > + > + > ngx_int_t ngx_ssl_handshake(ngx_connection_t *c); > ssize_t ngx_ssl_recv(ngx_connection_t *c, u_char *buf, size_t size); > ssize_t ngx_ssl_write(ngx_connection_t *c, u_char *data, size_t size); > diff --git a/src/http/modules/ngx_http_ssl_module.c b/src/http/modules/ngx_http_ssl_module.c > index d759489..ab91670 100644 > --- a/src/http/modules/ngx_http_ssl_module.c > +++ b/src/http/modules/ngx_http_ssl_module.c > @@ -48,6 +48,7 @@ static ngx_conf_enum_t ngx_http_ssl_verify[] = { > { ngx_string("off"), 0 }, > { ngx_string("on"), 1 }, > { ngx_string("optional"), 2 }, > + { ngx_string("optional_no_ca"), 3 }, > { ngx_null_string, 0 } > }; > > @@ -466,7 +467,7 @@ ngx_http_ssl_merge_srv_conf(ngx_conf_t *cf, void *parent, void *child) > > if (conf->verify) { > > - if (conf->client_certificate.len == 0) { > + if (conf->verify != 3 && conf->client_certificate.len == 0) { > ngx_log_error(NGX_LOG_EMERG, cf->log, 0, > "no ssl_client_certificate for ssl_client_verify"); > return NGX_CONF_ERROR; > diff --git a/src/http/ngx_http_request.c b/src/http/ngx_http_request.c > index cb970c5..96cec55 100644 > --- a/src/http/ngx_http_request.c > +++ b/src/http/ngx_http_request.c > @@ -1642,7 +1642,9 @@ ngx_http_process_request(ngx_http_request_t *r) > if (sscf->verify) { > rc = SSL_get_verify_result(c->ssl->connection); > > - if (rc != X509_V_OK) { > + if ((sscf->verify != 3 && rc != X509_V_OK) > + || !(sscf->verify == 3 && ngx_ssl_verify_error_is_optional(rc))) > + { > ngx_log_error(NGX_LOG_INFO, c->log, 0, > "client SSL certificate verify error: (%l:%s)", > rc, X509_verify_cert_error_string(rc)); > -- > 1.7.12 > > > -- > Mike Kazantsev // fraggod.net From leon at kingdest.com Sun Sep 23 16:24:34 2012 From: leon at kingdest.com (Leon) Date: Mon, 24 Sep 2012 00:24:34 +0800 Subject: How to prohibit a client certificate In-Reply-To: <505DEFCE.50909@kingdest.com> References: <505DEFCE.50909@kingdest.com> Message-ID: <505F37C2.20204@kingdest.com> Hi, I have a normal running nginx https server with mutual authentication, if i want to prohibit access a client certificate, how should I do? In addition, how to set up the mutual authentication of client access different pages? Regards -- Leon +86.13168330000 www.kingdest.com From yaoweibin at gmail.com Mon Sep 24 10:29:49 2012 From: yaoweibin at gmail.com (=?GB2312?B?0qbOsLHz?=) Date: Mon, 24 Sep 2012 18:29:49 +0800 Subject: Update the Chinese documentation with the nginx.org website. Message-ID: Hi folks, We have translated the nginx documentation into Chinese to the latest version (nginx.org repository, September 14th). Modules whose Chinese description are also available: ngx_http_access_module.html ngx_http_addition_module.html ngx_http_auth_basic_module.html ngx_http_autoindex_module.html ngx_http_browser_module.htm ngx_http_gzip_static_module.html ngx_http_index_module.html ngx_http_limit_conn_module.html Now you can preview the translation at http://tengine.taobao.org/nginx_docs/cn/. We'll really appreciate it if you report any errors or incorrectness to us. BTW, you are welcome to join the Nginx Chinese Documentation Translation Project (https://github.com/taobao/nginx-docs-cn). The process is quite easy: fork the repository, do some translation then pull a request :) Other parts of the translation will be posted when they are ready. -- Weibin Yao Developer @ Server Platform Team of Taobao -------------- next part -------------- A non-text attachment was scrubbed... Name: nginx_org_cn_20120914.diff Type: application/octet-stream Size: 534305 bytes Desc: not available URL: From maxim at nginx.com Mon Sep 24 11:10:30 2012 From: maxim at nginx.com (Maxim Konovalov) Date: Mon, 24 Sep 2012 15:10:30 +0400 Subject: Update the Chinese documentation with the nginx.org website. In-Reply-To: References: Message-ID: <50603FA6.6090500@nginx.com> Hello, On 9/24/12 2:29 PM, ??? wrote: > Hi folks, > > We have translated the nginx documentation into Chinese to the latest > version (nginx.org repository, September 14th). > > Modules whose Chinese description are also available: > ngx_http_access_module.html > ngx_http_addition_module.html > ngx_http_auth_basic_module.html > ngx_http_autoindex_module.html > ngx_http_browser_module.htm > ngx_http_gzip_static_module.html > ngx_http_index_module.html > ngx_http_limit_conn_module.html > Thanks for the patch! I reviewed it briefly and here are some observations: -- some of the files in the diff are original English documents, not their translated counterparts. To name some of them: xml/cn/links.xml, xml/cn/download.xml, xml/cn/docs/control.xml. It's not clear for my why you imported them under xml/cn hierarchy; -- we don't use tabs for indentation (xml/cn/docs/http/ngx_http_addition_module.xml, just befor "translator" tag); -- you added several copyright lines, namely Copyright (C) Cfsego Copyright (C) moonbingbing at gmail.com Copyright (c) litianqing How can we get a confirmation for these people/entities that they give us a permission to use their work under BSD lic terms (http://nginx.org/LICENSE)? Apart from that it would be nice to have a consistent style here, i.e. upper case "C" in (C) part. -- Maxim Konovalov +7 (910) 4293178 http://nginx.com/support.html From airforce.e at gmail.com Mon Sep 24 11:37:18 2012 From: airforce.e at gmail.com (=?UTF-8?B?5q+b5Yqf57yY?=) Date: Mon, 24 Sep 2012 19:37:18 +0800 Subject: Update the Chinese documentation with the nginx.org website. Message-ID: Cosmo Mao Maxim Konovalov ??? >Hello, > >On 9/24/12 2:29 PM, ??? wrote: >> Hi folks, >> >> We have translated the nginx documentation into Chinese to the latest >> version (nginx.org repository, September 14th). >> >> Modules whose Chinese description are also available: >> ngx_http_access_module.html >> ngx_http_addition_module.html >> ngx_http_auth_basic_module.html >> ngx_http_autoindex_module.html >> ngx_http_browser_module.htm >> ngx_http_gzip_static_module.html >> ngx_http_index_module.html >> ngx_http_limit_conn_module.html >> >Thanks for the patch! > >I reviewed it briefly and here are some observations: > >-- some of the files in the diff are original English documents, not >their translated counterparts. To name some of them: >xml/cn/links.xml, xml/cn/download.xml, xml/cn/docs/control.xml. >It's not clear for my why you imported them under xml/cn hierarchy; > >-- we don't use tabs for indentation >(xml/cn/docs/http/ngx_http_addition_module.xml, just befor >"translator" tag); > >-- you added several copyright lines, namely > >Copyright (C) Cfsego >Copyright (C) moonbingbing at gmail.com >Copyright (c) litianqing > >How can we get a confirmation for these people/entities that they >give us a permission to use their work under BSD lic terms >(http://nginx.org/LICENSE)? Apart from that it would be nice to >have a consistent style here, i.e. upper case "C" in (C) part. > >-- >Maxim Konovalov >+7 (910) 4293178 >http://nginx.com/support.html > >_______________________________________________ >nginx-devel mailing list >nginx-devel at nginx.org >http://mailman.nginx.org/mailman/listinfo/nginx-devel From yaoweibin at gmail.com Mon Sep 24 14:38:33 2012 From: yaoweibin at gmail.com (=?GB2312?B?0qbOsLHz?=) Date: Mon, 24 Sep 2012 22:38:33 +0800 Subject: Update the Chinese documentation with the nginx.org website. In-Reply-To: <50603FA6.6090500@nginx.com> References: <50603FA6.6090500@nginx.com> Message-ID: Hi, Maxim Thanks for your quick reply. We are translating the English documentation to be Chinese. First we just imported all the English modules. It's easy for us to port the makefile and preview in the browser. Then we invited the volunteers the translate every module. This work doesn't finish yet. We are planning to translate all the modules this year. If you don't like this patch, we can give a more purely version. Other modifications will be in the next patch. I'll ask the translators send you a confirmation email for their agreement with the nginx's BSD license, is that OK? 2012/9/24 Maxim Konovalov : > Hello, > > On 9/24/12 2:29 PM, ??? wrote: >> Hi folks, >> >> We have translated the nginx documentation into Chinese to the latest >> version (nginx.org repository, September 14th). >> >> Modules whose Chinese description are also available: >> ngx_http_access_module.html >> ngx_http_addition_module.html >> ngx_http_auth_basic_module.html >> ngx_http_autoindex_module.html >> ngx_http_browser_module.htm >> ngx_http_gzip_static_module.html >> ngx_http_index_module.html >> ngx_http_limit_conn_module.html >> > Thanks for the patch! > > I reviewed it briefly and here are some observations: > > -- some of the files in the diff are original English documents, not > their translated counterparts. To name some of them: > xml/cn/links.xml, xml/cn/download.xml, xml/cn/docs/control.xml. > It's not clear for my why you imported them under xml/cn hierarchy; > > -- we don't use tabs for indentation > (xml/cn/docs/http/ngx_http_addition_module.xml, just befor > "translator" tag); > > -- you added several copyright lines, namely > > Copyright (C) Cfsego > Copyright (C) moonbingbing at gmail.com > Copyright (c) litianqing > > How can we get a confirmation for these people/entities that they > give us a permission to use their work under BSD lic terms > (http://nginx.org/LICENSE)? Apart from that it would be nice to > have a consistent style here, i.e. upper case "C" in (C) part. > > -- > Maxim Konovalov > +7 (910) 4293178 > http://nginx.com/support.html > > _______________________________________________ > nginx-devel mailing list > nginx-devel at nginx.org > http://mailman.nginx.org/mailman/listinfo/nginx-devel -- Weibin Yao Developer @ Server Platform Team of Taobao From maxim at nginx.com Mon Sep 24 14:50:26 2012 From: maxim at nginx.com (Maxim Konovalov) Date: Mon, 24 Sep 2012 18:50:26 +0400 Subject: Update the Chinese documentation with the nginx.org website. In-Reply-To: References: <50603FA6.6090500@nginx.com> Message-ID: <50607332.4020106@nginx.com> Hi Weibin, On 9/24/12 6:38 PM, ??? wrote: > Hi, Maxim > > Thanks for your quick reply. > > We are translating the English documentation to be Chinese. First we > just imported all the English modules. It's easy for us to port the > makefile and preview in the browser. Then we invited the volunteers > the translate every module. This work doesn't finish yet. We are > planning to translate all the modules this year. > > If you don't like this patch, we can give a more purely version. > Yeah, it'd be nice to get a patch we'll be able to commit to the nginx.org tree. > Other modifications will be in the next patch. > > I'll ask the translators send you a confirmation email for their > agreement with the nginx's BSD license, is that OK? > That's great. Thanks for your work again. -- Maxim Konovalov +7 (910) 4293178 http://nginx.com/support.html From mdounin at mdounin.ru Mon Sep 24 18:24:07 2012 From: mdounin at mdounin.ru (mdounin at mdounin.ru) Date: Mon, 24 Sep 2012 18:24:07 +0000 Subject: [nginx] svn commit: r4847 - in branches/stable-1.2/src: core http/modules/perl Message-ID: <20120924182407.CB45C3F9C44@mail.nginx.com> Author: mdounin Date: 2012-09-24 18:24:07 +0000 (Mon, 24 Sep 2012) New Revision: 4847 URL: http://trac.nginx.org/nginx/changeset/4847/nginx Log: Version bump. Modified: branches/stable-1.2/src/core/nginx.h branches/stable-1.2/src/http/modules/perl/nginx.pm Modified: branches/stable-1.2/src/core/nginx.h =================================================================== --- branches/stable-1.2/src/core/nginx.h 2012-09-12 10:41:56 UTC (rev 4846) +++ branches/stable-1.2/src/core/nginx.h 2012-09-24 18:24:07 UTC (rev 4847) @@ -9,8 +9,8 @@ #define _NGINX_H_INCLUDED_ -#define nginx_version 1002003 -#define NGINX_VERSION "1.2.3" +#define nginx_version 1002004 +#define NGINX_VERSION "1.2.4" #define NGINX_VER "nginx/" NGINX_VERSION #define NGINX_VAR "NGINX" Modified: branches/stable-1.2/src/http/modules/perl/nginx.pm =================================================================== --- branches/stable-1.2/src/http/modules/perl/nginx.pm 2012-09-12 10:41:56 UTC (rev 4846) +++ branches/stable-1.2/src/http/modules/perl/nginx.pm 2012-09-24 18:24:07 UTC (rev 4847) @@ -50,7 +50,7 @@ HTTP_INSUFFICIENT_STORAGE ); -our $VERSION = '1.2.3'; +our $VERSION = '1.2.4'; require XSLoader; XSLoader::load('nginx', $VERSION); From mdounin at mdounin.ru Mon Sep 24 18:30:26 2012 From: mdounin at mdounin.ru (mdounin at mdounin.ru) Date: Mon, 24 Sep 2012 18:30:26 +0000 Subject: [nginx] svn commit: r4848 - in branches/stable-1.2: . docs docs/dtd docs/xml/nginx docs/xsls misc Message-ID: <20120924183026.BCCB13F9C1B@mail.nginx.com> Author: mdounin Date: 2012-09-24 18:30:26 +0000 (Mon, 24 Sep 2012) New Revision: 4848 URL: http://trac.nginx.org/nginx/changeset/4848/nginx Log: Merge of r4777, r4780, r4831, r4832: release process changes. *) Simplified makefile that builds CHANGES. *) Removed the need in Perl to generate ZIP archive of nginx/Windows. *) Converted DOS-style newlines in dump.xslt. *) Pass changes.xml thru xmllint when generating CHANGES and CHANGES.ru. Modified: branches/stable-1.2/ branches/stable-1.2/docs/GNUmakefile branches/stable-1.2/docs/dtd/changes.dtd branches/stable-1.2/docs/xml/nginx/changes.xml branches/stable-1.2/docs/xsls/dump.xsls branches/stable-1.2/misc/GNUmakefile Index: branches/stable-1.2 =================================================================== --- branches/stable-1.2 2012-09-24 18:24:07 UTC (rev 4847) +++ branches/stable-1.2 2012-09-24 18:30:26 UTC (rev 4848) Property changes on: branches/stable-1.2 ___________________________________________________________________ Modified: svn:mergeinfo ## -1 +1 ## -/trunk:4611-4632,4636-4657,4671-4672,4674-4676,4682,4684-4699,4704-4706,4713,4736-4741,4754,4756-4771,4775 +/trunk:4611-4632,4636-4657,4671-4672,4674-4676,4682,4684-4699,4704-4706,4713,4736-4741,4754,4756-4771,4775,4777,4780,4831-4832 \ No newline at end of property Modified: branches/stable-1.2/docs/GNUmakefile =================================================================== --- branches/stable-1.2/docs/GNUmakefile 2012-09-24 18:24:07 UTC (rev 4847) +++ branches/stable-1.2/docs/GNUmakefile 2012-09-24 18:30:26 UTC (rev 4848) @@ -5,53 +5,43 @@ TEMP= tmp CP= $(HOME)/java -define XSLScript - java -cp $(CP)/xsls/saxon.jar:$(CP)/xsls/xsls.jar \ - com.pault.StyleSheet \ - -x com.pault.XX -y com.pault.XX \ - $(1) docs/xsls/dump.xsls \ - | sed 's/ *$$//;/^ *$$/N;/\n *$$/D' > $(2) - if [ ! -s $(2) ]; then rm $(2); fi; test -s $(2) -endef - -define XSLT - xmllint --noout --valid $2 - xsltproc -o $3 \ - $(shell echo $4 \\ - | sed -e "s/\([^= ]*\)=\([^= ]*\)/--param \1 \"'\2'\"/g") \ - $1 $2 -endef - - all: changes changes: $(TEMP)/$(NGINX)/CHANGES.ru \ $(TEMP)/$(NGINX)/CHANGES -$(TEMP)/$(NGINX)/CHANGES.ru: docs/xml/nginx/changes.xml \ +$(TEMP)/$(NGINX)/CHANGES.ru: docs/dtd/changes.dtd \ + docs/xml/nginx/changes.xml \ docs/xml/change_log_conf.xml \ docs/xslt/changes.xslt - test -d $(TEMP)/$(NGINX) || mkdir -p $(TEMP)/$(NGINX) + mkdir -p $(TEMP)/$(NGINX) + xmllint --noout --valid docs/xml/nginx/changes.xml xsltproc --stringparam lang ru \ - -o $(TEMP)/$(NGINX)/CHANGES.ru \ - docs/xslt/changes.xslt docs/xml/nginx/changes.xml + -o $@ docs/xslt/changes.xslt docs/xml/nginx/changes.xml -$(TEMP)/$(NGINX)/CHANGES: docs/xml/nginx/changes.xml \ +$(TEMP)/$(NGINX)/CHANGES: docs/dtd/changes.dtd \ + docs/xml/nginx/changes.xml \ docs/xml/change_log_conf.xml \ docs/xslt/changes.xslt - test -d $(TEMP)/$(NGINX) || mkdir -p $(TEMP)/$(NGINX) + mkdir -p $(TEMP)/$(NGINX) + xmllint --noout --valid docs/xml/nginx/changes.xml xsltproc --stringparam lang en \ - -o $(TEMP)/$(NGINX)/CHANGES \ - docs/xslt/changes.xslt docs/xml/nginx/changes.xml + -o $@ docs/xslt/changes.xslt docs/xml/nginx/changes.xml -docs/xslt/changes.xslt: docs/xsls/changes.xsls +docs/xslt/changes.xslt: docs/xsls/changes.xsls docs/xsls/dump.xsls - $(call XSLScript, docs/xsls/changes.xsls, $@) + java -cp $(CP)/xsls/saxon.jar:$(CP)/xsls/xsls.jar \ + com.pault.StyleSheet \ + -x com.pault.XX -y com.pault.XX \ + $< docs/xsls/dump.xsls \ + | sed 's/ *$$//;/^ *$$/N;/\n *$$/D' > $@ + + if [ ! -s $@ ]; then rm $@; fi; test -s $@ Modified: branches/stable-1.2/docs/dtd/changes.dtd =================================================================== --- branches/stable-1.2/docs/dtd/changes.dtd 2012-09-24 18:24:07 UTC (rev 4847) +++ branches/stable-1.2/docs/dtd/changes.dtd 2012-09-24 18:30:26 UTC (rev 4848) @@ -12,10 +12,10 @@ > - + - + Modified: branches/stable-1.2/docs/xml/nginx/changes.xml =================================================================== --- branches/stable-1.2/docs/xml/nginx/changes.xml 2012-09-24 18:24:07 UTC (rev 4847) +++ branches/stable-1.2/docs/xml/nginx/changes.xml 2012-09-24 18:30:26 UTC (rev 4848) @@ -2,13 +2,9 @@ - + -????????? ? nginx -nginx changelog - Modified: branches/stable-1.2/docs/xsls/dump.xsls =================================================================== --- branches/stable-1.2/docs/xsls/dump.xsls 2012-09-24 18:24:07 UTC (rev 4847) +++ branches/stable-1.2/docs/xsls/dump.xsls 2012-09-24 18:30:26 UTC (rev 4848) @@ -1,29 +1,29 @@ -X:stylesheet { - -X:output method="xml" -X:param indent-increment="' '"; - -X:template noname(indent="' '") = "*" { - !{$indent} - - X:if "name()='xsl:template'" { - !{$indent} - } - - X:copy { - X:copy-of "@*" - !!( indent = "concat($indent, $indent-increment)" ); - X:if "./* " { !{$indent} } - } -} - - -X:template = "comment()|processing-instruction()" { - X:copy; -} - - - -} +X:stylesheet { + +X:output method="xml" +X:param indent-increment="' '"; + +X:template noname(indent="' '") = "*" { + !{$indent} + + X:if "name()='xsl:template'" { + !{$indent} + } + + X:copy { + X:copy-of "@*" + !!( indent = "concat($indent, $indent-increment)" ); + X:if "./* " { !{$indent} } + } +} + + +X:template = "comment()|processing-instruction()" { + X:copy; +} + + + +} Modified: branches/stable-1.2/misc/GNUmakefile =================================================================== --- branches/stable-1.2/misc/GNUmakefile 2012-09-24 18:24:07 UTC (rev 4847) +++ branches/stable-1.2/misc/GNUmakefile 2012-09-24 18:30:26 UTC (rev 4848) @@ -93,7 +93,7 @@ mkdir -p $(TEMP)/$(NGINX)/temp svn export -rHEAD conf $(TEMP)/$(NGINX)/conf/ - perl -pi -e 's/$$/\r/' $(TEMP)/$(NGINX)/conf/* + sed -i '' -e "s/$$/`printf '\r'`/" $(TEMP)/$(NGINX)/conf/* svn export -rHEAD contrib $(TEMP)/$(NGINX)/contrib/ svn export -rHEAD docs/html $(TEMP)/$(NGINX)/html/ @@ -112,7 +112,7 @@ cp -p $(OBJS)/lib/$(PCRE)/LICENCE \ $(TEMP)/$(NGINX)/docs/PCRE.LICENCE - perl -ne 'print if /^ \(C\) 1995-20/ .. /^ jloup\@gzip.org/' \ + sed -ne '/^ (C) 1995-20/,/^ jloup at gzip\.org/p' \ $(OBJS)/lib/$(ZLIB)/README \ > $(TEMP)/$(NGINX)/docs/zlib.LICENSE From mdounin at mdounin.ru Mon Sep 24 18:34:04 2012 From: mdounin at mdounin.ru (mdounin at mdounin.ru) Date: Mon, 24 Sep 2012 18:34:04 +0000 Subject: [nginx] svn commit: r4849 - in branches/stable-1.2: . src/core src/http Message-ID: <20120924183404.CAB1C3F9C11@mail.nginx.com> Author: mdounin Date: 2012-09-24 18:34:04 +0000 (Mon, 24 Sep 2012) New Revision: 4849 URL: http://trac.nginx.org/nginx/changeset/4849/nginx Log: Merge of r4778, r4782, r4783, r4824, r4830, r4834: minor fixes. *) Reorder checks in ngx_shared_memory_add() for more consistent error messages. *) Added "const" to ngx_memcpy() with NGX_MEMCPY_LIMIT defined. This fixes warning produced during compilation of the ngx_http_geoip_module due to const qualifier being discarded. *) Fixed possible use of old cached times if runtime went backwards. If ngx_time_sigsafe_update() updated only ngx_cached_err_log_time, and then clock was adjusted backwards, the cached_time[slot].sec might accidentally match current seconds on next ngx_time_update() call, resulting in various cached times not being updated. Fix is to clear the cached_time[slot].sec to explicitly mark cached times are stale and need updating. *) Radix tree preallocation fix. The preallocation size was calculated incorrectly and was always 8 due to sizeof(ngx_radix_tree_t) accidentally used instead of sizeof(ngx_radix_node_t). *) Fixed overflow if ngx_slab_alloc() is called with very big "size" argument. *) Write filter: replaced unneeded loop with one to free chains. Noted by Gabor Lekeny. Modified: branches/stable-1.2/ branches/stable-1.2/src/core/ngx_cycle.c branches/stable-1.2/src/core/ngx_radix_tree.c branches/stable-1.2/src/core/ngx_slab.c branches/stable-1.2/src/core/ngx_string.c branches/stable-1.2/src/core/ngx_string.h branches/stable-1.2/src/core/ngx_times.c branches/stable-1.2/src/http/ngx_http_write_filter_module.c Index: branches/stable-1.2 =================================================================== --- branches/stable-1.2 2012-09-24 18:30:26 UTC (rev 4848) +++ branches/stable-1.2 2012-09-24 18:34:04 UTC (rev 4849) Property changes on: branches/stable-1.2 ___________________________________________________________________ Modified: svn:mergeinfo ## -1 +1 ## -/trunk:4611-4632,4636-4657,4671-4672,4674-4676,4682,4684-4699,4704-4706,4713,4736-4741,4754,4756-4771,4775,4777,4780,4831-4832 +/trunk:4611-4632,4636-4657,4671-4672,4674-4676,4682,4684-4699,4704-4706,4713,4736-4741,4754,4756-4771,4775,4777-4778,4780,4782-4783,4824,4830-4832,4834 \ No newline at end of property Modified: branches/stable-1.2/src/core/ngx_cycle.c =================================================================== --- branches/stable-1.2/src/core/ngx_cycle.c 2012-09-24 18:30:26 UTC (rev 4848) +++ branches/stable-1.2/src/core/ngx_cycle.c 2012-09-24 18:34:04 UTC (rev 4849) @@ -1285,6 +1285,14 @@ continue; } + if (tag != shm_zone[i].tag) { + ngx_conf_log_error(NGX_LOG_EMERG, cf, 0, + "the shared memory zone \"%V\" is " + "already declared for a different use", + &shm_zone[i].shm.name); + return NULL; + } + if (size && size != shm_zone[i].shm.size) { ngx_conf_log_error(NGX_LOG_EMERG, cf, 0, "the size %uz of shared memory zone \"%V\" " @@ -1293,14 +1301,6 @@ return NULL; } - if (tag != shm_zone[i].tag) { - ngx_conf_log_error(NGX_LOG_EMERG, cf, 0, - "the shared memory zone \"%V\" is " - "already declared for a different use", - &shm_zone[i].shm.name); - return NULL; - } - return &shm_zone[i]; } Modified: branches/stable-1.2/src/core/ngx_radix_tree.c =================================================================== --- branches/stable-1.2/src/core/ngx_radix_tree.c 2012-09-24 18:30:26 UTC (rev 4848) +++ branches/stable-1.2/src/core/ngx_radix_tree.c 2012-09-24 18:34:04 UTC (rev 4849) @@ -60,7 +60,7 @@ */ if (preallocate == -1) { - switch (ngx_pagesize / sizeof(ngx_radix_tree_t)) { + switch (ngx_pagesize / sizeof(ngx_radix_node_t)) { /* amd64 */ case 128: Modified: branches/stable-1.2/src/core/ngx_slab.c =================================================================== --- branches/stable-1.2/src/core/ngx_slab.c 2012-09-24 18:30:26 UTC (rev 4848) +++ branches/stable-1.2/src/core/ngx_slab.c 2012-09-24 18:34:04 UTC (rev 4849) @@ -162,8 +162,8 @@ ngx_log_debug1(NGX_LOG_DEBUG_ALLOC, ngx_cycle->log, 0, "slab alloc: %uz", size); - page = ngx_slab_alloc_pages(pool, (size + ngx_pagesize - 1) - >> ngx_pagesize_shift); + page = ngx_slab_alloc_pages(pool, (size >> ngx_pagesize_shift) + + ((size % ngx_pagesize) ? 1 : 0)); if (page) { p = (page - pool->pages) << ngx_pagesize_shift; p += (uintptr_t) pool->start; Modified: branches/stable-1.2/src/core/ngx_string.c =================================================================== --- branches/stable-1.2/src/core/ngx_string.c 2012-09-24 18:30:26 UTC (rev 4848) +++ branches/stable-1.2/src/core/ngx_string.c 2012-09-24 18:34:04 UTC (rev 4849) @@ -1827,7 +1827,7 @@ #if (NGX_MEMCPY_LIMIT) void * -ngx_memcpy(void *dst, void *src, size_t n) +ngx_memcpy(void *dst, const void *src, size_t n) { if (n > NGX_MEMCPY_LIMIT) { ngx_log_error(NGX_LOG_ALERT, ngx_cycle->log, 0, "memcpy %uz bytes", n); Modified: branches/stable-1.2/src/core/ngx_string.h =================================================================== --- branches/stable-1.2/src/core/ngx_string.h 2012-09-24 18:30:26 UTC (rev 4848) +++ branches/stable-1.2/src/core/ngx_string.h 2012-09-24 18:34:04 UTC (rev 4849) @@ -89,7 +89,7 @@ #if (NGX_MEMCPY_LIMIT) -void *ngx_memcpy(void *dst, void *src, size_t n); +void *ngx_memcpy(void *dst, const void *src, size_t n); #define ngx_cpymem(dst, src, n) (((u_char *) ngx_memcpy(dst, src, n)) + (n)) #else Modified: branches/stable-1.2/src/core/ngx_times.c =================================================================== --- branches/stable-1.2/src/core/ngx_times.c 2012-09-24 18:30:26 UTC (rev 4848) +++ branches/stable-1.2/src/core/ngx_times.c 2012-09-24 18:34:04 UTC (rev 4849) @@ -211,6 +211,10 @@ slot++; } + tp = &cached_time[slot]; + + tp->sec = 0; + ngx_gmtime(sec + cached_gmtoff * 60, &tm); p = &cached_err_log_time[slot][0]; Modified: branches/stable-1.2/src/http/ngx_http_write_filter_module.c =================================================================== --- branches/stable-1.2/src/http/ngx_http_write_filter_module.c 2012-09-24 18:30:26 UTC (rev 4848) +++ branches/stable-1.2/src/http/ngx_http_write_filter_module.c 2012-09-24 18:34:04 UTC (rev 4849) @@ -185,23 +185,19 @@ } if (size == 0 && !(c->buffered & NGX_LOWLEVEL_BUFFERED)) { - if (last) { + if (last || flush) { + for (cl = r->out; cl; /* void */) { + ln = cl; + cl = cl->next; + ngx_free_chain(r->pool, ln); + } + r->out = NULL; c->buffered &= ~NGX_HTTP_WRITE_BUFFERED; return NGX_OK; } - if (flush) { - do { - r->out = r->out->next; - } while (r->out); - - c->buffered &= ~NGX_HTTP_WRITE_BUFFERED; - - return NGX_OK; - } - ngx_log_error(NGX_LOG_ALERT, c->log, 0, "the http output chain is empty"); From mdounin at mdounin.ru Mon Sep 24 18:37:14 2012 From: mdounin at mdounin.ru (mdounin at mdounin.ru) Date: Mon, 24 Sep 2012 18:37:14 +0000 Subject: [nginx] svn commit: r4850 - in branches/stable-1.2: . auto/lib/zlib misc src/os/win32 Message-ID: <20120924183715.023853F9C1B@mail.nginx.com> Author: mdounin Date: 2012-09-24 18:37:14 +0000 (Mon, 24 Sep 2012) New Revision: 4850 URL: http://trac.nginx.org/nginx/changeset/4850/nginx Log: Merge of r4779, r4840, r4843, r4844: win32 changes. *) Fixed build with Visual Studio 2005 Express. It is available via winetricks which makes it still usable, and has an old crtdefs.h which uses _CRT_SECURE_NO_DEPRECATE instead of _CRT_SECURE_NO_WARNINGS to suppress warnings. Reported by HAYASHI Kentaro, http://mailman.nginx.org/pipermail/nginx-devel/2012-August/002542.html *) Configure: provide inflate() when building zlib on win32. *) Configure: fixed make macros to use parentheses instead of braces. Parentheses are more portable, in particular they are understood by nmake while braces aren't. *) Helper target "win32" to run configure for win32 builds. *) Updated zlib used for win32 builds. Modified: branches/stable-1.2/ branches/stable-1.2/auto/lib/zlib/makefile.bcc branches/stable-1.2/auto/lib/zlib/makefile.msvc branches/stable-1.2/auto/lib/zlib/makefile.owc branches/stable-1.2/misc/GNUmakefile branches/stable-1.2/src/os/win32/ngx_win32_config.h Index: branches/stable-1.2 =================================================================== --- branches/stable-1.2 2012-09-24 18:34:04 UTC (rev 4849) +++ branches/stable-1.2 2012-09-24 18:37:14 UTC (rev 4850) Property changes on: branches/stable-1.2 ___________________________________________________________________ Modified: svn:mergeinfo ## -1 +1 ## -/trunk:4611-4632,4636-4657,4671-4672,4674-4676,4682,4684-4699,4704-4706,4713,4736-4741,4754,4756-4771,4775,4777-4778,4780,4782-4783,4824,4830-4832,4834 +/trunk:4611-4632,4636-4657,4671-4672,4674-4676,4682,4684-4699,4704-4706,4713,4736-4741,4754,4756-4771,4775,4777-4780,4782-4783,4824,4830-4832,4834,4840,4843-4844 \ No newline at end of property Modified: branches/stable-1.2/auto/lib/zlib/makefile.bcc =================================================================== --- branches/stable-1.2/auto/lib/zlib/makefile.bcc 2012-09-24 18:34:04 UTC (rev 4849) +++ branches/stable-1.2/auto/lib/zlib/makefile.bcc 2012-09-24 18:37:14 UTC (rev 4850) @@ -8,8 +8,10 @@ zlib.lib: cd $(ZLIB) - bcc32 -c $(CFLAGS) adler32.c crc32.c deflate.c trees.c zutil.c \ - compress.c + bcc32 -c $(CFLAGS) adler32.c crc32.c deflate.c \ + trees.c zutil.c compress.c \ + inflate.c inffast.c inftrees.c tlib zlib.lib +adler32.obj +crc32.obj +deflate.obj \ - +trees.obj +zutil.obj +compress.obj + +trees.obj +zutil.obj +compress.obj \ + +inflate.obj +inffast.obj +inftrees.obj Modified: branches/stable-1.2/auto/lib/zlib/makefile.msvc =================================================================== --- branches/stable-1.2/auto/lib/zlib/makefile.msvc 2012-09-24 18:34:04 UTC (rev 4849) +++ branches/stable-1.2/auto/lib/zlib/makefile.msvc 2012-09-24 18:37:14 UTC (rev 4850) @@ -8,7 +8,10 @@ zlib.lib: cd $(ZLIB) - cl -c $(CFLAGS) adler32.c crc32.c deflate.c trees.c zutil.c compress.c + cl -c $(CFLAGS) adler32.c crc32.c deflate.c \ + trees.c zutil.c compress.c \ + inflate.c inffast.c inftrees.c link -lib -out:zlib.lib adler32.obj crc32.obj deflate.obj \ - trees.obj zutil.obj compress.obj + trees.obj zutil.obj compress.obj \ + inflate.obj inffast.obj inftrees.obj Modified: branches/stable-1.2/auto/lib/zlib/makefile.owc =================================================================== --- branches/stable-1.2/auto/lib/zlib/makefile.owc 2012-09-24 18:34:04 UTC (rev 4849) +++ branches/stable-1.2/auto/lib/zlib/makefile.owc 2012-09-24 18:37:14 UTC (rev 4850) @@ -9,6 +9,6 @@ cd $(ZLIB) wcl386 -c $(CFLAGS) adler32.c crc32.c deflate.c trees.c zutil.c & - compress.c + compress.c inflate.c inffast.c inftrees.c wlib -n zlib.lib adler32.obj crc32.obj deflate.obj trees.obj & - zutil.obj compress.obj + zutil.obj compress.obj inflate.obj inffast.obj inftrees.obj Modified: branches/stable-1.2/misc/GNUmakefile =================================================================== --- branches/stable-1.2/misc/GNUmakefile 2012-09-24 18:34:04 UTC (rev 4849) +++ branches/stable-1.2/misc/GNUmakefile 2012-09-24 18:37:14 UTC (rev 4850) @@ -7,7 +7,7 @@ OBJS = objs.msvc8 OPENSSL = openssl-1.0.1c -ZLIB = zlib-1.2.5 +ZLIB = zlib-1.2.7 PCRE = pcre-8.31 @@ -83,6 +83,43 @@ tar -c -z -f $(NGINX).tar.gz --directory $(TEMP) $(NGINX) +win32: + ./auto/configure \ + --with-cc=cl \ + --builddir=${OBJS} \ + --with-debug \ + --prefix= \ + --conf-path=conf/nginx.conf \ + --pid-path=logs/nginx.pid \ + --http-log-path=logs/access.log \ + --error-log-path=logs/error.log \ + --sbin-path=nginx.exe \ + --http-client-body-temp-path=temp/client_body_temp \ + --http-proxy-temp-path=temp/proxy_temp \ + --http-fastcgi-temp-path=temp/fastcgi_temp \ + --http-scgi-temp-path=temp/scgi_temp \ + --http-uwsgi-temp-path=temp/uwsgi_temp \ + --with-cc-opt=-DFD_SETSIZE=1024 \ + --with-pcre=${OBJS}/lib/${PCRE} \ + --with-zlib=${OBJS}/lib/${ZLIB} \ + --with-select_module \ + --with-http_realip_module \ + --with-http_addition_module \ + --with-http_sub_module \ + --with-http_dav_module \ + --with-http_stub_status_module \ + --with-http_flv_module \ + --with-http_mp4_module \ + --with-http_gunzip_module \ + --with-http_gzip_static_module \ + --with-http_random_index_module \ + --with-http_secure_link_module \ + --with-mail \ + --with-openssl=${OBJS}/lib/${OPENSSL} \ + --with-openssl-opt=enable-tlsext \ + --with-http_ssl_module \ + --with-mail_ssl_module \ + --with-ipv6 zip: rm -rf $(TEMP) Modified: branches/stable-1.2/src/os/win32/ngx_win32_config.h =================================================================== --- branches/stable-1.2/src/os/win32/ngx_win32_config.h 2012-09-24 18:34:04 UTC (rev 4849) +++ branches/stable-1.2/src/os/win32/ngx_win32_config.h 2012-09-24 18:37:14 UTC (rev 4850) @@ -18,6 +18,7 @@ /* enable getenv() and gmtime() in msvc8 */ #define _CRT_SECURE_NO_WARNINGS +#define _CRT_SECURE_NO_DEPRECATE /* * we need to include explicitly before because From mdounin at mdounin.ru Mon Sep 24 18:46:50 2012 From: mdounin at mdounin.ru (mdounin at mdounin.ru) Date: Mon, 24 Sep 2012 18:46:50 +0000 Subject: [nginx] svn commit: r4851 - in branches/stable-1.2: . auto auto/lib/libatomic auto/lib/perl Message-ID: <20120924184650.A3E933F9C44@mail.nginx.com> Author: mdounin Date: 2012-09-24 18:46:50 +0000 (Mon, 24 Sep 2012) New Revision: 4851 URL: http://trac.nginx.org/nginx/changeset/4851/nginx Log: Merge of r4842: fixed make macros to use parentheses. Configure: fixed make macros to use parentheses instead of braces. Parentheses are more portable, in particular they are understood by nmake while braces aren't. Modified: branches/stable-1.2/ branches/stable-1.2/auto/install branches/stable-1.2/auto/lib/libatomic/make branches/stable-1.2/auto/lib/perl/make Index: branches/stable-1.2 =================================================================== --- branches/stable-1.2 2012-09-24 18:37:14 UTC (rev 4850) +++ branches/stable-1.2 2012-09-24 18:46:50 UTC (rev 4851) Property changes on: branches/stable-1.2 ___________________________________________________________________ Modified: svn:mergeinfo ## -1 +1 ## -/trunk:4611-4632,4636-4657,4671-4672,4674-4676,4682,4684-4699,4704-4706,4713,4736-4741,4754,4756-4771,4775,4777-4780,4782-4783,4824,4830-4832,4834,4840,4843-4844 +/trunk:4611-4632,4636-4657,4671-4672,4674-4676,4682,4684-4699,4704-4706,4713,4736-4741,4754,4756-4771,4775,4777-4780,4782-4783,4824,4830-4832,4834,4840,4842-4844 \ No newline at end of property Modified: branches/stable-1.2/auto/install =================================================================== --- branches/stable-1.2/auto/install 2012-09-24 18:37:14 UTC (rev 4850) +++ branches/stable-1.2/auto/install 2012-09-24 18:46:50 UTC (rev 4851) @@ -8,7 +8,7 @@ cat << END >> $NGX_MAKEFILE install_perl_modules: - cd $NGX_OBJS/src/http/modules/perl && \${MAKE} install + cd $NGX_OBJS/src/http/modules/perl && \$(MAKE) install END NGX_INSTALL_PERL_MODULES=install_perl_modules Modified: branches/stable-1.2/auto/lib/libatomic/make =================================================================== --- branches/stable-1.2/auto/lib/libatomic/make 2012-09-24 18:37:14 UTC (rev 4850) +++ branches/stable-1.2/auto/lib/libatomic/make 2012-09-24 18:46:50 UTC (rev 4851) @@ -6,7 +6,7 @@ cat << END >> $NGX_MAKEFILE $NGX_LIBATOMIC/src/libatomic_ops.a: $NGX_LIBATOMIC/Makefile - cd $NGX_LIBATOMIC && \${MAKE} + cd $NGX_LIBATOMIC && \$(MAKE) $NGX_LIBATOMIC/Makefile: $NGX_MAKEFILE cd $NGX_LIBATOMIC && ./configure Modified: branches/stable-1.2/auto/lib/perl/make =================================================================== --- branches/stable-1.2/auto/lib/perl/make 2012-09-24 18:37:14 UTC (rev 4850) +++ branches/stable-1.2/auto/lib/perl/make 2012-09-24 18:46:50 UTC (rev 4851) @@ -12,7 +12,7 @@ $NGX_OBJS/src/http/modules/perl/Makefile cp -p src/http/modules/perl/nginx.* $NGX_OBJS/src/http/modules/perl/ - cd $NGX_OBJS/src/http/modules/perl && \${MAKE} + cd $NGX_OBJS/src/http/modules/perl && \$(MAKE) rm -rf $NGX_OBJS/install_perl From mdounin at mdounin.ru Mon Sep 24 18:50:26 2012 From: mdounin at mdounin.ru (mdounin at mdounin.ru) Date: Mon, 24 Sep 2012 18:50:26 +0000 Subject: [nginx] svn commit: r4852 - in branches/stable-1.2: . src/core Message-ID: <20120924185026.A3B283F9C44@mail.nginx.com> Author: mdounin Date: 2012-09-24 18:50:25 +0000 (Mon, 24 Sep 2012) New Revision: 4852 URL: http://trac.nginx.org/nginx/changeset/4852/nginx Log: Merge of r4784: fixed the -p parameter handling. Ensure that the path supplied always ends with a `/' except when empty. An empty value now corresponds to the current directory instead of `/'. Modified: branches/stable-1.2/ branches/stable-1.2/src/core/nginx.c Index: branches/stable-1.2 =================================================================== --- branches/stable-1.2 2012-09-24 18:46:50 UTC (rev 4851) +++ branches/stable-1.2 2012-09-24 18:50:25 UTC (rev 4852) Property changes on: branches/stable-1.2 ___________________________________________________________________ Modified: svn:mergeinfo ## -1 +1 ## -/trunk:4611-4632,4636-4657,4671-4672,4674-4676,4682,4684-4699,4704-4706,4713,4736-4741,4754,4756-4771,4775,4777-4780,4782-4783,4824,4830-4832,4834,4840,4842-4844 +/trunk:4611-4632,4636-4657,4671-4672,4674-4676,4682,4684-4699,4704-4706,4713,4736-4741,4754,4756-4771,4775,4777-4780,4782-4784,4824,4830-4832,4834,4840,4842-4844 \ No newline at end of property Modified: branches/stable-1.2/src/core/nginx.c =================================================================== --- branches/stable-1.2/src/core/nginx.c 2012-09-24 18:46:50 UTC (rev 4851) +++ branches/stable-1.2/src/core/nginx.c 2012-09-24 18:50:25 UTC (rev 4852) @@ -836,7 +836,7 @@ len = ngx_strlen(ngx_prefix); p = ngx_prefix; - if (!ngx_path_separator(*p)) { + if (len && !ngx_path_separator(p[len - 1])) { p = ngx_pnalloc(cycle->pool, len + 1); if (p == NULL) { return NGX_ERROR; From mdounin at mdounin.ru Mon Sep 24 18:54:29 2012 From: mdounin at mdounin.ru (mdounin at mdounin.ru) Date: Mon, 24 Sep 2012 18:54:29 +0000 Subject: [nginx] svn commit: r4853 - in branches/stable-1.2: . src/core src/event src/http/modules src/os/unix Message-ID: <20120924185429.4F4BF3F9E74@mail.nginx.com> Author: mdounin Date: 2012-09-24 18:54:28 +0000 (Mon, 24 Sep 2012) New Revision: 4853 URL: http://trac.nginx.org/nginx/changeset/4853/nginx Log: Merge of r4785, r4795, r4811, r4812, r4816, r4822: coverity. *) Resolver: fixed possible memory leak in ngx_resolver_create(). *) Explicitly ignore returned value from unlink() in ngx_open_tempfile(). *) Explicitly ignore returned value from close() in ngx_event_core_init_conf(). *) Added three missing checks for NULL after ngx_array_push() calls. *) Crypt: fixed handling of corrupted SSHA entries in password file. *) Mark logically dead code with corresponding comment. Found by / prodded by Coverity. Modified: branches/stable-1.2/ branches/stable-1.2/src/core/ngx_crypt.c branches/stable-1.2/src/core/ngx_resolver.c branches/stable-1.2/src/event/ngx_event.c branches/stable-1.2/src/http/modules/ngx_http_fastcgi_module.c branches/stable-1.2/src/http/modules/ngx_http_limit_conn_module.c branches/stable-1.2/src/http/modules/ngx_http_limit_req_module.c branches/stable-1.2/src/http/modules/ngx_http_ssi_filter_module.c branches/stable-1.2/src/os/unix/ngx_files.c Index: branches/stable-1.2 =================================================================== --- branches/stable-1.2 2012-09-24 18:50:25 UTC (rev 4852) +++ branches/stable-1.2 2012-09-24 18:54:28 UTC (rev 4853) Property changes on: branches/stable-1.2 ___________________________________________________________________ Modified: svn:mergeinfo ## -1 +1 ## -/trunk:4611-4632,4636-4657,4671-4672,4674-4676,4682,4684-4699,4704-4706,4713,4736-4741,4754,4756-4771,4775,4777-4780,4782-4784,4824,4830-4832,4834,4840,4842-4844 +/trunk:4611-4632,4636-4657,4671-4672,4674-4676,4682,4684-4699,4704-4706,4713,4736-4741,4754,4756-4771,4775,4777-4780,4782-4785,4795,4811-4812,4816,4822,4824,4830-4832,4834,4840,4842-4844 \ No newline at end of property Modified: branches/stable-1.2/src/core/ngx_crypt.c =================================================================== --- branches/stable-1.2/src/core/ngx_crypt.c 2012-09-24 18:50:25 UTC (rev 4852) +++ branches/stable-1.2/src/core/ngx_crypt.c 2012-09-24 18:54:28 UTC (rev 4853) @@ -194,6 +194,7 @@ ngx_crypt_ssha(ngx_pool_t *pool, u_char *key, u_char *salt, u_char **encrypted) { size_t len; + ngx_int_t rc; ngx_str_t encoded, decoded; ngx_sha1_t sha1; @@ -204,13 +205,19 @@ encoded.data = salt + sizeof("{SSHA}") - 1; encoded.len = ngx_strlen(encoded.data); - decoded.data = ngx_pnalloc(pool, ngx_base64_decoded_length(encoded.len)); + len = ngx_max(ngx_base64_decoded_length(encoded.len), 20); + + decoded.data = ngx_pnalloc(pool, len); if (decoded.data == NULL) { return NGX_ERROR; } - ngx_decode_base64(&decoded, &encoded); + rc = ngx_decode_base64(&decoded, &encoded); + if (rc != NGX_OK || decoded.len < 20) { + decoded.len = 20; + } + /* update SHA1 from key and salt */ ngx_sha1_init(&sha1); Modified: branches/stable-1.2/src/core/ngx_resolver.c =================================================================== --- branches/stable-1.2/src/core/ngx_resolver.c 2012-09-24 18:50:25 UTC (rev 4852) +++ branches/stable-1.2/src/core/ngx_resolver.c 2012-09-24 18:54:28 UTC (rev 4853) @@ -113,15 +113,6 @@ return NULL; } - if (n) { - if (ngx_array_init(&r->udp_connections, cf->pool, n, - sizeof(ngx_udp_connection_t)) - != NGX_OK) - { - return NULL; - } - } - cln->data = r; r->event = ngx_calloc(sizeof(ngx_event_t), cf->log); @@ -153,6 +144,15 @@ r->log = &cf->cycle->new_log; r->log_level = NGX_LOG_ERR; + if (n) { + if (ngx_array_init(&r->udp_connections, cf->pool, n, + sizeof(ngx_udp_connection_t)) + != NGX_OK) + { + return NULL; + } + } + for (i = 0; i < n; i++) { if (ngx_strncmp(names[i].data, "valid=", 6) == 0) { s.len = names[i].len - 6; Modified: branches/stable-1.2/src/event/ngx_event.c =================================================================== --- branches/stable-1.2/src/event/ngx_event.c 2012-09-24 18:50:25 UTC (rev 4852) +++ branches/stable-1.2/src/event/ngx_event.c 2012-09-24 18:54:28 UTC (rev 4853) @@ -1214,7 +1214,7 @@ fd = epoll_create(100); if (fd != -1) { - close(fd); + (void) close(fd); module = &ngx_epoll_module; } else if (ngx_errno != NGX_ENOSYS) { Modified: branches/stable-1.2/src/http/modules/ngx_http_fastcgi_module.c =================================================================== --- branches/stable-1.2/src/http/modules/ngx_http_fastcgi_module.c 2012-09-24 18:50:25 UTC (rev 4852) +++ branches/stable-1.2/src/http/modules/ngx_http_fastcgi_module.c 2012-09-24 18:54:28 UTC (rev 4853) @@ -1626,6 +1626,9 @@ } part = ngx_array_push(f->split_parts); + if (part == NULL) { + return NGX_ERROR; + } part->start = part_start; part->end = part_end; Modified: branches/stable-1.2/src/http/modules/ngx_http_limit_conn_module.c =================================================================== --- branches/stable-1.2/src/http/modules/ngx_http_limit_conn_module.c 2012-09-24 18:50:25 UTC (rev 4852) +++ branches/stable-1.2/src/http/modules/ngx_http_limit_conn_module.c 2012-09-24 18:54:28 UTC (rev 4853) @@ -721,6 +721,10 @@ } limit = ngx_array_push(&lccf->limits); + if (limit == NULL) { + return NGX_CONF_ERROR; + } + limit->conn = n; limit->shm_zone = shm_zone; Modified: branches/stable-1.2/src/http/modules/ngx_http_limit_req_module.c =================================================================== --- branches/stable-1.2/src/http/modules/ngx_http_limit_req_module.c 2012-09-24 18:50:25 UTC (rev 4852) +++ branches/stable-1.2/src/http/modules/ngx_http_limit_req_module.c 2012-09-24 18:54:28 UTC (rev 4853) @@ -937,6 +937,9 @@ } limit = ngx_array_push(&lrcf->limits); + if (limit == NULL) { + return NGX_CONF_ERROR; + } limit->shm_zone = shm_zone; limit->burst = burst * 1000; Modified: branches/stable-1.2/src/http/modules/ngx_http_ssi_filter_module.c =================================================================== --- branches/stable-1.2/src/http/modules/ngx_http_ssi_filter_module.c 2012-09-24 18:50:25 UTC (rev 4852) +++ branches/stable-1.2/src/http/modules/ngx_http_ssi_filter_module.c 2012-09-24 18:54:28 UTC (rev 4853) @@ -1024,6 +1024,7 @@ switch (state) { case ssi_start_state: + /* not reached */ break; case ssi_tag_state: Modified: branches/stable-1.2/src/os/unix/ngx_files.c =================================================================== --- branches/stable-1.2/src/os/unix/ngx_files.c 2012-09-24 18:50:25 UTC (rev 4852) +++ branches/stable-1.2/src/os/unix/ngx_files.c 2012-09-24 18:54:28 UTC (rev 4853) @@ -139,7 +139,7 @@ access ? access : 0600); if (fd != -1 && !persistent) { - unlink((const char *) name); + (void) unlink((const char *) name); } return fd; From mdounin at mdounin.ru Mon Sep 24 18:58:13 2012 From: mdounin at mdounin.ru (mdounin at mdounin.ru) Date: Mon, 24 Sep 2012 18:58:13 +0000 Subject: [nginx] svn commit: r4854 - in branches/stable-1.2: . src/mail Message-ID: <20120924185813.CB44B3F9C18@mail.nginx.com> Author: mdounin Date: 2012-09-24 18:58:13 +0000 (Mon, 24 Sep 2012) New Revision: 4854 URL: http://trac.nginx.org/nginx/changeset/4854/nginx Log: Merge of r4813, r4814, r4818, r4819, r4820, r4823: mail fixes. *) Corrected the directive name in the ngx_mail_auth_http_module error message. *) Don't let the well-known port in the "listen" directive to override the already set "protocol". *) Fixed sorting of listen addresses (ticket #187). For http module this problem was already fixed in r4756. *) Removed a stale "AF_INET only" comment. IPv6 client connections in mail modules have been supported since r2856. *) Fixed handling of AF_UNIX addresses in "listen". This makes AF_UNIX addresses in mail officially supported. *) Whitespace fix. Modified: branches/stable-1.2/ branches/stable-1.2/src/mail/ngx_mail.c branches/stable-1.2/src/mail/ngx_mail_auth_http_module.c branches/stable-1.2/src/mail/ngx_mail_core_module.c branches/stable-1.2/src/mail/ngx_mail_handler.c Index: branches/stable-1.2 =================================================================== --- branches/stable-1.2 2012-09-24 18:54:28 UTC (rev 4853) +++ branches/stable-1.2 2012-09-24 18:58:13 UTC (rev 4854) Property changes on: branches/stable-1.2 ___________________________________________________________________ Modified: svn:mergeinfo ## -1 +1 ## -/trunk:4611-4632,4636-4657,4671-4672,4674-4676,4682,4684-4699,4704-4706,4713,4736-4741,4754,4756-4771,4775,4777-4780,4782-4785,4795,4811-4812,4816,4822,4824,4830-4832,4834,4840,4842-4844 +/trunk:4611-4632,4636-4657,4671-4672,4674-4676,4682,4684-4699,4704-4706,4713,4736-4741,4754,4756-4771,4775,4777-4780,4782-4785,4795,4811-4814,4816,4818-4820,4822-4824,4830-4832,4834,4840,4842-4844 \ No newline at end of property Modified: branches/stable-1.2/src/mail/ngx_mail.c =================================================================== --- branches/stable-1.2/src/mail/ngx_mail.c 2012-09-24 18:54:28 UTC (rev 4853) +++ branches/stable-1.2/src/mail/ngx_mail.c 2012-09-24 18:58:13 UTC (rev 4854) @@ -263,6 +263,12 @@ break; #endif +#if (NGX_HAVE_UNIX_DOMAIN) + case AF_UNIX: + p = 0; + break; +#endif + default: /* AF_INET */ sin = (struct sockaddr_in *) sa; p = sin->sin_port; @@ -539,6 +545,11 @@ return 1; } + if (second->wildcard) { + /* a wildcard must be the last resort, shift it to the end */ + return -1; + } + if (first->bind && !second->bind) { /* shift explicit bind()ed addresses to the start */ return -1; Modified: branches/stable-1.2/src/mail/ngx_mail_auth_http_module.c =================================================================== --- branches/stable-1.2/src/mail/ngx_mail_auth_http_module.c 2012-09-24 18:54:28 UTC (rev 4853) +++ branches/stable-1.2/src/mail/ngx_mail_auth_http_module.c 2012-09-24 18:58:13 UTC (rev 4854) @@ -1332,7 +1332,7 @@ if (conf->peer == NULL) { ngx_log_error(NGX_LOG_EMERG, cf->log, 0, - "no \"http_auth\" is defined for server in %s:%ui", + "no \"auth_http\" is defined for server in %s:%ui", conf->file, conf->line); return NGX_CONF_ERROR; Modified: branches/stable-1.2/src/mail/ngx_mail_core_module.c =================================================================== --- branches/stable-1.2/src/mail/ngx_mail_core_module.c 2012-09-24 18:54:28 UTC (rev 4853) +++ branches/stable-1.2/src/mail/ngx_mail_core_module.c 2012-09-24 18:58:13 UTC (rev 4854) @@ -340,6 +340,14 @@ break; #endif +#if (NGX_HAVE_UNIX_DOMAIN) + case AF_UNIX: + off = offsetof(struct sockaddr_un, sun_path); + len = sizeof(((struct sockaddr_un *) sa)->sun_path); + port = 0; + break; +#endif + default: /* AF_INET */ off = offsetof(struct sockaddr_in, sin_addr); len = 4; @@ -374,21 +382,23 @@ ls->wildcard = u.wildcard; ls->ctx = cf->ctx; - for (m = 0; ngx_modules[m]; m++) { - if (ngx_modules[m]->type != NGX_MAIL_MODULE) { - continue; - } + if (cscf->protocol == NULL) { + for (m = 0; ngx_modules[m]; m++) { + if (ngx_modules[m]->type != NGX_MAIL_MODULE) { + continue; + } - module = ngx_modules[m]->ctx; + module = ngx_modules[m]->ctx; - if (module->protocol == NULL) { - continue; - } + if (module->protocol == NULL) { + continue; + } - for (i = 0; module->protocol->port[i]; i++) { - if (module->protocol->port[i] == u.port) { - cscf->protocol = module->protocol; - break; + for (i = 0; module->protocol->port[i]; i++) { + if (module->protocol->port[i] == u.port) { + cscf->protocol = module->protocol; + break; + } } } } Modified: branches/stable-1.2/src/mail/ngx_mail_handler.c =================================================================== --- branches/stable-1.2/src/mail/ngx_mail_handler.c 2012-09-24 18:54:28 UTC (rev 4853) +++ branches/stable-1.2/src/mail/ngx_mail_handler.c 2012-09-24 18:58:13 UTC (rev 4854) @@ -38,8 +38,6 @@ /* find the server configuration for the address:port */ - /* AF_INET only */ - port = c->listening->servers; if (port->naddrs > 1) { From mdounin at mdounin.ru Mon Sep 24 19:00:04 2012 From: mdounin at mdounin.ru (mdounin at mdounin.ru) Date: Mon, 24 Sep 2012 19:00:04 +0000 Subject: [nginx] svn commit: r4855 - in branches/stable-1.2: . src/http/modules Message-ID: <20120924190005.704233F9C18@mail.nginx.com> Author: mdounin Date: 2012-09-24 19:00:02 +0000 (Mon, 24 Sep 2012) New Revision: 4855 URL: http://trac.nginx.org/nginx/changeset/4855/nginx Log: Merge of r4815: map: fixed optimization of variables as values. Previous code incorrectly used ctx->var_values as an array of pointers to ngx_http_variable_value_t, but the array contains structures, not pointers. Additionally, ctx->var_values inspection failed to properly set var on match. Modified: branches/stable-1.2/ branches/stable-1.2/src/http/modules/ngx_http_map_module.c Index: branches/stable-1.2 =================================================================== --- branches/stable-1.2 2012-09-24 18:58:13 UTC (rev 4854) +++ branches/stable-1.2 2012-09-24 19:00:02 UTC (rev 4855) Property changes on: branches/stable-1.2 ___________________________________________________________________ Modified: svn:mergeinfo ## -1 +1 ## -/trunk:4611-4632,4636-4657,4671-4672,4674-4676,4682,4684-4699,4704-4706,4713,4736-4741,4754,4756-4771,4775,4777-4780,4782-4785,4795,4811-4814,4816,4818-4820,4822-4824,4830-4832,4834,4840,4842-4844 +/trunk:4611-4632,4636-4657,4671-4672,4674-4676,4682,4684-4699,4704-4706,4713,4736-4741,4754,4756-4771,4775,4777-4780,4782-4785,4795,4811-4816,4818-4820,4822-4824,4830-4832,4834,4840,4842-4844 \ No newline at end of property Modified: branches/stable-1.2/src/http/modules/ngx_http_map_module.c =================================================================== --- branches/stable-1.2/src/http/modules/ngx_http_map_module.c 2012-09-24 18:58:13 UTC (rev 4854) +++ branches/stable-1.2/src/http/modules/ngx_http_map_module.c 2012-09-24 19:00:02 UTC (rev 4855) @@ -416,11 +416,12 @@ for (i = 0; i < ctx->var_values.nelts; i++) { if (index == (ngx_int_t) var[i].data) { + var = &var[i]; goto found; } } - var = ngx_palloc(ctx->keys.pool, sizeof(ngx_http_variable_value_t)); + var = ngx_array_push(&ctx->var_values); if (var == NULL) { return NGX_CONF_ERROR; } @@ -431,13 +432,6 @@ var->len = 0; var->data = (u_char *) index; - vp = ngx_array_push(&ctx->var_values); - if (vp == NULL) { - return NGX_CONF_ERROR; - } - - *vp = var; - goto found; } From mdounin at mdounin.ru Mon Sep 24 19:02:16 2012 From: mdounin at mdounin.ru (mdounin at mdounin.ru) Date: Mon, 24 Sep 2012 19:02:16 +0000 Subject: [nginx] svn commit: r4856 - in branches/stable-1.2: . src/http/modules Message-ID: <20120924190216.BCD6E3F9C10@mail.nginx.com> Author: mdounin Date: 2012-09-24 19:02:16 +0000 (Mon, 24 Sep 2012) New Revision: 4856 URL: http://trac.nginx.org/nginx/changeset/4856/nginx Log: Merge of r4817: geo: fixed handling of ranges without default set. The bug had appeared in 0.8.43 (r3653). Patch by Weibin Yao. Modified: branches/stable-1.2/ branches/stable-1.2/src/http/modules/ngx_http_geo_module.c Index: branches/stable-1.2 =================================================================== --- branches/stable-1.2 2012-09-24 19:00:02 UTC (rev 4855) +++ branches/stable-1.2 2012-09-24 19:02:16 UTC (rev 4856) Property changes on: branches/stable-1.2 ___________________________________________________________________ Modified: svn:mergeinfo ## -1 +1 ## -/trunk:4611-4632,4636-4657,4671-4672,4674-4676,4682,4684-4699,4704-4706,4713,4736-4741,4754,4756-4771,4775,4777-4780,4782-4785,4795,4811-4816,4818-4820,4822-4824,4830-4832,4834,4840,4842-4844 +/trunk:4611-4632,4636-4657,4671-4672,4674-4676,4682,4684-4699,4704-4706,4713,4736-4741,4754,4756-4771,4775,4777-4780,4782-4785,4795,4811-4820,4822-4824,4830-4832,4834,4840,4842-4844 \ No newline at end of property Modified: branches/stable-1.2/src/http/modules/ngx_http_geo_module.c =================================================================== --- branches/stable-1.2/src/http/modules/ngx_http_geo_module.c 2012-09-24 19:00:02 UTC (rev 4855) +++ branches/stable-1.2/src/http/modules/ngx_http_geo_module.c 2012-09-24 19:02:16 UTC (rev 4856) @@ -400,15 +400,15 @@ } } + if (ctx.high.default_value == NULL) { + ctx.high.default_value = &ngx_http_variable_null_value; + } + geo->u.high = ctx.high; var->get_handler = ngx_http_geo_range_variable; var->data = (uintptr_t) geo; - if (ctx.high.default_value == NULL) { - ctx.high.default_value = &ngx_http_variable_null_value; - } - ngx_destroy_pool(ctx.temp_pool); ngx_destroy_pool(pool); From mdounin at mdounin.ru Mon Sep 24 19:05:03 2012 From: mdounin at mdounin.ru (mdounin at mdounin.ru) Date: Mon, 24 Sep 2012 19:05:03 +0000 Subject: [nginx] svn commit: r4857 - in branches/stable-1.2: . src/core src/http src/http/modules Message-ID: <20120924190503.D1E433F9C10@mail.nginx.com> Author: mdounin Date: 2012-09-24 19:05:02 +0000 (Mon, 24 Sep 2012) New Revision: 4857 URL: http://trac.nginx.org/nginx/changeset/4857/nginx Log: Merge of r4828: "include" with wildcards in map and types blocks. The "include" directive should be able to include multiple files if given a filename mask. Fixed this to work for "include" directives inside the "map" or "types" blocks. The "include" directive inside the "geo" block is still not fixed. Modified: branches/stable-1.2/ branches/stable-1.2/src/core/ngx_conf_file.c branches/stable-1.2/src/core/ngx_conf_file.h branches/stable-1.2/src/http/modules/ngx_http_map_module.c branches/stable-1.2/src/http/ngx_http_core_module.c Index: branches/stable-1.2 =================================================================== --- branches/stable-1.2 2012-09-24 19:02:16 UTC (rev 4856) +++ branches/stable-1.2 2012-09-24 19:05:02 UTC (rev 4857) Property changes on: branches/stable-1.2 ___________________________________________________________________ Modified: svn:mergeinfo ## -1 +1 ## -/trunk:4611-4632,4636-4657,4671-4672,4674-4676,4682,4684-4699,4704-4706,4713,4736-4741,4754,4756-4771,4775,4777-4780,4782-4785,4795,4811-4820,4822-4824,4830-4832,4834,4840,4842-4844 +/trunk:4611-4632,4636-4657,4671-4672,4674-4676,4682,4684-4699,4704-4706,4713,4736-4741,4754,4756-4771,4775,4777-4780,4782-4785,4795,4811-4820,4822-4824,4828,4830-4832,4834,4840,4842-4844 \ No newline at end of property Modified: branches/stable-1.2/src/core/ngx_conf_file.c =================================================================== --- branches/stable-1.2/src/core/ngx_conf_file.c 2012-09-24 19:02:16 UTC (rev 4856) +++ branches/stable-1.2/src/core/ngx_conf_file.c 2012-09-24 19:05:02 UTC (rev 4857) @@ -12,7 +12,6 @@ static ngx_int_t ngx_conf_handler(ngx_conf_t *cf, ngx_int_t last); static ngx_int_t ngx_conf_read_token(ngx_conf_t *cf); -static char *ngx_conf_include(ngx_conf_t *cf, ngx_command_t *cmd, void *conf); static ngx_int_t ngx_conf_test_full_name(ngx_str_t *name); static void ngx_conf_flush_files(ngx_cycle_t *cycle); @@ -731,7 +730,7 @@ } -static char * +char * ngx_conf_include(ngx_conf_t *cf, ngx_command_t *cmd, void *conf) { char *rv; Modified: branches/stable-1.2/src/core/ngx_conf_file.h =================================================================== --- branches/stable-1.2/src/core/ngx_conf_file.h 2012-09-24 19:02:16 UTC (rev 4856) +++ branches/stable-1.2/src/core/ngx_conf_file.h 2012-09-24 19:05:02 UTC (rev 4857) @@ -317,6 +317,7 @@ char *ngx_conf_param(ngx_conf_t *cf); char *ngx_conf_parse(ngx_conf_t *cf, ngx_str_t *filename); +char *ngx_conf_include(ngx_conf_t *cf, ngx_command_t *cmd, void *conf); ngx_int_t ngx_conf_full_name(ngx_cycle_t *cycle, ngx_str_t *name, Modified: branches/stable-1.2/src/http/modules/ngx_http_map_module.c =================================================================== --- branches/stable-1.2/src/http/modules/ngx_http_map_module.c 2012-09-24 19:02:16 UTC (rev 4856) +++ branches/stable-1.2/src/http/modules/ngx_http_map_module.c 2012-09-24 19:05:02 UTC (rev 4857) @@ -369,7 +369,7 @@ ngx_http_map(ngx_conf_t *cf, ngx_command_t *dummy, void *conf) { ngx_int_t rc, index; - ngx_str_t *value, file, name; + ngx_str_t *value, name; ngx_uint_t i, key; ngx_http_map_conf_ctx_t *ctx; ngx_http_variable_value_t *var, **vp; @@ -391,15 +391,7 @@ } if (ngx_strcmp(value[0].data, "include") == 0) { - file = value[1]; - - if (ngx_conf_full_name(cf->cycle, &file, 1) != NGX_OK) { - return NGX_CONF_ERROR; - } - - ngx_log_debug1(NGX_LOG_DEBUG_CORE, cf->log, 0, "include %s", file.data); - - return ngx_conf_parse(cf, &file); + return ngx_conf_include(cf, dummy, conf); } if (value[1].data[0] == '$') { Modified: branches/stable-1.2/src/http/ngx_http_core_module.c =================================================================== --- branches/stable-1.2/src/http/ngx_http_core_module.c 2012-09-24 19:02:16 UTC (rev 4856) +++ branches/stable-1.2/src/http/ngx_http_core_module.c 2012-09-24 19:05:02 UTC (rev 4857) @@ -3193,7 +3193,7 @@ { ngx_http_core_loc_conf_t *clcf = conf; - ngx_str_t *value, *content_type, *old, file; + ngx_str_t *value, *content_type, *old; ngx_uint_t i, n, hash; ngx_hash_key_t *type; @@ -3206,15 +3206,8 @@ " in \"include\" directive"); return NGX_CONF_ERROR; } - file = value[1]; - if (ngx_conf_full_name(cf->cycle, &file, 1) != NGX_OK) { - return NGX_CONF_ERROR; - } - - ngx_log_debug1(NGX_LOG_DEBUG_CORE, cf->log, 0, "include %s", file.data); - - return ngx_conf_parse(cf, &file); + return ngx_conf_include(cf, dummy, conf); } content_type = ngx_palloc(cf->pool, sizeof(ngx_str_t)); From mdounin at mdounin.ru Mon Sep 24 19:06:48 2012 From: mdounin at mdounin.ru (mdounin at mdounin.ru) Date: Mon, 24 Sep 2012 19:06:48 +0000 Subject: [nginx] svn commit: r4858 - in branches/stable-1.2: . src/http src/http/modules Message-ID: <20120924190649.1162D3F9C14@mail.nginx.com> Author: mdounin Date: 2012-09-24 19:06:48 +0000 (Mon, 24 Sep 2012) New Revision: 4858 URL: http://trac.nginx.org/nginx/changeset/4858/nginx Log: Merge of r4829: fixed strict aliasing with ipv6 (ticket #201). Fixed strict aliasing bugs when dealing with IPv4-mapped IPv6 addresses. Modified: branches/stable-1.2/ branches/stable-1.2/src/http/modules/ngx_http_geo_module.c branches/stable-1.2/src/http/modules/ngx_http_geoip_module.c branches/stable-1.2/src/http/ngx_http_core_module.c Index: branches/stable-1.2 =================================================================== --- branches/stable-1.2 2012-09-24 19:05:02 UTC (rev 4857) +++ branches/stable-1.2 2012-09-24 19:06:48 UTC (rev 4858) Property changes on: branches/stable-1.2 ___________________________________________________________________ Modified: svn:mergeinfo ## -1 +1 ## -/trunk:4611-4632,4636-4657,4671-4672,4674-4676,4682,4684-4699,4704-4706,4713,4736-4741,4754,4756-4771,4775,4777-4780,4782-4785,4795,4811-4820,4822-4824,4828,4830-4832,4834,4840,4842-4844 +/trunk:4611-4632,4636-4657,4671-4672,4674-4676,4682,4684-4699,4704-4706,4713,4736-4741,4754,4756-4771,4775,4777-4780,4782-4785,4795,4811-4820,4822-4824,4828-4832,4834,4840,4842-4844 \ No newline at end of property Modified: branches/stable-1.2/src/http/modules/ngx_http_geo_module.c =================================================================== --- branches/stable-1.2/src/http/modules/ngx_http_geo_module.c 2012-09-24 19:05:02 UTC (rev 4857) +++ branches/stable-1.2/src/http/modules/ngx_http_geo_module.c 2012-09-24 19:06:48 UTC (rev 4858) @@ -233,12 +233,21 @@ #if (NGX_HAVE_INET6) if (addr.sockaddr->sa_family == AF_INET6) { + u_char *p; + in_addr_t inaddr; struct in6_addr *inaddr6; inaddr6 = &((struct sockaddr_in6 *) addr.sockaddr)->sin6_addr; if (IN6_IS_ADDR_V4MAPPED(inaddr6)) { - return ntohl(*(in_addr_t *) &inaddr6->s6_addr[12]); + p = inaddr6->s6_addr; + + inaddr = p[12] << 24; + inaddr += p[13] << 16; + inaddr += p[14] << 8; + inaddr += p[15]; + + return inaddr; } } Modified: branches/stable-1.2/src/http/modules/ngx_http_geoip_module.c =================================================================== --- branches/stable-1.2/src/http/modules/ngx_http_geoip_module.c 2012-09-24 19:05:02 UTC (rev 4857) +++ branches/stable-1.2/src/http/modules/ngx_http_geoip_module.c 2012-09-24 19:06:48 UTC (rev 4858) @@ -226,12 +226,21 @@ #if (NGX_HAVE_INET6) if (addr.sockaddr->sa_family == AF_INET6) { + u_char *p; + in_addr_t inaddr; struct in6_addr *inaddr6; inaddr6 = &((struct sockaddr_in6 *) addr.sockaddr)->sin6_addr; if (IN6_IS_ADDR_V4MAPPED(inaddr6)) { - return ntohl(*(in_addr_t *) &inaddr6->s6_addr[12]); + p = inaddr6->s6_addr; + + inaddr = p[12] << 24; + inaddr += p[13] << 16; + inaddr += p[14] << 8; + inaddr += p[15]; + + return inaddr; } } Modified: branches/stable-1.2/src/http/ngx_http_core_module.c =================================================================== --- branches/stable-1.2/src/http/ngx_http_core_module.c 2012-09-24 19:05:02 UTC (rev 4857) +++ branches/stable-1.2/src/http/ngx_http_core_module.c 2012-09-24 19:06:48 UTC (rev 4858) @@ -2733,7 +2733,15 @@ if (IN6_IS_ADDR_V4MAPPED(inaddr6)) { family = AF_INET; - inaddr = *(in_addr_t *) &inaddr6->s6_addr[12]; + + p = inaddr6->s6_addr; + + inaddr = p[12] << 24; + inaddr += p[13] << 16; + inaddr += p[14] << 8; + inaddr += p[15]; + + inaddr = htonl(inaddr); } } #endif From mdounin at mdounin.ru Mon Sep 24 19:11:46 2012 From: mdounin at mdounin.ru (mdounin at mdounin.ru) Date: Mon, 24 Sep 2012 19:11:46 +0000 Subject: [nginx] svn commit: r4859 - in branches/stable-1.2: . src/http/modules Message-ID: <20120924191146.4A6D13F9C0C@mail.nginx.com> Author: mdounin Date: 2012-09-24 19:11:45 +0000 (Mon, 24 Sep 2012) New Revision: 4859 URL: http://trac.nginx.org/nginx/changeset/4859/nginx Log: Merge of r4833: limit req: fix of rbtree node insertion. Limit req: fix of rbtree node insertion on hash collisions. The rbtree used in ngx_http_limit_req_module has two level of keys, the top is hash, and the next is the value string itself. However, when inserting a new node, only hash has been set, while the value string has been left empty. The bug was introduced in r4419 (1.1.14). Found by Charles Chen. Modified: branches/stable-1.2/ branches/stable-1.2/src/http/modules/ngx_http_limit_req_module.c Index: branches/stable-1.2 =================================================================== --- branches/stable-1.2 2012-09-24 19:06:48 UTC (rev 4858) +++ branches/stable-1.2 2012-09-24 19:11:45 UTC (rev 4859) Property changes on: branches/stable-1.2 ___________________________________________________________________ Modified: svn:mergeinfo ## -1 +1 ## -/trunk:4611-4632,4636-4657,4671-4672,4674-4676,4682,4684-4699,4704-4706,4713,4736-4741,4754,4756-4771,4775,4777-4780,4782-4785,4795,4811-4820,4822-4824,4828-4832,4834,4840,4842-4844 +/trunk:4611-4632,4636-4657,4671-4672,4674-4676,4682,4684-4699,4704-4706,4713,4736-4741,4754,4756-4771,4775,4777-4780,4782-4785,4795,4811-4820,4822-4824,4828-4834,4840,4842-4844 \ No newline at end of property Modified: branches/stable-1.2/src/http/modules/ngx_http_limit_req_module.c =================================================================== --- branches/stable-1.2/src/http/modules/ngx_http_limit_req_module.c 2012-09-24 19:06:48 UTC (rev 4858) +++ branches/stable-1.2/src/http/modules/ngx_http_limit_req_module.c 2012-09-24 19:11:45 UTC (rev 4859) @@ -444,17 +444,17 @@ node->key = hash; - ngx_rbtree_insert(&ctx->sh->rbtree, node); - lr = (ngx_http_limit_req_node_t *) &node->color; - ngx_queue_insert_head(&ctx->sh->queue, &lr->queue); - lr->len = (u_char) len; lr->excess = 0; ngx_memcpy(lr->data, data, len); + ngx_rbtree_insert(&ctx->sh->rbtree, node); + + ngx_queue_insert_head(&ctx->sh->queue, &lr->queue); + if (account) { lr->last = now; lr->count = 0; From mdounin at mdounin.ru Mon Sep 24 19:14:17 2012 From: mdounin at mdounin.ru (mdounin at mdounin.ru) Date: Mon, 24 Sep 2012 19:14:17 +0000 Subject: [nginx] svn commit: r4860 - in branches/stable-1.2: . docs/html Message-ID: <20120924191417.B09C83F9C46@mail.nginx.com> Author: mdounin Date: 2012-09-24 19:14:17 +0000 (Mon, 24 Sep 2012) New Revision: 4860 URL: http://trac.nginx.org/nginx/changeset/4860/nginx Log: Merge of r4835, r4841: improved welcome and 50x error pages. Modified: branches/stable-1.2/ branches/stable-1.2/docs/html/50x.html branches/stable-1.2/docs/html/index.html Index: branches/stable-1.2 =================================================================== --- branches/stable-1.2 2012-09-24 19:11:45 UTC (rev 4859) +++ branches/stable-1.2 2012-09-24 19:14:17 UTC (rev 4860) Property changes on: branches/stable-1.2 ___________________________________________________________________ Modified: svn:mergeinfo ## -1 +1 ## -/trunk:4611-4632,4636-4657,4671-4672,4674-4676,4682,4684-4699,4704-4706,4713,4736-4741,4754,4756-4771,4775,4777-4780,4782-4785,4795,4811-4820,4822-4824,4828-4834,4840,4842-4844 +/trunk:4611-4632,4636-4657,4671-4672,4674-4676,4682,4684-4699,4704-4706,4713,4736-4741,4754,4756-4771,4775,4777-4780,4782-4785,4795,4811-4820,4822-4824,4828-4835,4840-4844 \ No newline at end of property Modified: branches/stable-1.2/docs/html/50x.html =================================================================== --- branches/stable-1.2/docs/html/50x.html 2012-09-24 19:11:45 UTC (rev 4859) +++ branches/stable-1.2/docs/html/50x.html 2012-09-24 19:14:17 UTC (rev 4860) @@ -1,18 +1,21 @@ + -The page is temporarily unavailable +Error - - - - - -
-The page you are looking for is temporarily unavailable.
-Please try again later. -
+ +

An error occurred.

+

Sorry, the page you are looking for is currently unavailable.
+Please try again later.

+

If you are the system administrator of this resource then you should check +the error log for details.

+

Faithfully yours, nginx.

Modified: branches/stable-1.2/docs/html/index.html =================================================================== --- branches/stable-1.2/docs/html/index.html 2012-09-24 19:11:45 UTC (rev 4859) +++ branches/stable-1.2/docs/html/index.html 2012-09-24 19:14:17 UTC (rev 4860) @@ -1,8 +1,25 @@ + Welcome to nginx! + - -

Welcome to nginx!

+ +

Welcome to nginx!

+

If you see this page, the nginx web server is successfully installed and +working. Further configuration is required.

+ +

For online documentation and support please refer to +nginx.org.
+Commercial support is available at +nginx.com.

+ +

Thank you for using nginx.

From mdounin at mdounin.ru Tue Sep 25 04:39:28 2012 From: mdounin at mdounin.ru (Maxim Dounin) Date: Tue, 25 Sep 2012 08:39:28 +0400 Subject: [PATCH] (re-post) Add "optional_no_ca" option to ssl_verify_client to enable app-only CA chain validation In-Reply-To: <20120922170953.3f8f50b1@sacrilege> References: <20120918074355.GQ40452@mdounin.ru> <20120922170953.3f8f50b1@sacrilege> Message-ID: <20120925043928.GE40452@mdounin.ru> Hello! On Sat, Sep 22, 2012 at 05:09:53PM +0600, Mike Kazantsev wrote: [...] > > In particular, I would like someone to actually test if the > > error_page 495 aproach works instead as suggested here: > > > > http://mailman.nginx.org/pipermail/nginx-devel/2012-August/002650.html > > > > It doesn't seem to be of much use in current state, problems: > > - Requires "ssl_client_certificate" to be set to some valid > certificate, which then shouldn't actually never be used in this > case. This actually counts as a separate problem. I think it should be possible to specify empty certificate authorities list to be sent to clients even if nginx does actual verification. Part of the problem is addressed by the ssl_trusted_certificate directive which is going to be introduced along with OCSP stapling patches. > - In case of "ssl_verify_client" set to "on" or "optional", setting > "error_page 400 495 496 =200 /altpath;" doesn't seem to stop nginx > from returning "HTTP/1.1 400 Bad Request" response with "400 The SSL > certificate error" in html body. > > This code in ngx_http_request.c is probable cause of that: > > if (sscf->verify) { > rc = SSL_get_verify_result(c->ssl->connection); > > if (rc != X509_V_OK) { > ngx_log_error(NGX_LOG_INFO, c->log, 0, > "client SSL certificate verify error: (%l:%s)", > rc, X509_verify_cert_error_string(rc)); > > ngx_ssl_remove_cached_session(sscf->ssl.ctx, > (SSL_get0_session(c->ssl->connection))); > > ngx_http_finalize_request(r, NGX_HTTPS_CERT_ERROR); > return; > } > > I'm still unsure if it's a general bug in ssl module or some error > in my configuration above, because docs clearly state that error > should be handled by error_page and it doesn't seem to be. Works fine here: ssl_verify_client on; ssl_client_certificate ca.crt; error_page 495 = /foo; location = /foo { return 200 "dn: $ssl_client_s_dn, verify: $ssl_client_verify\n"; } $ openssl s_client -connect ... -cert ... -key ... ... GET / HTTP/1.0 HTTP/1.1 200 OK Server: nginx/1.3.6 Date: Tue, 25 Sep 2012 02:37:45 GMT Content-Type: text/plain Content-Length: 35 Connection: close dn: /CN=mdounin.ru, verify: FAILED > - "ssl_verify_client off;" isn't much useful, because it doesn't return > clent certificate and doesn't check it in any way. > > - "ssl_verify_client on;", still gives all-or-nothing check, though > I see that it's what might indeed be desirable, as Eric indicated. > > - I think it's really non-obvious way to do it. > > > > And a quick comment for your patch: I tend to think that > > introduction of ngx_http_ssl_variable_get_client_verify() is > > misleading. We shouldn't try to claim the certificate was > > verified unless it actually was. > > It might be misleading, indeed, I see your point. > Attached patch doesn't try to alter ssl_client_verify result. > > On a completely unrelated note - nginx fails to build from svn here due > to 'Exception in thread "main" java.lang.NoClassDefFoundError: > com/pault/StyleSheet' (so I just disabled building changelog for tests). Just checkout docs/xslt/changes.xslt again (or touch it). (It's dependencies was changed without any changes to commit in the resulting file, hence make tries to rebuild it but fails as you have no com.pault.StyleSheet java class needed to convert XSLScript to XSLT.) On the other hand, building changelog is completely separate process which is needed to build release tarball, but not to build nginx itself. > > URL for the revised patch: https://gist.github.com/3319062 > > > Inline patch follows. See below for some comments. > > diff --git a/src/event/ngx_event_openssl.h b/src/event/ngx_event_openssl.h > index cd6d885..97da051 100644 > --- a/src/event/ngx_event_openssl.h > +++ b/src/event/ngx_event_openssl.h > @@ -141,6 +141,14 @@ ngx_int_t ngx_ssl_get_client_verify(ngx_connection_t *c, ngx_pool_t *pool, > ngx_str_t *s); > > > +#define ngx_ssl_verify_error_is_optional(errnum) \ > + ((errnum == X509_V_ERR_DEPTH_ZERO_SELF_SIGNED_CERT) \ > + || (errnum == X509_V_ERR_SELF_SIGNED_CERT_IN_CHAIN) \ > + || (errnum == X509_V_ERR_UNABLE_TO_GET_ISSUER_CERT_LOCALLY) \ > + || (errnum == X509_V_ERR_CERT_UNTRUSTED) \ > + || (errnum == X509_V_ERR_UNABLE_TO_VERIFY_LEAF_SIGNATURE)) This should be adapted to match nginx style. > + > + > ngx_int_t ngx_ssl_handshake(ngx_connection_t *c); > ssize_t ngx_ssl_recv(ngx_connection_t *c, u_char *buf, size_t size); > ssize_t ngx_ssl_write(ngx_connection_t *c, u_char *data, size_t size); > diff --git a/src/http/modules/ngx_http_ssl_module.c b/src/http/modules/ngx_http_ssl_module.c > index d759489..ab91670 100644 > --- a/src/http/modules/ngx_http_ssl_module.c > +++ b/src/http/modules/ngx_http_ssl_module.c > @@ -48,6 +48,7 @@ static ngx_conf_enum_t ngx_http_ssl_verify[] = { > { ngx_string("off"), 0 }, > { ngx_string("on"), 1 }, > { ngx_string("optional"), 2 }, > + { ngx_string("optional_no_ca"), 3 }, > { ngx_null_string, 0 } > }; > > @@ -466,7 +467,7 @@ ngx_http_ssl_merge_srv_conf(ngx_conf_t *cf, void *parent, void *child) > > if (conf->verify) { > > - if (conf->client_certificate.len == 0) { > + if (conf->verify != 3 && conf->client_certificate.len == 0) { >From readability point of view I would like to see this reversed, i.e. if (conf->client_certificate.len == 0 && conf->verify != 3) { This way it would be more obvious that we check client certificate to be present (but allow it to be empty in the optional_no_ca case). > ngx_log_error(NGX_LOG_EMERG, cf->log, 0, > "no ssl_client_certificate for ssl_client_verify"); > return NGX_CONF_ERROR; > diff --git a/src/http/ngx_http_request.c b/src/http/ngx_http_request.c > index cb970c5..96cec55 100644 > --- a/src/http/ngx_http_request.c > +++ b/src/http/ngx_http_request.c > @@ -1642,7 +1642,9 @@ ngx_http_process_request(ngx_http_request_t *r) > if (sscf->verify) { > rc = SSL_get_verify_result(c->ssl->connection); > > - if (rc != X509_V_OK) { > + if ((sscf->verify != 3 && rc != X509_V_OK) > + || !(sscf->verify == 3 && ngx_ssl_verify_error_is_optional(rc))) > + { What happens here if sscf->verify == 3 and rc == X509_V_OK? Is this intentional? It probably should be if (rc != X509_V_OK && (sscf->verify != 3 || !ngx_ssl_verify_error_is_optional(rc))) { instead. > ngx_log_error(NGX_LOG_INFO, c->log, 0, > "client SSL certificate verify error: (%l:%s)", > rc, X509_verify_cert_error_string(rc)); > -- > 1.7.12 Maxim Dounin From mdounin at mdounin.ru Tue Sep 25 05:18:34 2012 From: mdounin at mdounin.ru (Maxim Dounin) Date: Tue, 25 Sep 2012 09:18:34 +0400 Subject: [PATCH] Unix domain sockets in abstract namespace In-Reply-To: References: <20120919135608.GH40452@mdounin.ru> Message-ID: <20120925051834.GH40452@mdounin.ru> Hello! On Fri, Sep 21, 2012 at 09:28:16AM +0200, Dirk Feytons wrote: > On Wed, Sep 19, 2012 at 3:56 PM, Maxim Dounin wrote: > > Hello! > > > > On Wed, Sep 12, 2012 at 11:46:52AM +0200, Dirk Feytons wrote: > > > >> Hi, > >> > >> I was playing around with ngx_lua and its socket API, which is built > >> on top of nginx functionality. The API supports unix domain sockets > >> but not in the abstract namespace, at least not officially. When I > >> tried specifying an abstract address I found it didn't work. I tracked > >> it down to two calls to ngx_cpystrn() in > >> core/ngx_inet.c:ngx_parse_unix_domain_url(). That function gets > >> confused by the leading '\0' in an abstract namespace address. > > > > This is because the abstract namespace is actually linux extension > > to unix domain sockets, "The abstract socket namespace is a > > nonportable Linux extension" (from [1]). > > > > Strictly speaking, it's not something even permitted by POSIX as > > socket path ends with first null byte as per POSIX. > > Yeah; I know it's Linux specific. Too bad in my opinion; I find it > easier to work with than the regular namespace. > > >> Changing it to ngx_memcpy() fixes this and then everything works. > >> > >> I attached a patch agains nginx 1.2.3. The fix seems trivial but this > >> is my first look into the nginx internals so I don't know whether > >> there are unwanted side effects from this change. Feedback welcome. > > > > I suspect there may be some, in particular during logging of > > errors and probably while parsing untrusted urls. > > Okay, I'll have a look at the code again. > Are you willing to accept a patch to support the abstract namespace? > Or does the fact that it's a nonportable Linux extension make it a > no-go for inclusion in nginx? It would be fine to add support, though I would like to make sure various possible side effects are checked/addressed. Maxim Dounin From leon at kingdest.com Tue Sep 25 06:16:53 2012 From: leon at kingdest.com (Leon) Date: Tue, 25 Sep 2012 14:16:53 +0800 Subject: how to set up the mutual authentication of client access different pages? Message-ID: how to set up the mutual authentication of client access different pages? -- Leon www.kingdest.com Sent from my Android phone with K-9 Mail. Please excuse my brevity. From mdounin at mdounin.ru Tue Sep 25 10:41:23 2012 From: mdounin at mdounin.ru (mdounin at mdounin.ru) Date: Tue, 25 Sep 2012 10:41:23 +0000 Subject: [nginx] svn commit: r4861 - branches/stable-1.2/misc Message-ID: <20120925104124.1AA3E3F9C1B@mail.nginx.com> Author: mdounin Date: 2012-09-25 10:41:22 +0000 (Tue, 25 Sep 2012) New Revision: 4861 URL: http://trac.nginx.org/nginx/changeset/4861/nginx Log: Stable: removed gunzip module from win32 helper target. Gunzip filter isn't merged into stable branch. Modified: branches/stable-1.2/misc/GNUmakefile Modified: branches/stable-1.2/misc/GNUmakefile =================================================================== --- branches/stable-1.2/misc/GNUmakefile 2012-09-24 19:14:17 UTC (rev 4860) +++ branches/stable-1.2/misc/GNUmakefile 2012-09-25 10:41:22 UTC (rev 4861) @@ -110,7 +110,6 @@ --with-http_stub_status_module \ --with-http_flv_module \ --with-http_mp4_module \ - --with-http_gunzip_module \ --with-http_gzip_static_module \ --with-http_random_index_module \ --with-http_secure_link_module \ From mdounin at mdounin.ru Tue Sep 25 13:42:44 2012 From: mdounin at mdounin.ru (mdounin at mdounin.ru) Date: Tue, 25 Sep 2012 13:42:44 +0000 Subject: [nginx] svn commit: r4862 - branches/stable-1.2/docs/xml/nginx Message-ID: <20120925134244.D79403FA005@mail.nginx.com> Author: mdounin Date: 2012-09-25 13:42:43 +0000 (Tue, 25 Sep 2012) New Revision: 4862 URL: http://trac.nginx.org/nginx/changeset/4862/nginx Log: nginx-1.2.4-RELEASE Modified: branches/stable-1.2/docs/xml/nginx/changes.xml Modified: branches/stable-1.2/docs/xml/nginx/changes.xml =================================================================== --- branches/stable-1.2/docs/xml/nginx/changes.xml 2012-09-25 10:41:22 UTC (rev 4861) +++ branches/stable-1.2/docs/xml/nginx/changes.xml 2012-09-25 13:42:43 UTC (rev 4862) @@ -5,6 +5,101 @@ + + + + +? ????????? "limit_req"; +?????? ????????? ? 1.1.14.
+??????? Charles Chen. +
+ +in the "limit_req" directive; +the bug had appeared in 1.1.14.
+Thanks to Charles Chen. +
+
+ + + +nginx ?? ????????? gcc 4.7 ? ???????????? -O2 +???? ????????????? ???????? --with-ipv6. + + +nginx could not be built by gcc 4.7 with -O2 optimization +if the --with-ipv6 option was used. + + + + + +? ??????? ???????? ??? ????????? segmentation fault, +???? ? ????????? map ? ???????? ???????? ?????????????? ??????????. + + +a segmentation fault might occur in a worker process +if the "map" directive was used with variables as values. + + + + + +? ??????? ???????? ??? ????????? segmentation fault +??? ????????????? ????????? geo ? ?????????? ranges, +?? ??? ????????? default; ?????? ????????? ? 0.8.43.
+??????? Zhen Chen ? Weibin Yao. +
+ +a segmentation fault might occur in a worker process +if the "geo" directive was used with the "ranges" parameter +but without the "default" parameter; the bug had appeared in 0.8.43.
+Thanks to Zhen Chen and Weibin Yao. +
+
+ + + +? ????????? ????????? ????????? ?????? -p. + + +in the -p command-line parameter handling. + + + + + +? ???????? ??????-???????. + + +in the mail proxy server. + + + + + +?????????????? ????????????? ??????.
+??????? Coverity. +
+ +of minor potential bugs.
+Thanks to Coverity. +
+
+ + + +nginx/Windows ?? ????????? ? Visual Studio 2005 Express.
+??????? HAYASHI Kentaro. +
+ +nginx/Windows could not be built with Visual Studio 2005 Express.
+Thanks to HAYASHI Kentaro. +
+
+ +
+ + From mdounin at mdounin.ru Tue Sep 25 13:43:03 2012 From: mdounin at mdounin.ru (mdounin at mdounin.ru) Date: Tue, 25 Sep 2012 13:43:03 +0000 Subject: [nginx] svn commit: r4863 - in tags: . release-1.2.4 Message-ID: <20120925134303.775ED3F9F2F@mail.nginx.com> Author: mdounin Date: 2012-09-25 13:43:03 +0000 (Tue, 25 Sep 2012) New Revision: 4863 URL: http://trac.nginx.org/nginx/changeset/4863/nginx Log: release-1.2.4 tag Added: tags/release-1.2.4/ Index: tags/release-1.2.4 =================================================================== --- branches/stable-1.2 2012-09-25 13:42:43 UTC (rev 4862) +++ tags/release-1.2.4 2012-09-25 13:43:03 UTC (rev 4863) Property changes on: tags/release-1.2.4 ___________________________________________________________________ Added: svn:ignore ## -0,0 +1,14 ## +access.log +client_body_temp +fastcgi_temp +proxy_temp +scgi_temp +uwsgi_temp +GNUmakefile +Makefile +makefile +nginx +nginx.conf +nginx-*.tar.gz +objs* +tmp Added: svn:mergeinfo ## -0,0 +1 ## +/trunk:4611-4632,4636-4657,4671-4672,4674-4676,4682,4684-4699,4704-4706,4713,4736-4741,4754,4756-4771,4775,4777-4780,4782-4785,4795,4811-4820,4822-4824,4828-4835,4840-4844 \ No newline at end of property From thierry.magnien at sfr.com Wed Sep 26 10:07:02 2012 From: thierry.magnien at sfr.com (MAGNIEN, Thierry) Date: Wed, 26 Sep 2012 10:07:02 +0000 Subject: Dedicatng a worker to a single URL ? Message-ID: <5D103CE839D50E4CBC62C9FD7B83287C1D7007@EXCN015.encara.local.ads> Hi, I'm writing an nginx module that needs to perform a quite long task when receving a POST request on a defined URL. However, at the moment, the worker that took the request is blocked until the task finishes, blocking possibly several dozens of GET requests. I would like to avoid this, so is there a way to dedicate a worker for this URL only ? In such a case, GET requests would be treated by other workers, therefore not blocked when the POST request comes in. Or could I "spawn" a separate thread for the task and release the worker asap ? Thanks, Thierry From info at tvdw.eu Wed Sep 26 10:19:40 2012 From: info at tvdw.eu (Tom van der Woerdt) Date: Wed, 26 Sep 2012 12:19:40 +0200 Subject: Dedicatng a worker to a single URL ? In-Reply-To: <5D103CE839D50E4CBC62C9FD7B83287C1D7007@EXCN015.encara.local.ads> References: <5D103CE839D50E4CBC62C9FD7B83287C1D7007@EXCN015.encara.local.ads> Message-ID: <5062D6BC.8090905@tvdw.eu> Hi Thierry, What kind of task is performed? Is it in a nginx module or an external source? I usually use a simple uwsgi or fastcgi setup for these. Simply only spawn one worker, then make nginx proxy it there. Nginx shouldn't be handling the hard tasks itself, and proxying is non-blocking, so for me this solution has always been ideal. Tom Op 9/26/12 12:07 PM, MAGNIEN, Thierry schreef: > Hi, > > I'm writing an nginx module that needs to perform a quite long task when receving a POST request on a defined URL. However, at the moment, the worker that took the request is blocked until the task finishes, blocking possibly several dozens of GET requests. > > I would like to avoid this, so is there a way to dedicate a worker for this URL only ? In such a case, GET requests would be treated by other workers, therefore not blocked when the POST request comes in. > > Or could I "spawn" a separate thread for the task and release the worker asap ? > > Thanks, > Thierry > > _______________________________________________ > nginx-devel mailing list > nginx-devel at nginx.org > http://mailman.nginx.org/mailman/listinfo/nginx-devel -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 3729 bytes Desc: S/MIME-cryptografische ondertekening URL: From thierry.magnien at sfr.com Wed Sep 26 10:27:13 2012 From: thierry.magnien at sfr.com (MAGNIEN, Thierry) Date: Wed, 26 Sep 2012 10:27:13 +0000 Subject: Dedicatng a worker to a single URL ? In-Reply-To: <5062D6BC.8090905@tvdw.eu> References: <5D103CE839D50E4CBC62C9FD7B83287C1D7007@EXCN015.encara.local.ads> <5062D6BC.8090905@tvdw.eu> Message-ID: <5D103CE839D50E4CBC62C9FD7B83287C1D710C@EXCN015.encara.local.ads> Hi, The task is internal to the module: either updating memory with data from the POST request itself, or triggering memory update from an external source. But in both cases I need data to be in nginx shared memory, not in an external source to which I could proxy. Thierry -----Message d'origine----- De?: nginx-devel-bounces at nginx.org [mailto:nginx-devel-bounces at nginx.org] De la part de Tom van der Woerdt Envoy??: mercredi 26 septembre 2012 12:20 ??: nginx-devel at nginx.org Objet?: Re: Dedicatng a worker to a single URL ? Hi Thierry, What kind of task is performed? Is it in a nginx module or an external source? I usually use a simple uwsgi or fastcgi setup for these. Simply only spawn one worker, then make nginx proxy it there. Nginx shouldn't be handling the hard tasks itself, and proxying is non-blocking, so for me this solution has always been ideal. Tom Op 9/26/12 12:07 PM, MAGNIEN, Thierry schreef: > Hi, > > I'm writing an nginx module that needs to perform a quite long task when receving a POST request on a defined URL. However, at the moment, the worker that took the request is blocked until the task finishes, blocking possibly several dozens of GET requests. > > I would like to avoid this, so is there a way to dedicate a worker for this URL only ? In such a case, GET requests would be treated by other workers, therefore not blocked when the POST request comes in. > > Or could I "spawn" a separate thread for the task and release the worker asap ? > > Thanks, > Thierry > > _______________________________________________ > nginx-devel mailing list > nginx-devel at nginx.org > http://mailman.nginx.org/mailman/listinfo/nginx-devel From yaoweibin at gmail.com Wed Sep 26 13:48:18 2012 From: yaoweibin at gmail.com (=?GB2312?B?0qbOsLHz?=) Date: Wed, 26 Sep 2012 21:48:18 +0800 Subject: Update the Chinese documentation with the nginx.org website. In-Reply-To: <50607332.4020106@nginx.com> References: <50603FA6.6090500@nginx.com> <50607332.4020106@nginx.com> Message-ID: Hi Maxim, The attachment is the updated patch as your suggestion. After discussion with the translation, all the translators agreed to remove their copyright with the translated documentation, just keep the translator attribute. There is no need for the confirmation with the license. It's also easy for your team to publish these translated document. If there is any problem, let us know. Thanks. 2012/9/24 Maxim Konovalov : > Hi Weibin, > > On 9/24/12 6:38 PM, ??? wrote: >> Hi, Maxim >> >> Thanks for your quick reply. >> >> We are translating the English documentation to be Chinese. First we >> just imported all the English modules. It's easy for us to port the >> makefile and preview in the browser. Then we invited the volunteers >> the translate every module. This work doesn't finish yet. We are >> planning to translate all the modules this year. >> >> If you don't like this patch, we can give a more purely version. >> > Yeah, it'd be nice to get a patch we'll be able to commit to the > nginx.org tree. > >> Other modifications will be in the next patch. >> >> I'll ask the translators send you a confirmation email for their >> agreement with the nginx's BSD license, is that OK? >> > That's great. Thanks for your work again. > > -- > Maxim Konovalov > +7 (910) 4293178 > http://nginx.com/support.html > > _______________________________________________ > nginx-devel mailing list > nginx-devel at nginx.org > http://mailman.nginx.org/mailman/listinfo/nginx-devel -- Weibin Yao Developer @ Server Platform Team of Taobao -------------- next part -------------- A non-text attachment was scrubbed... Name: nginx_org_cn_20120914_2.diff Type: application/octet-stream Size: 104006 bytes Desc: not available URL: From maxim at nginx.com Wed Sep 26 14:16:27 2012 From: maxim at nginx.com (Maxim Konovalov) Date: Wed, 26 Sep 2012 18:16:27 +0400 Subject: Update the Chinese documentation with the nginx.org website. In-Reply-To: References: <50603FA6.6090500@nginx.com> <50607332.4020106@nginx.com> Message-ID: <50630E3B.1080503@nginx.com> Hi Weibin, On 9/26/12 5:48 PM, ??? wrote: > Hi Maxim, > > The attachment is the updated patch as your suggestion. > > After discussion with the translation, all the translators agreed to > remove their copyright with the translated documentation, just keep > the translator attribute. There is no need for the confirmation with > the license. It's also easy for your team to publish these translated > document. > > If there is any problem, let us know. > The patch looks good for me. I'll ask Ruslan to review and commit it. Thanks for your work! -- Maxim Konovalov +7 (910) 4293178 http://nginx.com/support.html From mdounin at mdounin.ru Wed Sep 26 15:52:06 2012 From: mdounin at mdounin.ru (mdounin at mdounin.ru) Date: Wed, 26 Sep 2012 15:52:06 +0000 Subject: [nginx] svn commit: r4864 - in trunk/src: core http/modules/perl Message-ID: <20120926155206.9B79C3F9C4E@mail.nginx.com> Author: mdounin Date: 2012-09-26 15:52:06 +0000 (Wed, 26 Sep 2012) New Revision: 4864 URL: http://trac.nginx.org/nginx/changeset/4864/nginx Log: Version bump. Modified: trunk/src/core/nginx.h trunk/src/http/modules/perl/nginx.pm Modified: trunk/src/core/nginx.h =================================================================== --- trunk/src/core/nginx.h 2012-09-25 13:43:03 UTC (rev 4863) +++ trunk/src/core/nginx.h 2012-09-26 15:52:06 UTC (rev 4864) @@ -9,8 +9,8 @@ #define _NGINX_H_INCLUDED_ -#define nginx_version 1003006 -#define NGINX_VERSION "1.3.6" +#define nginx_version 1003007 +#define NGINX_VERSION "1.3.7" #define NGINX_VER "nginx/" NGINX_VERSION #define NGINX_VAR "NGINX" Modified: trunk/src/http/modules/perl/nginx.pm =================================================================== --- trunk/src/http/modules/perl/nginx.pm 2012-09-25 13:43:03 UTC (rev 4863) +++ trunk/src/http/modules/perl/nginx.pm 2012-09-26 15:52:06 UTC (rev 4864) @@ -50,7 +50,7 @@ HTTP_INSUFFICIENT_STORAGE ); -our $VERSION = '1.3.6'; +our $VERSION = '1.3.7'; require XSLoader; XSLoader::load('nginx', $VERSION); From mdounin at mdounin.ru Wed Sep 26 16:25:12 2012 From: mdounin at mdounin.ru (mdounin at mdounin.ru) Date: Wed, 26 Sep 2012 16:25:12 +0000 Subject: [nginx] svn commit: r4865 - trunk/src/os/unix Message-ID: <20120926162512.A04AF3F9C44@mail.nginx.com> Author: mdounin Date: 2012-09-26 16:25:12 +0000 (Wed, 26 Sep 2012) New Revision: 4865 URL: http://trac.nginx.org/nginx/changeset/4865/nginx Log: Added clearing of cpu_affinity after process spawn. This fixes unwanted/incorrect cpu_affinity use on dead worker processes respawn. While this is not ideal, it's expected to be better when previous situation where multiple processes were spawn with identical CPU affinity set. Reported by Charles Chen. Modified: trunk/src/os/unix/ngx_process_cycle.c Modified: trunk/src/os/unix/ngx_process_cycle.c =================================================================== --- trunk/src/os/unix/ngx_process_cycle.c 2012-09-26 15:52:06 UTC (rev 4864) +++ trunk/src/os/unix/ngx_process_cycle.c 2012-09-26 16:25:12 UTC (rev 4865) @@ -371,6 +371,8 @@ ngx_pass_open_channel(cycle, &ch); } + + cpu_affinity = 0; } From mdounin at mdounin.ru Wed Sep 26 16:33:44 2012 From: mdounin at mdounin.ru (Maxim Dounin) Date: Wed, 26 Sep 2012 20:33:44 +0400 Subject: [PATCH] Update the help message of configure In-Reply-To: References: Message-ID: <20120926163343.GJ40452@mdounin.ru> Hello! On Fri, Sep 21, 2012 at 12:31:48PM +0800, Joshua Zhu wrote: > Hi, > > It seems the help message of configure was not updated when > ngx_http_upstream_least_conn_module and ngx_http_upstream_keepalive_module > were imported. > Below attaches a patch for this. Hope it helps :) > > $ svn diff > Index: auto/options > =================================================================== > --- auto/options (revision 4846) > +++ auto/options (working copy) > @@ -388,6 +388,10 @@ > --without-http_browser_module disable ngx_http_browser_module > --without-http_upstream_ip_hash_module > disable > ngx_http_upstream_ip_hash_module > + --without-http_upstream_least_conn_module > + disable > ngx_http_upstream_least_conn_module > + --without-http_upstream_keepalive_module > + disable > ngx_http_upstream_keepalive_module > > --with-http_perl_module enable ngx_http_perl_module > --with-perl_modules_path=PATH set Perl modules path Committed, thanks. -- Maxim Dounin http://nginx.com/support.html From mdounin at mdounin.ru Wed Sep 26 16:39:39 2012 From: mdounin at mdounin.ru (mdounin at mdounin.ru) Date: Wed, 26 Sep 2012 16:39:39 +0000 Subject: [nginx] svn commit: r4866 - trunk/auto Message-ID: <20120926163939.2A4BB3F9FDA@mail.nginx.com> Author: mdounin Date: 2012-09-26 16:39:38 +0000 (Wed, 26 Sep 2012) New Revision: 4866 URL: http://trac.nginx.org/nginx/changeset/4866/nginx Log: Configure: help updated to list upstream keepalive and least_conn. Patch by Joshua Zhu. Modified: trunk/auto/options Modified: trunk/auto/options =================================================================== --- trunk/auto/options 2012-09-26 16:25:12 UTC (rev 4865) +++ trunk/auto/options 2012-09-26 16:39:38 UTC (rev 4866) @@ -388,6 +388,10 @@ --without-http_browser_module disable ngx_http_browser_module --without-http_upstream_ip_hash_module disable ngx_http_upstream_ip_hash_module + --without-http_upstream_least_conn_module + disable ngx_http_upstream_least_conn_module + --without-http_upstream_keepalive_module + disable ngx_http_upstream_keepalive_module --with-http_perl_module enable ngx_http_perl_module --with-perl_modules_path=PATH set Perl modules path From indtiny at gmail.com Wed Sep 26 17:01:46 2012 From: indtiny at gmail.com (Indtiny s) Date: Wed, 26 Sep 2012 13:01:46 -0400 Subject: ngnix support for the android Message-ID: Hi, I want to cross build the nginx for the android using Android-ndk .. Is Nginx support for android ..? has any one tried to build nginx for android , if so pls suggest some docs to start on . Rgds Indu -------------- next part -------------- An HTML attachment was scrubbed... URL: From mk.fraggod at gmail.com Thu Sep 27 14:05:25 2012 From: mk.fraggod at gmail.com (Mike Kazantsev) Date: Thu, 27 Sep 2012 20:05:25 +0600 Subject: [PATCH] (re-post) Add "optional_no_ca" option to ssl_verify_client to enable app-only CA chain validation In-Reply-To: <20120925043928.GE40452@mdounin.ru> References: <20120918074355.GQ40452@mdounin.ru> <20120922170953.3f8f50b1@sacrilege> <20120925043928.GE40452@mdounin.ru> Message-ID: <20120927200525.64de9ef3@sacrilege> On Tue, 25 Sep 2012 08:39:28 +0400 Maxim Dounin wrote: > Hello! > > On Sat, Sep 22, 2012 at 05:09:53PM +0600, Mike Kazantsev wrote: > > [...] > > > > In particular, I would like someone to actually test if the > > > error_page 495 aproach works instead as suggested here: > > > > > > http://mailman.nginx.org/pipermail/nginx-devel/2012-August/002650.html > > > > > > > It doesn't seem to be of much use in current state, problems: > > > > - Requires "ssl_client_certificate" to be set to some valid > > certificate, which then shouldn't actually never be used in this > > case. > > This actually counts as a separate problem. I think it should be > possible to specify empty certificate authorities list to be sent > to clients even if nginx does actual verification. Part of the > problem is addressed by the ssl_trusted_certificate directive which is > going to be introduced along with OCSP stapling patches. > Ok. Don't think finding any cert to put there should be a problem anyway. > > - In case of "ssl_verify_client" set to "on" or "optional", setting > > "error_page 400 495 496 =200 /altpath;" doesn't seem to stop nginx > > from returning "HTTP/1.1 400 Bad Request" response with "400 The SSL > > certificate error" in html body. > > > > This code in ngx_http_request.c is probable cause of that: > > ... > > > > I'm still unsure if it's a general bug in ssl module or some error > > in my configuration above, because docs clearly state that error > > should be handled by error_page and it doesn't seem to be. > > Works fine here: > > ssl_verify_client on; > ssl_client_certificate ca.crt; > > error_page 495 = /foo; > > location = /foo { > return 200 "dn: $ssl_client_s_dn, verify: $ssl_client_verify\n"; > } > > $ openssl s_client -connect ... -cert ... -key ... > ... > GET / HTTP/1.0 > > HTTP/1.1 200 OK > Server: nginx/1.3.6 > Date: Tue, 25 Sep 2012 02:37:45 GMT > Content-Type: text/plain > Content-Length: 35 > Connection: close > > dn: /CN=mdounin.ru, verify: FAILED > With a bit more universal catch-all config like this: error_page 495 = @cert_err; location @cert_err { ... It seem to work pretty much as desired, was some misconfiguration on my part before, I guess. Since it's possible to use and rewrite $uri in named locations, I don't see any obvious flaws in this approach, so I think "error_page" might make this patch redundant indeed. I'll try it out (and ping other users) and re-raise the issue with modified patch if there'll be any less obvious issues with it. Eric, does it work for your use-case as well? > > - "ssl_verify_client off;" isn't much useful, because it doesn't return > > clent certificate and doesn't check it in any way. > > > > - "ssl_verify_client on;", still gives all-or-nothing check, though > > I see that it's what might indeed be desirable, as Eric indicated. > > > > - I think it's really non-obvious way to do it. > > > > ... > See below for some comments. I'll look into addressing these, thanks, though as noted, the patch itself doesn't seem to be necessary, since "error_page 495 ..." works. -- Mike Kazantsev // fraggod.net -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 198 bytes Desc: not available URL: From defan at nginx.com Thu Sep 27 15:01:58 2012 From: defan at nginx.com (defan at nginx.com) Date: Thu, 27 Sep 2012 15:01:58 +0000 Subject: [nginx] svn commit: r4867 - trunk/auto/lib/perl Message-ID: <20120927150158.369B53F9FC0@mail.nginx.com> Author: defan Date: 2012-09-27 15:01:57 +0000 (Thu, 27 Sep 2012) New Revision: 4867 URL: http://trac.nginx.org/nginx/changeset/4867/nginx Log: Configure: additional test for ExtUtils::Embed perl module presence. Now perl configure will correctly fail if ExtUtils::Embed perl module is not present in the system (found on Amazon Linux AMI, as of release 2012.03). Modified: trunk/auto/lib/perl/conf Modified: trunk/auto/lib/perl/conf =================================================================== --- trunk/auto/lib/perl/conf 2012-09-26 16:39:38 UTC (rev 4866) +++ trunk/auto/lib/perl/conf 2012-09-27 15:01:57 UTC (rev 4867) @@ -12,7 +12,7 @@ if test -n "$NGX_PERL_VER"; then echo " + perl version: $NGX_PERL_VER" - if [ "`echo 'use 5.006001; print "OK"' | $NGX_PERL 2>&1`" != OK ]; then + if [ "`$NGX_PERL -e 'use 5.006001; print "OK"'`" != "OK" ]; then echo echo "$0: error: perl 5.6.1 or higher is required" echo @@ -20,6 +20,14 @@ exit 1; fi + if [ "`$NGX_PERL -MExtUtils::Embed -e 'print "OK"'`" != "OK" ]; then + echo + echo "$0: error: perl module ExtUtils::Embed is required" + echo + + exit 1; + fi + NGX_PERL_CFLAGS="$CFLAGS `$NGX_PERL -MExtUtils::Embed -e ccopts`" NGX_PM_CFLAGS=`$NGX_PERL -MExtUtils::Embed -e ccopts` From mailtogroe at gmail.com Thu Sep 27 15:30:10 2012 From: mailtogroe at gmail.com (=?ISO-8859-1?Q?Benjamin_Gr=F6ssing?=) Date: Thu, 27 Sep 2012 17:30:10 +0200 Subject: Can not use $bytes_sent in module (complex values) In-Reply-To: <1348215438.687512098@f258.mail.ru> References: <20120919000656.GA40452@mdounin.ru> <1348215438.687512098@f258.mail.ru> Message-ID: Hello, thanks for your further explanations. I think I understand the concepts of these variables and how they are calculated now. However, I am not sure how calling nxg_http_add_variable and implementing that specific getter would affect performance for modules (other than the log module) using it. I have also had a look at the changeset 4686, adding the $status variable. Thats exactly what I had in mind! Since $body_bytes_sent already exists (it actually just subtracts the header length from the total length!), adding $bytes_sent is actually quite easy (or did I miss something?). I have attached a patch (based on nginx 1.3.6) that adds $bytes_sent to ngx_http_variables. It works great for me - I'd be happy to see it in a future nginx release. Let me know if I can help any further! Regards Benjamin 2012/9/21 Vladimir Shebordaev : > Benjamin, > > it seems I used not quite neat wording, so my earlier explanations might > look obfuscating. > > Basically, the indexed variable is any "automatic" variable referenced thow > the request->variables array. The variables of the current log module do use > index slots, so they always have space for the value preallocated in > ngx_http_init_request(). But those variables can be referenced from a script > or by another module neither by name since they are missing from > cmcf->variables_hash, nor by index, e.g. by a complex value, since they > have no getter method thus are rejected in ngx_http_variables_init_vars() as > unknown. > > If you modify the log module to register its variables with > nxg_http_add_variable() and implement correct get handlers, they would > surely be available everywhere like any other variable. For performance > reasons the log module in itself should still use its variables as indexed > ones, since log operations are expensive and rather frequent. If you > implement the generic getter method for the http variable infrastructure, > you might immediatly use it in the log module handler probably using their > own machinery. > > Hope it helps. > > Regards, > Vladimir > > > > > Wed, 19 Sep 2012 23:11:57 +0200 ?? Benjamin Gr?ssing : > > Hi Vladimir, hi Maxim! > > > Thanks for your answers. I've had a look at the rewrite module - > unfortunately it does not support the $bytes_sent variable either as > configuring "if ($bytes_sent) { }" throws the error: [emerg] unknown > "bytes_sent" variable. > > Nice to hear that there is a plan to include all variables in the > complex value system - I think that would make that part a of nginx a > lot more clear! I'd be happy if I could contribute here, but I am not > sure about the details. Is there any guide to contributing to nginx > available or do I just have to create a patch file and post it here? > Can I just start with the nginx 1.2.3 source code from nginx.org? > > Regards > Benjamin > > 2012/9/19 Maxim Dounin : >> Hello! >> >> On Tue, Sep 18, 2012 at 03:55:38PM +0200, Benjamin Gr?ssing wrote: >> >>> Hello, >>> >>> I am currently working on a nginx module, that parses user defined >>> strings for variables using ngx_http_complex_value_t. That works great >>> with all variables defined in ngx_http_variables.c (i.e. >>> $body_bytes_sent). However, variables as $bytes_sent do not exist here >>> since ngx_http_log_module uses its own variable parsing system. What >>> is the best way to use these extra variables in my module? I have >>> thought of the following options, I am not very sure about what is the >>> best way to go: >>> a) Copy the ngx_http_log_module code in my own module and rename every >>> single type and function to avoid double definitions. Reuse functions >>> and types from there. >>> b) Move the ngx_http_log_module code in my own module directory and >>> compile it only there. Reuse functions and types from there. >>> c) Write own variable parsing system, reuse a few snippets from >>> ngx_http_variables and ngx_http_log_module, whatever fits best. >>> d) Use complex values and try to extend the ngx_http_core_variables >>> array to include $body_bytes_sent and the other missing variables (I >>> don't understand why they are not included here anyway?). >>> e) Any better way to achieve that? >>> >>> I'd really appreciate your help. >> >> All variables that are currently present only in log module should >> really be made available as normal variables as well. They aren't >> yet mostly due to historical reasons (log module appeared before >> normal variables were implemented) and the fact that most of >> them are only usable for logging. >> >> No idea what would be best for your particular module, but in >> general it's better to use complex values and what's available via >> normal variables. Unless you need $bytes_sent right now - it's >> probably the best aproach, and missing variables will be available >> once they appear in nginx. >> >> (The latter is something you may speedup by submitting a good >> patch. Most tricky part here would be to follow coding style.) >> >> Maxim Dounin >> >> _______________________________________________ >> nginx-devel mailing list >> nginx-devel at nginx.org >> http://mailman.nginx.org/mailman/listinfo/nginx-devel > > _______________________________________________ > nginx-devel mailing list > nginx-devel at nginx.org > http://mailman.nginx.org/mailman/listinfo/nginx-devel > > > > _______________________________________________ > nginx-devel mailing list > nginx-devel at nginx.org > http://mailman.nginx.org/mailman/listinfo/nginx-devel -------------- next part -------------- A non-text attachment was scrubbed... Name: add_bytes_sent_variable.patch Type: application/octet-stream Size: 1664 bytes Desc: not available URL: From eoconnor at coincident.com Thu Sep 27 16:10:37 2012 From: eoconnor at coincident.com (Eric O'Connor) Date: Thu, 27 Sep 2012 12:10:37 -0400 Subject: [PATCH] (re-post) Add "optional_no_ca" option to ssl_verify_client to enable app-only CA chain validation In-Reply-To: <20120927200525.64de9ef3@sacrilege> References: <20120918074355.GQ40452@mdounin.ru> <20120922170953.3f8f50b1@sacrilege> <20120925043928.GE40452@mdounin.ru> <20120927200525.64de9ef3@sacrilege> Message-ID: It appears to be technically correct, but not semantically correct. I would describe it as a nasty kludge. I'm not sure why that would be preferable to a small ~3 line + a macro patch that solves the issue and provides a nice interface. Error page handling should be used for handling errors, not supporting workarounds for a new set of technologies. On Thu, Sep 27, 2012 at 10:05 AM, Mike Kazantsev wrote: > On Tue, 25 Sep 2012 08:39:28 +0400 > Maxim Dounin wrote: > >> Hello! >> >> On Sat, Sep 22, 2012 at 05:09:53PM +0600, Mike Kazantsev wrote: >> >> [...] >> >> > > In particular, I would like someone to actually test if the >> > > error_page 495 aproach works instead as suggested here: >> > > >> > > http://mailman.nginx.org/pipermail/nginx-devel/2012-August/002650.html >> > > >> > >> > It doesn't seem to be of much use in current state, problems: >> > >> > - Requires "ssl_client_certificate" to be set to some valid >> > certificate, which then shouldn't actually never be used in this >> > case. >> >> This actually counts as a separate problem. I think it should be >> possible to specify empty certificate authorities list to be sent >> to clients even if nginx does actual verification. Part of the >> problem is addressed by the ssl_trusted_certificate directive which is >> going to be introduced along with OCSP stapling patches. >> > > Ok. > > Don't think finding any cert to put there should be a problem anyway. > > >> > - In case of "ssl_verify_client" set to "on" or "optional", setting >> > "error_page 400 495 496 =200 /altpath;" doesn't seem to stop nginx >> > from returning "HTTP/1.1 400 Bad Request" response with "400 The SSL >> > certificate error" in html body. >> > >> > This code in ngx_http_request.c is probable cause of that: >> > > > ... > >> > >> > I'm still unsure if it's a general bug in ssl module or some error >> > in my configuration above, because docs clearly state that error >> > should be handled by error_page and it doesn't seem to be. >> >> Works fine here: >> >> ssl_verify_client on; >> ssl_client_certificate ca.crt; >> >> error_page 495 = /foo; >> >> location = /foo { >> return 200 "dn: $ssl_client_s_dn, verify: $ssl_client_verify\n"; >> } >> >> $ openssl s_client -connect ... -cert ... -key ... >> ... >> GET / HTTP/1.0 >> >> HTTP/1.1 200 OK >> Server: nginx/1.3.6 >> Date: Tue, 25 Sep 2012 02:37:45 GMT >> Content-Type: text/plain >> Content-Length: 35 >> Connection: close >> >> dn: /CN=mdounin.ru, verify: FAILED >> > > With a bit more universal catch-all config like this: > > error_page 495 = @cert_err; > location @cert_err { ... > > It seem to work pretty much as desired, was some misconfiguration on my > part before, I guess. > > Since it's possible to use and rewrite $uri in named locations, I don't > see any obvious flaws in this approach, so I think "error_page" might > make this patch redundant indeed. > > I'll try it out (and ping other users) and re-raise the issue with > modified patch if there'll be any less obvious issues with it. > > Eric, does it work for your use-case as well? > > >> > - "ssl_verify_client off;" isn't much useful, because it doesn't return >> > clent certificate and doesn't check it in any way. >> > >> > - "ssl_verify_client on;", still gives all-or-nothing check, though >> > I see that it's what might indeed be desirable, as Eric indicated. >> > >> > - I think it's really non-obvious way to do it. >> > >> > > > ... > >> See below for some comments. > > I'll look into addressing these, thanks, though as noted, the patch > itself doesn't seem to be necessary, since "error_page 495 ..." works. > > > -- > Mike Kazantsev // fraggod.net From eoconnor at coincident.com Thu Sep 27 16:30:29 2012 From: eoconnor at coincident.com (Eric O'Connor) Date: Thu, 27 Sep 2012 12:30:29 -0400 Subject: [PATCH] (re-post) Add "optional_no_ca" option to ssl_verify_client to enable app-only CA chain validation In-Reply-To: References: <20120918074355.GQ40452@mdounin.ru> <20120922170953.3f8f50b1@sacrilege> <20120925043928.GE40452@mdounin.ru> <20120927200525.64de9ef3@sacrilege> Message-ID: Here is a modified patch addressing issues that Maxim brought up earlier: diff --git a/src/event/ngx_event_openssl.h b/src/event/ngx_event_openssl.h index cd6d885..97da051 100644 --- a/src/event/ngx_event_openssl.h +++ b/src/event/ngx_event_openssl.h @@ -141,6 +141,14 @@ ngx_int_t ngx_ssl_get_client_verify(ngx_connection_t *c, ngx_pool_t *pool, ngx_str_t *s); +#define ngx_ssl_verify_error_is_optional(errnum) \ + ((errnum == X509_V_ERR_DEPTH_ZERO_SELF_SIGNED_CERT) || \ + (errnum == X509_V_ERR_SELF_SIGNED_CERT_IN_CHAIN) || \ + (errnum == X509_V_ERR_UNABLE_TO_GET_ISSUER_CERT_LOCALLY) || \ + (errnum == X509_V_ERR_CERT_UNTRUSTED) || \ + (errnum == X509_V_ERR_UNABLE_TO_VERIFY_LEAF_SIGNATURE)) + + ngx_int_t ngx_ssl_handshake(ngx_connection_t *c); ssize_t ngx_ssl_recv(ngx_connection_t *c, u_char *buf, size_t size); ssize_t ngx_ssl_write(ngx_connection_t *c, u_char *data, size_t size); diff --git a/src/http/modules/ngx_http_ssl_module.c b/src/http/modules/ngx_http_ssl_module.c index d759489..ab91670 100644 --- a/src/http/modules/ngx_http_ssl_module.c +++ b/src/http/modules/ngx_http_ssl_module.c @@ -48,6 +48,7 @@ static ngx_conf_enum_t ngx_http_ssl_verify[] = { { ngx_string("off"), 0 }, { ngx_string("on"), 1 }, { ngx_string("optional"), 2 }, + { ngx_string("optional_no_ca"), 3 }, { ngx_null_string, 0 } }; @@ -466,7 +467,7 @@ ngx_http_ssl_merge_srv_conf(ngx_conf_t *cf, void *parent, void *child) if (conf->verify) { - if (conf->client_certificate.len == 0) { + if (conf->client_certificate.len == 0 && conf->verify != 3) { ngx_log_error(NGX_LOG_EMERG, cf->log, 0, "no ssl_client_certificate for ssl_client_verify"); return NGX_CONF_ERROR; diff --git a/src/http/ngx_http_request.c b/src/http/ngx_http_request.c index cb970c5..96cec55 100644 --- a/src/http/ngx_http_request.c +++ b/src/http/ngx_http_request.c @@ -1642,7 +1642,9 @@ ngx_http_process_request(ngx_http_request_t *r) if (sscf->verify) { rc = SSL_get_verify_result(c->ssl->connection); - if (rc != X509_V_OK) { + if (rc != X509_V_OK + && (sscf->verify != 3 || !ngx_ssl_verify_error_is_optional(rc))) + { ngx_log_error(NGX_LOG_INFO, c->log, 0, "client SSL certificate verify error: (%l:%s)", rc, X509_verify_cert_error_string(rc)); On Thu, Sep 27, 2012 at 12:10 PM, Eric O'Connor wrote: > It appears to be technically correct, but not semantically correct. I > would describe it as a nasty kludge. > > I'm not sure why that would be preferable to a small ~3 line + a macro > patch that solves the issue and provides a nice interface. Error page > handling should be used for handling errors, not supporting > workarounds for a new set of technologies. > > On Thu, Sep 27, 2012 at 10:05 AM, Mike Kazantsev wrote: >> On Tue, 25 Sep 2012 08:39:28 +0400 >> Maxim Dounin wrote: >> >>> Hello! >>> >>> On Sat, Sep 22, 2012 at 05:09:53PM +0600, Mike Kazantsev wrote: >>> >>> [...] >>> >>> > > In particular, I would like someone to actually test if the >>> > > error_page 495 aproach works instead as suggested here: >>> > > >>> > > http://mailman.nginx.org/pipermail/nginx-devel/2012-August/002650.html >>> > > >>> > >>> > It doesn't seem to be of much use in current state, problems: >>> > >>> > - Requires "ssl_client_certificate" to be set to some valid >>> > certificate, which then shouldn't actually never be used in this >>> > case. >>> >>> This actually counts as a separate problem. I think it should be >>> possible to specify empty certificate authorities list to be sent >>> to clients even if nginx does actual verification. Part of the >>> problem is addressed by the ssl_trusted_certificate directive which is >>> going to be introduced along with OCSP stapling patches. >>> >> >> Ok. >> >> Don't think finding any cert to put there should be a problem anyway. >> >> >>> > - In case of "ssl_verify_client" set to "on" or "optional", setting >>> > "error_page 400 495 496 =200 /altpath;" doesn't seem to stop nginx >>> > from returning "HTTP/1.1 400 Bad Request" response with "400 The SSL >>> > certificate error" in html body. >>> > >>> > This code in ngx_http_request.c is probable cause of that: >>> > >> >> ... >> >>> > >>> > I'm still unsure if it's a general bug in ssl module or some error >>> > in my configuration above, because docs clearly state that error >>> > should be handled by error_page and it doesn't seem to be. >>> >>> Works fine here: >>> >>> ssl_verify_client on; >>> ssl_client_certificate ca.crt; >>> >>> error_page 495 = /foo; >>> >>> location = /foo { >>> return 200 "dn: $ssl_client_s_dn, verify: $ssl_client_verify\n"; >>> } >>> >>> $ openssl s_client -connect ... -cert ... -key ... >>> ... >>> GET / HTTP/1.0 >>> >>> HTTP/1.1 200 OK >>> Server: nginx/1.3.6 >>> Date: Tue, 25 Sep 2012 02:37:45 GMT >>> Content-Type: text/plain >>> Content-Length: 35 >>> Connection: close >>> >>> dn: /CN=mdounin.ru, verify: FAILED >>> >> >> With a bit more universal catch-all config like this: >> >> error_page 495 = @cert_err; >> location @cert_err { ... >> >> It seem to work pretty much as desired, was some misconfiguration on my >> part before, I guess. >> >> Since it's possible to use and rewrite $uri in named locations, I don't >> see any obvious flaws in this approach, so I think "error_page" might >> make this patch redundant indeed. >> >> I'll try it out (and ping other users) and re-raise the issue with >> modified patch if there'll be any less obvious issues with it. >> >> Eric, does it work for your use-case as well? >> >> >>> > - "ssl_verify_client off;" isn't much useful, because it doesn't return >>> > clent certificate and doesn't check it in any way. >>> > >>> > - "ssl_verify_client on;", still gives all-or-nothing check, though >>> > I see that it's what might indeed be desirable, as Eric indicated. >>> > >>> > - I think it's really non-obvious way to do it. >>> > >>> > >> >> ... >> >>> See below for some comments. >> >> I'll look into addressing these, thanks, though as noted, the patch >> itself doesn't seem to be necessary, since "error_page 495 ..." works. >> >> >> -- >> Mike Kazantsev // fraggod.net From mdounin at mdounin.ru Thu Sep 27 17:59:59 2012 From: mdounin at mdounin.ru (mdounin at mdounin.ru) Date: Thu, 27 Sep 2012 17:59:59 +0000 Subject: [nginx] svn commit: r4868 - trunk/src/event Message-ID: <20120927180000.147963F9C46@mail.nginx.com> Author: mdounin Date: 2012-09-27 17:59:59 +0000 (Thu, 27 Sep 2012) New Revision: 4868 URL: http://trac.nginx.org/nginx/changeset/4868/nginx Log: SSL: fixed compression workaround to remove all methods. Previous code used sk_SSL_COMP_delete(ssl_comp_methods, i) while iterating stack from 0 to n, resulting in removal of only even compression methods. In real life this change is a nop, as there is only one compression method which is enabled by default in OpenSSL. Modified: trunk/src/event/ngx_event_openssl.c Modified: trunk/src/event/ngx_event_openssl.c =================================================================== --- trunk/src/event/ngx_event_openssl.c 2012-09-27 15:01:57 UTC (rev 4867) +++ trunk/src/event/ngx_event_openssl.c 2012-09-27 17:59:59 UTC (rev 4868) @@ -100,14 +100,14 @@ * Disable gzip compression in OpenSSL prior to 1.0.0 version, * this saves about 522K per connection. */ - int i, n; + int n; STACK_OF(SSL_COMP) *ssl_comp_methods; ssl_comp_methods = SSL_COMP_get_compression_methods(); n = sk_SSL_COMP_num(ssl_comp_methods); - for (i = 0; i < n; i++) { - (void) sk_SSL_COMP_delete(ssl_comp_methods, i); + while (n--) { + (void) sk_SSL_COMP_pop(ssl_comp_methods); } } #endif From mdounin at mdounin.ru Thu Sep 27 18:01:07 2012 From: mdounin at mdounin.ru (mdounin at mdounin.ru) Date: Thu, 27 Sep 2012 18:01:07 +0000 Subject: [nginx] svn commit: r4869 - trunk/src/event Message-ID: <20120927180107.8825F3F9F8D@mail.nginx.com> Author: mdounin Date: 2012-09-27 18:01:06 +0000 (Thu, 27 Sep 2012) New Revision: 4869 URL: http://trac.nginx.org/nginx/changeset/4869/nginx Log: SSL: added version checks for ssl compression workaround. The SSL_COMP_get_compression_methods() is only available as an API function in OpenSSL 0.9.8+, require it explicitly to unbreak build with OpenSSL 0.9.7. Modified: trunk/src/event/ngx_event_openssl.c Modified: trunk/src/event/ngx_event_openssl.c =================================================================== --- trunk/src/event/ngx_event_openssl.c 2012-09-27 17:59:59 UTC (rev 4868) +++ trunk/src/event/ngx_event_openssl.c 2012-09-27 18:01:06 UTC (rev 4869) @@ -94,6 +94,7 @@ OpenSSL_add_all_algorithms(); +#if OPENSSL_VERSION_NUMBER >= 0x0090800fL #ifndef SSL_OP_NO_COMPRESSION { /* @@ -111,6 +112,7 @@ } } #endif +#endif ngx_ssl_connection_index = SSL_get_ex_new_index(0, NULL, NULL, NULL, NULL); From agentzh at gmail.com Thu Sep 27 19:25:22 2012 From: agentzh at gmail.com (agentzh) Date: Thu, 27 Sep 2012 12:25:22 -0700 Subject: [ANN] Test::Nginx 0.21 released In-Reply-To: References: Message-ID: Hello, folks! I've just uploaded Test::Nginx 0.21 to CPAN, which features embeded TCP/UDP mocking server support and SystemTap integration for white-box testing: http://search.cpan.org/perldoc?Test::Nginx It will appear on the CPAN mirror near you in the next few hours or so. Special thanks go to all our contributors and users :) Here's the complete change log for this release, as compared to the last CPAN release, 0.20, released three months ago: * feature: added support for SystemTap: the SystemTap white-box testing mode is enabled by setting TEST_NGINX_USE_STAP=1 and by using new test sections "--- stap", "--- stap_out", and "--- stap_out_like". * feature: added support for new sections "--- udp_listen" and "--- udp_reply" for mocking an internal UDP server. * feature: added new section "--- udp_reply_delay" and also make "--- udp_reply_delay" and "--- timeout" sections accept time values with a time unit which can be either "s" or "ms". * feature: added new section "--- udp_query" to test if the UDP query sent to the embeded UDP server is expected. * feature: added new section "--- tcp_query_len" to work with mockeagain testing modes. * feature: added support for embedded TCP server by introducing new sections "--- tcp_listen", "--- tcp_query", "--- tcp_reply", and "--- tcp_reply_delay". * feature: added new section "--- tcp_no_close" to tell the embedded TCP server (if any) never close the connection actively. this is feature is mainly for mocking upstream reply read time out in the middle. * feature: added new section "--- log_level" to specify the error log level on a test-by-test basis. * bugfix: we might lose logs at config-time when TEST_NGINX_USE_HUP=1 because we sent HUP before USR1. * bugfix: be more tolarent when removing the nginx pid file while the nginx process is not running. * optimize: now we can automatically check the version of the current nginx configuration and retry dozens of times when TEST_NGINX_USE_HUP=1. this speed up the test suite in this mode a lot and also avoid accidental failures due to slow nginx reloads. * bugfix: try harder when failing to connect to the Nginx server. * refactor: now we use the standard "return" directive instead of "echo" for detecting nginx config reload completion for the "HUP reload" testing mode. * refactor: use waitpid() wherever possible. * bugfix: Parallel::ForkManager does not allow the user calling waitpid directly, or inifinite looping might happen when calling its start() method. now we no longer use this module any more and use the fork() builtin function exclusively. * bugfix: disabled the ending output "Looks like you ..." when the client socket timed out. * refactor: now we disabled the access log by default. * updated the bundled versions of Test::Builder and Test::More modules. * docs: documented the "--- skip_nginx" and "--- skip_nginx2" sections. thanks Weibin Yao for requesting it. This Perl module provides a test scaffold based on IO::Socket or LWP for automated testing in Nginx C module or ngx_lua-based Lua library development. This class inherits from Test::Base, thus bringing all its declarative power to the Nginx C module testing practices. Please check out the full documentation on CPAN: http://search.cpan.org/perldoc?Test::Nginx::Socket All of our Nginx modules (as well as our lua-resty-* libraries) are using Test::Nginx to drive their test suites. Please note that this module is completely different from the Test::Nginx module created by Maxim Dounin. The git repository for this Perl module is hosted on GitHub: https://github.com/agentzh/test-nginx Enjoy! -agentzh From defan at nginx.com Fri Sep 28 13:15:12 2012 From: defan at nginx.com (defan at nginx.com) Date: Fri, 28 Sep 2012 13:15:12 +0000 Subject: [nginx] svn commit: r4870 - trunk/src/core Message-ID: <20120928131512.6063B3F9C0F@mail.nginx.com> Author: defan Date: 2012-09-28 13:15:11 +0000 (Fri, 28 Sep 2012) New Revision: 4870 URL: http://trac.nginx.org/nginx/changeset/4870/nginx Log: Made sure to initialize the entire ngx_file_t structure. Found by Coverity. Modified: trunk/src/core/ngx_cycle.c Modified: trunk/src/core/ngx_cycle.c =================================================================== --- trunk/src/core/ngx_cycle.c 2012-09-27 18:01:06 UTC (rev 4869) +++ trunk/src/core/ngx_cycle.c 2012-09-28 13:15:11 UTC (rev 4870) @@ -1038,6 +1038,8 @@ ccf = (ngx_core_conf_t *) ngx_get_conf(cycle->conf_ctx, ngx_core_module); + ngx_memzero(&file, sizeof(ngx_file_t)); + file.name = ccf->pid; file.log = cycle->log; From defan at nginx.com Fri Sep 28 13:49:27 2012 From: defan at nginx.com (defan at nginx.com) Date: Fri, 28 Sep 2012 13:49:27 +0000 Subject: [nginx] svn commit: r4871 - in trunk: auto src/core src/os/unix src/os/win32 Message-ID: <20120928134927.E0D3F3F9C10@mail.nginx.com> Author: defan Date: 2012-09-28 13:49:26 +0000 (Fri, 28 Sep 2012) New Revision: 4871 URL: http://trac.nginx.org/nginx/changeset/4871/nginx Log: Correct plural form for "path" in the whole source base. Modified: trunk/auto/make trunk/src/core/ngx_cycle.c trunk/src/core/ngx_cycle.h trunk/src/core/ngx_file.c trunk/src/core/ngx_file.h trunk/src/os/unix/ngx_process_cycle.c trunk/src/os/win32/ngx_process_cycle.c Modified: trunk/auto/make =================================================================== --- trunk/auto/make 2012-09-28 13:15:11 UTC (rev 4870) +++ trunk/auto/make 2012-09-28 13:49:26 UTC (rev 4871) @@ -49,7 +49,7 @@ ngx_all_srcs="$CORE_SRCS" -# the core dependences and include pathes +# the core dependences and include paths ngx_deps=`echo $CORE_DEPS $NGX_AUTO_CONFIG_H $NGX_PCH \ | sed -e "s/ *\([^ ][^ ]*\)/$ngx_regex_cont\1/g" \ @@ -69,7 +69,7 @@ END -# the http dependences and include pathes +# the http dependences and include paths if [ $HTTP = YES ]; then @@ -95,7 +95,7 @@ fi -# the mail dependences and include pathes +# the mail dependences and include paths if [ $MAIL = YES ]; then Modified: trunk/src/core/ngx_cycle.c =================================================================== --- trunk/src/core/ngx_cycle.c 2012-09-28 13:15:11 UTC (rev 4870) +++ trunk/src/core/ngx_cycle.c 2012-09-28 13:49:26 UTC (rev 4871) @@ -118,18 +118,18 @@ } - n = old_cycle->pathes.nelts ? old_cycle->pathes.nelts : 10; + n = old_cycle->paths.nelts ? old_cycle->paths.nelts : 10; - cycle->pathes.elts = ngx_pcalloc(pool, n * sizeof(ngx_path_t *)); - if (cycle->pathes.elts == NULL) { + cycle->paths.elts = ngx_pcalloc(pool, n * sizeof(ngx_path_t *)); + if (cycle->paths.elts == NULL) { ngx_destroy_pool(pool); return NULL; } - cycle->pathes.nelts = 0; - cycle->pathes.size = sizeof(ngx_path_t *); - cycle->pathes.nalloc = n; - cycle->pathes.pool = pool; + cycle->paths.nelts = 0; + cycle->paths.size = sizeof(ngx_path_t *); + cycle->paths.nalloc = n; + cycle->paths.pool = pool; if (old_cycle->open_files.part.nelts) { @@ -334,7 +334,7 @@ } - if (ngx_create_pathes(cycle, ccf->user) != NGX_OK) { + if (ngx_create_paths(cycle, ccf->user) != NGX_OK) { goto failed; } Modified: trunk/src/core/ngx_cycle.h =================================================================== --- trunk/src/core/ngx_cycle.h 2012-09-28 13:15:11 UTC (rev 4870) +++ trunk/src/core/ngx_cycle.h 2012-09-28 13:49:26 UTC (rev 4871) @@ -48,7 +48,7 @@ ngx_queue_t reusable_connections_queue; ngx_array_t listening; - ngx_array_t pathes; + ngx_array_t paths; ngx_list_t open_files; ngx_list_t shared_memory; Modified: trunk/src/core/ngx_file.c =================================================================== --- trunk/src/core/ngx_file.c 2012-09-28 13:15:11 UTC (rev 4870) +++ trunk/src/core/ngx_file.c 2012-09-28 13:49:26 UTC (rev 4871) @@ -412,8 +412,8 @@ path = *slot; - p = cf->cycle->pathes.elts; - for (i = 0; i < cf->cycle->pathes.nelts; i++) { + p = cf->cycle->paths.elts; + for (i = 0; i < cf->cycle->paths.nelts; i++) { if (p[i]->name.len == path->name.len && ngx_strcmp(p[i]->name.data, path->name.data) == 0) { @@ -457,7 +457,7 @@ } } - p = ngx_array_push(&cf->cycle->pathes); + p = ngx_array_push(&cf->cycle->paths); if (p == NULL) { return NGX_ERROR; } @@ -469,14 +469,14 @@ ngx_int_t -ngx_create_pathes(ngx_cycle_t *cycle, ngx_uid_t user) +ngx_create_paths(ngx_cycle_t *cycle, ngx_uid_t user) { ngx_err_t err; ngx_uint_t i; ngx_path_t **path; - path = cycle->pathes.elts; - for (i = 0; i < cycle->pathes.nelts; i++) { + path = cycle->paths.elts; + for (i = 0; i < cycle->paths.nelts; i++) { if (ngx_create_dir(path[i]->name.data, 0700) == NGX_FILE_ERROR) { err = ngx_errno; Modified: trunk/src/core/ngx_file.h =================================================================== --- trunk/src/core/ngx_file.h 2012-09-28 13:15:11 UTC (rev 4870) +++ trunk/src/core/ngx_file.h 2012-09-28 13:49:26 UTC (rev 4871) @@ -130,7 +130,7 @@ ngx_int_t ngx_create_path(ngx_file_t *file, ngx_path_t *path); ngx_err_t ngx_create_full_path(u_char *dir, ngx_uint_t access); ngx_int_t ngx_add_path(ngx_conf_t *cf, ngx_path_t **slot); -ngx_int_t ngx_create_pathes(ngx_cycle_t *cycle, ngx_uid_t user); +ngx_int_t ngx_create_paths(ngx_cycle_t *cycle, ngx_uid_t user); ngx_int_t ngx_ext_rename_file(ngx_str_t *src, ngx_str_t *to, ngx_ext_rename_file_t *ext); ngx_int_t ngx_copy_file(u_char *from, u_char *to, ngx_copy_file_t *cf); Modified: trunk/src/os/unix/ngx_process_cycle.c =================================================================== --- trunk/src/os/unix/ngx_process_cycle.c 2012-09-28 13:15:11 UTC (rev 4870) +++ trunk/src/os/unix/ngx_process_cycle.c 2012-09-28 13:49:26 UTC (rev 4871) @@ -386,8 +386,8 @@ manager = 0; loader = 0; - path = ngx_cycle->pathes.elts; - for (i = 0; i < ngx_cycle->pathes.nelts; i++) { + path = ngx_cycle->paths.elts; + for (i = 0; i < ngx_cycle->paths.nelts; i++) { if (path[i]->manager) { manager = 1; @@ -1341,8 +1341,8 @@ next = 60 * 60; - path = ngx_cycle->pathes.elts; - for (i = 0; i < ngx_cycle->pathes.nelts; i++) { + path = ngx_cycle->paths.elts; + for (i = 0; i < ngx_cycle->paths.nelts; i++) { if (path[i]->manager) { n = path[i]->manager(path[i]->data); @@ -1370,8 +1370,8 @@ cycle = (ngx_cycle_t *) ngx_cycle; - path = cycle->pathes.elts; - for (i = 0; i < cycle->pathes.nelts; i++) { + path = cycle->paths.elts; + for (i = 0; i < cycle->paths.nelts; i++) { if (ngx_terminate || ngx_quit) { break; Modified: trunk/src/os/win32/ngx_process_cycle.c =================================================================== --- trunk/src/os/win32/ngx_process_cycle.c 2012-09-28 13:15:11 UTC (rev 4870) +++ trunk/src/os/win32/ngx_process_cycle.c 2012-09-28 13:49:26 UTC (rev 4871) @@ -963,8 +963,8 @@ next = 60 * 60; - path = ngx_cycle->pathes.elts; - for (i = 0; i < ngx_cycle->pathes.nelts; i++) { + path = ngx_cycle->paths.elts; + for (i = 0; i < ngx_cycle->paths.nelts; i++) { if (path[i]->manager) { n = path[i]->manager(path[i]->data); @@ -1002,8 +1002,8 @@ cycle = (ngx_cycle_t *) ngx_cycle; - path = cycle->pathes.elts; - for (i = 0; i < cycle->pathes.nelts; i++) { + path = cycle->paths.elts; + for (i = 0; i < cycle->paths.nelts; i++) { if (ngx_terminate || ngx_quit) { break; From vl at nginx.com Fri Sep 28 13:53:28 2012 From: vl at nginx.com (Homutov Vladimir) Date: Fri, 28 Sep 2012 17:53:28 +0400 Subject: ngnix support for the android In-Reply-To: References: Message-ID: <20120928135328.GA19340@vlpc> On Wed, Sep 26, 2012 at 01:01:46PM -0400, Indtiny s wrote: > Hi, > I want to cross build the nginx for the android using Android-ndk .. Is > Nginx support for android ..? > > has any one tried to build nginx for android , if so pls suggest some docs > to start on . The easiest way to get nginx on android is to use install debian in chroot with the stock android linux kernel. You will have working GCC and no need to deal with cross-building. The process is pretty straightforward and does not involve any magic. As of build using android NDK, I'm almost sure you'll have to hack a lot - at least libc(bionic) is not full and I have no idea how much limited NDK is in terms of available calls. From ywu at about.com Fri Sep 28 15:01:12 2012 From: ywu at about.com (YongFeng Wu) Date: Fri, 28 Sep 2012 11:01:12 -0400 Subject: A possible bug in ngx_rbtree Message-ID: <000001cd9d8a$19dacdb0$4d906910$@com> Hi, We just found a worker process was stuck in an infinite loop, in function ngx_open_file_lookup(). Checking the open file cache RB tree with GDB shows the following: (gdb) p cache->rbtree.root->right $3 = (ngx_rbtree_node_t *) 0x80122f900 (gdb) p cache->rbtree.root->right->right $4 = (ngx_rbtree_node_t *) 0x8040ea400 (gdb) p cache->rbtree.root->right->right->left $5 = (ngx_rbtree_node_t *) 0x801236980 (gdb) p cache->rbtree.root->right->right->left->right $6 = (ngx_rbtree_node_t *) 0x8090ee080 (gdb) p cache->rbtree.root->right->right->left->right->right $7 = (ngx_rbtree_node_t *) 0x804aab280 (gdb) p cache->rbtree.root->right->right->left->right->right->left $8 = (ngx_rbtree_node_t *) 0x804aabf00 (gdb) p cache->rbtree.root->right->right->left->right->right->left->left $9 = (ngx_rbtree_node_t *) 0x8090ee080 (gdb) p cache->rbtree.root->right->right->left->right->right->left->left->right $10 = (ngx_rbtree_node_t *) 0x804aab280 (gdb) p cache->rbtree.root->right->right->left->right->right->left->left->right->lef t $11 = (ngx_rbtree_node_t *) 0x804aabf00 (gdb) p cache->rbtree.root->right->right->left->right->right->left->left->right->lef t->left $12 = (ngx_rbtree_node_t *) 0x8090ee080 (gdb) p cache->rbtree.root->right->right->left->right->right->left->left->right->lef t->left->right $13 = (ngx_rbtree_node_t *) 0x804aab280 (gdb) p cache->rbtree.root->right->right->left->right->right->left->left->right->lef t->left->right->left $14 = (ngx_rbtree_node_t *) 0x804aabf00 Please look at the address of cache->rbtree.root->right->right->left->right->right->left->left ($9) It is the same as that of cache->rbtree.root->right->right->left->right ($6) That means the $9 == $9->parent->parent->parent, so the infinite loop. I think there might be a bug in ngx_rbtree.c. I'll really appreciate it if somebody can look into it. Thanks a lot, Yongfeng Wu -------------- next part -------------- An HTML attachment was scrubbed... URL: From mdounin at mdounin.ru Fri Sep 28 18:28:39 2012 From: mdounin at mdounin.ru (mdounin at mdounin.ru) Date: Fri, 28 Sep 2012 18:28:39 +0000 Subject: [nginx] svn commit: r4872 - trunk/src/core Message-ID: <20120928182839.751443F9C0D@mail.nginx.com> Author: mdounin Date: 2012-09-28 18:28:38 +0000 (Fri, 28 Sep 2012) New Revision: 4872 URL: http://trac.nginx.org/nginx/changeset/4872/nginx Log: Resolver: cached addresses are returned with random rotation now. This ensures balancing when working with dynamically resolved upstream servers with multiple addresses. Based on patch by Anton Jouline. Modified: trunk/src/core/ngx_resolver.c Modified: trunk/src/core/ngx_resolver.c =================================================================== --- trunk/src/core/ngx_resolver.c 2012-09-28 13:49:26 UTC (rev 4871) +++ trunk/src/core/ngx_resolver.c 2012-09-28 18:28:38 UTC (rev 4872) @@ -88,6 +88,8 @@ static void ngx_resolver_free(ngx_resolver_t *r, void *p); static void ngx_resolver_free_locked(ngx_resolver_t *r, void *p); static void *ngx_resolver_dup(ngx_resolver_t *r, void *src, size_t size); +static in_addr_t *ngx_resolver_rotate(ngx_resolver_t *r, in_addr_t *src, + ngx_uint_t n); static u_char *ngx_resolver_log_error(ngx_log_t *log, u_char *buf, size_t len); @@ -445,8 +447,7 @@ if (naddrs != 1) { addr = 0; - addrs = ngx_resolver_dup(r, rn->u.addrs, - naddrs * sizeof(in_addr_t)); + addrs = ngx_resolver_rotate(r, rn->u.addrs, naddrs); if (addrs == NULL) { return NGX_ERROR; } @@ -2135,6 +2136,28 @@ } +static in_addr_t * +ngx_resolver_rotate(ngx_resolver_t *r, in_addr_t *src, ngx_uint_t n) +{ + void *dst, *p; + ngx_uint_t j; + + dst = ngx_resolver_alloc(r, n * sizeof(in_addr_t)); + + j = ngx_random() % n; + + if (j == 0) { + ngx_memcpy(dst, src, n * sizeof(in_addr_t)); + return dst; + } + + p = ngx_cpymem(dst, &src[j], (n - j) * sizeof(in_addr_t)); + ngx_memcpy(p, src, j * sizeof(in_addr_t)); + + return dst; +} + + char * ngx_resolver_strerror(ngx_int_t err) { From mdounin at mdounin.ru Fri Sep 28 18:51:34 2012 From: mdounin at mdounin.ru (Maxim Dounin) Date: Fri, 28 Sep 2012 22:51:34 +0400 Subject: [PATCH] Random peer selection for implicit upstream defined by proxy_pass In-Reply-To: References: <20120907144500.GL40452@mdounin.ru> <20120919152450.GI40452@mdounin.ru> Message-ID: <20120928185133.GO40452@mdounin.ru> Hello! On Wed, Sep 19, 2012 at 10:39:01PM -0700, Anton Jouline wrote: [...] > So, i decided instead to take a look at ngx_resolver code and > it turned out that it's actually very simple to randomize there, > since there is code already that copies the array from one place > in memory to another. Looked like that was ideal place for > changing the order via simple rotation. > Here is the new patch: > > > > --- a/src/core/ngx_resolver.c > +++ b/src/core/ngx_resolver.c > @@ -88,6 +88,8 @@ static void *ngx_resolver_calloc(ngx_resolver_t *r, > size_t size); > static void ngx_resolver_free(ngx_resolver_t *r, void *p); > static void ngx_resolver_free_locked(ngx_resolver_t *r, void *p); > static void *ngx_resolver_dup(ngx_resolver_t *r, void *src, size_t size); > +static in_addr_t *ngx_resolver_dup_rotated(ngx_resolver_t *r, in_addr_t *src, > + u_short n); > static u_char *ngx_resolver_log_error(ngx_log_t *log, u_char *buf, size_t len); [...] Slightly modified version committed, thanks: http://trac.nginx.org/nginx/changeset/4872/nginx I'm a bit unsure - would you prefer Russian attribution in CHANGES.ru? (We usually write names in Russian in CHANGES.ru if contributor is from exUSSR/speak Russian.) Maxim Dounin From juce66 at gmail.com Fri Sep 28 20:08:30 2012 From: juce66 at gmail.com (Anton Jouline) Date: Fri, 28 Sep 2012 13:08:30 -0700 Subject: [PATCH] Random peer selection for implicit upstream defined by proxy_pass In-Reply-To: <20120928185133.GO40452@mdounin.ru> References: <20120907144500.GL40452@mdounin.ru> <20120919152450.GI40452@mdounin.ru> <20120928185133.GO40452@mdounin.ru> Message-ID: hi Maxim, yes, attribution in CHANGES.ru would be great, thanks! In Russian, my name is spelled as: ????? ????? One comment on changeset 4872: On line #1385, wouldn't you want to call ngx_resolver_rotate() instead of ngx_resolver_dup() ? To cover the case of the name not being cached at all for some odd reason - like, if TTL is set to 0 seconds. Or, if TTL is really small, which would also skew the distribution to use the 1st addr a lot more than others. On Fri, Sep 28, 2012 at 11:51 AM, Maxim Dounin wrote: > Hello! > > On Wed, Sep 19, 2012 at 10:39:01PM -0700, Anton Jouline wrote: > > [...] > >> So, i decided instead to take a look at ngx_resolver code and >> it turned out that it's actually very simple to randomize there, >> since there is code already that copies the array from one place >> in memory to another. Looked like that was ideal place for >> changing the order via simple rotation. >> Here is the new patch: >> >> >> >> --- a/src/core/ngx_resolver.c >> +++ b/src/core/ngx_resolver.c >> @@ -88,6 +88,8 @@ static void *ngx_resolver_calloc(ngx_resolver_t *r, >> size_t size); >> static void ngx_resolver_free(ngx_resolver_t *r, void *p); >> static void ngx_resolver_free_locked(ngx_resolver_t *r, void *p); >> static void *ngx_resolver_dup(ngx_resolver_t *r, void *src, size_t size); >> +static in_addr_t *ngx_resolver_dup_rotated(ngx_resolver_t *r, in_addr_t *src, >> + u_short n); >> static u_char *ngx_resolver_log_error(ngx_log_t *log, u_char *buf, size_t len); > > [...] > > Slightly modified version committed, thanks: > > http://trac.nginx.org/nginx/changeset/4872/nginx > > I'm a bit unsure - would you prefer Russian attribution in > CHANGES.ru? (We usually write names in Russian in CHANGES.ru if > contributor is from exUSSR/speak Russian.) > > Maxim Dounin > > _______________________________________________ > nginx-devel mailing list > nginx-devel at nginx.org > http://mailman.nginx.org/mailman/listinfo/nginx-devel From mdounin at mdounin.ru Sat Sep 29 07:24:38 2012 From: mdounin at mdounin.ru (Maxim Dounin) Date: Sat, 29 Sep 2012 11:24:38 +0400 Subject: [PATCH] Random peer selection for implicit upstream defined by proxy_pass In-Reply-To: References: <20120907144500.GL40452@mdounin.ru> <20120919152450.GI40452@mdounin.ru> <20120928185133.GO40452@mdounin.ru> Message-ID: <20120929072438.GP40452@mdounin.ru> Hello! On Fri, Sep 28, 2012 at 01:08:30PM -0700, Anton Jouline wrote: > hi Maxim, > > yes, attribution in CHANGES.ru would be great, thanks! > In Russian, my name is spelled as: ????? ????? Yep, that's what I supposed but wasn't sure. Thanks for clarification. :) > One comment on changeset 4872: > > On line #1385, wouldn't you want to call ngx_resolver_rotate() > instead of ngx_resolver_dup() ? To cover the case of the name > not being cached at all for some odd reason - like, if TTL is set > to 0 seconds. Or, if TTL is really small, which would also skew > the distribution to use the 1st addr a lot more than others. I think in this case it's up to DNS server to do proper rotation/randomization, and if it doesn't do that - it's probably for reason. -- Maxim Dounin http://nginx.com/support.html From vshebordaev at mail.ru Sun Sep 30 03:55:49 2012 From: vshebordaev at mail.ru (=?UTF-8?B?VmxhZGltaXIgU2hlYm9yZGFldg==?=) Date: Sun, 30 Sep 2012 07:55:49 +0400 Subject: [PATCH] export variables from the log module Message-ID: <1348977349.931833915@f345.mail.ru> Index: src/http/ngx_http_variables.c =================================================================== --- src/http/ngx_http_variables.c?? ?(revision 4872) +++ src/http/ngx_http_variables.c?? ?(working copy) @@ -487,12 +487,14 @@ ???????? } else { ? ???????????? vv = ngx_palloc(r->pool, sizeof(ngx_http_variable_value_t)); - -??????????? if (vv && v->get_handler(r, vv, v->data) == NGX_OK) { -??????????????? return vv; +??????????? if (vv) { +?????????????? ? +??????????????? vv->len = 0; +??????????????? if (v->get_handler(r, vv, v->data) != NGX_OK) { +??????????????????? vv = NULL; +??????????????? } ???????????? } - -??????????? return NULL; +??????????? return vv; ???????? } ???? } ? Index: src/http/modules/ngx_http_log_module.c =================================================================== --- src/http/modules/ngx_http_log_module.c?? ?(revision 4872) +++ src/http/modules/ngx_http_log_module.c?? ?(working copy) @@ -114,6 +114,7 @@ ?static void *ngx_http_log_create_loc_conf(ngx_conf_t *cf); ?static char *ngx_http_log_merge_loc_conf(ngx_conf_t *cf, void *parent, ???? void *child); +static ngx_int_t ngx_http_log_add_variables(ngx_conf_t *cf); ?static char *ngx_http_log_set_log(ngx_conf_t *cf, ngx_command_t *cmd, ???? void *conf); ?static char *ngx_http_log_set_format(ngx_conf_t *cf, ngx_command_t *cmd, @@ -154,7 +155,7 @@ ? ? ?static ngx_http_module_t? ngx_http_log_module_ctx = { -??? NULL,????????????????????????????????? /* preconfiguration */ +??? ngx_http_log_add_variables,??????????? /* preconfiguration */ ???? ngx_http_log_init,???????????????????? /* postconfiguration */ ? ???? ngx_http_log_create_main_conf,???????? /* create main configuration */ @@ -1338,6 +1339,70 @@ ? ? ?static ngx_int_t +ngx_http_log_get_variable(ngx_http_request_t *r, ngx_variable_value_t *value, uintptr_t data) +{ +??? ngx_http_log_var_t *v; + +??? v = (ngx_http_log_var_t *)data; + +??? if (!value->len) { +??????? value->data = ngx_pnalloc(r->pool, v->len); +??????? if (value->data == NULL) { +??????????? return NGX_ERROR; +??????? } +??? } + +??? value->len = v->run(r, value->data, NULL) - value->data; +?? ? +??? value->not_found = 0; +??? value->valid = 1; + +??? return NGX_OK; +} + + +static ngx_int_t +ngx_http_log_add_variables(ngx_conf_t *cf) +{ +??? ngx_http_log_var_t *v; +??? ngx_http_variable_t *var; +??? ngx_http_core_main_conf_t *cmcf; + +??? ngx_int_t rc; + +??? cmcf = ngx_http_conf_get_module_main_conf(cf, ngx_http_core_module); +?? ? +??? var = NULL; + +??? for (v = ngx_http_log_vars; v->name.len; v++) { + +??????? if (var == NULL) { +??????????? var = ngx_pcalloc(cf->pool, sizeof(ngx_http_variable_t)); +??????????? if (var == NULL) { +??????????????? return NGX_ERROR; +??????????? } +??????? } + +??????? var->name = v->name; +?????? ? +??????? rc = ngx_hash_add_key(cmcf->variables_keys, &v->name, var, +????????????????????????????? NGX_HASH_READONLY_KEY); + +??????? if (rc == NGX_BUSY) { +??????????? continue; +??????? } + +??????? var->get_handler = ngx_http_log_get_variable; +??????? var->data = (uintptr_t)v; +??????? var->flags = NGX_HTTP_VAR_CHANGEABLE|NGX_HTTP_VAR_NOCACHEABLE; + +??????? var = NULL; +??? } + +??? return NGX_OK; +} +?????? ? +static ngx_int_t ?ngx_http_log_init(ngx_conf_t *cf) ?{ ???? ngx_str_t????????????????? *value; -------------- next part -------------- An HTML attachment was scrubbed... URL: From vshebordaev at mail.ru Sun Sep 30 04:19:25 2012 From: vshebordaev at mail.ru (=?UTF-8?B?VmxhZGltaXIgU2hlYm9yZGFldg==?=) Date: Sun, 30 Sep 2012 08:19:25 +0400 Subject: Can not use $bytes_sent in module (complex values) In-Reply-To: References: <1348215438.687512098@f258.mail.ru> <20120919000656.GA40452@mdounin.ru> Message-ID: <1348978765.412780332@f345.mail.ru> Hi, Benjamin, Thu, 27 Sep 2012 17:30:10 +0200 ?? Benjamin Gr?ssing : >Hello, > > thanks for your further explanations. I think I understand the > concepts of these variables and how they are calculated now. However, > I am not sure how calling nxg_http_add_variable and implementing that > specific getter would affect performance for modules (other than the > log module) using it. >No wonder, it is just not true. The log module can reference indexed variable in the same way that the rewite module does, but it immediatly outputs its value into the log file. Please excuse me. It seems, for some reason I was trying to explain that it uses unattended indexed variables internally. > I have also had a look at the changeset 4686, adding the $status > variable. Thats exactly what I had in mind! Since $body_bytes_sent > already exists (it actually just subtracts the header length from the > total length!), adding $bytes_sent is actually quite easy (or did I > miss something?). > > I have attached a patch (based on nginx 1.3.6) that adds $bytes_sent > to ngx_http_variables. It works great for me - I'd be happy to see it > in a future nginx release. >Also please have a look at my recent patch. It suggests rather similar but slightly more generalized solution. Regards, Vladimir > Let me know if I can help any further! > > Regards > Benjamin > > 2012/9/21 Vladimir Shebordaev : > > Benjamin, > > > > it seems I used not quite neat wording, so my earlier explanations might > > look obfuscating. > > > > Basically, the indexed variable is any "automatic" variable referenced thow > > the request->variables array. The variables of the current log module do use > > index slots, so they always have space for the value preallocated in > > ngx_http_init_request(). But those variables can be referenced from a script > > or by another module neither by name since they are missing from > > cmcf->variables_hash, nor by index, e.g. by a complex value, since they > > have no getter method thus are rejected in ngx_http_variables_init_vars() as > > unknown. > > > > If you modify the log module to register its variables with > > nxg_http_add_variable() and implement correct get handlers, they would > > surely be available everywhere like any other variable. For performance > > reasons the log module in itself should still use its variables as indexed > > ones, since log operations are expensive and rather frequent. If you > > implement the generic getter method for the http variable infrastructure, > > you might immediatly use it in the log module handler probably using their > > own machinery. > > > > Hope it helps. > > > > Regards, > > Vladimir > > > > > > > > > > Wed, 19 Sep 2012 23:11:57 +0200 ?? Benjamin Gr?ssing : > > > > Hi Vladimir, hi Maxim! > > > > > > Thanks for your answers. I've had a look at the rewrite module - > > unfortunately it does not support the $bytes_sent variable either as > > configuring "if ($bytes_sent) { }" throws the error: [emerg] unknown > > "bytes_sent" variable. > > > > Nice to hear that there is a plan to include all variables in the > > complex value system - I think that would make that part a of nginx a > > lot more clear! I'd be happy if I could contribute here, but I am not > > sure about the details. Is there any guide to contributing to nginx > > available or do I just have to create a patch file and post it here? > > Can I just start with the nginx 1.2.3 source code from nginx.org? > > > > Regards > > Benjamin > > > > 2012/9/19 Maxim Dounin : > >> Hello! > >> > >> On Tue, Sep 18, 2012 at 03:55:38PM +0200, Benjamin Gr?ssing wrote: > >> > >>> Hello, > >>> > >>> I am currently working on a nginx module, that parses user defined > >>> strings for variables using ngx_http_complex_value_t. That works great > >>> with all variables defined in ngx_http_variables.c (i.e. > >>> $body_bytes_sent). However, variables as $bytes_sent do not exist here > >>> since ngx_http_log_module uses its own variable parsing system. What > >>> is the best way to use these extra variables in my module? I have > >>> thought of the following options, I am not very sure about what is the > >>> best way to go: > >>> a) Copy the ngx_http_log_module code in my own module and rename every > >>> single type and function to avoid double definitions. Reuse functions > >>> and types from there. > >>> b) Move the ngx_http_log_module code in my own module directory and > >>> compile it only there. Reuse functions and types from there. > >>> c) Write own variable parsing system, reuse a few snippets from > >>> ngx_http_variables and ngx_http_log_module, whatever fits best. > >>> d) Use complex values and try to extend the ngx_http_core_variables > >>> array to include $body_bytes_sent and the other missing variables (I > >>> don't understand why they are not included here anyway?). > >>> e) Any better way to achieve that? > >>> > >>> I'd really appreciate your help. > >> > >> All variables that are currently present only in log module should > >> really be made available as normal variables as well. They aren't > >> yet mostly due to historical reasons (log module appeared before > >> normal variables were implemented) and the fact that most of > >> them are only usable for logging. > >> > >> No idea what would be best for your particular module, but in > >> general it's better to use complex values and what's available via > >> normal variables. Unless you need $bytes_sent right now - it's > >> probably the best aproach, and missing variables will be available > >> once they appear in nginx. > >> > >> (The latter is something you may speedup by submitting a good > >> patch. Most tricky part here would be to follow coding style.) > >> > >> Maxim Dounin > >> > >> _______________________________________________ > >> nginx-devel mailing list > >> nginx-devel at nginx.org > >> http://mailman.nginx.org/mailman/listinfo/nginx-devel > > > > _______________________________________________ > > nginx-devel mailing list > > nginx-devel at nginx.org > > http://mailman.nginx.org/mailman/listinfo/nginx-devel > > > > > > > > _______________________________________________ > > nginx-devel mailing list > > nginx-devel at nginx.org > > http://mailman.nginx.org/mailman/listinfo/nginx-devel > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From vshebordaev at mail.ru Sun Sep 30 05:11:15 2012 From: vshebordaev at mail.ru (Vladimir Shebordaev) Date: Sun, 30 Sep 2012 09:11:15 +0400 Subject: [PATCH] export variables from the log module In-Reply-To: <1348977349.931833915@f345.mail.ru> References: <1348977349.931833915@f345.mail.ru> Message-ID: <5067D473.8050105@mail.ru> On 30.09.2012 07:55, Vladimir Shebordaev wrote: > Index: src/http/ngx_http_variables.c > =================================================================== > --- src/http/ngx_http_variables.c (revision 4872) > +++ src/http/ngx_http_variables.c (working copy) > @@ -487,12 +487,14 @@ > } else { > > vv = ngx_palloc(r->pool, > sizeof(ngx_http_variable_value_t)); > - > - if (vv && v->get_handler(r, vv, v->data) == NGX_OK) { > - return vv; > + if (vv) { > + > + vv->len = 0; The idea behind this is to indicate unallocated value data to the getter. Probably, zero vv-data field would be cleaner solution, since the getter can just do nothing that gives zero value length while the space has been actually allocated. > + if (v->get_handler(r, vv, v->data) != NGX_OK) { > + vv = NULL; > + } > } > - > - return NULL; > + return vv; > } > } > > Index: src/http/modules/ngx_http_log_module.c > =================================================================== > --- src/http/modules/ngx_http_log_module.c (revision 4872) > +++ src/http/modules/ngx_http_log_module.c (working copy) > @@ -114,6 +114,7 @@ > static void *ngx_http_log_create_loc_conf(ngx_conf_t *cf); > static char *ngx_http_log_merge_loc_conf(ngx_conf_t *cf, void > *parent, > void *child); > +static ngx_int_t ngx_http_log_add_variables(ngx_conf_t *cf); > static char *ngx_http_log_set_log(ngx_conf_t *cf, ngx_command_t > *cmd, > void *conf); > static char *ngx_http_log_set_format(ngx_conf_t *cf, > ngx_command_t *cmd, > @@ -154,7 +155,7 @@ > > > static ngx_http_module_t ngx_http_log_module_ctx = { > - NULL, /* preconfiguration */ > + ngx_http_log_add_variables, /* preconfiguration */ > ngx_http_log_init, /* postconfiguration */ > > ngx_http_log_create_main_conf, /* create main > configuration */ > @@ -1338,6 +1339,70 @@ > > > static ngx_int_t > +ngx_http_log_get_variable(ngx_http_request_t *r, > ngx_variable_value_t *value, uintptr_t data) > +{ > + ngx_http_log_var_t *v; > + > + v = (ngx_http_log_var_t *)data; > + > + if (!value->len) { The same field should be checked here. Regards, Vladimir > + value->data = ngx_pnalloc(r->pool, v->len); > + if (value->data == NULL) { > + return NGX_ERROR; > + } > + } > + > + value->len = v->run(r, value->data, NULL) - value->data; > + > + value->not_found = 0; > + value->valid = 1; > + > + return NGX_OK; > +} > + > + > +static ngx_int_t > +ngx_http_log_add_variables(ngx_conf_t *cf) > +{ > + ngx_http_log_var_t *v; > + ngx_http_variable_t *var; > + ngx_http_core_main_conf_t *cmcf; > + > + ngx_int_t rc; > + > + cmcf = ngx_http_conf_get_module_main_conf(cf, > ngx_http_core_module); > + > + var = NULL; > + > + for (v = ngx_http_log_vars; v->name.len; v++) { > + > + if (var == NULL) { > + var = ngx_pcalloc(cf->pool, > sizeof(ngx_http_variable_t)); > + if (var == NULL) { > + return NGX_ERROR; > + } > + } > + > + var->name = v->name; > + > + rc = ngx_hash_add_key(cmcf->variables_keys, &v->name, var, > + NGX_HASH_READONLY_KEY); > + > + if (rc == NGX_BUSY) { > + continue; > + } > + > + var->get_handler = ngx_http_log_get_variable; > + var->data = (uintptr_t)v; > + var->flags = > NGX_HTTP_VAR_CHANGEABLE|NGX_HTTP_VAR_NOCACHEABLE; > + > + var = NULL; > + } > + > + return NGX_OK; > +} > + > +static ngx_int_t > ngx_http_log_init(ngx_conf_t *cf) > { > ngx_str_t *value; > > > _______________________________________________ > nginx-devel mailing list > nginx-devel at nginx.org > http://mailman.nginx.org/mailman/listinfo/nginx-devel > From mdounin at mdounin.ru Sun Sep 30 18:54:16 2012 From: mdounin at mdounin.ru (Maxim Dounin) Date: Sun, 30 Sep 2012 22:54:16 +0400 Subject: [PATCH] export variables from the log module In-Reply-To: <1348977349.931833915@f345.mail.ru> References: <1348977349.931833915@f345.mail.ru> Message-ID: <20120930185416.GR40452@mdounin.ru> Hello! On Sun, Sep 30, 2012 at 07:55:49AM +0400, Vladimir Shebordaev wrote: [...] > +static ngx_int_t > +ngx_http_log_add_variables(ngx_conf_t *cf) > +{ > +??? ngx_http_log_var_t *v; > +??? ngx_http_variable_t *var; > +??? ngx_http_core_main_conf_t *cmcf; > + > +??? ngx_int_t rc; > + > +??? cmcf = ngx_http_conf_get_module_main_conf(cf, ngx_http_core_module); > +?? ? > +??? var = NULL; > + > +??? for (v = ngx_http_log_vars; v->name.len; v++) { > + > +??????? if (var == NULL) { > +??????????? var = ngx_pcalloc(cf->pool, sizeof(ngx_http_variable_t)); > +??????????? if (var == NULL) { > +??????????????? return NGX_ERROR; > +??????????? } > +??????? } > + > +??????? var->name = v->name; > +?????? ? > +??????? rc = ngx_hash_add_key(cmcf->variables_keys, &v->name, var, > +????????????????????????????? NGX_HASH_READONLY_KEY); > + > +??????? if (rc == NGX_BUSY) { > +??????????? continue; > +??????? } No, it's layering violation. It's not log modules business to hack cmcf->variables_keys. And, as a side note, it's very-very wrong to just continue on conflicting variable names. I've already pointed out correct aproach to this: log module not-really-variables should be reimplemented/duplicated as real variables in ngx_http_variables.c. [...] Maxim Dounin From vshebordaev at mail.ru Sun Sep 30 23:02:10 2012 From: vshebordaev at mail.ru (Vladimir Shebordaev) Date: Mon, 01 Oct 2012 03:02:10 +0400 Subject: [PATCH] export variables from the log module In-Reply-To: <20120930185416.GR40452@mdounin.ru> References: <1348977349.931833915@f345.mail.ru> <20120930185416.GR40452@mdounin.ru> Message-ID: <5068CF72.4080608@mail.ru> Hi, On 30.09.2012 22:54, Maxim Dounin wrote: > Hello! > > On Sun, Sep 30, 2012 at 07:55:49AM +0400, Vladimir Shebordaev wrote: > > [...] > >> +static ngx_int_t >> +ngx_http_log_add_variables(ngx_conf_t *cf) >> +{ >> + ngx_http_log_var_t *v; >> + ngx_http_variable_t *var; >> + ngx_http_core_main_conf_t *cmcf; >> + >> + ngx_int_t rc; >> + >> + cmcf = ngx_http_conf_get_module_main_conf(cf, ngx_http_core_module); >> + >> + var = NULL; >> + >> + for (v = ngx_http_log_vars; v->name.len; v++) { >> + >> + if (var == NULL) { >> + var = ngx_pcalloc(cf->pool, sizeof(ngx_http_variable_t)); >> + if (var == NULL) { >> + return NGX_ERROR; >> + } >> + } >> + >> + var->name = v->name; >> + >> + rc = ngx_hash_add_key(cmcf->variables_keys, &v->name, var, >> + NGX_HASH_READONLY_KEY); >> + >> + if (rc == NGX_BUSY) { >> + continue; >> + } > > No, it's layering violation. It's not log modules business to > hack cmcf->variables_keys. And, as a side note, it's very-very > wrong to just continue on conflicting variable names. Well, the only conflicting variable is "status", but this is only a patch. I guess, emergency stop would look rather strange here, since this is a static array, Actually, there are variables in http/ngx_http_variables.c whose getters implemented in other modules. Basically, I intentionally suggested to implement those ones in the log module to preserve existing module dependencies. > I've already pointed out correct aproach to this: log module > not-really-variables should be reimplemented/duplicated as real > variables in ngx_http_variables.c. > I think that is exactly what Benjamin would like to do. By the way, passing zeroed vv->data field to v->get_handler() in ngx_http_get_variable() would be of some interest in itself. The handlers could rely on it to reuse the value's data when it is feasible. I think it could be done just for the sake of consistency. As of now, it is possible just for the indexed variables whose value data is explicitly zeroed out in ngx_http_init_request() upon request->variables array allocation. Hope it helps. Regards, Vladimir > [...] > > Maxim Dounin > > From mailtogroe at gmail.com Sun Sep 30 23:20:29 2012 From: mailtogroe at gmail.com (=?ISO-8859-1?Q?Benjamin_Gr=F6ssing?=) Date: Mon, 1 Oct 2012 01:20:29 +0200 Subject: [PATCH] export variables from the log module In-Reply-To: <5068CF72.4080608@mail.ru> References: <1348977349.931833915@f345.mail.ru> <20120930185416.GR40452@mdounin.ru> <5068CF72.4080608@mail.ru> Message-ID: Hey guys, thanks for your work, Vladimir! To me, both the solution using getters in the log module from Vladimir as well as just re-implementing the methods in ngx_http_variables.c seem to have pros and cons. Using the log module as a dependency from core might be a problem when compiling without the log module (is that even possible?). However, duplicate code for log/ngx_http variables also don't seem to be the ideal solution. Anyway, I am not very familiar with your conventions and what is rated higher in nginx development, DRY code or core/module layering models. I've attached my patch again; it does what you suggested, Maxim, in a hardly intrusive way. Looking forward to see one of the solutions merged. Regards Benjamin 2012/10/1 Vladimir Shebordaev : > Hi, > > > On 30.09.2012 22:54, Maxim Dounin wrote: >> >> Hello! >> >> On Sun, Sep 30, 2012 at 07:55:49AM +0400, Vladimir Shebordaev wrote: >> >> [...] >> >>> +static ngx_int_t >>> +ngx_http_log_add_variables(ngx_conf_t *cf) >>> +{ >>> + ngx_http_log_var_t *v; >>> + ngx_http_variable_t *var; >>> + ngx_http_core_main_conf_t *cmcf; >>> + >>> + ngx_int_t rc; >>> + >>> + cmcf = ngx_http_conf_get_module_main_conf(cf, ngx_http_core_module); >>> + >>> + var = NULL; >>> + >>> + for (v = ngx_http_log_vars; v->name.len; v++) { >>> + >>> + if (var == NULL) { >>> + var = ngx_pcalloc(cf->pool, sizeof(ngx_http_variable_t)); >>> + if (var == NULL) { >>> + return NGX_ERROR; >>> + } >>> + } >>> + >>> + var->name = v->name; >>> + >>> + rc = ngx_hash_add_key(cmcf->variables_keys, &v->name, var, >>> + NGX_HASH_READONLY_KEY); >>> + >>> + if (rc == NGX_BUSY) { >>> + continue; >>> + } >> >> >> No, it's layering violation. It's not log modules business to >> hack cmcf->variables_keys. And, as a side note, it's very-very >> wrong to just continue on conflicting variable names. > > > > Well, the only conflicting variable is "status", but this is only a patch. I > guess, emergency stop would look rather strange here, since this is a static > array, Actually, there are variables in http/ngx_http_variables.c whose > getters implemented in other modules. Basically, I intentionally suggested > to implement those ones in the log module to preserve existing module > dependencies. > > >> I've already pointed out correct aproach to this: log module >> not-really-variables should be reimplemented/duplicated as real >> variables in ngx_http_variables.c. >> > > I think that is exactly what Benjamin would like to do. > > By the way, passing zeroed vv->data field to v->get_handler() in > ngx_http_get_variable() would be of some interest in itself. The handlers > could rely on it to reuse the value's data when it is feasible. I think it > could be done just for the sake of consistency. As of now, it is possible > just for the indexed variables whose value data is explicitly zeroed out in > ngx_http_init_request() upon request->variables array allocation. > > Hope it helps. > > Regards, > Vladimir > > >> [...] >> >> Maxim Dounin >> >> > > _______________________________________________ > nginx-devel mailing list > nginx-devel at nginx.org > http://mailman.nginx.org/mailman/listinfo/nginx-devel -------------- next part -------------- A non-text attachment was scrubbed... Name: add_bytes_sent_variable.patch Type: application/octet-stream Size: 1664 bytes Desc: not available URL: From vshebordaev at mail.ru Sun Sep 30 23:47:21 2012 From: vshebordaev at mail.ru (Vladimir Shebordaev) Date: Mon, 01 Oct 2012 03:47:21 +0400 Subject: [PATCH] export variables from the log module In-Reply-To: References: <1348977349.931833915@f345.mail.ru> <20120930185416.GR40452@mdounin.ru> <5068CF72.4080608@mail.ru> Message-ID: <5068DA09.6030901@mail.ru> Hi, Benjamin, On 01.10.2012 03:20, Benjamin Gr?ssing wrote: > Hey guys, > > thanks for your work, Vladimir! To me, both the solution using getters > in the log module from Vladimir as well as just re-implementing the > methods in ngx_http_variables.c seem to have pros and cons. Using the > log module as a dependency from core might be a problem when compiling > without the log module (is that even possible?). However, duplicate > code for log/ngx_http variables also don't seem to be the ideal > solution. As for me, I've also suggested to export the variables that way due to they are documented in the log module. Also, having separate implementations will require keeping 'em consistent upon change but this is hardly probable though. > Anyway, I am not very familiar with your conventions and what is rated > higher in nginx development, DRY code or core/module layering models. > I've attached my patch again; it does what you suggested, Maxim, in a > hardly intrusive way. > I meant that you would like to implement other variables from the log module in the way you did it with $bytes_sent. Sure, except for the $status variable. I think so Maxim did. Regards, Vladimir > Looking forward to see one of the solutions merged. > > Regards > Benjamin > > 2012/10/1 Vladimir Shebordaev : >> Hi, >> >> >> On 30.09.2012 22:54, Maxim Dounin wrote: >>> >>> Hello! >>> >>> On Sun, Sep 30, 2012 at 07:55:49AM +0400, Vladimir Shebordaev wrote: >>> >>> [...] >>> >>>> +static ngx_int_t >>>> +ngx_http_log_add_variables(ngx_conf_t *cf) >>>> +{ >>>> + ngx_http_log_var_t *v; >>>> + ngx_http_variable_t *var; >>>> + ngx_http_core_main_conf_t *cmcf; >>>> + >>>> + ngx_int_t rc; >>>> + >>>> + cmcf = ngx_http_conf_get_module_main_conf(cf, ngx_http_core_module); >>>> + >>>> + var = NULL; >>>> + >>>> + for (v = ngx_http_log_vars; v->name.len; v++) { >>>> + >>>> + if (var == NULL) { >>>> + var = ngx_pcalloc(cf->pool, sizeof(ngx_http_variable_t)); >>>> + if (var == NULL) { >>>> + return NGX_ERROR; >>>> + } >>>> + } >>>> + >>>> + var->name = v->name; >>>> + >>>> + rc = ngx_hash_add_key(cmcf->variables_keys, &v->name, var, >>>> + NGX_HASH_READONLY_KEY); >>>> + >>>> + if (rc == NGX_BUSY) { >>>> + continue; >>>> + } >>> >>> >>> No, it's layering violation. It's not log modules business to >>> hack cmcf->variables_keys. And, as a side note, it's very-very >>> wrong to just continue on conflicting variable names. >> >> >> >> Well, the only conflicting variable is "status", but this is only a patch. I >> guess, emergency stop would look rather strange here, since this is a static >> array, Actually, there are variables in http/ngx_http_variables.c whose >> getters implemented in other modules. Basically, I intentionally suggested >> to implement those ones in the log module to preserve existing module >> dependencies. >> >> >>> I've already pointed out correct aproach to this: log module >>> not-really-variables should be reimplemented/duplicated as real >>> variables in ngx_http_variables.c. >>> >> >> I think that is exactly what Benjamin would like to do. >> >> By the way, passing zeroed vv->data field to v->get_handler() in >> ngx_http_get_variable() would be of some interest in itself. The handlers >> could rely on it to reuse the value's data when it is feasible. I think it >> could be done just for the sake of consistency. As of now, it is possible >> just for the indexed variables whose value data is explicitly zeroed out in >> ngx_http_init_request() upon request->variables array allocation. >> >> Hope it helps. >> >> Regards, >> Vladimir >> >> >>> [...] >>> >>> Maxim Dounin >>> >>>