nginx-devel May 2015

nginx-devel@nginx.org

23 participants
36 discussions

[RFC] event/openssl: Add dynamic record size support for serving ssl trafic

by chen

Hi list: This is v1 of the patchset the implementing the feature SSL Dynamic Record Sizing, inspiring by Google Front End (https://www.igvita.com/2013/10/24/optimizing-tls-record-size-and-buffering-…) . There are 3 conditions, if true at the same time, may trigger SSL_write to send small record over the link, hard coded 1400 bytes at this time to keep it fit into MTU size. We just send out 3 of this small record at most to reduce framing overhead when serving large object, that is enough for browser to discovery other dependency of the page at top of html file. If the buffer chain is smaller than 4096 bytes, it will not justify the overhead of sending small record. After idle for 60s(hard coded at this moment), start all over again. Any comments is welcome. Regard YM hg export tip # HG changeset patch # User YM Chen <gzchenym(a)126.com> # Date 1430828974 -28800 # Node ID 31bfe6403c340bdc4c04e8e87721736c07bceef8 # Parent 162b2d27d4e1ce45bb9217d6958348c64f726a28 [RFC] event/openssl: Add dynamic record size support for serving ssl trafic SSL Dynamic Record Sizing is a long sought after feature for website that serving huge amount of encrypted traffic. The rational behide this is that SSL record should not overflow the congestion window at the beginning of slow-start period and by doing so, we can let the browser decode the first ssl record within 1 rtt and establish other connections to fetch other resources that are referenced at the top of the html file. diff -r 162b2d27d4e1 -r 31bfe6403c34 src/event/ngx_event_openssl.c --- a/src/event/ngx_event_openssl.c Wed Apr 29 14:59:02 2015 +0300 +++ b/src/event/ngx_event_openssl.c Tue May 05 20:29:34 2015 +0800 @@ -1508,6 +1508,11 @@ ngx_uint_t flush; ssize_t send, size; ngx_buf_t *buf; + ngx_msec_t last_sent_timer_diff; + ngx_uint_t loop_count; + + last_sent_timer_diff = ngx_current_msec - c->ssl->last_write_msec; + loop_count = 0; if (!c->ssl->buffer) { @@ -1517,7 +1522,13 @@ continue; } - n = ngx_ssl_write(c, in->buf->pos, in->buf->last - in->buf->pos); + size = in->buf->last - in->buf->pos; + + if(last_sent_timer_diff > 1000*60 && loop_count < 3 && size > 4096) { + size = 1400; + } + + n = ngx_ssl_write(c, in->buf->pos, size); if (n == NGX_ERROR) { return NGX_CHAIN_ERROR; @@ -1532,8 +1543,11 @@ if (in->buf->pos == in->buf->last) { in = in->next; } + + loop_count ++; } + c->ssl->last_write_msec = ngx_current_msec; return in; } @@ -1614,9 +1628,14 @@ if (size == 0) { buf->flush = 0; c->buffered &= ~NGX_SSL_BUFFERED; + c->ssl->last_write_msec = ngx_current_msec; return in; } + if(last_sent_timer_diff > 1000*60 && loop_count < 3 && size > 4096) { + size = 1400; + } + n = ngx_ssl_write(c, buf->pos, size); if (n == NGX_ERROR) { @@ -1633,14 +1652,18 @@ break; } - flush = 0; - - buf->pos = buf->start; - buf->last = buf->start; + if(buf->last == buf->pos) { + flush = 0; + + buf->pos = buf->start; + buf->last = buf->start; + } if (in == NULL || send == limit) { break; } + + loop_count++; } buf->flush = flush; @@ -1652,6 +1675,7 @@ c->buffered &= ~NGX_SSL_BUFFERED; } + c->ssl->last_write_msec = ngx_current_msec; return in; } diff -r 162b2d27d4e1 -r 31bfe6403c34 src/event/ngx_event_openssl.h --- a/src/event/ngx_event_openssl.h Wed Apr 29 14:59:02 2015 +0300 +++ b/src/event/ngx_event_openssl.h Tue May 05 20:29:34 2015 +0800 @@ -51,6 +51,8 @@ ngx_buf_t *buf; size_t buffer_size; + ngx_msec_t last_write_msec; + ngx_connection_handler_pt handler; ngx_event_handler_pt saved_read_handler;

7 years

wrong $bytes_sent on nginx-1.8.0 if aio threads is enabled

by George .

Hi, I found following bug in nginx-1.8.0: if aio is configured with threads support - sometime (one in thousands requests) $bytes_sent contains only length of the header. I'm attaching my nginx.conf, build params and simple python script I'm using the reproduce this issue. Here is the output of test script when the problem appears: . . . received: 101700000 from access_log : 101700000 on 26 iteration 127.0.0.1 - - [15/May/2015 17:27:45] "GET /test HTTP/1.0" 200 - 127.0.0.1 - - [15/May/2015 17:27:47] "GET /test HTTP/1.0" 200 - received: 101700000 from access_log : 101700000 on 27 iteration 127.0.0.1 - - [15/May/2015 17:27:58] "GET /test HTTP/1.0" 200 - 127.0.0.1 - - [15/May/2015 17:28:00] "GET /test HTTP/1.0" 200 - received: 101700000 from access_log : 101690000 on 28 iteration test failed!! also in access_log file" . . . 10170 GET /test HTTP/1.1 10170 GET /test HTTP/1.1 10170 GET /test HTTP/1.1 170 GET /test HTTP/1.1 10170 GET /test HTTP/1.1 10170 GET /test HTTP/1.1 . . Best regards, George

7 years, 1 month

Fwd: Windows shmem fix: makes shared memory fully ASLR and DEP compliant (ea. cache zone, limit zone, etc.)

by Sergey Brester

Hi, enclosed you will find an attached changeset, that fixes a ASLR/DEP problem on windows platforms (example Win 7/2008 x64). To find shared addr offset with ASLR, we have successful used the same resp. similar solution on various open source projects (ex.: postgresql etc.). Also nginx with suggested patch works fine over 5 months in production on several machines (Win-2k8-R2). BTW(1): I have interest to fix a problem with multiple workers under windows (ex.: http://forum.nginx.org/read.php?2,188714,188714#msg-188714 [1]). @Maxim Dounin: can you possibly give me more information, what you mean here, resp. what it currently depends on (see http://forum.nginx.org/read.php?2,188714,212568#msg-212568 [2]). BTW(2): I would like to fix iocp under windows also. Thanks in advance for any information about. P.S. speak fluently russian and german... Regards, Serg G. Brester (sebres) ------------------------- Links: ------ [1] http://forum.nginx.org/read.php?2,188714,188714#msg-188714 [2] http://forum.nginx.org/read.php?2,188714,212568#msg-212568

7 years, 2 months

答复: problems when use fastcgi_pass to deliver request to backend

by 林谡

/* we support the single request per connection */ 2573<http://trac.nginx.org/nginx/browser/nginx/src/http/modules/ngx_http_fastcgi…> 2574<http://trac.nginx.org/nginx/browser/nginx/src/http/modules/ngx_http_fastcgi…> case ngx_http_fastcgi_st_request_id_hi: 2575<http://trac.nginx.org/nginx/browser/nginx/src/http/modules/ngx_http_fastcgi…> if (ch != 0) { 2576<http://trac.nginx.org/nginx/browser/nginx/src/http/modules/ngx_http_fastcgi…> ngx_log_error(NGX_LOG_ERR, r->connection->log, 0, 2577<http://trac.nginx.org/nginx/browser/nginx/src/http/modules/ngx_http_fastcgi…> "upstream sent unexpected FastCGI " 2578<http://trac.nginx.org/nginx/browser/nginx/src/http/modules/ngx_http_fastcgi…> "request id high byte: %d", ch); 2579<http://trac.nginx.org/nginx/browser/nginx/src/http/modules/ngx_http_fastcgi…> return NGX_ERROR; 2580<http://trac.nginx.org/nginx/browser/nginx/src/http/modules/ngx_http_fastcgi…> } 2581<http://trac.nginx.org/nginx/browser/nginx/src/http/modules/ngx_http_fastcgi…> state = ngx_http_fastcgi_st_request_id_lo; 2582<http://trac.nginx.org/nginx/browser/nginx/src/http/modules/ngx_http_fastcgi…> break; 2583<http://trac.nginx.org/nginx/browser/nginx/src/http/modules/ngx_http_fastcgi…> 2584<http://trac.nginx.org/nginx/browser/nginx/src/http/modules/ngx_http_fastcgi…> case ngx_http_fastcgi_st_request_id_lo: 2585<http://trac.nginx.org/nginx/browser/nginx/src/http/modules/ngx_http_fastcgi…> if (ch != 1) { 2586<http://trac.nginx.org/nginx/browser/nginx/src/http/modules/ngx_http_fastcgi…> ngx_log_error(NGX_LOG_ERR, r->connection->log, 0, 2587<http://trac.nginx.org/nginx/browser/nginx/src/http/modules/ngx_http_fastcgi…> "upstream sent unexpected FastCGI " 2588<http://trac.nginx.org/nginx/browser/nginx/src/http/modules/ngx_http_fastcgi…> "request id low byte: %d", ch); 2589<http://trac.nginx.org/nginx/browser/nginx/src/http/modules/ngx_http_fastcgi…> return NGX_ERROR; 2590<http://trac.nginx.org/nginx/browser/nginx/src/http/modules/ngx_http_fastcgi…> } 2591<http://trac.nginx.org/nginx/browser/nginx/src/http/modules/ngx_http_fastcgi…> state = ngx_http_fastcgi_st_content_length_hi; 2592<http://trac.nginx.org/nginx/browser/nginx/src/http/modules/ngx_http_fastcgi…> break; By reading source code, I saw the reason , so can nginx support multi request per connection in future? 发件人: 林谡发送时间: 2015年5月29日 11:37 收件人: 'nginx-devel(a)nginx.org' 主题: problems when use fastcgi_pass to deliver request to backend Hi, I write a fastcgi server and use nginx to pass request to my server. It works till now. But I find a problem. Nginx always set requestId = 1 when sending fastcgi record. I was a little upset for this, cause according to fastcgi protocol, web server can send fastcgi records belonging to different request simultaneously, and requestIds are different and keep unique. I really need this feature, because requests can be handled simultaneously just over one connetion. Can I find a way out?

7 years, 2 months

Kernel stall while testing high-speed HTTPS traffic.

by Ben Greear

We are seeing problems with Nginx (mostly)locking up the server when running high loads of HTTPS traffic. This scenario we had nginx configured to bind to eth3 but our ssh sessions on eth0 were frozen during this condition as well. The system restores itself after a few minutes, (the load generation would have stopped after a minute or two of lockup, that may be what lets things recover). We tested different kernels (4.0.4+, 4.0.0+, 3.17.8+ with local patches, and stock 3.14.27-100.fc19.x86_64, all with same results), different NICs (Intel 10G, Intel 40G), and Apache as web server. Apache can sustain about 10.8Gbps of HTTPS traffic and shows no instability/lockups. nginx maxes out at 2.2Gbps (until it locks up machine). Some kernel splats indicated some files writing to the file system journal were blocked > 180 seconds, but they recover, so it is not a hard lock. The system should not be doing any heavy disk access since we have 32GB RAM. Swap shows no usage. === Scenario === Load testing box has a direct connection to eth3->eth3 over 10Gbps port. Curl clients using https, keepalive, requesting a 1MB file: 1000 clients @ 0.25 req/sec = 243 req/sec, 2.2Gbps tx, load 8.3 400 clients @ 0.65 req/sec = 260 req/sec, 2.2Gbps tx, load 9.2 === Environment === processor : 7 vendor_id : GenuineIntel cpu family : 6 model : 63 model name : Intel(R) Xeon(R) CPU E5-1630 v3 @ 3.70GHz > free total used free shared buffers cached Mem: 32840296 1394884 31445412 0 132792 632068 -/+ buffers/cache: 630024 32210272 Swap: 16457724 0 16457724 > cat /etc/issue Fedora release 19 (Schrödinger’s Cat) Kernel \r on an \m (\l) # uname -a Linux e5-1630-v3-qc 3.14.27-100.fc19.x86_64 #1 SMP Wed Dec 17 19:36:34 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux # /usr/local/lanforge/nginx/sbin/nginx -v nginx version: nginx/1.9.1 We are running small patch to allow nginx to bind to a particular interface. We tried with this option disabled, and that causes the same trouble. The exact source is found below: https://github.com/greearb/nginx/commits/master We are compiling nginx with these options: ./configure --prefix=/usr/local/lanforge/nginx/ --with-http_ssl_module --with-ipv6 --without-http_rewrite_module === Nginx Config === worker_processes auto; worker_rlimit_nofile 100000; error_log logs/eth3_error.log; pid /home/lanforge/vr_conf/nginx_eth3.pid; events { use epoll; worker_connections 8096; multi_accept on; } http { include /usr/local/lanforge/nginx/conf/mime.types; default_type application/octet-stream; access_log off; sendfile on; directio 1m; disable_symlinks on; gzip off; tcp_nopush on; tcp_nodelay on; open_file_cache max=1000 inactive=10s; open_file_cache_valid 600s; open_file_cache_min_uses 2000; open_file_cache_errors off; etag off; server { listen 1.1.1.1:80 so_keepalive=on bind_dev=eth3; server_name nginx.local nginx web.local web; location / { root /var/www/html; index index.html index.htm; } error_page 500 502 503 504 /50x.html; location = /50x.html { root html; } } server { listen 1.1.1.1:443 so_keepalive=on ssl bind_dev=eth3; server_name nginx.local nginx web.local web; ssl_certificate /usr/local/lanforge/apache.crt; ssl_certificate_key /usr/local/lanforge/apache.key; location / { root /var/www/html; index index.html index.htm; } error_page 500 502 503 504 /50x.html; location = /50x.html { root html; } } } Any help or suggestions is appreciated. Thanks, Ben -- Ben Greear <greearb(a)candelatech.com> Candela Technologies Inc http://www.candelatech.com

7 years, 2 months

Strange status 500 with empty response

by Carlos Eduardo Ferreira Rodrigues

Hi, Since upgrading to nginx 1.8.0, we started seeing some requests being logged with status 500 and a response of 0 bytes ($bytes_sent). Not many, only one for about 36000 requests. I've been trying to reliably reproduce this and/or figure out the cause without success, at least to know if this is an nginx bug or some module's. Any ideas or suggestions of how I can go about doing this? These requests use proxy_pass (with cache) and header_filter_by_lua, so I've been able to figure out some things for these failed requests: * They are always a cache hit; * Headers created by the Lua code are correctly logged with $sent_http_*; * Errors in Lua code result in status 500 with a non-empty response (our custom error page); Also, the error log shows nothing along with these errors. Thanks in advance, -- Carlos Rodrigues

7 years, 2 months

[PATCH] Adds $orig_remote_addr in realip module

by Jon Nalley

# HG changeset patch # User Jon Nalley <code(a)bluebot.org> # Date 1432848566 18000 # Thu May 28 16:29:26 2015 -0500 # Node ID 4e59c130d468ee6757b5ba97f912b6c72c3f7c0d # Parent 0a096e2e51fcbb536007d94bf3edfc308e214f56 Adds $orig_remote_addr in realip module. When the realip module sets $remote_addr, the connecting IP is no longer available for logging etc. This change preserves the connecting IP as $orig_remote_addr. diff -r 0a096e2e51fc -r 4e59c130d468 src/http/modules/ngx_http_realip_module.c --- a/src/http/modules/ngx_http_realip_module.c Tue May 26 16:49:51 2015 +0300 +++ b/src/http/modules/ngx_http_realip_module.c Thu May 28 16:29:26 2015 -0500 @@ -33,6 +33,10 @@ } ngx_http_realip_ctx_t; +static ngx_int_t + ngx_http_realip_orig_remote_addr_variable(ngx_http_request_t *r, + ngx_http_variable_value_t *v, uintptr_t data); +static ngx_int_t ngx_http_realip_add_variables(ngx_conf_t *cf); static ngx_int_t ngx_http_realip_handler(ngx_http_request_t *r); static ngx_int_t ngx_http_realip_set_addr(ngx_http_request_t *r, ngx_addr_t *addr); @@ -75,7 +79,7 @@ static ngx_http_module_t ngx_http_realip_module_ctx = { - NULL, /* preconfiguration */ + ngx_http_realip_add_variables, /* preconfiguration */ ngx_http_realip_init, /* postconfiguration */ NULL, /* create main configuration */ @@ -105,6 +109,15 @@ }; +static ngx_http_variable_t ngx_http_realip_vars[] = { + + { ngx_string("orig_remote_addr"), NULL, + ngx_http_realip_orig_remote_addr_variable, 0, NGX_HTTP_VAR_NOHASH, 0 }, + + { ngx_null_string, NULL, NULL, 0, 0, 0 } +}; + + static ngx_int_t ngx_http_realip_handler(ngx_http_request_t *r) { @@ -369,6 +382,48 @@ } +static ngx_int_t +ngx_http_realip_orig_remote_addr_variable(ngx_http_request_t *r, + ngx_http_variable_value_t *v, uintptr_t data) +{ + ngx_http_realip_ctx_t *ctx; + + ctx = ngx_http_get_module_ctx(r, ngx_http_realip_module); + + if (ctx == NULL) { + v->not_found = 1; + return NGX_OK; + } + + v->len = ctx->addr_text.len; + v->valid = 1; + v->no_cacheable = 0; + v->not_found = 0; + v->data = ctx->addr_text.data; + + return NGX_OK; +} + + +static ngx_int_t +ngx_http_realip_add_variables(ngx_conf_t *cf) +{ + ngx_http_variable_t *var, *v; + + for (v = ngx_http_realip_vars; v->name.len; v++) { + var = ngx_http_add_variable(cf, &v->name, v->flags); + if (var == NULL) { + return NGX_ERROR; + } + + var->get_handler = v->get_handler; + var->data = v->data; + } + + return NGX_OK; +} + + static void * ngx_http_realip_create_loc_conf(ngx_conf_t *cf) {

7 years, 2 months

[nginx] Fixed bullying style of comments.

by Ruslan Ermilov

details: http://hg.nginx.org/nginx/rev/e034af368274 branches: changeset: 6161:e034af368274 user: Ruslan Ermilov <ru(a)nginx.com> date: Fri May 29 09:26:33 2015 +0300 description: Fixed bullying style of comments. diffstat: src/core/ngx_log.h | 8 ++++---- 1 files changed, 4 insertions(+), 4 deletions(-) diffs (39 lines): diff -r 8edec63bd14d -r e034af368274 src/core/ngx_log.h --- a/src/core/ngx_log.h Fri May 29 09:26:27 2015 +0300 +++ b/src/core/ngx_log.h Fri May 29 09:26:33 2015 +0300 @@ -111,7 +111,7 @@ void ngx_log_error_core(ngx_uint_t level /*********************************/ -#else /* NO VARIADIC MACROS */ +#else /* no variadic macros */ #define NGX_HAVE_VARIADIC_MACROS 0 @@ -123,7 +123,7 @@ void ngx_cdecl ngx_log_debug_core(ngx_lo const char *fmt, ...); -#endif /* VARIADIC MACROS */ +#endif /* variadic macros */ /*********************************/ @@ -166,7 +166,7 @@ void ngx_cdecl ngx_log_debug_core(ngx_lo arg1, arg2, arg3, arg4, arg5, arg6, arg7, arg8) -#else /* NO VARIADIC MACROS */ +#else /* no variadic macros */ #define ngx_log_debug0(level, log, err, fmt) \ if ((log)->log_level & level) \ @@ -211,7 +211,7 @@ void ngx_cdecl ngx_log_debug_core(ngx_lo #endif -#else /* NO NGX_DEBUG */ +#else /* !NGX_DEBUG */ #define ngx_log_debug0(level, log, err, fmt) #define ngx_log_debug1(level, log, err, fmt, arg1)

7 years, 2 months

[nginx] Version bump.

by Ruslan Ermilov

details: http://hg.nginx.org/nginx/rev/8edec63bd14d branches: changeset: 6160:8edec63bd14d user: Ruslan Ermilov <ru(a)nginx.com> date: Fri May 29 09:26:27 2015 +0300 description: Version bump. diffstat: src/core/nginx.h | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) diffs (14 lines): diff -r 0a096e2e51fc -r 8edec63bd14d src/core/nginx.h --- a/src/core/nginx.h Tue May 26 16:49:51 2015 +0300 +++ b/src/core/nginx.h Fri May 29 09:26:27 2015 +0300 @@ -9,8 +9,8 @@ #define _NGINX_H_INCLUDED_ -#define nginx_version 1009001 -#define NGINX_VERSION "1.9.1" +#define nginx_version 1009002 +#define NGINX_VERSION "1.9.2" #define NGINX_VER "nginx/" NGINX_VERSION #ifdef NGX_BUILD

7 years, 2 months

problems when use fastcgi_pass to deliver request to backend

by 林谡

Hi, I write a fastcgi server and use nginx to pass request to my server. It works till now. But I find a problem. Nginx always set requestId = 1 when sending fastcgi record. I was a little upset for this, cause according to fastcgi protocol, web server can send fastcgi records belonging to different request simultaneously, and requestIds are different and keep unique. I really need this feature, because requests can be handled simultaneously just over one connetion. Can I find a way out?

7 years, 2 months

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

nginx-devel May 2015