Hi,
There is a problem with ngx_http_slice_module when it is used together
with an internal redirection to a named location. It results in Nginx
workers caching incorrect content and spinning in an infinite loop.
Consider the following fragment:
location /foo {
error_page 550 = @bar;
return 550;
}
location @bar {
slice ...;
proxy_pass ...;
proxy_cache ...;
...
}
In ngx_http_slice_body_filter(), the slice context is associated with
the subrequest:
ngx_http_set_ctx(sr, ctx, ngx_http_slice_filter_module);
However, the location re-match is triggered for the subrequests trying
to fetch each slice. In the ngx_http_named_location() function:
/* clear the modules contexts */
ngx_memzero(r->ctx, sizeof(void *) * ngx_http_max_module);
So, the slice context gets lost. There are several ways to fix this.
Unless you want an API to preserve the context, one simple fix would be
to get the context via r->parent (if ctx == NULL && r != r->main).
What would be the preferred way to fix this?
Thanks.
--
Mindaugas
Hi,
This patch adds a build option to the server_tokens directive. When build is specified the Server header will have a value of NGINX_VER_BUILD. Also the tail of built in error pages will be NGINX_VER_BUILD. off and on are retained as valid values and behave as they always have.
This makes it possible to specify semi-custom server tokens of the form "nginx/<version> (<value>)". <value> is specified with the --build flag for ./configure.
A potential use case is including the name of a custom build for example with --build="super-nginx/0.1". This would give: nginx/1.9.15 (super-nginx/0.1).
Another potential use case of this feature is to include the git revision of a parent build with --build="`git rev-parse --short HEAD`". This would give: nginx/1.9.15 (c3ebabd). In that case the feature can be used to enhance troubleshooting and/or provide more information to end users.
Kind Regards,
Tom Thorogood.
Hi,
I run NGINX in an embedded environment with limited resources. We have NGINX configured with a small number of maximum connections (worker_connections is 16). I’ve found the ngx_drain_connection() logic uses a hard-coded value to limit the number of reaped reusable connections. This works fine with the default 512, but is too aggressive for small number of connections and ends up reaping all reusable connections. These include new connections just accepted on the previous event loop cycle (which had not yet had a chance to receive a request) at the head of the list.
The below patch switches to the minimum of 32 or 1/4 of the number of connections. This should be less aggressive, even when the number of connections is slightly larger than 32, e.g. 48 or 64.
Joel
# HG changeset patch
# User Joel Cunningham <joel.cunningham(a)me.com>
# Date 1481145862 21600
# Wed Dec 07 15:24:22 2016 -0600
# Node ID b103dddcee7322522651f9aca764d499d5822ac1
# Parent 75dbab4ea930bc73cca98d183c2f556eb5125462
Fix drain logic for small number of connections
This commit fixes the ngx_drain_connections logic when maximum number of
connections are small (16/32/64). Using a hardcoded value of 32 when
the number of connections is small is overlly aggressive and will result
in repeaing the entire (or large portion of) the reusable_connections_queue,
which includes at the tail newly accepted connections that have not received
a request yet
The logic is updated to use the minimum of 1/4 the number of connections
or 32, which ever is smaller
diff -r 75dbab4ea930 -r b103dddcee73 src/core/ngx_connection.c
--- a/src/core/ngx_connection.c Mon Nov 21 16:03:42 2016 +0300
+++ b/src/core/ngx_connection.c Wed Dec 07 15:24:22 2016 -0600
@@ -1232,7 +1232,7 @@
ngx_queue_t *q;
ngx_connection_t *c;
- for (i = 0; i < 32; i++) {
+ for (i = 0; i < ngx_min(32, ngx_cycle->connection_n / 4); i++) {
if (ngx_queue_empty(&ngx_cycle->reusable_connections_queue)) {
break;
}
Hi
I am a newbie to nginx.
I am integrating a public key hardware accelerator in OpenSSL using engine interface.
The engine is async capable.
Recently openssl-1.1.0 has added support for ASYNC_JOB.
IIUC, nginx would also require changes in order to do SSL handshake in async way.
Any pointers where can I get the nginx code changes done for async openssl
Regards
Vakul
Hi,
there is map:
map $http_host $hosts {
hostnames;
default 0;
.example.com 1;
.example.net 1;
...
.example.org 1;
}
How to read the value of $hosts in C module code?
How to make it right, that's mean do not evaluate it more than once per request?
I want to do some modifications of body response based on 'type' of host and map
would simplify the code, at least hope so.
Hi,
We use Nginx to locally cache large binary content. Cache efficiency
is of high importance, so the slice filter is excellent for this
purpose! Here's a (very) short relevant configuration snippet for a
particular location we use:
slice 1m;
proxy_cache data;
proxy_pass http://$host$request_uri;
proxy_cache_key $uri$slice_range;
proxy_set_header Range $slice_range;
Since we've adopted the slice filter, we've been seeing a lot of 'etag
mismatch in slice response' errors because the upstream provider
serves content from multiple CDNs that provide the exact same URI
namespace. For example, consider the following resource:
/tpr/hs/data/7c/bf/7cbf4dbce9233a57ff6c1cbf84db77d8
Level3 serves this resource with ETag 74bc439-54261c855fd64.
Akamai serves this resource with ETag
1273157742232ec1b8d3afc4c41a2a9c:1480362585.
Since we value cache efficiency over anything else, adding the host to
the cache key is not an option. The alternative is pinning proxy_pass
to one specific CDN, but that approach has several other downsides.
So, I decided to try my hand at (slightly) altering the filter to make
it more configurable, I've posted the result below. Further
improvements might include:
- A (regex?) transformation for the ETag field, but that wouldn't be
useful in the scenario above
- exporting the ETag field out of the slice filter as a variable,
effectively making the cache mutable
'slice_etag' might be slightly too obscure to people unfamiliar with
the internals, perhaps 'slice_match_etag' would be more descriptive,
but I'll leave that up to the maintainers.
I've seen 'ngx_flag_t' in other *_conf_t typedefs, I hope that's
appropriate. I'm not a C programmer, but I did the best I could. Any
feedback is highly appreciated.
Take care,
Timo
# HG changeset patch
# User Timo Beckers <timo(a)incline.eu>
# Date 1482843958 -3600
# Tue Dec 27 14:05:58 2016 +0100
# Node ID 9e8c70191f0a9db9ad08bfb959fc1b1404d312ec
# Parent 54cf51c4f07add000b824036d4dce4bc60e67bcf
Slice filter: add slice_etag config var to disable etag handling
When caching 206 requests from multiple CDNs that serve identical resources
but have a different ETag scheme, this setting can be used to disable the
ETag checking during processing of response headers.
This comes in handy when seeing an excessive amount of 'etag mismatch in
slice response' errors and the cache is short-lived enough to ensure
clients don't receive stale content.
diff -r 54cf51c4f07a -r 9e8c70191f0a
src/http/modules/ngx_http_slice_filter_module.c
--- a/src/http/modules/ngx_http_slice_filter_module.c Mon Dec 26
14:27:05 2016 +0300
+++ b/src/http/modules/ngx_http_slice_filter_module.c Tue Dec 27
14:05:58 2016 +0100
@@ -12,6 +12,7 @@
typedef struct {
size_t size;
+ ngx_flag_t etag;
} ngx_http_slice_loc_conf_t;
@@ -48,6 +49,13 @@
static ngx_command_t ngx_http_slice_filter_commands[] = {
+ { ngx_string("slice_etag"),
+ NGX_HTTP_MAIN_CONF|NGX_HTTP_SRV_CONF|NGX_HTTP_LOC_CONF|NGX_CONF_TAKE1,
+ ngx_conf_set_flag_slot,
+ NGX_HTTP_LOC_CONF_OFFSET,
+ offsetof(ngx_http_slice_loc_conf_t, etag),
+ NULL },
+
{ ngx_string("slice"),
NGX_HTTP_MAIN_CONF|NGX_HTTP_SRV_CONF|NGX_HTTP_LOC_CONF|NGX_CONF_TAKE1,
ngx_conf_set_size_slot,
@@ -123,22 +131,26 @@
return NGX_ERROR;
}
- h = r->headers_out.etag;
+ slcf = ngx_http_get_module_loc_conf(r, ngx_http_slice_filter_module);
- if (ctx->etag.len) {
- if (h == NULL
- || h->value.len != ctx->etag.len
- || ngx_strncmp(h->value.data, ctx->etag.data, ctx->etag.len)
- != 0)
- {
- ngx_log_error(NGX_LOG_ERR, r->connection->log, 0,
- "etag mismatch in slice response");
- return NGX_ERROR;
+ if (slcf->etag) {
+ h = r->headers_out.etag;
+
+ if (ctx->etag.len) {
+ if (h == NULL
+ || h->value.len != ctx->etag.len
+ || ngx_strncmp(h->value.data, ctx->etag.data, ctx->etag.len)
+ != 0)
+ {
+ ngx_log_error(NGX_LOG_ERR, r->connection->log, 0,
+ "etag mismatch in slice response");
+ return NGX_ERROR;
+ }
}
- }
- if (h) {
- ctx->etag = h->value;
+ if (h) {
+ ctx->etag = h->value;
+ }
}
if (ngx_http_slice_parse_content_range(r, &cr) != NGX_OK) {
@@ -157,8 +169,6 @@
"http slice response range: %O-%O/%O",
cr.start, cr.end, cr.complete_length);
- slcf = ngx_http_get_module_loc_conf(r, ngx_http_slice_filter_module);
-
end = ngx_min(cr.start + (off_t) slcf->size, cr.complete_length);
if (cr.start != ctx->start || cr.end != end) {
@@ -480,6 +490,7 @@
}
slcf->size = NGX_CONF_UNSET_SIZE;
+ slcf->etag = NGX_CONF_UNSET;
return slcf;
}
@@ -492,6 +503,7 @@
ngx_http_slice_loc_conf_t *conf = child;
ngx_conf_merge_size_value(conf->size, prev->size, 0);
+ ngx_conf_merge_off_value(conf->etag, prev->etag, 1);
return NGX_CONF_OK;
}
details: http://hg.nginx.org/nginx/rev/4591da489a30
branches:
changeset: 6864:4591da489a30
user: Maxim Dounin <mdounin(a)mdounin.ru>
date: Tue Dec 27 17:23:07 2016 +0300
description:
nginx-1.11.8-RELEASE
diffstat:
docs/xml/nginx/changes.xml | 107 +++++++++++++++++++++++++++++++++++++++++++++
1 files changed, 107 insertions(+), 0 deletions(-)
diffs (117 lines):
diff --git a/docs/xml/nginx/changes.xml b/docs/xml/nginx/changes.xml
--- a/docs/xml/nginx/changes.xml
+++ b/docs/xml/nginx/changes.xml
@@ -5,6 +5,113 @@
<change_log title="nginx">
+<changes ver="1.11.8" date="27.12.2016">
+
+<change type="feature">
+<para lang="ru">
+директива absolute_redirect.
+</para>
+<para lang="en">
+the "absolute_redirect" directive.
+</para>
+</change>
+
+<change type="feature">
+<para lang="ru">
+параметр escape директивы log_format.
+</para>
+<para lang="en">
+the "escape" parameter of the "log_format" directive.
+</para>
+</change>
+
+<change type="feature">
+<para lang="ru">
+проверка клиентских SSL-сертификатов в модуле stream.
+</para>
+<para lang="en">
+client SSL certificates verification in the stream module.
+</para>
+</change>
+
+<change type="feature">
+<para lang="ru">
+директива ssl_session_ticket_key поддерживает
+шифрование TLS session tickets с помощью AES256
+при использовании с 80-байтными ключами.
+</para>
+<para lang="en">
+the "ssl_session_ticket_key" directive supports
+AES256 encryption of TLS session tickets
+when used with 80-byte keys.
+</para>
+</change>
+
+<change type="feature">
+<para lang="ru">
+поддержка vim-commentary в скриптах для vim.<br/>
+Спасибо Armin Grodon.
+</para>
+<para lang="en">
+vim-commentary support in vim scripts.<br/>
+Thanks to Armin Grodon.
+</para>
+</change>
+
+<change type="bugfix">
+<para lang="ru">
+рекурсия при получении значений переменных не ограничивалась.
+</para>
+<para lang="en">
+recursion when evaluating variables was not limited.
+</para>
+</change>
+
+<change type="bugfix">
+<para lang="ru">
+в модуле ngx_stream_ssl_preread_module.
+</para>
+<para lang="en">
+in the ngx_stream_ssl_preread_module.
+</para>
+</change>
+
+<change type="bugfix">
+<para lang="ru">
+если сервер, описанный в блоке upstream в модуле stream,
+был признан неработающим, то после истечения fail_timeout он
+признавался работающим только после завершения тестового соединения;
+теперь достаточно, чтобы соединение было успешно установлено.
+</para>
+<para lang="en">
+if a server in an upstream in the stream module failed,
+it was considered alive only when a test connection sent
+to it after fail_timeout was closed;
+now a successfully established connection is enough.
+</para>
+</change>
+
+<change type="bugfix">
+<para lang="ru">
+nginx/Windows не собирался с 64-битным Visual Studio.
+</para>
+<para lang="en">
+nginx/Windows could not be built with 64-bit Visual Studio.
+</para>
+</change>
+
+<change type="bugfix">
+<para lang="ru">
+nginx/Windows не собирался с OpenSSL 1.1.0.
+</para>
+<para lang="en">
+nginx/Windows could not be built with OpenSSL 1.1.0.
+</para>
+</change>
+
+</changes>
+
+
<changes ver="1.11.7" date="13.12.2016">
<change type="change">