How does nginx caching handle multiple cache control headers sent from a
backend?
I had a situation where I was sending both expires and cache-control and
it seemed that the order in which they were sent controlled. I solved
that problem by ignoring the expires header.
I thought I recalled that x-accel-expires would override any other
headers regardless of order, but that doesn't seem to be the case.
Is there a priority, or does order control?
Posted at Nginx Forum: http://forum.nginx.org/read.php?2,219520,219520#msg-219520
On Tue, Feb 1, 2011 at 11:45 PM, Ryan Malayter <malayter(a)gmail.com> wrote:
>
> It does in fact work in production on nginx 0.7.6x. Below is my actual
> configuration (trimmed to the essentials and with a few substitutions
> of actual URIs).
>
Well, ngx_proxy module's directive inheritance is in action here,
which gives you nice side effects that you want :)
I'll analyze some examples here such that people *may* get some light.
[Case 1]
location /proxy {
set $a 32;
if ($a = 32) {
set $a 56;
}
set $a 76;
proxy_pass http://127.0.0.1:$server_port/$a;
}
location ~ /(\d+) {
echo $1;
}
Calling /proxy gives 76 because it works in the following steps:
1. Nginx runs all the rewrite phase directives in the order that
they're in the config file, i.e.,
set $a 32;
if ($a = 32) {
set $a 56;
}
set $a 76;
and $a gets the final value of 76.
2. Nginx traps into the "if" inner block because its condition $a = 32
was met in step 1.
3. The inner block does not has any content handler, ngx_proxy
inherits the content handler (that of ngx_proxy) in the outer scope
(see src/http/modules/ngx_http_proxy_module.c:2025).
4. Also the config specified by proxy_pass also gets inherited by the
inner "if" block (see src/http/modules/ngx_http_proxy_module.c:2015)
5. Request terminates (and the control flow never goes outside of the
"if" block).
That is, the proxy_pass directive in the outer scope will never run in
this example. It is "if" inner block that actually serves you.
Let's see what happens when we override the inner "if" block's content
handler with out own:
[Case 2]
location /proxy {
set $a 32;
if ($a = 32) {
set $a 56;
echo "a = $a";
}
set $a 76;
proxy_pass http://127.0.0.1:$server_port/$a;
}
location ~ /(\d+) {
echo $1;
}
You will get this while accessing /proxy:
a = 76
Looks counter-intuitive? Oh, well, let's see what's happening this time:
1. Nginx runs all the rewrite phase directives in the order that
they're in the config file, i.e.,
set $a 32;
if ($a = 32) {
set $a 56;
}
set $a 76;
and $a gets the final value of 76.
2. Nginx traps into the "if" inner block because its condition $a = 32
was met in step 1.
3. The inner block *does* has a content handler specified by "echo",
then the value of $a (76) gets emitted to the client side.
4. Request terminates (and the control flow never goes outside of the
"if" block), as in Case 1.
We do have a choice to make Case 2 work as we like:
[Case 3]
location /proxy {
set $a 32;
if ($a = 32) {
set $a 56;
break;
echo "a = $a";
}
set $a 76;
proxy_pass http://127.0.0.1:$server_port/$a;
}
location ~ /(\d+) {
echo $1;
}
This time, we just add a "break" directive inside the if block. This
will stop nginx from running the rest ngx_rewrite directives. So we
get
a = 56
So this time, nginx works this way:
1. Nginx runs all the rewrite phase directives in the order that
they're in the config file, i.e.,
set $a 32;
if ($a = 32) {
set $a 56;
break;
}
and $a gets the final value of 56.
2. Nginx traps into the "if" inner block because its condition $a = 32
was met in step 1.
3. The inner block *does* has a content handler specified by "echo",
then the value of $a (56) gets emitted to the client side.
4. Request terminates (and the control flow never goes outside of the
"if" block), just as in Case 1.
Okay, you see how ngx_proxy module's config inheritance among nested
locations take the key role here, and make you *believe* it works the
way that you want. But other modules (like "echo" mentioned in one of
my earlier emails) may not inherit content handlers in nested
locations (in fact, most content handler modules, including upstream
ones, don't).
And one must be careful about bad side effects of config inheritance
of "if" blocks in other cases, consider the following example:
[Case 5]
location /proxy {
set $a 32;
if ($a = 32) {
return 404;
}
set $a 76;
proxy_pass http://127.0.0.1:$server_port/$a;
more_set_headers "X-Foo: $a";
}
location ~ /(\d+) {
echo $1;
}
Here, ngx_header_more's "more_set_headers" will also be inherited by
the implicit location created by the "if" block. So you will get:
curl localhost/proxy
HTTP/1.1 404 Not Found
Server: nginx/0.8.54 (without pool)
Date: Mon, 14 Feb 2011 05:24:00 GMT
Content-Type: text/html
Content-Length: 184
Connection: keep-alive
X-Foo: 32
which may or may not what you want :)
BTW, the "add_header" directive will not emit a "X-Foo" header in this
case, and it does not mean no directive inheritance happens here, but
add_header's header filter will skip 404 responses.
You see, how tricky it is behind the scene! No wonder people keep
saying "nginx's if is evil".
Cheers,
-agentzh
Disclaimer: There may be other corner cases that I've missed here, and
other more knowledgeable people can correct me wherever I'm wrong :)
Hello,
I'm dealing with a problem. When reloading the nginx configuration,
all keepalived connections receive the TCP reset flag after I send a
HUP signal to the master process. If I comment the line responsible for
enabling the keepalive feature in the configuration, the problem
disappear (nginx version is 0.9.7).
Thanks in advance,
Jocelyn Mocquant
Posted at Nginx Forum: http://forum.nginx.org/read.php?2,197927,197927#msg-197927
I have nginx running in front of apache2/mod_wsgi and I'm not sure how
to resolve this error:
upstream timed out (110: Connection timed out) while reading response
header from upstream
any ideas on where to start?
J
I have a nginx server setup and everything is working fine. I have a
virtual server setup which allows directory listing (autoindex on;).
It appears to truncate the file names at around 50 characters.
Is it possible to disable this so that it will show the entire filename?
Or is there a way to increase it to say 100 or 150 characters?
Thanks!
Posted at Nginx Forum: http://forum.nginx.org/read.php?2,124400,124400#msg-124400
Hi,
I tried to configure nginx 0.8.29 with libatomic 1.2 (manually built and installed) by specifying
--with-libatomic=/usr/local/lib
(=> /usr/local/lib/libatomic_ops.a)
OR
--with-libatomic=../libatomic_ops-1.2
(=> source code only)
but nginx shows no information to indicate if that configuration works or not
--------------------------------
checking for OS
+ Linux 2.6.18-164.el5PAE i686
checking for C compiler ... found
+ using GNU C compiler
+ gcc version: 4.1.2 20080704 (Red Hat 4.1.2-46)
checking for gcc -pipe switch ... found
checking for gcc builtin atomic operations ... not found
checking for gcc variadic macros ... found
checking for C99 variadic macros ... found
checking for unistd.h ... found
checking for inttypes.h ... found
checking for limits.h ... found
checking for sys/filio.h ... not found
checking for sys/param.h ... found
checking for sys/mount.h ... found
checking for sys/statvfs.h ... found
checking for crypt.h ... found
checking for Linux specific features
checking for epoll ... found
checking for sendfile() ... found
checking for sendfile64() ... found
checking for sys/prctl.h ... found
checking for prctl(PR_SET_DUMPABLE) ... found
checking for sched_setaffinity() ... found
checking for crypt_r() ... found
checking for sys/vfs.h ... found
checking for poll() ... found
checking for /dev/poll ... not found
checking for kqueue ... not found
checking for crypt() ... not found
checking for crypt() in libcrypt ... found
checking for F_READAHEAD ... not found
checking for posix_fadvise() ... found
checking for O_DIRECT ... found
checking for F_NOCACHE ... not found
checking for directio() ... not found
checking for statfs() ... found
checking for statvfs() ... found
checking for dlopen() ... not found
checking for dlopen() in libdl ... found
checking for sched_yield() ... found
checking for OpenSSL library ... found
checking for zlib library ... found
checking for perl
+ perl version: v5.8.8 built for i386-linux-thread-multi
+ perl interpreter multiplicity found
creating objs/Makefile
checking for int size ... 4 bytes
checking for long size ... 4 bytes
checking for long long size ... 8 bytes
checking for void * size ... 4 bytes
checking for uint64_t ... found
checking for sig_atomic_t ... found
checking for sig_atomic_t size ... 4 bytes
checking for socklen_t ... found
checking for in_addr_t ... found
checking for in_port_t ... found
checking for rlim_t ... found
checking for uintptr_t ... uintptr_t found
checking for system endianess ... little endianess
checking for size_t size ... 4 bytes
checking for off_t size ... 8 bytes
checking for time_t size ... 4 bytes
checking for setproctitle() ... not found
checking for pread() ... found
checking for pwrite() ... found
checking for strerror_r() ... found but is not working
checking for gnu style strerror_r() ... found
checking for localtime_r() ... found
checking for posix_memalign() ... found
checking for memalign() ... found
checking for mmap(MAP_ANON|MAP_SHARED) ... found
checking for mmap("/dev/zero", MAP_SHARED) ... found
checking for System V shared memory ... found
checking for struct msghdr.msg_control ... found
checking for ioctl(FIONBIO) ... found
checking for struct tm.tm_gmtoff ... found
checking for struct dirent.d_namlen ... not found
checking for struct dirent.d_type ... found
Configuration summary
+ using PCRE library: /usr/local/src/pcre-8.00
+ using system OpenSSL library
+ md5: using OpenSSL library
+ using sha1 library: auto/lib/sha1
+ using system zlib library
nginx path prefix: "/usr/local/nginx"
nginx binary file: "/usr/local/sbin/nginx"
nginx configuration prefix: "/usr/local/nginx"
nginx configuration file: "/usr/local/nginx/nginx.conf"
nginx pid file: "/usr/local/nginx/nginx.pid"
nginx error log file: "/usr/local/nginx/logs/error.log"
nginx http access log file: "/usr/local/nginx/logs/access.log"
nginx http client request body temporary files: "/usr/local/nginx/temp/client_body"
nginx http proxy temporary files: "/usr/local/nginx/temp/proxy"
nginx http fastcgi temporary files: "/usr/local/nginx/temp/fastcgi"
The warning "checking for gcc builtin atomic operations ... not found" shows that my default compiler gcc 4.1.2 20080704 does not support atomic operations. I tried to configure CC=gcc44 CXX=g++44 to tell Nginx use newer GCC44 but it seems to ignore it.
Am I doing wrong here?
Thanks
Dinh
Posted at Nginx Forum: http://forum.nginx.org/read.php?2,28343,28343#msg-28343
(Warning this is Linux only)
Hi all, the attached path adds support for unix socket in abstract namespace.
They are special sockets without filesystem correspondence (so you can use
them without thinking about permissions or in chroot). In netstat they are
reported with a '@' prefix.
For example (using uWSGI):
uwsgi -s @funnysock
netstat -l
unix 2 [ ACC ] STREAM LISTENING 23813 @funnysock
After applying the patch you can connect to it with
uwsgi_pass unix:@funnysock;
I hope it can be useful
--
Roberto De Ioris
http://unbit.it
Hello there,
I have a remote site containing lots of Windows Clients, which have no
access to WSUS server but nginx.
What I want to achieve is ; get the WSUS updates over the nginx proxy
saving bandwith for me.
I checked this URL http://wiki.nginx.org/WSUSProxy and made the proxy
work. I am testing with a client and can see that nginx is proxy'ing the
file for the client.
But it is not caching. Everytime I start a new download for a specific
cab file, nginx downloads from the cab file from the WSUS server again.
(checking with netstat) The /cache/wsus is empty, there is no error.
Eventhough there is no /cache directory nginx is not complaining about
that. How can I open full debug logging ( HIT MISS stuff)? I am not
familiar with the syntax.
How can I validate if the static caching is going to work? I am testing
with 1 test client but I can not test it with all clients ( over 800)
OS:Centos 5.7 i386
nginx version: nginx/1.0.11
conf file is the same as wikipedia ( I am not running any wpad file,
clients have proxy set on internet explorer)
There is the logs I get, nginx is putting the some files under the
/var/tmp/nginx which is not set anywhere,
also the file is not there after the check..
==> access.log <==
192.168.CLIENT.IP- - [28/Dec/2011:18:38:12 +0200] "HEAD
/selfupdate/wuident.cab?1112281638 HTTP/1.1" 200 0 "-"
"Windows-Update-Agent" "-"
192.168.CLIENT.IP- - [28/Dec/2011:18:38:12 +0200] "HEAD
/selfupdate/WSUS3/x86/Other/wsus3setup.cab?1112281638 HTTP/1.1" 200 0
"-" "Windows-Update-Agent" "-"
192.168.CLIENT.IP- - [28/Dec/2011:18:38:13 +0200] "GET
/selfupdate/WSUS3/x86/Other/wsus3setup.cab?1112281638 HTTP/1.1" 200
20742 "-" "Windows-Update-Agent" "-"
==> error.log <==
2011/12/28 18:38:14 [warn] 25164#0: *8 a client request body is buffered
to a temporary file /var/cache/nginx/client_temp/0000000001, client:
192.168.MY.IP, server: rusty.sbs.net.tr, request: "POST
/ClientWebService/client.asmx HTTP/1.1", host: "192.168.NGINX.IP:8081"
==> access.log <==
192.168.CLIENT.IP- - [28/Dec/2011:18:38:15 +0200] "POST
/ClientWebService/client.asmx HTTP/1.1" 200 869 "-"
"Windows-Update-Agent" "-"
192.168.CLIENT.IP- - [28/Dec/2011:18:38:15 +0200] "POST
/ClientWebService/client.asmx HTTP/1.1" 200 868 "-"
"Windows-Update-Agent" "-"
==> error.log <==
2011/12/28 18:38:15 [warn] 25164#0: *8 a client request body is buffered
to a temporary file /var/cache/nginx/client_temp/0000000002, client:
192.168.MY.IP, server: rusty.sbs.net.tr, request: "POST
/ClientWebService/client.asmx HTTP/1.1", host: "192.168.NGINX.IP:8081"
==> access.log <==
192.168.CLIENT.IP- - [28/Dec/2011:18:38:20 +0200] "POST
/ClientWebService/client.asmx HTTP/1.1" 200 1267 "-"
"Windows-Update-Agent" "-"
192.168.CLIENT.IP- - [28/Dec/2011:18:38:43 +0200] "HEAD
/selfupdate/wuident.cab?1112281638 HTTP/1.1" 200 0 "-"
"Windows-Update-Agent" "-"
192.168.CLIENT.IP- - [28/Dec/2011:18:38:43 +0200] "HEAD
/selfupdate/WSUS3/x86/Other/wsus3setup.cab?1112281638 HTTP/1.1" 200 0
"-" "Windows-Update-Agent" "-"
192.168.CLIENT.IP- - [28/Dec/2011:18:38:43 +0200] "GET
/selfupdate/WSUS3/x86/Other/wsus3setup.cab?1112281638 HTTP/1.1" 200
20742 "-" "Windows-Update-Agent" "-"
192.168.CLIENT.IP- - [28/Dec/2011:18:38:44 +0200] "POST
/ClientWebService/client.asmx HTTP/1.1" 200 869 "-"
"Windows-Update-Agent" "-"
192.168.CLIENT.IP- - [28/Dec/2011:18:38:44 +0200] "POST
/ClientWebService/client.asmx HTTP/1.1" 200 868 "-"
"Windows-Update-Agent" "-"
==> error.log <==
2011/12/28 18:38:44 [warn] 25164#0: *8 a client request body is buffered
to a temporary file /var/cache/nginx/client_temp/0000000003, client:
192.168.MY.IP, server: rusty.sbs.net.tr, request: "POST
/ClientWebService/client.asmx HTTP/1.1", host: "192.168.NGINX.IP:8081"
2011/12/28 18:38:44 [warn] 25164#0: *8 a client request body is buffered
to a temporary file /var/cache/nginx/client_temp/0000000004, client:
192.168.MY.IP, server: rusty.sbs.net.tr, request: "POST
/ClientWebService/client.asmx HTTP/1.1", host: "192.168.NGINX.IP:8081"
==> access.log <==
192.168.CLIENT.IP- - [28/Dec/2011:18:38:49 +0200] "POST
/ClientWebService/client.asmx HTTP/1.1" 200 1267 "-"
"Windows-Update-Agent" "-"
Should I use a different approach regarding the wiki? ( there is not
proxy_cache stuff included on that page, I tried that but could not make
it due the lack of experience with syntax)
Posted at Nginx Forum: http://forum.nginx.org/read.php?2,220614,220614#msg-220614
Hi,
I am currently planning to use nginx on several thousand devices as a reverse-proxy caching system.
It currently work as expected (thanks Igor!), caches files as they are being requested by the devices.
The only problem we hit is when nginx starts faster than the dns sytem is available on the units. Nginx will crash saying it is unable to connect to the remote host being proxied.
1739#0: host not found in upstream "content.dev.local" in /usr/local/nginx/conf/nginx.conf 33
Staring nginx again and it work (as the DNS is now responding properly).
Any idea on how to work around this or should i fill a bug report (Nginx shouldn't crash when the remote is not available, but should try on requests to access it).
Nginx does not die when the remote drops and come back (by pulling the network cable for example). It only crash when nginx is launched and the dns sytem is not yet available.
Posted at Nginx Forum: http://forum.nginx.org/read.php?2,15995,15995#msg-15995
