nginx-1.4 proxy requests being continious

[Daniel Black]

Just to prove I'm not making it up (even though I'm having a hard time replicating it).

log_format extended '$remote_addr - $remote_user [$time_local]  '
                    '"$request" $status $request_time $body_bytes_sent '
                    '$upstream_cache_status $upstream_addr $upstream_status $upstream_response_time'
                    '"$http_referer" "$http_user_agent"';
          

length of log line 3412217 characters (is that a record?)
58.169.18.35 - - [08/May/2013:19:58:13 -0400]  "GET //img/covers/medium/587/9781844454581.jpg HTTP/1.1" 499 100.820 0 - 192.168.
129.90:80, 192.168.129.90:80, 192.168.129.90:80, 192.168.129.90:80, 192.168.129.90:80, 192.168.129.90:80, 192.168.129.90:80, 192.168.129.90:80, 192.168.129.90:80, 192.168.129.90:80, 192.168.129.90:80, 192.168.129.90:80, 192.168.129.90:80, 192.168.129.90:80, 192.168.129.90:80, 192.168.129.90:80, 192.168.129.90:80, 192.168.129.90:80, 192.168.129.90:80, 192.168.129.90:80, 192.168.129.90:80, 192.168.129.90:80, 192.168.129.90:80, 192.168.129.90:80, 192.168.129.90:80 (many many pages)... 404, 404, 404, 404, 404, 404, 404, 404, 404, 404, 404, 404, 404, 404, 404, 404, 404, 404, 404, 404, 404, 404, 404, 404, 404, 404, 404, 404, 404, 404, 404, 404, 404, 404, 404, 404, 404, 404, 404, 404, 404, 404, 404, 404, 404, 404, 404, 404, 404, 404, 404, 404, 404, 404, 404..........., - 0.014,  0.001, 0.000, 0.001, 0.001, 0.000, 0.001, 0.001, 0.000, 0.001, 0.001, 0.001, 0.001, 0.001, 0.001.. , - "-" "Wget/1.13.4 (linux-gnu)"



192.168.131.254 - - [08/May/2013:19:58:13 -0400]  "GET //img/covers/medium/587/9781844454581.jpg HTTP/1.0" 404 0.000 169 "-" "IMGWEB"
192.168.131.254 - - [08/May/2013:19:58:13 -0400]  "GET //img/covers/medium/587/9781844454581.jpg HTTP/1.0" 404 0.000 169 "-" "IMGWEB"
192.168.131.254 - - [08/May/2013:19:58:13 -0400]  "GET //img/covers/medium/587/9781844454581.jpg HTTP/1.0" 404 0.000 169 "-" "IMGWEB"
192.168.131.254 - - [08/May/2013:19:58:13 -0400]  "GET //img/covers/medium/587/9781844454581.jpg HTTP/1.0" 404 0.000 169 "-" "IMGWEB"
192.168.131.254 - - [08/May/2013:19:58:13 -0400]  "GET //img/covers/medium/587/9781844454581.jpg HTTP/1.0" 404 0.000 169 "-" "IMGWEB"  
192.168.131.254 - - [08/May/2013:19:58:13 -0400]  "GET //img/covers/medium/587/9781844454581.jpg HTTP/1.0" 404 0.000 169 "-" "IMGWEB"
192.168.131.254 - - [08/May/2013:19:58:13 -0400]  "GET //img/covers/medium/587/9781844454581.jpg HTTP/1.0" 404 0.000 169 "-" "IMGWEB"
192.168.131.254 - - [08/May/2013:19:58:13 -0400]  "GET //img/covers/medium/587/9781844454581.jpg HTTP/1.0" 404 0.000 169 "-" "IMGWEB"
192.168.131.254 - - [08/May/2013:19:58:13 -0400]  "GET //img/covers/medium/587/9781844454581.jpg HTTP/1.0" 404 0.000 169 "-" "IMGWEB"
192.168.131.254 - - [08/May/2013:19:58:13 -0400]  "GET //img/covers/medium/587/9781844454581.jpg HTTP/1.0" 404 0.000 169 "-" "IMGWEB"
192.168.131.254 - - [08/May/2013:19:58:13 -0400]  "GET //img/covers/medium/587/9781844454581.jpg HTTP/1.0" 404 0.000 169 "-" "IMGWEB"
192.168.131.254 - - [08/May/2013:19:58:13 -0400]  "GET //img/covers/medium/587/9781844454581.jpg HTTP/1.0" 404 0.000 169 "-" "IMGWEB

----- Original Message -----
> A request for /img/file_doesnt_exist.jpg results in the backend server
> (192.168.129.90) getting continuous requests for the same file (which
> doesn't exist there either so 404 each time), while the original
> requester waits and nginx keeps asking the backend the same.
> 
> I'm using the nginx-1.4.1 from the debian squeeze repository.
> 
> Is there a better way do to this config? The aim for for all web
> servers to have the same config so a resource that aren't synced yet
> still get served a response if it exists somewhere but without the
> requests ending up in a circular loop.
> 
> My current, hopefully not too cut down, config is:
> 
> upstream imgweb_other {
> server 192.168.129.90;
> server 173.230.136.6 backup;
> }
> 
> server {
> 
> proxy_read_timeout 15;
> proxy_connect_timeout 3;
> proxy_next_upstream error timeout invalid_header http_500 http_502
> http_503 http_504 http_404;
> 
> location ~ ^/img/(.*)
> {
> expires 2592000;
> add_header Cache-Control public;
> alias /var/www/live_site_resources/$1;
> error_page 404 = @imgweb_other;
> }
> 
> location @imgweb_other {
> # we only want to fallback once so use user_agent as a flag
> if ( $http_user_agent = IMGWEB ) {
> return 404;
> }
> proxy_pass http://imgweb_other;
> proxy_set_header Host $host;
> proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
> proxy_set_header X-Real-IP $remote_addr;
> proxy_set_header User-Agent IMGWEB;
> }
> 
> }
> 
> _______________________________________________
> nginx mailing list
> nginx at nginx.org
> http://mailman.nginx.org/mailman/listinfo/nginx

-- 
-- 
Daniel Black, Engineer @ Open Query (http://openquery.com)
Remote expertise & maintenance for MySQL/MariaDB server environments.