proxy_cache not serving file from edge server !!

[shahzaib shahzaib]

We've an origin and edge server with nginx-1.6 . Origin web-server(Located
in U.S) is configured with nginx_geo_module and edge(Local ISP) is
configured with proxy_cache in order to cache files from origin server and
serve from their lately. We're using following method for caching with
proxy_cache :-

1. client (1.1.1.1) sends mp4 request to origin webserver and geo_module in
origin checks, if the ip is 1.1.1.1 then pass that client to the edge
server using proxy_pass.

2. Edge, checks if the file is in proxy_cache than it should serve the file
locally and if file is not in proxy_cache, it'll pass back the request to
origin server and client will be served from origin server as well as
requested file will also be cached in local server, so next time the edge
will not have to pass request again to origin server and serve the same
file via locally.

But, looks like our caching is not working as expected. Our ISP is
complaining that, whenever edge server serves the file, instead of serving
that file to local client (1.1.1.1) it serves the file back to origin
server(U.S) and all outgoing bandwidth is going back to U.S instead of
local clients (Offcourse bandwidth not being saved).

So i want to ask, if the origin server is passing request to edge server,
the cached file must be served locally but the request going back to the
origin server even the cache status: HIT. Following are my configs :-

ORIGIN :-

geo $TW {
  default 0;
1.1.1.1 1;

}



server {
        listen  80;
        server_name  origin.files.com origin.gear.net  origin.gear.com;
        location / {
            root   /var/www/html/files;
            index index.html index.htm index.php;

}


location ~ \.(mp4|jpg)$ {

                proxy_set_header X-Real-IP $remote_addr;
                proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
                proxy_set_header Host $http_host;
                if ($TW) {
                        proxy_pass http://tw002.edge.com:80;
                }
                 mp4;
                root /var/www/html/files;


                expires 7d;
        valid_referers none blocked  video.pk *.video.pk blog.video.pk *.
facebook.com *.twitter.com *.files.com *.gear.net video.tv *.video.tv
videomedia.tv www.videomedia.tv embed.videomedia.tv;
                if ($invalid_referer) {
                    return   403;
                }
                }

 # pass the PHP scripts to FastCGI server listening on 127.0.0.1:9000
        location ~ \.php$ {
            root /var/www/html/files;
            fastcgi_pass   127.0.0.1:9000;
           fastcgi_index  index.php;
            fastcgi_param  SCRIPT_FILENAME
$document_root$fastcgi_script_name;
            include        fastcgi_params;
        }

        location ~ /\.ht {
            deny  all;
        }
}

EDGE :-

#proxy_ignore_headers "Set-Cookie";
proxy_cache_path /data/nginx/cache levels=1:2 keys_zone=static:100m
                        loader_threshold=200 loader_files=500 inactive=1d
                        max_size=62g;


server {

        listen       80;
        server_name  tw002.edge.com;
        root /var/www/html/files;
        location ~ \.(mp4|jpeg|jpg)$ {
               root   /var/www/html/files;
                mp4;
                try_files $uri @getfrom_origin;

            }


        location @getfrom_origin {
        proxy_pass http://origin.files.com:80;
#       proxy_cache_valid 200 302   60m;
        proxy_cache_valid  15d;
        proxy_cache static;
        proxy_cache_min_uses 1;
        }



}

Help will be highly appreciated.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx/attachments/20140701/727b74bb/attachment-0001.html>