Forward single request to upstream server via proxy_store !!

[Reinis Rozitis]

> 3 clients requested for test.mp4 (file size is 4mb) --> nginx --> file not 
> existed (proxy_store) --> varnish --> backend (fetch the file from 
> origin).
> When nginx proxied these three requests subsequently towards the varnish,, 
> despite of filling 4mb of tmp dir it was filled with 12MB which means 
> nginx is proxying all three requests towards the varnish server and 
> creating tmp files as long as the file is not downloaded. (The method was 
> failed)

That is expected, this setup only “guards” the content server.



> Now varnish also has a flaw to send subsequent requests for same file 
> towards the nginx i.e

It's not a really flaw but default behaviour (different urls mean different 
content/cachable objects), but of course you can implement your own 
scenario:


By adding:

sub vcl_recv {
    set req.url = regsub(req.url, "\?.*", "");
}

will remove all the the arguments behind ? from the uri when forwarding to 
the content backend.


For static content I usually also add something like:

unset req.http.Cookie;
unset req.http.Accept-Encoding;
unset req.http.Cache-Control;

to normalise the request and so varnish doesnt try to cache different 
versions of the same object.


If you insist on using proxy_store I would probably also add 
proxy_ignore_client_abort on;  ( 
http://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_ignore_client_abort 
 ) to the nginx configuration. So the requests don't get repeated if the 
client closes/aborts the request early etc.

rr