how to setup as caching reverse proxy for/rewritign global URLs

[Thomas Hartmann]

Hi Francis and  and Reinis,

many thanks for the ideas and hints!
At least, I feel assured, that my idea seems to be not completely
unreasonable ;)

Crawling would be no necessity, assuming that the first client will make
'all' the requests to populate the cache.

I will try my luck.

Cheers and thanks,
  Thomas

On 04/03/2019 12.57, Reinis Rozitis wrote:
>> "caching reverse proxy" is what nginx is built for.
>>
>> "rewriting the body content" is not.
> 
> Well you can rewrite body with the sub  module ( http://nginx.org/en/docs/http/ngx_http_sub_module.html )
> 
> The only caveat is that the module  doesn't support compression (gzip) and you need to explicitly set that nginx disables it for the upstream requests.
> 
> 
> For example a config which changes "//upstreamdomain" to "//mydomain" in the proxied requests:
> 
> location / {
>     proxy_set_header Accept-Encoding "";
>     proxy_pass http://upstream.site/;
>     sub_filter_types text/html  text/css;
>     sub_filter_once off;
>     sub_filter //upstreamdomain  //mydomain;
> 
> }
> 
> Also note that you can/need to configure sub_filter_types if there are different proxied objects like javascripts etc.
> 
> 
> I use it for development environments - while with dynamic/scripting languages it's easy to have a dev domain/url with static content unless you use relative urls (which is sometimes harder/impossible if you use CDNs on different domain) it's easy for the developers to always use static production urls without the need to mangle them afterwards in the deploy process as nginx can dynamically rewrite everything
> (sub_filter supports also variables).
> 
> 
> 
>> and rewrite all subsequent request and cache them.
> 
> As Francis already wrote "probably going to involve you writing your own code" or use some external tools - nginx while can rewrite and cache the body/object it won't make any sub requests unless the client asks.
> 
> So if you want to actively mirror/cache a site you need some sort of a crawler to either request every link via nginx to fill the cache or as an example wget --mirror --convert-links --adjust-extension --page-requisites --no-parent http://somesite theoretically can store a static mirror.
> 
> rr
> 
> _______________________________________________
> nginx mailing list
> nginx at nginx.org
> http://mailman.nginx.org/mailman/listinfo/nginx
> 

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 5334 bytes
Desc: S/MIME Cryptographic Signature
URL: <http://mailman.nginx.org/pipermail/nginx/attachments/20190304/464c7d23/attachment.bin>