fastcgi_pass and caching with request rewrite

[0815 at lenhardt.in]

On 15.10.20 18:25, 0815 at lenhardt.in wrote:
> Hi!
> 
> This is the first time I am doing rewrites with a fastcgi backend 
> (php-fpm).
> 
> This is my fpm location which is working fine on a ubuntu 18.04 VM:
> 
>         # fpm-config
>          location ~ \.php$ {
> 
> 
>                  include snippets/fastcgi-php.conf;
>                  fastcgi_pass unix:/run/php/php-fpm-typo3.sock;
>                  fastcgi_param   HTTPS 'on';
> 
>                  fastcgi_read_timeout 240;
>                  # cache settings
>          fastcgi_cache            html_cache;
>          fastcgi_cache_valid        200 404 60m;
>          fastcgi_cache_bypass        $no_cache_allowed;
> 
>          fastcgi_cache_bypass        $cookie_be_typo_user;
>          fastcgi_cache_bypass        $eID_search_no_cache;
> 
> 
>          }
> 
> 
> in the http section:
> 
> http {
>      ...
>      ...
> 
>          fastcgi_cache_path /var/cache/nginx/proxy_cache levels=1:2
> keys_zone=html_cache:10m max_size=5120m inactive=60m use_temp_path=off;
> 
>          fastcgi_cache_key "$scheme$request_method$host$request_uri";
> 
>          fastcgi_buffer_size          128k;
>          fastcgi_buffers              4 256k;
>          fastcgi_busy_buffers_size    256k;
> 
> 
>      ...
> }
> 
> 
> After adding this location in the fpm vhost the caching is not working
> as expected and delivers the same page for each request:
> 
> 
>         location ~ ^/hotel/([0-9]+)/.* {
>                  rewrite ^/hotel/([0-9]+)/.*
> /index.php?id=3238&user_kuoniibefe_pi3[iff]=$1 break;
> 
> 
>                  include snippets/fastcgi-php.conf;
>                  fastcgi_param  REQUEST_URI
> $fastcgi_script_name?$query_string;
>                  fastcgi_pass unix:/run/php/php-fpm-typo3.sock;
>                  fastcgi_param   HTTPS 'on';
> 
>                  fastcgi_read_timeout 240;
>                  # cache settings
>                  include /etc/nginx/conf.d/fcgi_cache_settings.inc;
> 
>          }
> 
> The location block is used to rewrite URLs like
> /hotel/1234/bla?x=1&...
> 
> to
> /index.php?h=1234&.....
> 
> but in this case the cache key is missing all the query parameters:
> 
> 2020/10/15 17:27:51 [debug] 825#825: *1 http2 request line: "GET 
> /hotel/3/bl-zzz-xxxccccresort-spa-antigua/?ddate=2020-11-01&rdate=2021-04-30&adult=2&aid=3&dur=13,15&ibe=package 
> HTTP/2.0"
> 2020/10/15 17:27:51 [debug] 825#825: *1 http cache key: 
> "httpsGETdev1.restplatzboerse.at"
> 
> 
> with other URLs the cache key is correct:
> 2020/10/15 17:27:30 [debug] 32644#32644: *761 http2 request line: "GET 
> /index.php?id=3238&user_kuoniibefe_pi3[iff]=3&ddate=2020-11-01&rdate=2021-04-30&adult=2&aid=3&dur=13,15&ibe=package 
> HTTP/2.0"
> 2020/10/15 17:27:30 [debug] 32644#32644: *761 http cache key: 
> "httpsGETdev1.restplatzboerse.at/index.php?id=3238&user_kuoniibefe_pi3[iff]=3&ddate=2020-11-01&rdate=2021-04-30&adult=2&aid=3&dur=13,15&ibe=package" 
> 
> 
> 
> My question is why is the cache key incomplete with request that are 
> rewritten? How could I get the correct cache key after rewriting the 
> /hotel/1234/bla request.
> 
> The fastcgi_cache_key parameter in the http block is only once allowed, 
> so I can not use another fastcgi_cache_key parameter in the rewrite 
> location block.


I found a include file in my vhost that is removing some unwanted query 
parameters (to be not part of the cache key). That include was also in 
the /hotel rewrite location and removed all query parameters. So I 
removed it and the cache key looks now ok.

        set $c_uri $args; # e.g. "param1=true&param4=false"

         # get url path into variable for cache_key
         set $u_path $request_uri;
         if ($request_uri ~ (.*)(\?.*)) {
            set $u_path $1;
         }

         # remove unwanted get params
         if ($c_uri ~ (.*)(?:&|^)utm_source=[^&]*(.*)) {
            set $c_uri $1$2;
         }

         if ($c_uri ~ (.*)(?:&|^)utm_term=[^&]*(.*)) {
            set $c_uri $1$2;
         }

         if ($c_uri ~ (.*)(?:&|^)utm_campaign=[^&]*(.*)) {
            set $c_uri $1$2;
         }

         set $c_uri $is_args$c_uri;

         if ($c_uri ~ ^\?$) {
                 set $c_uri "";
         }

         # finally we have stripped out utms and has nice cache key
         set $c_uri $u_path$c_uri;

         # DEBUG (disable in prd env)
         #add_header X-RURI $request_uri;
         #add_header X-CACHE-KEY $c_uri;
         #add_header X-U-Path $u_path;

         # set cleaned $c_uri as cache_key
         fastcgi_cache_key "$scheme$request_method$host$c_uri";


2020/10/15 18:42:24 [debug] 10179#10179: *66 http2 request line: "GET 
/hotel/3/bl-zzz-xxxccccccresort-spa-antigua/?ddate=2020-11-01&rdate=2021-04-30&adult=2&aid=3&dur=13,15&ibe=package 
HTTP/2.0"
2020/10/15 18:42:24 [debug] 10179#10179: *66 http cache key: 
"httpsGETdev1.restplatzboerse.at/hotel/3/bl-zzz-xxxccccccresort-spa-antigua/?ddate=2020-11-01&rdate=2021-04-30&adult=2&aid=3&dur=13,15&ibe=package"


br,
Marco

> 
> br,
> Marco
> 
> _______________________________________________
> nginx mailing list
> nginx at nginx.org
> http://mailman.nginx.org/mailman/listinfo/nginx