proxy_cache_background_update ignores regular expression match when updating

[Roman Arutyunyan]

Hello Richard,

On Tue, Dec 04, 2018 at 06:57:15PM +0100, Richard Stanway via nginx wrote:
> Hello,
> I'm running into an issue where a proxied location with a regular
> expression match does not correctly update the cache when using
> proxy_cache_background_update. The update request to the backend seems
> to be missing the captured parameters from the regex. I've created a
> small test case that demonstrates this in nginx 1.15.7. Hopefully I'm
> not missing anything, I checked the docs and didn't seem to find
> anything that would explain this behavior.

This indeed looks like a bug in the way nginx creates cloned subrequests.
Because of it unnamed captures matched in the parent request are not
available in the subrequest.

A simple workaround is to use named captures instead.

[..]

>         location ~ /test/(regular|expression)$ {
>                 proxy_pass http://127.0.0.1:8010/test/$1;

This should solve the issue:

         location ~ /test/($<name>regular|expression)$ {
                 proxy_pass http://127.0.0.1:8010/test/$name;


>                 proxy_cache test;
>                 proxy_cache_background_update on;
>                 proxy_cache_use_stale updating;
>                 proxy_cache_valid 10s;
>         }
> }
> 
> Initial testing with proxy_cache_background_update off. Log excerpts
> show requests to both servers.
> 
> First request (one to frontend, one to backend as expected):
> 127.0.0.1 - - [04/Dec/2018:17:42:31 +0000] "GET /test/regular
> HTTP/1.0" 200 8 "-" "curl/7.52.1" "-"
> 127.0.0.1 - - [04/Dec/2018:17:42:31 +0000] "GET /test/regular
> HTTP/1.1" 200 8 "-" "curl/7.52.1" "-"
> 
> Second request (served from frontend cache, all good):
> 127.0.0.1 - - [04/Dec/2018:17:42:35 +0000] "GET /test/regular
> HTTP/1.1" 200 8 "-" "curl/7.52.1" "-"
> 
> Third request (cache expired, so a new request to backend, also good):
> 127.0.0.1 - - [04/Dec/2018:17:43:14 +0000] "GET /test/regular
> HTTP/1.0" 200 8 "-" "curl/7.52.1" "-"
> 127.0.0.1 - - [04/Dec/2018:17:43:14 +0000] "GET /test/regular
> HTTP/1.1" 200 8 "-" "curl/7.52.1" "-"
> 
> After setting proxy_cache_background_update on, every request tries to
> do a background update with the wrong URL once the content is expired.
> The stale content is still served in the meantime.
> 127.0.0.1 - - [04/Dec/2018:17:44:01 +0000] "GET /test/ HTTP/1.0" 403
> 153 "-" "curl/7.52.1" "-"
> 127.0.0.1 - - [04/Dec/2018:17:44:01 +0000] "GET /test/regular
> HTTP/1.1" 200 8 "-" "curl/7.52.1" "-"
> 
> 127.0.0.1 - - [04/Dec/2018:17:44:15 +0000] "GET /test/ HTTP/1.0" 403
> 153 "-" "curl/7.52.1" "-"
> 127.0.0.1 - - [04/Dec/2018:17:44:15 +0000] "GET /test/regular
> HTTP/1.1" 200 8 "-" "curl/7.52.1" "-"
> 
> 127.0.0.1 - - [04/Dec/2018:17:44:17 +0000] "GET /test/ HTTP/1.0" 403
> 153 "-" "curl/7.52.1" "-"
> 127.0.0.1 - - [04/Dec/2018:17:44:17 +0000] "GET /test/regular
> HTTP/1.1" 200 8 "-" "curl/7.52.1" "-"
> 
> 127.0.0.1 - - [04/Dec/2018:17:44:19 +0000] "GET /test/ HTTP/1.0" 403
> 153 "-" "curl/7.52.1" "-"
> 127.0.0.1 - - [04/Dec/2018:17:44:19 +0000] "GET /test/regular
> HTTP/1.1" 200 8 "-" "curl/7.52.1" "-"
> 
> 127.0.0.1 - - [04/Dec/2018:17:44:21 +0000] "GET /test/ HTTP/1.0" 403
> 153 "-" "curl/7.52.1" "-"
> 127.0.0.1 - - [04/Dec/2018:17:44:21 +0000] "GET /test/regular
> HTTP/1.1" 200 8 "-" "curl/7.52.1" "-"
> 
> Is this a bug or am I misunderstanding how this is supposed to work?
> _______________________________________________
> nginx mailing list
> nginx at nginx.org
> http://mailman.nginx.org/mailman/listinfo/nginx

-- 
Roman Arutyunyan