No revalidation when using stale-while-revalidate

[Roman Arutyunyan]

Hi,

On Mon, Jul 27, 2020 at 04:42:00AM +0300, Maxim Dounin wrote:
> Hello!
> 
> On Fri, Jul 24, 2020 at 03:21:31PM +0200, Adam Volek wrote:
> 
> > On 24. 07. 20 4:33, Maxim Dounin wrote:
> > > As long as the response returned isn't cacheable (either
> > > as specified in the response Cache-Control / Expires
> > > headers, or per proxy_cache_valid), nginx won't put
> > > the response into cache and will continue serving previously
> > > cached response till stale-while-revalidate timeout expires.
> > >
> > > Most likely "specific status code" in your tests in fact means
> > > responses returned by your upstream server without Cache-Control
> > > headers, and hence not cached by nginx.
> > 
> > This is not the case as far as I can tell. In our tests, the upstream server was set up to send these two responses, the 204 first, and then then the 404:
> > 
> > HTTP/1.1 204 No Content
> > Date: Fri, 24 Jul 2020 11:32:33 GMT
> > Connection: keep-alive
> > cache-control: max-age=5, stale-while-revalidate=10
> > 
> > HTTP/1.1 404 Not Found
> > Date: Fri, 24 Jul 2020 11:32:35 GMT
> > Content-Type: text/plain
> > Connection: close
> > cache-control: max-age=5, stale-while-revalidate=10
> > 
> > In this scenario, nginx returns fresh 204 for five seconds and then it returns stale 204 for ten seconds even though it's attempting revalidation according
> > to access log at the upstream server. If we send the following 410 response instead of 404 however, nginx behaves as we would expect: it returns the fresh
> > 204 for five seconds, then it revalidates it almost instantly and starts returning the fresh 410:
> > 
> > HTTP/1.1 410 Gone
> > Date: Fri, 24 Jul 2020 11:41:56 GMT
> > Content-Type: text/plain
> > Connection: close
> > cache-control: max-age=5, stale-while-revalidate=10
> 
> You are right, this seems to be an incorrect behaviour of 
> stale-while-revalidate / stale-if-error handling.
> 
> Internally, stale-if-error (and stale-while-revalidate) currently 
> behave as if "proxy_cache_use_stale" was set with all possible 
> flags (http://nginx.org/r/proxy_cache_use_stale) when handling 
> upstream server responses.  Notably this includes http_403, 
> http_404, and http_429 flags, and this causes the effect you 
> observe.
> 
> This probably should be fixed.
> 
> Just in case, the following configuration can be used to reproduce 
> the issue within nginx itself:
> 
>     proxy_cache_path cache keys_zone=one:1m;
> 
>     server {
>         listen 8080;
> 
>         location / {
>             proxy_pass http://127.0.0.1:8081;
>             proxy_cache one;
>             add_header X-Cache-Status $upstream_cache_status always;
>         }
>     }
> 
>     server {
>         listen 8081;
> 
>         location / {
>             add_header cache-control "max-age=5, stale-while-revalidate=10"
>                        always;
> 
>             if ($connection = "3") {
>                 return 204;
>             }
> 
>             return 404;
>         }
>     }

The fix was committed:

https://hg.nginx.org/nginx/rev/7015f26aef90

-- 
Roman Arutyunyan