No revalidation when using stale-while-revalidate

[Maxim Dounin]

Hello!

On Thu, Jul 23, 2020 at 08:06:36PM +0200, Adam Volek wrote:

> We're running into some strange behaviour with the 
> stale-while-revalidate extension of the cache-control header 
> when using nginx as a reverse proxy. When
> there is a stale response in the cache with nonzero 
> stale-while-revalidate time, nginx attempts revalidation but 
> seems to ignore the upstream answer if it
> has specific status code, such as 404 or 500, and server a stale 
> response to the client. Other response codes such as 200 or 410 
> don't trigger this
> behaviour. Is this intended, and if so, is there a way to 
> configure nginx to treat 404 as any other response?

As long as the response returned isn't cacheable (either 
as specified in the response Cache-Control / Expires 
headers, or per proxy_cache_valid), nginx won't put 
the response into cache and will continue serving previously 
cached response till stale-while-revalidate timeout expires.

Most likely "specific status code" in your tests in fact means 
responses returned by your upstream server without Cache-Control 
headers, and hence not cached by nginx.

> I understand that this behaviour might be desirable in some 
> situations (especially for the responses with 5xx status codes), 
> but in our case, if the
> upstream return a 404 response, we would want the cache to start 
> returning it as well as soon as the revalidation is finished.

As long as the above analysis is correct, the solution is simple: 
make sure responses you want nginx to cache are cacheable, that 
is, make sure appropriate headers are present or configure 
proxy_cache_valid (http://nginx.org/r/proxy_cache_valid).

-- 
Maxim Dounin
http://mdounin.ru/