nginx subrequests / background operations

[Maxim Dounin]

Hello!

On Sat, Jun 07, 2014 at 01:36:59AM -0400, Ameir Abdeldayem wrote:

> Hello Maxim,
> 
> Thanks for your feedback.  Yes, "proxy_cache_use_stale updating" does do a
> great job, but "the first request" is the first request per TTL, which
> relates to the problem we're facing.
> 
> We run some high-profile sites, oftentimes with low TTLs (1m or so).  The
> queries we run on the backend are very complex and time-consuming, and
> oftentimes take on the order of 10s to complete.  Because of the low TTL,
> although most users get immediate responses, the user who makes the request
> when the entry is EXPIRED has to suffer.

>From the description it looks like the real problem is that your 
backend response time is inadequate.  Instead of trying to use 
nginx cache to mitigate this, you may want to focus on the real 
problem instead, and make backend to respond in a reasonable time.

Following the "nginx mitigation" approach, a solution you may want to 
consider is a helper script to use proxy_cache_bypass to update 
cached items before they expire.

-- 
Maxim Dounin
http://nginx.org/