Question about proxy cache when it expires

[Arvind Jayaprakash]

On May 12, Igor Sysoev wrote:
>On Tue, May 12, 2009 at 10:38:04AM +0200, J?r?me Loyet wrote:
>
>> Hello igor,
>> 
>> I have a question about the cache behaviour in proxy mode.
>> 
>> I have nginx in front head which redirect to an apache back end. Nginx
>> caches eveything for M minutes.
>> 
>> If I have a large number of requests for the same page and this page
>> is cached : nginx returns the cached page ... no problems
>> After M minutes, the cached page expires
>> The first request coming after the expiration makes nginx to ask the
>> backend for refresh
>> When nginx receives the backend fresh response, it's saved to cache
>> and then nginx serves the fresh cached page
>> 
>> But what happen between the start of the request to the backend and
>> the end of the response from the backend ? (let's assume that the
>> backend serves the page in 5s ... and in 5s I can have a lot of
>> request to this page).:
>> - Are the request queued waiting for the backend response ?
>> - Every request makes try to refresh the cache from the backend ? (in
>> this case, I have multiple request for the same page to the backend
>> ... I can have a burst of request and my apache can be overflowed by
>> request -- that's why I'm using nignx with cache).
>> - Do the requests serve the cached page even if it's expired until the
>> backend response has been received ?
>> - Maybe something else :)
>
>Currently all requests which found that a cached response expired
>are proxied to backend. I plan to implement busy locks to pass the single
>request and leave others to wait the response up to specified time.

How about the notion of soft timeouts. Say the ttl is set to 300
seconds. We pick a value like say 10% and say that any request received
for the next 30 seconds will still get the stale content without any
waits and somewhere in there, we initiate a request to the backends and
refresh the cache. Beyond this window, we can take the busy wait
approach.

Squid 2.7.x has something like this.
http://www.squid-cache.org/Versions/v2/2.7/cfgman/refresh_stale_hit.html