Disable NGINX caching 304 Responses from Origin Server

[Ryan Barclay]

Hi Peter,

When we used this:

proxy_set_header If-Modified-Since $http_if_modified_since

...Nginx was caching 304 responses from the upstream when the client 
requested If-Modified-Since (if it was the first request for that 
object).  Future requests by clients that didn't set the 
If-Modified-Since header were also being served the 304 (with no body) 
as that was what was in the cache.

I hope that makes it clearer?

Thanks Peter.

On 26/07/2017 10:43, Peter Booth wrote:
> Ryan,
>
> Just to be pedantic, can you spell out  exactly what you meant when 
> you said "and deliver future responses as 304 to clients even without 
> the If-Modified-Since header?"
> What requests were triggering the 304 response?
> Were you observing what a browser was seeing or were you using curl or 
> wget to trigger the response?
>
> Peter
>
> Sent from my iPhone
>
> On Jul 26, 2017, at 5:10 AM, Ryan Barclay <ryan at rbftpnetworks.com 
> <mailto:ryan at rbftpnetworks.com>> wrote:
>
>> So this config seems to work:
>>
>> proxy_cache_valid 200 3M;
>> proxy_cache_valid 304 0;
>> proxy_cache_valid 404 0;
>> proxy_cache_revalidate on;
>> proxy_ignore_headers Cache-Control Expires;
>>
>>
>> There is no need for:
>>
>> proxy_set_header If-Modified-Since $http_if_modified_since;
>>
>>
>>
>> On 26/07/2017 09:57, Ryan Barclay wrote:
>>>
>>> Thanks for the reply Peter.
>>>
>>> I've noticed something interesting and wondered if you could shed 
>>> some light on it.
>>>
>>> Simply adding:
>>>
>>> proxy_ignore_headers Cache-Control Expires;
>>>
>>> Enables 304 responses from the origin server without setting:
>>>
>>> proxy_set_header If-Modified-Since $http_if_modified_since;
>>>
>>> I'm confused.
>>>
>>> On 26/07/2017 09:11, Peter Booth wrote:
>>>> I can’t see an obvious issue, but I can say that there is no such 
>>>> thing as a simple web server setup where caching is involved.
>>>> I have gray hairs that appeared after working with a high traffic 
>>>> retail website that had seven levels of caching
>>>> (browser cache, CDN, hardware load balancer, nginx reverse proxy, 
>>>> servlets that write content, tangosol /oracle coherence, endeca 
>>>> caching)
>>>>
>>>> I’m hoping that you are living in  a saner world than that one but 
>>>> I’m sure that you will have some craziness.
>>>> I would encourage you to add $upstream_cache_status to your log format
>>>> and/or add the directive add_headerX-Cache-Status 
>>>> $upstream_cache_status;
>>>> Instrumenting the cache can be a real life-saver when things go awry.
>>>>
>>>> I’d also strongly encourage you to use redbot.org 
>>>> <http://redbot.org> to check for aberrant behavior and 
>>>> webpagetest.org <http://webpagetest.org>
>>>> to see how different browsers handle your site.
>>>>
>>>> Peter
>>>>
>>>>
>>>>
>>>>> On Jul 26, 2017, at 3:29 AM, Ryan Barclay <ryan at rbftpnetworks.com 
>>>>> <mailto:ryan at rbftpnetworks.com>> wrote:
>>>>>
>>>>> The following config seems to work for the situation I discussed:
>>>>>
>>>>> proxy_cache_valid 200 3M;
>>>>> proxy_cache_valid 304 0;
>>>>> proxy_cache_revalidate on;
>>>>> proxy_set_header If-Modified-Since $http_if_modified_since;
>>>>> proxy_ignore_headers Cache-Control Expires;
>>>>>
>>>>>
>>>>> ... can anybody see any problems with this config or future 
>>>>> problems that may arise?
>>>>>
>>>>>
>>>>> On 24/07/2017 16:20, Ryan Barclay wrote:
>>>>>> We have a pretty simple setup with NGINX sitting on the front and 
>>>>>> a backend server (on a separate physical server) that provides 
>>>>>> the content.
>>>>>>
>>>>>> Nginx then caches content based on the EXPIRES and Cache-Control 
>>>>>> headers set by the origin server.
>>>>>>
>>>>>> We noticed that NGINX was not issuing 304 headers to images that 
>>>>>> were not in the local NGINX cache when the If-Modified-Since 
>>>>>> header was sent by the client. Instead, it would issue a 200 with 
>>>>>> the full data file.
>>>>>>
>>>>>> To fix this, we applied:
>>>>>> proxy_set_header If-Modified-Since $http_if_modified_since
>>>>>>
>>>>>> So then the If-Modified-Since header was passed to the backend 
>>>>>> and of course, it returned correctly with the 304 header - great.
>>>>>>
>>>>>> But what we noticed was that NGINX would cache this 304 response 
>>>>>> and deliver future responses as 304 to clients even without the 
>>>>>> If-Modified-Since header.
>>>>>>
>>>>>> How can we disable caching of 304 responses and fix this issue?
>>>>>>
>>>>>> Thank you for your help, suggestions, and tips in advance.
>>>>>
>>>>> _______________________________________________
>>>>> nginx mailing list
>>>>> nginx at nginx.org <mailto:nginx at nginx.org>
>>>>> http://mailman.nginx.org/mailman/listinfo/nginx
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> nginx mailing list
>>>> nginx at nginx.org
>>>> http://mailman.nginx.org/mailman/listinfo/nginx
>>>
>>
>> _______________________________________________
>> nginx mailing list
>> nginx at nginx.org <mailto:nginx at nginx.org>
>> http://mailman.nginx.org/mailman/listinfo/nginx
>
>
> _______________________________________________
> nginx mailing list
> nginx at nginx.org
> http://mailman.nginx.org/mailman/listinfo/nginx

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx/attachments/20170726/602820e5/attachment-0001.html>