[prev in list] [next in list] [prev in thread] [next in thread]
List: nginx
Subject: Re: Disable NGINX caching 304 Responses from Origin Server
From: Ryan Barclay <ryan () rbftpnetworks ! com>
Date: 2017-07-26 10:18:33
Message-ID: d23f395a-a60e-2aa8-12d4-3c434059ee18 () rbftpnetworks ! com
[Download RAW message or body]
[Attachment #2 (multipart/alternative)]
Hi Peter,
When we used this:
proxy_set_header If-Modified-Since $http_if_modified_since
...Nginx was caching 304 responses from the upstream when the client
requested If-Modified-Since (if it was the first request for that
object). Future requests by clients that didn't set the
If-Modified-Since header were also being served the 304 (with no body)
as that was what was in the cache.
I hope that makes it clearer?
Thanks Peter.
On 26/07/2017 10:43, Peter Booth wrote:
> Ryan,
>
> Just to be pedantic, can you spell out exactly what you meant when
> you said "and deliver future responses as 304 to clients even without
> the If-Modified-Since header?"
> What requests were triggering the 304 response?
> Were you observing what a browser was seeing or were you using curl or
> wget to trigger the response?
>
> Peter
>
> Sent from my iPhone
>
> On Jul 26, 2017, at 5:10 AM, Ryan Barclay <ryan@rbftpnetworks.com
> <mailto:ryan@rbftpnetworks.com>> wrote:
>
>> So this config seems to work:
>>
>> proxy_cache_valid 200 3M;
>> proxy_cache_valid 304 0;
>> proxy_cache_valid 404 0;
>> proxy_cache_revalidate on;
>> proxy_ignore_headers Cache-Control Expires;
>>
>>
>> There is no need for:
>>
>> proxy_set_header If-Modified-Since $http_if_modified_since;
>>
>>
>>
>> On 26/07/2017 09:57, Ryan Barclay wrote:
>>>
>>> Thanks for the reply Peter.
>>>
>>> I've noticed something interesting and wondered if you could shed
>>> some light on it.
>>>
>>> Simply adding:
>>>
>>> proxy_ignore_headers Cache-Control Expires;
>>>
>>> Enables 304 responses from the origin server without setting:
>>>
>>> proxy_set_header If-Modified-Since $http_if_modified_since;
>>>
>>> I'm confused.
>>>
>>> On 26/07/2017 09:11, Peter Booth wrote:
>>>> I can't see an obvious issue, but I can say that there is no such
>>>> thing as a simple web server setup where caching is involved.
>>>> I have gray hairs that appeared after working with a high traffic
>>>> retail website that had seven levels of caching
>>>> (browser cache, CDN, hardware load balancer, nginx reverse proxy,
>>>> servlets that write content, tangosol /oracle coherence, endeca
>>>> caching)
>>>>
>>>> I'm hoping that you are living in a saner world than that one but
>>>> I'm sure that you will have some craziness.
>>>> I would encourage you to add $upstream_cache_status to your log format
>>>> and/or add the directive add_headerX-Cache-Status
>>>> $upstream_cache_status;
>>>> Instrumenting the cache can be a real life-saver when things go awry.
>>>>
>>>> I'd also strongly encourage you to use redbot.org
>>>> <http://redbot.org> to check for aberrant behavior and
>>>> webpagetest.org <http://webpagetest.org>
>>>> to see how different browsers handle your site.
>>>>
>>>> Peter
>>>>
>>>>
>>>>
>>>>> On Jul 26, 2017, at 3:29 AM, Ryan Barclay <ryan@rbftpnetworks.com
>>>>> <mailto:ryan@rbftpnetworks.com>> wrote:
>>>>>
>>>>> The following config seems to work for the situation I discussed:
>>>>>
>>>>> proxy_cache_valid 200 3M;
>>>>> proxy_cache_valid 304 0;
>>>>> proxy_cache_revalidate on;
>>>>> proxy_set_header If-Modified-Since $http_if_modified_since;
>>>>> proxy_ignore_headers Cache-Control Expires;
>>>>>
>>>>>
>>>>> ... can anybody see any problems with this config or future
>>>>> problems that may arise?
>>>>>
>>>>>
>>>>> On 24/07/2017 16:20, Ryan Barclay wrote:
>>>>>> We have a pretty simple setup with NGINX sitting on the front and
>>>>>> a backend server (on a separate physical server) that provides
>>>>>> the content.
>>>>>>
>>>>>> Nginx then caches content based on the EXPIRES and Cache-Control
>>>>>> headers set by the origin server.
>>>>>>
>>>>>> We noticed that NGINX was not issuing 304 headers to images that
>>>>>> were not in the local NGINX cache when the If-Modified-Since
>>>>>> header was sent by the client. Instead, it would issue a 200 with
>>>>>> the full data file.
>>>>>>
>>>>>> To fix this, we applied:
>>>>>> proxy_set_header If-Modified-Since $http_if_modified_since
>>>>>>
>>>>>> So then the If-Modified-Since header was passed to the backend
>>>>>> and of course, it returned correctly with the 304 header - great.
>>>>>>
>>>>>> But what we noticed was that NGINX would cache this 304 response
>>>>>> and deliver future responses as 304 to clients even without the
>>>>>> If-Modified-Since header.
>>>>>>
>>>>>> How can we disable caching of 304 responses and fix this issue?
>>>>>>
>>>>>> Thank you for your help, suggestions, and tips in advance.
>>>>>
>>>>> _______________________________________________
>>>>> nginx mailing list
>>>>> nginx@nginx.org <mailto:nginx@nginx.org>
>>>>> http://mailman.nginx.org/mailman/listinfo/nginx
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> nginx mailing list
>>>> nginx@nginx.org
>>>> http://mailman.nginx.org/mailman/listinfo/nginx
>>>
>>
>> _______________________________________________
>> nginx mailing list
>> nginx@nginx.org <mailto:nginx@nginx.org>
>> http://mailman.nginx.org/mailman/listinfo/nginx
>
>
> _______________________________________________
> nginx mailing list
> nginx@nginx.org
> http://mailman.nginx.org/mailman/listinfo/nginx
[Attachment #5 (text/html)]
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
</head>
<body text="#000000" bgcolor="#FFFFFF">
<p>Hi Peter,</p>
<p>When we used this:</p>
<p>proxy_set_header If-Modified-Since $http_if_modified_since</p>
<p>...Nginx was caching 304 responses from the upstream when the
client requested If-Modified-Since (if it was the first request
for that object). Future requests by clients that didn't set the
If-Modified-Since header were also being served the 304 (with no
body) as that was what was in the cache.</p>
<p>I hope that makes it clearer?</p>
<p>Thanks Peter.<br>
</p>
<pre class="moz-signature" cols="72">
</pre>
<div class="moz-cite-prefix">On 26/07/2017 10:43, Peter Booth wrote:<br>
</div>
<blockquote type="cite"
cite="mid:8729C230-278E-46AE-8E71-E7E05C351732@me.com">
<meta http-equiv="content-type" content="text/html; charset=utf-8">
<div>Ryan,</div>
<div id="AppleMailSignature"><br>
</div>
<div id="AppleMailSignature">Just to be pedantic, can you spell
out exactly what you meant when you said "<span
style="background-color: rgba(255, 255, 255, 0);">and deliver
future responses as 304 to clients even without the
If-Modified-Since header?"</span></div>
<div id="AppleMailSignature">What requests were triggering the 304
response?</div>
<div id="AppleMailSignature">Were you observing what a browser was
seeing or were you using curl or wget to trigger the response?</div>
<div id="AppleMailSignature"><br>
</div>
<div id="AppleMailSignature">Peter<br>
<br>
Sent from my iPhone</div>
<div><br>
On Jul 26, 2017, at 5:10 AM, Ryan Barclay <<a
href="mailto:ryan@rbftpnetworks.com" \
moz-do-not-send="true">ryan@rbftpnetworks.com</a>> wrote:<br>
<br>
</div>
<blockquote type="cite">
<div>
<meta http-equiv="Content-Type" content="text/html;
charset=utf-8">
<p>So this config seems to work:</p>
<p>proxy_cache_valid 200 3M;<br>
proxy_cache_valid 304 0;<br>
proxy_cache_valid 404 0;<br>
proxy_cache_revalidate on;<br>
proxy_ignore_headers Cache-Control Expires;</p>
<p><br>
</p>
<p>There is no need for: <br>
</p>
<p>proxy_set_header If-Modified-Since $http_if_modified_since;</p>
<p><br>
</p>
<p><br>
</p>
<div class="moz-cite-prefix">On 26/07/2017 09:57, Ryan Barclay
wrote:<br>
</div>
<blockquote type="cite"
cite="mid:6913bf6b-aaee-885a-41b1-a386ea788e34@rbftpnetworks.com">
<meta http-equiv="Content-Type" content="text/html;
charset=utf-8">
<p>Thanks for the reply Peter.</p>
<p>I've noticed something interesting and wondered if you
could shed some light on it.</p>
<p>Simply adding:<br>
</p>
<p>proxy_ignore_headers Cache-Control Expires;</p>
<p>Enables 304 responses from the origin server without
setting:</p>
<p>proxy_set_header If-Modified-Since
$http_if_modified_since;</p>
<p>I'm confused.<br>
</p>
<div class="moz-cite-prefix">On 26/07/2017 09:11, Peter
Booth wrote:<br>
</div>
<blockquote type="cite"
cite="mid:63438CEF-8E36-4496-A069-1A51F33F0C24@me.com">
<meta http-equiv="Content-Type" content="text/html;
charset=utf-8">
<div class="">I can’t see an obvious issue, but I can say
that there is no such thing as a simple web server setup
where caching is involved.</div>
<div class="">I have gray hairs that appeared after
working with a high traffic retail website that had
seven levels of caching</div>
<div class="">(browser cache, CDN, hardware load balancer,
nginx reverse proxy, servlets that write content,
tangosol /oracle coherence, endeca caching)</div>
<div class=""><br class="">
</div>
<div class="">I’m hoping that you are living in a saner
world than that one but I’m sure that you will have some
craziness. </div>
<div class="">I would encourage you to add <span style="font-family: \
Consolas, 'Andale Mono', 'DejaVu Sans Mono', monospace; font-size: \
10.199999809265137px; orphans: 2; white-space: pre-wrap; widows: 2; background-color: \
rgb(238, 238, 238);" class="">$upstream_cache_status</span> to your log \
format</div> <div class="">and/or add the directive <span style="font-family: \
inherit; font-style: inherit; orphans: 2; white-space: pre-wrap; widows: 2; \
box-sizing: inherit; border: 0px; font-size: 12.75px; font-weight: 700; margin: 0px; \
outline: 0px; padding: 0px; vertical-align: baseline;" \
class="">add_header</span><span style="font-family: Consolas, 'Andale Mono', 'DejaVu \
Sans Mono', monospace; font-size: 0.85em; font-style: inherit; orphans: 2; \
white-space: pre-wrap; widows: 2; background-color: rgb(238, 238, 238);" class=""> \
X-Cache-Status $upstream_cache_status;</span></div> <div class="">Instrumenting the \
cache can be a real life-saver when things go awry.</div>
<div class=""><br class="">
</div>
<div class="">I’d also strongly encourage you to use <a
href="http://redbot.org" class=""
moz-do-not-send="true">redbot.org</a> to check for
aberrant behavior and <a href="http://webpagetest.org"
class="" moz-do-not-send="true">webpagetest.org</a></div>
<div class="">to see how different browsers handle your
site. </div>
<div class=""><br class="">
</div>
<div class="">Peter</div>
<div class=""><br class="">
</div>
<div class=""><br class="">
</div>
<br class="">
<div>
<blockquote type="cite" class="">
<div class="">On Jul 26, 2017, at 3:29 AM, Ryan
Barclay <<a href="mailto:ryan@rbftpnetworks.com"
class="" moz-do-not-send="true">ryan@rbftpnetworks.com</a>>
wrote:</div>
<br class="Apple-interchange-newline">
<div class="">
<div class="">The following config seems to work for
the situation I discussed:<br class="">
<br class="">
proxy_cache_valid 200 3M;<br class="">
proxy_cache_valid 304 0;<br class="">
proxy_cache_revalidate on;<br class="">
proxy_set_header If-Modified-Since
$http_if_modified_since;<br class="">
proxy_ignore_headers Cache-Control Expires;<br
class="">
<br class="">
<br class="">
... can anybody see any problems with this config
or future problems that may arise?<br class="">
<br class="">
<br class="">
On 24/07/2017 16:20, Ryan Barclay wrote:<br
class="">
<blockquote type="cite" class="">We have a pretty
simple setup with NGINX sitting on the front and
a backend server (on a separate physical server)
that provides the content.<br class="">
<br class="">
Nginx then caches content based on the EXPIRES
and Cache-Control headers set by the origin
server.<br class="">
<br class="">
We noticed that NGINX was not issuing 304
headers to images that were not in the local
NGINX cache when the If-Modified-Since header
was sent by the client. Instead, it would issue
a 200 with the full data file.<br class="">
<br class="">
To fix this, we applied:<br class="">
proxy_set_header If-Modified-Since
$http_if_modified_since<br class="">
<br class="">
So then the If-Modified-Since header was passed
to the backend and of course, it returned
correctly with the 304 header - great.<br
class="">
<br class="">
But what we noticed was that NGINX would cache
this 304 response and deliver future responses
as 304 to clients even without the
If-Modified-Since header.<br class="">
<br class="">
How can we disable caching of 304 responses and
fix this issue?<br class="">
<br class="">
Thank you for your help, suggestions, and tips
in advance.<br class="">
</blockquote>
<br class="">
_______________________________________________<br
class="">
nginx mailing list<br class="">
<a href="mailto:nginx@nginx.org" class=""
moz-do-not-send="true">nginx@nginx.org</a><br
class="">
<a class="moz-txt-link-freetext"
href="http://mailman.nginx.org/mailman/listinfo/nginx"
\
moz-do-not-send="true">http://mailman.nginx.org/mailman/listinfo/nginx</a><br \
class=""> </div>
</div>
</blockquote>
</div>
<br class="">
<br>
<fieldset class="mimeAttachmentHeader"></fieldset>
<br>
<pre wrap="">_______________________________________________
nginx mailing list
<a class="moz-txt-link-abbreviated" href="mailto:nginx@nginx.org" \
moz-do-not-send="true">nginx@nginx.org</a> <a class="moz-txt-link-freetext" \
href="http://mailman.nginx.org/mailman/listinfo/nginx" \
moz-do-not-send="true">http://mailman.nginx.org/mailman/listinfo/nginx</a></pre> \
</blockquote> <br>
</blockquote>
<br>
</div>
</blockquote>
<blockquote type="cite">
<div><span>_______________________________________________</span><br>
<span>nginx mailing list</span><br>
<span><a href="mailto:nginx@nginx.org" \
moz-do-not-send="true">nginx@nginx.org</a></span><br> <span><a
href="http://mailman.nginx.org/mailman/listinfo/nginx"
moz-do-not-send="true">http://mailman.nginx.org/mailman/listinfo/nginx</a></span></div>
</blockquote>
<br>
<fieldset class="mimeAttachmentHeader"></fieldset>
<br>
<pre wrap="">_______________________________________________
nginx mailing list
<a class="moz-txt-link-abbreviated" href="mailto:nginx@nginx.org">nginx@nginx.org</a>
<a class="moz-txt-link-freetext" \
href="http://mailman.nginx.org/mailman/listinfo/nginx">http://mailman.nginx.org/mailman/listinfo/nginx</a></pre>
</blockquote>
<br>
</body>
</html>
_______________________________________________
nginx mailing list
nginx@nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx
[prev in list] [next in list] [prev in thread] [next in thread]
Configure |
About |
News |
Add a list |
Sponsored by KoreLogic