Debugging Nginx Cache Misses: Hitting high number of MISS despite high proxy valid
Cherian Thomas
cherian.in at gmail.com
Sun May 13 05:30:00 UTC 2018
Thanks for this Michael.
This is so surprising. If someone decides to Dos and crawls the website
with a rogue header, this will essentially bypass the cache and put a
strain on the website. In fact, I was hit by a dos attack that’s when I
started looking at logs and realized the large number of MISSes.
Can someone please help?
- Cherian
On Sat, May 12, 2018 at 12:01 PM, Friscia, Michael <michael.friscia at yale.edu
> wrote:
> I'm not sure if this will help, but I ignore/hide a lot, this is in my
> config
>
>
> proxy_ignore_headers X-Accel-Expires Expires Cache-Control Set-Cookie;
> proxy_hide_header X-Accel-Expires;
> proxy_hide_header Pragma;
> proxy_hide_header Server;
> proxy_hide_header Request-Context;
> proxy_hide_header X-Powered-By;
> proxy_hide_header X-AspNet-Version;
> proxy_hide_header X-AspNetMvc-Version;
>
>
> I have not experienced the problem you mention, I just thought I would
> offer my config.
>
>
> ___________________________________________
>
> Michael Friscia
>
> Office of Communications
>
> Yale School of Medicine
>
> (203) 737-7932 – office
>
> (203) 931-5381 – mobile
>
> http://web.yale.edu
> <https://mailtrack.io/trace/link/8357a0bdd8c40c2ff5b7d91c7797cbc7a8535ffb?url=http%3A%2F%2Fweb.yale.edu%2F&userId=74734&signature=d652edf1f4f21323>
>
>
> ------------------------------
> *From:* nginx <nginx-bounces at nginx.org> on behalf of Quintin Par <
> quintinpar at gmail.com>
> *Sent:* Saturday, May 12, 2018 1:32 PM
> *To:* nginx at nginx.org
> *Subject:* Re: Debugging Nginx Cache Misses: Hitting high number of MISS
> despite high proxy valid
>
>
> That’s the tricky part. These MISSes are intermittent. Whenever I run curl
> I get HITs but I end up seeing a lot of MISS in the logs.
>
>
>
> How do I log these MiSSes with the reason? I want to know what headers
> ended up bypassing the cache.
>
>
>
> Here’s my caching config
>
>
>
> proxy_pass http://127.0.0.1:8000
> <https://urldefense.proofpoint.com/v2/url?u=http-3A__127.0.0.1-3A8000&d=DwMFaQ&c=cjytLXgP8ixuoHflwc-poQ&r=wvXEDjvtDPcv7AlldT5UvDx32KXBEM6um_lS023SJrs&m=F-qGMOyS74uE8JM-dOLmNH92bQ1xQ-7Rj1d6k-_WST4&s=NHvlb1WColNwTWBF36P1whJdu5iWHK9_50IDHugaEdQ&e=>
> ;
>
> proxy_set_header X-Real-IP $remote_addr;
>
> proxy_set_header X-Forwarded-For
> $proxy_add_x_forwarded_for;
>
> proxy_set_header X-Forwarded-Proto https;
>
> proxy_set_header X-Forwarded-Port 443;
>
>
>
> # If logged in, don't cache.
>
> if ($http_cookie ~* "comment_author_|wordpress_(?!test_cookie)|wp-postpass_"
> ) {
>
> set $do_not_cache 1;
>
> }
>
> proxy_cache_key "$scheme://$host$request_uri$
> do_not_cache";
>
> proxy_cache staticfilecache;
>
> add_header Cache-Control public;
>
> proxy_cache_valid 200 120d;
>
> proxy_hide_header "Set-Cookie";
>
> proxy_ignore_headers "Set-Cookie";
>
> proxy_ignore_headers "Cache-Control";
>
> proxy_hide_header "Cache-Control";
>
> proxy_pass_header X-Accel-Expires;
>
>
>
> proxy_set_header Accept-Encoding "";
>
> proxy_ignore_headers Expires;
>
> add_header X-Cache-Status $upstream_cache_status;
>
> proxy_cache_use_stale timeout;
>
> proxy_cache_bypass $arg_nocache $do_not_cache;
> - Quintin
>
>
> On Sat, May 12, 2018 at 10:29 AM Lucas Rolff <lucas at lucasrolff.com> wrote:
>
> It can be as simple as doing a curl to your “origin” url (the one you
> proxy_pass to) for the files you see that gets a lot of MISS’s – if there’s
> odd headers such as cookies etc, then you’ll most likely experience a bad
> cache if your nginx is configured to not ignore those headers.
>
>
>
> *From: *nginx <nginx-bounces at nginx.org> on behalf of Quintin Par <
> quintinpar at gmail.com>
> *Reply-To: *"nginx at nginx.org" <nginx at nginx.org>
> *Date: *Saturday, 12 May 2018 at 18.26
> *To: *"nginx at nginx.org" <nginx at nginx.org>
> *Subject: *Debugging Nginx Cache Misses: Hitting high number of MISS
> despite high proxy valid
>
>
>
> [image:
> https://mailtrack.io/trace/mail/86a613eb1ce46a4e7fa6f9eb96989cddae639800.png?u=74734]
>
> My proxy cache path is set to a very high size
>
>
>
> proxy_cache_path /var/lib/nginx/cache levels=1:2
> keys_zone=staticfilecache:180m max_size=700m;
>
> and the size used is only
>
>
>
> sudo du -sh *
>
> 14M cache
>
> 4.0K proxy
>
> Proxy cache valid is set to
>
>
>
> proxy_cache_valid 200 120d;
>
> I track HIT and MISS via
>
>
>
> add_header X-Cache-Status $upstream_cache_status;
>
> Despite these settings I am seeing a lot of MISSes. And this is for pages
> I intentionally ran a cache warmer an hour ago.
>
>
>
> How do I debug why these MISSes are happening? How do I find out if the
> miss was due to eviction, expiration, some rogue header etc? Does Nginx
> provide commands for this?
>
>
>
> - Quintin
> _______________________________________________
> nginx mailing list
> nginx at nginx.org
> http://mailman.nginx.org/mailman/listinfo/nginx
> <https://mailtrack.io/trace/link/122c3dbd333c388f47f5c2776af9ebc3fc75ae10?url=https%3A%2F%2Furldefense.proofpoint.com%2Fv2%2Furl%3Fu%3Dhttp-3A__mailman.nginx.org_mailman_listinfo_nginx%26d%3DDwMFaQ%26c%3DcjytLXgP8ixuoHflwc-poQ%26r%3DwvXEDjvtDPcv7AlldT5UvDx32KXBEM6um_lS023SJrs%26m%3DF-qGMOyS74uE8JM-dOLmNH92bQ1xQ-7Rj1d6k-_WST4%26s%3DD3LnZhfobOtlEStCvCDrcwmHydEHaGRFC4gnWvRT5Uk%26e%3D&userId=74734&signature=56c7a7ad18b2c057>
>
>
> _______________________________________________
> nginx mailing list
> nginx at nginx.org
> http://mailman.nginx.org/mailman/listinfo/nginx
> <https://mailtrack.io/trace/link/92c2700d67bd6891ca1606e2df4e0f11c6d82260?url=http%3A%2F%2Fmailman.nginx.org%2Fmailman%2Flistinfo%2Fnginx&userId=74734&signature=3763121afa828bb7>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx/attachments/20180512/6215b709/attachment-0001.html>
More information about the nginx
mailing list