<html><head><meta http-equiv="Content-Type" content="text/html; charset=utf-8"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class=""><br class=""><div>Quintin,</div><div><br class=""></div><div>I dont know anything about your context, but your setup looks over simplistic. Here are some things that I learned </div><div>painfully over a few years of supporting a high traffic retail website</div><div><br class=""></div><div>1. Is this a website that's on the internet, and thus exposed to random queries from bots and scrapers that you can’t control?</div><div><br class=""></div><div>2. For your cache misses, how long best case, typical and worse case does your back-end take to build the pages?</div><div><br class=""></div><div>3. You need to log everything that could feasibly affect the status of the site. For example, here’s a log config urationfrom one gnarly site that I worked on:</div><div class=""><div class=""><br class=""></div><div class=""> <font face="Consolas" class=""> log_format main '$http_x_forwarded_for $http_true_client_ip $remote_addr - $remote_user [$time_local] $host "$request" '</font></div><div class=""><font face="Consolas" class=""> '$status $body_bytes_sent $upstream_cache_status $cookie_jsessionid $http_akamai_country $cookie_e4x_country $cookie_e4x_currency "$http_referer" '</font></div><div class=""><font face="Consolas" class=""> '"$http_user_agent" "$request_time”’;</font></div></div><div class=""><br class=""></div><div>4. the first problem is your cache key, and that it includes $request_uri which is the original uri</div><div><b class=""> including all arguments. </b>So you are already exposed to DOS requests that could be unintentional,</div><div>as anyone can bust your cache by adding an extra parameter.</div><div><br class=""></div><div><div dir="auto" style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class=""><div class=""><blockquote type="cite" class=""><div class="gmail_extra"><div class="gmail_quote"><blockquote class="gmail_quote" style="margin: 0px 0px 0px 0.8ex; border-left-width: 1px; border-left-style: solid; border-left-color: rgb(204, 204, 204); padding-left: 1ex;"><div dir="ltr" class=""><div id="m_-267008425796858038divtagdefaultwrapper" dir="ltr" class="" style="font-size: 12pt; font-family: Calibri, Helvetica, sans-serif;"><div class=""><div class=""><div class="h5"><div class=""><div dir="ltr" class=""><div class=""><div class="" style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: Calibri, sans-serif;"><span class=""> </span>proxy_cache_key "$<a href="scheme://$host$request_uri$" class="">scheme://$host$request_uri$</a><wbr class="">do_not_cache";</div></div></div></div></div></div></div></div></div></blockquote></div></div></blockquote></div><div class=""><br class=""></div><div class="">5. Not caching requests from logged in users is a very blunt tool. Is this a site where only administrative users are logged in?</div><div class=""><br class=""></div><div class="">Imagine a retail site that sells clothing. It’s possible that a dynamic page that lists all the red dresses is something </div><div class="">a logged in user sees. Perhaps the page can be cached ? But if there is a version of the page that shows 30 entries and other </div><div class="">that shows 60 then they need to disambiguated by the cache key. Perhaps users can choose to see prices in Euro instead of USD?</div><div class="">Then this also belongs in the key. If I am an American vacationing in Pari s then perhaps the default behavior should be to show me</div><div class=""> Euro prices, based n the value of a cookie that the CDN sets. In the situation the customer may want to override this default behavior </div><div class="">and insist he sees USD prices. You can see how complex this can get. </div></div></div><div><br class=""></div><div>7. The default behavior is to not cache responses that contain a set-cookie - imagine how cache pollution - sending someone another person’s personal data stored in a cookie could be much worse than a cache miss. But there are also settings where your backend is some legacy software that you dont control</div><div>and the correct behavior isn’t to not cache but instead to remove the set-cookie from the response and cache the response without it.</div><div><br class=""></div><div>8 How you prime the cache , monitor the cache, and clear the cache are crucial . Perhaps you have a script that uses curl or wget to retrieve a series of pages from your site. If the script is written naively then each step might cause a new servlet session to be created on the backend producing a memory issue. </div><div><br class=""></div><div>9. script is very useful to track the health of your cache:</div><div><br class=""></div><div><a href="https://github.com/perusio/nginx-cache-inspector" class="">https://github.com/perusio/nginx-cache-inspector</a></div><div><br class=""></div><div>10. The if directive in nginx has some issues (see <a href="https://www.nginx.com/resources/wiki/start/topics/depth/ifisevil/" class="">https://www.nginx.com/resources/wiki/start/topics/depth/ifisevil/</a> )</div><div class="">When I need to use complex configuration logic I use OpenResty. OpenResty is a bundle that </div><div class="">combines the standard nginx with some additional lua modules. It’s still standard nginx -</div><div class=""> not forked or customized in any way.</div><div class=""><br class=""></div><div class="">11.</div><div class=""><br class=""></div><div class="">A very cut down version of a cache config for one page follows:</div><div class=""><br class=""></div><div class=""><div class=""><font face="Consolas" class=""># Product arrays get cached</font></div><div class=""><font face="Consolas" class=""> location ~ /shop/ {</font></div><div class=""><font face="Consolas" class=""> rewrite "/(.*)/2];ord.*$" $1 ;</font></div><div class=""><font face="Consolas" class=""> proxy_no_cache $arg_mid $arg_siteID;</font></div><div class=""><font face="Consolas" class=""> proxy_cache_bypass $arg_mid $arg_siteID;</font></div><div class=""><font face="Consolas" class=""> proxy_cache_use_stale updating;</font></div><div class=""><font face="Consolas" class=""> default_type text/html;</font></div><div class=""><font face="Consolas" class=""> proxy_cache_valid 200 302 301 15m;</font></div><div class=""><font face="Consolas" class=""> proxy_ignore_headers Set-Cookie Cache-Control; </font></div><div class=""><font face="Consolas" class=""> proxy_pass_header off;</font></div><div class=""><font face="Consolas" class=""> proxy_hide_header Set-Cookie;</font></div><div class=""><font face="Consolas" class=""> expires 900s;</font></div><div class=""><font face="Consolas" class=""> add_header Last-Modified "";</font></div><div class=""><font face="Consolas" class=""> add_header ETag ""; </font></div><div class=""><font face="Consolas" class=""> # Build cache key </font></div><div class=""><font face="Consolas" class=""> set $e4x_currency $cookie_e4x_currency;</font></div><div class=""><font face="Consolas" class=""> set_if_empty $e4x_currency 'USD';</font></div><div class=""><font face="Consolas" class=""> set $num_items $cookie_EndecaNumberOfItems;</font></div><div class=""><font face="Consolas" class=""> set_if_empty $num_items 'LOW'; </font></div><div class=""><font face="Consolas" class=""> proxy_cache_key "$uri|$e4x_currency|$num_items";</font></div><div class=""><font face="Consolas" class=""> proxy_cache product_arrays; </font></div><div class=""><font face="Consolas" class=""> # Add Canonical URL string</font></div><div class=""><font face="Consolas" class=""> set $folder_id $arg_FOLDER%3C%3Efolder_id;</font></div><div class=""><font face="Consolas" class=""> set $canonical_url "<a href="http://$http_host$uri" class="">http://$http_host$uri</a>";</font></div><div class=""><font face="Consolas" class=""> add_header Link "<$canonical_url>; rel=\"canonical\"";</font></div><div class=""><font face="Consolas" class=""> proxy_pass <a href="http://apache$request_uri" class="">http://apache$request_uri</a>;</font></div><div class=""><font face="Consolas" class=""> }</font></div></div><div><font face="Consolas" class=""><br class=""></font></div><div><font face="Consolas" class=""><br class=""></font></div><div>Tis snippet shows a key made of three parts. The real version has seven parts.</div><div><font face="Consolas" class=""><br class=""></font></div><div><font face="Consolas" class="">Goo</font>d luck!</div><div><br class=""></div><div>Peter</div><div><br class=""></div><div><br class=""><blockquote type="cite" class=""><div class="">On 14 May 2018, at 12:06 AM, Quintin Par <<a href="mailto:quintinpar@gmail.com" class="">quintinpar@gmail.com</a>> wrote:</div><br class="Apple-interchange-newline"><div class=""><div dir="ltr" class=""><img width="0" height="0" class="mailtrack-img" alt="" style="display:flex" src="https://mailtrack.io/trace/mail/830e676b314f1b30986adfc1c7df5f967b9aa282.png?u=74734"><div class=""><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: Calibri, sans-serif;" class="">Thanks all for the response. Michael, I am going to add those header ignores. <span class=""></span></div><p class="MsoNormal" style="margin:0in 0in 0.0001pt;font-size:12pt;font-family:Calibri,sans-serif"><span class=""> </span></p><div style="margin: 0in 0in 0.0001pt; font-size: 12pt; font-family: Calibri, sans-serif;" class="">Still puzzled by the large number of MISSEs and I’ve no clue why they are happening. Leads appreciated. <span class=""></span></div><p class="MsoNormal" style="margin:0in 0in 0.0001pt;font-size:12pt;font-family:Calibri,sans-serif"><span class=""> </span></p><p class="MsoNormal" style="margin:0in 0in 0.0001pt;font-size:12pt;font-family:Calibri,sans-serif"><span class=""> </span></p>
</div><div class="gmail_default" style="font-family: arial, helvetica, sans-serif;"><br class=""></div></div><div class="gmail_extra"><br clear="all" class=""><div class=""><div class="gmail_signature" data-smartmail="gmail_signature">- Quintin</div></div>
<br class=""><div class="gmail_quote">On Sun, May 13, 2018 at 6:12 PM, c0nw0nk <span dir="ltr" class=""><<a href="mailto:nginx-forum@forum.nginx.org" target="_blank" class="">nginx-forum@forum.nginx.org</a>></span> wrote:<br class=""><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">You know you can DoS sites with Cache MISS via switching up URL params and<br class="">
arguements.<br class="">
<br class="">
Examples :<br class="">
<br class="">
HIT :<br class="">
index.php?var1=one&var2=two<br class="">
MISS :<br class="">
index.php?var2=two&var1=one<br class="">
<br class="">
MISS :<br class="">
index.php?random=1<br class="">
index.php?random=2<br class="">
index.php?random=3<br class="">
etc etc<br class="">
<br class="">
Inserting random arguements to URL's will cause cache misses and changing<br class="">
the order of existing valid URL arguements will also cause misses.<br class="">
<br class="">
Cherian Thomas Wrote:<br class="">
------------------------------<wbr class="">-------------------------<br class="">
<div class=""><div class="h5">> Thanks for this Michael.<br class="">
> <br class="">
> <br class="">
> <br class="">
> This is so surprising. If someone decides to Dos and crawls the<br class="">
> website<br class="">
> with a rogue header, this will essentially bypass the cache and put a<br class="">
> strain on the website. In fact, I was hit by a dos attack that’s when<br class="">
> I<br class="">
> started looking at logs and realized the large number of MISSes.<br class="">
> <br class="">
> <br class="">
> <br class="">
> Can someone please help?<br class="">
> <br class="">
> <br class="">
> - Cherian<br class="">
> <br class="">
> On Sat, May 12, 2018 at 12:01 PM, Friscia, Michael<br class="">
> <<a href="mailto:michael.friscia@yale.edu" class="">michael.friscia@yale.edu</a><br class="">
> > wrote:<br class="">
> <br class="">
> > I'm not sure if this will help, but I ignore/hide a lot, this is in<br class="">
> my<br class="">
> > config<br class="">
> ><br class="">
> ><br class="">
> > proxy_ignore_headers X-Accel-Expires Expires Cache-Control<br class="">
> Set-Cookie;<br class="">
> > proxy_hide_header X-Accel-Expires;<br class="">
> > proxy_hide_header Pragma;<br class="">
> > proxy_hide_header Server;<br class="">
> > proxy_hide_header Request-Context;<br class="">
> > proxy_hide_header X-Powered-By;<br class="">
> > proxy_hide_header X-AspNet-Version;<br class="">
> > proxy_hide_header X-AspNetMvc-Version;<br class="">
> ><br class="">
> ><br class="">
> > I have not experienced the problem you mention, I just thought I<br class="">
> would<br class="">
> > offer my config.<br class="">
> ><br class="">
> ><br class="">
> > ______________________________<wbr class="">_____________<br class="">
> ><br class="">
> > Michael Friscia<br class="">
> ><br class="">
> > Office of Communications<br class="">
> ><br class="">
> > Yale School of Medicine<br class="">
> ><br class="">
> > (203) 737-7932 – office<br class="">
> ><br class="">
> > (203) 931-5381 – mobile<br class="">
> ><br class="">
> > <a href="https://mailtrack.io/trace/link/a61adbc81bbb4743e50220408108f7e1b8f3af40?url=http%3A%2F%2Fweb.yale.edu&userId=74734&signature=0767ce63378dc575" rel="noreferrer" target="_blank" class="">http://web.yale.edu</a><br class="">
> ><br class="">
</div></div>> <<a href="https://mailtrack.io/trace/link/661443b9951f60c19cd0ed2ec67ca9c38485a127?url=https%3A%2F%2Fmailtrack.io%2Ftrace%2Flink%2F8357a0bdd8c40c2ff5b7d91c7797cbc7a8535&userId=74734&signature=fd94611bb5198158" rel="noreferrer" target="_blank" class="">https://mailtrack.io/trace/<wbr class="">link/<wbr class="">8357a0bdd8c40c2ff5b7d91c7797cb<wbr class="">c7a8535</a><br class="">
> ffb?url=http%3A%2F%<a href="https://mailtrack.io/trace/link/8d2b22d027b9e7af0a2468545c2e35529237af19?url=http%3A%2F%2F2Fweb.yale.edu&userId=74734&signature=5ab2d28a496b50f6" rel="noreferrer" target="_blank" class="">2Fweb.yale.<wbr class="">edu</a>%2F&userId=74734&signature=<wbr class="">d652edf1f4<br class="">
> f21323><br class="">
> ><br class="">
> ><br class="">
> > ------------------------------<br class="">
> > *From:* nginx <<a href="mailto:nginx-bounces@nginx.org" class="">nginx-bounces@nginx.org</a>> on behalf of Quintin Par <<br class="">
> > <a href="mailto:quintinpar@gmail.com" class="">quintinpar@gmail.com</a>><br class="">
> > *Sent:* Saturday, May 12, 2018 1:32 PM<br class="">
> > *To:* <a href="mailto:nginx@nginx.org" class="">nginx@nginx.org</a><br class="">
> > *Subject:* Re: Debugging Nginx Cache Misses: Hitting high number of<br class="">
<span class="">> MISS<br class="">
> > despite high proxy valid<br class="">
> ><br class="">
> ><br class="">
</span><span class="">> > That’s the tricky part. These MISSes are intermittent. Whenever I<br class="">
> run curl<br class="">
> > I get HITs but I end up seeing a lot of MISS in the logs.<br class="">
> ><br class="">
> ><br class="">
> ><br class="">
> > How do I log these MiSSes with the reason? I want to know what<br class="">
> headers<br class="">
> > ended up bypassing the cache.<br class="">
> ><br class="">
> ><br class="">
> ><br class="">
> > Here’s my caching config<br class="">
> ><br class="">
> ><br class="">
> ><br class="">
> > proxy_pass <a href="https://mailtrack.io/trace/link/071291057b0a07a97c3170df6ceb9706ad0e553d?url=http%3A%2F%2F127.0.0.1%3A8000&userId=74734&signature=21d883fe1973c407" rel="noreferrer" target="_blank" class="">http://127.0.0.1:8000</a><br class="">
> ><br class="">
</span>> <<a href="https://mailtrack.io/trace/link/6864e1b6645eae9d83bd78154bd244cbd3132407?url=https%3A%2F%2Furldefense.proofpoint.com%2Fv2%2Furl%3Fu%3Dhttp-3A__127.0.0.1-3A8000%26&userId=74734&signature=05baa72c55f6e580" rel="noreferrer" target="_blank" class="">https://urldefense.<wbr class="">proofpoint.com/v2/url?u=http-<wbr class="">3A__127.0.0.1-3A8000&</a><br class="">
> d=DwMFaQ&c=cjytLXgP8ixuoHflwc-<wbr class="">poQ&r=<wbr class="">wvXEDjvtDPcv7AlldT5UvDx32KXBEM<wbr class="">6um_<br class="">
> lS023SJrs&m=F-qGMOyS74uE8JM-<wbr class="">dOLmNH92bQ1xQ-7Rj1d6k-_WST4&s=<wbr class="">NHvlb1WColNw<br class="">
> TWBF36P1whJdu5iWHK9_<wbr class="">50IDHugaEdQ&e=><br class="">
<div class=""><div class="h5">> > ;<br class="">
> ><br class="">
> > proxy_set_header X-Real-IP $remote_addr;<br class="">
> ><br class="">
> > proxy_set_header X-Forwarded-For<br class="">
> > $proxy_add_x_forwarded_for;<br class="">
> ><br class="">
> > proxy_set_header X-Forwarded-Proto https;<br class="">
> ><br class="">
> > proxy_set_header X-Forwarded-Port 443;<br class="">
> ><br class="">
> ><br class="">
> ><br class="">
> > # If logged in, don't cache.<br class="">
> ><br class="">
> > if ($http_cookie ~*<br class="">
> "comment_author_|wordpress_(?!<wbr class="">test_cookie)|wp-postpass_"<br class="">
> > ) {<br class="">
> ><br class="">
> > set $do_not_cache 1;<br class="">
> ><br class="">
> > }<br class="">
> ><br class="">
> > proxy_cache_key "$<a href="scheme://$host$request_uri$" class="">scheme://$host$request_uri$</a><br class="">
> > do_not_cache";<br class="">
> ><br class="">
> > proxy_cache staticfilecache;<br class="">
> ><br class="">
> > add_header Cache-Control public;<br class="">
> ><br class="">
> > proxy_cache_valid 200 120d;<br class="">
> ><br class="">
> > proxy_hide_header "Set-Cookie";<br class="">
> ><br class="">
> > proxy_ignore_headers "Set-Cookie";<br class="">
> ><br class="">
> > proxy_ignore_headers "Cache-Control";<br class="">
> ><br class="">
> > proxy_hide_header "Cache-Control";<br class="">
> ><br class="">
> > proxy_pass_header X-Accel-Expires;<br class="">
> ><br class="">
> ><br class="">
> ><br class="">
> > proxy_set_header Accept-Encoding "";<br class="">
> ><br class="">
> > proxy_ignore_headers Expires;<br class="">
> ><br class="">
> > add_header X-Cache-Status $upstream_cache_status;<br class="">
> ><br class="">
> > proxy_cache_use_stale timeout;<br class="">
> ><br class="">
> > proxy_cache_bypass $arg_nocache $do_not_cache;<br class="">
> > - Quintin<br class="">
> ><br class="">
> ><br class="">
> > On Sat, May 12, 2018 at 10:29 AM Lucas Rolff <<a href="mailto:lucas@lucasrolff.com" class="">lucas@lucasrolff.com</a>><br class="">
> wrote:<br class="">
> ><br class="">
> > It can be as simple as doing a curl to your “origin” url (the one<br class="">
> you<br class="">
> > proxy_pass to) for the files you see that gets a lot of MISS’s – if<br class="">
> there’s<br class="">
> > odd headers such as cookies etc, then you’ll most likely experience<br class="">
> a bad<br class="">
> > cache if your nginx is configured to not ignore those headers.<br class="">
> ><br class="">
> ><br class="">
> ><br class="">
</div></div>> > *From: *nginx <<a href="mailto:nginx-bounces@nginx.org" class="">nginx-bounces@nginx.org</a>> on behalf of Quintin Par <<br class="">
> > <a href="mailto:quintinpar@gmail.com" class="">quintinpar@gmail.com</a>><br class="">
> > *Reply-To: *"<a href="mailto:nginx@nginx.org" class="">nginx@nginx.org</a>" <<a href="mailto:nginx@nginx.org" class="">nginx@nginx.org</a>><br class="">
> > *Date: *Saturday, 12 May 2018 at 18.26<br class="">
> > *To: *"<a href="mailto:nginx@nginx.org" class="">nginx@nginx.org</a>" <<a href="mailto:nginx@nginx.org" class="">nginx@nginx.org</a>><br class="">
> > *Subject: *Debugging Nginx Cache Misses: Hitting high number of MISS<br class="">
<div class=""><div class="h5">> > despite high proxy valid<br class="">
> ><br class="">
> ><br class="">
> ><br class="">
> > [image:<br class="">
> ><br class="">
> <a href="https://mailtrack.io/trace/mail/86a613eb1ce46a4e7fa6f9eb96989cddae6398" rel="noreferrer" target="_blank" class="">https://mailtrack.io/trace/<wbr class="">mail/<wbr class="">86a613eb1ce46a4e7fa6f9eb96989c<wbr class="">ddae6398</a><br class="">
> 00.png?u=74734]<br class="">
> ><br class="">
> > My proxy cache path is set to a very high size<br class="">
> ><br class="">
> ><br class="">
> ><br class="">
> > proxy_cache_path /var/lib/nginx/cache levels=1:2<br class="">
> > keys_zone=staticfilecache:180m max_size=700m;<br class="">
> ><br class="">
> > and the size used is only<br class="">
> ><br class="">
> ><br class="">
> ><br class="">
> > sudo du -sh *<br class="">
> ><br class="">
> > 14M cache<br class="">
> ><br class="">
> > 4.0K proxy<br class="">
> ><br class="">
> > Proxy cache valid is set to<br class="">
> ><br class="">
> ><br class="">
> ><br class="">
> > proxy_cache_valid 200 120d;<br class="">
> ><br class="">
> > I track HIT and MISS via<br class="">
> ><br class="">
> ><br class="">
> ><br class="">
> > add_header X-Cache-Status $upstream_cache_status;<br class="">
> ><br class="">
> > Despite these settings I am seeing a lot of MISSes. And this is for<br class="">
> pages<br class="">
> > I intentionally ran a cache warmer an hour ago.<br class="">
> ><br class="">
> ><br class="">
> ><br class="">
> > How do I debug why these MISSes are happening? How do I find out if<br class="">
> the<br class="">
> > miss was due to eviction, expiration, some rogue header etc? Does<br class="">
> Nginx<br class="">
> > provide commands for this?<br class="">
> ><br class="">
> ><br class="">
> ><br class="">
> > - Quintin<br class="">
> > ______________________________<wbr class="">_________________<br class="">
> > nginx mailing list<br class="">
> > <a href="mailto:nginx@nginx.org" class="">nginx@nginx.org</a><br class="">
> > <a href="https://mailtrack.io/trace/link/956685bf1c269e5b5e505d57769f24a31e3e2442?url=http%3A%2F%2Fmailman.nginx.org%2Fmailman%2Flistinfo%2Fnginx&userId=74734&signature=61a29f8655dde16e" rel="noreferrer" target="_blank" class="">http://mailman.nginx.org/<wbr class="">mailman/listinfo/nginx</a><br class="">
> ><br class="">
</div></div>> <<a href="https://mailtrack.io/trace/link/0f96ef0fff2b29b47c79cd24c346157878aaf2e5?url=https%3A%2F%2Fmailtrack.io%2Ftrace%2Flink%2F122c3dbd333c388f47f5c2776af9ebc3fc75a&userId=74734&signature=0b1e1864a472eee2" rel="noreferrer" target="_blank" class="">https://mailtrack.io/trace/<wbr class="">link/<wbr class="">122c3dbd333c388f47f5c2776af9eb<wbr class="">c3fc75a</a><br class="">
> e10?url=https%3A%2F%<a href="https://mailtrack.io/trace/link/5a068de37a59a883da6fd59fdd4026a152a7fc91?url=http%3A%2F%2F2Furldefense.proofpoint.com&userId=74734&signature=ca8f6ddc8276a370" rel="noreferrer" target="_blank" class="">2Furldefen<wbr class="">se.proofpoint.com</a>%2Fv2%2Furl%<wbr class="">3Fu%3Dhttp-<br class="">
> 3A__mailman.nginx.org_mailman_<wbr class="">listinfo_nginx%26d%3DDwMFaQ%<wbr class="">26c%3DcjytLX<br class="">
> gP8ixuoHflwc-poQ%26r%<wbr class="">3DwvXEDjvtDPcv7AlldT5UvDx32KXB<wbr class="">EM6um_lS023SJrs%26m<br class="">
> %3DF-qGMOyS74uE8JM-<wbr class="">dOLmNH92bQ1xQ-7Rj1d6k-_WST4%<wbr class="">26s%3DD3LnZhfobOtlEStCv<br class="">
> CDrcwmHydEHaGRFC4gnWvRT5Uk%<wbr class="">26e%3D&userId=74734&signature=<wbr class="">56c7a7ad18b2c<br class="">
> 057><br class="">
<span class="">> ><br class="">
> ><br class="">
> > ______________________________<wbr class="">_________________<br class="">
> > nginx mailing list<br class="">
> > <a href="mailto:nginx@nginx.org" class="">nginx@nginx.org</a><br class="">
> > <a href="https://mailtrack.io/trace/link/f500ef35fc0275c82402a7af89180ae2c67cea6a?url=http%3A%2F%2Fmailman.nginx.org%2Fmailman%2Flistinfo%2Fnginx&userId=74734&signature=aa7675f47e061eec" rel="noreferrer" target="_blank" class="">http://mailman.nginx.org/<wbr class="">mailman/listinfo/nginx</a><br class="">
> ><br class="">
</span>> <<a href="https://mailtrack.io/trace/link/d6afed06499ad18204cf041056d4781772869d72?url=https%3A%2F%2Fmailtrack.io%2Ftrace%2Flink%2F92c2700d67bd6891ca1606e2df4e0f11c6d82&userId=74734&signature=59dcf4fe89ac3c3c" rel="noreferrer" target="_blank" class="">https://mailtrack.io/trace/<wbr class="">link/<wbr class="">92c2700d67bd6891ca1606e2df4e0f<wbr class="">11c6d82</a><br class="">
> 260?url=http%3A%2F%<a href="https://mailtrack.io/trace/link/3ec600220aa90db4d165256c22910f3c97fa118d?url=http%3A%2F%2F2Fmailman.nginx.org&userId=74734&signature=c116773b55639f01" rel="noreferrer" target="_blank" class="">2Fmailman.<wbr class="">nginx.org</a>%2Fmailman%<wbr class="">2Flistinfo%2Fnginx&us<br class="">
> erId=74734&signature=<wbr class="">3763121afa828bb7><br class="">
<span class="">> ><br class="">
> ______________________________<wbr class="">_________________<br class="">
> nginx mailing list<br class="">
> <a href="mailto:nginx@nginx.org" class="">nginx@nginx.org</a><br class="">
> <a href="https://mailtrack.io/trace/link/8e6777181b5012ff78b980aafec44306b2954bae?url=http%3A%2F%2Fmailman.nginx.org%2Fmailman%2Flistinfo%2Fnginx&userId=74734&signature=2adebca7901eccce" rel="noreferrer" target="_blank" class="">http://mailman.nginx.org/<wbr class="">mailman/listinfo/nginx</a><br class="">
<br class="">
</span>Posted at Nginx Forum: <a href="https://mailtrack.io/trace/link/89e8f350a5c632ccafaadd90a9a8114ecac2e688?url=https%3A%2F%2Fforum.nginx.org%2Fread.php%3F2%2C279764%2C279771%23msg-279771&userId=74734&signature=3a01022d1b56bd07" rel="noreferrer" target="_blank" class="">https://forum.nginx.org/read.<wbr class="">php?2,279764,279771#msg-279771</a><br class="">
<div class="HOEnZb"><div class="h5"><br class="">
______________________________<wbr class="">_________________<br class="">
nginx mailing list<br class="">
<a href="mailto:nginx@nginx.org" class="">nginx@nginx.org</a><br class="">
<a href="https://mailtrack.io/trace/link/8e6777181b5012ff78b980aafec44306b2954bae?url=http%3A%2F%2Fmailman.nginx.org%2Fmailman%2Flistinfo%2Fnginx&userId=74734&signature=2adebca7901eccce" rel="noreferrer" target="_blank" class="">http://mailman.nginx.org/<wbr class="">mailman/listinfo/nginx</a></div></div></blockquote></div><br class=""></div>
_______________________________________________<br class="">nginx mailing list<br class=""><a href="mailto:nginx@nginx.org" class="">nginx@nginx.org</a><br class="">http://mailman.nginx.org/mailman/listinfo/nginx</div></blockquote></div><br class=""></body></html>