nginx cache growing well above max_size threshold

[Jeff Dyke]

You can actually can run H/2 through HAProxy, using ALPN to determine if
the client understands H/2  I have the following (snippet of a) config that
sends to different nginx ports based on the ALPN response.

frontend https
  mode tcp
  bind 0.0.0.0:443 ssl crt /etc/haproxy/certs alpn h2,http/1.1 ecdhe
secp384r1
  http-response set-header Strict-Transport-Security "max-age=16000000;
includeSubDomains; preload;"
  timeout http-request 10s
  #send all HTTP/2 traffic to a specific backend
  use_backend http2-nodes if { ssl_fc_alpn -i h2 }
  #send HTTP/1.1 and HTTP/1.0 to default, which don't speak HTTP/2
  default_backend http1-nodes

backend http1-nodes
  mode http
  balance roundrobin
  default-server inter 1s fall 2

  server web01 10.1.1.12:80 check send-proxy
  server web03 10.1.1.14:80 check send-proxy

backend http2-nodes
  mode tcp
  balance roundrobin
  default-server inter 1s fall 2

  server web01 10.1.1.12:81 check send-proxy
  server web03 10.1.1.14:81 check send-proxy

Sounds like you may not want to complicate this system ATM, but just
throwing it out there.  It's worked really well for me, i have had i live
for about a year.


On Thu, Sep 14, 2017 at 1:09 PM, Lucas Rolff <lucas at lucasrolff.com> wrote:

> Okay cool, I'll give it a try
>
> In our case we do not run http2 on the machines since haproxy runs in
> front as well (which doesn't support http2)
>
> I'll also try enable a bit more verbose logging on one of the machines to
> see what the logs say
>
> Thanks a lot Maxim!
>
> Best regards,
> Lucas Rolff
>
> Get Outlook for iOS <https://aka.ms/o0ukef>
> ------------------------------
> *From:* nginx <nginx-bounces at nginx.org> on behalf of Maxim Dounin <
> mdounin at mdounin.ru>
> *Sent:* Thursday, September 14, 2017 6:55:57 PM
> *To:* nginx at nginx.org
> *Subject:* Re: nginx cache growing well above max_size threshold
>
> Hello!
>
> On Thu, Sep 14, 2017 at 04:34:09PM +0000, Lucas Rolff wrote:
>
> > I have a minor question, so I have an nginx box using
> > proxy_cache, it has a key zone of 40 gigabyte (so it can cache
> > 320 million files), a max_size of 1500 gigabyte for the cache
> > and the inactive set to 30 days.
> >
> > However we experience that nginx goes well above the defined
> > limit - in our case the max size is 1500 gigabyte, but the cache
> > directory takes goes well above 1700 gigabyte.
> >
> > There's a total of 42.000.000 files currently on the system,
> > meaning the average filesize is about 43 kilobyte.
> >
> > Normally I know that nginx can go slightly above the limit,
> > until the cache manager purges the files, but it stays at about
> > 1700 gigabyte constantly unless we manually clear out the size.
> >
> > I see there's a change in 1.13.1 that ignores long locked cache
> > entries, is it possible that this bugfix actually fixes above
> > issue?
> >
> > Upgrading is rather time consuming and we have to ensure nginx
> > versions across the platform, so I wonder if anyone has some
> > pointers if the above bugfix would maybe solve our issue.
> > (currently the custom nginx version is based on nginx 1.10.3).
>
> https://trac.nginx.org/nginx/ticket/1163
>
> TL;DR:
>
> This behaviour indicate there is a problem somewhere, likely
> socket leaks or process crashes.  Reports suggests it might be
> related to HTTP/2.  The change in 1.13.1 don't fix the root cause,
> but will allow nginx to keep cache under max_size regardless of
> the problem.
>
> --
> Maxim Dounin
> http://nginx.org/
> _______________________________________________
> nginx mailing list
> nginx at nginx.org
> http://mailman.nginx.org/mailman/listinfo/nginx
>
> _______________________________________________
> nginx mailing list
> nginx at nginx.org
> http://mailman.nginx.org/mailman/listinfo/nginx
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx/attachments/20170914/32209472/attachment.html>