limit_conn is dropping valid connections and causing memory leaks on nginx reload

[Lucas Rolff]

Anoop,

He added v and double quotes around $binary_remote_addr.

Best Regards,

From: nginx <nginx-bounces at nginx.org> on behalf of Anoop Alias <anoopalias01 at gmail.com>
Reply-To: "nginx at nginx.org" <nginx at nginx.org>
Date: Saturday, 30 September 2017 at 12.14
To: Nginx <nginx at nginx.org>
Subject: Re: limit_conn is dropping valid connections and causing memory leaks on nginx reload

What is the change (workaround) you made ?I don't see a difference?

On Sat, Sep 30, 2017 at 3:35 PM, Dejan Grofelnik Pelzel <nginx-forum at forum.nginx.org<mailto:nginx-forum at forum.nginx.org>> wrote:
Hello,

We are running the nginx 1.13.5  with HTTP/2 in a proxy_pass proxy_cache
configuration with clients having relatively long open connections. Our
system does automatic reloads for any new configuration and we recently
introduced a limit_conn to some of the config files. After that, I've
started noticing a rapid drop in connections and outgoing network every-time
the system would perform a configuration reload. Even stranger, on every
reload the memory usage would go up for about 1-2GB until ultimately
everything crashed if the reloads were too frequent. The memory usage did go
down after old workers were released, but that could take up to 30 minutes,
while the configuration could get reloaded up to twice per minute.

We used the following configuration as recommended by pretty much any
example:
limit_conn_zone $binary_remote_addr zone=1234con:10m;
limit_conn zone1234con 10;

I was able to verify the connection drop by doing a simple ab test, for
example, I would run ab -c 100 -n -k 1000 https://127.0.0.1/file.bin
990 of the connections went through, however, 10 would still be active.
Immediately after the reload, those would get dropped as well. Adding -r
option would help the problem, but that doesn't fix our problem.

Finally, after I tried to create a workaround, I've configured the limit
zone to:
limit_conn_zone "v$binary_remote_addr" zone=1234con:10m;

Suddenly everything magically started to work. The connections were not
being dropped, the limit worked as expected and even more surprisingly the
memory usage was not going up anymore. I've been tearing my hair out almost
all day yesterday trying to figure this out. While I was very happy to see
this resolved, I am now confused as to why nginx behaves in such a way.

I'm thinking this might likely be a bug, so I'm just wondering if anyone
could explain why it is happening or has a similar problem.

Thank you!

Posted at Nginx Forum: https://forum.nginx.org/read.php?2,276633,276633#msg-276633

_______________________________________________
nginx mailing list
nginx at nginx.org<mailto:nginx at nginx.org>
http://mailman.nginx.org/mailman/listinfo/nginx



--
Anoop P Alias

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx/attachments/20170930/3729f547/attachment-0001.html>