New nginx module: ngx_softlimit

[Zeljko Tomic]

On Thu, 1 Jul 2010 07:43:15 +0400
Maxim Dounin <mdounin at mdounin.ru> wrote:

> Hello!
> 
> On Thu, Jul 01, 2010 at 02:29:12AM +0200, Zeljko Tomic wrote:
> 
> > On Thu, 1 Jul 2010 02:55:04 +0400
> > Maxim Dounin <mdounin at mdounin.ru> wrote:
> > 
> > > Hello!
> > > 
> > > On Wed, Jun 30, 2010 at 07:35:06PM +0200, scyth wrote:
> > > 
> > > > On Wed, 30 Jun 2010 20:53:58 +0400
> > > > Maxim Dounin <mdounin at mdounin.ru> wrote:
> > > > 
> > > > > Hello!
> > > > > 
> > > > > On Wed, Jun 30, 2010 at 06:28:51PM +0200, Zeljko Tomic 
> > > > > wrote:
> > > > > 
> > > > > > I wrote a small, but sometimes very useful module for 
> > > > > > limiting the number of active connections, in a "soft" 
> > > > > > way.
> > > > > > 
> > > > > > Description and usage:
> > > > > > -----------------------
> > > > > > In situation when using nginx for serving static files 
> > > > > > and primarily as a proxy/proxy_cache with some slower 
> > > > > > backend (or even php fastcgi), limits of connections and 
> > > > > > requests that nginx can handle are much higher than they 
> > > > > > are on backend server(s).  When you know the limits of 
> > > > > > your backend, it is bad practice to apply those limits 
> > > > > > on nginx's worker_connections, because it's obvious that 
> > > > > > when limits are met, new requests won't come through and 
> > > > > > this leaves you with bad visitor experience.
> > > > > 
> > > > > It would be a good idea to clarify differences from 
> > > > > limit_conn module.  Right now it looks like use cases you 
> > > > > consider are better handled by limit_conn.
> > > > 
> > > > The difference is that limit_conn (from limit_zone module) 
> > > > limits the number of simultaneous connections PER IP.
> > > 
> > > No.  It limits number of simultaneous connections per variable 
> > > value.  You are free to use any variable, including one 
> > > identical for all requests.
> > > 
> > > See here for details:
> > > 
> > > http://wiki.nginx.org/NginxHttpLimitZoneModule#limit_zone
> > > 
> > 
> > I couldn't find variable which holds current number of 
> > connections, or any number of connections for that matter.
> 
> There is no need to find variable.  Just using something like 
> 
>     limit_zone one $constant 16k;
> 
>     ...
> 
>     location / { 
>         set $constant 1;
>         limit_conn one 100;
> 
>         ...
>     }
> 
> would limit number of currently running requests to 100 in the 
> location in question.  

I see. I guess documentation can be updated to explain this a bit more.

> 
> > > > This is useful in many cases, but soflimit_module, as stated 
> > > > above, compares its limit with total number of active 
> > > > connections, which is different. For example, site can have 
> > > > 2000 different clients with 1 connection each, but backend 
> > > > can handle only 300 without performance issues. Actual 
> > > > limits of course vary from site to site.
> > > 
> > > As long as you are trying to limit *backend* load - you 
> > > probably don't want to use number of active connections (you 
> > > are talking about ngx_stat_active, right? sorry, I haven't had 
> > > time to read your code yet) as it includes keepalive 
> > > connections, connections just sending requests to us and 
> > > connections for non-backend requests.
> > > 
> > > limit_conn limits number of currently running requests in 
> > > certain location (from preaccess phase till request pool 
> > > cleanup) and this is much closer to real number of backend 
> > > connections than ngx_stat_active (though it's still not 
> > > perfect as it includes reading request body and sending big 
> > > buffered responses).
> > > 
> > > ...not even talking about multiple backends with quite a 
> > > different limits running behind the same nginx frontend...
> > 
> > I could not agree more, but there are no precise internal 
> > counters for what counts as backend connection. What I found is 
> > that there is a close relationship in real world between total 
> > active connections and backend connections (with keepalive 
> > ones), so setting the value a bit higher is not a problem. 
> > Finding the appropriate relationship is up to sysadmin, and 
> > isn't very hard to do. But as I said earlier, I will update the 
> > code to use more precise counter as soon as one becomes 
> > available. Also, next version will have "mode" setting, for 
> > users to choose which counter to use, where I'll probably cover 
> > most useful combinations.
> 
> I hardly understand idea of "finding appropriate relationship" 
> while with limit_conn you may just to set exact number.  
> Additionally, "relationships" tend to be quite different under 
> extreme conditions which we are trying to prevent.
> 
> And no, "more precise counters" aren't likely to be ever available 
> as long as you consider multiple locations and so on.
> 
> > Another thing I don't like about limit_conn is that returns 
> > error 503. Sure I can redirect such errors to "maintenance.html" 
> > page, but I prefer having "maintenance" behavior separated from 
> > regular 50x.
> 
> This is quite questionable.  Status 503 was introduced in HTTP 
> specifically for this use case: report server overloads. 
> Moreover, not using some error code for maintenance are likely 
> hurt from SEO point of view.
> 
> Maxim Dounin
> 
> p.s. Looks like you wrote the module in question just because you 
> were sure it's for per-ip limits only.  It's probably a good idea 
> to say something like "why does this shit always happens to us?!" 
> and start hacking limit_conn instead.  ;)
> 

Because documentation lacks better descriptions and syntax examples?:)
Ok, I'm convinced. 503 is what should be used. I overlooked that one.


> The thing it (and limit_req) really lacks is multiple limits in 
> single location.  E.g. no more than 1 connection per ip, no more 
> than 100 connections in total.

Choosing what counter (or derivative of some simple calculation based on available counters) to use should also be a matter of preference. softlimit module currently works only with *ngx_stat_active, but that can be changed easily. My idea is to have more choices available. It's always a better idea to have such flexibility, because different environments require different settings. I've seen VPSs having trouble with number of connections (very small scale), while regular servers usualy care more about requests and not connections. It's definitely not something to be hardcoded. Any thoughts on putting this in limit_zone? (I know that limit_req handles request limits in similar matter, but other variables should be considered too).

Also, what constitutes a "connection" in conn_limit should be documented better. 


-- 
Zeljko Tomic <scyth at unixreich.com>