Understanding HttpLimitReqModule

[Adrian von Stechow]

Aha! I didn't know leaky bucket was a (common) thing - thanks for the
link and for clarifying!

On Fri, Feb 25, 2011 at 2:18 PM, Maxim Dounin <mdounin at mdounin.ru> wrote:
> Hello!
>
> On Fri, Feb 25, 2011 at 11:04:34AM +0100, Adrian von Stechow wrote:
>
>> Hi all:
>>
>> I'm trying to understand the HttpLimitReqModule, the wiki is a bit
>> unverbose about the terminology.
>> I'm trying to mimic Apache's mod_evasive module, specifically there is
>> an annoying user that likes to request the same image once every
>> second for hours at a time. I would like to log this and then use
>> fail2ban to block the IP for a specific time. The problem is that the
>> image in question is a legitimate request that shows up on every page
>> of the site in question. What I had in mind:
>>
>> limit_req_zone  $binary_remote_addr  zone=one:1m   rate=50r/m;
>> #offending user: 60r/m
>>
>>     server {
>>         location = /path/to/image.jpg {
>>             limit_req   zone=one  burst=???;
>>             limit_req_log_level error
>>         }
>>
>> The problem is the low rate with which the offending requests are
>> made. mod_evasive lets you set up a timespan in which a specific
>> number of requests are made, while nginx checks "online" if a second
>> request is made after 1/rate. In my case (1 offending request per
>> second), legitimate users would be blocked if they load 2 pages in one
>> second, which of course happens frequently.
>>
>> Any suggestions?
>
> Set burst= to be high enough to accomodate occasional request
> bursts from legitimate users, and rate= to be somewhere between
> typical one for legitimate users (on average during relatively
> long time period) and offending one.
>
> If you just want to block offending user which does 60 requests
> per minute, and your typical users do about 1 requests per
> minute, but occasinally may produce something like 100 requests in
> short time frame, you may set something like:
>
>    limit_req_zone $binary_remote_addr zone=one:1m rate=10r/m;
>
>    location ... {
>        limit_req zone=one burst=100 nodelay;
>    }
>
> This will allow legitimate users to do 10r/m on average and up to
> 100 requests in the very same second.  On the other hand,
> offending user with 60 r/m will start getting 503's after about 2
> minutes: 120 requests with 20 allowed by rate will overflow
> burst.
>
> Please refer to http://en.wikipedia.org/wiki/Leaky_Bucket for
> algorithm details.
>
> I also recommend using "nodelay" flag unless you really want to
> control rate at which requests are passed to backends or something
> like this.
>
> Maxim Dounin
>
> _______________________________________________
> nginx mailing list
> nginx at nginx.org
> http://nginx.org/mailman/listinfo/nginx
>