Behaviour of map changes when using it's result in limit_req/limit_req_zone

[Francis Daly]

On Fri, Oct 11, 2019 at 10:30:14AM +0200, Felix Gläske wrote:

Hi there,

> I want to throttle requests based on upstream response codes.

It is unlikely to be useful to try to limit incoming requests based on
something that is not in the incoming request.

(More specifically: based on something that is not available at the time
that the limiting should happen.)

> If a user (repeatedly) fails to login, and therefor the upstream server responds with 403, the requests should be throttled. Not for 2xx or other response codes. Only 403.
> 
> Online I found a solution using a `map` and `limit_req_zone`/`limit_req`.

Could you link to this solution? Perhaps it is valid in some circumstances.

>   http {
>     map $status $bad_guy {
>       ~^[2] 1;
>       ~^[4] $remote_addr;
>       default "default";
>     }
> 
>     map $status $wup {
>       ~^[2] 1;
>       ~^[4] $remote_addr;
>       default "default";
>     }

http://nginx.org/r/$status -- that is the response status.

>     log_format   main '$remote_addr - $status ->$wup<- ->$bad_guy<- ';
>     access_log   /dev/stdout  main;
> 
>     limit_req_zone $bad_guy zone=by_status:10m rate=60r/m;

>       location / {
>         proxy_pass http://127.0.0.1:8888$request_uri;
>         limit_req zone=by_status burst=2;

A request comes in. "limit_req" says "check if this should be limited".

limit_req_zone says "what value has $bad_guy got right now?".

map says "$status is empty, therefore $bad_guy is default".

limiting happens based on "default".

After the request completes, "log_format" wants the values of four
variables. The nginx-built-in ones have their appropriate values. The
extra ones have whatever value they have -- $bad_guy is "default" because
that is what "map" set it to previously; "$wup" was not set previously,
so now "map" sets it based on $status -- which now is not empty.

You can mark $bad_guy "volatile" (http://nginx.org/r/map) if you want
"map" to set it to a new value at log time; but that will not affect
the fact that limit_req was based on the value "default".


> I would assume that in the log statement the values for $wup and $bad_guy are the same since they are based on maps with the same rules.

No.

http://nginx.org/r/map:

"""
Since variables are evaluated only when they are used, the mere
declaration even of a large number of “map” variables does not add
any extra costs to request processing.
"""

> Let's change the configuration a little bit and comment out the line "limit_req zone=by_status burst=2;"
> 
> Scenario 3: Upstream reports 403
> Log:
>   127.0.0.1 - 403 ->127.0.0.1<- ->127.0.0.1<-
> 
> Ok, now the values seem all right.

Same analysis as above, except in this config $bad_guy is not used before
"log_format", so it has no value before logging, so "map" sets it based
on the value of $status at logging time.

> Final question: Why does using "limit_req" change the behaviour/outcome of the "map"?

Timing.

	f
-- 
Francis Daly        francis at daoine.org