Nginx Rate limiting for HTTPS requests

[Maxim Dounin]

Hello!

On Tue, May 15, 2018 at 12:20:31PM -0400, rickGsp wrote:

> I have been experimenting with Nginx rate limiting and I need some inputs on
> it’s working and what can be expected from this feature. I see some
> difference in what I expected from this feature going by the documentation
> and what I observed in my experiments.
> 
> Here is the detail on my testing:
> I have a test server running Nginx and a backend server. Nginx is configured
> as HTTPS server listening on 443. I have configured Nginx as reverse proxy
> to my backend. We have a proprietary tool which feeds configured number of
> HTTPS requests (one request/connection) to test server and generates reports
> at the end of test. Report will have details how many requests return status
> as 200 and 503. 
> 
> Observation 1: 
> As per my observations, more requests are getting processed with return
> status as 200 than expected if input request rate to Nginx is much higher
> than the rate limit configured.
> For example, with the following configuration in Nginx for rate limiting,
> Here are my tests:
> limit_req_zone $host zone=perhost:1m rate=100r/s;
> limit_req zone=perhost burst=100 nodelay;
> 
> Test1: With input as 250 req/sec and rate limit configured at 100r/s, rate
> limiting works as expected since on average ~100 requests return with 200
> status every second
>     
> Test2: With input as 500 req/sec and rate limit configured at 100r/s, rate
> limiting does not work as expected since on average ~150 requests return
> with 200 status every second

The question is: how did you get the ~150 r/s number?

As per your description, the tool you are using reports number of 
requests returned corresponding status, but not rate.  Make sure 
that calculation is not based on initial numbers, but counts real 
responses received and uses wall clock to calculate rate.

That is, if you tool is expected to generate 500 r/s load for 10 
seconds (5000 requests in total) and you've got 1500 requests with 
status 200, success rate is _not_ 150 r/s.  To calculate success rate 
properly we need to know how long requests processing took.  E.g., 
if it took 15 seconds from the load start to the last request 
finished, the real rate is 100 r/s.

[...]

> I am not very sure what is happening here for HTTPS based testing. One
> observation I have made is that in HTTP case, requests gets processed very
> quickly whereas for HTTPS case, complete transaction takes relatively
> longer. Also, for low input rate of HTTPS requests transaction completion is
> not taking very long where as when input rate goes up, this delay further
> increase and then rate limiting start behaving unexpectedly. Can this be the
> cause of this difference in anyway? Please share your inputs on this.

Sure, see above.  As long as request processing takes significant 
time, it becomes more important to measure time properly.  Failing 
to do so will result in wrong numbers.

-- 
Maxim Dounin
http://mdounin.ru/