[nginx] Limit req: improved handling of negative times.

[Maxim Dounin]

details:   http://hg.nginx.org/nginx/rev/bd6563e81cea
branches:  
changeset: 7281:bd6563e81cea
user:      Maxim Dounin <mdounin at mdounin.ru>
date:      Wed May 30 15:40:34 2018 +0300
description:
Limit req: improved handling of negative times.

Negative times can appear since workers only update time on an event
loop iteration start.  If a worker was blocked for a long time during
an event loop iteration, it is possible that another worker already
updated the time stored in the node.  As such, time since last update
of the node (ms) will be negative.

Previous code used ngx_abs(ms) in the calculations.  That is, negative
times were effectively treated as positive ones.  As a result, it was
not possible to maintain high request rates, where the same node can be
updated multiple times from during an event loop iteration.
In particular, this affected setups with many SSL handshakes, see
http://mailman.nginx.org/pipermail/nginx/2018-May/056291.html.

Fix is to only update the last update time stored in the node if the
new time is larger than previously stored one.  If a future time is
stored in the node, we preserve this time as is.

To prevent breaking things on platforms without monotonic time available
if system time is updated backwards, a safety limit of 60 seconds is
used.  If the time stored in the node is more than 60 seconds in the future,
we assume that the time was changed backwards and update lr->last
to the current time.

diffstat:

 src/http/modules/ngx_http_limit_req_module.c |  29 ++++++++++++++++++++++++---
 1 files changed, 25 insertions(+), 4 deletions(-)

diffs (58 lines):

diff --git a/src/http/modules/ngx_http_limit_req_module.c b/src/http/modules/ngx_http_limit_req_module.c
--- a/src/http/modules/ngx_http_limit_req_module.c
+++ b/src/http/modules/ngx_http_limit_req_module.c
@@ -399,7 +399,14 @@ ngx_http_limit_req_lookup(ngx_http_limit
 
             ms = (ngx_msec_int_t) (now - lr->last);
 
-            excess = lr->excess - ctx->rate * ngx_abs(ms) / 1000 + 1000;
+            if (ms < -60000) {
+                ms = 1;
+
+            } else if (ms < 0) {
+                ms = 0;
+            }
+
+            excess = lr->excess - ctx->rate * ms / 1000 + 1000;
 
             if (excess < 0) {
                 excess = 0;
@@ -413,7 +420,11 @@ ngx_http_limit_req_lookup(ngx_http_limit
 
             if (account) {
                 lr->excess = excess;
-                lr->last = now;
+
+                if (ms) {
+                    lr->last = now;
+                }
+
                 return NGX_OK;
             }
 
@@ -509,13 +520,23 @@ ngx_http_limit_req_account(ngx_http_limi
         now = ngx_current_msec;
         ms = (ngx_msec_int_t) (now - lr->last);
 
-        excess = lr->excess - ctx->rate * ngx_abs(ms) / 1000 + 1000;
+        if (ms < -60000) {
+            ms = 1;
+
+        } else if (ms < 0) {
+            ms = 0;
+        }
+
+        excess = lr->excess - ctx->rate * ms / 1000 + 1000;
 
         if (excess < 0) {
             excess = 0;
         }
 
-        lr->last = now;
+        if (ms) {
+            lr->last = now;
+        }
+
         lr->excess = excess;
         lr->count--;