[nginx] Improved connection draining with small number of connections.

[Maxim Dounin]

details:   http://hg.nginx.org/nginx/rev/426828549afc
branches:  
changeset: 6873:426828549afc
user:      Maxim Dounin <mdounin at mdounin.ru>
date:      Fri Jan 20 14:03:20 2017 +0300
description:
Improved connection draining with small number of connections.

Closing up to 32 connections might be too aggressive if worker_connections
is set to a comparable number (and/or there are only a small number of
reusable connections).  If an occasional connection shorage happens in
such a configuration, it leads to closing all reusable connections instead
of gradually reducing keepalive timeout to a smaller value.  To improve
granularity in such configurations we now close no more than 1/8 of all
reusable connections at once.

Suggested by Joel Cunningham.

diffstat:

 src/core/ngx_connection.c |  8 ++++++--
 src/core/ngx_cycle.h      |  1 +
 2 files changed, 7 insertions(+), 2 deletions(-)

diffs (46 lines):

diff --git a/src/core/ngx_connection.c b/src/core/ngx_connection.c
--- a/src/core/ngx_connection.c
+++ b/src/core/ngx_connection.c
@@ -1204,6 +1204,7 @@ ngx_reusable_connection(ngx_connection_t
 
     if (c->reusable) {
         ngx_queue_remove(&c->queue);
+        ngx_cycle->reusable_connections_n--;
 
 #if (NGX_STAT_STUB)
         (void) ngx_atomic_fetch_add(ngx_stat_waiting, -1);
@@ -1217,6 +1218,7 @@ ngx_reusable_connection(ngx_connection_t
 
         ngx_queue_insert_head(
             (ngx_queue_t *) &ngx_cycle->reusable_connections_queue, &c->queue);
+        ngx_cycle->reusable_connections_n++;
 
 #if (NGX_STAT_STUB)
         (void) ngx_atomic_fetch_add(ngx_stat_waiting, 1);
@@ -1228,11 +1230,13 @@ ngx_reusable_connection(ngx_connection_t
 static void
 ngx_drain_connections(ngx_cycle_t *cycle)
 {
-    ngx_int_t          i;
+    ngx_uint_t         i, n;
     ngx_queue_t       *q;
     ngx_connection_t  *c;
 
-    for (i = 0; i < 32; i++) {
+    n = ngx_max(ngx_min(32, cycle->reusable_connections_n / 8), 1);
+
+    for (i = 0; i < n; i++) {
         if (ngx_queue_empty(&cycle->reusable_connections_queue)) {
             break;
         }
diff --git a/src/core/ngx_cycle.h b/src/core/ngx_cycle.h
--- a/src/core/ngx_cycle.h
+++ b/src/core/ngx_cycle.h
@@ -53,6 +53,7 @@ struct ngx_cycle_s {
     ngx_uint_t                modules_used;    /* unsigned  modules_used:1; */
 
     ngx_queue_t               reusable_connections_queue;
+    ngx_uint_t                reusable_connections_n;
 
     ngx_array_t               listening;
     ngx_array_t               paths;