[PATCH][bugfix] Upstream: clear the delayed flag to prevent blocking from sending.

[胡聪 (hucc)]

Hi,

If the problem reproduction is hard, the use of OpenResty can make it easier,
and may be too easy. But the bug do exists, and belongs to nginx.


------------------ Original ------------------
From:  "胡聪 (hucc)";<hucong.c at foxmail.com>;
Send time: Friday, Feb 17, 2017 11:18 AM
To: "nginx-devel"<nginx-devel at nginx.org>;
Subject:  [PATCH][bugfix] Upstream: clear the delayed flag to prevent blockingfrom sending.

Hi,

To reproduce the scene of send blocking, there must be more than two
subrequest, and the more the better.

When I encounter this problem, my config is as follows (roughly):
slice                       1M;
limit_rate                2M;
proxy_buffering       on
proxy_buffer_size    32K;
proxy_buffers    4    64K;
proxy_pass       http://domain/uri;

And my system:
Linux ** 2.6.32-642.13.1.el6.x86_64 #1 SMP Wed Jan 11 20:56:24
UTC 2017 x86_64 x86_64 x86_64 GNU/Linux

Patchs bellow:

# HG changeset patch
# User hucongcong <hucong.c at foxmail.com>
# Date 1487298087 -28800
#      Fri Feb 17 10:21:27 2017 +0800
# Node ID 37b376790ecf83eaaff2f024eb3093a079091d93
# Parent  05fd0dc8f0dc808219f727dd18a5da2f078c4073
Upstream: clear the delayed flag to prevent blocking from sending.

Suppose that proxy_buffering is on and limit_rate is defined, the send will be
blocked in write filter when the following two conditions are met: First, the
previous upstream request sets the write->delayed flag and starts the timer of
downstream, and then finalized. Second, the timer timed out before the next
subrequest enters the function ngx_http_upstream_send_response(), which means
the delayed flag is not be cleared properly. Thus, the data transmission from
upstream to downstream will be blocked in subrequest.

By clearing the delayed flag of c->write in ngx_http_upstream_send_response()
when the timer of write event is deleted, to solve this problem.

diff -r 05fd0dc8f0dc -r 37b376790ecf src/http/ngx_http_upstream.c
--- a/src/http/ngx_http_upstream.c	Thu Feb 16 18:37:22 2017 +0300
+++ b/src/http/ngx_http_upstream.c	Fri Feb 17 10:21:27 2017 +0800
@@ -2848,6 +2848,10 @@ ngx_http_upstream_send_response(ngx_http
 
     c = r->connection;
 
+    if (c->write->delayed && !c->write->timer_set) {
+        c->write->delayed = 0;
+    }
+
     if (r->header_only) {
 
         if (!u->buffering) {