slice module + thread pools

[Maxim Dounin]

Hello!

On Tue, Feb 23, 2016 at 11:22:45AM +0300, Vadim Lazovskiy wrote:

[...]

> Проблема 2 (aio включено slice включен):
> Если запустить  скачивание файла через прокси, скорость очень низкая,
> соединение постоянно разрывается. В логах при этом:
> 2016/02/23 10:36:11 [alert] 11124#11124: task #1 already active
> 2016/02/23 10:36:12 [alert] 11124#11124: task #4 already active
> 2016/02/23 10:36:14 [alert] 11124#11124: task #7 already active
> 2016/02/23 10:36:17 [alert] 11124#11124: task #10 already active
> 2016/02/23 10:37:59 [alert] 11124#11124: task #23 already active
> 2016/02/23 10:38:07 [alert] 11124#11124: task #35 already active
> 2016/02/23 10:38:17 [alert] 11124#11124: task #37 already active
> 
> Если отключить slice module, все становится хорошо.

Это выглядит как проблема при aio threads + sendfile + подзапросы.  
Workaround - выключить что-нибудь из списка.  Just for the record, 
воспроизводится с помощью тестов как-то так:

TEST_NGINX_GLOBALS_HTTP="aio threads; sendfile on;" prove proxy_store.t

Патч:

# HG changeset patch
# User Maxim Dounin <mdounin at mdounin.ru>
# Date 1456417157 -10800
#      Thu Feb 25 19:19:17 2016 +0300
# Node ID 06da00f231e74bbb8dbb55fd6abd88ca8b207917
# Parent  463609ba52b07e8e669d83ca7ca7fa754ae5355a
Fixed sendfile in threads when used with subrequests.

If sendfile in threads where used, it was possible that multiple
subrequests will trigger multiple ngx_linux_sendfile_thread() calls,
as operations are only serialized in output chain based on r->aio,
that is, on subrequest level.

This resulted in "task #N already active" alerts, in particular, when
running proxy_store.t with "aio threads; sendfile on;".

Fix is to tolerate duplicate calls, with an additional safety check
that the file is the same as previously used.

diff --git a/src/os/unix/ngx_linux_sendfile_chain.c b/src/os/unix/ngx_linux_sendfile_chain.c
--- a/src/os/unix/ngx_linux_sendfile_chain.c
+++ b/src/os/unix/ngx_linux_sendfile_chain.c
@@ -354,6 +354,18 @@ ngx_linux_sendfile_thread(ngx_connection
         return (ctx->sent == ctx->size) ? NGX_DONE : NGX_AGAIN;
     }
 
+    if (task->event.active && ctx->file == file) {
+        /*
+         * tolerate duplicate calls with the same file;
+         * it can happen due to subrequests, as r->aio only serializes
+         * operations within a single subrequest
+         */
+
+        *sent = 0;
+
+        return NGX_OK;
+    }
+
     ctx->file = file;
     ctx->socket = c->fd;
     ctx->size = size;

-- 
Maxim Dounin
http://nginx.org/