0.8.52 AIO problem

[Maxim Dounin]

Hello!

On Fri, Oct 08, 2010 at 10:20:36PM +0400, Maxim Dounin wrote:

> On Fri, Oct 08, 2010 at 06:45:08PM +0200, "Brane F. Gračnar" wrote:
> 
> > On 10/07/2010 05:25 PM, Maxim Dounin wrote:
> > > 
> > > Backtrace is mostly useless here as it shows trace to check, not 
> > > to the real problem. Though it shows that you may want to
> > > recompile again with lower optimization to simplify debugging 
> > > (note this "<value optimized out>" things).  I usually debug 
> > > with -O0 as it minimizes compiler artifacts.  For CPU-bound hosts 
> > > you may want to use -O1 instead.
> > 
> > Ok, nginx recompiled again :)
> 
> [...]
> 
> > See attachments for all worker processes stacktraces.
> 
> As I already said, backtrace is useless.  You have to provide 
> output of various gdb commands I wrote in previous message.
> 
> It should enough to examine only one core, but please keep it (and 
> coresponding binary) for future reference.
> 
> > Main server error log (700KB):
> > http://frost.ath.cx/.tmp/main-err.log.gz
> > 
> > Vhost error log (2.3MB):
> > http://frost.ath.cx/.tmp/vhost-err.log.gz
> 
> It would be much better to have combined one to preserve order of 
> messages.  I.e. just comment out error_log in vhost.
> 
> And, more importantly, I don't see "open socket left" alerts in 
> these logs.  Looks like they were omitted somehow.

Forget, I was able to trace the problem (and reproduce it here) 
from debug logs provided.  Please try the attached patch.

Maxim Dounin
-------------- next part --------------
# HG changeset patch
# User Maxim Dounin <mdounin at mdounin.ru>
# Date 1286592074 -14400
# Node ID c8168bda2a1631dee3ade68f3fe112bbc6ebbfdb
# Parent  94b30aa5d8497cc409579f3253dfa3522c55a28b
Fix socket leak with AIO.

When output buffer size was less than file size we are going to send second
aio post happened if non-first aio operation on the file in question was not
able to complete immediately.

Additionally, on Linux aio event's ready flag wasn't reset after submitting
io request, and as a result there were no "[alert] ... second aio post"
messages.

diff --git a/src/core/ngx_output_chain.c b/src/core/ngx_output_chain.c
--- a/src/core/ngx_output_chain.c
+++ b/src/core/ngx_output_chain.c
@@ -74,18 +74,18 @@ ngx_output_chain(ngx_output_chain_ctx_t 
         }
     }
 
-#if (NGX_HAVE_FILE_AIO)
-    if (ctx->aio) {
-        return NGX_AGAIN;
-    }
-#endif
-
     out = NULL;
     last_out = &out;
     last = NGX_NONE;
 
     for ( ;; ) {
 
+#if (NGX_HAVE_FILE_AIO)
+        if (ctx->aio) {
+            return NGX_AGAIN;
+        }
+#endif
+
         while (ctx->in) {
 
             /*
diff --git a/src/http/ngx_http_copy_filter_module.c b/src/http/ngx_http_copy_filter_module.c
--- a/src/http/ngx_http_copy_filter_module.c
+++ b/src/http/ngx_http_copy_filter_module.c
@@ -211,6 +211,8 @@ ngx_http_copy_aio_handler(ngx_output_cha
 
     r->main->blocked++;
     r->aio = 1;
+
+    ctx->aio = 1;
 }
 
 
diff --git a/src/os/unix/ngx_linux_aio_read.c b/src/os/unix/ngx_linux_aio_read.c
--- a/src/os/unix/ngx_linux_aio_read.c
+++ b/src/os/unix/ngx_linux_aio_read.c
@@ -95,6 +95,10 @@ ngx_file_aio_read(ngx_file_t *file, u_ch
     n = io_submit(ngx_aio_ctx, 1, piocb);
 
     if (n == 1) {
+        ev->active = 1;
+        ev->ready = 0;
+        ev->complete = 0;
+
         return NGX_AGAIN;
     }