segfault in 1.1.4

[Maxim Dounin]

Hello!

On Wed, Sep 21, 2011 at 01:05:49PM +0200, Jérôme Loyet wrote:

> Hi,
> 
> after migrating to 1.1.4 from 1.0.4, nginx would segfault on some cases.
> 
> I've tracked down the problem to a double free in the upstream module.
> Here is a patch. I don't know if it's the right solution to fix this:
> 
> Index: src/http/ngx_http_upstream.c
> ===================================================================
> --- src/http/ngx_http_upstream.c        (revision 4142)
> +++ src/http/ngx_http_upstream.c        (working copy)
> @@ -2924,6 +2924,7 @@
> 
>          if (u->peer.connection->pool) {
>              ngx_destroy_pool(u->peer.connection->pool);
> +            u->peer.connection->pool = NULL;
>          }
> 
>          ngx_close_connection(u->peer.connection);
> @@ -3022,6 +3023,7 @@
> 
>          if (u->peer.connection->pool) {
>              ngx_destroy_pool(u->peer.connection->pool);
> +            u->peer.connection->pool = NULL;
>          }
> 
>          ngx_close_connection(u->peer.connection);
> 
> 
> Here are the last log I have:
> 
> 2011/09/21 13:03:29 [error] 2890#0: *13 no live upstreams while
> connecting to upstream, client: 78.226.14.200, server:
> direct.php.dev.fatbsd.com, request: "GET /ping HTTP/1.1", upstream:
> "fastcgi://127.0.0.1:9001", host: "direct.php.dev.fatbsd.com"
> *** glibc detected *** nginx: worker process: double free or
> corruption (!prev): 0x00000000011b84a0 ***
> 
> and the gdb stacktrace:
> 
> (gdb) bt
> #0  0x00007f200ae07d05 in raise () from /lib/x86_64-linux-gnu/libc.so.6
> #1  0x00007f200ae0bab6 in abort () from /lib/x86_64-linux-gnu/libc.so.6
> #2  0x00007f200ae40d7b in ?? () from /lib/x86_64-linux-gnu/libc.so.6
> #3  0x00007f200ae4ca8f in ?? () from /lib/x86_64-linux-gnu/libc.so.6
> #4  0x00007f200ae508e3 in free () from /lib/x86_64-linux-gnu/libc.so.6
> #5  0x000000000040ce5c in ngx_destroy_pool (pool=0x11b84a0) at
> src/core/ngx_palloc.c:86
> #6  0x0000000000448361 in ngx_http_upstream_finalize_request
> (r=0x1194610, u=0x1195af8, rc=502) at
> src/http/ngx_http_upstream.c:3024
> #7  0x000000000044a28e in ngx_http_upstream_process_header
> (r=0x1194610, u=0x1195af8) at src/http/ngx_http_upstream.c:1510
> #8  0x0000000000448a09 in ngx_http_upstream_handler (ev=<value
> optimized out>) at src/http/ngx_http_upstream.c:926
> #9  0x0000000000429e6b in ngx_epoll_process_events (cycle=<value
> optimized out>, timer=<value optimized out>, flags=<value optimized
> out>) at src/event/modules/ngx_epoll_module.c:678
> #10 0x0000000000422257 in ngx_process_events_and_timers
> (cycle=0x113acd0) at src/event/ngx_event.c:245
> #11 0x000000000042848c in ngx_worker_process_cycle (cycle=0x113acd0,
> data=<value optimized out>) at src/os/unix/ngx_process_cycle.c:801
> #12 0x0000000000426b4c in ngx_spawn_process (cycle=0x113acd0,
> proc=0x4283c0 <ngx_worker_process_cycle>, data=0x0, name=0x4cbae3
> "worker process", respawn=0) at src/os/unix/ngx_process.c:196
> #13 0x0000000000429328 in ngx_reap_children (cycle=0x113acd0) at
> src/os/unix/ngx_process_cycle.c:617
> #14 ngx_master_process_cycle (cycle=0x113acd0) at
> src/os/unix/ngx_process_cycle.c:180
> #15 0x000000000040b912 in main (argc=<value optimized out>,
> argv=<value optimized out>) at src/core/nginx.c:405
> 
> hope it helps

Good catch, thanks.  It looks like this resulted in "connection 
already closed" alerts previously and may resulted in problems 
with ssl upstream connections (up to similar segmentation faults).

Better patch would be

diff --git a/src/http/ngx_http_upstream.c b/src/http/ngx_http_upstream.c
--- a/src/http/ngx_http_upstream.c
+++ b/src/http/ngx_http_upstream.c
@@ -2927,6 +2927,7 @@ ngx_http_upstream_next(ngx_http_request_
         }

         ngx_close_connection(u->peer.connection);
+        u->peer.connection = NULL;
     }

 #if 0

Maxim Dounin