[PATCH] Fixed Nginx 1.1.1 eating 100% CPU time on occasions

Artyom Gavrichenkov ximaera at highloadlab.com
Tue Aug 30 17:22:20 UTC 2011


Problem:

2011/08/30 19:35:05 [debug] 3186#0: *5193 http upstream request:
"/ximaera/images/whiting_buddh.png?"
2011/08/30 19:35:05 [debug] 3186#0: *5193 http upstream process header
2011/08/30 19:35:05 [debug] 3186#0: *5193 malloc: 0000000002239950:65536
2011/08/30 19:35:05 [debug] 3186#0: *5193 recv: fd:252 0 of 65536
2011/08/30 19:35:05 [error] 3186#0: *5193 upstream prematurely closed
connection while reading response header from upstream, client:
217.26.0.104, server: , request: "GET
/ximaera/images/whiting_buddh.png HTTP/1.1", upstream:
"http://192.168.1.5:80/ximaera/images/whiting_buddh.png", host:
"www.ximaera.name", referrer: "http://twitter.com/"
2011/08/30 19:35:05 [debug] 3186#0: *5193 http next upstream, 2
2011/08/30 19:35:05 [debug] 3186#0: *5193 free keepalive peer
2011/08/30 19:35:05 [debug] 3186#0: *5193 free rr peer 1 4
2011/08/30 19:35:05 [debug] 3186#0: *5193 free rr peer failed: 0 -1
2011/08/30 19:35:05 [debug] 3186#0: *5193 close http upstream connection: 252
2011/08/30 19:35:05 [debug] 3186#0: *5193 event timer del: 252: 1314718565804
2011/08/30 19:35:05 [debug] 3186#0: *5193 reusable connection: 0
2011/08/30 19:35:05 [debug] 3186#0: *5193 get keepalive peer
2011/08/30 19:35:05 [debug] 3186#0: *5193 get rr peer, try: 0
2011/08/30 19:35:05 [debug] 3186#0: *5193 [XIMAERA] before
round_robin.c:505, try: 0
2011/08/30 19:35:05 [debug] 3186#0: *5193 [XIMAERA] before
round_robin.c:508, try: 18446744073709551615
2011/08/30 19:35:05 [debug] 3186#0: *5193 [XIMAERA] before
round_robin.c:514, try: 18446744073709551615

After unsuccessful attempt to
ngx_http_upstream_free_round_robin_peer() we had (pc->tries == 0).
Then we tried to ngx_http_upstream_get_round_robin_peer() with
(pc->tries == 0 && rrp->peers->number == 1).
On ngx_http_upstream_round_robin.c:505 we did pc->tries-- and started
to decrement 0xffffffffffffffff to zero.

Quick and probably dirty (but working) patch follows.

Signed-off-by: Artyom Gavrichenkov <ximaera at highloadlab.com>

diff -Nurp nginx-1.1.1.orig/src/http/ngx_http_upstream_round_robin.c
nginx-1.1.1/src/http/ngx_http_upstream_round_robin.c
--- nginx-1.1.1.orig/src/http/ngx_http_upstream_round_robin.c
2011-08-18 21:04:52.000000000 +0400
+++ nginx-1.1.1/src/http/ngx_http_upstream_round_robin.c
2011-08-30 21:22:40.000000000 +0400
@@ -473,6 +473,10 @@ ngx_http_upstream_get_round_robin_peer(n

         } else {

+            if (pc->tries == 0) {
+                goto failed;
+            }
+
             i = pc->tries;

             for ( ;; ) {



More information about the nginx-devel mailing list