Re: After reload, nginx's CPU usage jumped to 100 percent（backtrace provided)

[Maxim Dounin]

Hello!

On Tue, Jul 19, 2011 at 02:16:25PM +0800, 杨镭 wrote:

> Hi,
> 
> we reproduced the problem and below is the content of fcn:
> 
> (gdb) p *fcn
> $1 = {node = {key = 2417939894599905030, left = 0x2b822d64f780, right =
> 0x2b822ada9a00, parent = 0x2b822bac7080, color = 0 '\000',
>     data = 0 '\000'}, queue = {prev = 0x2b822afd63a8, next =
> 0x2b822ad45040}, key = "\177\215\220\215h!?T", count = 1, uses = 1,
>   valid_msec = 0, error = 0, exists = 1, updating = 0, deleting = 1, uniq =
> 7209021, expire = 1311055359, valid_sec = 0,
>   body_start = 905, fs_size = 2}
> (gdb)
> 
> So I guess we should apply the patch?

Yes, this is exactly the case I was thinking about.

With fcn->deleting == 1 cache manger will loop waiting for cache 
item to be actually deleted, but it's not going to happen as cache 
manager holds cache lock.

Maxim Dounin

> 
> 
> On Wed, Jul 13, 2011 at 4:20 PM, 杨镭 <clanherb at gmail.com> wrote:
> 
> > Hi,
> >
> > we didn't take core dump. If we can reproduce the problem, I'll bring you
> > the p *fcn content.
> >
> >
> > On Tue, Jul 12, 2011 at 10:08 PM, Maxim Dounin <mdounin at mdounin.ru> wrote:
> >
> >> Hello!
> >>
> >> On Tue, Jul 12, 2011 at 09:31:07PM +0800, 杨镭 wrote:
> >>
> >> > After reload, all the worker processes, including newly forked ones,
> >> goes
> >> > straight to 100% CPU:
> >>
> >> [...]
> >>
> >> > Also, we took backtrace for cache manger process:
> >> >
> >> >
> >> > Attaching to process 17306
> >> > ngx_http_file_cache_expire (data=<value optimized out>) at
> >> > src/http/ngx_http_file_cache.c:1096
> >> > 1096            if (fcn->count == 0) {
> >> > (gdb) bt
> >> > #0  ngx_http_file_cache_expire (data=<value optimized out>) at
> >> > src/http/ngx_http_file_cache.c:1096
> >>
> >> Could you please show
> >>
> >> p *fcn
> >>
> >> ?
> >>
> >> I think I see the problem, but just to make sure.
> >>
> >> If you are able to reproduce the problem please try the following
> >> patch:
> >>
> >> diff --git a/src/http/ngx_http_file_cache.c
> >> b/src/http/ngx_http_file_cache.c
> >> --- a/src/http/ngx_http_file_cache.c
> >> +++ b/src/http/ngx_http_file_cache.c
> >> @@ -1099,7 +1099,8 @@ ngx_http_file_cache_expire(ngx_http_file
> >>         }
> >>
> >>         if (fcn->deleting) {
> >> -            continue;
> >> +            wait = 1;
> >> +            break;
> >>         }
> >>
> >>         p = ngx_hex_dump(key, (u_char *) &fcn->node.key,
> >>
> >>
> >> Maxim Dounin
> >>
> >> _______________________________________________
> >> nginx mailing list
> >> nginx at nginx.org
> >> http://nginx.org/mailman/listinfo/nginx
> >>
> >
> >
> >
> > --
> > lei yang
> >
> 
> 
> 
> -- 
> lei yang

> _______________________________________________
> nginx mailing list
> nginx at nginx.org
> http://nginx.org/mailman/listinfo/nginx