[NGINX] cache process manager consume a lot of CPU at reload and has to be restarted

[Maxim Dounin]

Hello!

On Thu, May 19, 2011 at 03:51:11PM -0400, omega8cc wrote:

> Unfortunately, it still happens also for 1.0.2 release.
> 
> I have seen it over 10 times on two different servers since upgrade to
> 1.0.0, then 1.0.1 and 1.0.2.
> 
> Not sure how to debug this, as it is random issue, yet causes downtime
> until our auto-healing will kill all nginx processes and wait for load
> to stabilize to start nginx again.

There were two issues discussed in this thread which caused cpu 
hog in cache manager:

1. Earlier SIGSEGV in worker process while holding cache zone lock 
with resulting ngx_spinlock() loop in cache manager.  Fix is to
trace and fix this earlier SIGSEGV (looking though error log 
should help, btw).

2. Cache manager was looping if it wasn't able to made any 
progress.  This was fixed in 1.0.0 (patch present in this thread 
was committed).

I suspect your problem is (1), though it may be some other problem 
as well.  Please provide at least backtrace of runaway process to 
make further debugging possible.  This may be done with

# gdb /path/to/nginx <pid-of-runaway-process>
bt

(and please make sure your binary isn't stripped, else backtrace 
will contain only '??' instead of function names)

Running something like ktrace/strace may be also helpfull.

Maxim Dounin