[NGINX] cache process manager consume a lot of CPU at reload and has to be restarted

[Srebrenko Šehić]

2011/4/1 Jérôme Loyet <jerome at loyet.net>:

Hi,

> I had a similar issue 2 hours before when 2 worker processes segfault. As
> I'm using a custom mudule, I didn't link the two events.

I am seeing this issue as well. Based on my initial tests, the issue
seems to manifest itself easily with the following approach:

1. Configure a simple reverse proxy setup with caching enabled
2. Warm up the cache with some data
3. Shut down the upstreams so requests from clients start to timeout
(upstream must not answer on the IP level)
4. Send SIGHUP to master process few times.

After a few retries, nginx cache manager will go into an infinite loop
and start consuming 100% CPU. If the SIGHUP exercise is repeated, you
will also end up with more than one cache manager process. Then, you
will start seeing stuff like this in your error log:

2011/04/04 13:08:28 [crit] 5036#0: unlink()
"/cache/static/3/4/4b/3563f09ec1e13b6f479efb8db6ea94b4" failed (2: No
such file or directory)

I will try to get a gdb trace of running wild process.

How can I help find the root cause of this problem? It's pretty vital
that you can reliably reload configuration.

Thanks,
Srebrenko