nginx crash

[Maxim Dounin]

Hello!

On Sat, Nov 26, 2011 at 10:07:48PM +0800, MagicBear wrote:

> Here is the crash bt.

[...]

> warning: Can't read pathname for load map: Input/output error.
> [Thread debugging using libthread_db enabled]
> Core was generated by `nginx:'.
> Program terminated with signal 11, Segmentation fault.
> #0  ngx_rbtree_min (sentinel=<optimized out>, node=0x0) at
> src/core/ngx_rbtree.h:75
> 75	    while (node->left != sentinel) {
> (gdb) bt
> #0  ngx_rbtree_min (sentinel=<optimized out>, node=0x0) at
> src/core/ngx_rbtree.h:75
> #1  ngx_rbtree_delete (tree=0x7f463a38c000, node=0x7f463a38f000) at
> src/core/ngx_rbtree.c:178
> #2  0x0000000000451a24 in ngx_http_file_cache_free (c=0x2509a70,
> tf=0x250a3a8) at src/http/ngx_http_file_cache.c:956
> #3  0x0000000000447759 in ngx_http_upstream_process_request
> (r=0x24ab650) at src/http/ngx_http_upstream.c:2738
> #4  0x000000000044783b in ngx_http_upstream_process_upstream
> (r=0x24ab650, u=0x24db4c0) at src/http/ngx_http_upstream.c:2677
> #5  0x000000000044792c in ngx_http_upstream_handler (ev=<optimized
> out>) at src/http/ngx_http_upstream.c:935
> #6  0x0000000000425979 in ngx_event_expire_timers () at
> src/event/ngx_event_timer.c:149
> #7  0x00000000004255b0 in ngx_process_events_and_timers
> (cycle=0x2465bf0) at src/event/ngx_event.c:261

[...]

> I have increase the key size to 128M, the problem has disappear for 2
> weeks, but just has happen again.

Unfortunately, backtrace isn't really usefull here.  Bad thing 
happens before this - when a cache manager decides to cleanup 
long locked cache entry, while it's still required.

Could you please add $request_time and $upstream_response_time to 
logs and provide some stats about maximum values (note there may 
be many times in $upstream_response_time for a single request, sum 
of them all is intresting)?

After looking into code again I tend to think that it should work 
as intended, just default inactive= value isn't enough in your case.  
Bumping it to something like "5h" may also help, i.e.

    proxy_cache_path ... inactive=5h;

Maxim Dounin