worker 100% cpu

Alexander Azarov azarov на aha.ru
Чт Мар 11 11:36:17 MSK 2010


Раз в 3-4 дня случается такое, что nginx worker занимает 100% CPU.  
Приходится делать ему kill -KILL.

$ uname -a
Linux r1 2.6.26-2-vserver-amd64 #1 SMP Thu Nov 5 03:47:07 UTC 2009  
x86_64 GNU/Linux

$ /usr/local/sbin/nginx -V
nginx version: nginx/0.7.64
built by gcc 4.3.2 (Debian 4.3.2-1.1)
TLS SNI support enabled
configure arguments: --prefix=/usr/local --conf-path=/etc/nginx/ 
nginx.conf --pid-path=/var/run/nginx.pid --user=www-data --group=www- 
data --with-md5=/usr --http-proxy-temp-path=/var/cache/nginx/tmp_proxy  
--http-fastcgi-temp-path=/var/cache/nginx/tmp_fcgi --http-client-body- 
temp-path=/var/cache/nginx/tmp_body --with-http_stub_status_module -- 
with-cc-opt=-I/usr/include/pcre --with-ld-opt=-L/usr/lib --with- 
http_perl_module --with-http_realip_module --with-http_sub_module -- 
with-debug --without-mail_pop3_module --without-mail_imap_module -- 
without-mail_smtp_module --with-http_gzip_static_module --with- 
http_ssl_module --with-http_image_filter_module

Конфигурация использует proxy_pass, proxy_cache, image_filter.

Backtrace в gdb:

[Thread debugging using libthread_db enabled]
[New Thread 0x7f77ecd6c6e0 (LWP 31206)]
0x00007f77eb01c723 in writev () from /lib/libc.so.6
#0  0x00007f77eb01c723 in writev () from /lib/libc.so.6
#1  0x000000000042ce54 in ngx_linux_sendfile_chain (c=0x7f77e975afa0,  
in=0x0, limit=2147479551)
    at src/os/unix/ngx_linux_sendfile_chain.c:288
#2  0x000000000044017f in ngx_http_write_filter (r=0x92db20, in=0x0)
    at src/http/ngx_http_write_filter_module.c:238
#3  0x000000000044f250 in ngx_http_chunked_body_filter (r=0x92db20,  
in=0x7fff0f316ac0)
    at src/http/modules/ngx_http_chunked_filter_module.c:84
#4  0x000000000045417c in ngx_http_gzip_body_filter (r=0x92db20, in=0x0)
    at src/http/modules/ngx_http_gzip_filter_module.c:304
#5  0x00000000004550d1 in ngx_http_postpone_filter (r=0x92db20, in=0x0)
    at src/http/ngx_http_postpone_filter_module.c:82
#6  0x0000000000455ff0 in ngx_http_charset_body_filter (r=0x92db20,  
in=0x7fff0f316ac0)
    at src/http/modules/ngx_http_charset_filter_module.c:552
#7  0x00000000004577d2 in ngx_http_ssi_body_filter (r=0x92db20,  
in=0xffffffffffffffff)
    at src/http/modules/ngx_http_ssi_filter_module.c:395
#8  0x000000000045b56a in ngx_http_image_body_filter (r=0x92db20,  
in=0x0)
    at src/http/modules/ngx_http_image_filter_module.c:263
#9  0x000000000045c860 in ngx_http_sub_body_filter (r=0x1f, in=0x0)
    at src/http/modules/ngx_http_sub_filter_module.c:178
#10 0x0000000000412db5 in ngx_output_chain (ctx=0x7e9f38, in=0x0) at  
src/core/ngx_output_chain.c:58
#11 0x0000000000440505 in ngx_http_copy_filter (r=0x92db20, in=0x0)
    at src/http/ngx_http_copy_filter_module.c:110
#12 0x000000000044f4b2 in ngx_http_range_body_filter (r=0x92db20,  
in=0x0)
    at src/http/modules/ngx_http_range_filter_module.c:549
#13 0x0000000000434b31 in ngx_http_output_filter (r=0x92db20, in=0x0)
    at src/http/ngx_http_core_module.c:1690
#14 0x000000000042655c in ngx_event_pipe (p=0x7f6bd0, do_write=<value  
optimized out>)
    at src/event/ngx_event_pipe.c:625
#15 0x0000000000448faf in ngx_http_upstream_process_upstream  
(r=0x92db20, u=0x7f6828)
    at src/http/ngx_http_upstream.c:2502
#16 0x0000000000449134 in ngx_http_upstream_handler (ev=0x7f77e95b2e60)
    at src/http/ngx_http_upstream.c:830
#17 0x000000000042444a in ngx_event_process_posted (cycle=0x830290,  
posted=0x698328)
    at src/event/ngx_event_posted.c:39
#18 0x000000000042426e in ngx_process_events_and_timers (cycle=0x830290)
    at src/event/ngx_event.c:272
#19 0x000000000042ac12 in ngx_worker_process_cycle (cycle=0x830290,  
data=<value optimized out>)
    at src/os/unix/ngx_process_cycle.c:781
#20 0x0000000000429303 in ngx_spawn_process (cycle=0x830290,
    proc=0x42ab1f <ngx_worker_process_cycle>, data=0x0, name=0x47434e  
"worker process", respawn=-4)
    at src/os/unix/ngx_process.c:194
#21 0x000000000042a139 in ngx_start_worker_processes (cycle=0x830290,  
n=6, type=-4)
    at src/os/unix/ngx_process_cycle.c:345
#22 0x000000000042b989 in ngx_master_process_cycle (cycle=0x830290)
    at src/os/unix/ngx_process_cycle.c:239
#23 0x0000000000410151 in main (argc=<value optimized out>, argv=0x17)  
at src/core/nginx.c:389




Подробная информация о списке рассылки nginx-ru