nginx crash only when using Chromium (in ubuntu)

Sun Dec 30 16:10:03 UTC 2012

i could not find the cause that only when using Chromium i get a crash but
when using Firefox i never don't.
some hints to the nginx experts that might help:
1. i use my handler module + filter module. (when module is disabled - no
crash)
2. i use C++ code in shared lib and sometimes the crash is in the c++ object
deconstructor . the object is allocated on the stack (not ptr, just regular
declaration like: obj_t obj1) and freed automatically and end of function.
3. i attach here the headers of FF / CHR browsers.
4. when using valgrind - i get some warnings (see below) but never crash,
even in CHR
5. the nginx runs on vurtual machine (centos 6.3) under ubuntu 12.10. the
browser runs on the ubuntu.
6. the response handler runs when subrequest returns from an upstream
server, then the handler continues and goes to the filter module.
7. sometimes when using palloc i got alignment errors so i used pnalloc. is
it the source of the bug ? when to use palloc and when to use pnalloc ? (see
below the function that uses pnalloc)
8. when restarting nginx and doing CTRL+F5 in CHR browser (right after the
previous crash) - its easy to get another crash again with the same stack
trace, while when browsing to anbother page - it takes time to reproduce the
crash.

 ===============

Thread [1] (Suspended: Signal 'SIGABRT' received. Description: Aborted.)	
	15 raise()  0x00007ffff64e18a5	
	14 abort()  0x00007ffff64e3085	
	13 __libc_message()  0x00007ffff651efe7	
	12 malloc_printerr()  0x00007ffff6524916	
	11 _int_free()  0x00007ffff6527443	
	10 ngx_destroy_pool() ngx_palloc.c:87 0x0000000000406a22	
	9 ngx_http_free_request() ngx_http_request.c:3081 0x000000000044dbfb	
	8 ngx_http_close_request() ngx_http_request.c:3006 0x000000000044d9b3	
	7 ngx_http_terminate_handler() ngx_http_request.c:2176 0x000000000044bc38	
	6 ngx_http_run_posted_requests() ngx_http_request.c:1903
0x000000000044b1ad	
	5 ngx_http_request_handler() ngx_http_request.c:1869 0x000000000044b0b6	
	4 ngx_epoll_process_events() ngx_epoll_module.c:683 0x00000000004377d6	
	3 ngx_process_events_and_timers() ngx_event.c:247 0x00000000004281f4	
	2 ngx_single_process_cycle() ngx_process_cycle.c:316 0x0000000000434442	
	1 main() nginx.c:409 0x0000000000403cdc	

valgrind:
==27496==  Address 0x90c0b2d is 29 bytes inside a block of size 3,366
free'd
==27496==    at 0x4C2645F: operator delete(void*) (vg_replace_malloc.c:387)
==27496==    by 0x59B73AD: SBB::ResponseBean::~ResponseBean() (in
/usr/local/lib/libClientAPI-C-Lib.so)
==27496==    by 0x57ABB04: ngx_sbb_med_handle_va_response (in
/usr/local/lib/libngx_sbb_mediator.so)
==27496==    by 0x4A933D: ngx_sbb_va_response_handler
(ngx_sbb_module.c:274)
==27496==    by 0x4AA372: ngx_sbb_post_subrequest_handler
(ngx_sbb_mod_utils.c:89)
==27496==    by 0x44B3C0: ngx_http_finalize_request
(ngx_http_request.c:1961)
==27496==    by 0x465407: ngx_http_upstream_finalize_request
(ngx_http_upstream.c:3095)

CHR headers:
GET /index.php?cat=1&pag=1&det=108 HTTP/1.1
Host: ---
Connection: keep-alive
Cache-Control: max-age=0
User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.4 (KHTML, like
Gecko) Ubuntu/12.10 Chromium/22.0.1229.94 Chrome/22.0.1229.94 Safari/537.4
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Referer: http://yellowmockup.com/index.php?cat=1
Accept-Encoding: gzip,deflate,sdch
Accept-Language: en-US,en;q=0.8,he;q=0.6
Accept-Charset: UTF-8,*;q=0.5
Cookie: adOtr=4aYP5; PRLST=Ya; UTGv2=h4a59e6b096ada50ad0a1243f0549366c032;
x-autozoom=150f; SPSI=56aa48be644d6ac8ccec5dd82ade576d

FF headers:
GET /index.php?cat=1&pag=1&det=108 HTTP/1.1
Host: ---
User-Agent: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:16.0) Gecko/20100101
Firefox/16.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate
Connection: keep-alive
Cookie: UTGv2=h430c577bc94965b18d99cd502407af14a80;
SPSI=63c40df4be7823f2acbc8e966a8817df; PRLST=zi/Jv/DT; adOtr=04Hd6
Pragma: no-cache
Cache-Control: no-cache

another crash dump:
Thread [1] (Suspended: Signal 'SIGSEGV' received. Description: Segmentation
fault.)	
	16 memcpy()  0x00007ffff65381ab	
	15 sbb_strncpy() ngx_sbb_utils.c:12 0x00000000004a9e5f	
	14 ngx_sbb_utils_str2char() ngx_sbb_mod_utils.c:253 0x00000000004aaab7	
	13 ngx_sbb_med_prepare_va_request()  0x00007ffff725d7b4	
	12 ngx_sbb_handler() ngx_sbb_module.c:229 0x00000000004a913d	
	11 ngx_http_core_rewrite_phase() ngx_http_core_module.c:931
0x000000000043d2a1	
	10 ngx_http_core_run_phases() ngx_http_core_module.c:877
0x000000000043d103	
	9 ngx_http_handler() ngx_http_core_module.c:860 0x000000000043d07a	
	8 ngx_http_process_request() ngx_http_request.c:1687 0x000000000044ac51	
	7 ngx_http_process_request_headers() ngx_http_request.c:1135
0x0000000000449809	
	6 ngx_http_process_request_line() ngx_http_request.c:933
0x0000000000448fbe	
	5 ngx_http_init_request() ngx_http_request.c:519 0x000000000044873f	
	4 ngx_epoll_process_events() ngx_epoll_module.c:683 0x00000000004377d6	
	3 ngx_process_events_and_timers() ngx_event.c:247 0x00000000004281f4	
	2 ngx_single_process_cycle() ngx_process_cycle.c:316 0x0000000000434442	
	1 main() nginx.c:409 0x0000000000403cdc	
=============

// copies exactly n bytes from src to dest, then adds null in n+1 (alloc dst
to n+1 first !)
u_char * sbb_strncpy(u_char *dst, u_char *src, size_t n)
{
	memcpy(dst, src, n);
	dst[n] = '\0';

	return dst;
}

// allocate, copy and add terminating null. do not return null but null_str
to avoid segmentation fault later (dereferencing null ptr)
u_char* ngx_sbb_utils_str2char(ngx_http_request_t *r, ngx_str_t *ngx_str)
{
	u_char *res = NULL;

	if ( (!ngx_str) || (!r))
		return (u_char*)gv_null_str;

    res = ngx_pnalloc(r->pool, ngx_str->len+1);
    if (!res)
    	return (u_char*)gv_null_str;

    return sbb_strncpy(res, ngx_str->data, ngx_str->len); // adds
terminating null
}

Posted at Nginx Forum: http://forum.nginx.org/read.php?2,234580,234580#msg-234580