nginx segfaulting with mod_security

[Robert Paprocki]

I realized that I spoke too soon about gzip shortly after sending this.
My apologies for making a silly assumption; I am a sysadmin by trade and
not so skilled at developing or troubleshooting complex server software.
I've contact mod_security mailing list but haven't heard back from them.
Thank you for your time and patience in answering my questions!

Sincerely,
Robert Paprocki
On 04/14/2014 03:44 AM, Maxim Dounin wrote:
> Hello!
>
> On Sun, Apr 13, 2014 at 08:42:04PM -0700, Robert Paprocki wrote:
>
>> Hi Maxim!
>>
>> Thank you for your response, always nice to actually hear back from
>> someone knowledgeable. Once thing I had noticed while looking at
>> backtraces (coredumps seem to indicate segfaults occurring in a number
>> of places, not just filter_module.c:121) was that every bt seemed to
>> include gzip modules as well. i disabled gzip in both my server and http
>> sections but this did not stop the segfaults (which is interesting, but
> All filters, including gzip, are expected to be in backtraces, as 
> they are always called.  Depending on configuration, they either 
> do something or not.
>
>> not entirely surprising, given that even when I had set SecRuleEngine to
>> off in my modsecurity.conf file, segfaults still occured... so even the
>> mere presence of ModSecurityEnabled in the nginx configuration was
>> leading to a break).
>>
>> I have since recompiled nginx without both gunzip module and gzip static
>> module, and have not yet gotten any segfaults. i will continue to
>> research this and update if I have any more information
> As per backtrace, it's more or less obvious that the problem is in 
> modsecurity.  Compiling out other modules may result in less 
> frequent segfaults, but it won't fix the bug in modsecurity.
>
> If you want to actually fix the problem, you should either 
> switch off / compile out modsecurity, or find and fix the bug in 
> it.
>
>> On 04/13/2014 03:17 AM, Maxim Dounin wrote:
>>> Hello!
>>>
>>> On Sat, Apr 12, 2014 at 04:44:28PM -0700, Robert Paprocki wrote:
>>>
>>>> Hello,
>>>>
>>>> I have compiled nginx-1.5.13 with modsecurity-2.7.7 and am seeing
>>>> occasional segfaults when sending requests to the server. mod_security
>>>> was compiled as a standalone module per the instructions made available
>>>> at
>>>> https://github.com/SpiderLabs/ModSecurity/wiki/Reference-Manual#Installation_for_NGINX.
>>>> The segfaults appear sporadic and do not seem to match up with any given
>>>> request. Below is my nginx configuration:
>>> [...]
>>>
>>>> Also, a backtrace of the core dump:
>>>> (gdb) bt
>>>> #0  0x080a1827 in ngx_http_write_filter (r=0x83bb078, in=0x8baaa6c) at
>>>> src/http/ngx_http_write_filter_module.c:121
>>> This points to the following code line:
>>>
>>>         cl->buf = ln->buf;
>>>
>>> That is, dereferencing ln->buf fails, which may only happen if the 
>>> buffer chain ("in" argument) is broken.
>>>
>>> [...]
>>>
>>>> #8  0x080cfc78 in ngx_http_gunzip_body_filter (r=0x83bb078, in=0x8baaa6c)
>>>>     at src/http/modules/ngx_http_gunzip_filter_module.c:184
>>>> #9  0x081146bd in ngx_http_modsecurity_body_filter (r=0x83bb078,
>>>> in=0xbf7ff8b4)
>>>>     at
>>>> ../modsecurity-apache_2.7.7/nginx/modsecurity//ngx_http_modsecurity.c:1252
>>>> #10 0x08055381 in ngx_output_chain (ctx=0x8baa9b8, in=0xbf7ff8b4) at
>>>> src/core/ngx_output_chain.c:66
>>> And this clearly shows that the buffer chain was chaned by 
>>> mod_security output body filter.  Note "in" argument of 
>>> mod_security ("in=0xbf7ff8b4") and gunzip filter which follows it 
>>> ("in=0x8baaa6c").
>>>
>>> That is, from the backtrace it looks like mod_security changed the 
>>> buffer chain and did it wrong, with a segfault as a result.
>>>
>> _______________________________________________
>> nginx mailing list
>> nginx at nginx.org
>> http://mailman.nginx.org/mailman/listinfo/nginx