ngx_http_upstream_copy_allow_ranges() issue when using ModSecurity

Thu Dec 3 10:55:15 UTC 2015

Hey everyone,

looks like nginx versions >= 1.7.7 have issues with the modsecurity module,
because of the new proxy_force_ranges directive. The problem is that
modsecurity calls ngx_http_upstream_header_t->copy_handler() for all
ngx_http_upstream_headers_in headers specified in ngx_http_upstream.

And in ngx_http_upstream_copy_allow_ranges() the check for that
configuration value [1] results in a segfault, because r->upstream->conf is
probably NULL at that point, which causes nginx to crash.

One way to work around this is to set "proxy_force_ranges" to on in nginx
config. However another simple fix is to check if r->upstream->conf exists,
before accessing r->upstream->conf->force_ranges. And this shouldn't change
the behavior of nginx (which changing the value of this flag does).

More info: see @driehuls comment in
https://github.com/SpiderLabs/ModSecurity/issues/823

Also, here is a backtrace of the crash, taken on nginx 1.9.3.2:

$ gdb sbin/nginx cores/core
> GNU gdb (Ubuntu 7.7.1-0ubuntu5~14.04.2) 7.7.1
> Copyright (C) 2014 Free Software Foundation, Inc.
> License GPLv3+: GNU GPL version 3 or later <
> http://gnu.org/licenses/gpl.html>
> This is free software: you are free to change and redistribute it.
> There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
> and "show warranty" for details.
> This GDB was configured as "x86_64-linux-gnu".
> Type "show configuration" for configuration details.
> For bug reporting instructions, please see:
> <http://www.gnu.org/software/gdb/bugs/>.
> Find the GDB manual and other documentation resources online at:
> <http://www.gnu.org/software/gdb/documentation/>.
> For help, type "help".
> Type "apropos word" to search for commands related to "word"...
> Reading symbols from sbin/nginx...done.
>
> [New LWP 12171]
> [Thread debugging using libthread_db enabled]
> Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
> Core was generated by `nginx: worker
> process                                                         '.
> Program terminated with signal SIGSEGV, Segmentation fault.
> #0  0x00000000004682e6 in ngx_http_upstream_copy_allow_ranges (r=0xd11be0,
> h=0x7fff38119ff0, offset=<optimized out>)
>     at src/http/ngx_http_upstream.c:4780
>
> 4780        if (r->upstream->conf->force_ranges) {
> (gdb) bt
> #0  0x00000000004682e6 in ngx_http_upstream_copy_allow_ranges (r=0xd11be0,
> h=0x7fff38119ff0, offset=<optimized out>)
>     at src/http/ngx_http_upstream.c:4780
> #1  0x0000000000d11be0 in ?? ()
> #2  0x000000000000000d in ?? ()
> #3  0x00000000004fab3b in ngx_http_modsecurity_save_headers_out_visitor
> (data=0xd11be0, key=<optimized out>, value=0xf29c5e "bytes")
>     at
> /vagrant/openresty/vendor/mod_security/nginx/modsecurity/ngx_http_modsecurity.c:854
> #4  0x00007f924cd21d7d in apr_table_vdo () from
> /usr/lib/x86_64-linux-gnu/libapr-1.so.0
> #5  0x00007f924cd21e32 in apr_table_do () from
> /usr/lib/x86_64-linux-gnu/libapr-1.so.0
> #6  0x00000000004fb84e in ngx_http_modsecurity_save_headers_out
> (r=0xd11be0)
>     at
> /vagrant/openresty/vendor/mod_security/nginx/modsecurity/ngx_http_modsecurity.c:792
> #7  ngx_http_modsecurity_body_filter (r=<optimized out>, in=<optimized
> out>)
>     at
> /vagrant/openresty/vendor/mod_security/nginx/modsecurity/ngx_http_modsecurity.c:1413
> #8  0x00000000004d1194 in ngx_http_lua_capture_body_filter (r=0xd11be0,
> in=0x7fff3811a350)
>     at ../ngx_lua-0.9.19/src/ngx_http_lua_capturefilter.c:133
> #9  0x0000000000428015 in ngx_output_chain (ctx=ctx at entry=0xf29b10,
> in=in at entry=0x7fff3811a350) at src/core/ngx_output_chain.c:74
> #10 0x000000000045d9d3 in ngx_http_copy_filter (r=0xd11be0,
> in=0x7fff3811a350) at src/http/ngx_http_copy_filter_module.c:152
> #11 0x00000000004531eb in ngx_http_output_filter (r=r at entry=0xd11be0,
> in=in at entry=0x7fff3811a350)
>     at src/http/ngx_http_core_module.c:1969
> #12 0x0000000000456bd3 in ngx_http_send_special (r=r at entry=0xd11be0,
> flags=flags at entry=1) at src/http/ngx_http_request.c:3355
> #13 0x00000000004673e5 in ngx_http_upstream_finalize_request (r=0xd11be0,
> u=0xd570c8, rc=0) at src/http/ngx_http_upstream.c:4071
> #14 0x00000000004681c5 in ngx_http_upstream_process_request (r=0xd11be0,
> u=0xd570c8) at src/http/ngx_http_upstream.c:3667
> #15 0x000000000046a583 in ngx_http_upstream_send_response (u=0xd570c8,
> r=0xd11be0) at src/http/ngx_http_upstream.c:2963
> #16 ngx_http_upstream_process_header (r=0xd11be0, u=0xd570c8) at
> src/http/ngx_http_upstream.c:2165
> #17 0x00000000004674b9 in ngx_http_upstream_handler (ev=<optimized out>)
> at src/http/ngx_http_upstream.c:1092
> #18 0x000000000043d630 in ngx_event_process_posted (cycle=cycle at entry=0xd0bbb0,
> posted=0x7fd780 <ngx_posted_events>)
>     at src/event/ngx_event_posted.c:33
> #19 0x000000000043d210 in ngx_process_events_and_timers (cycle=cycle at entry=0xd0bbb0)
> at src/event/ngx_event.c:259
> #20 0x0000000000442a8d in ngx_worker_process_cycle (cycle=cycle at entry=0xd0bbb0,
> data=data at entry=0x0)
>     at src/os/unix/ngx_process_cycle.c:769
> #21 0x0000000000441534 in ngx_spawn_process (cycle=cycle at entry=0xd0bbb0,
> proc=0x4429e0 <ngx_worker_process_cycle>, data=0x0,
>     name=0x54b804 "worker process", respawn=respawn at entry=0) at
> src/os/unix/ngx_process.c:198
> #22 0x0000000000443b32 in ngx_reap_children (cycle=0xd0bbb0) at
> src/os/unix/ngx_process_cycle.c:621
> #23 ngx_master_process_cycle (cycle=cycle at entry=0xd0bbb0) at
> src/os/unix/ngx_process_cycle.c:174
> #24 0x00000000004236cf in main (argc=<optimized out>, argv=<optimized
> out>) at src/core/nginx.c:415
>

[1]
https://github.com/nginx/nginx/blob/master/src/http/ngx_http_upstream.c#L4788
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx-devel/attachments/20151203/51cc13af/attachment.html>