Seg fault in http read event handler caused by rouge event call without context

[Dave Brennan]

For the last few years we have been using the "nginx_upload" module to streamline result posting within our environment.

With the introduction of nginx 1.17.5 we saw a large number of segment faults, causing us to revert to 1.17.4 on our development system.

While isolating the fault we added an increase in debug messages to monitor the request and context variables being passed to event handlers.

So a good response in 1.17.4 looks like this:-

2019/11/14 10:24:21 [debug] 12398#12398: *9770 Upload handle pre alloc Request address = 0000563E9FE451F0 Context = 0000000000000000
2019/11/14 10:24:21 [debug] 12398#12398: *9770 Upload Handler post alloc Request address = 0000563E9FE451F0 Context = 0000563E9FE81CD8
2019/11/14 10:24:21 [debug] 12398#12398: *9770 Upload_eval_path Request address = 0000563E9FE451F0 Context = 0000563E9FE81CD8
2019/11/14 10:24:21 [debug] 12398#12398: *9770 Upload eval state path Request address = 0000563E9FE451F0 Context = 0000563E9FE81CD8
2019/11/14 10:24:21 [debug] 12398#12398: *9770 Upload client read Request address = 0000563E9FE451F0 Context = 0000563E9FE81CD8
2019/11/14 10:24:21 [debug] 12398#12398: *9770 do read upload client Request address = 0000563E9FE451F0 Context = 0000563E9FE81CD8
2019/11/14 10:24:21 [debug] 12398#12398: *9770 process request body Request address = 0000563E9FE451F0 Context = 0000563E9FE81CD8
2019/11/14 10:24:21 [debug] 12398#12398: *9770 Upload variable Request address = 0000563E9FE451F0 Context = 0000563E9FE81CD8
2019/11/14 10:24:21 [debug] 12398#12398: *9770 Upload variable Request address = 0000563E9FE451F0 Context = 0000563E9FE81CD8
2019/11/14 10:24:21 [debug] 12398#12398: *9770 Upload variable Request address = 0000563E9FE451F0 Context = 0000563E9FE81CD8
2019/11/14 10:24:21 [debug] 12398#12398: *9770 Upload variable Request address = 0000563E9FE451F0 Context = 0000563E9FE81CD8
2019/11/14 10:24:21 [debug] 12398#12398: *9770 Upload variable Request address = 0000563E9FE451F0 Context = 0000563E9FE81CD8
2019/11/14 10:24:21 [debug] 12398#12398: *9770 Upload variable Request address = 0000563E9FE451F0 Context = 0000563E9FE81CD8
2019/11/14 10:24:21 [debug] 12398#12398: *9770 Upload variable Request address = 0000563E9FE451F0 Context = 0000563E9FE81CD8
2019/11/14 10:24:21 [debug] 12398#12398: *9770 upload md5 variable Request address = 0000563E9FE451F0 Context = 0000563E9FE81CD8
2019/11/14 10:24:21 [debug] 12398#12398: *9770 Upload variable Request address = 0000563E9FE451F0 Context = 0000563E9FE81CD8
2019/11/14 10:24:21 [debug] 12398#12398: *9770 Upload File size variable Request address = 0000563E9FE451F0 Context = 0000563E9FE81CD8
2019/11/14 10:24:21 [debug] 12398#12398: *9770 Upload Body Handler Request address = 0000563E9FE451F0 Context = 0000563E9FE81CD8

In 1.17.5 the event stream looks like this:-

2019/11/13 14:21:52 [debug] 28086#28086: *3703 Upload handle pre alloc Request address = 0000558ADDD4F780 Context = 0000000000000000
2019/11/13 14:21:52 [debug] 28086#28086: *3703 Upload Handler post alloc Request address = 0000558ADDD4F780 Context = 0000558ADDD49FF8
2019/11/13 14:21:52 [debug] 28086#28086: *3703 Upload_eval_path Request address = 0000558ADDD4F780 Context = 0000558ADDD49FF8
2019/11/13 14:21:52 [debug] 28086#28086: *3703 Upload eval state path Request address = 0000558ADDD4F780 Context = 0000558ADDD49FF8
2019/11/13 14:21:52 [debug] 28086#28086: *3703 Upload client read Request address = 0000558ADDD4F780 Context = 0000558ADDD49FF8
2019/11/13 14:21:52 [debug] 28086#28086: *3703 do read upload client Request address = 0000558ADDD4F780 Context = 0000558ADDD49FF8
2019/11/13 14:21:52 [debug] 28086#28086: *3703 process request body Request address = 0000558ADDD4F780 Context = 0000558ADDD49FF8
2019/11/13 14:21:52 [debug] 28086#28086: *3703 Upload variable Request address = 0000558ADDD4F780 Context = 0000558ADDD49FF8
2019/11/13 14:21:52 [debug] 28086#28086: *3703 Upload variable Request address = 0000558ADDD4F780 Context = 0000558ADDD49FF8
2019/11/13 14:21:52 [debug] 28086#28086: *3703 Upload variable Request address = 0000558ADDD4F780 Context = 0000558ADDD49FF8
2019/11/13 14:21:52 [debug] 28086#28086: *3703 Upload variable Request address = 0000558ADDD4F780 Context = 0000558ADDD49FF8
2019/11/13 14:21:52 [debug] 28086#28086: *3703 Upload variable Request address = 0000558ADDD4F780 Context = 0000558ADDD49FF8
2019/11/13 14:21:52 [debug] 28086#28086: *3703 Upload variable Request address = 0000558ADDD4F780 Context = 0000558ADDD49FF8
2019/11/13 14:21:52 [debug] 28086#28086: *3703 process request body Request address = 0000558ADDD4F780 Context = 0000558ADDD49FF8
2019/11/13 14:21:52 [debug] 28086#28086: *3703 Upload variable Request address = 0000558ADDD4F780 Context = 0000558ADDD49FF8
2019/11/13 14:21:52 [debug] 28086#28086: *3703 upload md5 variable Request address = 0000558ADDD4F780 Context = 0000558ADDD49FF8
2019/11/13 14:21:52 [debug] 28086#28086: *3703 Upload variable Request address = 0000558ADDD4F780 Context = 0000558ADDD49FF8
2019/11/13 14:21:52 [debug] 28086#28086: *3703 Upload File size variable Request address = 0000558ADDD4F780 Context = 0000558ADDD49FF8
2019/11/13 14:21:52 [debug] 28086#28086: *3703 Upload Body Handler Request address = 0000558ADDD4F780 Context = 0000558ADDD49FF8

2019/11/13 14:21:52 [debug] 28086#28086: *3703 read upload clent request body Request address = 0000558ADDD4F780 Context = 0000000000000000
2019/11/13 14:21:52 [debug] 28086#28086: *3703 do read upload client Request address = 0000558ADDD4F780 Context = 0000000000000000


There appears to be  an extra call to the request "read event" and although the request address has not changed the context address returned by:-

ngx_http_upload_ctx_t     *u = ngx_http_get_module_ctx(r, ngx_http_upload_module);

Returns NULL, which causes any reference to the context table to cause a segment fault.

While it is possible to work round this by checking for a NULL context, the read event appears to be rouge when compared to the previous  version of nginx, and I can only assume has been generated by code changes in 1.17.5.



Dave Brennan
Cyber Protection Senior Technologist

CORVID Protect Holdings Limited, trading as CORVID Protect, is registered in Guernsey, company number FC034204, whose registered office is at Royal Bank Place, 1 Glategny Esplanade, St Peter Port, Guernsey GY1 4ND. CORVID Protect Holdings Limited is a subsidiary company of Ultra Electronics Holdings plc registered in England and Wales, company number 02830397, whose registered office is at 35 Portman Square, London W1H 6LR.

Ultra Electronics is committed to safeguarding the privacy of all personal data: data privacy notice. Email communications may be monitored by us, as permitted by applicable law and regulations. This email is confidential and may also be privileged. If you have received this message in error you should notify the sender immediately by reply e-mail and delete the message from your system.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx-devel/attachments/20191118/2e7a0183/attachment.htm>