RFC on C Library Safety in Nginx Module Callback

[Maxim Dounin]

Hello!

On Thu, Jun 13, 2019 at 09:08:56PM -0400, Sinan Kaya wrote:

> I wanted to hear opinions on this surprising observation while
> developing an nginx module.
> 
> We hit this issue while developing an nginx module. During nginx
> module's  startup process, it starts up nginx but also does a lot of
> other work, some of which involves using glibc's implementation of
> timer_create, a posix function.
> 
> https://code.woboq.org/userspace/glibc/sysdeps/unix/sysv/linux/timer_create.c.html
> https://code.woboq.org/userspace/glibc/sysdeps/unix/sysv/linux/timer_routines.c.html#__active_timer_sigev_thread_lock
> 
> 
> Looking at the glibc source code for timer_create, we can see it has a
> global mutex named __active_timer_sigev_thread_lock.  If nginx happens
> to call fork() while the rest of the nginx module code is calling
> timer_create, the fork() will make a copy of global memory which
> includes the global mutex as being owned by some other thread. In the
> newly-forked process, that thread will not exist, and the mutex will
> always be owned by a non-existent thread. I'm pasting in a GDB trace
> showing this below the rest of my explanatory text.

[...]

> By looking at this link below, almost majority of the C library
> functions are unsafe when called inside the nginx callback.

Almost all C library functions are safe when called inside nginx 
callbacks.  The only (or at least most obvious) exceptions are 
creating threads (you shouldn't, this is highly likely to cause 
problems described in your message) and processes (you'll end up 
confusing nginx process management code).

As long as you don't create your own threads, everything is 
expected to be fine.  If you don't create your own threads but 
still see the deadlock as shown in your trace - please provide 
more details.

-- 
Maxim Dounin
http://mdounin.ru/