[dev] threads support in nginx

Manlio Perillo manlio_perillo at libero.it
Mon Sep 17 00:02:27 MSD 2007

Igor Sysoev ha scritto:
> On Sun, Sep 16, 2007 at 08:51:20PM +0200, Manlio Perillo wrote:
>> Since mod_wsgi is almost ready, I have to find the best way to not block 
>>  nginx when running a WSGI application.
>> There are two solutions:
>> 1) Implements a Python module for accessing the nginx event module, so
>>    that the application can do asynchronous programming
>> 2) Use threads
>> The second solution is, unfortunately (I do not like threads) the only 
>> way to support existing applications.
> This is why I said that python, like perl, etc. is not suitable
> language to embed to the event-driven servers such as nginx.

The problem is not with Python or Perl, but with application programming.

It is true that interpreted languages are slower then C, but usually 
this is not a problem (as an example, Mercurial is written in Python and 
it is "fast").

>> I'm reading the nginx threads support code and, as far as I can understand:
>> - to execute a job in a separate thread, a module can post an event to
>>   a worker thread
>> - the code that runs in the separate thread can access the nginx
>>   "internals", since these are protected using a mutex
>> However there is an alternate method to support threads, and it is the 
>> solution adopted by Twisted (an asynchronous framework written in 
>> Python): when a code that runs in a separate thread needs to access 
>> shared code, it posts back a new event[1] to the main thread.
>> This solution has the great advantage to simplify the implementation of 
>> thread safe code, since this code is limited to a well defined place, 
>> and the rest of the framework can safely be thread unsafe.
>> Igor, is this solution applicable to nginx?
> Now the thread support is broken in nginx. They requires rethinking
> and locking in various places.
> The threads are not intended to run whole request in one thread.
> The request in his life time may be handled by various threads.

With the Twisted model, here is how I can handle a blocking code that 
produce data (pseudo code):

def handler(request):
    # execute the blocking function is a separate thread
    callInThread(handler_thread, request)
    return # return control to the reactor

def handler_thread(request):
    while True:
       data = slow_read()
       # execute unsafe code in the main thread
       callFromThread(handler_write, request, data)

    callFromThread(request_finish, request)

def handler_write(request, data):

def request_finish(request):

The problem with nginx, if I'm right, is that request.write can return a 

Its infortunate that with nginx it is not possible to use this simple 

Thanks and regards   Manlio Perillo

More information about the nginx mailing list