"Hacking" Nginx events second part

[François Battail]

Just a followup to my previous thread, but I think it can interest 
some developpers.

Nginx is great! I've tried before to "hack" Lighttpd, but it was a 
pain in the a*s. It takes me some times to enter Nginx but I don't 
regret it at all. Thanks Igor for this superb work.

So, I was trying to call (long) blocking functions inside Nginx. 
First approach has been to create worker threads, a queue and a 
thread which is doing the epoll_wait and put in the queue 
epoll related events as well as my workers events. It was fine 
but a bit complicated and performance was around 10000 
requests / s (a dynamic 10 kB document).

Clearly:

static request -> nginx -> msg -> nginx
dynamic request -> nginx > msg -> my worker -> msg -> nginx

I've just discovered signalfd(). It's now (with a recent Linux 
kernel) possible to associate a file descriptor with a signal 
handler and it's compatible with epoll_wait :) so now it's:

static request -> nginx
dynamic request -> nginx -> msg -> my worker -> signal -> nginx

So in the event loop, it's like :

nevents = epoll_wait (...event_list...) ;

for (i = 0 ; i < nevents ; i ++)
{
  if (event_list [i].data.fd == my_fd)
  {
    process_my_eventS (...) ;
    continue ;
  }
  ...
}

process_my_eventS (...)
{
  msg_t msg ;

  while (1)
  {
    poll_msg (q, & msg) ;
    if (read (myfd...) != sizeof (signalfd_siginfo))
      break ;
    handle_http_results (msg.request) ;
  }
}

Of course I still need a queue to get the context of the request, 
but it works fine (100.10^6 requests so far without a glitch 
using ab).

>From my benchmarks it's now ~ 15500 requests / s (10 kB dynamic 
document).

Just to compare:

Apache 2.2 / Prefork MPM / PHP 5 with index.php:

<?php
  echo '<10kB of ascii data>' ;
?>

1800 requests / s, 80 load!

And with Nginx:

15500 requests / s, ~ 2 load 8)

Using ab -k -c 50 -n 1000000 http://127.0.0.1:8000/

On an AMD X2 4200 / 2 GB / Ubuntu 8.04 RC1.

Sorry to bother you with this long message...