ngx_drizzle: make nginx talk directly to mysql, drizzle, and sqlite3 by libdrizzle

[agentzh]

Hi, guys!

This is the last day in 2009 and I'm too impatient to hold the first
release (version v0.0.1) of our ngx_drizzle module, an upstream module
that talks directly to RDBMS backends like mysql, drizzle, and the
drizzle server shipped with libdrizzle for sqlite3.

This module is initially started by my friend and colleague,
chaoslawful. His Chinese name is 王晓哲 :) He did the initial (most
difficult) work all in his own time :) I also shamelessly borrowed a
lot of code from Igor's ngx_http_upstream.c and
ngx_http_memcached_module.c in the nginx 0.8.30 core, as well as Maxim
Dounin's excellent upstream_keepalive module. These parts of code are
copyrighted by these authors, respectively.

This module is still at its very early phase of development and
considered highly experimental. But you're encouraged to test it out
on your side and report any quirks that you experience :)

Here's some sample configurations:

    http {
        ...

        upstream cluster {
            # simple round-robin
            drizzle_server 127.0.0.1:3306 dbname=test
                 password=some_pass user=monty protocol=mysql;
            drizzle_server 127.0.0.1:1234 dbname=test2
                 password=pass user=bob protocol=drizzle;
        }

        upstream backend {
            drizzle_server 127.0.0.1:3306 dbname=test
                 password=some_pass user=monty protocol=mysql;
        }

        server {
            location /mysql {
                set $my_sql 'select * from cats';
                drizzle_query $my_sql;

                drizzle_pass backend;
            }
            ...
        }
    }

Essentially it provides a very efficient and flexible way for nginx
internals to access mysql, drizzle, sqlite3, as well as other RDBMS's
that support the drizzle protocol or mysql protocol. Also it can serve
as a direct REST interface to those RDBMS backends.

It also has a builtin per-worker connection pool mechanism borrowed
from Maxim Dounin's upstream_keepalive module.

    Here's a sample configuration:

        upstream backend {
            drizzle_server 127.0.0.1:3306 dbname=test
                 password=some_pass user=monty protocol=mysql;
            drizzle_keepalive max=100 mode=single overflow=reject;
        }

You may wonder why it will be useful for your PHP/Python/Perl/Java
applications fastcgi'd or proxied by nginx. Well, we'll work out an
ngx_accel_subrequest module some time in the future to allow these
backend apps directly issue subrequests by means of the special
X-Accel-Subrequest header and continuation passing style.

Unlike the current X-Accel-Redirect trick we're already familiar with,
X-Accel-Subrequest is more like a function invocation that issues one
(or multiple parallel subrequests), and eventually *returns* back when
the resulting data the subrequest obtains is ready and gives control
back to your backend apps to go on processing the data.

Sadly this mdoule does not have a wiki page yet, just the source
repository on GitHub:

    http://github.com/chaoslawful/drizzle-nginx-module

I've done some work in the README file there. I promise I'll work on
the wiki doc in another day ;)

Release tarballs can be downloaded from the page below

    http://github.com/chaoslawful/drizzle-nginx-module/downloads

I've also cc'd the nginx-devel mailing list in hope to get interested
developers to join this project's development. __YOU__ are always
welcomed!

Happily, we've already got a working framework for integrating
third-party libraries like libdrizzle and libpq into nginx's upstream
system as long as the libraries meet the following prerequisites:

1. It exposes the underlying socket fd.
2. It has a truly non-blocking API for its operations.

libdrizzle and libpq meet these conditions while oracle's OCI library
needs some hack for the requirement #1. So, you can expect ngx_oracle
and even ngx_pgsql to be announced here in the next few weeks or so,
and you'll see even more! If you'd join the fun, please don't hesitate
to drop us a line :)

Happy new year!

Enjoy!
-agentzh

P.S. I was hoping to release v0.0.1 by this Christmas, but missed the
deadline because I had spent too much time on fixing weird bugs in the
ngx_chunkin module :P

The year 2009 is my first year in the nginx community. Thank you all
for creating such a live community and thanks Igor for creating nginx
in the first place ;) I'm going to do more nginx C hacking in the next
year! :D