RFC: Feasibility of a "dynamic module loader" built in to nginx?

[agentzh]

On Thu, Aug 26, 2010 at 7:14 PM, Michael Shadle <mike503 at gmail.com> wrote:
> I believe that for package management, as well as module adoption, it
> would be a definite move in the right direction to be able to load
> modules up on demand instead of having to custom compile nginx with
> the modules you want.
>

I totally agree here :) We wrote 15 nginx C modules in the last year
(just one or two are opensourced yet) and we found rpm packaging
especially hard in a corporation environment. For every different kind
of business, we have to provide a variation of nginx rpm package just
because they use a different set of modules. (Yeah, we could compile
all those modules into a giant nginx but it hurts performance.)

> That allows for module development to be done completely independently
> of nginx, modules to be added/removed, etc, without needing to
> recompile nginx every time.
>

I personally spent a lot of CPU cycles on recompiling nginx on my
laptop during nginx development. Fortunately I have a fast machine,
but still it's far from ideal :)

> I find that the majority of people would most likely say this is a
> good idea, a few may think it is not necessary, but everyone can
> benefit, and for those who don't, they can still maintain their nginx
> however they want, by recompiling each time.
>

Having more choices is always a Good Thing (TM) ;)

> I don't expect much overhead being added, as the modules would still
> only be loaded when the master process (or perhaps child processes?)
> startup.
>

chaoslawful and I have talked about the implementation details several
times and dynamic loader support for the nginx core is one of our TODO
items. And yeah, it also means that it's very likely that we'll
maintain our own patches to the core or even our own fork of nginx
(given the historic records in the nginx world) :)

> Ideally it would be as simple as
>
> load /usr/lib/nginx/mod_drizzle.so;
>

Cool, and by that way, we can re-arrange the order of nginx modules
just in the config file! Like so:

   load /path/to/ngx_rds_json.so
   load /path/to/ngx_srcache.so
   load /path/to/ngx_lua.so

or make it even more elegant and more portable for other OS's:

   use ngx_rds_json;
   use ngx_srcache;
   use ngx_lua;

And we can control the search paths for the module .so files from the
environment. Quite perl-ish, isn't it? ;)

> Or something in the configuration. This would allow plugins to
> flourish without complex build instructions or having to touch the
> nginx core; module developers could be informed to make version checks
> and issue a notice when trying to startup, something along the lines
> of "mod_drizzle is not compatible with this version of nginx (=>
> 0.8.43 needed)" or something of that nature.
>

*nod*

There we go!

Cheers,
-agentzh