Installing dynamic modules when Nginx itself is installed via yum/dnf (Linux)

[Francis Daly]

On Tue, Jan 05, 2021 at 07:27:20PM -0500, Phoenix Kiula wrote:

Hi there,

there seem to be a few different things clashing here, which combine to
make it very awkward for you to build the dynamic modules that you want.

I will try to explain the background as I see it, and hopefully point
you at things you can do to get to a working system.

Firstly - you only want to use software installed from a package. That
is a sensible choice; it makes it much easier to re-create a server when
the software comes from a known external source.

Secondly - you only want to use packages provided by a specific third
party -- your distribution. That's also sensible, but is a bit more
restrictive, because...

Thirdly - you want to use some software that your distribution does
not make available. Here's where the conflict arises -- between your
requirement#2 and requirement#3.

They can't both happen, so you must choose which to break.

You seem to be happy to self-compile some code that is not available
from the distribution; but not to replace code that is available from
the distribution. That's also a sensible choice.

On the nginx side...

Once upon a time, nginx was configured to be built with whatever modules
were wanted, and was compiled statically with that set. The only way to
add a module was to (re)compile the whole thing.

The reliable way to get "the same" nginx but with an extra module, was
to run "nginx -V" to find how this was configured, and then run the same
command with an extra "--add-module=".

Then in early 2016, "dynamic modules" were added to nginx -- some modules
could be compiled as "shared object" files, and could be loaded into nginx
at startup; so you could distribute one "main" binary, plus a handful
of individual modules, and the user could decide which to include and
which not to, in their nginx.conf.

It turns out that the modules do need to know some things about the
internal layout of the compiled nginx instance; and that layout can
(and does) change depending on how nginx was configured to be built. So
"dynamic modules" were good; but you could not reliably provide "a
module for this version of nginx". You had to provide "a module for
this version of nginx configured in this way", plus "a module for this
version of nginx configured in that way"; and the number of possible
ways was big enough that it was a headache to try to provide multiple
binary versions of "the same" module.

(A big thing was ssl -- the layout of some internal things were very
different if ssl was or was not enabled. But other features could also
change that layout.)

The reliable way to build an extra module for this nginx, was to run
"nginx -V" to find how this was configured, and then run the same command
with an extra "--add-dynamic-module=".

So, in late 2016, the "--with-compat" configure-time flag was added
as an option; this would more-or-less fix the internal layout that
modules needed to know about, to no longer vary based on the rest of
the configure-time choices.

With this, the nginx-internal layout would remain much more consistent
-- basically, a "compat version number" could be changed whenever a
known-breaking change would be made, and most changes would not affect
that "compat version number". You could now distribute one binary module
which could reasonably be expected to work with any configuration of a
specific version of nginx; and there was also a reasonable chance that
it would work with newer and older versions of nginx too, so long as this
"compat" part was not changed.

It does need that both nginx and the module are built with the
same expectations of the internal layout, but that is relatively
straightforward to ensure -- just use "--with-compat" at configure-time
before building, and most of the work is done.


In that world, the way to build a module is the few-step recipe that
was in the document referred to earlier:

https://www.nginx.com/blog/compiling-dynamic-modules-nginx-plus/

 * get the nginx source code
 * get your module source code
 * configure nginx "--with-compat" and "--add-dynamic-module=" your module
 * "make modules" to build the module
 * copy the module .so file to wherever you want your pre-existing
 nginx to be able to find it

All of the extra configure-options of the running nginx binary can be
ignored, so long as it includes --with-compat.

==

In your specific case, I am not sure from your mails if your distribution
included "--with-compat" or not. If it did, you should be able to just
follow that recipe.

If it did not, you will probably need to follow the recipe from 2016.

You may be able to omit some of the "--with-" module parts, if you
know that they do not affect the internal layout that affects dynamic
modules. (How would you know that? Probably studying the source code
would give hints; I suspect that not many people would be especially
interested in investigating, since there has been a configure-time option
to make it unnecessary for the past four years.)


If your running nginx shows "--with-compat" in its "nginx -V" output,
and if you follow exactly the recipe given, things have a better chance
of working.

If you get errors or unexpected output, then copy-pasting the commands
that you ran and the output that you got, may help someone point at what
could be done differently.

If your running nginx does not show "--with-compat" in its "nginx -V"
output, and you are using a well-known distribution, then maybe someone
has already built some modules for your distribution; if you are happy to
trust whatever external package repository you find instead of building
the module yourself, that might be a way to proceed. You may be able,
for example, to extract the files that you care about, in order to
avoid changing the server repository configuration. You would take on
the responsibility for updating things in that case, the same as if you
had built it yourself.

> Welcome any pointers to maintainable ways of installing so that a "dnf
> update nginx" will not break the modules.

That would require whoever compiled your nginx, and whoever compiles
your extra modules, choosing to update in lockstep. That is unlikely,
unless they are the same organization.

Good luck with it,

	f
-- 
Francis Daly        francis at daoine.org