shell environment variables in "include"-directive not working

[Brian H.]

Just wanted to respond that I had the similar need, and hopefully a 
better solution. We too have dev, staging, prod, and the configurations 
are similar, but not identical.
We have hundreds of servers, and the variances in files means we need to 
have a much longer runway for new sysadm recruits to get up to speed, 
and increases the opportunity for errors caused by divergence - 
especially in active/active or active/passive configurations in 
production.

Next - the link to sed which is a HORRIBLE idea to my OCD brain.  sed is 
a line parser, it fundamentally doesn't understand the hierarchy of an 
nginx conf file(s). The opportunity for something to go horribly 
horribly wrong and be very difficult to troubleshoot (especially with 
high degrees of automation/scripting in other areas of our platform) 
scares the crap out of me. Using sed to parse old style unix files where 
everything is on a single line and all lines are identical = great, 
using it to parse a nested configuration file where each line is 
different is a horrid idea (even with excellent naming conventions).
I am actually not aware of any valid NGINX hierarchy parser, and since 
it uses a very non standard, highly complex syntax I think it would be 
reasonably challenging to write one which could read, modify, write back 
the same file with comments, etc.  (I bet a lot of people do)

AND -- something as a simple __HOSTNAME__ or #IFHOSTNAME would be 
*amazingly* useful for a variety of situations.
I also wanted to share to others who may also have the same issues why 
user defined environment variables are a horrible idea too and why the 
devs are so resistant to them.   When trying to hot-swap executables 
since the environment won't carry across pids -- this is a cool feature 
nginx has, and using environment variables would break that.
So stop asking for environment variables -- instead we should be 
thinking about a few well known variables that can be interpolated at 
config parsing time would be useful for those of us with hundreds of 
servers to manage the 1-2% variances between each servers without 
needing an elaborate custom build script. one config file to bind them 
all and KISS.

**SO** I may write a plugin to do that in the future, but alas, no time. 
So I submit the slightly ghetto work around I devised.

First use host file trickery (this is IMHO pretty common for 
dev/prod/staging) to alias well known role based names "ex: api, www, 
etc"

Use symlinking with hostname to include the proper files based on 
hostname. Modify your init.d script so it links, or touches empty files 
when a corresponding file doesn't exist.

I'm not going to provide examples here because inevitably your 
environment will be different than mine, but I can tell you that nginx 
will load a empty (zero byte) include file with no issues.
Of course environment variables like hostname in a shell script is 
trivial.

So when you bake it all together - in the nginx.conf file:

include "some-role.conf"
include "lotsofcustom-roles/*.conf"
include "yetanother-role.conf"

** those includes point at symlinked or zero byte files.

In the /etc/init.d/nginx script do something like:

/bin/rm -f $NGINXROOT/conf/some-role.conf
/bin/rm -f $NGINXROOT/conf/lotsofcustom-roles
/bin/rm -f $NGINXROOT/conf/yetanother-role.conf
if [ -f "$NGINXROOT/conf/some-role-$HOSTNAME.conf" ] ; then
   ln -s "$NGINXROOT/conf/some-role-$HOSTNAME.conf" 
$NGINXROOT/conf/some-role.conf"
else
   touch $NGINXROOT/conf/some-role.conf
fi
if [ -d "$NGINXROOT/conf/lotsofcustom-roles-$HOSTNAME" ] ; then
   ln -sd "$NGINXROOT/conf/lotsofcustom-roles-$HOSTNAME" 
$NGINXROOT/lotsofcustom-roles"
else
   mkdir $NGINXROOT/conf/lotsofcustom-roles
   touch $NGINXROOT/conf/lotsofcustom-roles/nothing-to-see-here.conf
fi
if [ -f "$NGINXROOT/conf/yetanother-role-$HOSTNAME.conf" ] ; then
   ln -s "$NGINXROOT/conf/yetanother-role-$HOSTNAME.conf" 
$NGINXROOT/conf/some-role.conf"
else
   touch $NGINXROOT/conf/yetanother-role.conf
fi

Obviously this is highly dependent on exactly what you want to 
accomplish, but I think it strikes a much nicer balance than use a sed 
machete to hack through a config file using regular expressions it 
doesn't understand and accidentally clobbering something you didn't 
intend to.

Regards,

-Brian Horakh
Chief Technical Guy
anyCommerce





Maxim Dounin wrote in post #911875:
> Hello!
>
> On Thu, May 13, 2010 at 12:49:21AM +0200, Markus Grobelin wrote:
>
>> Hy everybody,
>> i'm doing my first steps with nginx/0.8.36 and trying to get *NIX
>> shell environment variables working inside the configuration files.
>> Sadly, it's seems they aren't working inside the
>> "include"-directive! :(
>
> nginx doesn't have syntax for expanding environment variables in
> configuration file.  Syntax $var used for runtime per-request
> variables (supported by some directives, support explicitly noted
> in directive descriptions).
>
> [...]
>
>> The [emerg] indicates, that the $INSTANCE environment variable isn't
>> expanded, whereas the "user" and "pid" directive doesn't raise an
>> exception??
>
> Because there is no syntax error in user "$USER" and config parser
> has nothing against it.  As you aren't running as root nginx
> just prints warning about being non-root and forgets about it.
> Under root you should see:
>
> [emerg]: getpwnam("$USER") failed in /path/to/nginx.conf:line
>
> (unless you actually have "$USER" in your /etc/passwd)
>
> Similar thing with pid.  It's syntactically correct and will only
> produce error when nginx will try to create pid file.  If you
> happen to cleanup other critical config errors you should see
> something like this on startup:
>
> [emerg]: open() "/nginx/$INSTANCE/run/nginx.pid" failed (2: No such file
> or directory)
>
> (again, unless you actually have "/nginx/$INSTANCE/run/" directory)
>
> Maxim Dounin

-- 
Posted via http://www.ruby-forum.com/.