Nginx headers to be set at the time of request processing

[CeeGeeDev]

Francis Daly Wrote:
-------------------------------------------------------
> On Mon, Nov 28, 2016 at 06:56:29PM -0500, CeeGeeDev wrote:
> 
> Hi there,
> 
> Thanks for expanding on what you are doing.
> 
> I confess that I am still not sure what it is; but that's ok -- I
> don't need to understand.

Some of our team members might say the same, so appreciate your patience. :)
So, other web proxies/servers e.g. Apache and others, when you indicate a
"set header"-style directive in configuration, apparently (I am told) any
custom modules/filters you may have automatically receive as input a merged
set of request headers: the "real" ones and the "fake" ones set in the
configuration files. So there is often in fact no distinguishing between the
two.

This allows (often complex) configuration rules to drive request header
values (the ones that need to be modified for whatever business logic
reasons), both in terms of module business logic and altering upstream view
of the request headers. So it seems like in nginx we're only getting
one-half of the equation, and there seemed no direct way (without incurring
the overhead of a server redirect or something like that) to tell the module
about a change to a request header in the config file.

This is the gap we're trying to solve: to unify the view of the headers from
the perspective of both the upstream and the custom request processing
module. If they see different views/values of the headers, the business
logic will make the wrong decisions.

> > So for the benefit of the community: our plan is to implement a
> > custom configuration directive in our http module to allow us to inform
> > ourselves about various header overrides made in the nginx
> > configuration file that should override various request headers in the
> > actual request data structures during request processing in our http
> > module code (in our business logic only... will have no effect on
> > downstream request header values). There seems to be no built-in
> > alternative for nginx custom http module developers (apologies if this
> > question is better suited to the development list), at least none that
> > we can find documented
> anywhere.
> 
>   location /test/ {
>     proxy_set_header a a;
>     fastcgi_param b b;
>     my_directive_a c c;
>     my_directive_b d d;
>   }
> 
> For a request that is handled in that location, three of those
> directives could send some "extra" information to the upstream,
> if a suitable "*_pass" directive were active. No "*_pass" directive
> is active, so those three directives are effectively unused for this
> request.
> 
> What do you expect your module to report for a request handled in this
> location?

Right, so in other words, our dev and actually the provisioning team who
discovered this (all more familiar with other web servers and new to nginx)
were surprised when the module wasn't getting the modified headers from the
stock directives like proxy_set_header. We just kind of assumed that if the
header was "altered", we would see it (granted in nginx, there are multiple
directives affecting headers, so I don't mean to imply we were/are really
sure what would/should happen if multiple directives affected the same
header, etc). We're more familiar with other web servers where this is
usually only a single "change this header" directive, and it clearly makes
sense to alter that header value "globally".

The nice thing of this custom directive approach is that we have
flexibility: a) both the upstream and the module are told the modified
header, b) the upstream is not told but the module is or c) the upstream is
told but the module is not. But it would have been sufficient to only
support (a), and always set both the standard proxy_set_header /
fastcgi_param etc and the my_directive_x to the same value, and then
everyone will think the request header has been modified to this new value.
That's the most common case anyway.

> That may make it clearer what you are trying to achieve.
> 
> (If it does not, feel free to ignore this mail.)
> 
> Thanks,
> 
> 	f
> -- 
> Francis Daly        francis at daoine.org

Further, we did discover "more_set_input_headers" from ngx_headers_more (a
public custom module) that does something close to what we need, but our
legal staff always objects to using various modules for all their fun legal
reasons. And we decided we liked the flexibility of the custom directive in
the end anyway.

So something to consider perhaps as a feature enhancement for nginx? Or at
least a documented approach. Appreciate your consideration anyway and
certainly if there is a feature somewhere we aren't aware of that might
accomplish the same thing, we would be interested.

Also, we're the same team interested in a published versioned custom module
true pluggable dynamic-linked module (i.e. no re-compile of nginx server
code required) versioned nginx SDK interface, which I understand has been
discussed but is not on immediate roadmap. So I imagine in a true module
SDK, perhaps this type of functionality would make more sense? Anyway, just
a thought.

Thank you very much

Posted at Nginx Forum: https://forum.nginx.org/read.php?2,271147,271273#msg-271273