Handling Large Number of Dynamic Hosts in Custom Module Subrequests

[Cloud Gate]

Hello nginx developers,

Background: Our custom C++ module for nginx 1.11.4 uses subrequests to perform REST calls to our server during the course of processing incoming proxy requests. The subrequest / server URL is dynamic and varies by request e.g. <host1>.domain.foo, <host2>.domain.foo, etc. There may be thousands of variations of host. As best we can tell, subrequests require explicit configuration in nginx.conf, where the host / URL is hard-coded. We'd rather not add thousands of config sections, which bloat the configuration and require constant maintenance.

Is there a way to configure nginx to proxy a large number of hosts, where the specific host is determined dynamically at runtime?

We are basing our development on these sample resources: http://nginx.org/en/docs/http/ngx_http_auth_request_module.html or https://github.com/openresty/echo-nginx-module. Configuration for those modules hard-codes a small number of known locations for upstream servers, e.g. auth request module (additional authentication servers would require additional location blocks):

location /private/ {
    auth_request /auth;
    ...
}

location = /auth {
    proxy_pass http://authserver
    ...

The first alternative approach we've come up with is to set an HTTP header in the subrequest containing the target URL (x_our_proxy_url=http://host1.domain.foo/...), target the subrequest at a static /ourproxy location, and in nginx.conf pass the URL from the header to proxy_pass:

location /ourproxy {
   proxy_pass $http_x_our_proxy_url;
}

While this works, it seems somewhat ad-hoc, and we're wondering if there are better options for a subrequest to target arbitrary hosts. Note that use of third-party modules such as OpenResty is not really an option at this time, as we are strictly a C++ module.

Thanks


--

CG Development
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx-devel/attachments/20160926/d5384879/attachment.html>