Changeset: Dynamic Virtual Server Selection Based on Host's Cnamed domain

[Maxim Dounin]

Hello!

On Mon, Jun 06, 2022 at 12:24:35PM +0530, Devarajan D via nginx-devel wrote:

> Dear Members,
> 
> # HG changeset patch
> # User devarajan.dk <mailto:devarajan.dk at zohocorp.com>
> # Date 1654428592 -19800
> #      Sun Jun 05 16:59:52 2022 +0530
> # Node ID 8d73e94b379091440ad8c836e2a052fb5edda160
> # Parent  e0cfab501dd11fdd23ad492419692269d3a01fc7
> Dynamic Virtual Server Selection Based on Host's Cnamed domain
> 
> Use Case:
> Preventing nginx.conf reload on new customer domain addition.
> 
> Many B2B Cloud Services provide option for rebranding customer domains through CNAME.
> eg. Customers - mail.schroeppel.me, mail.vey.de cnamed to ghs.googlehosted.com
> The hosting Service can be having multiple such services reverse proxied through
> same nginx instance. (eg. Virtual Servers - mail.ghost.com, docs.ghost.com, drive.ghost.com, etc.
> all virtual services listening on the same IP and port; consider 100s of such virtual services)
> 
> While deployment, the cloud service has to add the customer domain name in 'server_name' directive
> of corresponding virtual server for the request to reach the right virtual server block.
> Also, as the customer has given control to the hosting cloud service, the service will also
> purchase certificate for the customer domain and offload through nginx.
> 
> As and when the new rebranding customers starts to subscribe for the cloud service, the cloud service
> has to add the certificate name & key in ssl_certificate & ssl_certificate_key directives respectively
> and customer's domain name in server_name directive of corresponding virtual server.
> So the nginx.conf has to be reloaded every time a new customer starts using the service.
> 
> As the frequency of new customers joining the service increases, the hosting service
> may suffer from frequent nginx reloads and may not scale well.
> (consider 100s of customers joining the service every hour)
> Thanks to the Dynamic Certificate Loading feature in nginx that uses $ssl_server_name variable
> to dynamically select the certificate on runtime and no need to reload conf after deploying certificate.
> But still there is a need to reload the conf due to the addition of customer's domain name in server_name directive.
> 
> To prevent this reload, we can dynamically choose Virtual server based on the cname of the customer domains.
> Only the cnamed domain ie. mail.ghost.com will be written in the server_name directive of the virtual server.
> All customer domains cnamed to mail.ghost.com will be routed to right virtual server based on the cname.
> This cname check can be turned on/off by a http level directive - 'resolve_cname' (off by default)
> 
> Note:
> 1. Similar to Dynamic Certificate Loading, the pros of dynamic Virtual Server selection come 
>    at the cost of increase in latency during request processing time due to DNS lookup.
> 2. The cloud service can choose to deploy their customer domains in hybrid mode too. 
>    (ie. old customers domains can be explicitly configured in server_name and 
>    customer domains who newly subscribed for the cloud service in between two nginx.conf reloads
>    can be deployed in nginx dynamically)

[...]

Thank you for the patch.

It doesn't look like a good idea to me though, for multiple 
reasons.  In no particular order:

- It is generally a bad idea to do DNS lookups on per-request 
  basis.  Further, even if you do DNS lookups for some reason, it 
  is generally a bad idea to do user-controlled DNS lookups.  It 
  can be easily used as an attack vector.

- You should never ever use gethostbyname() at runtime in nginx 
  (or any other event-based servers): it is blocking and a 
  long-running DNS request will stop all the request in the worker 
  processes.  Combined with user-controlled name, this is an 
  immediate DoS on the server.  To make it possible to do DNS 
  lookups when needed, nginx provides the resolver infrastructure 
  (http://nginx.org/r/resolver).

Further, it looks like what you are trying to do can be easily 
achieved with at least the following options:

- The most simple solution to all issues involving configuration 
  reloads is to do reloads periodically, applying all the changes 
  accumulated at once.  For example, reloading configurations once 
  in several minutes might be an option - especially considering 
  that typical DNS update time is more than 1 hour.

- In this particular case, it is trivial to configure nginx to 
  process all the names by using distinct IP-based virtual servers 
  (if needed, with further additional restrictions applied via nginx 
  configuration, such as checking if appropriate files exists, 
  and/or at the backend level).  If using just one IP address is 
  absolutely a must, further distinction between services can be 
  implemented within a single server{} block.

Hope this helps.

-- 
Maxim Dounin
http://mdounin.ru/