<html xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:Helvetica;
panose-1:0 0 0 0 0 0 0 0 0 0;}
@font-face
{font-family:"Angsana New";
panose-1:2 2 6 3 5 4 5 2 3 4;}
@font-face
{font-family:"Cordia New";
panose-1:2 11 3 4 2 2 2 2 2 4;}
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:DengXian;
panose-1:2 1 6 0 3 1 1 1 1 1;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:"\@DengXian";
panose-1:2 1 6 0 3 1 1 1 1 1;}
@font-face
{font-family:Consolas;
panose-1:2 11 6 9 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
pre
{mso-style-priority:99;
mso-style-link:"HTML Preformatted Char";
margin:0cm;
margin-bottom:.0001pt;
font-size:10.0pt;
font-family:"Courier New";}
p.msonormal0, li.msonormal0, div.msonormal0
{mso-style-name:msonormal;
mso-margin-top-alt:auto;
margin-right:0cm;
mso-margin-bottom-alt:auto;
margin-left:0cm;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
span.HTMLPreformattedChar
{mso-style-name:"HTML Preformatted Char";
mso-style-priority:99;
mso-style-link:"HTML Preformatted";
font-family:"Consolas",serif;}
span.EmailStyle20
{mso-style-type:personal-reply;
font-family:"Calibri",sans-serif;
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:612.0pt 792.0pt;
margin:3.0cm 2.0cm 3.0cm 2.0cm;}
div.WordSection1
{page:WordSection1;}
--></style>
</head>
<body lang="DA" link="blue" vlink="purple">
<div class="WordSection1">
<p class="MsoNormal"><span lang="EN-GB">In haproxy, you simply specify a path where you have all your certificates.<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB">frontend https_frontend<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB"> bind *:443 ssl crt /etc/haproxy/certs/default-cert.pem crt /etc/haproxy/certs alpn h2,http/1.1<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB">This way, haproxy will read all certs, and when stuff comes in, it uses the host header to determine which certificate it should serve.<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB">There was a thread on the haproxy mailing list not long ago, with managing more than 100k certificates per haproxy instance, and they’re working on further optimizations with those kinds of deployments (if it’s not already
done.. haven’t checked to be honest).<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB">Best Regards,<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB"><o:p> </o:p></span></p>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0cm 0cm 0cm">
<p class="MsoNormal" style="margin-left:36.0pt"><b><span style="font-size:12.0pt;color:black">From:
</span></b><span style="font-size:12.0pt;color:black">nginx <nginx-bounces@nginx.org> on behalf of Richard Paul <Richard@primarysite.net><br>
<b>Reply-To: </b>"nginx@nginx.org" <nginx@nginx.org><br>
<b>Date: </b>Tuesday, 12 February 2019 at 10.04<br>
<b>To: </b>"nginx@nginx.org" <nginx@nginx.org><br>
<b>Subject: </b>Re: I'm about to embark on creating 12000 vhosts<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:36.0pt"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:36.0pt"><a name="_MailOriginalBody">Hi Jeff<o:p></o:p></a></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:36.0pt"><span style="mso-bookmark:_MailOriginalBody"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:36.0pt"><span style="mso-bookmark:_MailOriginalBody">That's interesting, how do you manage the progamming to load the right certificate for the right domain coming in as the server name? We need to load the right certificate
for the incoming domain and the 12000 figure is the number of unique vanity domains without the www. subdomains.<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:36.0pt"><span style="mso-bookmark:_MailOriginalBody"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:36.0pt"><span style="mso-bookmark:_MailOriginalBody">We're planning to follow the same path as you though, we're essentially putting these Nginx TLS terminators (fronted by GCP load balancers) in front of our existing
Varnish caching and Nginx backend infrastructure which currently only listen on port 80.<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:36.0pt"><span style="mso-bookmark:_MailOriginalBody"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:36.0pt"><span style="mso-bookmark:_MailOriginalBody">I couldn't work out what the limits are at LE as it's not clear with regards to adding new unique domains limits. I'm going to have to ask in the forums at some point
so that I can work out what our daily batches are going to be.<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:36.0pt"><span style="mso-bookmark:_MailOriginalBody"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:36.0pt"><span style="mso-bookmark:_MailOriginalBody">Kind regards,<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:36.0pt"><span style="mso-bookmark:_MailOriginalBody">Richard<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:36.0pt"><span style="mso-bookmark:_MailOriginalBody"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:36.0pt"><span style="mso-bookmark:_MailOriginalBody">On Mon, 2019-02-11 at 14:33 -0500, Jeff Dyke wrote:<o:p></o:p></span></p>
</div>
<blockquote style="border:none;border-left:solid #729FCF 1.5pt;padding:0cm 0cm 0cm 6.0pt;margin-left:4.8pt;margin-right:0cm">
<div>
<div>
<p class="MsoNormal" style="margin-left:36.0pt"><span style="mso-bookmark:_MailOriginalBody">I use haproxy in a similar way as stated by Rainer, rather than having hundreds and hundreds of config files (yes there are other ways), i have 1 for haproxy and 2(on
multiple machines defined in HAProxy). One for my main domain that listens to an "real" server_name and another that listens to `server_name _;` All of the nginx servers simply listen on 80 and 81 to handle non H2 clients and the application does the correct
thing with the domain. Which is where YMMV as all applications differ. <o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:36.0pt"><span style="mso-bookmark:_MailOriginalBody"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:36.0pt"><span style="mso-bookmark:_MailOriginalBody">I found this much simpler and easier to maintain over time. I got around the LE limits by a staggered migration, so i was only requesting what was in the limit each
day, then have a custom script that calls LE (which is also on the same machine as HAProxy) when certs are about 10 days out, so the staggering stays within the limits. When i was using custom configuration, i was build them via python using a yaml file and
nginx would effectively be a jinja2 template. But even that became onerous. When going down the nginx path ensure you pay attention to the variables that control domain hash sizes. </span><a href="http://nginx.org/en/docs/hash.html"><span style="mso-bookmark:_MailOriginalBody">http://nginx.org/en/docs/hash.html</span><span style="mso-bookmark:_MailOriginalBody"></span></a><span style="mso-bookmark:_MailOriginalBody">
<o:p></o:p></span></p>
<div>
<p class="MsoNormal" style="margin-left:36.0pt"><span style="mso-bookmark:_MailOriginalBody"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:36.0pt"><span style="mso-bookmark:_MailOriginalBody">HTH, good luck!<br>
Jeff<o:p></o:p></span></p>
</div>
</div>
</div>
<p class="MsoNormal" style="margin-left:36.0pt"><span style="mso-bookmark:_MailOriginalBody"><o:p> </o:p></span></p>
<div>
<div>
<p class="MsoNormal" style="margin-left:36.0pt"><span style="mso-bookmark:_MailOriginalBody">On Mon, Feb 11, 2019 at 1:58 PM Rainer Duffner <</span><a href="mailto:rainer@ultra-secure.de"><span style="mso-bookmark:_MailOriginalBody">rainer@ultra-secure.de</span><span style="mso-bookmark:_MailOriginalBody"></span></a><span style="mso-bookmark:_MailOriginalBody">>
wrote:<o:p></o:p></span></p>
</div>
<blockquote style="border:none;border-left:solid #729FCF 1.5pt;padding:0cm 0cm 0cm 6.0pt;margin-left:4.8pt;margin-right:0cm">
<div>
<p class="MsoNormal" style="margin-left:36.0pt"><span style="mso-bookmark:_MailOriginalBody"><o:p> </o:p></span></p>
<div>
<p class="MsoNormal" style="margin-left:36.0pt"><span style="mso-bookmark:_MailOriginalBody"><br>
<br>
<o:p></o:p></span></p>
<blockquote style="border:none;border-left:solid #729FCF 1.5pt;padding:0cm 0cm 0cm 6.0pt;margin-left:4.8pt;margin-right:0cm">
<div>
<p class="MsoNormal" style="margin-left:36.0pt"><span style="mso-bookmark:_MailOriginalBody">Am 11.02.2019 um 16:16 schrieb rick_pri <</span><a href="mailto:nginx-forum@forum.nginx.org" target="_blank"><span style="mso-bookmark:_MailOriginalBody">nginx-forum@forum.nginx.org</span><span style="mso-bookmark:_MailOriginalBody"></span></a><span style="mso-bookmark:_MailOriginalBody">>:<o:p></o:p></span></p>
</div>
<p class="MsoNormal" style="margin-left:36.0pt"><span style="mso-bookmark:_MailOriginalBody"><o:p> </o:p></span></p>
<div>
<p class="MsoNormal" style="margin-left:36.0pt"><span style="mso-bookmark:_MailOriginalBody"><span style="font-size:9.0pt;font-family:Helvetica">However, our customers, with about 12000 domain names at present have</span></span><span style="mso-bookmark:_MailOriginalBody"><o:p></o:p></span></p>
</div>
</blockquote>
</div>
<p class="MsoNormal" style="margin-left:36.0pt"><span style="mso-bookmark:_MailOriginalBody"><o:p> </o:p></span></p>
<div>
<p class="MsoNormal" style="margin-left:36.0pt"><span style="mso-bookmark:_MailOriginalBody"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:36.0pt"><span style="mso-bookmark:_MailOriginalBody">Let’s Encrypt rate limits will likely make these very difficult to obtain and also to renew.<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:36.0pt"><span style="mso-bookmark:_MailOriginalBody"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:36.0pt"><span style="mso-bookmark:_MailOriginalBody">If you own the DNS, maybe using Wildcard DNS entries is more practical.<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:36.0pt"><span style="mso-bookmark:_MailOriginalBody"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:36.0pt"><span style="mso-bookmark:_MailOriginalBody">Then, HAProxy allows to just drop all the certificates in a directory and let itself figure out the domain-names it has to answer.<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:36.0pt"><span style="mso-bookmark:_MailOriginalBody">At least, that’s what my co-worker told me.<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:36.0pt"><span style="mso-bookmark:_MailOriginalBody"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:36.0pt"><span style="mso-bookmark:_MailOriginalBody">Also, there’s the fabio LB with similar goal-posts.<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:36.0pt"><span style="mso-bookmark:_MailOriginalBody"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:36.0pt"><span style="mso-bookmark:_MailOriginalBody"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:36.0pt"><span style="mso-bookmark:_MailOriginalBody"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:36.0pt"><span style="mso-bookmark:_MailOriginalBody"><o:p> </o:p></span></p>
</div>
</div>
<p class="MsoNormal" style="margin-left:36.0pt"><span style="mso-bookmark:_MailOriginalBody">_______________________________________________<br>
nginx mailing list<br>
</span><a href="mailto:nginx@nginx.org" target="_blank"><span style="mso-bookmark:_MailOriginalBody">nginx@nginx.org</span><span style="mso-bookmark:_MailOriginalBody"></span></a><span style="mso-bookmark:_MailOriginalBody"><br>
</span><a href="http://mailman.nginx.org/mailman/listinfo/nginx" target="_blank"><span style="mso-bookmark:_MailOriginalBody">http://mailman.nginx.org/mailman/listinfo/nginx</span><span style="mso-bookmark:_MailOriginalBody"></span></a><span style="mso-bookmark:_MailOriginalBody"><o:p></o:p></span></p>
</blockquote>
</div>
<pre style="margin-left:36.0pt"><span style="mso-bookmark:_MailOriginalBody">_______________________________________________<o:p></o:p></span></pre>
<pre style="margin-left:36.0pt"><span style="mso-bookmark:_MailOriginalBody">nginx mailing list<o:p></o:p></span></pre>
<p class="MsoNormal" style="margin-left:36.0pt"><span style="mso-bookmark:_MailOriginalBody"></span><a href="mailto:nginx@nginx.org"><span style="mso-bookmark:_MailOriginalBody"><o:p></o:p></span></a></p>
<pre style="margin-left:36.0pt"><span style="mso-bookmark:_MailOriginalBody"></span><a href="mailto:nginx@nginx.org"><span style="mso-bookmark:_MailOriginalBody">nginx@nginx.org</span><span style="mso-bookmark:_MailOriginalBody"><span style="color:windowtext;text-decoration:none"><o:p></o:p></span></span></a></pre>
<p class="MsoNormal" style="margin-left:36.0pt"><span style="mso-bookmark:_MailOriginalBody"><o:p> </o:p></span></p>
<pre style="margin-left:36.0pt"><span style="mso-bookmark:_MailOriginalBody"><o:p> </o:p></span></pre>
<p class="MsoNormal" style="margin-left:36.0pt"><span style="mso-bookmark:_MailOriginalBody"></span><a href="http://mailman.nginx.org/mailman/listinfo/nginx"><span style="mso-bookmark:_MailOriginalBody"><o:p></o:p></span></a></p>
<pre style="margin-left:36.0pt"><span style="mso-bookmark:_MailOriginalBody"></span><a href="http://mailman.nginx.org/mailman/listinfo/nginx"><span style="mso-bookmark:_MailOriginalBody">http://mailman.nginx.org/mailman/listinfo/nginx</span><span style="mso-bookmark:_MailOriginalBody"><span style="color:windowtext;text-decoration:none"><o:p></o:p></span></span></a></pre>
<p class="MsoNormal" style="margin-left:36.0pt"><span style="mso-bookmark:_MailOriginalBody"><o:p> </o:p></span></p>
</blockquote>
</div>
</body>
</html>