proxy_pass Not Working on Port 80

[figshta]

Thank you Francis!

I realize that some of these are probably rhetorical questions, but in the
interest of learning, I will try to answer them anyway.

<What request do you make of nginx-frontend?

I am mostly working with http/https 'get' requests for now.

<What request do you want nginx to make of the backend/upstream?

I want all requests for specific domains to pass to the backend (Server2)
(The idea is that Server2 will eventually replace Server1 as domains are
eventually moved over to it.)

<What request does nginx actually make of the backend?
The backend (server2) is also an nginx server.
I have seen the access logs and error logs for the backend (Server2), but
since I'm new to this, I'm slow to understand it all.

<The logs, or tcpdump, should show you exactly what is happening.
I will keep looking at the logs and study tcpdump, thank you.

<Certbot requires port 80 on the frontend.
<You get to decide for yourself what happens on the backend - certbot should
not know or care.

Right, and perhaps my scheme is erroneous.
I am trying to keep certificates on both servers.
Originally, I was trying to keep the certificates for domains on the backend
(Server2) on that machine, but I couldn't proxy_pass encrypted traffic
easily.

Here is that story:
https://community.letsencrypt.org/t/nginx-proxied-server-running-certbot-wrong-certificate/132635/2

In short, I ran Cerbot twice, once for each server (backend first), and in
order to run it on the backend I needed port 80.
It worked.
I'm trying to do that again the same way because I think it will be easier
to promote Server2 to the frontend later.
Maybe that is a misconception though, not sure.

<That is the 404 return that you get, because your frontend nginx did not
send the Host: header that you want. 
<(Instead, it sent the Host:header that you configured it to send.)

I commented out 'return 404; # managed by Certbot' and that did the trick.
Now I can use port 80. Thank you!
That said, I don't really understand where I configured the Host: header or
how to do it correctly.

>There is, in this case, an implicit "listen 80 default;" in this server{}.
So...

>> server {
>> listen 80;
>> server_name threedaystubble.com www.threedaystubble.com;

>....this server{} will only be used if you include a Host: header of one of
those two strings.
>Add some logging; or (temporarily)
>return 200 "this is the backend you want: $request_uri\n";
>to see that it is (or is not) being used.

It is clear that you have given me the guidance I need to try figure it
out.
I will play with it and try to learn it.
Thank you!

Posted at Nginx Forum: https://forum.nginx.org/read.php?2,289348,289371#msg-289371