proxy_pass Not Working on Port 80

[Francis Daly]

On Thu, Sep 10, 2020 at 01:31:48AM -0400, figshta wrote:

Hi there,

> I realize that some of these are probably rhetorical questions, but in the
> interest of learning, I will try to answer them anyway.

No, not rhetorical.

Also, not general.

I mean: when you reply with "does not work", what was the one specific
test case that you ran?

Something like: I run the command

  curl -v http://www.example.com/one/two.html

and I expect the response "http 301" and a redirect to /one/three.html;
or I expect the response "http 200" and the content of the file
/usr/local/nginx/html/one/two.html from the machine server2; or whatever
exact specific response that you want for this one specific request

> <What request do you make of nginx-frontend?
> 
> I am mostly working with http/https 'get' requests for now.

So, let's say "http://www.example.com/one/two.html".

> <What request do you want nginx to make of the backend/upstream?
> 
> I want all requests for specific domains to pass to the backend (Server2)
> (The idea is that Server2 will eventually replace Server1 as domains are
> eventually moved over to it.)

And let's say "/one/two.html", talking to backend "www.example.com".

> <What request does nginx actually make of the backend?
> The backend (server2) is also an nginx server.
> I have seen the access logs and error logs for the backend (Server2), but
> since I'm new to this, I'm slow to understand it all.

If you have more than one backend server{} block, and you write all
the logs for all server{} blocks into one file, then you possibly will
not easily know which one server{} block nginx used to process this
one request.

If you make it possible to see what is happening, it will be easier to
see what is happening.

> <The logs, or tcpdump, should show you exactly what is happening.
> I will keep looking at the logs and study tcpdump, thank you.

tcpdump will be useful for http content; it can be slightly useful
for https content, to show that you are or are not using SNI (which is
relevant when you have more than one "virtual host" on the same IP:port).

> <Certbot requires port 80 on the frontend.
> <You get to decide for yourself what happens on the backend - certbot should
> not know or care.
> 
> Right, and perhaps my scheme is erroneous.
> I am trying to keep certificates on both servers.
> Originally, I was trying to keep the certificates for domains on the backend
> (Server2) on that machine, but I couldn't proxy_pass encrypted traffic
> easily.

The certbot side seems... complicated. If that wants discussing here,
it probably should be in a dedicated thread. It is not directly related
to the subject of this thread.

(In very short: you need to decide how exactly you want inbound traffic
to your public IP address to be handled. After you have a clear design
for how you want each request to be handled, you will be able to see
how and whether it is possible.)

> <That is the 404 return that you get, because your frontend nginx did not
> send the Host: header that you want. 
> <(Instead, it sent the Host:header that you configured it to send.)
> 
> I commented out 'return 404; # managed by Certbot' and that did the trick.
> Now I can use port 80. Thank you!
> That said, I don't really understand where I configured the Host: header or
> how to do it correctly.

There is (potentially) lots in the configuration that you want. So it
is worth taking one step at a time.

There is also lots of documentation available on the nginx.org web
site. For example, every directive (e.g "location") is documented at
(e.g.) http://nginx.org/r/location.

For nginx http -- a request comes in, and is handled in exactly one
server{} block (based on the configuration, and the incoming IP:port and
hostname within the request). Then (after rewrite-module directives)
the request is handled in exactly one location{} block (based on the
configuration and the request). Then either a subrequest is created,
and a new rewrite-module/location-selection happens; or the request is
just processed.

On your frontend, this means that your /one/two.html
request to threedaystubble.com is proxy_pass'ed to
http://192.168.3.5:80/one/two.html.

On your backend, the http://192.168.3.5:80/one/two.html is handled
in the "houseofavi.com" server{} block, where it will return
404; or, now that you have removed that, return the content of
/var/www/houseofavi.com/one/two.html


If you want the request to be handled in the server2 "threedaystubble.com"
server block, you'll probably want the proxy_set_header that was mentioned
previously.


If you use "curl" for testing, rather than a more full-featured web
browser, you will have a better chance of seeing what exactly is
happening, and you will avoid things like browser caching that can
interfere with useful testing.

Good luck with it,

	f
-- 
Francis Daly        francis at daoine.org