Proxy_Protocol with local Proxy_Pass to local secondary nginx process

[Joseph Wonesh]

Hello,

I am trying to route requests such that those requiring websockets will
route to a long-lived nginx process, and all others will go to the general
reverse-proxy which handles all other traffic. These nginx processes exist
in our AWS cloud behind an ELB that has been configured to use Proxy
Protocol. Note that all of this works correctly with our current setup
which uses only one nginx process that is configured to use proxy_protocol.

The change to this setup is as follows:

The first nginx server handling all ingress uses proxy_protocol and
forwards requests to either the websocket or non-websocket nginx servers
locally:

server {
    listen 8080 proxy_protocol;
    real_ip_header proxy_protocol;
    charset utf-8;
    client_max_body_size 20M;

    #send to websocket process
    location /client {
      proxy_pass http://localhost:8084;
      proxy_set_header Host $host;
      proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
      proxy_set_header X-Real-IP $proxy_protocol_addr;
      proxy_set_header X-NginX-Proxy true;
      proxy_set_header X-Proxy-Scheme $scheme;
      proxy_set_header X-Proxy-Port $proxy_port;
      proxy_set_header X-ELB-Proxy-Scheme "https";
      proxy_set_header X-ELB-Proxy-Port "443";

      # Always support web socket connection upgrades
      proxy_http_version 1.1;
      proxy_set_header Upgrade $http_upgrade;
      proxy_set_header Connection "upgrade";
    }

  #send to non-websocket process
  location / {
          proxy_pass http://localhost:8082;
          proxy_set_header Host $host;
          proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
          proxy_set_header X-Real-IP $proxy_protocol_addr;
          proxy_set_header X-NginX-Proxy true;
          proxy_set_header X-Proxy-Scheme $scheme;
          proxy_set_header X-Proxy-Port $proxy_port;
          proxy_set_header X-ELB-Proxy-Scheme "https";
          proxy_set_header X-ELB-Proxy-Port "443";

          # Always support web socket connection upgrades
          proxy_http_version 1.1;
          proxy_set_header Upgrade $http_upgrade;
          proxy_set_header Connection "upgrade";
  }


When any non-websocket request is sent to localhost:8082, I get an empty
reply. If I remove proxy_protocol from the first server, I get a response
as expected. Obviously, I need proxy_protocl to support the ingress from
our ELB, so removing it is not an option. However, I would like to know
what pieces I am missing to route traffic correctly -- and I would also
like to know why proxying a request locally from a proxy_protocol enabled
server to another nginx process (regardless of this second process using
proxy_protocol or not) fails.

For reference, the basic configuration of this secondary nginx process is
below:

upstream console {
   server localhost:3000 max_fails=3 fail_timeout=60 weight=1;
}

server {
    listen 8082;
   client_max_body_size 20M;

   location /console {
     proxy_pass http://console
   }

   .
   .
   .


}



Thank you all for your time,
Joseph Wonesh

-- 
This message is private and confidential. If you have received message in 
error, please notify us and remove from your system. 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx/attachments/20180822/079445b0/attachment.html>