<div dir="ltr"><div>Thank you for your reply, Roman.</div><div><br></div><div>It turns out the issue was adding the various proxy and forwarded headers in that entry. Changing it to the below block fixed my issue:</div><div><br></div><div><pre style="white-space:pre-wrap;text-decoration-style:initial;text-decoration-color:initial;background-color:rgb(43,43,43);color:rgb(169,183,198);font-family:Menlo;font-size:9pt">#send to non-websocket process<br> location / {<br> proxy_pass <a href="http://localhost:8082/" target="_blank" style="color:rgb(17,85,204)">http://localhost:8082</a>;<br> proxy_set_header Host $host;<br> }</pre></div><div><br></div><div>Now everything is working as expected.</div><div><br></div><div>Cheers,</div><div>Joseph Wonesh</div><div class="gmail_extra"><br><div class="gmail_quote">On Thu, Aug 23, 2018 at 6:46 AM, Roman Arutyunyan <span dir="ltr"><<a href="mailto:arut@nginx.com" target="_blank">arut@nginx.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Hello Joseph,<br>
<div><div class="h5"><br>
On Wed, Aug 22, 2018 at 10:24:23AM -0400, Joseph Wonesh wrote:<br>
> Hello,<br>
> <br>
> I am trying to route requests such that those requiring websockets will<br>
> route to a long-lived nginx process, and all others will go to the general<br>
> reverse-proxy which handles all other traffic. These nginx processes exist<br>
> in our AWS cloud behind an ELB that has been configured to use Proxy<br>
> Protocol. Note that all of this works correctly with our current setup<br>
> which uses only one nginx process that is configured to use proxy_protocol.<br>
> <br>
> The change to this setup is as follows:<br>
> <br>
> The first nginx server handling all ingress uses proxy_protocol and<br>
> forwards requests to either the websocket or non-websocket nginx servers<br>
> locally:<br>
> <br>
> server {<br>
> listen 8080 proxy_protocol;<br>
> real_ip_header proxy_protocol;<br>
> charset utf-8;<br>
> client_max_body_size 20M;<br>
> <br>
> #send to websocket process<br>
> location /client {<br>
> proxy_pass <a href="http://localhost:8084" rel="noreferrer" target="_blank">http://localhost:8084</a>;<br>
> proxy_set_header Host $host;<br>
> proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;<br>
> proxy_set_header X-Real-IP $proxy_protocol_addr;<br>
> proxy_set_header X-NginX-Proxy true;<br>
> proxy_set_header X-Proxy-Scheme $scheme;<br>
> proxy_set_header X-Proxy-Port $proxy_port;<br>
> proxy_set_header X-ELB-Proxy-Scheme "https";<br>
> proxy_set_header X-ELB-Proxy-Port "443";<br>
> <br>
> # Always support web socket connection upgrades<br>
> proxy_http_version 1.1;<br>
> proxy_set_header Upgrade $http_upgrade;<br>
> proxy_set_header Connection "upgrade";<br>
> }<br>
> <br>
> #send to non-websocket process<br>
> location / {<br>
> proxy_pass <a href="http://localhost:8082" rel="noreferrer" target="_blank">http://localhost:8082</a>;<br>
> proxy_set_header Host $host;<br>
> proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;<br>
> proxy_set_header X-Real-IP $proxy_protocol_addr;<br>
> proxy_set_header X-NginX-Proxy true;<br>
> proxy_set_header X-Proxy-Scheme $scheme;<br>
> proxy_set_header X-Proxy-Port $proxy_port;<br>
> proxy_set_header X-ELB-Proxy-Scheme "https";<br>
> proxy_set_header X-ELB-Proxy-Port "443";<br>
> <br>
> # Always support web socket connection upgrades<br>
> proxy_http_version 1.1;<br>
> proxy_set_header Upgrade $http_upgrade;<br>
> proxy_set_header Connection "upgrade";<br>
> }<br>
> <br>
> <br>
> When any non-websocket request is sent to localhost:8082, I get an empty<br>
> reply. If I remove proxy_protocol from the first server, I get a response<br>
> as expected.<br>
<br>
</div></div>If removing the proxy_protocol parameter from a server's listen directive fixes<br>
the issue with a specific connection, this means that this connection does not<br>
support the PROXY protocol, whether it is curl or a proxied connection from AWS.<br>
So please make sure the PROXY protocol header is actually sent over the<br>
connection. It's easy to check by tcpdumping at most 107 first bytes,<br>
especially if PROXY protocol v1 is used.<br>
<span class=""><br>
> Obviously, I need proxy_protocl to support the ingress from<br>
> our ELB, so removing it is not an option. However, I would like to know<br>
> what pieces I am missing to route traffic correctly -- and I would also<br>
> like to know why proxying a request locally from a proxy_protocol enabled<br>
> server to another nginx process (regardless of this second process using<br>
> proxy_protocol or not) fails.<br>
<br>
</span>If you configure nginx to expect PROXY protocol (listen ... proxy_protocol),<br>
then the client should actually send it. AWS balancers can send it, as well<br>
as nginx stream module can (proxy_protocol on). The nginx http module<br>
cannot send PROXY protocol header (use X-Forwarded-For instead), so you<br>
cannot proxy an HTTP connection to a PROXY-protocol-enabled server.<br>
<div class="HOEnZb"><div class="h5"><br>
> For reference, the basic configuration of this secondary nginx process is<br>
> below:<br>
> <br>
> upstream console {<br>
> server localhost:3000 max_fails=3 fail_timeout=60 weight=1;<br>
> }<br>
> <br>
> server {<br>
> listen 8082;<br>
> client_max_body_size 20M;<br>
> <br>
> location /console {<br>
> proxy_pass <a href="http://console" rel="noreferrer" target="_blank">http://console</a><br>
> }<br>
> <br>
> .<br>
> .<br>
> .<br>
> <br>
> <br>
> }<br>
> <br>
> <br>
> <br>
> Thank you all for your time,<br>
> Joseph Wonesh<br>
> <br>
</div></div><span class="HOEnZb"><font color="#888888">> -- <br>
> This message is private and confidential. If you have received message in <br>
> error, please notify us and remove from your system. <br>
<br>
> ______________________________<wbr>_________________<br>
> nginx mailing list<br>
> <a href="mailto:nginx@nginx.org">nginx@nginx.org</a><br>
> <a href="http://mailman.nginx.org/mailman/listinfo/nginx" rel="noreferrer" target="_blank">http://mailman.nginx.org/<wbr>mailman/listinfo/nginx</a><br>
<br>
<br>
-- <br>
Roman Arutyunyan<br>
______________________________<wbr>_________________<br>
nginx mailing list<br>
<a href="mailto:nginx@nginx.org">nginx@nginx.org</a><br>
<a href="http://mailman.nginx.org/mailman/listinfo/nginx" rel="noreferrer" target="_blank">http://mailman.nginx.org/<wbr>mailman/listinfo/nginx</a><br>
</font></span></blockquote></div><br></div></div>
<br>
<span style="color:rgb(102,102,102);font-family:Arial,sans-serif;font-size:x-small;background-color:rgb(255,255,255)">This message is private and confidential. If you have received message in error, please notify us and remove from your system. </span>