<div dir="ltr"><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div dir="ltr"><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">@all : Can someone help /point-out what i have missed in proxy_protocol here? <br><br>
> I am using *NGINX 1.13.5 as a Load Balancer for one of my<br>
> CUSTOM-APPLICATION *which will listen on* UDP port 2231,67 and 68.*<br>
> <br>
> I am trying for Load Balancing with IP-Transparency.<br>
> <br>
> <br>
> <br>
> When I using the proxy_protocol method the packets received from a remote<br>
> client is modified and send to upstream by NGINX LB not sure why/how the<br>
> packet is modified and also the remote client IP is NOT as source IP.<br>
<br>
The proxy_protocol directive adds a PROXY protocol header to the datagram,<br>
that's why it's modified. The directive does not change the source address.<br>
Instead, the remote client address is passed in the PROXY protocol header.<br>
<br class="gmail-m_-3649759410976817533gmail-Apple-interchange-newline"><span style="color:rgb(0,0,0);font-family:monospace;white-space:pre-wrap"><jeya> : Okay. Do we have any options to send remote client IP as source address?
Due to additional proxy header the packet is dropped by the application running in the upstream.</span><span style="color:rgb(0,0,0);white-space:pre-wrap"> How can the proxy header can be stripped in the upstream end?</span> </blockquote><div> Do we need to do configuration/rules on the upstream end?</div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
> When I using proxy_bind, the packet is forwarded to configured upstream but<br>
> the source IP is not updated with Remote Client IP.<br>
<br>
What is the reason for the port next to $remote_addr in proxy_bind?<br>
Also make sure nginx master runs with sufficient privileges.<br></blockquote><div> </div><div><<span style="color:rgb(0,0,0);white-space:pre-wrap">Jeya> : Yes, application running with root privilege as specified in the conf file</span></div><pre style="white-space:pre-wrap;color:rgb(0,0,0)">Also, the proxy_bind syntax is referred in the below link.'</pre><pre style="white-space:pre-wrap"><a href="https://www.nginx.com/blog/ip-transparency-direct-server-return-nginx-plus-transparent-proxy/#proxy_bind" style="color:rgb(0,0,0)" target="_blank">https://www.nginx.com/blog/ip-transparency-direct-server-return-nginx-plus-transparent-proxy/#proxy_bind</a><font color="#000000"> </font></pre><div><span style="background-color:rgb(238,238,238);color:rgba(0,0,0,0);font-family:Consolas,"Andale Mono","DejaVu Sans Mono",monospace;font-style:inherit;font-weight:inherit;white-space:pre-wrap">proxy_bind $remote_addr:$remote_port transparent;</span> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<br>
> *Basically, in both methods, the remote client address was not used as a<br>
> source IP. I hope I missed some minor parts. Can someone help to resolve<br>
> this issue?*<br>
> <br>
> <br>
> <br>
> The following are the detailed configuration for your reference.<br>
> <br>
> <br>
> <br>
> *Method 1 :- proxy_protocol*<br>
> <br>
> <br>
> <br>
> *Configuration:*<br>
> <br>
> <br>
> <br>
> user *root;*<br>
> worker_processes 1;<br>
> error_log /var/log/nginx/error.log debug;<br>
> pid /var/run/nginx.pid;<br>
> events {<br>
> worker_connections 1024;<br>
> <br>
> }<br>
> <br>
> stream {<br>
> server {<br>
> listen <a href="http://10.43.18.107:2231" rel="noreferrer" target="_blank">10.43.18.107:2231</a> udp;<br>
> proxy_protocol on;<br>
> proxy_pass <a href="http://10.43.18.172:2231" rel="noreferrer" target="_blank">10.43.18.172:2231</a>;<br>
> }<br>
> server {<br>
> listen <a href="http://10.43.18.107:67" rel="noreferrer" target="_blank">10.43.18.107:67</a> udp;<br>
> proxy_protocol on;<br>
> proxy_pass <a href="http://10.43.18.172:67" rel="noreferrer" target="_blank">10.43.18.172:67</a>;<br>
> }<br>
> server {<br>
> listen <a href="http://10.43.18.107:68" rel="noreferrer" target="_blank">10.43.18.107:68</a> udp;<br>
> proxy_protocol on;<br>
> proxy_pass <a href="http://10.43.18.172:68" rel="noreferrer" target="_blank">10.43.18.172:68</a>;<br>
> }<br>
> }<br>
> <br>
> *TCPDUMP O/P :*<br>
> <br>
> <br>
> <br>
> *From LB:*<br>
> <br>
> 10:05:07.284259 IP 10.43.18.116.2231 > 10.43.18.107.2231: UDP, length 43<br>
> <br>
> 10:05:07.284555 IP 10.43.18.107.51775 > 10.43.18.172.2231: UDP, length 91<br>
> <br>
> <br>
> <br>
> *From upstream[Custom application]:*<br>
> <br>
> 10:05:07.284442 IP 10.43.18.107.51775 > 10.43.18.172.2231: UDP, length 91<br>
> <br>
> <br>
> <br>
> *Method 2:- [ proxy_bind ]*<br>
> <br>
> <br>
> <br>
> *Configuration:*<br>
> <br>
> <br>
> <br>
> user root;<br>
> worker_processes 1;<br>
> error_log /var/log/nginx/error.log debug;<br>
> pid /var/run/nginx.pid;<br>
> events {<br>
> worker_connections 1024;<br>
> }<br>
> <br>
> stream {<br>
> server {<br>
> listen <a href="http://10.43.18.107:2231" rel="noreferrer" target="_blank">10.43.18.107:2231</a> udp;<br>
> proxy_bind $remote_addr:2231 transparent;<br>
> proxy_pass <a href="http://10.43.18.172:2231" rel="noreferrer" target="_blank">10.43.18.172:2231</a>;<br>
> }<br>
> server {<br>
> listen <a href="http://10.43.18.107:67" rel="noreferrer" target="_blank">10.43.18.107:67</a> udp;<br>
> proxy_bind $remote_addr:67 transparent;<br>
> proxy_pass <a href="http://10.43.18.172:67" rel="noreferrer" target="_blank">10.43.18.172:67</a>;<br>
> }<br>
> server {<br>
> listen <a href="http://10.43.18.107:68" rel="noreferrer" target="_blank">10.43.18.107:68</a> udp;<br>
> proxy_bind $remote_addr:68 transparent;<br>
> proxy_pass <a href="http://10.43.18.172:68" rel="noreferrer" target="_blank">10.43.18.172:68</a>;<br>
> }<br>
> <br>
> }<br>
> <br>
> <br>
> <br>
> *Also, added the below rules :*<br>
> <br>
> <br>
> <br>
> ip rule add fwmark 1 lookup 100<br>
> <br>
> ip route add local <a href="http://0.0.0.0/0" rel="noreferrer" target="_blank">0.0.0.0/0</a> dev lo table 100<br>
> iptables -t mangle -A PREROUTING -p udp -s <a href="http://10.43.18.0/24" rel="noreferrer" target="_blank">10.43.18.0/24</a> --sport 2231 -j<br>
> MARK --set-xmark 0x1/0xffffffff<br>
> iptables -t mangle -A PREROUTING -p udp -s <a href="http://10.43.18.0/24" rel="noreferrer" target="_blank">10.43.18.0/24</a> --sport 67 -j MARK<br>
> --set-xmark 0x1/0xffffffff<br>
> iptables -t mangle -A PREROUTING -p udp -s <a href="http://10.43.18.0/24" rel="noreferrer" target="_blank">10.43.18.0/24</a> --sport 68 -j MARK<br>
> --set-xmark 0x1/0xffffffff<br>
> <br>
> <br>
> <br>
> However, still, the packet is sent from NGINX LB with its own IP, not with<br>
> the remote client IP address.<br>
> <br>
> <br>
> <br>
> *TCPDUMP O/P from LB:*<br>
> <br>
> <br>
> <br>
> 11:49:51.999829 IP 10.43.18.116.2231 > 10.43.18.107.2231: UDP, length 43<br>
> <br>
> 11:49:52.000161 IP 10.43.18.107.2231 > 10.43.18.172.2231: UDP, length 43<br>
> <br>
> <br>
> <br>
> *TPCDUM O/P from Upstream:*<br>
> <br>
> <br>
> <br>
> 11:49:52.001155 IP 10.43.18.107.2231 > 10.43.18.172.2231: UDP, length 43<br>
> <br>
> <br>
> <br>
> *Note:* I have followed the below link.<br>
> <br>
> <br>
> <br>
> <a href="https://www.nginx.com/blog/ip-transparency-direct-server-return-nginx-plus-transparent-proxy/" rel="noreferrer" target="_blank">https://www.nginx.com/blog/ip-transparency-direct-server-return-nginx-plus-transparent-proxy/</a><br>
<br>
> _______________________________________________<br>
> nginx mailing list<br>
> <a href="mailto:nginx@nginx.org" target="_blank">nginx@nginx.org</a><br>
> <a href="http://mailman.nginx.org/mailman/listinfo/nginx" rel="noreferrer" target="_blank">http://mailman.nginx.org/mailman/listinfo/nginx</a><br>
<br>
<br>
-- <br>
Roman Arutyunyan<br>
_______________________________________________<br>
nginx mailing list<br>
<a href="mailto:nginx@nginx.org" target="_blank">nginx@nginx.org</a><br>
<a href="http://mailman.nginx.org/mailman/listinfo/nginx" rel="noreferrer" target="_blank">http://mailman.nginx.org/mailman/listinfo/nginx</a><br>
</blockquote></div></div>
</div>
</blockquote></div></div>