<div dir="ltr"><div class="gmail_quote"><div dir="ltr"><div class="gmail_quote"><div dir="ltr"><div><p class="MsoNormal" style="margin:0in 0in 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif"><span style="font-family:Arial,sans-serif">Hi
all,</span></p>
<p class="MsoNormal" style="margin:0in 0in 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif"><span style="font-family:Arial,sans-serif"><br>
<span style="background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial">I am using </span><b>NGINX 1.13.5 as a Load Balancer for one of my CUSTOM-APPLICATION </b><span style="background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial">which will listen on</span><b> UDP port 2231,67 and 68.</b><span style="background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"> </span><br>
<br>
<span style="background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial">I am trying for Load Balancing with IP-Transparency. </span></span></p>
<p class="MsoNormal" style="margin:0in 0in 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif"><span style="font-family:Arial,sans-serif;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"> </span></p>
<p class="MsoNormal" style="margin:0in 0in 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif"><span style="font-family:Arial,sans-serif;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial">When I using the proxy_protocol method the packets received from
a remote client is modified and send to upstream by NGINX LB not sure why/how
the packet is modified and also the remote client IP is NOT as source IP.</span></p>
<p class="MsoNormal" style="margin:0in 0in 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif"><span style="font-family:Arial,sans-serif;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"> </span></p>
<p class="MsoNormal" style="margin:0in 0in 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif"><span style="font-family:Arial,sans-serif;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial">When I using proxy_bind, the packet is forwarded to
configured upstream but the source IP is not updated with Remote Client IP. </span></p>
<p class="MsoNormal" style="margin:0in 0in 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif"><span style="font-family:Arial,sans-serif;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"> </span></p>
<p class="MsoNormal" style="margin:0in 0in 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif"><span style="font-family:Arial,sans-serif;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"><b><u>Basically, in both methods, the remote client address was not
used as a source IP. I hope I missed some minor parts. Can someone help to
resolve this issue?</u></b></span></p>
<p class="MsoNormal" style="margin:0in 0in 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif"><span style="font-family:Arial,sans-serif;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial"> </span></p>
<p class="MsoNormal" style="margin:0in 0in 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif"><span style="font-family:Arial,sans-serif;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial">The following are the detailed configuration for your reference.</span></p>
<p class="MsoNormal" style="margin:0in 0in 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif"><span style="font-family:Arial,sans-serif"> </span></p>
<p class="MsoNormal" style="margin:0in 0in 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif"><b><u><span style="font-family:Arial,sans-serif">Method 1 :- proxy_protocol</span></u></b><span style="font-family:Arial,sans-serif"></span></p>
<p class="MsoNormal" style="margin:0in 0in 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif"><span style="font-family:Arial,sans-serif"> </span></p>
<p class="MsoNormal" style="margin:0in 0in 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif"><b><span style="font-family:Arial,sans-serif">Configuration:</span></b><span style="font-family:Arial,sans-serif"></span></p>
<p class="MsoNormal" style="margin:0in 0in 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif"><span style="font-family:Arial,sans-serif"> </span></p>
<p class="MsoNormal" style="margin:0in 0in 0.0001pt 0.5in;font-size:11pt;font-family:Calibri,sans-serif"><span style="font-family:"Courier New"">user <b>root;</b><br>
worker_processes 1;<br>
error_log /var/log/nginx/error.log debug;<br>
pid /var/run/nginx.pid;<br>
events {<br>
worker_connections 1024;</span><span style="font-family:"Times New Roman",serif"></span></p>
<p class="MsoNormal" style="margin:0in 0in 0.0001pt 0.5in;font-size:11pt;font-family:Calibri,sans-serif"><span style="font-family:"Courier New"">}</span><span style="font-family:Arial,sans-serif"></span></p>
<p class="MsoNormal" style="margin:0in 0in 12pt 0.5in;font-size:11pt;font-family:Calibri,sans-serif"><span style="font-family:"Courier New"">stream
{<br>
server {<br>
listen <a href="http://10.43.18.107:2231" target="_blank">10.43.18.107:2231</a> udp;<br>
proxy_protocol on;<br>
proxy_pass <a href="http://10.43.18.172:2231" target="_blank">10.43.18.172:2231</a>;<br>
}<br>
server {<br>
listen <a href="http://10.43.18.107:67" target="_blank">10.43.18.107:67</a> udp;<br>
proxy_protocol on;<br>
proxy_pass <a href="http://10.43.18.172:67" target="_blank">10.43.18.172:67</a>;<br>
}<br>
server {<br>
listen <a href="http://10.43.18.107:68" target="_blank">10.43.18.107:68</a> udp;<br>
proxy_protocol on;<br>
proxy_pass <a href="http://10.43.18.172:68" target="_blank">10.43.18.172:68</a>;<br>
}<br>
}</span><span style="font-family:Arial,sans-serif"></span></p>
<p class="MsoNormal" style="margin:0in 0in 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif"><b><u><span style="font-family:Arial,sans-serif">TCPDUMP O/P :</span></u></b><span style="font-family:Arial,sans-serif"></span></p>
<p class="MsoNormal" style="margin:0in 0in 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif"><span style="font-family:Arial,sans-serif"> </span></p>
<p class="MsoNormal" style="margin:0in 0in 0.0001pt 0.5in;font-size:11pt;font-family:Calibri,sans-serif"><b><u><span style="font-family:Arial,sans-serif">From LB:</span></u></b><span style="font-family:Arial,sans-serif"></span></p>
<p class="MsoNormal" style="margin:0in 0in 0.0001pt 0.5in;font-size:11pt;font-family:Calibri,sans-serif"><span style="font-family:"Courier New"">10:05:07.284259 IP 10.43.18.116.2231 > 10.43.18.107.2231:
UDP, length 43</span><span style="font-family:Arial,sans-serif"></span></p>
<p class="MsoNormal" style="margin:0in 0in 0.0001pt 0.5in;font-size:11pt;font-family:Calibri,sans-serif"><span style="font-family:"Courier New"">10:05:07.284555 IP 10.43.18.107.51775 > 10.43.18.172.2231:
UDP, length 91</span><span style="font-family:Arial,sans-serif"></span></p>
<p class="MsoNormal" style="margin:0in 0in 0.0001pt 0.5in;font-size:11pt;font-family:Calibri,sans-serif"><span style="font-family:Arial,sans-serif"> </span></p>
<p class="MsoNormal" style="margin:0in 0in 0.0001pt 0.5in;font-size:11pt;font-family:Calibri,sans-serif"><b><u><span style="font-family:Arial,sans-serif">From upstream[Custom application]:</span></u></b><span style="font-family:Arial,sans-serif"></span></p>
<p class="MsoNormal" style="margin:0in 0in 0.0001pt 0.5in;font-size:11pt;font-family:Calibri,sans-serif"><span style="font-family:"Courier New"">10:05:07.284442 IP 10.43.18.107.51775 > 10.43.18.172.2231:
UDP, length 91</span><span style="font-family:Arial,sans-serif"></span></p>
<p class="MsoNormal" style="margin:0in 0in 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif"><span style="font-family:Arial,sans-serif"> </span></p>
<p class="MsoNormal" style="margin:0in 0in 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif"><b><u><span style="font-family:Arial,sans-serif">Method 2:- [ proxy_bind ]</span></u></b><span style="font-family:Arial,sans-serif"></span></p>
<p class="MsoNormal" style="margin:0in 0in 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif"><span style="font-family:Arial,sans-serif"> </span></p>
<p class="MsoNormal" style="margin:0in 0in 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif"><b><u><span style="font-family:Arial,sans-serif">Configuration:</span></u></b><span style="font-family:Arial,sans-serif"></span></p>
<p class="MsoNormal" style="margin:0in 0in 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif"><span style="font-family:Arial,sans-serif"> </span></p>
<p class="MsoNormal" style="margin:0in 0in 0.0001pt 0.5in;font-size:11pt;font-family:Calibri,sans-serif"><span style="font-family:"Courier New"">user root;<br>
worker_processes 1;<br>
error_log /var/log/nginx/error.log debug;<br>
pid /var/run/nginx.pid;<br>
events {<br>
worker_connections 1024;<br>
}</span><span style="font-family:Arial,sans-serif"></span></p>
<p class="MsoNormal" style="margin:0in 0in 0.0001pt 0.5in;font-size:11pt;font-family:Calibri,sans-serif"><span style="font-family:"Courier New"">stream {<br>
server {<br>
listen <a href="http://10.43.18.107:2231" target="_blank">10.43.18.107:2231</a> udp;<br>
proxy_bind $remote_addr:2231 transparent;<br>
proxy_pass <a href="http://10.43.18.172:2231" target="_blank">10.43.18.172:2231</a>;<br>
}<br>
server {<br>
listen <a href="http://10.43.18.107:67" target="_blank">10.43.18.107:67</a> udp;<br>
proxy_bind $remote_addr:67 transparent;<br>
proxy_pass <a href="http://10.43.18.172:67" target="_blank">10.43.18.172:67</a>;<br>
}<br>
server {<br>
listen <a href="http://10.43.18.107:68" target="_blank">10.43.18.107:68</a> udp;<br>
proxy_bind $remote_addr:68 transparent;<br>
proxy_pass <a href="http://10.43.18.172:68" target="_blank">10.43.18.172:68</a>;<br>
}</span><span style="font-family:Arial,sans-serif"></span></p>
<p class="MsoNormal" style="margin:0in 0in 0.0001pt 0.5in;font-size:11pt;font-family:Calibri,sans-serif"><span style="font-family:"Courier New"">}</span><span style="font-family:Arial,sans-serif"></span></p>
<p class="MsoNormal" style="margin:0in 0in 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif"><span style="font-family:Arial,sans-serif"> </span></p>
<p class="MsoNormal" style="margin:0in 0in 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif"><b><u><span style="font-family:Arial,sans-serif">Also, added the below
rules :</span></u></b></p>
<p class="MsoNormal" style="margin:0in 0in 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif"><span style="font-family:Arial,sans-serif"> </span></p>
<p class="MsoNormal" style="margin:0in 0in 0.0001pt 0.5in;font-size:11pt;font-family:Calibri,sans-serif"><span style="font-family:"Courier New"">ip rule add fwmark 1 lookup 100</span><span style="font-family:Arial,sans-serif"></span></p>
<p class="MsoNormal" style="margin:0in 0in 0.0001pt 0.5in;font-size:11pt;font-family:Calibri,sans-serif"><span style="font-family:"Courier New"">ip route add local <a href="http://0.0.0.0/0" target="_blank">0.0.0.0/0</a> dev lo table 100<br>
iptables -t mangle -A PREROUTING -p udp -s <a href="http://10.43.18.0/24" target="_blank">10.43.18.0/24</a> --sport 2231 -j MARK
--set-xmark 0x1/0xffffffff<br>
iptables -t mangle -A PREROUTING -p udp -s <a href="http://10.43.18.0/24" target="_blank">10.43.18.0/24</a> --sport 67 -j MARK
--set-xmark 0x1/0xffffffff<br>
iptables -t mangle -A PREROUTING -p udp -s <a href="http://10.43.18.0/24" target="_blank">10.43.18.0/24</a> --sport 68 -j MARK
--set-xmark 0x1/0xffffffff</span><span style="font-family:Arial,sans-serif"></span></p>
<p class="MsoNormal" style="margin:0in 0in 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif"><span style="font-family:Arial,sans-serif"> </span></p>
<p class="MsoNormal" style="margin:0in 0in 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif"><span style="font-family:Arial,sans-serif">However,
still, the packet is sent from NGINX LB with its own IP, not with the remote
client IP address.</span></p>
<p class="MsoNormal" style="margin:0in 0in 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif"><span style="font-family:Arial,sans-serif"> </span></p>
<p class="MsoNormal" style="margin:0in 0in 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif"><b><u><span style="font-size:10pt;font-family:Arial,sans-serif">TCPDUMP O/P from LB:</span></u></b></p>
<p class="MsoNormal" style="margin:0in 0in 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif"><span style="font-family:Arial,sans-serif"> </span></p>
<p class="MsoNormal" style="margin:0in 0in 0.0001pt 0.5in;font-size:11pt;font-family:Calibri,sans-serif"><span style="font-family:"Courier New"">11:49:51.999829
IP 10.43.18.116.2231 > 10.43.18.107.2231: UDP, length 43</span></p>
<p class="MsoNormal" style="margin:0in 0in 0.0001pt 0.5in;font-size:11pt;font-family:Calibri,sans-serif"><span style="font-family:"Courier New"">11:49:52.000161
IP 10.43.18.107.2231 > 10.43.18.172.2231: UDP, length 43</span></p>
<p class="MsoNormal" style="margin:0in 0in 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif"><span style="font-family:"Courier New""> </span></p>
<p class="MsoNormal" style="margin:0in 0in 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif"><b><u><span style="font-family:"Courier New"">TPCDUM O/P from Upstream:</span></u></b></p>
<p class="MsoNormal" style="margin:0in 0in 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif"><span style="font-family:"Courier New""> </span></p>
<p class="MsoNormal" style="text-indent:0.5in;margin:0in 0in 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif"><span style="font-family:"Courier New"">11:49:52.001155 IP 10.43.18.107.2231 > 10.43.18.172.2231:
UDP, length 43</span></p>
<p class="MsoNormal" style="margin:0in 0in 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif"><span style="font-family:Arial,sans-serif"> </span></p>
<p class="MsoNormal" style="margin:0in 0in 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif"><b><span style="font-family:Arial,sans-serif">Note:</span></b><span style="font-family:Arial,sans-serif"> I have followed the below
link. </span></p>
<p class="MsoNormal" style="margin:0in 0in 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif"><span style="font-family:Arial,sans-serif"> </span></p>
<p class="MsoNormal" style="text-indent:0.5in;margin:0in 0in 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif"><span style="font-family:Arial,sans-serif"><a href="https://www.nginx.com/blog/ip-transparency-direct-server-return-nginx-plus-transparent-proxy/" style="color:rgb(5,99,193)" target="_blank"><span style="color:rgb(17,85,204)">https://www.nginx.com/blog/ip-transparency-direct-server-return-nginx-plus-transparent-proxy/</span></a> </span></p>
<p class="MsoNormal" style="margin:0in 0in 0.0001pt;font-size:11pt;font-family:Calibri,sans-serif"> </p></div></div>
</div></div>
</div></div>