How to configure Nginx LB IP-Transparency for custom UDP application
Aleksandar Lazic
al-nginx at none.at
Fri Jul 19 20:49:48 UTC 2019
Am 16.07.2019 um 13:29 schrieb Jeya Murugan:
> @all : Can someone help /point-out what i have missed in proxy_protocol
> here?
the proxy protocol is only designed for tcp not udp.
> > I am using *NGINX 1.13.5 as a Load Balancer for one of my
> > CUSTOM-APPLICATION *which will listen on* UDP port 2231,67 and 68.*
> >
> > I am trying for Load Balancing with IP-Transparency.
> >
> >
> >
> > When I using the proxy_protocol method the packets received from a remote
> > client is modified and send to upstream by NGINX LB not sure why/how the
> > packet is modified and also the remote client IP is NOT as source IP.
>
> The proxy_protocol directive adds a PROXY protocol header to the datagram,
> that's why it's modified. The directive does not change the source address.
> Instead, the remote client address is passed in the PROXY protocol header.
>
> <jeya> : Okay. Do we have any options to send remote client IP as source
> address? Due to additional proxy header the packet is dropped by the
> application running in the upstream.How can the proxy header can be
> stripped in the upstream end?
>
> Do we need to do configuration/rules on the upstream end?
>
>
> > When I using proxy_bind, the packet is forwarded to configured
> upstream but
> > the source IP is not updated with Remote Client IP.
>
> What is the reason for the port next to $remote_addr in proxy_bind?
> Also make sure nginx master runs with sufficient privileges.
>
>
> <Jeya> : Yes, application running with root privilege as specified in the
> conf file
>
> Also, the proxy_bind syntax is referred in the below link.'
>
> https://www.nginx.com/blog/ip-transparency-direct-server-return-nginx-plus-transparent-proxy/#proxy_bind
>
> proxy_bind $remote_addr:$remote_port transparent;
>
>
> > *Basically, in both methods, the remote client address was not used as a
> > source IP. I hope I missed some minor parts. Can someone help to resolve
> > this issue?*
> >
> >
> >
> > The following are the detailed configuration for your reference.
> >
> >
> >
> > *Method 1 :- proxy_protocol*
> >
> >
> >
> > *Configuration:*
> >
> >
> >
> > user *root;*
> > worker_processes 1;
> > error_log /var/log/nginx/error.log debug;
> > pid /var/run/nginx.pid;
> > events {
> > worker_connections 1024;
> >
> > }
> >
> > stream {
> > server {
> > listen 10.43.18.107:2231 <http://10.43.18.107:2231> udp;
> > proxy_protocol on;
> > proxy_pass 10.43.18.172:2231 <http://10.43.18.172:2231>;
> > }
> > server {
> > listen 10.43.18.107:67 <http://10.43.18.107:67> udp;
> > proxy_protocol on;
> > proxy_pass 10.43.18.172:67 <http://10.43.18.172:67>;
> > }
> > server {
> > listen 10.43.18.107:68 <http://10.43.18.107:68> udp;
> > proxy_protocol on;
> > proxy_pass 10.43.18.172:68 <http://10.43.18.172:68>;
> > }
> > }
> >
> > *TCPDUMP O/P :*
> >
> >
> >
> > *From LB:*
> >
> > 10:05:07.284259 IP 10.43.18.116.2231 > 10.43.18.107.2231: UDP, length 43
> >
> > 10:05:07.284555 IP 10.43.18.107.51775 > 10.43.18.172.2231: UDP, length 91
> >
> >
> >
> > *From upstream[Custom application]:*
> >
> > 10:05:07.284442 IP 10.43.18.107.51775 > 10.43.18.172.2231: UDP, length 91
> >
> >
> >
> > *Method 2:- [ proxy_bind ]*
> >
> >
> >
> > *Configuration:*
> >
> >
> >
> > user root;
> > worker_processes 1;
> > error_log /var/log/nginx/error.log debug;
> > pid /var/run/nginx.pid;
> > events {
> > worker_connections 1024;
> > }
> >
> > stream {
> > server {
> > listen 10.43.18.107:2231 <http://10.43.18.107:2231> udp;
> > proxy_bind $remote_addr:2231 transparent;
> > proxy_pass 10.43.18.172:2231 <http://10.43.18.172:2231>;
> > }
> > server {
> > listen 10.43.18.107:67 <http://10.43.18.107:67> udp;
> > proxy_bind $remote_addr:67 transparent;
> > proxy_pass 10.43.18.172:67 <http://10.43.18.172:67>;
> > }
> > server {
> > listen 10.43.18.107:68 <http://10.43.18.107:68> udp;
> > proxy_bind $remote_addr:68 transparent;
> > proxy_pass 10.43.18.172:68 <http://10.43.18.172:68>;
> > }
> >
> > }
> >
> >
> >
> > *Also, added the below rules :*
> >
> >
> >
> > ip rule add fwmark 1 lookup 100
> >
> > ip route add local 0.0.0.0/0 <http://0.0.0.0/0> dev lo table 100
> > iptables -t mangle -A PREROUTING -p udp -s 10.43.18.0/24
> <http://10.43.18.0/24> --sport 2231 -j
> > MARK --set-xmark 0x1/0xffffffff
> > iptables -t mangle -A PREROUTING -p udp -s 10.43.18.0/24
> <http://10.43.18.0/24> --sport 67 -j MARK
> > --set-xmark 0x1/0xffffffff
> > iptables -t mangle -A PREROUTING -p udp -s 10.43.18.0/24
> <http://10.43.18.0/24> --sport 68 -j MARK
> > --set-xmark 0x1/0xffffffff
> >
> >
> >
> > However, still, the packet is sent from NGINX LB with its own IP, not with
> > the remote client IP address.
> >
> >
> >
> > *TCPDUMP O/P from LB:*
> >
> >
> >
> > 11:49:51.999829 IP 10.43.18.116.2231 > 10.43.18.107.2231: UDP, length 43
> >
> > 11:49:52.000161 IP 10.43.18.107.2231 > 10.43.18.172.2231: UDP, length 43
> >
> >
> >
> > *TPCDUM O/P from Upstream:*
> >
> >
> >
> > 11:49:52.001155 IP 10.43.18.107.2231 > 10.43.18.172.2231: UDP, length 43
> >
> >
> >
> > *Note:* I have followed the below link.
> >
> >
> >
> >
> https://www.nginx.com/blog/ip-transparency-direct-server-return-nginx-plus-transparent-proxy/
>
> > _______________________________________________
> > nginx mailing list
> > nginx at nginx.org <mailto:nginx at nginx.org>
> > http://mailman.nginx.org/mailman/listinfo/nginx
>
>
> --
> Roman Arutyunyan
> _______________________________________________
> nginx mailing list
> nginx at nginx.org <mailto:nginx at nginx.org>
> http://mailman.nginx.org/mailman/listinfo/nginx
>
>
> _______________________________________________
> nginx mailing list
> nginx at nginx.org
> http://mailman.nginx.org/mailman/listinfo/nginx
>
More information about the nginx
mailing list