can't setup nginx as transparent proxy server

[Roman Arutyunyan]

Hi,

On Tue, Aug 09, 2016 at 01:20:46PM +0800, Peng Xie wrote:
> Hi,
> 
> I am relatively new to nginx.  I would like to setup nginx as a
> transparent reverse proxy.
> 
> Here is the topology of my network.
> ,----
> | +------------------------+
> | |                        |
> | |   192.168.56.109:80    | <-- upstream which is the real http server on port 80
> | |                        |
> | +------------------------+
> |            ^
> |            |
> |            |
> | +------------------------+
> | |                        |
> | |   192.168.56.108:800   | <-- proxy_server which run nginx as a reverse proxy server on port 800
> | |                        |
> | +------------------------+
> |            ^
> |            |
> |            |
> | +------------------------+
> | |                        |
> | |     192.168.56.1       | <-- client
> | |                        |
> | +------------------------+
> `----
> 
> Here is my nginx.conf.
> ,----
> | server {
> |        listen       800;
> |        server_name  localhost;
> | 
> |        location / {
> |                 proxy_pass       http://192.168.56.109:80;
> |                 proxy_bind $remote_addr  transparent;
> | }
> `----
> 
> If not use proxy_bind, Cient can access upstream through
> 192.168.56.108:800. Of course, the proxy is not transparent in this
> situation.
> 
> To make the proxy_server transparent, I read these documents: doc1)
> [http://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_bind]
> 
> doc2) [https://www.kernel.org/doc/Documentation/networking/tproxy.tx]
> 
> Add proxy_bind into nginx.conf according to doc1. Reload nginx:
> ,----
> | nginx -s reload
> `----
> 
> According to doc2, I write a shell-script as follow:
> ,----
> | #!/bin/bash
> | set -x
> | sudo iptables -F
> | sudo iptables -X
> | 
> | sudo iptables -t mangle -N DIVERT;
> | sudo iptables -t mangle -A PREROUTING -p tcp -m socket -j DIVERT;
> | sudo iptables -t mangle -A DIVERT -j MARK --set-mark 1;
> | sudo iptables -t mangle -A DIVERT -j ACCEPT;
> | sudo ip rule add fwmark 1 lookup 100;
> | sudo ip route add local 0.0.0.0/0 dev lo table 100;
> | sudo iptables -t mangle -A PREROUTING -p tcp --dport 80 -j TPROXY  --tproxy-mark 0x1/0x1 --on-port 800;
> `----
> 
> Now, I access proxy on client:
> ,----
> | ➜  ~ curl -v http://192.168.56.108:800
> | * Rebuilt URL to: http://192.168.56.108:800/
> | *   Trying 192.168.56.108...
> | * Connected to 192.168.56.108 (192.168.56.108) port 800 (#0)
> | > GET / HTTP/1.1
> | > Host: 192.168.56.108:800
> | > User-Agent: curl/7.43.0
> | > Accept: */*
> | >
> `----
> 
> And then I try port 80:
> ,----
> | ➜  ~ curl -v http://192.168.56.108:80
> | * Rebuilt URL to: http://192.168.56.108:80/
> | *   Trying 192.168.56.108...
> `----
> 
> Client can't access the upstream now!
> 
> Use proxy_bind to set a transparent proxy server may be a new feature on
> nginx. I've searched for a long time. Does anybody have a suggestion?
> 
> Thanks Peng Xie

Did you try to tcpdump the packets at proxy and upstream?

-- 
Roman Arutyunyan