Varnish + nginx with php and REMOTE_ADDR issue using http_realip_module

[Maxim Dounin]

Hello!

On Fri, Jun 10, 2011 at 08:41:30PM -0300, Flavio Torres wrote:

> Hello,
> 
> Please, I'm trying to use http_realip_module
> (http://wiki.nginx.org/NginxHttpRealIpModule#real_ip_header) to get the
> X-Forwarded-For IP in my .php, but I'm suspecting something is wrong,
> because that looks like it should work.
> 
> My setup is:
> 
> LB > Varnish FE network pool (10.110.0.0/22) > Varnish BE - same server,
> another iface - (10.214.0.0/22) > LB 10.214.0.0/22 > nginx BE pool
> 10.214.0.0/22
> 
> When I set:
> 
>         location ~ \.php$ {
>                 fastcgi_pass 127.0.0.1:9000;
>                 fastcgi_index index.php;
>                 include fastcgi_params;
>                 fastcgi_param SCRIPT_FILENAME
> /net/nfs_parceiros_1/ftorres.com.br$fastcgi_script_name;
>         }
> 
> 
> I get these values from my simple php test page:
> 
> <?php
>     echo '<table border="1">';
> 
>     foreach($_SERVER as $k => $v) {
>         echo '<tr><td>'.$k.'</td><td>'.$v.'</td></tr>';
>     }
>     echo '</table>';
> ?>
> 
> 
> 
> SERVER_SOFTWARE    nginx/0.8.53
> REMOTE_ADDR    10.214.3.250
> SERVER_ADDR    10.214.0.56
> HTTP_X_FORWARDED_FOR    10.214.0.47

Just a side note: HTTP_X_FORWARDED_FOR should be something like 
"200.226.123.253, 10.110.3.250" here according to what you've 
showed later, it's unclear why you see only one address here.

> When I set:
> 
>         location ~ \.php$ {
> 
>                 set_real_ip_from 10.214.0.0/22;
>                 #set_real_ip_from 10.110.0.0/22;
>                 real_ip_header X-Forwarded-For;
>                 fastcgi_pass 127.0.0.1:9000;
>                 fastcgi_index index.php;
>                 include fastcgi_params;
>                 fastcgi_param SCRIPT_FILENAME
> /net/nfs_parceiros_1/ftorres.com.br$fastcgi_script_name;
>         }
> 
> I get:
> 
> REMOTE_ADDR    10.110.3.250
> SERVER_ADDR    10.214.0.56
> HTTP_X_FORWARDED_FOR    10.214.0.48
> 
> 
> Oops, now I can see 10.110.0.0/22 (my front end network), but nothing
> about my x-forwarded-for, now I fixed it and add the FE network :
> 
>         location ~ \.php$ {
> 
>                 add_header X-Fw-For $proxy_add_x_forwarded_for;
>                 set_real_ip_from 10.214.0.0/22;
>                 set_real_ip_from 10.110.0.0/22;
>                 real_ip_header X-Forwarded-For;
>                 fastcgi_pass 127.0.0.1:9000;
>                 fastcgi_index index.php;
>                 include fastcgi_params;
>                 fastcgi_param SCRIPT_FILENAME
> /net/nfs_parceiros_1/ftorres.com.br$fastcgi_script_name;
>         }
> 
> 
> 
> Then, I get:
> 
> REMOTE_ADDR    10.110.3.250
> SERVER_ADDR    10.214.0.56
> HTTP_X_FORWARDED_FOR    10.214.0.46

Realip module only uses last address from X-Forwarded-For header, 
the one which was added by last (trusted) proxy.  It doesn't try 
to follow X-Forwarded-For chain until non-trusted address appears 
(though it probably should, at least with some configuration 
option).  That's why you see the same REMOTE_ADDR here as in 
previous test.

Simpliest solution would be to set X-Real-Ip header on your real 
frontend and use it instead.  Not sure if Varnish can do it 
though.

> Any of these configurations shown the X-forwarded-for at remote_addr
> field, and the x-fw-for Header its working:
> 
> HTTP/1.1 200 OK
> Server: nginx
> Date: Fri, 10 Jun 2011 23:27:19 GMT
> Content-Type: text/html
> Connection: keep-alive
> *X-Fw-For: 200.226.123.253, 10.110.3.250, 10.110.3.250*
> 
> 
> I fixed it adding the following configuration at fastcgi_param file:
> 
>                 set  $addr  $remote_addr;
>                 if ($proxy_add_x_forwarded_for ~
> "^(?:^|,)\s*(\d+\.\d+\.\d+\.\d+)\s*") {
>                         set  $addr  $1;
>                 }
> 
> fastcgi_param  HTTP_X_FORWARDED_FOR    $addr;

This is *not* correct and insecure.  You use first ip address from 
X-Forwarded-For header which may be easily set by malicious client 
(and likely will contain private ip addresses from non-malicious 
clients behind real corporate proxies and so on).

Maxim Dounin