totally transparent proxying with nginx on openbsd

[Maxim Dounin]

Hello!

On Fri, Oct 29, 2010 at 12:14:48AM +1000, David Gwynne wrote:

> openbsd has a setsockopt option called SO_BINDANY that allows a
> process to bind to any ip address, even if it is not local to the
> system. the patch below uses it to allow nginx to connect to a
> backend server using the ip of the client making the request.
> 
> my main goal here is to allow the backend server to know the ip of
> the client actually making the request without having to look at
> extra hhtp headers
> 
> i thought id throw this out there to get some help since this is
> my first attempt at tweaking nginx. there are a few issues with
> this implementation:
> 
> 1. it is completely specific to openbsd.

You may want to look at IP_TRANSPARENT in Linux as well.  AFAIR 
somebody was working implementing IP_TRANSPARENT support in nginx, 
though I don't know details.

> 2. it needs root privileges to use the SO_BINDANY sockopt.

Some fine-grained access control in OS is required to allow 
SO_BINDANY for non-root processes.  Quick look suggests it's not 
currently possible in OpenBSD.  In Linux it's possible to 
allow usage of IP_TRANSPARENT for non-root processes via 
CAP_NET_ADMIN capability.

> 3. im not sure if connections to backends are cached. if so then
> it is probable that a different client will reuse a previous clients
> proxy connection, so it will appear that the same client made both
> requests to the backend.

Connections aren't cached in official nginx yet, though there are 
third party modules which allow connection caching (only in some 
limited cases though).

Obviously it's not possible to change source address once 
connection is established, so the only simple solution I see is to 
don't cache such connections.  This is probably good enough 
solution though.

> --- src/event/ngx_event_connect.c.orig	Thu Nov 26 04:03:59 2009
> +++ src/event/ngx_event_connect.c	Thu Oct 28 23:22:37 2010
> @@ -11,7 +11,7 @@
>  
>  
>  ngx_int_t
> -ngx_event_connect_peer(ngx_peer_connection_t *pc)
> +ngx_event_connect_peer(ngx_peer_connection_t *pc, ngx_connection_t *cc)

There is already pc->local which is used for bind().

The only thing that should be passed is some flag in 
ngx_peer_connection_t structure to trigger setsockopt(SO_BINDANY) 
before bind.

[...]

> +        if (setsockopt(s, SOL_SOCKET, SO_BINDANY,
> +                    &bindany, sizeof(bindany)) == -1)

This is obviously needs some configure tests and #ifdef's.

[...]

> +	if (bind(s, cc->sockaddr, cc->socklen) == -1)

As I already said - there is no need to duplication bind() code.

Additional question to consider is what you are going to do if 
client's connection protocol doesn't match upstream connection's 
one (e.g. ipv6 client vs ipv4 upstream).

Maxim Dounin