Reverse Proxy with 500k connections

[Andrei Belov]

Yes, split_clients solution fits perfectly in the described use case.

Also, nginx >= 1.11.4 has support for IP_BIND_ADDRESS_NO_PORT socket
option ([1], [2]) on supported systems (Linux kernel >= 4.2, glibc >= 2.23) which
may be helpful as well.

Quote from [1]:

[..]
Add IP_BIND_ADDRESS_NO_PORT to overcome bind(0) limitations: When an
application needs to force a source IP on an active TCP socket it has to use
bind(IP, port=x). As most applications do not want to deal with already used
ports, x is often set to 0, meaning the kernel is in charge to find an
available port. But kernel does not know yet if this socket is going to be a
listener or be connected. This patch adds a new SOL_IP socket option, asking
kernel to ignore the 0 port provided by application in bind(IP, port=0) and
only remember the given IP address. The port will be automatically chosen at
connect() time, in a way that allows sharing a source port as long as the
4-tuples are unique.
[..]


[1] https://kernelnewbies.org/Linux_4.2#head-8ccffc90738ffcb0c20caa96bae6799694b8ba3a
[2] https://git.kernel.org/torvalds/c/90c337da1524863838658078ec34241f45d8394d


> On 08 Mar 2017, at 01:10, Tolga Ceylan <tolga.ceylan at gmail.com> wrote:
> 
> How about using
> 
>    split_clients "${remote_addr}AAA" $proxy_ip {
>                   10%               192.168.1.10;
>                   10%               192.168.1.11;
>                   ...
>                   *                     192.168.1.19;
>    }
> 
>    proxy_bind $proxy_ip;
> 
> where $proxy_ip is populated via split clients module to spread the
> traffic to 10 internal IPs.
> 
> or add 10 new listener ports (or ips) to your backend server instead,
> (and perhaps use least connected load balancing) in upstream {} set of
> 10 backends. eg:
> 
> upstream backend {
>  least_conn;
>  server 192.168.1.21:443;
>  server 192.168.1.21:444;
>  server 192.168.1.21:445;
>  server 192.168.1.21:446;
>  server 192.168.1.21:447;
>  server 192.168.1.21:448;
>  server 192.168.1.21:449;
>  server 192.168.1.21:450;
>  server 192.168.1.21:451;
>  server 192.168.1.21:452;
> }
> 
> 
> 
> 
> On Tue, Mar 7, 2017 at 1:21 PM, Rainer Duffner <rainer at ultra-secure.de> wrote:
>> 
>> Am 07.03.2017 um 22:12 schrieb Nelson Marcos <nelsonmarcos at gmail.com>:
>> 
>> Do you really need to use different source ips or it's a solution that you
>> picked?
>> 
>> Also, is it a option to set the keepalive option in your upstream configure
>> section?
>> http://nginx.org/en/docs/http/ngx_http_upstream_module.html#keepalive
>> 
>> 
>> 
>> 
>> I’m not sure if you can proxy web socket connections like http-connections.
>> 
>> After all, they are persistent (hence the large number of connections).
>> 
>> Why can’t you (OP) do the upgrade to 1.10? I thought it’s the only
>> „supported" version anyway?
>> 
>> 
>> 
>> 
>> 
>> _______________________________________________
>> nginx mailing list
>> nginx at nginx.org
>> http://mailman.nginx.org/mailman/listinfo/nginx
> _______________________________________________
> nginx mailing list
> nginx at nginx.org
> http://mailman.nginx.org/mailman/listinfo/nginx