Reverse Proxy with 500k connections

[Maxim Konovalov]

This is just a matter of number of ip addresses you have in a
proxy_bind pool and suitable hash function for the split_clients map.

Adding additional logic to proxy_bind ip address selection you still
can face the same problem.

On 3/8/17 9:45 PM, Tolga Ceylan wrote:
> is IP_BIND_ADDRESS_NO_PORT the best solution for OP's case? Unlike the
> blog post with two backends, OP's case has one backend server. If any
> of the hash slots exceed the 65K port limit, there's no chance to
> recover. Despite having enough port capacity, the client will receive
> an error if the client ip/port hashed to a full slot.
> 
> IMHO picking bind IP based on a client ip/port hash is not very robust
> in this case since
> you can't really make sure you really are directing %10 of the
> traffic. This solution does
> not consider long connections (web sockets) and the hash slot could
> get out of balance
> over time.
> 
> 
> On Wed, Mar 8, 2017 at 3:20 AM, Maxim Konovalov <maxim at nginx.com> wrote:
>> On 3/7/17 10:50 PM, larsg wrote:
>>> Hi,
>>>
>>> we are operating native nginx 1.8.1 on RHEL as a reverse proxy.
>>> The nginx routes requests to a backend server that can be reached from the
>>> proxy via a single internal IP address.
>>> We have to support a large number of concurrent websocket connections - say
>>> 100k to 500k.
>>>
>>> As we don't want to increase the number of proxy instances (with different
>>> IPs) and we cannot use the "proxy_bind transarent" option (was introduced in
>>> a later nginx release, upgrade is not possible) we wanted to configure the
>>> nginx to use different source IPs then routing to the backend. Thus, we want
>>> nginx to select an available source ip + source port when a connection is
>>> established with the backend.
>>>
>>> For that we assigned ten internal IPs to the proxy server and used the
>>> proxy_bind directive bound to 0.0.0.0.
>>> But this approach seems not to work. The nginx instance seems always use the
>>> first IP as source IP.
>>> Using multiple proxy_bind's is not possible.
>>>
>>> So my question is: How can I configure nginx to select from a pool of source
>>> IPs? Or generally: to overcome the 64k problem?
>>>
>> We ever wrote a blog post for you!
>>
>> https://www.nginx.com/blog/overcoming-ephemeral-port-exhaustion-nginx-plus/
>>
>> As a side note: I'd really encourage all of you to add our blog rss
>> to your feeds. While there is some marketing "noise" we are still
>> trying to make it useful for tech people too.
>>
>> --
>> Maxim Konovalov
>> _______________________________________________
>> nginx mailing list
>> nginx at nginx.org
>> http://mailman.nginx.org/mailman/listinfo/nginx
> _______________________________________________
> nginx mailing list
> nginx at nginx.org
> http://mailman.nginx.org/mailman/listinfo/nginx
> 


-- 
Maxim Konovalov