Load balancing with least_conn strategy does round-robin

[emil.vikstrom]

Hi.

I want to use nginx as a load balancer for tcp streams and because I want to
share the load evenly over all machines I thought that least_conn would be
the right strategy to use in this case. But I have stumbled upon an problem
I don't really understand.

In my lab environment I have three machines running server software.

I start my testing establishing 12 connections to nginx and can see that
they are getting evenly across all hosts. 4 connections per host.
I then stop one of the machines and the client reestablishes the lost
connections and they are getting spread evenly to the remaining hosts. Now
we have 6 connections on 2 hosts.
I then start up the closed machine and proceed to close 2 connections each
on the servers that have been up all this time. The client tries to set up
new connections and this is where I notice that the load not being equally
shared anymore.
The results I get is host1: 6 connections, host2: 5 connections, host3: 1
connection.

As far as I can see it seems like nginx uses round-robin to balance the
connections and not setting up connections to the host with least
connections. Am I misinterpreting what least connections mean?

>From the documentation

Syntax:	least_conn;
Default:	—
Context:	upstream
Specifies that a group should use a load balancing method where a connection
is passed to the server with the least number of active connections, taking
into account weights of servers. If there are several such servers, they are
tried in turn using a weighted round-robin balancing method.

How I interpret this is that it will only use weighted round-robin in the
case where there are 2 or more servers with the same amount connections that
satisfy the condition of having the least number of connections, which
should not be the case in this where host3 should be the one with least
amount of connections.

I had the hypothesis that I used a shared memory zone in the upstream might
be the cause of the problem, but removing that only made the results even
more weird.

After removing the zone the initial step of setting up 12 connections got me
the following results
Host1: 6 connections, Host2: 5 connections, Host3: 1 connection.

I suspect that it is the core config that might be faulty but tweaking the
parameters don't seem to get me further to solve this.

My configuration

user nginx;
worker_processes 16;
error_log /opt/nginx/logs/errors.log;
worker_rlimit_nofile 10000;
events {
    worker_connections 2048;
    worker_aio_requests 8;
}

stream {
    upstream stream1 {
        least_conn;
        zone stream1 128k;
        server host3:19953;
        server host2:19953;
        server host1:19953;
    }
    server {
        listen 9953;
        proxy_pass stream1;
    }

    access_log off;
}

Posted at Nginx Forum: https://forum.nginx.org/read.php?2,283093,283093#msg-283093