Achieving strong client -> upstrem_server affinity

Srirang Doddihal om.brahmana at
Thu Aug 30 18:28:02 UTC 2012


I am using Nginx 1.1.19 on Ubuntu 12.04 (LTS) server.

Nginx is used to load balance traffic between two instances of the
Punjab XMPP-BOSH server.

Below is the relevant part from my nginx configuration :

upstream chat_bosh {

server {
        location /http-bind {
                proxy_next_upstream off;
                proxy_pass http://chat_bosh;
                expires off;

I am using ip_hash to make sure that a client will always be served by
the same upstream server. This is essential.
I am using "proxy_next_upstream off;" to prevent a request being tried
on multiple upstream servers, because such requests will invariably
fail. I realize that this will cost me redundancy and fallback in case
a particular upstream server goes down, but that isn't useful in this
particular case. I plan to handle it separately via monitoring and

Anomalies :

1) Despite ip_hash being specified the request from a particular
client IP sometimes (close to 7% of requests) get routed to a second
upstream server
2) Despute proxy_next_upstream off; some requests (about 5%) are tried
over multiple upstream servers.

What could be causing these and how do I go about fixing these?

Here are two sets of log line captures which depict the above
mentioned problems. - upstrem_next_server on; (i.e. default value) - - upstrem_next_server off; (as specified above)

These log lines were created with this command : tail -F
/var/log/nginx/access.log | grep "POST /http-bind" | awk '{print $1
"|"  $3 "|" $8 "|" $14}'

$1 = $remote_addr
$2 = $upstream_addr
$3 = $msec
$4 = $status

To see the ip_hash anamoly search for "|404" and look at the adjacent
lines. The same $remote_addr will be forwarded to two different
upstream servers.
To see the upstream_next_server off; anamoly search for "HTTP" -
Because of my brittle awk statement the status is replace by the
string "HTTP/1.1" when upstream_addr has multiple addresses.

Srirang G Doddihal

The LIGHT shows the way.
The WISE see it.
The BRAVE walk it.
The PERSISTENT endure and complete it.

I want to do it all ALONE.

More information about the nginx mailing list