upstream tomcatcluster { ip_hash; server test1:8080 max_fails=0; server test2:8080 max_fails=0; } It seems that the size of ip_hash is too small and it can't hold enough keys - or perhaps nginx clears out keys after a period of time? I'm curious what the limitations of ip_hash are because it's not working 100% of the time. Thank you.