How to solve the "out of ports" problem?TIME_WAIT reusing seems not work

[mowaken]

I'm using nginx as reverse proxy, and find more than 30k TIME_WAIT state
ports in upstream server. I know my servers are "out of ports" which
discussed here(http://nginx.org/pipermail/nginx/2009-April/011255.html),
and set both nginx and upstream server to reuse TIME_WAIT and  to
recycle more quickly.

[sysctl -p]
……
net.ipv4.ip_local_port_range = 1024 65000
net.ipv4.tcp_tw_recycle = 1
net.ipv4.tcp_tw_reuse = 1

But nginx hangs and "connection timed out while connection to upstream
server" error still can be found on nginx error log, when RPS of
upstream is higher than 1000 within 1 minutes. When upstream is Windows,
server will be "out of ports" in seconds.

Any ideas? A connection pool with a waiting queue? Maxim Dounin wrote a
useful module to keep connection with memcached, but why can't it
support Web Server?

Posted at Nginx Forum: http://forum.nginx.org/read.php?2,181493,181493#msg-181493