lots of connections on TIME_WAIT state
mdounin at mdounin.ru
Wed Apr 8 19:41:11 MSD 2009
On Wed, Apr 08, 2009 at 05:56:51PM +0300, Anıl Çetin wrote:
> So, what is the solution? I have exactly the same problem, my nginx is
> in a virtual server (openvz), working as a proxy server in front of
> apache and oftenly (after 2k-3k requests) server becomes "out of
> sockets" even I raise the allowed numbers of sockets to a very big
You probably "out of ports", not out of sockets. Solution is to
configure TIME_WAIT reusing (tw_reuse, tw_recyle or something like
depending on your OS). You may also allow your system to use more
ports for outgoing connections.
Under FreeBSD reusing of TIME_WAIT sockets is the default, and
portrange for outgoing connections may be tuned via
net.inet.ip.portrange.hifirst and net.inet.ip.portrange.hilast
Not sure about Linux, but Google suggests reusing of TIME_WAIT
sockets may be turned on via /proc/sys/net/ipv4/tcp_tw_recycle.
> Igor Sysoev yazmış:
>> On Wed, Apr 08, 2009 at 10:47:16AM +0300, Artis Caune wrote:
>>> 2009/4/7 Deepan Chakravarthy <codeshepherd at gmail.com>:
>>>> I am using nginx with fast-cgi . When I run
>>>> $netstat -np | grep 127.0.0.1:9000
>>>> I find lot of connections in TIME_WAIT state. Is this because of high
>>>> keepalive_timeout value ? When lot of people use (5 requests per second)
>>>> nginx takes more time to respond. System load goes more than 10 during
>>>> peak hours.
>>> This is because of how TCP works.
>>>> debian:~# netstat -np | grep 127.0.0.1:9000
>>>> tcp 0 0 127.0.0.1:9000 127.0.0.1:45603
>>>> TIME_WAIT -
>>>> tcp 0 0 127.0.0.1:9000 127.0.0.1:45601
>>>> TIME_WAIT -
>>> If you were on FreeBSD, you could disable TIME_WAIT on loopback
>>> completely by setting:
>>> sysctl net.inet.tcp.nolocaltimewait=1
>> Due to the incorrect implementation this remedy is worse than the disease.
>> The net.inet.tcp.nolocaltimewait relys on unlimited RST delivery, therefore
>> if there are too many RSTs, they will be limited by net.inet.icmp.icmplim
>> and you will have a lot of sockets in the LAST_ACK state on server side
>> instead of lot of sockets in the TIME_WAIT on client side.
More information about the nginx