sudden nginx hang -- restart fails, "98: Address already in use"

[Maxim Dounin]

Hello!

On Thu, Feb 06, 2014 at 07:23:51AM -0500, dwirth wrote:

> Thanks. I am fairly certain (?) at this point that NFS is the culprit. I had
> a lot of trouble unmounting one of my NFS directories. Eventually I resorted
> to rebooting, at which point it went into a permanent hang until a reboot
> was forced via hypervisor.

Well, if you use NFS it perfectly explains observed behaviour.

> Is this particular situation, where NFS causes nginx to shut down, specific
> to nginx? We just switched from apache to nginx at the start of the year. I
> didn't have NFS problems before that. I don't know if that's coincidence or
> not.

Basic NFS problems are the same regardless of software you use: if 
something goes wrong, it blocks processes trying to access NFS 
share.  With nginx, results are usually a bit more severe than 
with process-based servers like Apache, because a) blocking an 
nginx worker process affects multiple requests, and b) blocking 
all nginx processes is easier as typically there are only small 
number of nginx worker processes.

> At any rate, my takeaway from this: nginx + NFS = bad.

I would take nginx out of the equation.

If you are going to use NFS it may be a good idea to make sure 
you've read and understood all the mount options NFS has.  In 
particular, it is believed that using the "soft" option and small 
timeouts may help a bit.  I wouldn't recommend using NFS at all 
though.

-- 
Maxim Dounin
http://nginx.org/