Weird timeouts, not sure if I've set the right threshholds

[Igor Sysoev]

On Sat, May 03, 2008 at 11:14:14AM +1000, Rob Mueller wrote:

> 
> >Can anyone explain the prejudice against NFS?
> 
> NFS *can* cause blocking problems.
> 
> I haven't done a detailed analysis of how nginx is serving files, but I've 
> seen NFS flakiness cause massive problems because lots of processes trying 
> to access the NFS share end up "locking up" waiting on the NFS server to 
> respond. Because the lock occurs inside the kernel (eg when the process is 
> doing a read() call, or accessing an mmaped() region of a file), the 
> processes are almost completely uninterruptable.
> 
> With nginx, this would be even worse. nginx uses a small process count + 
> non-blocking event loop model for serving files. If something is causes 
> that loop to "block" (eg. waiting on the NFS server), nginx will basically 
> freeze up and stop serving files completely. In theory, using epoll() and 
> sendfile() should push that blocking down into the kernel which shouldn't 
> affect nginx, but as I said, I haven't done a detailed analysis. Even doing 
> things like stat() which can't be made non-blocking on an NFS mounted file 
> can block badly causing an entire nginx process to freeze.

You are right completely except epoll()/sendfile().

> Some things then to check.
> 
> How does nginx handle file IO?
> 
> If you're not using sendfile(), does nginx use read() with O_NONBLOCK? Does 
> the linux kernel block a read() call on an NFS file if the NFS server is 
> having problems even if you're using O_NONBLOCK?

Yes, kernel blocks.

> If you're using sendfile() on an NFS file, does the linux kernel block the 
> call if there's a problem accessing the NFS server?

Yes, kernel blocks.

> Does nginx use stat() calls to verify a file/path exists? Does the linux 
> kernel block a stat() call on an NFS file if the NFS server is having 
> problems?

Yes, kernel blocks.


-- 
Igor Sysoev
http://sysoev.ru/en/