Weird timeouts, not sure if I've set the right threshholds
Denis S. Filimonov
den.lists at gmail.com
Sat May 3 06:36:09 MSD 2008
On Friday 02 May 2008 19:56:45 Cliff Wells wrote:
> On Fri, 2008-05-02 at 16:44 -0400, Denis S. Filimonov wrote:
> > Hi guys,
> > Can anyone explain the prejudice against NFS?
> I hadn't noticed any. It's more a question of using the wrong tool for
> this job. NFS is great if you need normal filesystem access for a
> workgroup. It's the wrong tool for serving web content.
That's exactly the prejudice I was talking about :-)
I've yet to see any proof of this statement while my experience suggests that
NFS suits serving web content very well (at least in some cases).
> > Specifically, why would additional proxy hop be faster than serving files
> > from NFS?
> Because I'm betting Nginx is faster than NFS.
> > I can see two points in favor of NFS:
> > - NFS client caches files while Nginx doesn't (yet)
> But you *must* use Nginx if you want your files available on the web (or
> more exactly, some HTTP server). Caching is almost wholly irrelevant if
> you go through a layer that doesn't cache. The only overhead being
> eliminated by NFS caching is that of NFS itself.
That's not true.
If I understand the proposed schemes correctly, they are
proxy <-(1)-> nginx/fcgi <-(2)-> NFS server
proxy <-(1)-> nginx <-(2)-> nginx/fcgi (over local fs)
Even though all traffic goes through link (1) anyway, we still want to reduce
the latency in the link (2). And I maintain that the scheme with NFS can do
better, partly because of caching.
> > - Nginx doesn't support keepalive connections to upstream, hence
> > additional latencies and traffic for TCP handshake/finalization. NFS
> > doesn't have this issue since it typically works over UDP.
> Except that you ignore all the other overhead NFS imposes at a different
> layer. NFS may use a fairly efficient network protocol, however it's
> also a fairly complete virtual filesystem, and this emulation imposes
> significant overhead that a simple HTTP server doesn't have.
The only significant overhead NFS imposes is due to default consistency
requirements which are excessive for most web applications. Relaxing those
requirements goes a long way in improving NFS performance but for some reason
it's usually neglected. Seriously, having disabled close-to-open consistency
(totally irrelevant to a typical web application) I have observed *multifold*
reduction of NFS packets and huge drop in latency.
> > I do have a couple boxes serving a lot of traffic (mostly PHP) from NFS.
> > It works just fine, though it did take some NFS tuning.
> I'm sure apps would run just fine off NFS, since they are typically
> read-once, run-many. Serving a few static files from NFS would probably
> be okay too, because the caching would help.
> Serving lots of static
> files over NFS is probably going to suck badly unless your NFS server
> has tons of RAM for cache and even then I doubt it will perform very
True, it can perform poorly in extreme circumstances but I'm sure a web server
would do even worse under the same conditions.
> At the end of the day, I suggest the OP *try* both methods and let the
> tests decide. In the time it took to write this reply the OP could have
> tested both cases and this discussion would have been moot.
Agreed. Hope he posts the results, I did my research on this subject years
ago. Much has changed, it'd be nice to have an update.
More information about the nginx