nginx 0.7.59 open_file_cache and NFS

[Igor Sysoev]

On Wed, Jun 10, 2009 at 01:06:32PM +0200, andan andan wrote:

> 2009/6/10 Igor Sysoev <is at rambler-co.ru>:
> > On Wed, Jun 10, 2009 at 12:24:02PM +0200, andan andan wrote:
> >
> >> 2009/6/10 Igor Sysoev <is at rambler-co.ru>:
> >> > On Wed, Jun 10, 2009 at 11:49:00AM +0200, andan andan wrote:
> >> >
> >> >> We have two nginx serving static objects from a shared NFS partition,
> >> >> and we are using open_file_cache for performance purposes:
> >> >>
> >> >> With nginx 0.6.x, when the developers uploads files, during a few
> >> >> seconds (less than 30s) we see this errors in logs:
> >> >>
> >> >> [alert] 2721#0: *75303 sendfile() failed (116: Stale NFS file handle)
> >> >> while sending response to client, client: XXX ....
> >> >>
> >> >> I understand that is a normal behaviour because we are using a open_file_cache.
> >> >>
> >> >> But, when we upgrade to nginx 0.7.59 the same errors appears
> >> >> indefinitely (and of course we are sending corrupted objects or
> >> >> blank), it seems that, open_file_cache_valid is been ignored in new
> >> >> version.
> >> >>
> >> >> Any idea, tip or recomendation ?
> >> >>
> >> >> In other hand, is recomended the use of open_file_cache, until now, we
> >> >> are very happy with it.
> >> >
> >> > What OS do you using on NFS client and server side ?
> >>
> >> Client: RHEL5
> >> 2.6.18-128.1.10.el5 x86_64
> >>
> >> Server: NAS EMC Celerra NS40
> >>
> >> > How files are uploaded - simple cp or
> >> >
> >> > cp file file.temp
> >> > rm file
> >> > mv file.temp file
> >>
> >> Normally, they uploads files via scp, but editing a file by hand the
> >> errors appears on the same way.
> >
> > This is because it's no atomic method.
> > On local filesystem you may get mixed content from old and new versions
> > without any errors.
> 
> mmm, then, is recommended to disable open_file_cache in this scenario ?

Even if you will disable open_file_cache, the race condition still remains.
open_file_cache simply makes it distinct: 30s vs 20ms and so.

> I'm curious about the question that with nginx 0.6.x in a few seconds
> the file is "ready" again, but no with 0.7.x

Can not say right now.


-- 
Igor Sysoev
http://sysoev.ru/en/