Serving files from a slow NFS storage

[Scott Larson]

     Without knowing everything in the mix my first thought would be the
NFS head node is being tapped out and can't keep up. Generally you'd solve
this with some type of caching, either at a CDN level or you could look at
the SlowFS module. I've not checked to see if it still compiles against the
current releases but if you're dealing with short-life hot data or a
consistent group of commonly accessed files, either solution would make a
significant impact in reducing NFS load without having to resort to other
potentially dodgy solutions.



*__________________Scott LarsonSystems AdministratorWiredrive/LA310 823
8238 ext. 1106310 943 2078 faxwww.wiredrive.com
<http://www.wiredrive.com/>www.twitter.com/wiredrive
<http://www.twitter.com/wiredrive>www.facebook.com/wiredrive
<http://www.wiredrive.com/facebook>*

On Mon, Dec 29, 2014 at 12:52 PM, erankor2 <nginx-forum at nginx.us> wrote:

> Hi all,
>
> In our production environment, we have several nginx servers running on
> Ubuntu that serve files from a very large (several PBs) NFS mounted
> storage.
>
> Usually the storage responds pretty fast, but occasionally it can be very
> slow. Since we are using aio, when a file read operation runs slow, it's
> not
> that bad - the specific HTTP request will just take a longer time to
> complete.
> However, we have seen cases in which the file open itself can take a long
> time to complete. Since the open is synchronous, when the open is slow all
> active requests on the same nginx worker process are delayed.
> One way to mitigate the problem may be to increase the number of nginx
> workers, to some number well above the number of CPU cores. This will make
> each worker handle less requests and therefore less requests will be
> delayed
> due to a slow open, but this solution sounds far from ideal.
> Another possibility (that requires some development) may be to create a
> thread that will perform the task of opening the file. The main thread will
> wait on the completion of the open-thread asynchronously, and will be
> available to handle other requests until the open completes. The child
> thread can either be created per request (many requests probably won't even
> need to open a file, thanks to the caching of open file handles), or
> alternatively some fancier thread pooling mechanism could be developed.
>
> I'd love to hear any thoughts / ideas on this subject
>
> Thanks,
>
> Eran
>
> Posted at Nginx Forum:
> http://forum.nginx.org/read.php?2,255847,255847#msg-255847
>
> _______________________________________________
> nginx mailing list
> nginx at nginx.org
> http://mailman.nginx.org/mailman/listinfo/nginx
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx/attachments/20141229/e178717c/attachment-0001.html>