Serving files from a slow NFS storage
nginx-forum at nginx.us
Tue Dec 30 07:34:45 UTC 2014
Thank you all for your replies.
Since all 3 replies suggest some form of caching I'll respond to them
together here -
The nginx servers that I mentioned in my post do not serve client requests
directly, the clients always hit the CDN first (we use mostly Akamai), and
the CDN then pulls from these nginx servers. In other words, these servers
act as the CDN origin. Therefore, hot / popular content is already taken
care of - I have no problem there.
Since the files we serve are large (video) the CDN isn't caching them for
too long (we send caching header of 3 months, and the files usually get
cached for a couple of days), so the servers are getting quite a few
requests, and these requests hardly repeat themselves. Each server is
delivering roughly 1/2TB of data per day, so to get any hits on an NFS cache
we'll probably need a very large cache. And even if do that, we'll still
have this problem with the non-popular content (e.g. videos that are watched
on average once a week) - such a request may hang the process if opening the
file takes a long time.
Posted at Nginx Forum: http://forum.nginx.org/read.php?2,255847,255868#msg-255868
More information about the nginx