Serving files from a slow NFS storage

[Steve Holdoway]

On Tue, 2014-12-30 at 02:34 -0500, erankor2 wrote:
> Thank you all for your replies.
> 
> Since all 3 replies suggest some form of caching I'll respond to them
> together here - 
> The nginx servers that I mentioned in my post do not serve client requests
> directly, the clients always hit the CDN first (we use mostly Akamai), and
> the CDN then pulls from these nginx servers. In other words, these servers
> act as the CDN origin. Therefore, hot / popular content is already taken
> care of - I have no problem there. 
> Since the files we serve are large (video) the CDN isn't caching them for
> too long (we send caching header of 3 months, and the files usually get
> cached for a couple of days), so the servers are getting quite a few
> requests, and these requests hardly repeat themselves. Each server is
> delivering roughly 1/2TB of data per day, so to get any hits on an NFS cache
> we'll probably need a very large cache. And even if do that, we'll still
> have this problem with the non-popular content (e.g. videos that are watched
> on average once a week) - such a request may hang the process if opening the
> file takes a long time. 
> 
> Thanks,
> 
> Eran

I'm a bit confused here. Are you saying that the CDN is pulling from
NFS? If so, then surely the solution is under your control... deliver
all this content from a single server. If the web servers never deliver
it, then mount this content via NFS on them so they know it exists, but
no more.

Steve


-- 
Steve Holdoway BSc(Hons) MIITP
http://www.greengecko.co.nz
Linkedin: http://www.linkedin.com/in/steveholdoway
Skype: sholdowa