Scaling nginx caching storage

[Amir Keshavarz]

Sorry for the confusion .
My problem is that i need to cache items as much as possible so even if one
node had the storage capacity to satisfy my needs it couldn't handle all
the requests and we can't afford multiple nginx nodes request to our main
server each time an item is requested on a different nginx node .

For that problem i have afew scenarios but they either have huge overhead
on our servers and our network or are not suitable for sensitive production
env because it causes weird problems ( sharing storage ) .

But at this point i'm beginning to think if it's even worth it . Should i
settle for having multiple nginx nodes requesting the same item to our
upstream server ?


On Sat, Sep 23, 2017 at 1:48 PM, Lucas Rolff <lucas at lucasrolff.com> wrote:

> > is there any way to share a cache directory between two nginx instances ?
>
> > If it can't be done what do you think is the best way to go when we need
> to scale the nginx caching storage ?
>
>
>
> One is about using same storage for two nginx instances, the other one is
> scaling the nginx cache storage.
>
> I believe it’s two different things.
>
>
>
> There’s nothing that prevents you from having two nginx instances reading
> from the same cache storage – however you will get into scenarios where if
> you try to write from both machines (Let’s say it tries to cache the same
> file on both nginx instances), you might have some issues.
>
>
>
> Why exactly would you need two instances to share the same storage?
>
> And what scale do you mean by scaling the nginx caching storage?
>
>
>
> Currently there’s really only a limit to your disk size and the size of
> your keys_zone – if you have 50 terabytes of storage, just set the
> keys_zone size to be big enough to contain the amount of files you wanna
> manage (you can store about 8000 files per 1 megabyte).
>
>
>
>
>
>
>
> *From: *nginx <nginx-bounces at nginx.org> on behalf of Amir Keshavarz <
> amirkekh at gmail.com>
> *Reply-To: *"nginx at nginx.org" <nginx at nginx.org>
> *Date: *Saturday, 23 September 2017 at 10.58
> *To: *"nginx at nginx.org" <nginx at nginx.org>
> *Subject: *Scaling nginx caching storage
>
>
>
> Hello,
>
> Since nginx stores some cache metadata in memory , is there any way to
> share a cache directory between two nginx instances ?
>
>
>
> If it can't be done what do you think is the best way to go when we need
> to scale the nginx caching storage ?
>
>
>
> Thanks
>
> _______________________________________________
> nginx mailing list
> nginx at nginx.org
> http://mailman.nginx.org/mailman/listinfo/nginx
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx/attachments/20170923/31cd9f4d/attachment-0001.html>