Proposal: new caching backend for nginx
mdounin at mdounin.ru
Tue Jan 22 18:17:34 UTC 2013
On Tue, Jan 22, 2013 at 06:15:08PM +0200, Aliaksandr Valialkin wrote:
> Hi all,
> I'm the author of ybc - fast in-process key-value caching library -
> https://github.com/valyala/ybc . This library has the following features,
> which may be essential to nginx:
> - compact source code written in C;
> - optimized for speed. Avoids using syscalls, dynamic memory allocations
> and memory copying in hot paths;
> - has no external dependencies;
> - cross-platform design - all platform-specific code is located in one
> place - https://github.com/valyala/ybc/tree/master/platform . Currently
> only linux is supported, but other platforms can be easily added by
> implementing the corresponding platform-specific functions;
Cross-platform with only linux supported sounds cool. :)
> - designed for efficient caching of millions big objects on both SSDs and
> HDDs. Easily deals with huge caches exceeding available RAM by multiple
> orders of magnitude.
> - persists cached objects across application restarts.
> - provides functionality similar to proxy_cache_lock and
> proxy_cache_use_stale from
> http://nginx.org/en/docs/http/ngx_http_proxy_module.html out of the box.
> - unlike nginx's file-based cache, is free of additional "cache manager"
> - unlike nginx's file-based cache, never exceeds maximum cache size set at
> - unlike nginx's file-based cache, stores all cached data in two files -
> one file is used for index, while the other file is used for cached data.
> These files aren't vulnerable to fragmentation, since their sizes never
> change after the application start.
> Other ybc features can be found at
> https://github.com/valyala/ybc/blob/master/README .
> ybc API is located at https://github.com/valyala/ybc/blob/master/ybc.h .
> It would be great if somebody familiar with nginx internals, who is
> interested in faster and cleaner caching backend for nginx could help me
> substituting file-based backend with ybc.
I don't think that it will fit as a cache store for nginx. In
particular, with quick look through sources I don't see any
interface to store data with size not known in advance, which
happens often in HTTP world. Additionally, it looks like it
doesn't provide async disk IO support.
More information about the nginx-devel