ybc (was: nginx-devel Digest, Vol 39, Issue 27)

[Igor Sysoev]

On Jan 23, 2013, at 21:40 , Aliaksandr Valialkin wrote:

> On Wed, Jan 23, 2013 at 2:00 PM, Igor Sysoev <igor at sysoev.ru> wrote:
> > - unlike nginx's file-based cache, stores all cached data in two files - one file is used for index, while the other file is used for cached data. These files aren't vulnerable to fragmentation, since their sizes never change after the application start.
> 
> Well it eliminates the cache file fragmentation on file system, but how do you eliminate
> cached objects fragmentation inside the cache file?
> 
> Cached objects fragmentation is eliminated by the following mechanisms build into ybc:
> - cached object data always occupies a single contiguous memory region.
> - cached objects are stored in a circular log. So newly objects may only overwrite the oldest objects stored in the log. This way the log cannot contain holes (except for stale, rolled back or overwritten objects).

These holes mean that the algorithm has sources of the fragmentation.

> - frequently requested objects with sizes smaller than the given threshold are periodically moved into the log's head. This guarantees that small hot objects will be always packed into the smallest possible memory region located near the log's head. Big objects with sizes exceeding multiple memory pages aren't moved, because there is no much sense in their defragmentation.


What will happen with hot ~100K size objects? Will they be eventually overwritten with some new objects, probably
requested only once?


--
Igor Sysoev
http://nginx.com/support.html

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx-devel/attachments/20130124/d4d2471a/attachment.html>