Proposal: new caching backend for nginx

[Andrew Alexeev]

On Jan 23, 2013, at 10:19 PM, Maxim Dounin wrote:

> Hello!
> 
> On Wed, Jan 23, 2013 at 07:27:42PM +0200, Aliaksandr Valialkin wrote:
> 
>> On Wed, Jan 23, 2013 at 5:47 AM, Maxim Dounin <mdounin at mdounin.ru> wrote:
>> 
>> 
>>> I don't think that it will fit as a cache store for nginx.  In
>>> particular, with quick look through sources I don't see any
>>> interface to store data with size not known in advance, which
>>> happens often in HTTP world.
>> 
>> 
>> Yes, ybc doesn't allow storing data with size not known in advance due to
>> performance and architectural reasons. There are several workarounds for
>> this problem:
>> - objects with unknown sizes may be streamed into a temporary location
>> before storing them into ybc.
> 
> This will effectively double work needed to store response, and 
> writing to cache is already one of major performance bottlenecks 
> on many setups.
> 
>> - objects with unknown sizes may be cached in ybc using fixed-sized chunks,
>> except for the last chunk, which may have smaller size. Here is a
> 
> Thus effectively reinventing what is known as blocks in the 
> filsystem world, doing all the maintanance work by hand.
> 
> Actually, this is the basic problem with phk@'s aproach (with all 
> respect to phk@): reinventing the filesystem.

I personally don't think reinventing a filesystem is always bad :) Generic filesystems might not be that suitable for high load (caching/streaming) scenarions.

A better question might be - what filesystem architecture/concept fits best. One option would be to bypass VM layer completely and work with raw block devices, right?

> [...]
> 
>>> Additionally, it looks like it
>>> doesn't provide async disk IO support.
>> 
>> Ybc works with memory mapped files. It doesn't use disk I/O directly. Disk
>> I/O may be triggered if the given memory page is missing in RAM. It's
>> possible to determine whether the given virtual memory location is cached
>> in RAM or not - OSes provide special syscalls designed for this case - for
>> example, mincore(2) in linux. But I think it's better relying on caching
>> mechanisms provided by OS for memory mapped files than using such syscalls
>> directly. Ybc may block nginx worker when reading swapped out memory pages,
>> but this should be rare event if frequently accessed cached objects fit RAM.
> 
> The key words are "if ... cached objects fit RAM".
> 
> But bad things happen if they aren't, and that's why support for 
> AIO was introduced in nginx 0.8.11 several years ago.  
> Surprisingly enough, it just works for cache without any special 
> code - because it's just files.
> 
> (Well, not exactly, there is special code to handle async reading 
> of a response header.  But it's rather addition to normal AIO 
> code.)
> 
> And another major problem with mmap is that it doesn't tolerate IO 
> errors, and if e.g. disk is unable to read a particular block - 
> you'll end up with SIGBUS to the whole process leaving it no 
> possibilities to recover instead of just an error returned for a 
> single read() operation.  While this may be acceptable in many use 
> cases, it is really bad for an event-based server with thousands 
> of clients served within a single process.
> 
>> Also as I understood from the http://www.aosabook.org/en/nginx.html , nginx
>> currently may block on disk I/O too:
>> 
>>> One major problem that the developers of nginx will be solving in
>>> upcoming versions is how to avoid most of the blocking on disk I/O.
>>> At the moment, if there's not enough
>>> storage performance to serve disk operations generated by a particular
>>> worker, that
>>> worker may still block on reading/writing from disk.

As the author of that chapter I can't help mentioning that this is an excerpt :)

Next phrase is actually:

"A number of mechanisms and configuration file directives exist to mitigate such disk I/O blocking scenarios. Most notably, combinations of options like sendfile and AIO typically produce a lot of headroom for disk performance. An nginx installation should be planned based on the data set, the amount of memory available for nginx, and the underlying storage architecture."

> Yep, not all OSes have AIO support (and some, like Linux, 
> require you to choose just one option, VM cache or AIO), and even on 
> systems with good one (FreeBSD, notably) there are still 
> operations like open() and stat(), which doesn't have async 
> counterparts at all, but still may block.  That's why we are 
> experimenting with various ways to make things better.  But it's 
> all about improving async IO support, not about dropping it.
> 
> -- 
> Maxim Dounin
> http://nginx.com/support.html
> 
> _______________________________________________
> nginx-devel mailing list
> nginx-devel at nginx.org
> http://mailman.nginx.org/mailman/listinfo/nginx-devel