using Perl Mod to populate Memcached on the fly

[Marcus Clyne]

When I tested using memcached vs files on the filesystem, there wasn't 
much difference between them, and sometimes using memcached fared worse 
than directly sending the files from the filesystem (this would probably 
vary quite a bit, depending on the system).

The reason for this is that the overhead of creating a request to a 
memcached server, sending the request, waiting for the response (which 
is done asynchronously so as not to block the server), then handling the 
response is 'relatively' high.

If the memory cache was instead integrated into Nginx, rather than using 
an external process like memcached, then it would almost certainly be 
quite a bit quicker.  As far as I know, there isn't an internal 
memory-cache module for Nginx yet (I'm not sure if ncache has included 
memory caching yet, or if they intend on doing so).

As Atif says, there is little benefit to using memcache for serving 
static files, but there definitely is for dynamically-generated files.

As for Perl, I believe the current implementation is still blocking 
(i.e. you can't currently generate Perl in the background whilst still 
serving static files, you can either do one or the other). This would 
kill the performance terribly.

For populating the cache, have you looked at the error_file directive.  
You can set the error file to script which could be passed to an FCGI 
process, so that on a cache-miss the script is called to generate the 
page, which could in the process put the file into the memcached cache.  
This of course only really makes sense for dynamically-generate content.

Marcus.
 

Atif Ghaffar wrote:
> Daniel,
>
> I see now your other posts,
>
> I do not think that you will get any benifit of using memcached with 
> static files.
> Nginx is already very optimized at serving static files.
>
>
> On Wed, Mar 4, 2009 at 11:17 PM, Daniel Rhoden <drhoden at iiwinc.com 
> <mailto:drhoden at iiwinc.com>> wrote:
>
>     This is purely speculative, so please don't think I know how to do
>     this.  I'm throwing this out so hopefully, if a good idea, the
>     right people can create the example.
>
>     Memcached has a good assortment of Perl clients.
>
>     Nginx has a means of embedding Perl into the configuration.
>
>     Couldn't there be a way of combining these to immediately populate
>     the cache when the cache returns missing?
>
>     By the way, I'm looking at this as a means of improving I/O for
>     static pages (on SliceHost).  Ideally nginx's Memcached Module
>     would have the ability to do this when the requested filename
>     exists on the hard drive:
>     (-f $request_filename/index.html)
>
>
>     Daniel Rhoden
>     Director of Information Technology
>     Interactive Internet Websites, Inc.
>     3854 - 2 Killearn Court
>     Tallahassee, Florida 32309
>     Voice: (256) 878-5554
>     E-Mail: drhoden at iiwinc.com <mailto:drhoden at iiwinc.com>
>     Website: iiwinc.com <http://iiwinc.com>
>
>
>
>
>
> -- 
> best regards
> Atif Ghaffar

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://nginx.org/pipermail/nginx/attachments/20090305/a87ccbb5/attachment.html>