using Perl Mod to populate Memcached on the fly

[Atif Ghaffar]

Marcus,

I agree with you. It was a very strange scenario.


On Thu, Mar 5, 2009 at 8:09 PM, Marcus Clyne <eugaia at gmail.com> wrote:

>  Atif,
>
> I understood it the same way you did, and was saying that serving files
> from memcached as opposed to the filesystem has negligible benefits, and
> sometimes actually performs worse.
>
> The reference to error_page (not error_file as I said before - memory slip)
> was just as a means to populate memcache when there's no data in the cache -
> you could easily use this method to put the file into memcache from the
> filesystem.  You wouldn't need to use an Nginx module to put the file into
> memcache (at least not with the current Perl module - it's blocking, so
> would slow everything down too much, though with a non-blocking version it
> could be ok) - though I understand that that's what he's talking about and
> where he's talking about hooking it in.
>
> Another method would be to use the try_files directive.
>
> Overall, though, I can only see populating memcache with a file to serve
> through Nginx as being overall slower than serving the file from the
> filesystem.  In my tests, memcache vs filesystem is pretty similar, and
> adding any kind of overhead to put a file in memcache will mean just serving
> the file statically will be more efficient (and won't waste memory
> unnecessarily).  I could see storing static files in an internal memory
> cache being a bit quicker than serving files statically, but actually not
> all that much, and would only possibly be of real benefit for files that are
> under very high demand (e.g. a homepage or a logo on a high-traffic site).
>
> Cheers,
>
> Marcus.
>
>
> Given that the
>
>
> Atif Ghaffar wrote:
>
>
>
> On Thu, Mar 5, 2009 at 12:34 PM, Marcus Clyne <eugaia at gmail.com> wrote:
>
>>
>> For populating the cache, have you looked at the error_file directive.
>> You can set the error file to script which could be passed to an FCGI
>> process, so that on a cache-miss the script is called to generate the page,
>> which could in the process put the file into the memcached cache.
>>
>
> Marcus,
>
> I believe Daniel  was talking about the opposite. He want to populate the
> cache if nginx hits the file instead of when it does not find it.
>
> So something like this (this is how nginx would do it
>
> request comes from /files/1.txt
> 1. Check if memcache exists and serve from there.
> 2. Check if file exist and serve from here.  <---- This is where he wants
> to hook it :-)
> 3. If file does not exists, handle error.
>
> What Daniel wants (if I understood correctly)
> request comes from /files/1.txt
> 1. Check if memcache exists and serve from there.
> 2. if(-f file ) call the perl/fcgi process that populates the memcache
> cache and returns the file. (so this would be done only where there is no
> cache in memcache)
> 3. If file does not exists, handle error.
>
>
> Still I dont see whats the point of this but yes it is doable.
>
> best regards
>
>
>
>
>
>
>> This of course only really makes sense for dynamically-generate content.
>>
>> Marcus.
>>
>>
>> Atif Ghaffar wrote:
>>
>> Daniel,
>>
>> I see now your other posts,
>>
>> I do not think that you will get any benifit of using memcached with
>> static files.
>> Nginx is already very optimized at serving static files.
>>
>>
>> On Wed, Mar 4, 2009 at 11:17 PM, Daniel Rhoden <drhoden at iiwinc.com>wrote:
>>
>>> This is purely speculative, so please don't think I know how to do this.
>>>  I'm throwing this out so hopefully, if a good idea, the right people can
>>> create the example.
>>>
>>> Memcached has a good assortment of Perl clients.
>>>
>>> Nginx has a means of embedding Perl into the configuration.
>>>
>>> Couldn't there be a way of combining these to immediately populate the
>>> cache when the cache returns missing?
>>>
>>> By the way, I'm looking at this as a means of improving I/O for static
>>> pages (on SliceHost).  Ideally nginx's Memcached Module would have the
>>> ability to do this when the requested filename exists on the hard drive:
>>> (-f $request_filename/index.html)
>>>
>>>
>>> Daniel Rhoden
>>> Director of Information Technology
>>> Interactive Internet Websites, Inc.
>>> 3854 - 2 Killearn Court
>>> Tallahassee, Florida 32309
>>> Voice: (256) 878-5554
>>> E-Mail: drhoden at iiwinc.com
>>> Website: iiwinc.com
>>>
>>>
>>>
>>
>>
>> --
>> best regards
>> Atif Ghaffar
>>
>>
>>
>
>
> --
> best regards
> Atif Ghaffar
>
>
>


-- 
best regards
Atif Ghaffar
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://nginx.org/pipermail/nginx/attachments/20090305/74b547da/attachment.html>