How to make Nginx work with distributed/multiple memcached servers?

[agentzh]

On Wed, Apr 20, 2011 at 10:09 AM, Andy <nginx-forum at nginx.us> wrote:
> agentzh Wrote:
> -------------------------------------------------------
>> > And if it's Django, how do i make sure that
>> given the same key, both
>> > Django and Nginx will hash to the same memcached
>> server?
>> >
>>
>> If you want your python app to access memcached as
>> well, the
>> recommended approach is to pass the nginx variable
>> holding the hashed
>> memcached upstream name back to your fastcgi app.
>>
>
> How do I pass the nginx variable holding the hashed memcached upstream?
> Which variable is that?
>

Here's a complete nginx.conf example for this:

   http {
        upstream A {
           server 10.32.110.5:11211;
        }
        upstream B {
           server 10.32.110.16:11211;
        }
        upstream C {
           server 10.32.110.27:11211;
        }

        upstream_list my_cluster A B C;

        server {
           location = /memc {
               internal;
               set $memc_key $query_string;
               set $memc_exptime 3600; # cache one hour

               # hashing the $memc_key to an upstream backend
               #  in the my_cluster upstream list, and set $backend:
               set_hashed_upstream $backend my_cluster $memc_key;

               # pass $backend to memc_pass:
               memc_pass $backend;
           }

           location /myapp {
                set $key 'my page key';  # you may want to construct a
dynamic cache key here...

                set_hashed_upstream $backend my_cluster $key;

                srcache_fetch GET /memc $key;
                srcache_store PUT /memc $key;

                fastcgi_param MEMC_BACKEND $backend;
                fastcgi_pass unix:/path/to/your/python/app.sock;
           }
        }
    }

Another way is to use Lua to pick up a memcached backend from your
memcached cluster, such that you can define your own hashing
algorithm, including your custom consistent hashing ones.

Here we use fastcgi_param to pass nginx variable $backend to your
fastcgi app such that your fastcgi app can read that from its
environment MEMC_BACKEND. Another approach is to append the $backend
value to the query string of the forwarded http request.

Cheers,
-agentzh