DHT upstream module + nginx patches

[W-Mark Kubacki]

Am 9. Oktober 2009 10:46 schrieb Peter Schüller <scode at spotify.com>:
> [...]
>
> I am attaching two things; a module (spdht) which implements DHT based
> routing of requests to multiple upstream servers, and a patchset for
> nginx itself (against 0.7.61) that are needed, in part, in order to
> use the module.

Thank you for sharing! At least, I am interested.

> [...] In addition the DHT ring is configured through DNS. [...]

In a project of mine in academia this turned out to be a very good
approach - if you cached DNS responses in the configuration reader,
though.

>
>  * Add support for SHA1 in the caching module. [...]

No need to use cryptographic hashes in such an application. They're
slower than those I will mention below and you don't really need the
"from a given output bit you cannot judge to an input bit". It is
sufficient that every output bit will be toggled at a probability of
~50%, thus of a almost perfect dispersion.

For faster hashing try Murmurhash64, FNV1A and friends:
[1] http://murmurhash.googlepages.com/
[2] http://isthe.com/chongo/tech/comp/fnv/
[3] http://burtleburtle.net/bob/hash/doobs.html (see the very bottom
of that page)

BTW, replacing MD5, SHA1 etc. by one of those you can accelerate a lot
of Key/Value storages out there.

-- 
W-Mark Kubacki
http://mark.ossdl.de/