[p2p-hackers] Hybrid Static/Dynamic DHT Systems

[Lemon Obrien]

when i designed my hashing system i gave up trying to understand all the academic stuff. i don't get it....i don't even know why you need hashing in peer to peer systems. I used it for search and db entries to make it supper fast; cause somparing strings would be slow; the only real requirements are 1) speed (fast hashing), 2) wide distribution (little collision) and 3) all software running on my netowrk use the same function (not hard since you need software to search). I then found one on the internet; and now just use that.

Cortland Klein <me at pixelcort.com> wrote:
This design proposal aims to describe a DHT system which has both a 
functional mutable hash table and a replicating static content hash table.

The problem I see with traditional DHT systems is that they can not 
easilly replicate data without making it much more difficult to maintain 
up-to-date state. In addition, it becomes impossible to provide an 
immutable reference to particular data.

A Hybrid DHT System would have two subsystems, the traditional dynamic 
mutable hash table and the added replicating static content hash table.

The static content hash table is a simple table where the keys are the 
SHA-1 of the content and the content is a length prefixed amount of data 
limited to 256KB. When a key is requested, it's value is automatically 
cached among the path of nodes to the node requesting the data. Nodes 
maintain their own caches and replace old keys with new keys as they 
pass through. Old keys are chosen based not only on their last request 
time, but also how close the key is to the node's ID. If the node is 
'responsible' for a key, it remains in the cache indefinitely. Thus, we 
can achieve replication while ensuring at least one copy is always on 
the network. Data larger than 256KB is split up into chunks and a 
splitfile containing a list of the static content keys for each chunk is 
used to represent the data.

The dynamic mutable hash table provides for each key a revisioned list 
of pointers to static content keys. Thus, each dynamic key's content has 
a table with datestamps and static content keys. To get the current 
value for a dynamic key, the system then does a lookup for the static 
content key. The node responsible for the key maintains this revisioned 
list and updates it when a put is made on the key. It can then expire 
old revisions after both a set number of revisions and at least a 
certain amount of time have passed.

My reasoning behind having two tables is that dynamic data cannot 
easilly be replicated due to its mutability, however it can point to 
static data which can be replicated. By having the static data retain a 
'responsible' node, we can ensure that at least one copy of the static 
data is available.

Any thoughts?

-- 
Cortland Klein (XMPP/SMTP)
nuWeb Project Founder - http://nuweb.org/ "Infinite Scalability"
Journal - http://pixelcort.com/ "Ranting and Raving"
The geeks shall take over the world and the bits shall be set free!
_______________________________________________
p2p-hackers mailing list
p2p-hackers at zgp.org
http://zgp.org/mailman/listinfo/p2p-hackers
_______________________________________________
Here is a web page listing P2P Conferences:
http://www.neurogrid.net/twiki/bin/view/Main/PeerToPeerConferences



You don't get no juice unless you squeeze
Lemon Obrien, the Third.
		
---------------------------------
Do you Yahoo!?
 Make Yahoo! your home page   
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://zgp.org/pipermail/p2p-hackers/attachments/20050414/a165290e/attachment.htm