[p2p-hackers] [connellybarnes@yahoo.com: [i2p] Python filesharing Merkle hash tree magic]

[Jacek Sieka]

Hi,

Nice, just wanted to note that there are many links out there that use Tiger (which apparently just got included in the linux kernel, so someone trusts it a little bit at least) instead of SHA1 together with merkle trees (dubbed tth's often), just in case you want to benefit from existing link sites such as bitzi.com...on the other hand, yet another hash type is always fun =)

FYI 2; if anyone's interested, the DC++ source code contains a C++ templated merkle tree implementation that can be used to generate trees with any hash (oh, and there's probably a chunk checker in there as well, although it's probably ugly =) - if anyone's interested it could be scavenged, polished and made cooperative with some standard hash function library, currently it expects a (fairly simple) dc++ specific interface to the hash fcn...

regards
/Jacek 

Eugen Leitl wrote:
> ----- Forwarded message from "C. Barnes" <connellybarnes at yahoo.com> -----
> 
> Topic: P2P filesharing ala BitTorrent.
> 
> Presenting public domain module 'chunk.py':
> 
>   http://oregonstate.edu/~barnesc/temp/chunk.py
> 
> This module does some magic with Merkle hash trees.
> It allows one to publish a 20 byte SHA-1 digest for
> a file, and encode 'chunks' from the file with
> minimal overhead, such that any given chunk can be
> verified against the file's 20 byte digest.
> Cryptographic hashing is used to verify each chunk,
> so chunks can be transmitted in any order, and it is
> impossible to save corrupt data to disk if one has
> the correct file hash.
> 
> One could use this module in any file sharing
> application which divides files up into blocks.
> 
> For example, in a BitTorrent-like project it might
> be desirable to keep the .torrent files minimal in
> size.  Using this module, one needs to only publish
> a hash like DA39A3EE5E6B4B0D3255BFEF95601890AFD80709
> instead of a digest of all blocks in the file.
> 
> Pros:
>  - Smaller digest size.
>  - Can send chunks in any order.
> 
> Cons:
>  - Each 'chunk' contains checksumming information.
>    This overhead is less than 4% for files of size
>    less than 1e20 bytes.
> 
>  - Connelly Barnes