[p2p-hackers] Generalizing BitTorrent..

Bryan Turner bryan.turner at pobox.com
Fri Jan 14 01:34:13 UTC 2005

Hello p2p-hackers,

    I've been thinking about BitTorrent recently and had some ideas on
improving the protocol.  I'm sure others have had similar thoughts, and I'm
interested to hear your opinions.

    The basic idea is best described with a real-world example.  There are a
number of "Full-MAME" torrents, one for each version of MAME (for those who
don't know, MAME is an arcade emulator and the torrents contain the ROMs for
the arcade games).  As MAME is updated frequently, there is a string of
these torrents on the net.  Each one is very large (10 GB), and contains 95%
of the EXACT SAME data from the previous torrent.  The other 5% is a small
amount of changes, and some new content.

    If a user is running the v0.7 torrent and has become a Seed, he serves
ONLY the v0.7 peers.  When v0.8 is released, his Seed status is essentially
useless to the peers in the v0.8 crowd, even though he has > 90% of the same
data.  And again, when v0.9 is released, the same problem.  It seems like
there should be an extension to the protocol to allow for this type of
'shared data' among torrents.

    Let me take this concept a bit further.  You could think of the v0.7,
v0.8, and v0.9 torrents as being three separate torrents - or you could
think of them as ONE torrent, with overlapping pieces.  If there were a
tracker that tracked the "meta-torrent" of all three versions, it would be
valuable to ALL peers interested in ANY of the versions.

    Taken to the extreme, if you were to gather up other (non MAME) torrents
in the same manner and glue them all together, the resulting meta-torrent
(and tracker) would be valuable to any client interested in any data tracked
by the meta-torrent tracker.

    Likewise, clients could exchange with other clients for any piece of
data in the meta-torrent shared between the clients.  For example, if Client
A is interested in Fedora 3, a some MP3s, and MAME v0.7, while Client B is
interested in Fedora 2, some MP3s, and MAME v0.8, then their shared
meta-torrent is all the data shared between Fedora 2 & 3, the
mutually-interesting MP3s, and the data shared between MAME v0.7 & v0.8.

    Conceptually, nothing has changed, we're just aggregating the set of
files being exchanged, and allowing torrents to overlap where ever it is
natural.  Many BitTorrent client applications already allow a similar
selection process, they allow the user to choose some subset of data to
transfer from a complete torrent.  The proposed changes are essentially the
same idea, except that the client software automatically selects the pieces
to download/ignore from the meta-torrent based on the sub-torrent files that
the user has selected.

    I have a few ideas on how to implement this as well, but I don't want to
waste bandwidth if the group isn't interested.

Thanks for reading!
bryan.turner at pobox.com

More information about the P2p-hackers mailing list