[p2p-hackers] network coding put to practical p2p content dist. use

[coderman]

On 6/20/05, Nick Johnson <arachnid at notdot.net> wrote:
> ...
> Trackers in BitTorrent don't deal with individual data blocks - only
> peer discovery.

i explained that very poorly, my apologies.  by centralized i meant
the metadata contained in the torrent, which consists of finite blocks
and their SHA1 digest (among other details) and the authoritative
introduction performed by the tracker(s) referenced in the torrent
metadata.

in this case peers can only exchange the exact blocks defined in the
torrent and will (currently) only discover peers through the
authoritative tracker(s).  there are ways to improve efficiency in
this situation dealing with coordination between peers and/or trackers
(for example, intelligent trackers that order peers in the list in
particular ways, the super seeder modes for rapid seeding of popular
torrents, etc) but these all deal with scheduling of the same static
blocks.

authenticated network codes would allow peers to send freshly coded
chunks of a current dataset without any predefined block offsets.  the
two advantages this could provide (assuming CPU/memory costs are not
prohibitive) are:

- introduction outside of a .torrent/tracker, since peers no longer
need to be exchanging the exact same fixed blocks, only the same data
(and yet still be able to trust a given network coded chunk)
- less complicated scheduling because of the way novel bits are
incorporated into subsequent re-encoded blocks, thus obviating the
need for some of the complicated scheduling used by trackers and/or
peers / superseeders.

this is the core difference i meant to reference in earlier posts. 
bittorrent as it currently works with distinct authenticated blocks in
metadata and tracker introduction will not benefit from network coding
or other online / error coding (over the internet), the efficiency
tweaks possible can be done on a blocking / scheduling basis.

network codes might be able to approach the same high efficiencies
without a fixed set of blocks and with less need for smarts in a
tracker or seeder peer.  which in turn makes them well suited for the
types of edge cases the authors cite as problematic for bittorrent
efficiency: large heterogeneous networks (tracker overload?), high
peer churn (block scheduling/tracker overload?), and flash crowds (the
superseeders).