[p2p-hackers] Error correcting codes to prevent failure in
BitTorrent like systems?
Nick Johnson
arachnid at notdot.net
Mon Dec 5 03:28:03 UTC 2005
Systems like BitTorrent have a rather annoying failure mode - the last
'seed' goes offline while there are still several 'peers' (without the
complete file) online. Attempts by the peers to reconstruct the original
file are rarely successful, as the chances of every single block being
present on one of the seeds are generally very low - it's likely that at
least one block is missing.
However, what if one were to precode files to be distributed using a
standard error correcting code such as a reed-solomon code? By
generating 10% check blocks, and treating the composite file the same as
you would the original (with the exception that you can stop downloading
when you reach 90%, and reconstruct using check blocks from there), you
can reduce the chance of the last departing seed ensuring nobody can
complete the file.
If we assume there are 4 peers left on the network, each with 50% of the
file remaining, on average they will be able to reconstruct 50% + 25% +
12.5% + 6.25% = 93.75% of the file, which exceeds the threshold required
to reconstruct with check blocks.
So, a couple of questions:
1) How common is this failure mode? Does it occur often enough to
justify the extra complexity?
2) Do peers generally have enough pieces between them to reach or exceed
the 90% threshold?
-Nick Johnson
More information about the P2p-hackers
mailing list