[p2p-hackers] Error correcting codes to prevent failure in BitTorrent like systems?

Nick Johnson arachnid at notdot.net
Mon Dec 5 03:28:03 UTC 2005


Systems like BitTorrent have a rather annoying failure mode - the last 
'seed' goes offline while there are still several 'peers' (without the 
complete file) online. Attempts by the peers to reconstruct the original 
file are rarely successful, as the chances of every single block being 
present on one of the seeds are generally very low - it's likely that at 
least one block is missing.

However, what if one were to precode files to be distributed using a 
standard error correcting code such as a reed-solomon code? By 
generating 10% check blocks, and treating the composite file the same as 
you would the original (with the exception that you can stop downloading 
when you reach 90%, and reconstruct using check blocks from there), you 
can reduce the chance of the last departing seed ensuring nobody can 
complete the file.

If we assume there are 4 peers left on the network, each with 50% of the 
file remaining, on average they will be able to reconstruct 50% + 25% + 
12.5% + 6.25% = 93.75% of the file, which exceeds the threshold required 
to reconstruct with check blocks.

So, a couple of questions:
1) How common is this failure mode? Does it occur often enough to 
justify the extra complexity?
2) Do peers generally have enough pieces between them to reach or exceed 
the 90% threshold?

-Nick Johnson




More information about the P2p-hackers mailing list