[p2p-hackers] Error correcting codes to prevent failure in BitTorrent like systems?

[Arnaud Legout]

Hi,

Nick Johnson wrote:
> Systems like BitTorrent have a rather annoying failure mode - the last 
> 'seed' goes offline while there are still several 'peers' (without the 
> complete file) online. Attempts by the peers to reconstruct the 
> original file are rarely successful, as the chances of every single 
> block being present on one of the seeds are generally very low - it's 
> likely that at least one block is missing.
from my point of view this is pure myth.
I often see such claims that bittorrent suffers from last pieces 
problem; that if there is no seed, the torrent is dead; etc.
 From all the experiments I performed, the reality is very different.
Rarest first does a very good job at  replicating the rarest pieces in a 
torrent, so that  the probability to have a piece that is not
replicated at all is very low.

Of course, one can always build toys model that show problems in extreme 
cases. But, it is clear that BitTorrent is
not a one fit all solution, and BitTorrent is very successful for its 
targeted applications: large scale replication for medium to large files.

Outside this target, it makes sense to design other classes of 
applications. However, I am not convinced that error correcting code are the
solution. Such codes are terribly sexy, but when it comes to real 
applications, things are far less sexy.
It is hard to tune such codes as their relevance really comes from the 
context. If the context is a moving target, then the problem becomes 
very complex.
Consider the simple case of a improving reliability of a satellite link. 
It is not trivial at all to find the good tradeoff between reliability 
and overhead.
When it comes to a distributed system with heterogeneous clients, the 
problem is several order of magnitude more complex.

Regards,

Arnaud.