[p2p-hackers] First use of FEC in P2P Systems?

[Justin Chapweske]

On Tue, 2004-03-09 at 11:14, Jim McCoy wrote:
> On Mar 9, 2004, at 2:03 AM, Justin Chapweske wrote:
> 
> > Well it depends on your definition of P2P, but the first usage of FEC 
> > in
> > a modern "P2P" system that I am aware of was when I invented swarming
> > downloads.
> 
> Actually, Digital Fountain and the work of the reliable multicast group 
> predated you by quite a while.
> 

I never claimed otherwise.  If you re-read my message I specifically
refer to Luigi Rizzo's (reliable multicast) work as being the
inspiration for Swarmcast, which is certainly evidenced by the fact that
our FEC library is a direct port of his from C to Java.  However, the
reliable multicast systems are not typically thought of as P2P and
certainly do not feature any kind of swarming.  

As I said, if one were to extend the reliable multicast work to include
application layer/P2P multicast, then that would certainly be considered
P2P.  However, from what I've seen of the early application layer
multicast systems such as Yoid, they did not at that time contemplate
the use of FEC-style reliable multicast.

> While Shamir may have contemplated such a use I am not aware of him 
> publishing anything about it.  The oldest precedents for the 
> application of error-correction to increase data retrieval reliability 
> probably go back to the RAID work initially done at Berkeley and if you 
> expand the scope slightly to what most of us would consider "p2p" then 
> the earliest such system is probably Zebra, although this was more of a 
> "traditional" RAID network filesystem designed as a distributed set of 
> storage agents.  The earliest system that most of us would look at and 
> call "p2p storage" I think would be the Archival Intermemory research 
> from NEC Research in 97 (and Frangipani if you are thinking more of a 
> replicated data layer that error-correction could be applied to...)

I had forgotten about Archival Intermemory, and seeing as that was in
'97 as well, I'm guessing the synchronicity of the early uses of
software-based FEC was simply due to CPU power reaching a sweet spot to
enable these applications.  I'd guess that Luigi's paper, "On the
Feasibility of Software FEC", also played a large role in raising of the
awareness of researchers to the possibilities of software-based FEC.

As an aside, Hakim Weatherspoon's paper on "Erasure Coding vs.
Replication: A Quantitative Comparison"
(http://oceanstore.cs.berkeley.edu/publications/papers/pdf/erasure_iptps.pdf) is a good read for anyone wanting to gain a greater appreciation for the performance trade offs when using FEC.

Thanks,

-Justin