[p2p-hackers] Overlay Weaver: An Overlay Construction Toolkit

shudo at computer.org shudo at computer.org
Mon Jan 23 02:49:12 UTC 2006

Hi Michael,

Thanks for your interest in Overlay Weaver.
There are mailing lists provided. Please consider joining it
(but I've not posted yet).

> It seems a great toolkit , .I think Overlay Weaver will greatly speed the developing of p2p applications.
> I have one question, is Overlay Weaver similar to the Jxta ? or it's more powerful than Jxta ?

JXTA is oriented to programming interface and protocol, and Overlay
Weaver (OW) is more seeds-oriented. The seeds are overlay-related
technologies. Structured overlays are more powerful than JXTA's
discovery mechanism, I believe.

JXTA does routed and fowarded messaging supported by relay peers to
get over NA(P)T. OW currently provides only UDP hole punching, which
works with lots of routers but not all ones.

Advertisement is a common idea to constructs of JXTA, peer, peer
group, pipe and so on. OW does not have such higher-level common
ideas. There are 2 higher-level services, DHT and Group Manager, and
they can work together in the same application and share a routing
service. But they are independent services and do not share an unified
idea like advertisement.

I have been an enthusiast for JXTA for years and support its ideal,
providing common protocol for P2P applications. But I've been also
disappointed by a few points, especially ad-hoc-ness and
scalability. As you may know, JXTA applications requires pre-deployed
super peers (rendezvous and relay peers). A normal (edge) peer can
turn into a super peer but we cannot rely on the transform mechanism
when deploying an application. It is not very ad-hoc when deployed in
a large-scale.

JXTA's discovery mechanism called SRDI is DHT-like but less scalable
in principle. There have been a strong requirement for JXTA J2SE to
support multiple discovery mechanism and JXTA core members have
asserted it's the case. But I have not been convinced.

JXTA project set a scalability target in a paper in 2002, in which a
super peer hosts about 1,000 normal (edge) peers and several hundreds
of super peers coordinates. But early in 2004, I found that a super
peer can host only about 2 dozens of normal peers in case of frequent
messaging. I guess that very limited people have tested JXTA with
dozens of computers.


> ----- Original Message ----- 
> From: <shudo at computer.org>
> To: <p2p-hackers at zgp.org>
> Sent: Monday, January 16, 2006 11:21 PM
> Subject: [p2p-hackers] Overlay Weaver: An Overlay Construction Toolkit
> > I'm pleased to announce the initial release of Overlay Weaver.
> >
> >   Overlay Weaver: An Overlay Construction Toolkit
> >   http://overlayweaver.sf.net/
> >
> > It supports overlay algorithm designers in addition to application
> > developers.
> >
> > For application developers, the toolkit provides a common API for
> > higher-level services such as distributed hashtable (DHT) and
> > multicast.  Applications relying on the common API depend no specific
> > transport protocol, database implementation and routing algorithm.
> >
> > The toolkit provides multiple routing algorithms, Chord, Kademlia,
> > Pastry and Tapestry. These algorithms could be implemented only in
> > hundreds lines of code because of routing layer decomposition. Routing
> > layer under the higher-level services has been decomposed into
> > multiple components, routing driver, routing algorithm and messaging
> > service. The decomposition also facilitates implementation of a new
> > algorithm. A newly implemented algorithm can be tested, evaluated and
> > compared on emulator, which can host thousands of virtual nodes It
> > enables large-scale emulation and fair comparison between algorithms.
> >
> > Features:
> >
> > - Implemented in Java 5.
> >   (except part of IPv4 multicast router which is in C.)
> >
> > - Provides multiple routing algorithms, Chord, Kademlia, Pastry and Tapestry.
> >
> > - Two routing drivers respectively performing iterative and recursive routing
> >   work with all routing algorithm (except recursive routing with Kademlia).
> >
> > - Provides a distributed environment emulator. It has demonstrated
> >   that it can host 4000 (virtual) nodes on a single 32 bit computer
> >   with 1 GB memory.
> >
> > - There are multiple implementations of communication layer,
> >   with UDP, TCP and emulated messaging layer.
> >   Note that the UDP implementation does UDP hole punching.
> >
> > - A visualization tool, Messaging Visualizer provided.
> >   It shows nodes and communications just in time
> >   and works both on the emulator and a real network.
> >
> > There are screenshots and a demonstration provided on the web site.
> > Please take a look.
> >
> > We have written a paper but now it's in Japanese. I will prepare an
> > English paper in few months.
> >
> > We'd appreciate activities utilizing this toolkit such as application
> > development, algorithm researches, testbed construction and operation.
> > We'll support them. Please contact us or subscribe a mailing list.
> >
> > Thanks,

  Kazuyuki Shudo, Ph.D.		shudo at ni.aist.go.jp, http://www.shudo.net/
  Grid Technology Research Center
  National Institute of Advanced Industrial Science and Technology (AIST)

More information about the P2p-hackers mailing list