[p2p-hackers] Re: scalability (was: p2p framework)

[Greg Bildson]

Your UDP based protocol sounds similar to our experimental GUESS protocol.
As such, I have always felt that a connectionless UDP protocol is more at
risk from abuse.  While the cost of delivering queries to many hosts is
higher, there is also less opportunity to monitor the activity of the query
source.   Given a network where there is a true cost and delay to finding
available TCP connections, this ways against a greedy client churning
connections as well.  Our experience of massive query traffic from greedy
clients in 2003 led to my being very suspicious of short term connections
for querying.

The other benefit that you can get from longer lived TCP connections is
index forwarding.  In the modern Gnutella network, QRP (keyword bloom
filtering) indexes are rolled up and propagated two hops from leaf to
ultrapeer to neighboring ultrapeers.  This allows for query routing on the
last hop(s) that a query may travel.  With a high outdegree and a low TTL,
large routing efficiencies are gained.  Queries are not delivered to
ultrapeers where keyword matches would not occur.  Out-of-band UDP results
still allow direct delivery of responses and delivery as needed as well.

Thanks
-greg

> -----Original Message-----
> From: p2p-hackers-bounces at zgp.org [mailto:p2p-hackers-bounces at zgp.org]On
> Behalf Of coderman
> Sent: Wednesday, November 30, 2005 4:39 PM
> To: Peer-to-peer development.
> Subject: Re: [p2p-hackers] Re: scalability (was: p2p framework)
>
>
> On 11/30/05, Daniel Stutzbach <agthorr at cs.uoregon.edu> wrote:
> > Instead of saying "Gnutella does not scale", I think it would be more
> > accurate to say "Gnutella does not perform the operation I am
> > interested in".
>
> perhaps the following are more accurate observations:
> - it is difficult to protect gnutella protocol against malicious /
> aggressive peers
> - it is difficult to conserve bandwidth consumption for large gnutella
> peer groups
>
> i implemented a weighted iterative discovery protocol a while back
> called 'alpine' that ordered destination peer lists used for resource
> discovery by the relative quality associated with a given peer derived
> from past interactions (past responses, usefulness of resources
> provided, etc).
>
> the transport level was a logical UDP connection based protocol that
> persisted between application instances and supported reconnecting
> when endpoints changed (NAT, etc).  this made scaling characteristics
> simple as bandwidth and memory were the limiting factors.  (i tested a
> 4.5million logical connection pool between two gigabit servers at
> OSDL)
>
> the reason i prefer an iterative unicast without forwarding is that it
> greatly improves the ability to optimize queries and avoid resource
> consumption attacks.  a peer performing a search is in direct control
> of the rate of discovery, can terminate early, and may order the
> search according to local criteria.  lack of forwarding makes it
> difficult for a peer to abuse the resources of another (in gnutella a
> well connected silent client can inject a large number of queries
> which propagate according to TTL / other metrics.  a single query
> packet consumes many times more peer resources than the originating
> sender expended.
>
> one major point of note with this approach is that it requires much
> larger peer groups since they must all be directly addressed.  i
> considered 1k to 10k peers a common size but expected individual nodes
> with bandwidth to be capable of 100k or more.
>
> transitive introduction was also weighted by peer quality with the
> expectation that high quality peers would know additional high quality
> peers that share similar interests / relevant resources.
>
> the list archives and archive.org have additional details if you are
> curious / bored.
>
> best regards,
> _______________________________________________
> p2p-hackers mailing list
> p2p-hackers at zgp.org
> http://zgp.org/mailman/listinfo/p2p-hackers
> _______________________________________________
> Here is a web page listing P2P Conferences:
> http://www.neurogrid.net/twiki/bin/view/Main/PeerToPeerConferences