[p2p-hackers] Re: scalability

[Adam Fisk]

I always thought alpine was an interesting protocol, and it was one of a 
mix of protocols and papers that went into the thinking behind GUESS and 
Gnutella's current indexing/walking approach.  I agree particularly on 
the protection against malicious attacks -- you have to devote 
significant resources to get other nodes to give up resources.  DHTs and 
index/walking are far more susceptible to DDOS attacks.

I was always amazed you were able to make so many connections, although 
I know the eDonkey guys maintain incredibly high numbers of connections 
as well.  That's actually a fascinating finding in light of the paper 
someone mentioned earlier on discovering the optimal number of 
connections to maintain in DHTs -- you might maintain more if you have 
the right connection scheme.  I like the approach of just keeping as 
many connections as you can.  4.5 million?  Wow.  Connections on 
anywhere near that scale (I'd take 100,000 any day) combined with 
indexing sounds like a great approach to me.  Indexing 2 hops away would 
go even further, and just don't use TTLs at all (you'd just be indexing 
it all and searching your local database).  Then again, I guess indexing 
10,000,000,000 nodes on each machine might be, well, optimistic.  
Somewhere in there would make the system scale better while also solving 
your DDOS problem, though.

What ever became of Alpine?

-Adam


coderman wrote:

>On 11/30/05, Daniel Stutzbach <agthorr at cs.uoregon.edu> wrote:
>  
>
>>Instead of saying "Gnutella does not scale", I think it would be more
>>accurate to say "Gnutella does not perform the operation I am
>>interested in".
>>    
>>
>
>perhaps the following are more accurate observations:
>- it is difficult to protect gnutella protocol against malicious /
>aggressive peers
>- it is difficult to conserve bandwidth consumption for large gnutella
>peer groups
>
>i implemented a weighted iterative discovery protocol a while back
>called 'alpine' that ordered destination peer lists used for resource
>discovery by the relative quality associated with a given peer derived
>from past interactions (past responses, usefulness of resources
>provided, etc).
>
>the transport level was a logical UDP connection based protocol that
>persisted between application instances and supported reconnecting
>when endpoints changed (NAT, etc).  this made scaling characteristics
>simple as bandwidth and memory were the limiting factors.  (i tested a
>4.5million logical connection pool between two gigabit servers at
>OSDL)
>
>the reason i prefer an iterative unicast without forwarding is that it
>greatly improves the ability to optimize queries and avoid resource
>consumption attacks.  a peer performing a search is in direct control
>of the rate of discovery, can terminate early, and may order the
>search according to local criteria.  lack of forwarding makes it
>difficult for a peer to abuse the resources of another (in gnutella a
>well connected silent client can inject a large number of queries
>which propagate according to TTL / other metrics.  a single query
>packet consumes many times more peer resources than the originating
>sender expended.
>
>one major point of note with this approach is that it requires much
>larger peer groups since they must all be directly addressed.  i
>considered 1k to 10k peers a common size but expected individual nodes
>with bandwidth to be capable of 100k or more.
>
>transitive introduction was also weighted by peer quality with the
>expectation that high quality peers would know additional high quality
>peers that share similar interests / relevant resources.
>
>the list archives and archive.org have additional details if you are
>curious / bored.
>
>best regards,
>_______________________________________________
>p2p-hackers mailing list
>p2p-hackers at zgp.org
>http://zgp.org/mailman/listinfo/p2p-hackers
>_______________________________________________
>Here is a web page listing P2P Conferences:
>http://www.neurogrid.net/twiki/bin/view/Main/PeerToPeerConferences
>
>  
>