[p2p-hackers] Where do bright minds discuss p2p technology?

[Emin Gun Sirer]

David Barrett writes:
>> I am not sure what the Britney-problem is exactly (rather, I think the
>> Britney problem is the sheer crappiness of her music, but you seem to be
>> referring to something far more technical and possibly fixable), but 
>> three things struck me about your note:
>
>Heh, the problem you cite is probably the more severe one, but my 
>impression is the "Britney problem" is also the "hotspot" problem -- how 
>do I avoid overloading any single node in my DHT when a given keyword 
>becomes incredibly popular.

Ah, that's an interesting problem. The only way I know to deal with
hotspots is to dedicate more resources to the query terms causing the
hotspot. The difficult question is then "how many nodes should I
dedicate to serving an item that is ranked k in popularity? How many
copies of popular items should I make and propagate throughout my
network ?" This is exactly the question that Beehive tackles. The
quick insight is, if you make no copies of an item, the home node
bears all the load. If you copy a hot item to all N nodes in the
network, load is reduced by a factor of N, and performance improves
drastically since the object is available at the first node you
consult.  Unfortunately, the system would not scale if you tried to
copy all the items to all the nodes. So somewhere in the middle is the
optimal level of replication, at which the load placed by an item on
each node carrying its replicas will be equal to the load placed by
all other items on the nodes carrying their replicas, modulo some slop
due to discretization, and subject to space and bandwidth constraints
on the participating nodes. The Beehive result shows how to compute
this optimal level of replication. The derivation is difficult to do
in email, but the final formula is pretty succinct, and it doesn't
take much more than a few days of coding to layer Beehive onto a DHT
like Pastry, Chord, etc.

Wish that was the only problem with Britney,
- egs