[p2p-hackers] The 'MetaWeb' and the Slashdot effect

Bert bert at akamail.com
Fri Nov 29 08:46:01 UTC 2002


Nick Johnson wrote:

> Hi folks,
>
> I was just thinking some random thoughts (you know the type), and a 
> fairly simple idea occurred to me for mitigation of Slashdot type 
> effects, increasing reliability and uptime, etc.
> The basic idea is this: Instead of attempting to fetch a page 
> directly, the user attempts to fetch it via a script. If the script 
> has a cached copy of the page in it's database (the script would 
> follow standard caching directives as a proxy would), it simply 
> returns the page. If not, it has a number of choices: [snip]

Below is a paper that explores this idea, though it's more a combination 
of this idea + BitTorrent  in that the peers who download are the ones 
doing the caching. The idea here is that clients requesting the content 
provide a special pragma field that indicates their willingness to serve 
that content to subsequent requestors. The server can then issue a 
redirect to any such peers instead of serving the content itself. *

The Case for Cooperative Networking*, Venkata N. Padmanabhan and 
Kunwadee Sripanidkulchai. IPTPS '02 
<http://www.cs.rice.edu/Conferences/IPTPS02/>.
http://detache.cmcl.cs.cmu.edu/~kunwadee/research/papers/coopnetiptps.pdf

I've also planned on adding a similar function to YouServ, as described 
in section 3.4 of  
http://www.almaden.ibm.com/cs/people/bayardo/userv/plugin/plugin.html. 
Here though we want to use DNS to do the "load balancing" (ala Akamai) 
instead of requiring the originating server to always either send 
redirects or serve the content itself.





More information about the P2p-hackers mailing list