[p2p-hackers] The 'MetaWeb' and the Slashdot effect

[Bert]

Nick Johnson wrote:

> Hi folks,
>
> I was just thinking some random thoughts (you know the type), and a 
> fairly simple idea occurred to me for mitigation of Slashdot type 
> effects, increasing reliability and uptime, etc.
> The basic idea is this: Instead of attempting to fetch a page 
> directly, the user attempts to fetch it via a script. If the script 
> has a cached copy of the page in it's database (the script would 
> follow standard caching directives as a proxy would), it simply 
> returns the page. If not, it has a number of choices: [snip]

Below is a paper that explores this idea, though it's more a combination 
of this idea + BitTorrent  in that the peers who download are the ones 
doing the caching. The idea here is that clients requesting the content 
provide a special pragma field that indicates their willingness to serve 
that content to subsequent requestors. The server can then issue a 
redirect to any such peers instead of serving the content itself. *

The Case for Cooperative Networking*, Venkata N. Padmanabhan and 
Kunwadee Sripanidkulchai. IPTPS '02 
<http://www.cs.rice.edu/Conferences/IPTPS02/>.
http://detache.cmcl.cs.cmu.edu/~kunwadee/research/papers/coopnetiptps.pdf

I've also planned on adding a similar function to YouServ, as described 
in section 3.4 of  
http://www.almaden.ibm.com/cs/people/bayardo/userv/plugin/plugin.html. 
Here though we want to use DNS to do the "load balancing" (ala Akamai) 
instead of requiring the originating server to always either send 
redirects or serve the content itself.