[p2p-hackers] Tighter HTTP and P2P integration??

[Charles Iliya Krempeaux]

Hello Ian,

I was thinking about how to integrate digger with my "P2P conditional get"
idea.  The steps when making certain kinds of requests for a URL (like for
RSS/Atom enclosures) would roughtly be llike the followig.  (Although
configuration options would allow you to change this.)


   1. Look in "cache" for URL.  If in there, then get "ETag" or
   "Last-Modified" value.
   2. Make an HTTP request on the server. Send a "If-None-Match" or
   "If-Modified-Since" "conditional get" if found "ETag" or "Last-Modified"
   from previous step.  Also, send a "X-If-No-Link" for P2P protocols we
   understand.
   3. If the server sends back a "304 Not Modified" then we have the up
   to date file.  (So we're done.  Else....)  If the server sends back a "204
   No Content" then get the "Link" headers (the server returned) and shunt over
   to the "preferred" P2P network.  Else get dijjer to get it.


Step #1 is where I wanted to look into dijjer's cache.  (Step #2 is in there
so that the publisher can provide "sanctioned" ways of getting the file off
of P2P networks.)


P.S.
I don't think that the mailing list is sending all the messages/replies I've
sent.  Although the messages are in the mailing list archive, mailman -- the
mailing list software -- sent 3 messages to me says it wasn't able to send
(3) messages.


See ya


On 2/17/06, Ian Clarke <ian at locut.us> wrote:
>
> No, Dijjer peers don't cache entire files, rather they just cache 256k
> blocks of files, and these are stored in a single binary file.  Typically
> any given Dijjer peer would probably only be caching one block from any
> given file.
>
> Can you elaborate on why you might want to do this?
>
> Ian.
>
> On 17 Feb 2006, at 00:09, Charles Iliya Krempeaux wrote:
>
> Hello Ian,
>
> Is there a way to get at dijjer's cached files besides going through it's
> local HTTP server?  (I.e., with direct file I/O.)
>
>
> See ya
>
> On 2/14/06, Ian Clarke <ian at locut.us> wrote:
> >
> > On 14 Feb 2006, at 16:38, Charles Iliya Krempeaux wrote:
> > > I've been actually considering using a local HTTP server to implement
> > > things.  (Which is something you are doing too.)  Although I thought
> > > some extra "sugar" was warranted, to help with (user) usabiliy and
> > > developer usability.
> >
> > We have added some sugar too.  For example, we have javascript
> > available on our site that can detect whether Dijjer is present on
> > the user's machine and redirect the user to a download page if it isn't.
> >
> >
> > > I skimmed through the stuff on dijjer.org, and I have a questions.
> > > Does digger pay attention to HTTP headers?  And act basically as a web
> > > cache?
> >
> > Yes, we have attempted to stick as closely to the HTTP protocol as
> > possible.
> >
> > > For example, here's some (but not all) "rules" that I was planning on
> > > following.... If HTTP headers "say" that something should NOT be
> > > cached, then it should NOT be on the P2P network.
> >
> > Yes, we obey the various HTTP no-cache headers.  We consider it
> > important for Dijjer to act as much like a HTTP cache as possible so
> > that we are afforded the protection the DMCA gives to caches for
> > copyright infringement.
> >
> > >   Stuff on the P2P
> > > network should be removed after the date for the "Expires" header
> > > passes?
> >
> > We don't obey "Expires" explicitly, but since Dijjer pings the
> > original web server each-time something is downloaded to ensure that
> > the file is still there and hasn't changed, if it is removed from the
> > web server, it will effectively be removed from Dijjer at the same
> > instant.
> >
> > > Things from non-HTTP GET methods should NOT be put onto the
> > > P2P network?  Etc.
> >
> > We attempt to be smart about our interaction with HTTP, if there are
> > shortcomings in this regard they will be easy to fix.
> >
> >

--
    Charles Iliya Krempeaux, B.Sc.

    charles @ reptile.ca
    supercanadian @ gmail.com

    developer weblog: http://ChangeLog.ca/
___________________________________________________________________________
Make Television                                http://maketelevision.com/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://zgp.org/pipermail/p2p-hackers/attachments/20060217/ecb844bb/attachment.htm