[p2p-hackers] Tighter HTTP and P2P integration??
Charles Iliya Krempeaux
supercanadian at gmail.com
Wed Feb 15 00:38:25 UTC 2006
Hello,
I've been actually considering using a local HTTP server to implement
things. (Which is something you are doing too.) Although I thought
some extra "sugar" was warranted, to help with (user) usabiliy and
developer usability.
I skimmed through the stuff on dijjer.org, and I have a questions.
Does digger pay attention to HTTP headers? And act basically as a web
cache?
For example, here's some (but not all) "rules" that I was planning on
following.... If HTTP headers "say" that something should NOT be
cached, then it should NOT be on the P2P network. Stuff on the P2P
network should be removed after the date for the "Expires" header
passes? Things from non-HTTP GET methods should NOT be put onto the
P2P network? Etc.
Does dijjer follow these web caching rules?
See ya
On 2/14/06, Ian Clarke <ian at locut.us> wrote:
> You should take a look at Dijjer - http://dijjer.org/.
>
> Dijjer is a Java client that runs on your computer, and emulates a
> local HTTP server.
>
> You can request (almost) any file through Dijjer that is available on
> the WWW by requesting http://127.0.0.1:9115/http://the.url.of/the/
> file.mov . Unlike BitTorrent, files are downloaded from Dijjer
> starting at the beginning, and will be streamed to the browser over
> the local HTTP connection as they are downloaded. This allows people
> to do neat things like embed Quicktime movies in web browsers and
> have the video play as it is downloaded from the P2P network.
>
> This will appear to your web browser as if you are downloading the
> file from an ordinary HTTP server, but in fact, the file will be
> downloaded through a distributed P2P cache. The first time the file
> is requested it will be pulled into the cache, subsequent requests
> will come from other nodes in the P2P network.
>
> Other Dijjer features:
>
> * NAT hole-punching (see http://en.wikipedia.org/wiki/
> UDP_hole_punching)
> * RESTful design (see above)
> * Decentralized algorithm (based on small-world principle)
> * Two complimentary forms of integrity checking (http://dijjer.org/
> wiki/IntegrityChecking)
> * TCP friendly data transfer (http://dijjer.org/wiki/DataTransmission)
> * Neat internal network abstraction layer that has formed the basis
> for the next
> major release of Freenet (http://dijjer.org/wiki/MessagingLayer)
> * Free as in speech (GPL license)
>
> If this sounds like advocacy, that's because it is :-) Dijjer needs
> developers, the code is relatively clean, and relatively well
> documented, but right now I am basically the only active developer,
> and I haven't had much free time lately with the next version of
> Freenet, not to mention my day job.
>
> In particular, it needs work on deployment, there is no OSX installer
> right now, and Windows version relies on Excelsior Jet - an easy to
> install but pure Java version would be nice.
>
> If you are a Java programmer and looking for something interesting to
> do, check out the Development section of the site and join our
> mailing list.
>
> Ian.
>
> On 14 Feb 2006, at 00:35, Charles Iliya Krempeaux wrote:
>
> > Hello,
> >
> > (First, I should say that I'm writing this around midnight, so let me
> > first apologise for all the spelling and grammar mistakes I know I'll
> > probably make but won't notice until I read this tomorrow. But
> > anyways....)
> >
> > One thing that's been on my mind for a while is making "tighter
> > integration" between HTTP and P2P technologies. (I've finally been
> > motivated to write this e-mail since I need to put this to practice
> > now. Although I really wanted to write a longer and more detailed
> > blog post on this. And do more research first.... Maybe later.)
> >
> > The current methods that bring some integration between HTTP and P2P
> > technologies include the magnet-uri and BitTorrent (via BitTorrent's
> > "application/x-bittorrent" MIME type).
> >
> > However, one could get even tighter integration. (Here's a few
> > different ways that may work... I'd like to get people's opinions on
> > these proposals.)
> >
> > One possible avenue for getting even tighter integration is the
> > creation of something like a "conditional get" for P2P technologies.
> > (For those unfamiliar with the HTTP "conditional get", look for
> > information of the "If-None-Match" and "If-Modified-Since" HTTP
> > headers.)
> >
> > A "P2P conditional get" (sent by the client) would tell the server NOT
> > to send the requested document if there is an alternative way of
> > getting the document via a P2P technology supported by the client.
> > (The client would tell the server which P2P technologies it supports.)
> > (This "P2P conditional get" would work similar to the "If-None-Match"
> > and "If-Modified-Since" "conditional get" headers.).
> >
> > For example, it might be something like:
> >
> > X-If-No-Alternate: bittorrent, something-else, magnet
> >
> > First, this says that the client supports the protocols: "bittorrent",
> > "something-else", and "magnet". It also says it prefers "bittorrent"
> > first, then "something-else", and then "magnet".
> >
> > (Really, there's nothing that mandates that these protocols be for P2P
> > technologies. But that is the use case that is motivating this.)
> >
> > This could also include "q" factors. Perhaps something like:
> >
> > X-If-No-Alternate: protocol1; q=1.0, protocol2; q=0.7,
> > protocol3; q=0.5
> >
> > And there could be ways of including more information too. (Which
> > could be used to add protocol specific information.) For example:
> >
> > X-If-No-Alternate: protocol1; q=1.0; a=b;c="d", magnet;
> > q=0.5;urn-allowed="sha1 md5"
> >
> > Note, I'm NOT trying to propose the exact format here or the names
> > used. Just trying to illustrate what I'm talking about through
> > examples. (I've tried to follow similar formats as other HTTP headers
> > though.)
> >
> > The server might then respond with an "HTTP 302" (Found) and then
> > redirect the user to a P2P version using a URI for the P2P technology.
> > This "redirect" would be done via the HTTP "Location" header. For
> > example:
> >
> > HTTP/1.1 302 Found
> > Location: magnet:?xt=urn:sha1:WRCIRZV5ZO56CWMNHFV4FRGNPWPPDVKT
> >
> > or:
> >
> > HTTP/1.1 302 Found
> > Location: x-some-new-p2p-protocol:etc/etc/etc
> >
> > However, I'm not sure if "302" is the correct HTTP response code to
> > use. (Anyone have any suggestions?)
> >
> >
> > Another possible solution would be to use the "Accept" header (sent by
> > the client). Now, this would only be useful for P2P technologies that
> > are shunted into HTTP via a MIME type (like BitTorrent and the
> > "application/x-bittorrent" MIME type.). But for technologies which
> > have this, the client could list the P2P shunt MIME types first, in
> > the HTTP "Accept" header. For example:
> >
> > Accept: application/x-bittorrent, application/ogg
> >
> > However, with BitTorrent's case, this brings the problem of confusing
> > the torrent and files in the torrent. So perhaps a "accept extension"
> > is needed:
> >
> > Accept: application/x-bittorrent;
> > enclosure-content-type="application/ogg", application/ogg
> >
> > Although I still think this still conveys the wrong semantics. So
> > perhaps this really isn't a good solution.
> >
> >
> > Another alternative is to use the "Upgrade" header (from the client)
> > and the "HTTP 101" (Switching Protocols) (from the server). This would
> > let you switch to a P2P protocol. However, this seems to somewhat
> > imply that whatever protocol is "upgraded to" will have communication
> > going from between the same client and server. (At least that's my
> > feeling.)
> >
> >
> > Any Comments? Anyone have any suggestions or advice?
> >
> > Also, are there already any proposals or specs along these lines? (No
> > sense in reinventing the wheel.)
--
Charles Iliya Krempeaux, B.Sc.
charles @ reptile.ca
supercanadian @ gmail.com
developer weblog: http://ChangeLog.ca/
___________________________________________________________________________
Make Television http://maketelevision.com/
More information about the P2p-hackers
mailing list