[p2p-hackers] Tighter HTTP and P2P integration??

Tue Feb 14 16:07:34 UTC 2006

Hello Charles,

A couple of weeks ago, there was some work done to integrate
both BitTorrent and Gnutella technologies with the Plone
CMS/Zope webserver.  http://plone.org/events/sprints/snow-sprint3

I've done some preliminary work (along with Matt Hamilton from NetSight)
using an Apache plugin to add the X-Alt and X-Node HTTP headers
that allow Gnutella clients downloading the same file to find each other
and take load off the server.

I haven't gone past proof-of-concept, but I have confirmed that two
LimeWire clients are able to find each other through an Apache
server using a plugin I helped write.  The two LimeWire clients
then share portions of the file with each other and thereby reduce
load on the server.

Note that Apache 2.2 on OS X 10.4 (Tiger) deals very poorly with a single
connection utilizing multiple 128 kb GET requests in random order.  We
got very high CPU useage on the PowerBook being used as the webserver,
and even got a kernel panic. We were forced to use the latest Apache 2.0,
which seemed to work fine.  (This was the weekend of Feb 4th, 2006.)

Good Luck,

Karl Magdsick
Software Engineer, Lime Wire LLC

Charles Iliya Krempeaux wrote:

>Hello,
>
>(First, I should say that I'm writing this around midnight, so let me
>first apologise for all the spelling and grammar mistakes I know I'll
>probably make but won't notice until I read this tomorrow.  But
>anyways....)
>
>One thing that's been on my mind for a while is making "tighter
>integration" between HTTP and P2P technologies.  (I've finally been
>motivated to write this e-mail since I need to put this to practice
>now.  Although I really wanted to write a longer and more detailed
>blog post on this.  And do more research first....  Maybe later.)
>
>The current methods that bring some integration between HTTP and P2P
>technologies include the magnet-uri and BitTorrent (via BitTorrent's
>"application/x-bittorrent" MIME type).
>
>However, one could get even tighter integration.  (Here's a few
>different ways that may work... I'd like to get people's opinions on
>these proposals.)
>
>One possible avenue for getting even tighter integration is the
>creation of something like a "conditional get" for P2P technologies. 
>(For those unfamiliar with the HTTP "conditional get", look for
>information of the "If-None-Match" and "If-Modified-Since" HTTP
>headers.)
>
>A "P2P conditional get" (sent by the client) would tell the server NOT
>to send the requested document if there is an alternative way of
>getting the document via a P2P technology supported by the client. 
>(The client would tell the server which P2P technologies it supports.)
> (This "P2P conditional get" would work similar to the "If-None-Match"
>and "If-Modified-Since" "conditional get" headers.).
>
>For example, it might be something like:
>
>    X-If-No-Alternate: bittorrent, something-else, magnet
>
>First, this says that the client supports the protocols: "bittorrent",
>"something-else", and "magnet".  It also says it prefers "bittorrent"
>first, then "something-else", and then "magnet".
>
>(Really, there's nothing that mandates that these protocols be for P2P
>technologies.  But that is the use case that is motivating this.)
>
>This could also include "q" factors.  Perhaps something like:
>
>    X-If-No-Alternate: protocol1; q=1.0, protocol2; q=0.7, protocol3; q=0.5
>
>And there could be ways of including more information too.  (Which
>could be used to add protocol specific information.)  For example:
>
>    X-If-No-Alternate: protocol1; q=1.0; a=b;c="d", magnet;
>q=0.5;urn-allowed="sha1 md5"
>
>Note, I'm NOT trying to propose the exact format here or the names
>used.  Just trying to illustrate what I'm talking about through
>examples.  (I've tried to follow similar formats as other HTTP headers
>though.)
>
>The server might then respond with an "HTTP 302" (Found) and then
>redirect the user to a P2P version using a URI for the P2P technology.
> This "redirect" would be done via the HTTP "Location" header.  For
>example:
>
>    HTTP/1.1 302 Found
>    Location: magnet:?xt=urn:sha1:WRCIRZV5ZO56CWMNHFV4FRGNPWPPDVKT
>
>or:
>
>    HTTP/1.1 302 Found
>    Location: x-some-new-p2p-protocol:etc/etc/etc
>
>However, I'm not sure if "302" is the correct HTTP response code to
>use.  (Anyone have any suggestions?)
>
>
>Another possible solution would be to use the "Accept" header (sent by
>the client).  Now, this would only be useful for P2P technologies that
>are shunted into HTTP via a MIME type (like BitTorrent and the
>"application/x-bittorrent" MIME type.).  But for technologies which
>have this, the client could list the P2P shunt MIME types first, in
>the HTTP "Accept" header.  For example:
>
>    Accept: application/x-bittorrent, application/ogg
>
>However, with BitTorrent's case, this brings the problem of confusing
>the torrent and files in the torrent.  So perhaps a "accept extension"
>is needed:
>
>    Accept: application/x-bittorrent;
>enclosure-content-type="application/ogg", application/ogg
>
>Although I still think this still conveys the wrong semantics.  So
>perhaps this really isn't a good solution.
>
>
>Another alternative is to use the "Upgrade" header (from the client)
>and the "HTTP 101" (Switching Protocols) (from the server). This would
>let you switch to a P2P protocol.  However, this seems to somewhat
>imply that whatever protocol is "upgraded to" will have communication
>going from between the same client and server.  (At least that's my
>feeling.)
>
>
>Any Comments?  Anyone have any suggestions or advice?
>
>Also, are there already any proposals or specs along these lines?  (No
>sense in reinventing the wheel.)
>
>
>TIA
>
>
>See ya
>
>--
>    Charles Iliya Krempeaux, B.Sc.
>
>    charles @ reptile.ca
>    supercanadian @ gmail.com
>
>    developer weblog: http://ChangeLog.ca/
>___________________________________________________________________________
> Make Television                                http://maketelevision.com/
>_______________________________________________
>p2p-hackers mailing list
>p2p-hackers at zgp.org
>http://zgp.org/mailman/listinfo/p2p-hackers
>_______________________________________________
>Here is a web page listing P2P Conferences:
>http://www.neurogrid.net/twiki/bin/view/Main/PeerToPeerConferences
>  
>