From coderman at gmail.com Wed Mar 1 06:02:55 2006 From: coderman at gmail.com (coderman) Date: Sat Dec 9 22:13:10 2006 Subject: [p2p-hackers] Re: Identify "defective" nodes In-Reply-To: <7FE01058-EEAF-4602-B0C7-57B9E48B8984@vt.edu> References: <6.2.3.4.0.20060228105303.01f980a0@mail.cs.cornell.edu> <19196d860602280800n2674cc64s4a07565d5651fd4b@mail.gmail.com> <20060228161418.GN5704@cs.uoregon.edu> <7FE01058-EEAF-4602-B0C7-57B9E48B8984@vt.edu> Message-ID: <4ef5fec60602282202x4d5ff8fbh83c9d3c0ab7180a4@mail.gmail.com> On 2/28/06, H. Lally Singh wrote: > ... Hell you could go as far as a CORBA. noooooooo! :) [actually, CORBA isn't that bad; it's just well suited for large enterprise distributed systems and not a lot else] From lally at vt.edu Wed Mar 1 18:08:07 2006 From: lally at vt.edu (H. Lally Singh) Date: Sat Dec 9 22:13:10 2006 Subject: [p2p-hackers] Re: Identify "defective" nodes In-Reply-To: <4ef5fec60602282202x4d5ff8fbh83c9d3c0ab7180a4@mail.gmail.com> References: <6.2.3.4.0.20060228105303.01f980a0@mail.cs.cornell.edu> <19196d860602280800n2674cc64s4a07565d5651fd4b@mail.gmail.com> <20060228161418.GN5704@cs.uoregon.edu> <7FE01058-EEAF-4602-B0C7-57B9E48B8984@vt.edu> <4ef5fec60602282202x4d5ff8fbh83c9d3c0ab7180a4@mail.gmail.com> Message-ID: Haha, I didn't want to get into any kind of debate; just mention that you can get lightweight ORBs that I've successfully used in things as remote as embedded realtime. But yeah, you've gotta be real careful (and selective of your ORB) to keep it lightweight. -- When the Boogeyman goes to sleep every night he checks his closet for Chuck Norris. On Mar 1, 2006, at 1:02 AM, coderman wrote: > On 2/28/06, H. Lally Singh wrote: >> ... Hell you could go as far as a CORBA. > > noooooooo! > > :) > > [actually, CORBA isn't that bad; it's just well suited for large > enterprise distributed systems and not a lot else] > _______________________________________________ > p2p-hackers mailing list > p2p-hackers@zgp.org > http://zgp.org/mailman/listinfo/p2p-hackers > _______________________________________________ > Here is a web page listing P2P Conferences: > http://www.neurogrid.net/twiki/bin/view/Main/PeerToPeerConferences From philip_matthews at magma.ca Thu Mar 2 21:20:20 2006 From: philip_matthews at magma.ca (Philip Matthews) Date: Sat Dec 9 22:13:10 2006 Subject: [p2p-hackers] Number of pinholes supported by low-end NATs? Message-ID: <8C1DD971-379F-45F8-8169-8238C52D03F6@magma.ca> Does anyone have any idea how many pinholes a P2P application can have open at one time through a typical low-end NAT box? Could a P2P application maintain connections to 30 peers simultaneously? What about 50 peers? Or 100 peers? Do the numbers differ if the messages are carried over UDP vs TCP? (Note: My interest is in limitations in the NAT box, and NOT in any limitations in the Windows, Mac, Linux, ... box on which the P2P application is running.) Just wondering if anyone has any solid data in this area. - Philip From lemonobrien at yahoo.com Thu Mar 2 21:45:24 2006 From: lemonobrien at yahoo.com (Lemon Obrien) Date: Sat Dec 9 22:13:10 2006 Subject: [p2p-hackers] NATs reconfiguring IPs and Port Numbers In-Reply-To: <8C1DD971-379F-45F8-8169-8238C52D03F6@magma.ca> Message-ID: <20060302214524.21119.qmail@web53604.mail.yahoo.com> It seems port numbers change after a certain amount of time due to the local NAT or local ISP, and the peer to peer application has to re-configure itself to find out what its new global address is...and broadcast that to others so they can re-connect; does anyone know what the average time is a port number is good for? I'm getting up to 24 hours testing through Comcast and SBC; but i have to sleep so...i'm not sure on this number. thanks lemon You don't get no juice unless you squeeze Lemon Obrien, the Third. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://zgp.org/pipermail/p2p-hackers/attachments/20060302/7e2b03db/attachment.html From coderman at gmail.com Thu Mar 2 22:44:33 2006 From: coderman at gmail.com (coderman) Date: Sat Dec 9 22:13:10 2006 Subject: [p2p-hackers] NATs reconfiguring IPs and Port Numbers In-Reply-To: <20060302214524.21119.qmail@web53604.mail.yahoo.com> References: <8C1DD971-379F-45F8-8169-8238C52D03F6@magma.ca> <20060302214524.21119.qmail@web53604.mail.yahoo.com> Message-ID: <4ef5fec60603021444k323d801icd0b4134e5450897@mail.gmail.com> On 3/2/06, Lemon Obrien wrote: > ... > It seems port numbers change after a certain amount of time due to the local > NAT or local ISP, and the peer to peer application has to re-configure > itself to find out what its new global address is...and broadcast that to > others so they can re-connect; does anyone know what the average time is a > port number is good for? I'm getting up to 24 hours testing through Comcast > and SBC; but i have to sleep so...i'm not sure on this number. the longest time i've experienced on a dynamic address assigned DSL connection provided by verizon using a linux 2.6 based NAT is 4 months continuous. if you are using a lot of traffic or endpoints, verizon, comcast, and other providers appear to roll your endpoint more frequently. i've got a friend on verizon FiOS who gets an endpoint roll every few score minutes (few times per hour) when running a large number of torrents. i'm not aware of a thorough analysis of provider and NAT behavior under varying conditions but this would certainly be useful to track over time. From hopper at omnifarious.org Thu Mar 2 22:54:07 2006 From: hopper at omnifarious.org (Eric Hopper) Date: Sat Dec 9 22:13:10 2006 Subject: [p2p-hackers] NATs reconfiguring IPs and Port Numbers In-Reply-To: <4ef5fec60603021444k323d801icd0b4134e5450897@mail.gmail.com> References: <8C1DD971-379F-45F8-8169-8238C52D03F6@magma.ca> <20060302214524.21119.qmail@web53604.mail.yahoo.com> <4ef5fec60603021444k323d801icd0b4134e5450897@mail.gmail.com> Message-ID: <20060302225407.GA14429@omnifarious.org> On Thu, Mar 02, 2006 at 02:44:33PM -0800, coderman wrote: > if you are using a lot of traffic or endpoints, verizon, comcast, and > other providers appear to roll your endpoint more frequently. i've > got a friend on verizon FiOS who gets an endpoint roll every few score > minutes (few times per hour) when running a large number of torrents. I wonder if that kind of behavior is one of the reasons IPv6 isn't being rolled out much by ISPs. It would make doing that kind of thing a lot harder. I think though that consumer level NAT hardware might have some sort of limit on the number of mappings it can keep in memory and apply to packets though. *sigh*, -- "It does me no injury for my neighbor to say there are twenty gods or no God. It neither picks my pocket nor breaks my leg." --- Thomas Jefferson "Go to Heaven for the climate, Hell for the company." -- Mark Twain -- Eric Hopper (http://www.omnifarious.org/~hopper) -- -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://zgp.org/pipermail/p2p-hackers/attachments/20060302/3b73bb3f/attachment.pgp From saikat at cs.cornell.edu Thu Mar 2 23:15:49 2006 From: saikat at cs.cornell.edu (Saikat Guha) Date: Sat Dec 9 22:13:10 2006 Subject: [p2p-hackers] NATs reconfiguring IPs and Port Numbers In-Reply-To: <20060302214524.21119.qmail@web53604.mail.yahoo.com> References: <20060302214524.21119.qmail@web53604.mail.yahoo.com> Message-ID: <1141341349.25273.40.camel@localhost.localdomain> On Thu, 2006-03-02 at 13:45 -0800, Lemon Obrien wrote: > It seems port numbers change after a certain amount of time due to the > local NAT or local ISP Do you mean the external port number allocated by the NAT? Most NATs will timeout idle connections; sending subsequent packets will cause a new allocation. The length of the timeout depends on the transport protocol; the new allocation depends on the mapping type of the NAT. For TCP, most NATs timeout somewhere between 1-2 hours. Some (rather aggressive) NATs will timeout within 10-15 minutes of inactivity. Detailed numbers here: http://nutss.net/stunt-results.php?sort=-33 Standards in the process are trying to peg the default timeout to 2h. For UDP, the timeout is significantly less. Standards are setting the UDP timeout to 5 minutes of inactivity. > , and the peer to peer application has to re-configure itself to find > out what its new global address is > ...and broadcast that to others so they can re-connect; That said, the standards are also trying to ensure NATs have consistent mapping / cone behavior / address and port independent mapping. Consequently, the need to re-publish this information should diminish over time. > I'm getting up to 24 hours testing through Comcast and SBC; but i have > to sleep so...i'm not sure on this number. Hmmm. This should not be an ISP issue unless the ISP is putting you behind a NAT. Also, your subject seems to suggest your IP address is changing -- is comcast/sbc assigning you new DHCP addresses every few days? Perhaps the DHCP lease time can hint on the value they are using. In any event, as long as your endpoint renews your DHCP lease before it expires, it shouldn't change IP addresses. -- Saikat -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: This is a digitally signed message part Url : http://zgp.org/pipermail/p2p-hackers/attachments/20060302/be23daf6/attachment.pgp From saikat at cs.cornell.edu Thu Mar 2 23:24:29 2006 From: saikat at cs.cornell.edu (Saikat Guha) Date: Sat Dec 9 22:13:10 2006 Subject: [p2p-hackers] NATs reconfiguring IPs and Port Numbers In-Reply-To: <20060302225407.GA14429@omnifarious.org> References: <8C1DD971-379F-45F8-8169-8238C52D03F6@magma.ca> <20060302214524.21119.qmail@web53604.mail.yahoo.com> <4ef5fec60603021444k323d801icd0b4134e5450897@mail.gmail.com> <20060302225407.GA14429@omnifarious.org> Message-ID: <1141341869.25273.49.camel@localhost.localdomain> On Thu, 2006-03-02 at 14:54 -0800, Eric Hopper wrote: > I think though that consumer level NAT hardware might have some sort of > limit on the number of mappings it can keep in memory and apply to > packets though. Certain (really old) NAT models (/firmware) did indeed limit the number of simultaneous connections to 256. They are really hard to find these days. The lowest limit I can find is 1000 (http://nutss.net/stunt-results.php?sort=-9), but most NATs support roughly 65K mappings these days. You'd think they'd allow memory to fill up before doing any garbage collection of stale connections; turns out vendors would rather do the timeout thing (some fud about DoS etc. check the behave list for messages from Hoffman and Srisuresh on the topic) -- so for the most part, 65K is more than the app can use anyways and inactivity timeouts are the primary concern here. -- Saikat -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: This is a digitally signed message part Url : http://zgp.org/pipermail/p2p-hackers/attachments/20060302/aa007f25/attachment.pgp From saikat at cs.cornell.edu Thu Mar 2 23:38:00 2006 From: saikat at cs.cornell.edu (Saikat Guha) Date: Sat Dec 9 22:13:10 2006 Subject: [p2p-hackers] Number of pinholes supported by low-end NATs? In-Reply-To: <8C1DD971-379F-45F8-8169-8238C52D03F6@magma.ca> References: <8C1DD971-379F-45F8-8169-8238C52D03F6@magma.ca> Message-ID: <1141342680.25273.59.camel@localhost.localdomain> On Thu, 2006-03-02 at 16:20 -0500, Philip Matthews wrote: > Does anyone have any idea how many pinholes a P2P application > can have open at one time through a typical low-end NAT box? I just posted about this and linked so some related data in the previous thread. Most NATs you can buy off the shelf support upwards of a few thousand; 65k in many cases. > Do the numbers differ if the messages are carried over UDP vs TCP? It may. There are two things to consider here (beside UDP/TCP). One is the NAT mapping type (think "cone" or not), and the other is NAT filtering type (think "full", "restricted" etc.) Intuitively, full cone NATs need only keep track of the mapping (just the local port) -- they can potentially support infinite simultaneous sessions from the same local port; and support ~65K such local ports. Non-cone, or restricted cone NATs need to track each session (both local port, and destination) -- they can support roughly ~65K simultaneous sessions combined over all choices of local ports. This number can differ for UDP and TCP; and in some cases, the combined number of simultaneous TCP and UDP sessions could be subject to a sum total of ~65K. If you are looking for some really absolute pessimistic lower bounds, the most conservative I'd go would be ~1K simultaneous sessions (TCP and UDP combined) through the NAT. cheers, -- Saikat -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: This is a digitally signed message part Url : http://zgp.org/pipermail/p2p-hackers/attachments/20060302/e83dc097/attachment.pgp From saikat at cs.cornell.edu Thu Mar 2 23:47:47 2006 From: saikat at cs.cornell.edu (Saikat Guha) Date: Sat Dec 9 22:13:10 2006 Subject: [p2p-hackers] NATs reconfiguring IPs and Port Numbers In-Reply-To: <1141341349.25273.40.camel@localhost.localdomain> References: <20060302214524.21119.qmail@web53604.mail.yahoo.com> <1141341349.25273.40.camel@localhost.localdomain> Message-ID: <1141343267.25273.65.camel@localhost.localdomain> On Thu, 2006-03-02 at 18:15 -0500, Saikat Guha wrote: > Detailed numbers here: http://nutss.net/stunt-results.php?sort=-33 The column of interest is on the far right, third last, titled 'Timer ESTD'. This is the time an established TCP connection will stay up through the NAT despite inactivity (e.g. SSH left on for days without activity). The column reports the lower-bound and upper-bound inside of which the true timer value lies. cheers, -- Saikat -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: This is a digitally signed message part Url : http://zgp.org/pipermail/p2p-hackers/attachments/20060302/8cb2dbe3/attachment.pgp From seth.johnson at RealMeasures.dyndns.org Fri Mar 3 00:10:31 2006 From: seth.johnson at RealMeasures.dyndns.org (Seth Johnson) Date: Sat Dec 9 22:13:10 2006 Subject: [p2p-hackers] Important Statement to Review for Signing Message-ID: <44078977.8F6EFFB0@RealMeasures.dyndns.org> Hello folks, Please review the important joint statement below, related to the WIPO Broadcaster's Treaty, and consider adding your signature if you are an American citizen. Also make sure those you know who should sign are also given the opportunity. Andy Oram has written a good letter to the US Delegation to WIPO on the subject: > http://www.oreillynet.com/pub/a/etel/2006/01/13/the-problem-with-webcasting.html?page=2 CPTech Links on the Treaty: > http://www.cptech.org/ip/wipo/bt/index.html#Coments Electronic Frontier Foundation Links: > http://www.eff.org/IP/WIPO/broadcasting_treaty/ IP Justice Links: > http://www.ipjustice.org/WIPO/broadcasters.shtml Union for the Public Domain Links: > http://www.public-domain.org/?q=node/47 The Latest Draft of the Treaty: > http://www.cptech.org/ip/wipo/sccr12.2rev2.doc A survey of relevant links: > http://www.hyperorg.com/blogger/mtarchive/wipo_and_the_war_against_the_i.html If you choose to sign, please send your name along with an affiliation or appropriate short phrase to attach to your name for identification purposes, to mailto:seth.p.johnson@gmail.com. If your organization endorses the statement, please indicate that separately, so your organization will be listed under that header. Thank you for consideration. Seth Johnson Corresponding Secretary New Yorkers for Fair Use Joint Statement to Congress: Dear (Relevant Congressional Committees) (cc the WIPO Delegation): Negotiations are currently underway at the World Intellectual Property Organization (WIPO) to develop a treaty giving broadcasters power to suppress currently lawful communications. The United States delegation is also advocating similar rights for "webcasters" through which the authors of new works communicate them to the public. Some provisions of the proposed "Treaty on the Protection of Broadcasting Organizations" would merely update and standardize existing legal norms, but several proposals would require Congress to enact sweeping new laws that give private parties control over information, communication, and even copyrighted works of others, whenever they have broadcast or "webcast" the work. The novel policy areas addressed by this treaty go beyond ordinary treaty-making that seeks worldwide adherence to U.S. policy. Instead, this initiative invades Congress? prerogative to develop and establish national policy. Indeed, even as Congress is debating how best to protect network neutrality, treaty negotiators are debating how to eliminate it. The threat to personal liberties presented by this treaty is too grave to allow these new policy initiatives be handed over to an unelected delegation to negotiate with foreign countries, leaving Congress with the sole option whether to acquiesce. When dealing with policies that are related to copyright and communications, Congress's assigned powers and responsibility under Article I, Section 8 of the Constitution become particularly important. We urge two important steps. First, the new proposed regulations should be published in the Federal Register, with an invitation to the public to comment. Second, the appropriate House and Senate committees should hold hearings to more fully explore the impact of these novel legal restrictions on commerce, freedom of speech, copyright holders, network neutrality, and communications policy. Americans currently enjoy substantial freedoms with respect to broadcast and webcast communications. Under the proposed treaty, the existing options available to commercial enterprises and entrepreneurs as well as the general public to communicate news, information and entertainment would be limited by a new private gatekeeper who adds nothing of value to the content. Communications policies currently under discussion at the FCC would be impacted. Individuals and small businesses would be limited in their freedom of speech. Copyright owners would find their freedom to license their works limited by whether the work had been broadcast or webcast. The principle of network neutrality, already the subject of congressional hearings, would be all but destroyed. As able as the staff of the United States Patent and Trademark Office and the Library of Congress may be, it was never intended that they alone should stake out the United States national policy to be promoted before an unelected international body in entirely new areas abridging civil liberties. Congress should be the first to establish America?s national policies in this new area so that our WIPO delegation will have sufficient guidance to achieve legitimate objectives without impairing Constitutional principles such as freedom of speech and assembly, without impairing the value of copyrights, and without granting to private parties arbitrary power to suppress existing freedoms or burden new technologies. We cannot afford for Congress to wait for the Senate to be presented with a fully formed treaty calling for the enacting of domestic law at odds with fundamental American liberties foreign to American and international legal norms, and that would bring to a close many of the benefits of widespread personal computing and the end-to-end connectivity brought by the Internet. We ask Congress to use its authority now to shape these important communications policies impacting constitutionally based copyright laws and First Amendment liberties. Signed, (Affiliations for individual signers are for identification only. Endorsing organizations are listed separately.) William Abernathy, Independent Technical Editor Scottie D. Arnett, President, Info-Ed, Inc. Jonathan Askin, Pulver.com John Bachir, Ibiblio.org Tom Barger, DMusic.com Fred Benenson, FreeCulture.org Daniel Berninger, VON Coalition Eric Blossom, GNU Radio Joshua Breitbart, Media Tank Dave Burstein, Editor, DSL Prime Michael Calabrese, Vice President, New America Foundation Dave A. Chakrabarti, Community Technologist, CTCNet Chicago Steven Cherry, Senior Associate Editor, IEEE Spectrum Steven Clift, Publicus.Net Roland J. Cole, J.D., Ph.D., Executive Director, Software Patent Institute Gordon Cook, Editor, Publisher and Owner since 1992 of the COOK Report on Internet Protocol Walt Crawford, Editor/Publisher, Cites & Insights Cynthia H. de Lorenzi, Washington Bureau for ISP Advocacy Cory Doctorow, Author, journalist, Fulbright Chair, EFF Fellow Marshall Eubanks, CEO, AmericaFree.tv Harold Feld, Senior Vice President, Media Access Project Miles R. Fidelman, President, The Center for Civic Networking Richard Forno (bio: http://www.infowarrior.org/rick.html) Laura N. Gasaway, Professor of Law, University of North Carolina Paul Gherman, University Librarian, Vanderbilt University Shubha Ghosh, Professor of Law, Southern Methodist University Paul Ginsparg, Cornell University Fred R. Goldstein, Ionary Consulting Robin Gross, IP Justice Michael Gurstein, New Jersey Institute for Technology Jon Hall, President, Linux International Chuck Hamaker, Atkins Library, University of North Carolina - Charlotte Charles M. Hannum, consultant, founder of The NetBSD Project Dewayne Hendricks, CEO, Dandin Group David R Hughes, CEO, Old Colorado City Communications, 1993 EFF Pioneer Award Paul Hyland, Computer Professionals for Social Responsibility David S. Isenberg, Ph.D., Founder & CEO, isen.com, LLC Seth Johnson, New Yorkers for Fair Use Paul Jones, School of Information and Library Science, University of North Carolina - Chapel Hill Peter D. Junger, Professor of Law Emeritus, Case Western Reserve University Brewster Kahle, Internet Archive Jerry Kang, Professor of Law, UCLA School of Law Dennis S. Karjala, Jack E. Brown Professor of Law, Arizona State University Dan Krimm, Independent Musician Michael J. Kurtz, Astronomer and Computer Scientist, Harvard- Smithsonian Center for Astrophysics Michael Maranda, President, Association For Community Networking Kevin Marks, mediAgora Anthony McCann, www.beyondthecommons.com Sascha Meinrath, Champaign-Urbana Community Wireless Network, Free Press Edmund Mierzwinski, Consumer Program Director, U.S. Public Interest Research Group Lee N. Miller, Ph.D., Editor Emeritus, Ecological Society of America John Mitchell, InteractionLaw Tom Moritz, Chief, Knowledge Managment, Getty Research Institute Andrew Odlyzko, University of Minnesota Ken Olthoff, Advisory Board, EFF Austin Andy Oram, Editor, O'Reilly Media Bruce Perens (bio at http://perens.com/Bio.html) Ian Peter, Senior Partner, Ian Peter and Associates Pty Ltd Malla Pollack, Law Professor, American Justice School of Law Jeff Pulver, Pulver.com Tom Raftery, PodLeaders.com David P. Reed, contributor to original Internet Protocol design Jerome H. Reichman, Bunyan S. Womble Professor of Law Lawrence Rosen, Rosenlaw & Einschlag; Stanford University Lecturer in Law Bruce Schneier, security technologist and CTO, Counterpane David J. Smith, Specialist of Distributed Content Distribution and Protocols, Michigan State University Michael E. Smith, LXNY Richard Stallman, President, Free Software Foundation Fred Stutzman, Ph.D. Student, UNC Chapel Hill Peter Suber, Open Access Project Director, Public Knowledge Jay Sulzberger, New Yorkers for Fair Use Aaron Swartz, infogami Stephen H. Unger, Professor, Computer Science Department, Columbia University Eric F. Van de Velde, Ph.D., Director, Library Information Technology, California Institute of Technology Tom Vogt, independent computer security researcher David Weinberger, Harvard Berkman Center Frannie Wellings, Free Press Adam Werbach, President, Ironweed Films Stephen Wolff, igewolff.net Brett Wynkoop, Wynn Data Ltd. John Young, Cryptome.org Endorsing Organizations: Association For Community Networking (AFCN) The Center for Civic Networking Computer Professionals for Social Responsibility Contact Communications The COOK Report on Internet Protocol Cryptome.org Champaign-Urbana Community Wireless Network Dandin Group FreeCulture.org Free Press Free Software Foundation Illinois Community Technology Coalition Internet Archive Ionary Consulting IP Justice isen.com, LLC mediAgora New Yorkers for Fair Use Old Colorado City Communications Podleaders.com Pulver.com Rosenlaw & Einschlag U.S. Public Interest Research Group Washington Bureau for ISP Advocacy Wyoming.com -- [separate one-page attachment] WHY PUBLIC SCRUTINY OF THE PROPOSED BROADCASTER TREATY IS NEEDED If Congress were to hold public hearings, or if the US delegation to WIPO were to publish the current proposal for public review and comment, myriad voices from various segments of society could come forward to show that the proposed Broadcaster's Treaty: * Is written to look like existing copyright treaties, but it is not based on the constitutional requirements for copyright protection, such as originality, and in fact is antagonistic to copyrights * Is promoted as a way of standardizing existing signal protection, but in fact extends well beyond signal protection by giving broadcasters and webcasters a monopoly, for 50 years, over the content created by others the moment it is broadcast or transmitted over the Internet * Gives broadcasters greater rights than producers of original works * Accords exclusive rights to non-authors in direct violation of fundamental rights guaranteed by the Constitution * Attacks the principle of network neutrality which serves as the basis by which the Internet has fostered a profound expansion in human capacities and innovation * Grants privileges that extend beyond broadcast signals to actually give broadcasters control over works conveyed within a broadcast -- including copyrighted and public domain works * Blocks fair use and other copyright provisions that enable the public to make use of and benefit from published information * Chills freedom of expression by extending unwarranted controls over broadcast publication * Benefits broadcasters at the expense of the web, the public and future innovation * Creates a de facto tax on copyrights, freedom of speech, communications and technological progress, all for the benefit of broadcasters and webcasters who have added nothing to deserve such a windfall. From slavitch at gmail.com Fri Mar 3 00:42:14 2006 From: slavitch at gmail.com (Michael Slavitch) Date: Sat Dec 9 22:13:10 2006 Subject: [p2p-hackers] Number of pinholes supported by low-end NATs? In-Reply-To: <8C1DD971-379F-45F8-8169-8238C52D03F6@magma.ca> References: <8C1DD971-379F-45F8-8169-8238C52D03F6@magma.ca> Message-ID: <1fc32c8f0603021642h4ba80b1dy@mail.gmail.com> In theory 65K-2. In practise, "less" but lots. Our testing seems to support values that are like this: 1 4 16 64 256 1K 4K 16K 64K With more recent boxes supporting the higher end in test cases if not in real life. CPU and bandwidth cack out before ports do. Gaming forced this. "Do the numbers differ if the messages are carried over UDP vs TCP?" . If you can guess how much RAM is available you can guess the size of the pinhole range. Look at ( http://openwrt.org/) On 02/03/06, Philip Matthews < philip_matthews@magma.ca> wrote: > > Does anyone have any idea how many pinholes a P2P application > can have open at one time through a typical low-end NAT box? > > Could a P2P application maintain connections to 30 peers simultaneously? > What about 50 peers? Or 100 peers? > > Do the numbers differ if the messages are carried over UDP vs TCP? > > (Note: My interest is in limitations in the NAT box, and NOT in any > limitations in > the Windows, Mac, Linux, ... box on which the P2P application is > running.) > > Just wondering if anyone has any solid data in this area. > > - Philip > _______________________________________________ > p2p-hackers mailing list > p2p-hackers@zgp.org > http://zgp.org/mailman/listinfo/p2p-hackers > _______________________________________________ > Here is a web page listing P2P Conferences: > http://www.neurogrid.net/twiki/bin/view/Main/PeerToPeerConferences > -- Michael Slavitch Ottawa, Ontario Canada -------------- next part -------------- An HTML attachment was scrubbed... URL: http://zgp.org/pipermail/p2p-hackers/attachments/20060302/034bd318/attachment.html From matthew at matthew.at Fri Mar 3 03:08:30 2006 From: matthew at matthew.at (Matthew Kaufman) Date: Sat Dec 9 22:13:10 2006 Subject: [p2p-hackers] NATs reconfiguring IPs and Port Numbers In-Reply-To: <20060302214524.21119.qmail@web53604.mail.yahoo.com> Message-ID: <00c401c63e6f$c29e15f0$0202fea9@matthewdesk> If your protocol supports IP address mobility (preferably with protection against using that against an endpoint for session hijacking, and preferably protecting that against replay as well), then existing peers wouldn't need to "re-connect", they'd just be able to keep their connections up through the change in address,... though you would want to re-determine your external address/port so any new peers could connect to that. Conveniently, knowledge that your address had undergone a mobility event would be exactly how you'd know when you needed to do that re-determination. I happen to have designed, published the specification for, and implemented a protocol that does just this. Matthew Kaufman matthew@matthew.at http://www.amicima.com _____ From: p2p-hackers-bounces@zgp.org [mailto:p2p-hackers-bounces@zgp.org] On Behalf Of Lemon Obrien Sent: Thursday, March 02, 2006 1:45 PM To: Peer-to-peer development. Subject: [p2p-hackers] NATs reconfiguring IPs and Port Numbers It seems port numbers change after a certain amount of time due to the local NAT or local ISP, and the peer to peer application has to re-configure itself to find out what its new global address is...and broadcast that to others so they can re-connect; does anyone know what the average time is a port number is good for? I'm getting up to 24 hours testing through Comcast and SBC; but i have to sleep so...i'm not sure on this number. thanks lemon You don't get no juice unless you squeeze Lemon Obrien, the Third. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://zgp.org/pipermail/p2p-hackers/attachments/20060302/662252da/attachment.htm From eugen at leitl.org Fri Mar 3 11:33:11 2006 From: eugen at leitl.org (Eugen Leitl) Date: Sat Dec 9 22:13:10 2006 Subject: [p2p-hackers] NATs reconfiguring IPs and Port Numbers In-Reply-To: <20060302225407.GA14429@omnifarious.org> References: <8C1DD971-379F-45F8-8169-8238C52D03F6@magma.ca> <20060302214524.21119.qmail@web53604.mail.yahoo.com> <4ef5fec60603021444k323d801icd0b4134e5450897@mail.gmail.com> <20060302225407.GA14429@omnifarious.org> Message-ID: <20060303113311.GD25017@leitl.org> On Thu, Mar 02, 2006 at 02:54:07PM -0800, Eric Hopper wrote: > I think though that consumer level NAT hardware might have some sort of > limit on the number of mappings it can keep in memory and apply to > packets though. Most consumer NAT implementations are indeed buggy and limited (P2P crashing consumer firewalls is well-documented), which is the main reason I've moved to a m0n0wall on a WRAP. -- Eugen* Leitl leitl http://leitl.org ______________________________________________________________ ICBM: 48.07100, 11.36820 http://www.ativel.com 8B29F6BE: 099D 78BA 2FD3 B014 B08A 7779 75B0 2443 8B29 F6BE -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: Digital signature Url : http://zgp.org/pipermail/p2p-hackers/attachments/20060303/95f26b61/attachment.pgp From googaya at gmail.com Fri Mar 3 12:33:55 2006 From: googaya at gmail.com (googaya) Date: Sat Dec 9 22:13:10 2006 Subject: [p2p-hackers] NATs reconfiguring IPs and Port Numbers In-Reply-To: <20060303113311.GD25017@leitl.org> References: <8C1DD971-379F-45F8-8169-8238C52D03F6@magma.ca> <20060302214524.21119.qmail@web53604.mail.yahoo.com> <4ef5fec60603021444k323d801icd0b4134e5450897@mail.gmail.com> <20060302225407.GA14429@omnifarious.org> <20060303113311.GD25017@leitl.org> Message-ID: <440837B3.3000206@gmail.com> Eugen Leitl wrote: >On Thu, Mar 02, 2006 at 02:54:07PM -0800, Eric Hopper wrote: > > > >>I think though that consumer level NAT hardware might have some sort of >>limit on the number of mappings it can keep in memory and apply to >>packets though. >> >> > >Most consumer NAT implementations are indeed buggy and limited (P2P >crashing consumer firewalls is well-documented), which is the main >reason I've moved to a m0n0wall on a WRAP. > > > Good point we have found excellent success using OpenVPN and can connect to an Asterisk/OpenSER with no problems no matter the location, Enterprise, Hotels/Motels, Starbucks, etc... -E http://googaya.com From m.rogers at cs.ucl.ac.uk Fri Mar 3 15:42:29 2006 From: m.rogers at cs.ucl.ac.uk (Michael Rogers) Date: Sat Dec 9 22:13:10 2006 Subject: [p2p-hackers] Number of pinholes supported by low-end NATs? In-Reply-To: <8C1DD971-379F-45F8-8169-8238C52D03F6@magma.ca> References: <8C1DD971-379F-45F8-8169-8238C52D03F6@magma.ca> Message-ID: <440863E5.2050009@cs.ucl.ac.uk> Hi Philip, Gnutella ultrapeers usually have 32 connections to other ultrapeers and around the same number to leaf peers. I'd guess a lot of ultrapeers are behind domestic NATs but I don't know how you'd find out for sure - maybe LimeWire collects some stats from its firewall detection logic? Cheers, Michael Philip Matthews wrote: > Does anyone have any idea how many pinholes a P2P application > can have open at one time through a typical low-end NAT box? > > Could a P2P application maintain connections to 30 peers simultaneously? > What about 50 peers? Or 100 peers? > > Do the numbers differ if the messages are carried over UDP vs TCP? > > (Note: My interest is in limitations in the NAT box, and NOT in any > limitations in > the Windows, Mac, Linux, ... box on which the P2P application is running.) > > Just wondering if anyone has any solid data in this area. > > - Philip > _______________________________________________ > p2p-hackers mailing list > p2p-hackers@zgp.org > http://zgp.org/mailman/listinfo/p2p-hackers > _______________________________________________ > Here is a web page listing P2P Conferences: > http://www.neurogrid.net/twiki/bin/view/Main/PeerToPeerConferences From bcg at utas.edu.au Tue Mar 7 01:35:56 2006 From: bcg at utas.edu.au (Brad Goldsmith) Date: Sat Dec 9 22:13:10 2006 Subject: [p2p-hackers] Mailing lists for P2P conference announcements? In-Reply-To: <440863E5.2050009@cs.ucl.ac.uk> References: <8C1DD971-379F-45F8-8169-8238C52D03F6@magma.ca> <440863E5.2050009@cs.ucl.ac.uk> Message-ID: <6103992410571a1b0388715d8f088442@utas.edu.au> Hi All, What mailing lists (or other resources) do people here use for conference announcements? I've just joined the p2p research group mailing list at the ietf (p2prg@ietf.org). Just wondering if anyone can suggest some other good lists for keeping up with calls for papers, etc? I always seem to miss them! Cheers, Brad --- Brad Goldsmith School of Computing University of Tasmania, Tasmania, Australia Office: Launceston Campus, Computing Building, V-177 Telephone: (03) 6324 3389 International: +61-3-6324 3389 Facsimile: (03) 6324 3368 International: +61-3-6324 3368 From lemonobrien at yahoo.com Tue Mar 7 01:47:13 2006 From: lemonobrien at yahoo.com (Lemon Obrien) Date: Sat Dec 9 22:13:11 2006 Subject: [p2p-hackers] Mailing lists for P2P conference announcements? In-Reply-To: <6103992410571a1b0388715d8f088442@utas.edu.au> Message-ID: <20060307014713.12935.qmail@web53607.mail.yahoo.com> why do you want to go to a conference on peer-to-peer technology? the state of the art is here. Brad Goldsmith wrote: Hi All, What mailing lists (or other resources) do people here use for conference announcements? I've just joined the p2p research group mailing list at the ietf (p2prg@ietf.org). Just wondering if anyone can suggest some other good lists for keeping up with calls for papers, etc? I always seem to miss them! Cheers, Brad --- Brad Goldsmith School of Computing University of Tasmania, Tasmania, Australia Office: Launceston Campus, Computing Building, V-177 Telephone: (03) 6324 3389 International: +61-3-6324 3389 Facsimile: (03) 6324 3368 International: +61-3-6324 3368 _______________________________________________ p2p-hackers mailing list p2p-hackers@zgp.org http://zgp.org/mailman/listinfo/p2p-hackers _______________________________________________ Here is a web page listing P2P Conferences: http://www.neurogrid.net/twiki/bin/view/Main/PeerToPeerConferences You don't get no juice unless you squeeze Lemon Obrien, the Third. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://zgp.org/pipermail/p2p-hackers/attachments/20060306/caff10c7/attachment.htm From dbarrett at quinthar.com Tue Mar 7 01:49:59 2006 From: dbarrett at quinthar.com (David Barrett) Date: Sat Dec 9 22:13:11 2006 Subject: [p2p-hackers] Mailing lists for P2P conference announcements? In-Reply-To: <20060307014713.12935.qmail@web53607.mail.yahoo.com> Message-ID: <20060307015002.D7A1A3FD10@capsicum.zgp.org> Speaking of which, we should have another P2P in SFC meetup. The last one was a good time. Any takers? _____ From: p2p-hackers-bounces@zgp.org [mailto:p2p-hackers-bounces@zgp.org] On Behalf Of Lemon Obrien Sent: Monday, March 06, 2006 5:47 PM To: Peer-to-peer development. Subject: Re: [p2p-hackers] Mailing lists for P2P conference announcements? why do you want to go to a conference on peer-to-peer technology? the state of the art is here. Brad Goldsmith wrote: Hi All, What mailing lists (or other resources) do people here use for conference announcements? I've just joined the p2p research group mailing list at the ietf (p2prg@ietf.org). Just wondering if anyone can suggest some other good lists for keeping up with calls for papers, etc? I always seem to miss them! Cheers, Brad --- Brad Goldsmith School of Computing University of Tasmania, Tasmania, Australia Office: Launceston Campus, Computing Building, V-177 Telephone: (03) 6324 3389 International: +61-3-6324 3389 Facsimile: (03) 6324 3368 International: +61-3-6324 3368 _______________________________________________ p2p-hackers mailing list p2p-hackers@zgp.org http://zgp.org/mailman/listinfo/p2p-hackers _______________________________________________ Here is a web page listing P2P Conferences: http://www.neurogrid.net/twiki/bin/view/Main/PeerToPeerConferences You don't get no juice unless you squeeze Lemon Obrien, the Third. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://zgp.org/pipermail/p2p-hackers/attachments/20060306/6a3e6d57/attachment.html From ap at hamachi.cc Tue Mar 7 01:53:15 2006 From: ap at hamachi.cc (Alex Pankratov) Date: Sat Dec 9 22:13:11 2006 Subject: [p2p-hackers] Mailing lists for P2P conference announcements? In-Reply-To: <20060307015002.D7A1A3FD10@capsicum.zgp.org> References: <20060307015002.D7A1A3FD10@capsicum.zgp.org> Message-ID: <440CE78B.10304@hamachi.cc> Is anyone from Vancouver BC here by the way ? Just curious :) Alex David Barrett wrote: > Speaking of which, we should have another P2P in SFC meetup. The last > one was a good time. Any takers? > > > > ------------------------------------------------------------------------ > > *From:* p2p-hackers-bounces@zgp.org [mailto:p2p-hackers-bounces@zgp.org] > *On Behalf Of *Lemon Obrien > *Sent:* Monday, March 06, 2006 5:47 PM > *To:* Peer-to-peer development. > *Subject:* Re: [p2p-hackers] Mailing lists for P2P conference announcements? > > > > why do you want to go to a conference on peer-to-peer technology? the > state of the art is here. > > */Brad Goldsmith /* wrote: > > Hi All, > > What mailing lists (or other resources) do people here use for > conference announcements? > > I've just joined the p2p research group mailing list at the ietf > (p2prg@ietf.org). > > Just wondering if anyone can suggest some other good lists for keeping > up with calls for papers, etc? > > I always seem to miss them! > > Cheers, > Brad > > --- > > Brad Goldsmith > School of Computing > University of Tasmania, Tasmania, Australia > Office: Launceston Campus, Computing Building, V-177 > Telephone: (03) 6324 3389 International: +61-3-6324 3389 > Facsimile: (03) 6324 3368 International: +61-3-6324 3368 > > _______________________________________________ > p2p-hackers mailing list > p2p-hackers@zgp.org > http://zgp.org/mailman/listinfo/p2p-hackers > _______________________________________________ > Here is a web page listing P2P Conferences: > http://www.neurogrid.net/twiki/bin/view/Main/PeerToPeerConferences > > > > > You don't get no juice unless you squeeze > Lemon Obrien, the Third. > > > ------------------------------------------------------------------------ > > _______________________________________________ > p2p-hackers mailing list > p2p-hackers@zgp.org > http://zgp.org/mailman/listinfo/p2p-hackers > _______________________________________________ > Here is a web page listing P2P Conferences: > http://www.neurogrid.net/twiki/bin/view/Main/PeerToPeerConferences From bcg at utas.edu.au Tue Mar 7 02:32:39 2006 From: bcg at utas.edu.au (Brad Goldsmith) Date: Sat Dec 9 22:13:11 2006 Subject: [p2p-hackers] Mailing lists for P2P conference announcements? In-Reply-To: <20060307014713.12935.qmail@web53607.mail.yahoo.com> References: <20060307014713.12935.qmail@web53607.mail.yahoo.com> Message-ID: <447cc400203a31f4c007ae882bb5b96f@utas.edu.au> On 07/03/2006, at 12:47 PM, Lemon Obrien wrote: > why do you want to go to a conference on peer-to-peer technology? the > state of the art is here. Unfortunately, some of us have to publish in order to justify our existences. Or at least to justify our PhD grant money... So, to make this post more than a poor attempt at a witty retort, I know that the 6th IEEE p2p conf is coming up: http://p2p2006.csc.ncsu.edu/ - Submission date is April 26, 2006. Cheers. --- Brad Goldsmith School of Computing University of Tasmania, Tasmania, Australia Office: Launceston Campus, Computing Building, V-177 Telephone: (03) 6324 3389 International: +61-3-6324 3389 Facsimile: (03) 6324 3368 International: +61-3-6324 3368 From lemonobrien at yahoo.com Tue Mar 7 02:33:53 2006 From: lemonobrien at yahoo.com (Lemon Obrien) Date: Sat Dec 9 22:13:11 2006 Subject: [p2p-hackers] Mailing lists for P2P conference announcements? In-Reply-To: <20060307015002.D7A1A3FD10@capsicum.zgp.org> Message-ID: <20060307023353.92187.qmail@web53604.mail.yahoo.com> HERE HERE !!! David Barrett wrote: v\:* {behavior:url(#default#VML);} o\:* {behavior:url(#default#VML);} w\:* {behavior:url(#default#VML);} .shape {behavior:url(#default#VML);} st1\:*{behavior:url(#default#ieooui) } Speaking of which, we should have another P2P in SFC meetup. The last one was a good time. Any takers? --------------------------------- From: p2p-hackers-bounces@zgp.org [mailto:p2p-hackers-bounces@zgp.org] On Behalf Of Lemon Obrien Sent: Monday, March 06, 2006 5:47 PM To: Peer-to-peer development. Subject: Re: [p2p-hackers] Mailing lists for P2P conference announcements? why do you want to go to a conference on peer-to-peer technology? the state of the art is here. Brad Goldsmith wrote: Hi All, What mailing lists (or other resources) do people here use for conference announcements? I've just joined the p2p research group mailing list at the ietf (p2prg@ietf.org). Just wondering if anyone can suggest some other good lists for keeping up with calls for papers, etc? I always seem to miss them! Cheers, Brad --- Brad Goldsmith School of Computing University of Tasmania, Tasmania, Australia Office: Launceston Campus, Computing Building, V-177 Telephone: (03) 6324 3389 International: +61-3-6324 3389 Facsimile: (03) 6324 3368 International: +61-3-6324 3368 _______________________________________________ p2p-hackers mailing list p2p-hackers@zgp.org http://zgp.org/mailman/listinfo/p2p-hackers _______________________________________________ Here is a web page listing P2P Conferences: http://www.neurogrid.net/twiki/bin/view/Main/PeerToPeerConferences You don't get no juice unless you squeeze Lemon Obrien, the Third. _______________________________________________ p2p-hackers mailing list p2p-hackers@zgp.org http://zgp.org/mailman/listinfo/p2p-hackers _______________________________________________ Here is a web page listing P2P Conferences: http://www.neurogrid.net/twiki/bin/view/Main/PeerToPeerConferences You don't get no juice unless you squeeze Lemon Obrien, the Third. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://zgp.org/pipermail/p2p-hackers/attachments/20060306/ea7191d5/attachment.htm From slavitch at gmail.com Tue Mar 7 03:59:03 2006 From: slavitch at gmail.com (Michael Slavitch) Date: Sat Dec 9 22:13:11 2006 Subject: [p2p-hackers] Mailing lists for P2P conference announcements? In-Reply-To: <440CE78B.10304@hamachi.cc> References: <20060307015002.D7A1A3FD10@capsicum.zgp.org> <440CE78B.10304@hamachi.cc> Message-ID: <1fc32c8f0603061959v7d008fb9v61229389f2d91593@mail.gmail.com> Let's try to be interesting and make all p2psip-hackers meeting virtual and distributed. Let's eat what we grow. M On 3/6/06, Alex Pankratov wrote: > > Is anyone from Vancouver BC here by the way ? Just curious :) > > Alex > > David Barrett wrote: > > Speaking of which, we should have another P2P in SFC meetup. The last > > one was a good time. Any takers? > > > > > > > > ------------------------------------------------------------------------ > > > > *From:* p2p-hackers-bounces@zgp.org [mailto: p2p-hackers-bounces@zgp.org > ] > > *On Behalf Of *Lemon Obrien > > *Sent:* Monday, March 06, 2006 5:47 PM > > *To:* Peer-to-peer development. > > *Subject:* Re: [p2p-hackers] Mailing lists for P2P conference > announcements? > > > > > > > > why do you want to go to a conference on peer-to-peer technology? the > > state of the art is here. > > > > */Brad Goldsmith /* wrote: > > > > Hi All, > > > > What mailing lists (or other resources) do people here use for > > conference announcements? > > > > I've just joined the p2p research group mailing list at the ietf > > (p2prg@ietf.org). > > > > Just wondering if anyone can suggest some other good lists for keeping > > up with calls for papers, etc? > > > > I always seem to miss them! > > > > Cheers, > > Brad > > > > --- > > > > Brad Goldsmith > > School of Computing > > University of Tasmania, Tasmania, Australia > > Office: Launceston Campus, Computing Building, V-177 > > Telephone: (03) 6324 3389 International: +61-3-6324 3389 > > Facsimile: (03) 6324 3368 International: +61-3-6324 3368 > > > > _______________________________________________ > > p2p-hackers mailing list > > p2p-hackers@zgp.org > > http://zgp.org/mailman/listinfo/p2p-hackers > > _______________________________________________ > > Here is a web page listing P2P Conferences: > > http://www.neurogrid.net/twiki/bin/view/Main/PeerToPeerConferences > > > > > > > > > > You don't get no juice unless you squeeze > > Lemon Obrien, the Third. > > > > > > ------------------------------------------------------------------------ > > > > _______________________________________________ > > p2p-hackers mailing list > > p2p-hackers@zgp.org > > http://zgp.org/mailman/listinfo/p2p-hackers > > _______________________________________________ > > Here is a web page listing P2P Conferences: > > http://www.neurogrid.net/twiki/bin/view/Main/PeerToPeerConferences > _______________________________________________ > p2p-hackers mailing list > p2p-hackers@zgp.org > http://zgp.org/mailman/listinfo/p2p-hackers > _______________________________________________ > Here is a web page listing P2P Conferences: > http://www.neurogrid.net/twiki/bin/view/Main/PeerToPeerConferences > -- Michael Slavitch Ottawa, Ontario Canada -------------- next part -------------- An HTML attachment was scrubbed... URL: http://zgp.org/pipermail/p2p-hackers/attachments/20060306/90172eee/attachment.html From rosejn at gmail.com Tue Mar 7 10:46:38 2006 From: rosejn at gmail.com (Jeff Rose) Date: Sat Dec 9 22:13:11 2006 Subject: [p2p-hackers] clustering Message-ID: <440D648E.302@gmail.com> Anyone out there doing work in clustering? I'd be interested in hearing what people are up to, or if you have pointers to good papers that would be great too. At this point I'm interested in any angle, geographic distance based, semantic distance based, hierarchical, non-hierarchical etc... My goal is to work towards a very amorphous clustering scheme that allows any kind of object to be located close to relatives in the network as long as they share a common distance function. Peace, Jeff From jdefarge at gmail.com Tue Mar 7 13:58:37 2006 From: jdefarge at gmail.com (jacques defarge) Date: Sat Dec 9 22:13:11 2006 Subject: [p2p-hackers] clustering In-Reply-To: <440D648E.302@gmail.com> References: <440D648E.302@gmail.com> Message-ID: Hello Jeff, I am finishing my MSc. dissertation on p2p systems grouped in a cluster like structure. I have used JXTA as the p2p substrate, but it was the worst choice even possible. After finishing my MSc. work I am searching for really nice projects on such area. If you would like we could discuss your ideas further, develop some things, and even collaborate on a paper or a prototype. []'s Jacques On 3/7/06, Jeff Rose wrote: > > Anyone out there doing work in clustering? I'd be interested in hearing > what people are up to, or if you have pointers to good papers that would > be great too. At this point I'm interested in any angle, geographic > distance based, semantic distance based, hierarchical, non-hierarchical > etc... My goal is to work towards a very amorphous clustering scheme > that allows any kind of object to be located close to relatives in the > network as long as they share a common distance function. > > Peace, > Jeff > _______________________________________________ > p2p-hackers mailing list > p2p-hackers@zgp.org > http://zgp.org/mailman/listinfo/p2p-hackers > _______________________________________________ > Here is a web page listing P2P Conferences: > http://www.neurogrid.net/twiki/bin/view/Main/PeerToPeerConferences > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://zgp.org/pipermail/p2p-hackers/attachments/20060307/5e07fad7/attachment.htm From rosejn at gmail.com Tue Mar 7 14:25:48 2006 From: rosejn at gmail.com (Jeff Rose) Date: Sat Dec 9 22:13:11 2006 Subject: [p2p-hackers] clustering In-Reply-To: References: <440D648E.302@gmail.com> Message-ID: <440D97EC.8060408@gmail.com> Why was the JXTA stuff the worst p2p substrate possible? I don't really care much for java and all the corporate speak, but technically what was it missing? Outside of the realm of true research I've been looking around for ideas to eventually roll into some kind of p2p infrastructure for ruby... (Probably a set of interfaces for various services, and then a couple implementations behind it. DHT search, unstructured search (random walk, smart flood), torrent style download, and some glue...) Is your thesis available on the web? -Jeff From jacob at mungo.dk Tue Mar 7 16:23:13 2006 From: jacob at mungo.dk (Jacob Madsen) Date: Sat Dec 9 22:13:11 2006 Subject: [p2p-hackers] Supernodes in the FastTrack network Message-ID: <440DB371.8040702@mungo.dk> Hey I was reading about FastTrack on wikipedia and checked http://www.slyck.com for stats on the network. And according to http://www.slyck.com/stats.php there is almost 3 millions users at the moment. Do someone know of a method to calculate the approx. number of supernodes? Thanks! From saikat at cs.cornell.edu Tue Mar 7 16:51:33 2006 From: saikat at cs.cornell.edu (Saikat Guha) Date: Sat Dec 9 22:13:11 2006 Subject: [p2p-hackers] Supernodes in the FastTrack network In-Reply-To: <440DB371.8040702@mungo.dk> References: <440DB371.8040702@mungo.dk> Message-ID: <1141750293.5300.32.camel@localhost.localdomain> On Tue, 2006-03-07 at 17:23 +0100, Jacob Madsen wrote: > I was reading about FastTrack [...] almost 3 millions users at the > moment. > Do someone know of a method to calculate the approx. number of supernodes? Bunch of really *rough* estimates. 1) In [1], the authors claim that roughly 50% of MSN users are NAT'ed; such nodes cannot become supernodes [2] (while [2] relates to Skype specifically, there is some evidence to suggest Skype uses the same underlying network as Kazaa = FastTrack; also, [2] finds roughly 4 million users, so in the same ballpark). Based on these two, figure 1.5--2M supernodes as an upper bound. My personal intuition is that it is much less. 2) In [2], a crawl of the supernode network identified some 250K, only some of which were online simultaneously. The crawl was not exhaustive. Figure this is some sort of weak lower bound, with the real number being higher. End result: we don't know for sure, but my guess is somewhere between 200K to 2M active supernodes. An order of magnitude doesn't matter much, right? :P A large-scale crawl should yield some tighter bounds. The crawl methodology is written up in [2]. [1] P Rodriguez, See-Mong Tan, C. Gkantsidis, "On the feasibility of commercial, legal P2P Content Distribution", In ACM/SIGCOMM CCR, Jan 2006. http://research.microsoft.com/~pablo/papers/CCR.pdf [2] S Guha, N Daswani and R Jain, "An Experimental Study of the Skype Peer-to-Peer VoIP System", In IPTPS'06, Feb 2006. http://saikat.guha.cc/pub/iptps06-skype/ cheers, -- Saikat -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: This is a digitally signed message part Url : http://zgp.org/pipermail/p2p-hackers/attachments/20060307/21a3ad08/attachment.pgp From jacob at mungo.dk Tue Mar 7 18:31:09 2006 From: jacob at mungo.dk (Jacob Madsen) Date: Sat Dec 9 22:13:11 2006 Subject: [p2p-hackers] Supernodes in the FastTrack network In-Reply-To: <1141750293.5300.32.camel@localhost.localdomain> References: <440DB371.8040702@mungo.dk> <1141750293.5300.32.camel@localhost.localdomain> Message-ID: <440DD16D.104@mungo.dk> Hey again Great answar Saikat - thanks alot! The stuff about Skype using the FastTrack protocol or something similar made me wonder. I thought that the whole network wasnt reachable from a client's point, since the search requests have a time-to-live that is decremented at each supernode it passes and dropped when the value is zero (so in the aspect of Skype, not all users can reach eachother, unless they have implemented the protocol differently) Thats actually 2 questions and i'm really only interested in the first about FastTrack and the time-to-live value limiting the reachability of the search requests. As far as i remember, the client has 1 connection to a supernode and send its search requests to that supernode. The supernode send (flood) its supernode neighbours with the request and the time-to-live value in the request is decremented at each supernode it passes and will reach the zero value at some point and be dropped. Again - thanks in advance! Saikat Guha wrote: > On Tue, 2006-03-07 at 17:23 +0100, Jacob Madsen wrote: > >> I was reading about FastTrack [...] almost 3 millions users at the >> moment. >> Do someone know of a method to calculate the approx. number of supernodes? >> > > Bunch of really *rough* estimates. > > 1) In [1], the authors claim that roughly 50% of MSN users are NAT'ed; > such nodes cannot become supernodes [2] (while [2] relates to Skype > specifically, there is some evidence to suggest Skype uses the same > underlying network as Kazaa = FastTrack; also, [2] finds roughly 4 > million users, so in the same ballpark). Based on these two, figure > 1.5--2M supernodes as an upper bound. My personal intuition is that it > is much less. > > 2) In [2], a crawl of the supernode network identified some 250K, only > some of which were online simultaneously. The crawl was not exhaustive. > Figure this is some sort of weak lower bound, with the real number being > higher. > > End result: we don't know for sure, but my guess is somewhere between > 200K to 2M active supernodes. An order of magnitude doesn't matter much, > right? :P > > A large-scale crawl should yield some tighter bounds. The crawl > methodology is written up in [2]. > > [1] P Rodriguez, See-Mong Tan, C. Gkantsidis, "On the feasibility of > commercial, legal P2P Content Distribution", In ACM/SIGCOMM CCR, Jan > 2006. http://research.microsoft.com/~pablo/papers/CCR.pdf > > [2] S Guha, N Daswani and R Jain, "An Experimental Study of the Skype > Peer-to-Peer VoIP System", In IPTPS'06, Feb 2006. > http://saikat.guha.cc/pub/iptps06-skype/ > > cheers, > > ------------------------------------------------------------------------ > > _______________________________________________ > p2p-hackers mailing list > p2p-hackers@zgp.org > http://zgp.org/mailman/listinfo/p2p-hackers > _______________________________________________ > Here is a web page listing P2P Conferences: > http://www.neurogrid.net/twiki/bin/view/Main/PeerToPeerConferences > From saikat at cs.cornell.edu Tue Mar 7 19:30:33 2006 From: saikat at cs.cornell.edu (Saikat Guha) Date: Sat Dec 9 22:13:11 2006 Subject: [p2p-hackers] Supernodes in the FastTrack network In-Reply-To: <440DD16D.104@mungo.dk> References: <440DB371.8040702@mungo.dk> <1141750293.5300.32.camel@localhost.localdomain> <440DD16D.104@mungo.dk> Message-ID: <1141759833.5300.39.camel@localhost.localdomain> On Tue, 2006-03-07 at 19:31 +0100, Jacob Madsen wrote: > Thats actually 2 questions and i'm really only interested in the first > about FastTrack and the time-to-live value limiting the reachability of > the search requests. Perhaps [1] may be of some interest to you. I don't know much about the internal reachability of the FastTrack network. [1] J Liang, R Kumar, and K W Ross, "The Kazaa Overlay: A Measurement Study", Computer Networks 49, 6. Oct. 2005. http://cis.poly.edu/~ross/papers/KazaaOverlay.pdf -- Saikat -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: This is a digitally signed message part Url : http://zgp.org/pipermail/p2p-hackers/attachments/20060307/66548116/attachment.pgp From salman at cs.columbia.edu Tue Mar 7 20:50:14 2006 From: salman at cs.columbia.edu (Salman Abdul Baset) Date: Sat Dec 9 22:13:11 2006 Subject: [p2p-hackers] Supernodes in the FastTrack network In-Reply-To: <440DB371.8040702@mungo.dk> References: <440DB371.8040702@mungo.dk> Message-ID: Please see http://www1.cs.columbia.edu/~salman/publications/skype1_4.pdf It provides some insights on Skype super nodes. regards, salman On Tue, 7 Mar 2006, Jacob Madsen wrote: > Hey > > I was reading about FastTrack on wikipedia and checked > http://www.slyck.com for stats on the network. And according to > http://www.slyck.com/stats.php there is almost 3 millions users at the > moment. > Do someone know of a method to calculate the approx. number of supernodes? > > Thanks! > _______________________________________________ > p2p-hackers mailing list > p2p-hackers@zgp.org > http://zgp.org/mailman/listinfo/p2p-hackers > _______________________________________________ > Here is a web page listing P2P Conferences: > http://www.neurogrid.net/twiki/bin/view/Main/PeerToPeerConferences > From dbarrett at quinthar.com Wed Mar 8 00:46:11 2006 From: dbarrett at quinthar.com (David Barrett) Date: Sat Dec 9 22:13:11 2006 Subject: [p2p-hackers] clustering In-Reply-To: <440D648E.302@gmail.com> Message-ID: <20060308004615.169FA3FD08@capsicum.zgp.org> I've been long interested in using synthetic coordinate systems to estimate the network distance (latency, throughput, etc) between any two nodes (even if they've never communicated directly) for purposes of clustering, but haven't really dug into it. Has anyone tried it in a real-world setting? -david > -----Original Message----- > From: p2p-hackers-bounces@zgp.org [mailto:p2p-hackers-bounces@zgp.org] On > Behalf Of Jeff Rose > Sent: Tuesday, March 07, 2006 2:47 AM > To: Peer-to-peer development. > Subject: [p2p-hackers] clustering > > Anyone out there doing work in clustering? I'd be interested in hearing > what people are up to, or if you have pointers to good papers that would > be great too. At this point I'm interested in any angle, geographic > distance based, semantic distance based, hierarchical, non-hierarchical > etc... My goal is to work towards a very amorphous clustering scheme > that allows any kind of object to be located close to relatives in the > network as long as they share a common distance function. > > Peace, > Jeff > _______________________________________________ > p2p-hackers mailing list > p2p-hackers@zgp.org > http://zgp.org/mailman/listinfo/p2p-hackers > _______________________________________________ > Here is a web page listing P2P Conferences: > http://www.neurogrid.net/twiki/bin/view/Main/PeerToPeerConferences From ian at locut.us Wed Mar 8 01:02:45 2006 From: ian at locut.us (Ian Clarke) Date: Sat Dec 9 22:13:11 2006 Subject: [p2p-hackers] clustering In-Reply-To: <440D648E.302@gmail.com> References: <440D648E.302@gmail.com> Message-ID: <5008189E-074B-43CC-A4F6-CEA901134073@locut.us> I think Chapters 1 and 2 of Oskar Sandberg's recent thesis may be extremely relevant: http://www.math.chalmers.se/~ossa/lic.pdf It outlines an algorithm to choose which nodes in a network should have edges between them such that "greedy" routing (routing to the peer closest to what you are looking for) has small world O( log^2 (N) ) path lengths. The algorithm is very simple, and pleasingly "natural". Basically you use greedy routing to find a path to your intended destination node, then you add a link from each node along this path to the destination. The number of outbound edges from each node can't obviously increase indefinitely, so they are deleted according to a least recently used scheme to make room for new edges. This is based on Freenet's approach to edge selection. The paper describes both experimental results that confirm that this algorithm does lead to a small world topology, and also delves into the mathematics behind why this might be the case. It is interesting not just because of the simplicity of the algorithm, but because it is extremely amenable to the construction and maintainence of decentralized P2P networks, and because it could tell us something about why human relationships tend to form small world networks. This algorithm is used by the Dijjer (http://dijjer.org/) P2P network. Ian. On 7 Mar 2006, at 02:46, Jeff Rose wrote: > Anyone out there doing work in clustering? I'd be interested in > hearing what people are up to, or if you have pointers to good > papers that would be great too. At this point I'm interested in > any angle, geographic distance based, semantic distance based, > hierarchical, non-hierarchical etc... My goal is to work towards a > very amorphous clustering scheme that allows any kind of object to > be located close to relatives in the network as long as they share > a common distance function. From networksimulator at gmail.com Wed Mar 8 02:17:23 2006 From: networksimulator at gmail.com (Ranus) Date: Sat Dec 9 22:13:11 2006 Subject: [p2p-hackers] clustering In-Reply-To: <440D648E.302@gmail.com> Message-ID: <000c01c64256$71f96bf0$ccc96fa6@thinkingfish> I ran into the idea of "small-world" last week and searched in the literature about it, there have been several papers using clustering to improve the system performance, and still more trying to understand the small-world networks. For P2P applications, Hui Zhang has published a = paper named "Using the Small-World Model to Improve Freenet Performance". It should correspond to your idea, so maybe you could read that. Actually I'm quite interested in the idea myself, in the p2p context, = the dynamics toward clustering is important... It should be efficient and simple, and self-organized. I'd be happy if we can have more = discussion. -- Ranus Yue Tsinghua University =B7=A2=BC=FE=C8=CB: p2p-hackers-bounces@zgp.org = [mailto:p2p-hackers-bounces@zgp.org] =B4=FA =B1=ED Jeff Rose =B7=A2=CB=CD=CA=B1=BC=E4: 2006=C4=EA3=D4=C27=C8=D5 18:47 =CA=D5=BC=FE=C8=CB: Peer-to-peer development. =D6=F7=CC=E2: [p2p-hackers] clustering Anyone out there doing work in clustering? I'd be interested in hearing what people are up to, or if you have pointers to good papers that would = be great too. At this point I'm interested in any angle, geographic = distance based, semantic distance based, hierarchical, non-hierarchical etc... = My goal is to work towards a very amorphous clustering scheme that allows = any kind of object to be located close to relatives in the network as long = as they share a common distance function. Peace, Jeff _______________________________________________ p2p-hackers mailing list p2p-hackers@zgp.org http://zgp.org/mailman/listinfo/p2p-hackers _______________________________________________ Here is a web page listing P2P Conferences: http://www.neurogrid.net/twiki/bin/view/Main/PeerToPeerConferences From lemonobrien at yahoo.com Wed Mar 8 02:59:54 2006 From: lemonobrien at yahoo.com (Lemon Obrien) Date: Sat Dec 9 22:13:11 2006 Subject: [p2p-hackers] clustering In-Reply-To: <000c01c64256$71f96bf0$ccc96fa6@thinkingfish> Message-ID: <20060308025954.84414.qmail@web53611.mail.yahoo.com> i read the book 'sync'; clustering is easy...at weblogic they udp to ping/pong each other...the same thing all peer-to-peer apps have to do to punch and keep open udp behind a firewall...so you have the messaging...within the pong or ping...sinc your session data...the smaller the better...when recieving a ping/pong and session data has changed...propagate that message with all other known peers...since all low class behind a firewall nodes only talk to relay nodes (aka any other cool name for a computer which is also used to relay messages to childern and to other relay station...leaf,super,open). The relay node will transform your net/mesh/grid's session...and or sessions as the french would say... Ranus wrote: I ran into the idea of "small-world" last week and searched in the literature about it, there have been several papers using clustering to improve the system performance, and still more trying to understand the small-world networks. For P2P applications, Hui Zhang has published a paper named "Using the Small-World Model to Improve Freenet Performance". It should correspond to your idea, so maybe you could read that. Actually I'm quite interested in the idea myself, in the p2p context, the dynamics toward clustering is important... It should be efficient and simple, and self-organized. I'd be happy if we can have more discussion. -- Ranus Yue Tsinghua University ??????: p2p-hackers-bounces@zgp.org [mailto:p2p-hackers-bounces@zgp.org] ?? ?? Jeff Rose ????????: 2006??3??7?? 18:47 ??????: Peer-to-peer development. ????: [p2p-hackers] clustering Anyone out there doing work in clustering? I'd be interested in hearing what people are up to, or if you have pointers to good papers that would be great too. At this point I'm interested in any angle, geographic distance based, semantic distance based, hierarchical, non-hierarchical etc... My goal is to work towards a very amorphous clustering scheme that allows any kind of object to be located close to relatives in the network as long as they share a common distance function. Peace, Jeff _______________________________________________ p2p-hackers mailing list p2p-hackers@zgp.org http://zgp.org/mailman/listinfo/p2p-hackers _______________________________________________ Here is a web page listing P2P Conferences: http://www.neurogrid.net/twiki/bin/view/Main/PeerToPeerConferences _______________________________________________ p2p-hackers mailing list p2p-hackers@zgp.org http://zgp.org/mailman/listinfo/p2p-hackers _______________________________________________ Here is a web page listing P2P Conferences: http://www.neurogrid.net/twiki/bin/view/Main/PeerToPeerConferences You don't get no juice unless you squeeze Lemon Obrien, the Third. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://zgp.org/pipermail/p2p-hackers/attachments/20060307/35e55d7e/attachment.html From garyjefferson123 at yahoo.com Wed Mar 8 03:43:39 2006 From: garyjefferson123 at yahoo.com (Gary Jefferson) Date: Sat Dec 9 22:13:11 2006 Subject: [p2p-hackers] clustering Message-ID: <20060308034339.54645.qmail@web35809.mail.mud.yahoo.com> Ranus wrote: > I ran into the idea of "small-world" last week and > searched in the literature about it, Just a general question for the list: DHTs don't exhibit small-world characteristics, at least not in the overlay network view of things, right? I mean, from each node's perspective, it might look a bit like a small world -- in Kademlia, for instance, my routing tables will center around my own ID space. But the network, as a whole, isn't small world, and isn't vulnerable to vertex-order attacks, right? Or am I missing something? Gary __________________________________________________ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com From agthorr at cs.uoregon.edu Wed Mar 8 04:55:52 2006 From: agthorr at cs.uoregon.edu (Daniel Stutzbach) Date: Sat Dec 9 22:13:11 2006 Subject: [p2p-hackers] clustering In-Reply-To: <20060308034339.54645.qmail@web35809.mail.mud.yahoo.com> References: <20060308034339.54645.qmail@web35809.mail.mud.yahoo.com> Message-ID: <20060308045551.GC5696@cs.uoregon.edu> On Tue, Mar 07, 2006 at 07:43:39PM -0800, Gary Jefferson wrote: > Just a general question for the list: DHTs don't > exhibit small-world characteristics, at least not in > the overlay network view of things, right? I mean, > from each node's perspective, it might look a bit like > a small world -- in Kademlia, for instance, my routing > tables will center around my own ID space. DHTs typically are small worlds. The small-world characteristics are short path lengths (comparable to a random graph) and high clustering (much more than a random graph). DHTs certainly have short path lengths (that's the point). All the DHTs I can think of also have high clustering. A node's neighbors are likely to also be neighbors. > But the network, as a whole, isn't small world, and isn't vulnerable > to vertex-order attacks, right? Or am I missing something? I believe you are thinking of power-law graphs (also called "scale-free") which have a few very-high-degree peers that make juicy targets. Power-law graphs frequently (but don't necessarily) exhibit small-world characteristics. Almost everyone is connected to some of the high-degree peers who are all connected together, so there is a lot of clustering. The high degree peers also provide short-paths everywhere. As a result, they're small worlds. However, not all small-worlds are power-law graphs. In particular, DHTs are not power-law graphs. There are not exceptionally high or low degree peers. -- Daniel Stutzbach Computer Science Ph.D Student http://www.barsoom.org/~agthorr University of Oregon From wesley at felter.org Wed Mar 8 03:55:26 2006 From: wesley at felter.org (Wes Felter) Date: Sat Dec 9 22:13:11 2006 Subject: [p2p-hackers] clustering In-Reply-To: <440D648E.302@gmail.com> References: <440D648E.302@gmail.com> Message-ID: <1F413594-2AC1-4FBE-A25D-08109D7F5FFC@felter.org> On Mar 7, 2006, at 4:46 AM, Jeff Rose wrote: > Anyone out there doing work in clustering? I'd be interested in > hearing what people are up to, or if you have pointers to good > papers that would be great too. At this point I'm interested in > any angle, geographic distance based, semantic distance based, > hierarchical, non-hierarchical etc... My goal is to work towards a > very amorphous clustering scheme that allows any kind of object to > be located close to relatives in the network as long as they share > a common distance function. IIRC, OASIS uses some sophisticated clustering under the hood. http:// www.coralcdn.org/oasis/ Wes Felter - wesley@felter.org - http://felter.org/wesley/ From networksimulator at gmail.com Wed Mar 8 06:08:20 2006 From: networksimulator at gmail.com (Ranus) Date: Sat Dec 9 22:13:11 2006 Subject: =?gb2312?B?tPC4tDogW3AycC1oYWNrZXJzXSBjbHVzdGVyaW5n?= In-Reply-To: <20060308045551.GC5696@cs.uoregon.edu> Message-ID: <000601c64276$b5b267a0$ccc96fa6@thinkingfish> Well... Small-world is characterized by both a high clustering degree = and a short average path length. While DHTs are designed to have short paths = from node to node, they do not necessarily present clustering.=20 I just thought about Chord... The nodes keep track of neighbors every = 2^n steps away. I don't think the nodes are clustering here, or am I wrong somewhere? -- Ranus Yue Tsinghua University -----=D3=CA=BC=FE=D4=AD=BC=FE----- =B7=A2=BC=FE=C8=CB: p2p-hackers-bounces@zgp.org = [mailto:p2p-hackers-bounces@zgp.org] =B4=FA =B1=ED Daniel Stutzbach =B7=A2=CB=CD=CA=B1=BC=E4: 2006=C4=EA3=D4=C28=C8=D5 12:56 =CA=D5=BC=FE=C8=CB: p2p-hackers@zgp.org =D6=F7=CC=E2: Re: [p2p-hackers] clustering On Tue, Mar 07, 2006 at 07:43:39PM -0800, Gary Jefferson wrote: > Just a general question for the list: DHTs don't exhibit small-world=20 > characteristics, at least not in the overlay network view of things,=20 > right? I mean, from each node's perspective, it might look a bit like = > a small world -- in Kademlia, for instance, my routing tables will=20 > center around my own ID space. DHTs typically are small worlds. The small-world characteristics are short path lengths (comparable to a random graph) and high clustering (much more than a random graph). DHTs certainly have short path lengths (that's the point). All the DHTs = I can think of also have high clustering. A node's neighbors are likely = to also be neighbors. > But the network, as a whole, isn't small world, and isn't vulnerable=20 > to vertex-order attacks, right? Or am I missing something? I believe you are thinking of power-law graphs (also called "scale-free") which have a few very-high-degree peers that make juicy targets. Power-law graphs frequently (but don't necessarily) exhibit small-world characteristics. Almost everyone is connected to some of = the high-degree peers who are all connected together, so there is a lot of clustering. The high degree peers also provide short-paths everywhere. = As a result, they're small worlds. However, not all small-worlds are power-law graphs. In particular, DHTs = are not power-law graphs. There are not exceptionally high or low degree = peers. --=20 Daniel Stutzbach Computer Science Ph.D Student http://www.barsoom.org/~agthorr University of Oregon _______________________________________________ p2p-hackers mailing list p2p-hackers@zgp.org http://zgp.org/mailman/listinfo/p2p-hackers _______________________________________________ Here is a web page listing P2P Conferences: http://www.neurogrid.net/twiki/bin/view/Main/PeerToPeerConferences From eugen at leitl.org Wed Mar 8 07:57:03 2006 From: eugen at leitl.org (Eugen Leitl) Date: Sat Dec 9 22:13:11 2006 Subject: [p2p-hackers] clustering In-Reply-To: <20060308004615.169FA3FD08@capsicum.zgp.org> References: <440D648E.302@gmail.com> <20060308004615.169FA3FD08@capsicum.zgp.org> Message-ID: <20060308075703.GM25017@leitl.org> On Tue, Mar 07, 2006 at 04:46:11PM -0800, David Barrett wrote: > I've been long interested in using synthetic coordinate systems to estimate > the network distance (latency, throughput, etc) between any two nodes (even > if they've never communicated directly) for purposes of clustering, but > haven't really dug into it. Has anyone tried it in a real-world setting? Has routing been converging towards geography lately, or is this an urban myth? -- Eugen* Leitl leitl http://leitl.org ______________________________________________________________ ICBM: 48.07100, 11.36820 http://www.ativel.com 8B29F6BE: 099D 78BA 2FD3 B014 B08A 7779 75B0 2443 8B29 F6BE -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: Digital signature Url : http://zgp.org/pipermail/p2p-hackers/attachments/20060308/e7555690/attachment.pgp From m.rogers at cs.ucl.ac.uk Wed Mar 8 08:35:54 2006 From: m.rogers at cs.ucl.ac.uk (Michael Rogers) Date: Sat Dec 9 22:13:11 2006 Subject: [p2p-hackers] clustering In-Reply-To: <20060308034339.54645.qmail@web35809.mail.mud.yahoo.com> References: <20060308034339.54645.qmail@web35809.mail.mud.yahoo.com> Message-ID: <440E976A.8010906@cs.ucl.ac.uk> Gary Jefferson wrote: > Just a general question for the list: DHTs don't > exhibit small-world characteristics, at least not in > the overlay network view of things, right? That depends on your definition of 'small world'. Most people take it to mean short paths and high clustering, but as far as I know the Freenet work uses a more specific definition due to Jon Kleinberg: a graph is a small world if it can be embedded in a metric space such that the length of the edges follows a power law distribution, where the magnitude of the power law exponent is equal to the number of dimensions in the metric space[1]. Kleinberg showed that greedy routing in such a graph is efficient, but if the power law exponent has any other value then greedy routing is inefficient[2,3]. However, it's also possible that the length distribution doesn't follow a power law at all (eg Chord, where the length distribution is exponential and greedy routing is efficient). Most DHTs aren't small worlds in the sense used in the Freenet work; Symphony[4] is an exception. Cheers, Michael [1] http://www.cs.cornell.edu/home/kleinber/nips14.ps [2] http://www.cs.cornell.edu/home/kleinber/nat00.pdf [3] http://www.cs.cornell.edu/home/kleinber/swn.pdf [4] http://www-db.stanford.edu/~bawa/Pub/symphony.pdf From networksimulator at gmail.com Wed Mar 8 16:38:58 2006 From: networksimulator at gmail.com (Ranus) Date: Sat Dec 9 22:13:11 2006 Subject: [p2p-hackers] clustering In-Reply-To: <440E976A.8010906@cs.ucl.ac.uk> Message-ID: <001401c642ce$cf4db3b0$ccc96fa6@thinkingfish> About your previous mail: I checked the definition of clustering coefficient, that of a chord vertex is 3/(n-2) (for a 2^n node ring), and it's the same for every vertex, so the overall clustering coefficient is not high when n is moderately large. The definition of small-world is indeed clustering and short path. Well, the network defined by Kleinberg falls into the category of "scale-free networks". In such networks there are a few nodes that connect to abundant other nodes. Following the power law distribution of degrees is one way to construct scale-free networks, which are also small-world networks. But there could be scale-free networks that are not small worlds. (E.g, two star networks connected at the centers, here the clustering coefficient is 0 because there's no triangle) On the other hand, a very famous illustration of a small world is a regular lattice with a few extra, randomly chosen edges. This does not conform the power law distribution of edges and there's no supernode, but it's a small world. Am I going too far from the point? Better turn around here... The scale-free networks seem quite appealing as performance and scalability are concerned, and it's simpler to guarantee short paths with powerful supernodes. Many P2P applications such as Gia (Making Gnutella-like P2P Systems Scalable, Sigcomm'03) introduce such supernodes to improve the scalability. But it seems harder, to build scalable networks with more or less the same number of edges for each node (this could be realistic when edges are interpreted as connections, rather than links/routing information), because at such time the supernodes are no where to be found. I think some work could be done here... what do you suggest? -- Ranus Yue Tsinghua University -----邮件原件----- 发件人: p2p-hackers-bounces@zgp.org [ mailto:p2p-hackers-bounces@zgp.org] 代表 Michael Rogers 发送时间: 2006年3月8日 16:36 收件人: Peer-to-peer development. 主题: Re: [p2p-hackers] clustering Gary Jefferson wrote: > Just a general question for the list: DHTs don't exhibit small-world > characteristics, at least not in the overlay network view of things, > right? That depends on your definition of 'small world'. Most people take it to mean short paths and high clustering, but as far as I know the Freenet work uses a more specific definition due to Jon Kleinberg: a graph is a small world if it can be embedded in a metric space such that the length of the edges follows a power law distribution, where the magnitude of the power law exponent is equal to the number of dimensions in the metric space[1]. Kleinberg showed that greedy routing in such a graph is efficient, but if the power law exponent has any other value then greedy routing is inefficient[2,3]. However, it's also possible that the length distribution doesn't follow a power law at all (eg Chord, where the length distribution is exponential and greedy routing is efficient). Most DHTs aren't small worlds in the sense used in the Freenet work; Symphony[4] is an exception. Cheers, Michael [1] http://www.cs.cornell.edu/home/kleinber/nips14.ps [2] http://www.cs.cornell.edu/home/kleinber/nat00.pdf [3] http://www.cs.cornell.edu/home/kleinber/swn.pdf [4] http://www-db.stanford.edu/~bawa/Pub/symphony.pdf _______________________________________________ p2p-hackers mailing list p2p-hackers@zgp.org http://zgp.org/mailman/listinfo/p2p-hackers _______________________________________________ Here is a web page listing P2P Conferences: http://www.neurogrid.net/twiki/bin/view/Main/PeerToPeerConferences -------------- next part -------------- An HTML attachment was scrubbed... URL: http://zgp.org/pipermail/p2p-hackers/attachments/20060309/9bf072ff/attachment.html From agthorr at cs.uoregon.edu Wed Mar 8 17:15:36 2006 From: agthorr at cs.uoregon.edu (Daniel Stutzbach) Date: Sat Dec 9 22:13:11 2006 Subject: [p2p-hackers] clustering In-Reply-To: <001401c642ce$cf4db3b0$ccc96fa6@thinkingfish> References: <440E976A.8010906@cs.ucl.ac.uk> <001401c642ce$cf4db3b0$ccc96fa6@thinkingfish> Message-ID: <20060308171535.GI5696@cs.uoregon.edu> On Thu, Mar 09, 2006 at 12:38:58AM +0800, Ranus wrote: > About your previous mail: I checked the definition of clustering > coefficient, that of a > chord vertex is 3/(n-2) (for a 2^n node ring), and it's the same for every > vertex, so the overall clustering coefficient is not high when n is > moderately large. That's actually a very larger clustering coefficient, compared to random graph with the same number of nodes and edges. An Erdos-Renyi random graph (the standard random graph) every edge exists with probability |E| / |V|^2. Consequently, the clustering coefficient is approximately |E| / |V|^2. In Chord, |E| = |V|*lg |V|, so we have a clustering coefficient in the comparable random graph of lg |V| / |V|. In your calculation, you're using n=2^|V|, so the clustering coefficient is 3/(lg |V| - 2). This is an enormous clustering coefficient. For |V| = 1 million, the clustering coefficient is 17%! For a comparable random graph, it's only 0.002%. (I use "lg x" to mean "log base 2 of x") > The scale-free networks seem quite appealing as performance and scalability > are concerned, and it's simpler to guarantee short paths with powerful > supernodes. Scale-free networks scale well in some ways and scale very badly in other ways. Yes, they maintain short path lengths as the network gets bigger. However, they require that you have these increasingly high-degree peers as the network gets bigger. You may have some peers that have more capacity than others, and it may make sense to utilize those resources, but it does not seem wise to assume that if your network grows by a factor of 100 that you will be able to a user who has 100 times more resources than your previous best-user. > Many P2P applications such as Gia (Making Gnutella-like P2P > Systems Scalable, Sigcomm'03) introduce such supernodes to improve the > scalability. But it seems harder, to build scalable networks with more or > less the same number of edges for each node (this could be realistic when > edges are interpreted as connections, rather than links/routing > information), because at such time the supernodes are no where to be found. It largely depends on what you're trying to achieve. For a network like Gnutella, global visibility is not one of the design requirements, so it's OK to use a constant number of edges. DHTs (typically) use O(log |V|) edges to guarantee a worst-case of O(log |V|) steps per lookup. That seems like a good compromise to me. -- Daniel Stutzbach Computer Science Ph.D Student http://www.barsoom.org/~agthorr University of Oregon From Pietro.Michiardi at eurecom.fr Wed Mar 8 18:31:38 2006 From: Pietro.Michiardi at eurecom.fr (Pietro Michiardi) Date: Sat Dec 9 22:13:11 2006 Subject: [p2p-hackers] PhD in networked and distributed systems, Institut Eurecom Message-ID: <440F230A.7030906@eurecom.fr> PhD position opening Networking and Security Dept. Institut Eurecom The Networking and Security Department of Institut EURECOM is seeking candidates with a background in networked and distributed systems. Candidates should have a MsC in Computer Science and demonstrate exceptional research potential. The successful candidate is expected to carry out research in an area related to peer-to-peer and content distribution systems; wireless and mobile systems (including ad hoc networking). Additional skills in cooperation issues in self-organizing systems would be a plus. Candidates should also have experience in programming in C/C++; using MATLAB; network simulation environments (ns, glomosim, ...). The position is available immediately. The selection process will begin immediately upon receipt of applications. To apply, send resume, a motivation letter and e-mail addresses of at least one reference to: Pietro.Michiardi@eurecom.fr Please, use the following subject: [EUR-CAS:PHD]. -------------- next part -------------- A non-text attachment was scrubbed... Name: Pietro.Michiardi.vcf Type: text/x-vcard Size: 317 bytes Desc: not available Url : http://zgp.org/pipermail/p2p-hackers/attachments/20060308/2b839362/Pietro.Michiardi.vcf From Pietro.Michiardi at eurecom.fr Wed Mar 8 18:32:24 2006 From: Pietro.Michiardi at eurecom.fr (Pietro Michiardi) Date: Sat Dec 9 22:13:11 2006 Subject: [p2p-hackers] Postdoc in networked and distributed systems, Institut Eurecom Message-ID: <440F2338.4090900@eurecom.fr> Post-Doctoral Research Fellowship Networking and Security Dept. Institut Eurecom Duration: 12 months (minimum) up to 18 months The Networking and Security Department of Institut EURECOM is seeking a post-doctoral fellow with a strong research background in networked and distributed systems. Candidates should have a PhD in Computer Science or equivalent and demonstrate exceptional research potential as well as basic project management skills. The successful candidate is expected work in the context of a funded European Project and to carry out research in an area related to peer-to-peer and content distribution systems; wireless and mobile systems (including ad hoc networking). Additional skills in game theoretical modeling of networks; network modeling; graph theory are highly recommended. This position also involves some project management. The successful applicant is expected to represent EURECOM at the project level, participate in technical meetings in European locations and handle interactions with the partners of EU project. The position is available immediately. The selection process will begin immediately upon receipt of applications. To apply, send resume, a statement of research and e-mail addresses of at least three references to: Pietro.Michiardi@eurecom.fr Please, use the following subject: [EUR-CAS:POSTDOC]. -------------- next part -------------- A non-text attachment was scrubbed... Name: Pietro.Michiardi.vcf Type: text/x-vcard Size: 317 bytes Desc: not available Url : http://zgp.org/pipermail/p2p-hackers/attachments/20060308/baad6004/Pietro.Michiardi.vcf From m.rogers at cs.ucl.ac.uk Wed Mar 8 18:36:09 2006 From: m.rogers at cs.ucl.ac.uk (Michael Rogers) Date: Sat Dec 9 22:13:11 2006 Subject: [p2p-hackers] clustering In-Reply-To: <001401c642ce$cf4db3b0$ccc96fa6@thinkingfish> References: <001401c642ce$cf4db3b0$ccc96fa6@thinkingfish> Message-ID: <440F2419.9030807@cs.ucl.ac.uk> Ranus wrote: > Well, the network defined by Kleinberg falls into the category of > "scale-free networks". Sorry to contradict you, but a scale-free network has a power law degree distribution. A Kleinberg small world has a power law length distribution, and there are no restrictions on the degree distribution[1]. > The scale-free networks seem quite appealing as performance and > scalability are concerned, and it's simpler to guarantee short paths > with powerful supernodes. I've come across a couple of papers suggesting scale-free topologies for P2P networks, but I'm a bit skeptical. Scale-free graphs typically have O(log n) diameter, so if you flood O(sqrt n) advertisements from any node and O(sqrt n) queries from any other node then the query is likely to find the advertisement, but on the other hand the same's true of random graphs, which don't depend on a small fraction of the nodes handling a large fraction of the messages... I guess it's a question of finding a degree distribution that matches the bandwidth distribution of the nodes. Maybe that's power law, maybe not... does anyone have any figures? Cheers, Michael [1] http://fleece.ucsd.edu/~massimo/Journal/SWorld-Submission.pdf From Pietro.Michiardi at eurecom.fr Wed Mar 8 18:47:59 2006 From: Pietro.Michiardi at eurecom.fr (Pietro Michiardi) Date: Sat Dec 9 22:13:11 2006 Subject: [p2p-hackers] PhD in networked and distributed systems, Institut Eurecom Message-ID: <440F26DF.4060406@eurecom.fr> (Apologies for the previous barely readable posting) PhD position opening Networking and Security Dept. Institut Eurecom The Networking and Security Department of Institut EURECOM is seeking candidates with a background in networked and distributed systems. Candidates should have a MsC in Computer Science and demonstrate exceptional research potential. The successful candidate is expected to carry out research in an area related to peer-to-peer and content distribution systems; wireless and mobile systems (including ad hoc networking). Additional skills in cooperation issues in self-organizing systems would be a plus. Candidates should also have experience in programming in C/C++; using MATLAB; network simulation environments (ns, glomosim, ...). The position is available immediately. The selection process will begin immediately upon receipt of applications. To apply, send resume, a motivation letter and e-mail addresses of at least one reference to: Pietro.Michiardi@eurecom.fr Please, use the following subject: [EUR-CAS:PHD]. -------------- next part -------------- A non-text attachment was scrubbed... Name: Pietro.Michiardi.vcf Type: text/x-vcard Size: 317 bytes Desc: not available Url : http://zgp.org/pipermail/p2p-hackers/attachments/20060308/70a9b87e/Pietro.Michiardi.vcf From Pietro.Michiardi at eurecom.fr Wed Mar 8 18:53:19 2006 From: Pietro.Michiardi at eurecom.fr (Pietro Michiardi) Date: Sat Dec 9 22:13:11 2006 Subject: [p2p-hackers] Postdoc in networked and distributed systems, Institut Eurecom Message-ID: <440F281F.9050908@eurecom.fr> (Apologies for the previous barely readable posting) Post-Doctoral Research Fellowship Networking and Security Dept. Institut Eurecom Duration: 12 months (minimum) up to 18 months The Networking and Security Department of Institut EURECOM is seeking a post-doctoral fellow with a strong research background in networked and distributed systems. Candidates should have a PhD in Computer Science or equivalent and demonstrate exceptional research potential as well as basic project management skills. The successful candidate is expected work in the context of a funded European Project and to carry out research in an area related to peer-to-peer and content distribution systems; wireless and mobile systems (including ad hoc networking). Additional skills in game theoretical modeling of networks; network modeling; graph theory are highly recommended. This position also involves some project management. The successful applicant is expected to represent EURECOM at the project level, participate in technical meetings in European locations and handle interactions with the partners of EU project. The position is available immediately. The selection process will begin immediately upon receipt of applications. To apply, send resume, a statement of research and e-mail addresses of at least three references to: Pietro.Michiardi@eurecom.fr Please, use the following subject: [EUR-CAS:POSTDOC] -------------- next part -------------- A non-text attachment was scrubbed... Name: Pietro.Michiardi.vcf Type: text/x-vcard Size: 317 bytes Desc: not available Url : http://zgp.org/pipermail/p2p-hackers/attachments/20060308/616cb50c/Pietro.Michiardi.vcf From ian at locut.us Thu Mar 9 03:19:33 2006 From: ian at locut.us (Ian Clarke) Date: Sat Dec 9 22:13:11 2006 Subject: [p2p-hackers] clustering In-Reply-To: <000c01c64256$71f96bf0$ccc96fa6@thinkingfish> References: <440D648E.302@gmail.com> <000c01c64256$71f96bf0$ccc96fa6@thinkingfish> Message-ID: <823242bd0603081919h7717ccecyad22dff89e93f0a4@mail.gmail.com> On 3/7/06, Ranus wrote: > > Hui Zhang has published a paper > named "Using the Small-World Model to Improve Freenet Performance". It > should correspond to your idea, so maybe you could read that. Be careful of this paper. If I recall correctly, most of their results can be attributed to the fact that they ensured that links existed between adjacent nodes in the graph, which obviously would have a dramatic beneficial effect relative to a network where local links may be missing as it means that in the worst case you will do an exhaustive search for the node you are looking for just by following local links. Our findings, as presented in Oskar's thesis, are that Freenet-style edge selection results in the desired degree of clustering without "artificial" help. Ian. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://zgp.org/pipermail/p2p-hackers/attachments/20060308/07c28188/attachment.htm From networksimulator at gmail.com Thu Mar 9 03:24:39 2006 From: networksimulator at gmail.com (Ranus) Date: Sat Dec 9 22:13:11 2006 Subject: [p2p-hackers] clustering In-Reply-To: <20060308171535.GI5696@cs.uoregon.edu> Message-ID: <002301c64329$09f29210$ccc96fa6@thinkingfish> Daniel: You are right on Chord. Thanks for pointing that out. Michael: No universally accepted definition of the scale-free network is available now. The definition of "scale-free metric" does not require a power-law degree distribution, but that's 99% the case in available research and you can say it's the only case that matters. Some deviation in real life case is reported (as the degree cannot grow ultimately)[1]. Many DHTs provide good connectivity but do not require global visibility, and usually there's not much need in adding it. We do not really care if it's scale free or not, we only care if we can find what we want. Under current condition you cannot really feel the difference when you're in this part of the overlay network or another, there's no way to measure. But sometimes you do want to connect to certain nodes more than others. Then starting at a random point and having no idea of the whole picture, how to find the right part you belong quickly? This does not always mean global visibility is required, though that does settle the problem. There could be other approaches, esp. when the network is large and decentralized. So it's not only a matter of join and stay somewhere, but choose the right cluster as well. Could we call this "heuristic clustering"? Again, the question fall back to the very first one: who do you call as your 'relatives' and how to ask for them from your current neighbors? [1] http://www.santafe.edu/research/publications/workingpapers/01-03-021.pdf -- Ranus Yue Tsinghua University > -----Original Message----- > From: p2p-hackers-bounces@zgp.org > [mailto:p2p-hackers-bounces@zgp.org] On Behalf Of Daniel Stutzbach > Sent: Thursday, March 09, 2006 1:16 AM > To: 'Peer-to-peer development.' > Subject: Re: [p2p-hackers] clustering > > That's actually a very larger clustering coefficient, > compared to random graph with the same number of nodes and edges. > > An Erdos-Renyi random graph (the standard random graph) every > edge exists with probability |E| / |V|^2. Consequently, the > clustering coefficient is approximately |E| / |V|^2. In > Chord, |E| = |V|*lg |V|, so we have a clustering coefficient > in the comparable random graph of lg |V| / |V|. > > In your calculation, you're using n=2^|V|, so the clustering > coefficient is 3/(lg |V| - 2). This is an enormous > clustering coefficient. For |V| = 1 million, the clustering > coefficient is 17%! > For a comparable random graph, it's only 0.002%. > > (I use "lg x" to mean "log base 2 of x") > > Scale-free networks scale well in some ways and scale very > badly in other ways. Yes, they maintain short path lengths > as the network gets bigger. However, they require that you > have these increasingly high-degree peers as the network gets > bigger. > > You may have some peers that have more capacity than others, > and it may make sense to utilize those resources, but it does > not seem wise to assume that if your network grows by a > factor of 100 that you will be able to a user who has 100 > times more resources than your previous best-user. > > It largely depends on what you're trying to achieve. For a > network like Gnutella, global visibility is not one of the > design requirements, so it's OK to use a constant number of edges. > > DHTs (typically) use O(log |V|) edges to guarantee a > worst-case of O(log |V|) steps per lookup. That seems like a > good compromise to me. > > -- > Daniel Stutzbach Computer Science > Ph.D Student > http://www.barsoom.org/~agthorr > University of Oregon > _______________________________________________ > p2p-hackers mailing list > p2p-hackers@zgp.org > http://zgp.org/mailman/listinfo/p2p-hackers > _______________________________________________ > Here is a web page listing P2P Conferences: > http://www.neurogrid.net/twiki/bin/view/Main/PeerToPeerConferences From ossa at math.chalmers.se Thu Mar 9 08:14:14 2006 From: ossa at math.chalmers.se (Oskar Sandberg) Date: Sat Dec 9 22:13:11 2006 Subject: [p2p-hackers] clustering In-Reply-To: <440F2419.9030807@cs.ucl.ac.uk> References: <001401c642ce$cf4db3b0$ccc96fa6@thinkingfish> <440F2419.9030807@cs.ucl.ac.uk> Message-ID: <440FE3D6.6010002@math.chalmers.se> Michael Rogers wrote: > Ranus wrote: > >> Well, the network defined by Kleinberg falls into the category of >> "scale-free networks". > > > Sorry to contradict you, but a scale-free network has a power law degree > distribution. A Kleinberg small world has a power law length > distribution, and there are no restrictions on the degree distribution[1]. If you mean by "Kleinberg small world" the model that Jon Kleinberg proposed for navigable networks, then this is not the case. That model has directed "long-range" edges, with a fixed out-degree and Poisson in-degree. While making the out-degree arbitrary is easy, changing the in-degrees while retaining the mathematical results would be non-trivial. I'm not sure if anybody has done it. // oskar From ossa at math.chalmers.se Thu Mar 9 08:23:33 2006 From: ossa at math.chalmers.se (Oskar Sandberg) Date: Sat Dec 9 22:13:11 2006 Subject: [p2p-hackers] clustering In-Reply-To: <440E976A.8010906@cs.ucl.ac.uk> References: <20060308034339.54645.qmail@web35809.mail.mud.yahoo.com> <440E976A.8010906@cs.ucl.ac.uk> Message-ID: <440FE605.8060605@math.chalmers.se> Michael Rogers wrote: > However, it's also possible that the length > distribution doesn't follow a power law at all (eg Chord, where the > length distribution is exponential and greedy routing is efficient). Actually, while the frequency of Chord links falls exponentially with the "level" (not sure what the Chord term is) the length of such links increases exponentially as well, so in fact the frequency of links with certain lengths do fall harmonically. One could see Chord as some sort of "mean field" version of the same dynamics as Kleinberg's model. > Most DHTs aren't small worlds in the sense used in the Freenet work; > Symphony[4] is an exception. All sensible DHTs are small-world networks. If our definition of the term doesn't imply this, we are getting lost in semantics. // oskar From m.rogers at cs.ucl.ac.uk Thu Mar 9 11:43:05 2006 From: m.rogers at cs.ucl.ac.uk (Michael Rogers) Date: Sat Dec 9 22:13:11 2006 Subject: [p2p-hackers] clustering In-Reply-To: <440FE3D6.6010002@math.chalmers.se> References: <001401c642ce$cf4db3b0$ccc96fa6@thinkingfish> <440F2419.9030807@cs.ucl.ac.uk> <440FE3D6.6010002@math.chalmers.se> Message-ID: <441014C9.1090500@cs.ucl.ac.uk> Oskar Sandberg wrote: > If you mean by "Kleinberg small world" the model that Jon Kleinberg > proposed for navigable networks, then this is not the case. That model > has directed "long-range" edges, with a fixed out-degree and Poisson > in-degree. True, that's why I linked to the Franceschetti & Meester paper which generalises Kleinberg's model to any out-degree distribution. > While making the out-degree arbitrary is easy, changing the > in-degrees while retaining the mathematical results would be > non-trivial. I'm not sure if anybody has done it. That's interesting - would you mind elaborating? As long as the distribution of edge lengths is right and the edges are independent, why does the in-degree matter? Cheers, Michael From ossa at math.chalmers.se Thu Mar 9 11:55:30 2006 From: ossa at math.chalmers.se (Oskar Sandberg) Date: Sat Dec 9 22:13:11 2006 Subject: [p2p-hackers] clustering In-Reply-To: <441014C9.1090500@cs.ucl.ac.uk> References: <001401c642ce$cf4db3b0$ccc96fa6@thinkingfish> <440F2419.9030807@cs.ucl.ac.uk> <440FE3D6.6010002@math.chalmers.se> <441014C9.1090500@cs.ucl.ac.uk> Message-ID: <441017B2.4080306@math.chalmers.se> Michael Rogers wrote: > Oskar Sandberg wrote: >> While making the out-degree arbitrary is easy, changing the >> in-degrees while retaining the mathematical results would be >> non-trivial. I'm not sure if anybody has done it. > > > That's interesting - would you mind elaborating? As long as the > distribution of edge lengths is right and the edges are independent, why > does the in-degree matter? I doubt it matters to the results, but just off the top of my head I would expect it to complicate things in the proofs because it means that lengths of the long range links are not purely independent at each step. There are probably conditioning arguments to get around it, but it is still non-trivial (for some value of trivial). // oskar From jacob at mungo.dk Thu Mar 9 20:10:02 2006 From: jacob at mungo.dk (Jacob Madsen) Date: Sat Dec 9 22:13:11 2006 Subject: [p2p-hackers] Structured P2P Networks Message-ID: <44108B9A.5040107@mungo.dk> Hey, I'm writing a small paper about DHTs and it got me wondering... I can't recall that I've ever came across other kind of structured P2P networks than DHT. Maybe one of you guys know about other kind of structured P2P networks? Thanks! From dbarrett at quinthar.com Thu Mar 9 21:27:15 2006 From: dbarrett at quinthar.com (David Barrett) Date: Sat Dec 9 22:13:11 2006 Subject: [p2p-hackers] Practical clustering Message-ID: <20060309212720.AB6063FC50@capsicum.zgp.org> So I?ve enjoyed the discussion on the mathematical foundation and taxonomy of networks vis-?-vis clustering, but what techniques have you found useful on a more nuts-and-bolts level? It seems to me the clustering problem can be broken down into (roughly): 1) Measuring the ?distance? between any two nodes: How have you defined ?distance? in this context: physical distance, latency, network hops, throughput, etc? And for your selection, what techniques have you used to measure it (IP geolocation, synthetic coordinate systems, embedded traceroute, network monitoring, etc)? 2) Measuring the ?strength? of any given node: How have you defined ?strength? in this context: uptime, CPU, memory, upload capacity, download capacity, etc? How have you measured these? 3) Join algorithm: When a new node comes into the network, how do you determine its placement? Is it a top-down, recursive approach, or a bottom-up, emergent approach? 4) Optimization algorithm: As new nodes come and go (or even as a ?background process?) how do you optimize the network by promoting and demoting nodes, and on what basis? 5) Repair algorithm: Given that any node can disappear at any time without warning, how do you mitigate the effect of this and clean up afterwards? And not all clustering need be in a network sense. With iGlance, for example, I have a ?video buddy list? feature where you have a super low-bandwidth video stream to all your peers. My initial thought for this was to have each client build up a ?pose library? in order to optimize for the massive similarity between frames in a video stream of you sitting in front of your computer. For example, there?s the ?looking at screen? pose and ?talking on phone? pose, and ?away from computer? pose, and so on. My goal was to accumulate this pose library over time, and then just send out indices into this library. I?d classify the clustering algorithm I used for my pose library generation as follows: 1) Distance: Use an image differencing function that roughly equated to ?number of pixels changed? (it was more sophisticated than this, but not by much) 2) Strength: If a new image coming in is within some tolerance of an existing image, just increment the ?strength? of the old image and discard the new. Thus we?d gradually identify which images were most common, and call those the strongest poses. 3) Join algorithm: Trickle a new image down from top to bottom. At each node, if it?s within tolerance, discard and increment its strength. If outside tolerance, compare to children and pick whichever is within second tolerance. If none are, call it a new child. 4) Optimization algorithm: Set a fixed limit on the number of nodes allowed in the tree (based on how much memory I was willing to allocate to the problem) and ? before a node creates a new leaf ? discard the weakest child branch if we?re at our limit. I think there was also a fan-out limit and when overrun, I?d measure the distances between all children, find the pair that was closest, and then demote the weaker under the stronger. 5) Repair algorithm: Not applicable to this problem because images didn?t just disappear without warning. Anyway, so that?s a clustering approach I used, and it worked surprisingly well. The big problems were it had too many ?magic numbers? (merge threshold, split threshold, total node numbers, etc) and it was hard choosing the right values. That and at the end of the day, the number of poses you have even staring at your computer is actually quite high, and simply sending a 1FPS video feed creates a better experience, is easier to implement, and is sufficiently low bandwidth for my needs. I?m sure with enough tweaking it could be made to work and support a huge number of peers, but meh. Work for a future day. Assuming I even kept the source, when it?d be just like me to throw it away. So, that?s my real-world clustering story. What story do you have, and what lessons did you learn? -david -------------- next part -------------- An HTML attachment was scrubbed... URL: http://zgp.org/pipermail/p2p-hackers/attachments/20060309/7f9a4a7c/attachment.html From m.rogers at cs.ucl.ac.uk Thu Mar 9 22:15:01 2006 From: m.rogers at cs.ucl.ac.uk (Michael Rogers) Date: Sat Dec 9 22:13:11 2006 Subject: [p2p-hackers] Structured P2P Networks In-Reply-To: <44108B9A.5040107@mungo.dk> References: <44108B9A.5040107@mungo.dk> Message-ID: <4410A8E5.1050303@cs.ucl.ac.uk> Hi Jacob, The only ones I've come across are Freedman and Vingralek's distributed trie (http://www.cs.rice.edu/Conferences/IPTPS02/167.pdf) and Law and Siu's random expander networks (http://www.ieee-infocom.org/2003/papers/52_02.PDF). Cheers, Michael From sam at neurogrid.com Fri Mar 10 02:05:16 2006 From: sam at neurogrid.com (Sam Joseph) Date: Sat Dec 9 22:13:11 2006 Subject: [p2p-hackers] AP2PC '06 Call for Participation Message-ID: <4410DEDC.3040306@neurogrid.com> **** list of accepted papers available **** early registration deadline: 10 March 2006 **** ********************************************************************* Fifth International Workshop on Agents and Peer-to-Peer Computing (AP2PC 2006) 9 May 2006 Future University Hakodate, Japan (held in conjunction with AAMAS 2006) URL: http://p2p.ingce.unibo.it/ ********************************************************************* CALL FOR PARTICIPATION ********************************************************************* The workshop on Agents and Peer-to-Peer Computing ( AP2PC), in its fifth edition this year, is a well-established forum for researchers interested in sharing their experiences in combining peer-to-peer based approaches those of agents and multiagent systems. AP2PC 2006 will be held as a satellite workshop of AAMAS 2006, the 5th Inter- national Joint Conference on Autonomous Agents and Multiagent Systems in May 2006 in Hakodate, Japan. The workshop programme will consist of presentations of several contributed papers a panel and an invited speaker. Everybody with an interest in the crossover between peer-to-peer and agent systems is cordially invited to attend. ********************************************************************* REGISTRATION ********************************************************************* Registration will be handled by the AAMAS organisers. Information is available at the AAMAS registration page: http://www.fun.ac.jp/aamas2006/registration.html Please note the deadline for early registration is already upon us: *** 10 March 2006 *** ********************************************************************* ACCEPTED PAPERS ********************************************************************* The PC has selected 10 full papers and 6 short papers for presentation at the workshop: *** Long Papers *** * Cooperative CBR System for Peer Agent Committee Formation Hager Karoui, Rushed Kanawati & Laure Petrucci * Hybrid DHT Design for Mobile Environments Stefan Zoels, Simon Schubert, Wolfgang Kellerer & Zoran Despotovic * Mitigating the Impact of Liars by Reflecting Peer's Credibility on P2P File Reputation Systems Soyoung Lee, O-Hoon Kwon, Jong Kim & Sung Je Hong * A Comparative Study of Reasoning Techniques for Service Selection Murat Sensoy & Pinar Yolum * Chora: Expert-based P2P Web Search Halldor Gylfason, Omar Khan & Grant Schoenebeck * K-link: A Peer-to-Peer Solution for Organizational Knowledge Management Giuseppe Pirro', Domenico Talia & Massimo Ruffolo * An Analysis of Interest Community Facilitated P2P Search Elth Ogston * Mobile agent-based approach for resource discovery in peer-to-peer networks Jaafar Gaber & Mohamed Bakhouya * Peer to Peer Grid Computing System based on Mobile Agents Joon-Min Gil & Sung-Jin Choi * DANTE: A Self-Adapting Peer-to-Peer System Luis Rodero, Luis Lo'pez, Antonio Ferna'ndez, Vicent Cholvi *** Short Papers *** * Studying viable free markets in Peer-to-Peer file exchange applications without Altruistic Agents David Cabanillas & Steven Willmott * PROSA: P2P Resource Organisation by Social Acquaintances Vincenza Carchiolo, Michele Malgeri, Giuseppe Mangioni & Vincenzo Nicosia * The Exclusion of Malicious Routing Peers in Structured P2P Systems O-Hoon Kwon, Bong Soo Roh, Sung Je Hong & Jong Kim * Facilitating collaboration in a distributed software development environment using P2P architecture Maryam Purvis, Martin Purvis & Bastin Tony Roy Savarimuthu * Reliable P2P File Sharing Service Jung Hwa Shin * Distributed Multilayer Network Management for NEC using Agents Richard Vaughan, James Wise, Paul Huey, Michael Alcock, Jonathan Vaughan & Graham Atkins ********************************************************************* From rosejn at gmail.com Fri Mar 10 10:08:57 2006 From: rosejn at gmail.com (Jeff Rose) Date: Sat Dec 9 22:13:11 2006 Subject: [p2p-hackers] clustering In-Reply-To: <823242bd0603081919h7717ccecyad22dff89e93f0a4@mail.gmail.com> References: <440D648E.302@gmail.com> <000c01c64256$71f96bf0$ccc96fa6@thinkingfish> <823242bd0603081919h7717ccecyad22dff89e93f0a4@mail.gmail.com> Message-ID: <44115039.6080107@gmail.com> It seems like people are always putting arbitrary restrictions on p2p systems and simulations in terms of connectivity, but is this really necessary? Unless you are trying to use NATed nodes (assume we can punch or route through a neighbor),just about any pair of computers on the internet can be neighbors. In essence the internet is a fully connected overlay graph. All DHT's and other less-structured schemes are doing is deciding which links to send messages down. So when you talk about "links existing" you just mean that a given pair maintains some amount of regular communication, or just that they know of each others existence in the network? Maybe since you are coming from the freenet side of things connectivity has a lot more meaning than in other schemes? -Jeff Ian Clarke wrote: > On 3/7/06, *Ranus* > wrote: > > Hui Zhang has published a paper > named "Using the Small-World Model to Improve Freenet Performance". It > should correspond to your idea, so maybe you could read that. > > > Be careful of this paper. If I recall correctly, most of their results > can be attributed to the fact that they ensured that links existed > between adjacent nodes in the graph, which obviously would have a > dramatic beneficial effect relative to a network where local links may > be missing as it means that in the worst case you will do an exhaustive > search for the node you are looking for just by following local links. > > Our findings, as presented in Oskar's thesis, are that Freenet-style > edge selection results in the desired degree of clustering without > "artificial" help. > > Ian. > > > ------------------------------------------------------------------------ > > _______________________________________________ > p2p-hackers mailing list > p2p-hackers@zgp.org > http://zgp.org/mailman/listinfo/p2p-hackers > _______________________________________________ > Here is a web page listing P2P Conferences: > http://www.neurogrid.net/twiki/bin/view/Main/PeerToPeerConferences From ian at locut.us Fri Mar 10 18:48:55 2006 From: ian at locut.us (Ian Clarke) Date: Sat Dec 9 22:13:11 2006 Subject: [p2p-hackers] clustering In-Reply-To: <44115039.6080107@gmail.com> References: <440D648E.302@gmail.com> <000c01c64256$71f96bf0$ccc96fa6@thinkingfish> <823242bd0603081919h7717ccecyad22dff89e93f0a4@mail.gmail.com> <44115039.6080107@gmail.com> Message-ID: <6534FBFF-07D4-48EC-BB0A-362B0D69A6B3@locut.us> You can't really ignore the implications of NATs and firewalled nodes that easily since most computers on the Internet these days are behind NATs or firewalls. But even if you do ignore their existence, the determining factor of whether two nodes in a P2P network can communicate is that they know of each other's existence, and that they know each-other's location in information space (ie. not just their location in IP space). It is not realistic to assume that every node in a P2P network will have this information for every other node in the P2P network, at least not if you want the network to be scalable, and so it is necessary for nodes to select a subset of all other nodes in the P2P network with which they can communicate. Of course, the practicalities of operating a P2P network, which include issues such as establishing cryptographic tunnels, and dealing with NATs and firewalls, provide significant additional motivation for restricting the subset of nodes with which a particular node might seek to communicate with. Ian. On 10 Mar 2006, at 02:08, Jeff Rose wrote: > It seems like people are always putting arbitrary restrictions on > p2p systems and simulations in terms of connectivity, but is this > really necessary? Unless you are trying to use NATed nodes (assume > we can punch or route through a neighbor),just about any pair of > computers on the internet can be neighbors. In essence the > internet is a fully connected overlay graph. All DHT's and other > less-structured schemes are doing is deciding which links to send > messages down. So when you talk about "links existing" you just > mean that a given pair maintains some amount of regular > communication, or just that they know of each others existence in > the network? Maybe since you are coming from the freenet side of > things connectivity has a lot more meaning than in other schemes? > > -Jeff > > Ian Clarke wrote: >> On 3/7/06, *Ranus* > > wrote: >> Hui Zhang has published a paper >> named "Using the Small-World Model to Improve Freenet >> Performance". It >> should correspond to your idea, so maybe you could read that. >> Be careful of this paper. If I recall correctly, most of their >> results can be attributed to the fact that they ensured that links >> existed between adjacent nodes in the graph, which obviously would >> have a dramatic beneficial effect relative to a network where >> local links may be missing as it means that in the worst case you >> will do an exhaustive search for the node you are looking for just >> by following local links. >> Our findings, as presented in Oskar's thesis, are that Freenet- >> style edge selection results in the desired degree of clustering >> without "artificial" help. >> Ian. >> --------------------------------------------------------------------- >> --- >> _______________________________________________ >> p2p-hackers mailing list >> p2p-hackers@zgp.org >> http://zgp.org/mailman/listinfo/p2p-hackers >> _______________________________________________ >> Here is a web page listing P2P Conferences: >> http://www.neurogrid.net/twiki/bin/view/Main/PeerToPeerConferences > > _______________________________________________ > p2p-hackers mailing list > p2p-hackers@zgp.org > http://zgp.org/mailman/listinfo/p2p-hackers > _______________________________________________ > Here is a web page listing P2P Conferences: > http://www.neurogrid.net/twiki/bin/view/Main/PeerToPeerConferences > From mfreed at cs.nyu.edu Sat Mar 11 15:49:26 2006 From: mfreed at cs.nyu.edu (Michael J Freedman) Date: Sat Dec 9 22:13:11 2006 Subject: [p2p-hackers] clustering In-Reply-To: <44115039.6080107@gmail.com> References: <440D648E.302@gmail.com> <000c01c64256$71f96bf0$ccc96fa6@thinkingfish> <823242bd0603081919h7717ccecyad22dff89e93f0a4@mail.gmail.com> <44115039.6080107@gmail.com> Message-ID: On Fri, 10 Mar 2006, Jeff Rose wrote: > Date: Fri, 10 Mar 2006 11:08:57 +0100 > From: Jeff Rose > Reply-To: Peer-to-peer development. > To: Peer-to-peer development. > Subject: Re: [p2p-hackers] clustering > > It seems like people are always putting arbitrary restrictions on p2p systems > and simulations in terms of connectivity, but is this really necessary? > Unless you are trying to use NATed nodes (assume we can punch or route > through a neighbor),just about any pair of computers on the internet can be > neighbors. In essence the internet is a fully connected overlay graph. The problem is that "just about every" and "every" node being able to communicate are not quite the same thing. Indeed, it's precisely the difference in these two assumptions which actually raises a lot of problems when actually deploying DHTs in the wide-area. We recently presented a short paper at WORLDS '05 which discusses the real-world problems that arises from non-transitivity in Internet routing: A can speak to B, B can speak to C, but A can't speak to C as we all independently discovered from running CoralCDN, OpenDHT, and i3. (Firewalls and NATs are actually an easier problem that this, as they express routing constraints much more symmetrically.) http://www.scs.stanford.edu/mfreed/docs/ntr-worlds05.pdf I sent an email about this paper to this mailing list a few months ago, and my apologies for the repeat. However, as our main audience for this paper was actually meant to be the hacker community, as opposed to the academic one, I thought it bears re-mention. --mike ----- www.michaelfreedman.org www.coralcdn.org From lemonobrien at yahoo.com Sat Mar 11 21:19:50 2006 From: lemonobrien at yahoo.com (Lemon Obrien) Date: Sat Dec 9 22:13:11 2006 Subject: [p2p-hackers] clustering In-Reply-To: Message-ID: <20060311211950.59943.qmail@web53605.mail.yahoo.com> >>A can speak to B, B can speak to C, but A can't speak to C this is why you have a relay/super node...to be the go between between A nad C; this is also why creating p2p systems is very difficult...b/c you'll have to test this along with tons of other different senarios. Michael J Freedman wrote: On Fri, 10 Mar 2006, Jeff Rose wrote: > Date: Fri, 10 Mar 2006 11:08:57 +0100 > From: Jeff Rose > Reply-To: Peer-to-peer development. > To: Peer-to-peer development. > Subject: Re: [p2p-hackers] clustering > > It seems like people are always putting arbitrary restrictions on p2p systems > and simulations in terms of connectivity, but is this really necessary? > Unless you are trying to use NATed nodes (assume we can punch or route > through a neighbor),just about any pair of computers on the internet can be > neighbors. In essence the internet is a fully connected overlay graph. The problem is that "just about every" and "every" node being able to communicate are not quite the same thing. Indeed, it's precisely the difference in these two assumptions which actually raises a lot of problems when actually deploying DHTs in the wide-area. We recently presented a short paper at WORLDS '05 which discusses the real-world problems that arises from non-transitivity in Internet routing: A can speak to B, B can speak to C, but A can't speak to C as we all independently discovered from running CoralCDN, OpenDHT, and i3. (Firewalls and NATs are actually an easier problem that this, as they express routing constraints much more symmetrically.) http://www.scs.stanford.edu/mfreed/docs/ntr-worlds05.pdf I sent an email about this paper to this mailing list a few months ago, and my apologies for the repeat. However, as our main audience for this paper was actually meant to be the hacker community, as opposed to the academic one, I thought it bears re-mention. --mike ----- www.michaelfreedman.org www.coralcdn.org _______________________________________________ p2p-hackers mailing list p2p-hackers@zgp.org http://zgp.org/mailman/listinfo/p2p-hackers _______________________________________________ Here is a web page listing P2P Conferences: http://www.neurogrid.net/twiki/bin/view/Main/PeerToPeerConferences You don't get no juice unless you squeeze Lemon Obrien, the Third. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://zgp.org/pipermail/p2p-hackers/attachments/20060311/4a80794c/attachment.htm From osokin at osokin.com Sun Mar 12 03:08:38 2006 From: osokin at osokin.com (Serguei Osokine) Date: Sat Dec 9 22:13:11 2006 Subject: Dijjer and Freenet (RE: [p2p-hackers] clustering) In-Reply-To: <5008189E-074B-43CC-A4F6-CEA901134073@locut.us> Message-ID: On Tuesday, March 07, 2006 Ian Clarke wrote: > I think Chapters 1 and 2 of Oskar Sandberg's recent thesis may be > extremely relevant: > > http://www.math.chalmers.se/~ossa/lic.pdf > > It outlines an algorithm to choose which nodes in a network should > have edges between them such that "greedy" routing (routing to the > peer closest to what you are looking for) has small world O( log^2 > (N) ) path lengths. Ian, Oskar, that was wonderful. I rarely see anything that has that much aesthetic value, but the analyitical proof that the graph rewiring caused by the random search actions actually makes it a small-world graph with log^2(N) path length, was certainly worth the one-year wait since last April, which is when Ian has preannounced this paper. I have to apologize in advance for my questions - some of them might be downright stupid (I won't even pretend that I followed the logic of all your proofs), and in any case there are many of them. Perhaps, too many. Feel free to answer any subset that does not seem stupid to you (and as a special case, of course, just ignore all of them :-) But I waited for this paper for a year, and as you might imagine, I had time to ponder quite a few issues... So here we go: 1. What is the difference between Freenet[1] and Sandberg/Dijjer[2] models that makes Dijjer easier to analyze? This is mentioned a few times in [2] and in other Freenet- and Dijjer- related places, but I did not notice the explicit explanation of the differences anywhere. Was it the directed links in the graph in [2]? Something else? I honestly tried to figure this out by myself, but couldn't; sorry... Maybe if I'd spend more time looking at both [1] and [2], I'd figure this out, but since you guys are already here, I thought that maybe you could answer that? 2. Whatever this difference is, how relevant do you think are the Sandberg/Dijjer results[2] for the original Freenet paper[1]? For example: a. section 5.2 of [1] says about Fig. 3: "We can see that the pathlength scales approximately logarithmically, with a change of slope near 40,0