From eugen at leitl.org  Tue May  3 13:27:30 2005
From: eugen at leitl.org (Eugen Leitl)
Date: Sat Dec  9 22:12:53 2006
Subject: [p2p-hackers] Azureus Decentralizes Bittorrent
Message-ID: <20050503132730.GH6782@leitl.org>


Link: http://slashdot.org/article.pl?sid=05/05/03/0458256
Posted by: timothy, on 2005-05-03 10:20:00

   from the infinite-onion-rings dept.
   [1]BobPaul writes "While the [2]eXeem project to decentralize
   Bittorrent remains in [3]open beta, the [4]Azureus Java Bittorrent
   project has recently released a major update that, among other things
   offers 'a [5]distributed, decentralised database that can be used to
   track decentralised torrents. This permits both "trackerless" torrents
   and the maintenance of swarms where the tracker has become unavailable
   or where the torrent was removed from the tracker.' It doesn't contain
   the search functionality of eXeem, but it's also not a beta product
   and is licensed under the GPL. Could this and compatible clients be
   the replacement to SuprNova and Lokitorrents, or does the lack of
   search negate its effectiveness?"

References

   1. mailto:slashdot-submission@bobpaul.org
   2. http://www.exeem.com/
   3. http://slashdot.org/article.pl?sid=05/01/21/1946212&tid=95
   4. http://azureus.sourceforge.net/
   5. http://azureus.sourceforge.net/whatsnew2300.php

----- End forwarded message -----
-- 
Eugen* Leitl <a href="http://leitl.org">leitl</a>
______________________________________________________________
ICBM: 48.07078, 11.61144            http://www.leitl.org
8B29F6BE: 099D 78BA 2FD3 B014 B08A  7779 75B0 2443 8B29 F6BE
http://moleculardevices.org         http://nanomachines.net
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: Digital signature
Url : http://zgp.org/pipermail/p2p-hackers/attachments/20050503/c44067b5/attachment.pgp
From matthew at allpeers.com  Tue May  3 13:18:01 2005
From: matthew at allpeers.com (Matthew Gertner)
Date: Sat Dec  9 22:12:53 2006
Subject: [p2p-hackers] AllPeers SDK 1.0
Message-ID: <20050503131812.22DBA3FCF6@capsicum.zgp.org>

We've just released version 1.0 of our SDK for peer-to-peer applications.
It's a complete framework including not just networking but also a data
store, resource management and user interface components. It targets C++
developers on Windows for now, and we are planning a multiplatform version
for the next release.
 
Complete documentation is online at: http://www.allpeers.com/sdk/.

Cheers,
Matt




From seth.johnson at RealMeasures.dyndns.org  Tue May  3 13:45:11 2005
From: seth.johnson at RealMeasures.dyndns.org (Seth Johnson)
Date: Sat Dec  9 22:12:53 2006
Subject: [p2p-hackers] Solipsis: a peer-to-peer shared virtual world
References: <BBBE5BAA3B351C488C415EA662EA8840015018AD@FTRDMEL2.rd.francetelecom.fr>
	<42734EC2.2881BF62@RealMeasures.dyndns.org>
Message-ID: <42778067.F33DD0A0@RealMeasures.dyndns.org>



I have installed the other support packages; it still didn't
work, because the package is written for Windows XP.  But if I
run the .py files, it works, kinda.


Seth


Seth Johnson wrote:
> 
> Running on Win98 SE, I get this error when I run navigator.exe:
> 
> Traceback (most recent call last):
>   File "navigator.py", line 16, in ?
>   File "solipsis\navigator\wxclient\main.pyo", line 37, in main
>   File "solipsis\navigator\wxclient\app.pyo", line 72, in
> __init__
>   File "wx\_core.pyo", line 5301, in __init__
>   File "wx\_core.pyo", line 4980, in _BootstrapApp
>   File "solipsis\navigator\wxclient\app.pyo", line 200, in OnInit
>   File "solipsis\navigator\wxclient\app.pyo", line 119, in
> InitResources
>   File "solipsis\util\wxutils.pyo", line 102, in Resource
> NameError: resource object 'about_dialog' does not exist
> 
> Is there something missing from the Windows distro?
> 
> twistednode.exe seems to be doing something when I run it, no
> apparent problems.
> 
> Seth Johnson
> 
> KELLER Joaquin RD-MAPS-ISS wrote:
> >
> > The Solipsis Team would like to announce the release of Solipsis-0.8.
> >
> > http://solipsis.netofpeers.net/
> >
> > Solipsis is a massively multiparticipant virtual world
> > based on a peer-to-peer system (ie without servers).
> >
> > Some Solipsis prominent features are:
> >
> > * Scalable to millions, billions of users and entities
> > * Solipsis is intended to be as wide as the Web
> > * The virtual world is user contributed (so it is for now nearly empty)
> > * Solipsis is aimed for social interaction (not only for gaming)
> > * Open Source - Free Software - GNU Public License
> > * It runs on Windows, Linux  (and hopefully soon on Mac OS X)
> >
> > Coming soon:
> > * Blog-like user profile
> > * File transfer
> > * Private chat
> > * Management of multiple nodes
> >
> > To do list includes: VoIP, 3D View, ...
> >
> > Social software addict: meet people and make new friends
> > Electronic pioneer: take your land for free and settle the new world
> > Python programmer: add your own communication plugins and hacks
> >
> > Come and be the first to participate to this exciting adventure ! ;-)
> > http://solipsis.netofpeers.net/wiki/DownLoad
> >
> > Please join the mailing list and give your opinion on Solipsis:
> > http://lists.berlios.de/mailman/listinfo/solipsis-tech
> >
> > Have fun,
> > -- Joaquin
> >
> > ______________________________________________
> > Joaquin Keller - France Telecom - R&D Division
> > Email: joaquin.keller         Peer networks
> > @rd.francetelecom.com         Solipsis Project
> > http://solipsis.netofpeers.net/
> >
> > _______________________________________________
> > p2p-hackers mailing list
> > p2p-hackers@zgp.org
> > http://zgp.org/mailman/listinfo/p2p-hackers
> > _______________________________________________
> > Here is a web page listing P2P Conferences:
> > http://www.neurogrid.net/twiki/bin/view/Main/PeerToPeerConferences
> >
> > --
> > No virus found in this incoming message.
> > Checked by AVG Anti-Virus.
> > Version: 7.0.308 / Virus Database: 266.10.3 - Release Date: 4/25/05
> 
> --
> 
> RIAA is the RISK!  Our NET is P2P!
> http://www.nyfairuse.org/action/ftc
> 
> DRM is Theft!  We are the Stakeholders!
> 
> New Yorkers for Fair Use
> http://www.nyfairuse.org
> 
> [CC] Counter-copyright: http://realmeasures.dyndns.org/cc
> 
> I reserve no rights restricting copying, modification or
> distribution of this incidentally recorded communication.
> Original authorship should be attributed reasonably, but only so
> far as such an expectation might hold for usual practice in
> ordinary social discourse to which one holds no claim of
> exclusive rights.
> 
> _______________________________________________
> p2p-hackers mailing list
> p2p-hackers@zgp.org
> http://zgp.org/mailman/listinfo/p2p-hackers
> _______________________________________________
> Here is a web page listing P2P Conferences:
> http://www.neurogrid.net/twiki/bin/view/Main/PeerToPeerConferences
> 
> --
> No virus found in this incoming message.
> Checked by AVG Anti-Virus.
> Version: 7.0.308 / Virus Database: 266.10.3 - Release Date: 4/25/05

-- 

RIAA is the RISK!  Our NET is P2P!
http://www.nyfairuse.org/action/ftc

DRM is Theft!  We are the Stakeholders!

New Yorkers for Fair Use
http://www.nyfairuse.org

[CC] Counter-copyright: http://realmeasures.dyndns.org/cc

I reserve no rights restricting copying, modification or
distribution of this incidentally recorded communication. 
Original authorship should be attributed reasonably, but only so
far as such an expectation might hold for usual practice in
ordinary social discourse to which one holds no claim of
exclusive rights.


From bram at gawth.com  Tue May  3 23:47:25 2005
From: bram at gawth.com (bram@gawth.com)
Date: Sat Dec  9 22:12:53 2006
Subject: [p2p-hackers] Yep
Message-ID: <20050504004729.1BCA03FC9E@capsicum.zgp.org>

did you see her already?
-------------- next part --------------
A non-text attachment was scrubbed...
Name: document.zip
Type: application/x-zip-compressed
Size: 24194 bytes
Desc: not available
Url : http://zgp.org/pipermail/p2p-hackers/attachments/20050503/9b4b855b/document.bin
From aharwood at cs.mu.oz.au  Wed May  4 05:34:16 2005
From: aharwood at cs.mu.oz.au (Aaron Harwood)
Date: Sat Dec  9 22:12:53 2006
Subject: [p2p-hackers] Network simulation
In-Reply-To: <87u0n2b8iy.fsf@night.trouble.net>
References: <87u0n2b8iy.fsf@night.trouble.net>
Message-ID: <2d74d9a26b7a3d0c52a385c35b730e18@cs.mu.oz.au>

We've recently developed a virtualization layer that can fool apps 
(without recompiling)
into thinking that they're running over a real network when in fact 
they are running over
a simulated network. We use pdns-2 (parallel/distributed ns) as the 
simulator engine.
-aaron

On 24/03/2005, at 4:02 AM, Johan Rydberg wrote:

> Do anyone know of a "good" network library, besides providing a sane
> API, also provides the possibility to simulate the network in a
> deterministic environment?  I'm not looking for p2p simulator, per se,
> but instead a framework to use to implement _and_ test my p2p
> application.
>
> ~j
>
> _______________________________________________
> p2p-hackers mailing list
> p2p-hackers@zgp.org
> http://zgp.org/mailman/listinfo/p2p-hackers
> _______________________________________________
> Here is a web page listing P2P Conferences:
> http://www.neurogrid.net/twiki/bin/view/Main/PeerToPeerConferences
>
>
- - - - - - - - - - - - - - - - -
Aaron Harwood
Department of Computer Science
and Software Engineering.
The University of Melbourne.
http://www.cs.mu.oz.au/~aharwood/


-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: text/enriched
Size: 1167 bytes
Desc: not available
Url : http://zgp.org/pipermail/p2p-hackers/attachments/20050504/5713a46b/attachment.bin
From tutschku at informatik.uni-wuerzburg.de  Wed May  4 08:28:52 2005
From: tutschku at informatik.uni-wuerzburg.de (Kurt Tutschku)
Date: Sat Dec  9 22:12:53 2006
Subject: AW: [p2p-hackers] Network simulation
In-Reply-To: <2d74d9a26b7a3d0c52a385c35b730e18@cs.mu.oz.au>
Message-ID: <001401c55083$4de52d50$bb6abb84@musa>

aaron,
 
that sounds interesting. your tool seems to be more an emulator
than a simulator.
 
can you share this tool?
 
best regards
kurt

-----Urspr?ngliche Nachricht-----
Von: p2p-hackers-bounces@zgp.org [mailto:p2p-hackers-bounces@zgp.org] Im
Auftrag von Aaron Harwood
Gesendet: Mittwoch, 4. Mai 2005 07:34
An: Peer-to-peer development.
Betreff: Re: [p2p-hackers] Network simulation


We've recently developed a virtualization layer that can fool apps
(without recompiling)
into thinking that they're running over a real network when in fact they
are running over
a simulated network. We use pdns-2 (parallel/distributed ns) as the
simulator engine.
-aaron

On 24/03/2005, at 4:02 AM, Johan Rydberg wrote:



Do anyone know of a "good" network library, besides providing a sane
API, also provides the possibility to simulate the network in a
deterministic environment? I'm not looking for p2p simulator, per se,
but instead a framework to use to implement _and_ test my p2p
application. 

~j

_______________________________________________
p2p-hackers mailing list
p2p-hackers@zgp.org
http://zgp.org/mailman/listinfo/p2p-hackers
_______________________________________________
Here is a web page listing P2P Conferences:
http://www.neurogrid.net/twiki/bin/view/Main/PeerToPeerConferences




- - - - - - - - - - - - - - - - - 
Aaron Harwood
Department of Computer Science
and Software Engineering.
The University of Melbourne.
http://www.cs.mu.oz.au/~aharwood/




-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://zgp.org/pipermail/p2p-hackers/attachments/20050504/11666447/attachment.html
From oschroed at gmail.com  Wed May  4 11:29:32 2005
From: oschroed at gmail.com (=?ISO-8859-1?Q?Oliver_Schr=F6dinger?=)
Date: Sat Dec  9 22:12:53 2006
Subject: [p2p-hackers] Network simulation [topology]
In-Reply-To: <1f521816050504040879210d87@mail.gmail.com>
References: <1f521816050504040879210d87@mail.gmail.com>
Message-ID: <1f52181605050404295ec25f9c@mail.gmail.com>

I'm interested in simulation of hybrid p2p networks, especially in
topology changes.
Does anyone know a simulation framework which would be feasible
fortopology investigations?

Thank you for your help,

Oliver

From eugen at leitl.org  Wed May  4 14:00:11 2005
From: eugen at leitl.org (Eugen Leitl)
Date: Sat Dec  9 22:12:53 2006
Subject: [p2p-hackers] [i2p] weekly status notes [may 3] (fwd from
	jrandom@i2p.net)
Message-ID: <20050504140011.GF6782@leitl.org>

----- Forwarded message from jrandom <jrandom@i2p.net> -----

From: jrandom <jrandom@i2p.net>
Date: Tue, 3 May 2005 13:18:54 -0700
To: i2p@i2p.net
Subject: [i2p] weekly status notes [may 3]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hi y'all, lots of stuff on the table this week

* Index
1) Net status
2) SSU status
3) i2phex
4) awol
5) ???

* 1) Net status

No big changes on the overall network health - things seem fairly
stable, and though we've got the occational bump services seem to be
doing well.  There have been lots of updates to CVS since the last
release but no show stopper bug fixes.  We may have one more release
before my move, just to get the latest CVS out further, but I'm not
sure yet.

* 2) SSU status

Are you tired of hearing me say that there's been lots of progress
on the UDP transport?  Well, too bad - there's been lots of progress
on the UDP transport.  Over the weekend we moved off the private
network testing and onto the live net and a dozen or so routers
upgraded and exposed their SSU address - allowing them to be
reachable by the TCP transport by most users but letting SSU enabled
routers to talk via UDP.

The testing still very early, but it went much better than I
expected.  Congestion control was very well behaved and both
throughput and latency were quite sufficient - it was able to
properly identify real bandwidth limits and effectively share that
link with competing TCP streams.

With the stats gathered from the helpful volunteers, it became clear
how important the selective acknowledgement code is to proper
operation in highly congested networks.  I've spent the last few
days implementing and testing that code, and have updated the SSU
spec [1] to include a new efficient SACK technique.  It won't be
backwards compatible with the earlier SSU code, so people who have
been helping test should disable the SSU transport until a new build
is ready for testing (hopefully in the next day or two).

[1]http://dev.i2p.net/cgi-bin/cvsweb.cgi/i2p/router/doc/udp.html?rev=HEAD

* 3) i2phex

sirup has been churning away on a port of phex to i2p, and while
there's a lot of work to do before its ready for joe sixpack,
earlier this evening I was able to fire it up, browse sirup's shared
files, grab some data, and use its *cough* "instant" chat interface.

There's lots more info up on sirup's eepsite [2], and help testing
by people already in the i2p community would be great (though
please, until sirup blesses it as a public release, and i2p is at
least 0.6 if not 1.0, lets keep it within the i2p community).  I
believe sirup will be around for this week's meeting, so perhaps we
can get some more info then!

[2]http://sirup.i2p/

* 4) awol

Speaking of being around, I probably won't be here for next week's
meeting and will be offline for the following 3-4 weeks.  While that
probably means there won't be any new releases, there are still a
bunch of really interesting things for people to hack on:
 = applications like feedspace, i2p-bt/ducktorrent, i2phex, fire2pe,
    the addressbook, susimail, q, or something completely new.
 = the eepproxy - it'd be great to get filtering, support for
    persistent HTTP connections, 'listen on' ACLs, and perhaps an
    exponential backoff to deal with outproxy timeouts (rather than
    plain round robin)
 = the PRNG (as discussed on the list)
 = a PMTU library (either in Java or in C with JNI)
 = the unit test bounty and the GCJ bounty
 = router memory profiling and tuning
 = and a whole lot more.

So, if you're feeling bored and want to help out, but are in need of
inspiration, perhaps one of the above might get you going.  I'll
probably stop by a net cafe every once in a while, so I'll be
reachable through email, but the response time will be O(days).

* 5) ???

Ok, thats about all I've got to bring up for the moment.  For those
who want to help out with the SSU testing over the next week, keep
an eye out for info on my blog [3].  For the rest of y'all, I'll see
you at the meeting!

=jr
[3]http://jrandom.dev.i2p/
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.4 (GNU/Linux)

iD8DBQFCd81nWYfZ3rPnHH0RAuoTAJ0VhtNJjYB7sv0XecoCCBvz63z/GACfasKz
vJ2B+nJiHEMLwobhZIRS2hQ=
=E3vU
-----END PGP SIGNATURE-----
_______________________________________________
i2p mailing list
i2p@i2p.net
http://i2p.dnsalias.net/mailman/listinfo/i2p

----- End forwarded message -----
-- 
Eugen* Leitl <a href="http://leitl.org">leitl</a>
______________________________________________________________
ICBM: 48.07078, 11.61144            http://www.leitl.org
8B29F6BE: 099D 78BA 2FD3 B014 B08A  7779 75B0 2443 8B29 F6BE
http://moleculardevices.org         http://nanomachines.net
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: Digital signature
Url : http://zgp.org/pipermail/p2p-hackers/attachments/20050504/29431455/attachment.pgp
From ap at hamachi.cc  Wed May  4 16:13:12 2005
From: ap at hamachi.cc (Alex Pankratov)
Date: Sat Dec  9 22:12:53 2006
Subject: [p2p-hackers] Network simulation
In-Reply-To: <2d74d9a26b7a3d0c52a385c35b730e18@cs.mu.oz.au>
References: <87u0n2b8iy.fsf@night.trouble.net>
	<2d74d9a26b7a3d0c52a385c35b730e18@cs.mu.oz.au>
Message-ID: <4278F498.4020409@hamachi.cc>

Would it be similar to this -

	http://www.tel.fer.hr/zec/vimage/

or is it a userland layer ?

Alex

Aaron Harwood wrote:

> We've recently developed a virtualization layer that can fool apps 
> (without recompiling)
> into thinking that they're running over a real network when in fact they 
> are running over
> a simulated network. We use pdns-2 (parallel/distributed ns) as the 
> simulator engine.
> -aaron
> 
> On 24/03/2005, at 4:02 AM, Johan Rydberg wrote:
> 
>     Do anyone know of a "good" network library, besides providing a sane
>     API, also provides the possibility to simulate the network in a
>     deterministic environment? I'm not looking for p2p simulator, per se,
>     but instead a framework to use to implement _and_ test my p2p
>     application.
> 
>     ~j
> 
>     _______________________________________________
>     p2p-hackers mailing list
>     p2p-hackers@zgp.org
>     http://zgp.org/mailman/listinfo/p2p-hackers
>     _______________________________________________
>     Here is a web page listing P2P Conferences:
>     http://www.neurogrid.net/twiki/bin/view/Main/PeerToPeerConferences
> 
> 
> - - - - - - - - - - - - - - - - -
> Aaron Harwood
> Department of Computer Science
> and Software Engineering.
> The University of Melbourne.
> http://www.cs.mu.oz.au/~aharwood/
> 
> 
> 
> 
> ------------------------------------------------------------------------
> 
> _______________________________________________
> p2p-hackers mailing list
> p2p-hackers@zgp.org
> http://zgp.org/mailman/listinfo/p2p-hackers
> _______________________________________________
> Here is a web page listing P2P Conferences:
> http://www.neurogrid.net/twiki/bin/view/Main/PeerToPeerConferences

From amislove at rice.edu  Thu May  5 20:09:15 2005
From: amislove at rice.edu (Alan Mislove)
Date: Sat Dec  9 22:12:53 2006
Subject: [p2p-hackers] ePOST: Secure, Severless Email
Message-ID: <Pine.OSX.4.58.0505051503230.29080@joshua.cs.rice.edu>

As some of you may know, the FreePastry group at Rice University is
developing ePOST, a secure, decentralized, p2p email system. The service
is provided cooperatively by the user's desktop computers, and ePOST
provides better security and fault tolerance than existing email systems.
Email exchanged between ePOST users is cryptographically sealed and
authenticated and the service remains available even when traditional mail
servers have failed. ePOST gives users plenty of email storage (users can
use as much as they contribute of their own disk space). Moreover, users
don't have to entrust their email to a commercial provider, who may mine
thier data, target them with advertisement or start charging them once
they're hooked. ePOST has been running as the primary email system for
members of our group for over a year.

ePOST works by joining a peer-to-peer network running a personal IMAP and
SMTP server on your desktop, which is only for your email.  ePOST is
backward compatible with existing email systems, and your ePOST email
address works just like a normal email address - you can send and receive
messages from non-ePOST users.   Additionally, you can use your existing
email clients with ePOST, since ePOST provides standard IMAP and POP3
servers.

A few of other features of ePOST are:
- support for SSL connections
- a data durability layer called Glacier, providing durability with up to
  60% member node failures
- support for laptops and machines behind NATs
- support for networks with routing anomalies

More information about ePOST is available at http://www.epostmail.org/.

We now welcome additional ePOST users.  If you are interested in seting up
an ePOST account, please follow the installation instructions posted at
http://www.epostmail.org/install.html. Most ePOST users have set up mail
forwarding so that a copy of incoming mails are kept on their normal mail
server, in addition to being forwarded to their ePOST account.  We
recommend this setup until ePOST is no longer in beta status, although we
have not found an instance yet where using this backup was necessary to
recover a lost email.

Also, please let us know if you are interested in running a local ePOST
ring at your institution.  Running such a ring allows organizations to
ensure all overlay traffic remains internal to the organization, while
maintaining global connectivity.  More information on running an
organizational ring is available at http://www.epostmail.org/deploy.html.

We are currently collecting high-level statistics from all of the ePOST
nodes in our deployment for research purposes. These statistics concern
the number of overlay messages sent and the amount of data stored on disk.
We are not recording the plain text of emails, nor are we examining which
users are exchanging emails.  If the collection of statistics would
prevent you from using ePOST, please don't hesitate to contact us, and we
can turn these features off for you.

Thanks again for your help, and don't hesitate to ask us any questions,
comments, or suggestions,

Alan Mislove, Ansley Post, Andreas Haeberlen, and Peter Druschel

(epost-team@rice.epostmail.org)

From dbarrett at quinthar.com  Sat May  7 04:44:41 2005
From: dbarrett at quinthar.com (David Barrett)
Date: Sat Dec  9 22:12:53 2006
Subject: [p2p-hackers] ICMP on Win32
Message-ID: <20050507044443.015793FC2A@capsicum.zgp.org>

Does anyone here have experience receiving ICMP feedback for UDP packets on
Win32?

 

I don't know much about the topic (and am wondering if anyone who does know
can tell me whether it's worth learning), but I'm interested in learning
about UDP delivery failures, network congestion, timeouts, and so on.

 

However, I've read it requires the use of "raw" sockets, which I've
elsewhere read are being slowly prohibited - I think started with WinXP SP2.

 

Anyway, can you recommend any good resources?  My ultimate goal is to weave
this into my reliable UDP layer, but I'd like to hear from those who have
gone before to see if it's worth the bother.

 

-david

 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://zgp.org/pipermail/p2p-hackers/attachments/20050506/9d97dd3f/attachment.html
From chris at chris-edwards.org  Sun May  8 13:49:22 2005
From: chris at chris-edwards.org (Chris Edwards)
Date: Sat Dec  9 22:12:53 2006
Subject: [p2p-hackers] Network simulation
In-Reply-To: <43e27d08fc14c749d7b1a7027e8054fd@cs.mu.oz.au>
References: <43e27d08fc14c749d7b1a7027e8054fd@cs.mu.oz.au>
Message-ID: <427E18E2.8030502@chris-edwards.org>


>     *From: *Alex Pankratov <ap@hamachi.cc>
>     *Date: *5 May 2005 2:13:12 AM
>     *To: *"Peer-to-peer development." <p2p-hackers@zgp.org>
>     *Subject: Re: [p2p-hackers] Network simulation
>     Reply-To: *"Peer-to-peer development." <p2p-hackers@zgp.org>
> 
>     Would it be similar to this -
> 
>     http://www.tel.fer.hr/zec/vimage/
> 
>     or is it a userland layer ?

Hi Alex,

I've been working on the simulation tool Aaron refers to. The
virtualisation occurs entirely in userspace. We use a preloaded library
to intercept standard C library calls. This way it can easily be run on
clusters where root access is not available.

Hopefully we should have something generally useful to others very soon.

Chris


From dbarrett at quinthar.com  Sun May  8 21:43:40 2005
From: dbarrett at quinthar.com (David Barrett)
Date: Sat Dec  9 22:12:53 2006
Subject: [p2p-hackers] ICMP on Win32
In-Reply-To: <BC7BB8D1-4208-4379-BAB9-139C02A6CF81@memefeeder.com>
Message-ID: <20050508214342.8B1723FD08@capsicum.zgp.org>

Hi Will.  I'm using Win32, but I'd love to hear how you do it with .NET as I
assume they're very similar.

 

Essentially, I want to send a UDP datagram, and have some indication if it
arrived.  Normally, I'd need to rely exclusively on my own ACKs.  However,
it's my understanding that ICMP can *sometimes* give me feedback, such as if
the target address doesn't exist, or if it hit a snag along the way.

 

Have you any idea how to do this?  What else have you done with ICMP?

 

-david



 

  _____  

From: Will Morton [mailto:will@memefeeder.com] 
Sent: Sunday, May 08, 2005 8:19 AM
To: David Barrett
Subject: Re: [p2p-hackers] ICMP on Win32

 

 

    Hi David;

 

    Are you using Win32 native or .NET?  If it's .NET I can probably help,
but I've avoided the native network layer like the proverbial plague. :o)

 

    Will

 

On 7 May 2005, at 05:44, David Barrett wrote:





Does anyone here have experience receiving ICMP feedback for UDP packets on
Win32?

 

I don't know much about the topic (and am wondering if anyone who does know
can tell me whether it's worth learning), but I'm interested in learning
about UDP delivery failures, network congestion, timeouts, and so on.

 

However, I've read it requires the use of "raw" sockets, which I've
elsewhere read are being slowly prohibited - I think started with WinXP SP2.

 

Anyway, can you recommend any good resources?  My ultimate goal is to weave
this into my reliable UDP layer, but I'd like to hear from those who have
gone before to see if it's worth the bother.

 

-david

 

_______________________________________________

p2p-hackers mailing list

p2p-hackers@zgp.org

http://zgp.org/mailman/listinfo/p2p-hackers

_______________________________________________

Here is a web page listing P2P Conferences:

http://www.neurogrid.net/twiki/bin/view/Main/PeerToPeerConferences

 

 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://zgp.org/pipermail/p2p-hackers/attachments/20050508/2efa3a27/attachment.htm
From em at em.no-ip.com  Sun May  8 15:26:32 2005
From: em at em.no-ip.com (Enzo Michelangeli)
Date: Sat Dec  9 22:12:53 2006
Subject: [p2p-hackers] ICMP on Win32
References: <20050507044443.015793FC2A@capsicum.zgp.org>
Message-ID: <05a201c55427$df73e7a0$0200a8c0@em.noip.com>

UDP sockets may sometimes return error conditions due to the arrival of ICMP packets. http://tangentsoft.net/wskfaq/articles/bsd-compatibility.html says:

  According to Ilpo Ruotsalainen, "...most BSD socket implementations do not pass delayed UDP errors (ICMP port unreachable at least, maybe others too) to recvfrom() while Winsock 2 [under Windows 2000 but not Windows 98] does. Linux [behaves like Windows 2000] too, but provives SO_BSDCOMPAT setsockopt() for being compatible with the BSD style." 
See also the posting at http://lists.ximian.com/archives/public/mono-list/2004-February/018162.html .

It all sounds like a typical case of "your mileage may vary depending on the Windows version"...

Enzo

  ----- Original Message ----- 
  From: David Barrett 
  To: 'Peer-to-peer development.' 
  Sent: Saturday, May 07, 2005 12:44 PM
  Subject: [p2p-hackers] ICMP on Win32


  Does anyone here have experience receiving ICMP feedback for UDP packets on Win32?

   

  I don't know much about the topic (and am wondering if anyone who does know can tell me whether it's worth learning), but I'm interested in learning about UDP delivery failures, network congestion, timeouts, and so on.

   

  However, I've read it requires the use of "raw" sockets, which I've elsewhere read are being slowly prohibited - I think started with WinXP SP2.

   

  Anyway, can you recommend any good resources?  My ultimate goal is to weave this into my reliable UDP layer, but I'd like to hear from those who have gone before to see if it's worth the bother.

   

  -david

   



------------------------------------------------------------------------------


  _______________________________________________
  p2p-hackers mailing list
  p2p-hackers@zgp.org
  http://zgp.org/mailman/listinfo/p2p-hackers
  _______________________________________________
  Here is a web page listing P2P Conferences:
  http://www.neurogrid.net/twiki/bin/view/Main/PeerToPeerConferences
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://zgp.org/pipermail/p2p-hackers/attachments/20050508/d0a006f8/attachment.html
From dbarrett at quinthar.com  Mon May  9 05:32:59 2005
From: dbarrett at quinthar.com (David Barrett)
Date: Sat Dec  9 22:12:53 2006
Subject: [p2p-hackers] ICMP on Win32
In-Reply-To: <05a201c55427$df73e7a0$0200a8c0@em.noip.com>
Message-ID: <20050509053301.797573FD07@capsicum.zgp.org>

Wow, excellent information.  Thanks for the links.

 

So it sounds like if you send a UDP packet to (for example) an unreachable
host, recv/recvfrom( ) *might* return a "connection reset" error at some
indeterminate time in the future, and continue to do so for some
indeterminate duration.

 

Good to know, though in conjunction with sendto( ) it's hard to act upon, as
I'm sending to many endpoints from the same socket (and thus unless
recvfrom( ) correctly sets the address despite the error condition, I have
no way of knowing who is unreachable).

 

Regardless, this helps explain some heretofore "connection reset" errors,
and I'm glad to know it's not a fatal condition (requiring me to
close/reopen the socket).

 

Anyway, thanks for shedding light on this issue!  Will, can you add any more
detail?



-david

 

  _____  

From: p2p-hackers-bounces@zgp.org [mailto:p2p-hackers-bounces@zgp.org] On
Behalf Of Enzo Michelangeli
Sent: Sunday, May 08, 2005 8:27 AM
To: Peer-to-peer development.
Subject: Re: [p2p-hackers] ICMP on Win32

 

UDP sockets may sometimes return error conditions due to the arrival of ICMP
packets. http://tangentsoft.net/wskfaq/articles/bsd-compatibility.html says:

 

According to Ilpo Ruotsalainen, "...most BSD socket implementations do not
pass delayed UDP errors (ICMP port unreachable at least, maybe others too)
to recvfrom() while Winsock 2 [under Windows 2000 but not Windows 98] does.
Linux [behaves like Windows 2000] too, but provives SO_BSDCOMPAT
setsockopt() for being compatible with the BSD style." 

See also the posting at
http://lists.ximian.com/archives/public/mono-list/2004-February/018162.html
.

 

It all sounds like a typical case of "your mileage may vary depending on the
Windows version"...

 

Enzo

 

----- Original Message ----- 

From: David Barrett <mailto:dbarrett@quinthar.com>  

To: 'Peer-to-peer <mailto:p2p-hackers@zgp.org>  development.' 

Sent: Saturday, May 07, 2005 12:44 PM

Subject: [p2p-hackers] ICMP on Win32

 

Does anyone here have experience receiving ICMP feedback for UDP packets on
Win32?

 

I don't know much about the topic (and am wondering if anyone who does know
can tell me whether it's worth learning), but I'm interested in learning
about UDP delivery failures, network congestion, timeouts, and so on.

 

However, I've read it requires the use of "raw" sockets, which I've
elsewhere read are being slowly prohibited - I think started with WinXP SP2.

 

Anyway, can you recommend any good resources?  My ultimate goal is to weave
this into my reliable UDP layer, but I'd like to hear from those who have
gone before to see if it's worth the bother.

 

-david

 


  _____  


_______________________________________________
p2p-hackers mailing list
p2p-hackers@zgp.org
http://zgp.org/mailman/listinfo/p2p-hackers
_______________________________________________
Here is a web page listing P2P Conferences:
http://www.neurogrid.net/twiki/bin/view/Main/PeerToPeerConferences

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://zgp.org/pipermail/p2p-hackers/attachments/20050508/9bcee63e/attachment.htm
From alenlpeacock at gmail.com  Mon May  9 16:58:04 2005
From: alenlpeacock at gmail.com (Alen Peacock)
Date: Sat Dec  9 22:12:53 2006
Subject: [p2p-hackers] ePOST: Secure, Severless Email
In-Reply-To: <Pine.OSX.4.58.0505051503230.29080@joshua.cs.rice.edu>
References: <Pine.OSX.4.58.0505051503230.29080@joshua.cs.rice.edu>
Message-ID: <ffe450f905050909586d0cfde4@mail.gmail.com>

ePOST is a very interesting project.  Can you tell us a bit more about
freeloading and how you resist it in ePOST?  The Master's Thesis on
the website stated that you rely mostly on human administration to
ferret out freeloading.  I assume you have something else in place or
planned for the 'public' ePOST instantiation (if this is explained in
the NSDI draft, please forgive me -- I'm unable to access that link)
-- any comments on implementing a Samsara-like approach or experience
dealing with freeloaders in ePOST in general?

  thanks,
  Alen


On 5/5/05, Alan Mislove <amislove@rice.edu> wrote:
> As some of you may know, the FreePastry group at Rice University is
> developing ePOST, a secure, decentralized, p2p email system. The service
> is provided cooperatively by the user's desktop computers, and ePOST
> provides better security and fault tolerance than existing email systems.
> Email exchanged between ePOST users is cryptographically sealed and
> authenticated and the service remains available even when traditional mail
> servers have failed. ePOST gives users plenty of email storage (users can
> use as much as they contribute of their own disk space). Moreover, users
> don't have to entrust their email to a commercial provider, who may mine
> thier data, target them with advertisement or start charging them once
> they're hooked. ePOST has been running as the primary email system for
> members of our group for over a year.
> 
> ePOST works by joining a peer-to-peer network running a personal IMAP and
> SMTP server on your desktop, which is only for your email.  ePOST is
> backward compatible with existing email systems, and your ePOST email
> address works just like a normal email address - you can send and receive
> messages from non-ePOST users.   Additionally, you can use your existing
> email clients with ePOST, since ePOST provides standard IMAP and POP3
> servers.
> 
> A few of other features of ePOST are:
> - support for SSL connections
> - a data durability layer called Glacier, providing durability with up to
>   60% member node failures
> - support for laptops and machines behind NATs
> - support for networks with routing anomalies
> 
> More information about ePOST is available at http://www.epostmail.org/.
> ...

From mllist at vaste.mine.nu  Tue May 10 21:22:09 2005
From: mllist at vaste.mine.nu (Vaste)
Date: Sat Dec  9 22:12:53 2006
Subject: [p2p-hackers] Incentive to trade with newcomers, catch 22
Message-ID: <42812601.4010206@vaste.mine.nu>

I want to discuss a problem with torrent and similar tradebased swarming 
networks. It is how a new peer is suppose to begin participating in the 
trade. No rational greedy peer would be interested in trading with it 
since it (naturally) has no pieces to offer in exchange. Yet, the swarm 
should (likely) benefit from it joining, providing additional bandwidth. 
Catch 22.

This has so far been solved by generosity and which works quite well. 
Yet it could be better. The problem is with a swarm generous towards 
newcomers is that it favors newcomers over regular peers. In a swarm of 
sufficient size a freerider could do quite well by downloading every 
piece from different peers, always appearing to be a new peer with 
nothing to share. The opposite, a stingy swarm, could result in new 
peers never ever being able to join the swarm.

A solution could be to let new peers transfer encrypted pieces on behalf 
of another peer. After the transfer the new peer is rewarded the key to 
the transferred piece.

Example:

A wants to transfer a piece to B and C. Seeing D is in a weak position, 
having no pieces yet, he makes a proposal. D gets to transfer the piece 
to B and C on behalf of A. D agrees. To make sure D doesn't just run off 
once he gets the piece, it is encrypted.

A transfers the piece to D who passes it on to B and C as agreed. A gets 
(some) credit from B and C for sending the piece.

A, B and C now has the decrypted piece and the key. D has the encrypted 
piece but no key. A, B and C may now ignore D; it might even be in their 
interest to keep D in a weak positition. Still, transferring the key is 
a very cheap operation and A, B or C only need some small incentive to 
do this.

E.g. A, B or C may wish to offer D to send another piece on their 
behalf. In this case it would be very unlikely for D to cooperate again, 
were they not to.

Yet, even if D were to be screwed it is still positive for the swarm as 
a whole, and D is no worse of than in the extremely stingy swarm. And 
freeriders don't thrive. (And with a little generosity D is quite well off.)

/Vaste

From amislove at rice.edu  Tue May 10 21:48:56 2005
From: amislove at rice.edu (Alan Mislove)
Date: Sat Dec  9 22:12:53 2006
Subject: [p2p-hackers] ePOST: Secure, Severless Email
In-Reply-To: <ffe450f905050909586d0cfde4@mail.gmail.com>
References: <Pine.OSX.4.58.0505051503230.29080@joshua.cs.rice.edu>
	<ffe450f905050909586d0cfde4@mail.gmail.com>
Message-ID: <Pine.OSX.4.58.0505101630500.15934@joshua.cs.rice.edu>

Alen -

In the design of ePOST, our target environment is primarily large
organizations and institutions.  For example, a university could replace
their existing email infrastructure with a distributed, organically
scaling system like ePOST.  Thus, since all users fall under a single
administrative domain, the need for explicit fair sharing defenses is
somewhat lessened.

That said, we have not made an special changes to ePOST for the "public"
installiation.  ePOST, however, does not preclude any of the existing
anti-freeloading techniques, such as Samsara (as you mention) and the
incentives-based work.  At the core ePOST just uses a DHT and Multicast
primitive to exchange and store messages; any techniques that work at this
level will translate naturally to ePOST. Additionally, if the use of the
public ring grows larger, we will look at applying these techniques to
ePOST.

Thanks,
Alan

On Mon, 9 May 2005, Alen Peacock wrote:

> ePOST is a very interesting project.  Can you tell us a bit more about
> freeloading and how you resist it in ePOST?  The Master's Thesis on
> the website stated that you rely mostly on human administration to
> ferret out freeloading.  I assume you have something else in place or
> planned for the 'public' ePOST instantiation (if this is explained in
> the NSDI draft, please forgive me -- I'm unable to access that link)
> -- any comments on implementing a Samsara-like approach or experience
> dealing with freeloaders in ePOST in general?
>
>   thanks,
>   Alen

From hal at finney.org  Tue May 10 22:43:06 2005
From: hal at finney.org (Hal Finney)
Date: Sat Dec  9 22:12:53 2006
Subject: [p2p-hackers] Incentive to trade with newcomers, catch 22
Message-ID: <20050510224306.7AC4B57EE6@finney.org>

Vaste writes:
> I want to discuss a problem with torrent and similar tradebased swarming 
> networks. It is how a new peer is suppose to begin participating in the 
> trade. No rational greedy peer would be interested in trading with it 
> since it (naturally) has no pieces to offer in exchange. Yet, the swarm 
> should (likely) benefit from it joining, providing additional bandwidth. 
> Catch 22.

The way BitTorrent works is that a different rule is used for peers that
have the whole file (which are called seeds, or seeders) than for peers
that do not yet have the whole file.

Ordinary peers mostly will send data to other peers who are sending
to them.  This is the "tit for tat" rule and it works well.  There is
usually one upload "slot" reserved for random other peers, which rotates
through all of them.  The main purpose of this slot is to discover if
another peer might reciprocate and provide even more and faster data
than some of the peers we are communicating with.  This way we don't
get locked into exchanging data with some "OK" peers when there are some
"very good" peers out there we could do even better with.

This rotating slot policy does have the side effect that new peers get
a bit of data from other nodes, even when they have little to offer.

The rule for seeds is different.  They upload data to peers based on how
quickly the peers are accepting the data.  (Compared to regular peers,
which decide based on how fast their partner peers are providing data
back to them.)  Peers that are downloading from the seeders the fastest
get priority.

This is one of the main ways that new peers can do well.  If they can
accept data quickly, they can get it very rapidly from seeder nodes,
even faster than other nodes that may be sharing with many peers.
And by biasing the data from seeder nodes towards peers that accept it
the fastest, BT spreads the data around as fast as possible, so that
there can be more peers with data and the whole network works well.

This set of policies works very well in practice and is a good part of
the reason for the success of BT.  It's pretty hard to tweak it and get
something that works better.

Hal

From eugen at leitl.org  Sun May 15 10:53:21 2005
From: eugen at leitl.org (Eugen Leitl)
Date: Sat Dec  9 22:12:53 2006
Subject: [p2p-hackers] [connellybarnes@yahoo.com: [i2p] Python filesharing
	Merkle hash tree magic]
Message-ID: <20050515105321.GA23832@leitl.org>

----- Forwarded message from "C. Barnes" <connellybarnes@yahoo.com> -----

From: "C. Barnes" <connellybarnes@yahoo.com>
Date: Thu, 12 May 2005 04:36:32 -0700 (PDT)
To: i2p@i2p.net
Subject: [i2p] Python filesharing Merkle hash tree magic


Topic: P2P filesharing ala BitTorrent.

Presenting public domain module 'chunk.py':

  http://oregonstate.edu/~barnesc/temp/chunk.py

This module does some magic with Merkle hash trees.
It allows one to publish a 20 byte SHA-1 digest for
a file, and encode 'chunks' from the file with
minimal overhead, such that any given chunk can be
verified against the file's 20 byte digest.
Cryptographic hashing is used to verify each chunk,
so chunks can be transmitted in any order, and it is
impossible to save corrupt data to disk if one has
the correct file hash.

One could use this module in any file sharing
application which divides files up into blocks.

For example, in a BitTorrent-like project it might
be desirable to keep the .torrent files minimal in
size.  Using this module, one needs to only publish
a hash like DA39A3EE5E6B4B0D3255BFEF95601890AFD80709
instead of a digest of all blocks in the file.

Pros:
 - Smaller digest size.
 - Can send chunks in any order.

Cons:
 - Each 'chunk' contains checksumming information.
   This overhead is less than 4% for files of size
   less than 1e20 bytes.

 - Connelly Barnes



		
__________________________________ 
Yahoo! Mail Mobile 
Take Yahoo! Mail with you! Check email on your mobile phone. 
http://mobile.yahoo.com/learn/mail 
_______________________________________________
i2p mailing list
i2p@i2p.net
http://i2p.dnsalias.net/mailman/listinfo/i2p

----- End forwarded message -----

From eugen at leitl.org  Sun May 15 10:53:52 2005
From: eugen at leitl.org (Eugen Leitl)
Date: Sat Dec  9 22:12:53 2006
Subject: [p2p-hackers] [connellybarnes@yahoo.com: Re: [i2p] Python
	filesharing Merkle hash tree magic]
Message-ID: <20050515105352.GB23832@leitl.org>

----- Forwarded message from "C. Barnes" <connellybarnes@yahoo.com> -----

From: "C. Barnes" <connellybarnes@yahoo.com>
Date: Thu, 12 May 2005 21:07:59 -0700 (PDT)
To: i2p@i2p.net
Subject: Re: [i2p] Python filesharing Merkle hash tree magic


I didn't consider multifile torrents.  I designed this
as a general purpose module, and didn't think too much
about BitTorrent.  But for a multifile torrent you
would need to publish the root Merkle hash for *each*
file.  This would take 20*(number of files) bytes.
Thanks for pointing that out.

Also, FYI the chunk.py module uses the bencode.py
module from the BitTorrent project.

--- closedshop <closedshop@gmx.de> wrote:
> What about multifile torrents ?
> 
> C. Barnes wrote:
> 
> >Topic: P2P filesharing ala BitTorrent.
> >
> >Presenting public domain module 'chunk.py':
> >
> >  http://oregonstate.edu/~barnesc/temp/chunk.py
> >
> >This module does some magic with Merkle hash trees.
> >It allows one to publish a 20 byte SHA-1 digest for
> >a file, and encode 'chunks' from the file with
> >minimal overhead, such that any given chunk can be
> >verified against the file's 20 byte digest.
> >Cryptographic hashing is used to verify each chunk,
> >so chunks can be transmitted in any order, and it
> is
> >impossible to save corrupt data to disk if one has
> >the correct file hash.
> >
> >One could use this module in any file sharing
> >application which divides files up into blocks.
> >
> >For example, in a BitTorrent-like project it might
> >be desirable to keep the .torrent files minimal in
> >size.  Using this module, one needs to only publish
> >a hash like
> DA39A3EE5E6B4B0D3255BFEF95601890AFD80709
> >instead of a digest of all blocks in the file.
> >
> >Pros:
> > - Smaller digest size.
> > - Can send chunks in any order.
> >
> >Cons:
> > - Each 'chunk' contains checksumming information.
> >   This overhead is less than 4% for files of size
> >   less than 1e20 bytes.
> >
> > - Connelly Barnes
> >
> >
> >
> >		
> >__________________________________ 
> >Yahoo! Mail Mobile 
> >Take Yahoo! Mail with you! Check email on your
> mobile phone. 
> >http://mobile.yahoo.com/learn/mail 
> >_______________________________________________
> >i2p mailing list
> >i2p@i2p.net
> >http://i2p.dnsalias.net/mailman/listinfo/i2p
> >
> >
> >  
> >
> 
> 


		
__________________________________ 
Yahoo! Mail Mobile 
Take Yahoo! Mail with you! Check email on your mobile phone. 
http://mobile.yahoo.com/learn/mail 
_______________________________________________
i2p mailing list
i2p@i2p.net
http://i2p.dnsalias.net/mailman/listinfo/i2p

----- End forwarded message -----

From arnetheduck at gmail.com  Sat May 14 10:29:00 2005
From: arnetheduck at gmail.com (Jacek Sieka)
Date: Sat Dec  9 22:12:53 2006
Subject: [p2p-hackers] [connellybarnes@yahoo.com: [i2p] Python filesharing
	Merkle hash tree magic]
In-Reply-To: <20050515105321.GA23832@leitl.org>
References: <20050515105321.GA23832@leitl.org>
Message-ID: <4285D2EC.80408@gmail.com>

Hi,

Nice, just wanted to note that there are many links out there that use Tiger (which apparently just got included in the linux kernel, so someone trusts it a little bit at least) instead of SHA1 together with merkle trees (dubbed tth's often), just in case you want to benefit from existing link sites such as bitzi.com...on the other hand, yet another hash type is always fun =)

FYI 2; if anyone's interested, the DC++ source code contains a C++ templated merkle tree implementation that can be used to generate trees with any hash (oh, and there's probably a chunk checker in there as well, although it's probably ugly =) - if anyone's interested it could be scavenged, polished and made cooperative with some standard hash function library, currently it expects a (fairly simple) dc++ specific interface to the hash fcn...

regards
/Jacek 

Eugen Leitl wrote:
> ----- Forwarded message from "C. Barnes" <connellybarnes@yahoo.com> -----
> 
> Topic: P2P filesharing ala BitTorrent.
> 
> Presenting public domain module 'chunk.py':
> 
>   http://oregonstate.edu/~barnesc/temp/chunk.py
> 
> This module does some magic with Merkle hash trees.
> It allows one to publish a 20 byte SHA-1 digest for
> a file, and encode 'chunks' from the file with
> minimal overhead, such that any given chunk can be
> verified against the file's 20 byte digest.
> Cryptographic hashing is used to verify each chunk,
> so chunks can be transmitted in any order, and it is
> impossible to save corrupt data to disk if one has
> the correct file hash.
> 
> One could use this module in any file sharing
> application which divides files up into blocks.
> 
> For example, in a BitTorrent-like project it might
> be desirable to keep the .torrent files minimal in
> size.  Using this module, one needs to only publish
> a hash like DA39A3EE5E6B4B0D3255BFEF95601890AFD80709
> instead of a digest of all blocks in the file.
> 
> Pros:
>  - Smaller digest size.
>  - Can send chunks in any order.
> 
> Cons:
>  - Each 'chunk' contains checksumming information.
>    This overhead is less than 4% for files of size
>    less than 1e20 bytes.
> 
>  - Connelly Barnes

From arachnid at notdot.net  Sat May 14 23:26:37 2005
From: arachnid at notdot.net (Nick Johnson)
Date: Sat Dec  9 22:12:53 2006
Subject: [p2p-hackers] [connellybarnes@yahoo.com: [i2p] Python filesharing
	Merkle hash tree magic]
In-Reply-To: <20050515105321.GA23832@leitl.org>
References: <20050515105321.GA23832@leitl.org>
Message-ID: <4286892D.1040201@notdot.net>

How does this work? You don't provide details here or in the .py file - 
does each chunk include the hashes of all its siblings, and its parents' 
siblings, etc?

-Nick Johnson

Eugen Leitl wrote:

>----- Forwarded message from "C. Barnes" <connellybarnes@yahoo.com> -----
>
>From: "C. Barnes" <connellybarnes@yahoo.com>
>Date: Thu, 12 May 2005 04:36:32 -0700 (PDT)
>To: i2p@i2p.net
>Subject: [i2p] Python filesharing Merkle hash tree magic
>
>
>Topic: P2P filesharing ala BitTorrent.
>
>Presenting public domain module 'chunk.py':
>
>  http://oregonstate.edu/~barnesc/temp/chunk.py
>
>This module does some magic with Merkle hash trees.
>It allows one to publish a 20 byte SHA-1 digest for
>a file, and encode 'chunks' from the file with
>minimal overhead, such that any given chunk can be
>verified against the file's 20 byte digest.
>Cryptographic hashing is used to verify each chunk,
>so chunks can be transmitted in any order, and it is
>impossible to save corrupt data to disk if one has
>the correct file hash.
>
>One could use this module in any file sharing
>application which divides files up into blocks.
>
>For example, in a BitTorrent-like project it might
>be desirable to keep the .torrent files minimal in
>size.  Using this module, one needs to only publish
>a hash like DA39A3EE5E6B4B0D3255BFEF95601890AFD80709
>instead of a digest of all blocks in the file.
>
>Pros:
> - Smaller digest size.
> - Can send chunks in any order.
>
>Cons:
> - Each 'chunk' contains checksumming information.
>   This overhead is less than 4% for files of size
>   less than 1e20 bytes.
>
> - Connelly Barnes
>
>
>
>		
>__________________________________ 
>Yahoo! Mail Mobile 
>Take Yahoo! Mail with you! Check email on your mobile phone. 
>http://mobile.yahoo.com/learn/mail 
>_______________________________________________
>i2p mailing list
>i2p@i2p.net
>http://i2p.dnsalias.net/mailman/listinfo/i2p
>
>----- End forwarded message -----
>_______________________________________________
>p2p-hackers mailing list
>p2p-hackers@zgp.org
>http://zgp.org/mailman/listinfo/p2p-hackers
>_______________________________________________
>Here is a web page listing P2P Conferences:
>http://www.neurogrid.net/twiki/bin/view/Main/PeerToPeerConferences
>
>!DSPAM:4285c8e4292214151599692!
>
>  
>


From bert at web2peer.com  Sun May 15 01:14:24 2005
From: bert at web2peer.com (bert@web2peer.com)
Date: Sat Dec  9 22:12:53 2006
Subject: [p2p-hackers] [connellybarnes@yahoo.com: [i2p] Python
	filesharing	Merkle hash tree magic]
Message-ID: <20050515011424.651B91096A0@ws6-4.us4.outblaze.com>

Merkle trees [1] are a cryptographic approach to authenticity/integrity that have several properties that make them useful in p2p settings. Each leaf contains the hash of one file block/chunk, and the internal nodes are themselves hashes of the hashes represented by their children.  To support verification, the server needs to provide only lg(N) hashes with each chunk served, where N is the total number of chunks. These "auxiliary hashes" that must be returned with each chunk correspond to the tree nodes that are siblings of those nodes along the "authentication path" from leaf to root.

I've just returned from WWW-2005 where I was advocating the use of Merkle trees for use in authenticating HTTP responses: http://www.almaden.ibm.com/cs/people/bayardo/ps/www2005.pdf.  This provides an efficient means for authenticity guarantees even when using p2p (untrusted) web content caching & delivery networks. No rocket science here, but an idea that I believe whose time has come.

And as an aside, multi-file torrents need not require multiple root hashes. A single merkle tree (& hence root hash) would suffice for authenticating arbitrary chunks so long as you appropriately combined hashes. I think if you read the above-linked proposal for authenticating HTTP responses (which requires one root hash per multi-file repository), it should be clear how multi-file torrents could be supported.


[1] R. C. Merkle. Protocols for public key cryptosystems. In Symposium
on Security and Privacy, 122-134, 1980.

----- Original Message -----

From: "Nick Johnson" <arachnid@notdot.net>
To: "Peer-to-peer development." <p2p-hackers@zgp.org>
Subject: Re: [p2p-hackers] [connellybarnes@yahoo.com: [i2p] Python filesharing	Merkle hash tree magic]
Date: Sun, 15 May 2005 11:26:37 +1200

> 
> How does this work? You don't provide details here or in the .py file - does 
> each chunk include the hashes of all its siblings, and its parents' siblings, 
> etc?
> 
> -Nick Johnson
> 
> Eugen Leitl wrote:
> 
> > ----- Forwarded message from "C. Barnes" <connellybarnes@yahoo.com> -----
> >
> > From: "C. Barnes" <connellybarnes@yahoo.com>
> > Date: Thu, 12 May 2005 04:36:32 -0700 (PDT)
> > To: i2p@i2p.net
> > Subject: [i2p] Python filesharing Merkle hash tree magic
> >
> >
> > Topic: P2P filesharing ala BitTorrent.
> >
> > Presenting public domain module 'chunk.py':
> >
> >  http://oregonstate.edu/~barnesc/temp/chunk.py
> >
> > This module does some magic with Merkle hash trees.
> > It allows one to publish a 20 byte SHA-1 digest for
> > a file, and encode 'chunks' from the file with
> > minimal overhead, such that any given chunk can be
> > verified against the file's 20 byte digest.
> > Cryptographic hashing is used to verify each chunk,
> > so chunks can be transmitted in any order, and it is
> > impossible to save corrupt data to disk if one has
> > the correct file hash.
> >
> > One could use this module in any file sharing
> > application which divides files up into blocks.
> >
> > For example, in a BitTorrent-like project it might
> > be desirable to keep the .torrent files minimal in
> > size.  Using this module, one needs to only publish
> > a hash like DA39A3EE5E6B4B0D3255BFEF95601890AFD80709
> > instead of a digest of all blocks in the file.
> >
> > Pros:
> > - Smaller digest size.
> > - Can send chunks in any order.
> >
> > Cons:
> > - Each 'chunk' contains checksumming information.
> >   This overhead is less than 4% for files of size
> >   less than 1e20 bytes.
> >
> > - Connelly Barnes
> >
> >
> >
> > 		
> > __________________________________ Yahoo! Mail Mobile Take Yahoo! Mail with 
> > you! Check email on your mobile phone. http://mobile.yahoo.com/learn/mail 
> > _______________________________________________
> > i2p mailing list
> > i2p@i2p.net
> > http://i2p.dnsalias.net/mailman/listinfo/i2p
> >
> > ----- End forwarded message -----
> > _______________________________________________
> > p2p-hackers mailing list
> > p2p-hackers@zgp.org
> > http://zgp.org/mailman/listinfo/p2p-hackers
> > _______________________________________________
> > Here is a web page listing P2P Conferences:
> > http://www.neurogrid.net/twiki/bin/view/Main/PeerToPeerConferences
> >
> > !DSPAM:4285c8e4292214151599692!
> >
> >
> >
> 
> _______________________________________________
> p2p-hackers mailing list
> p2p-hackers@zgp.org
> http://zgp.org/mailman/listinfo/p2p-hackers
> _______________________________________________
> Here is a web page listing P2P Conferences:
> http://www.neurogrid.net/twiki/bin/view/Main/PeerToPeerConferences


From verdimar at comp.nus.edu.sg  Tue May 17 09:53:49 2005
From: verdimar at comp.nus.edu.sg (Verdi March)
Date: Sat Dec  9 22:12:53 2006
Subject: [p2p-hackers] Skipgraph and global balancing operations
Message-ID: <Pine.GSO.4.58.0505171750300.21946@sf3.comp.nus.edu.sg>

Hi,

in the skipgraph paper [1], in section 2 it's mentioned that
skip lists require no global balancing operations.

I'm not clear at the "global balancing operations" statement:
what are the operations and why skip lists don't need those.

Any help is appreciated.


Regards,
Verdi


From verdimar at comp.nus.edu.sg  Tue May 17 09:56:19 2005
From: verdimar at comp.nus.edu.sg (Verdi March)
Date: Sat Dec  9 22:12:53 2006
Subject: [p2p-hackers] Re: Skipgraph and global balancing operations
In-Reply-To: <Pine.GSO.4.58.0505171750300.21946@sf3.comp.nus.edu.sg>
References: <Pine.GSO.4.58.0505171750300.21946@sf3.comp.nus.edu.sg>
Message-ID: <Pine.GSO.4.58.0505171754190.21946@sf3.comp.nus.edu.sg>

Forgot the reference, but I'm sure most of you have already
know that :).

[1] James Aspnes and Gauri Shah, Skip Graphs, Proceedings of the 14th
Annual ACM-SIAM Symposium on Discrete Algorithms, January 2003.

Regards,
Verdi


From wolfgang.mueller at wiai.uni-bamberg.de  Tue May 17 12:08:13 2005
From: wolfgang.mueller at wiai.uni-bamberg.de (Wolfgang =?iso-8859-1?q?M=FCller?=)
Date: Sat Dec  9 22:12:53 2006
Subject: [p2p-hackers] Rumorama: Scalable Summary Based Retrieval in P2P
	Networks
Message-ID: <200505171408.13191.wolfgang.mueller@wiai.uni-bamberg.de>

Dear colleagues,

In the tech report 
http://mi.wiai.uni-bamberg.de/research/publications/bibtex/mueller2005scalable 
we present Rumorama, a protocol to make summary-based retrieval in P2P 
networks scalable. 

Summary-based retrieval in P2P networks, as introduced by PlanetP, uses 
summaries of peer data to select peers to contact for a retrieval process. In 
other words, PlanetP adapts GlOSS-like distributed IR techniques to P2P 
networks. The advantage of this method is that only very little indexing data 
has to be shipped as compared to distributed indexing structures, giving room  
for redundant storage of these summaries. Furthermore, summary-based 
retrieval needs to ship orders of magnitude less data in some situations, 
e.g. when processing multi-word boolean queries, than methods based on 
inverted lists (see e.g. the Li et al. paper on the feasibility of keyword 
search in P2P-networks for some words about the problems to be solved for 
inverted file based indexing 
http://pdos.csail.mit.edu/~rtm/papers/search_feasibility.ps). The drawback of 
summary-based retrieval is - up to Rumorama - that PlanetP is not scalable.

Rumorama achieves scalability: the number of neighbors of each peer grows 
logarithmically with the number of peers in the network, the communication 
cost per peer for maintaining the network also grows logarithmically with 
network size. Lastly, queries are processed in a number of hops (i.e. depth 
of our multicast tree) that grows logarithmically with the network's size. 

However, the number of nodes to be contacted within a query grows linearly 
with the network size. This is a known effect from distributed information 
retrieval. However, depending on the query model, the absolute number of 
peers that has to be contacted for answering a query is very small with 
respect to network size.

Enjoy the paper! We would be thankful for some feedback and criticism on it.

Cheers,
Wolfgang M?ller
Martin Eisenhardt
Andreas Henrich
-- 
Dr. Wolfgang M?ller
LS Medieninformatik
Universit?t Bamberg

From m.rogers at cs.ucl.ac.uk  Tue May 17 17:06:52 2005
From: m.rogers at cs.ucl.ac.uk (Michael Rogers)
Date: Sat Dec  9 22:12:53 2006
Subject: [p2p-hackers] Skipgraph and global balancing operations
In-Reply-To: <Pine.GSO.4.58.0505171750300.21946@sf3.comp.nus.edu.sg>
References: <Pine.GSO.4.58.0505171750300.21946@sf3.comp.nus.edu.sg>
Message-ID: <428A24AC.5000208@cs.ucl.ac.uk>

I think it's referring to the fact that, unlike trees, skip lists don't 
need to be rebalanced as nodes are added and removed. AFAIK the reason 
is that the rank of each item in the list (ie the number of pointers the 
item keeps) is chosen randomly, so the higher "layers" are replaced 
incrementally as a natural result of churn.

Cheers,
Michael

Verdi March wrote:

>Hi,
>
>in the skipgraph paper [1], in section 2 it's mentioned that
>skip lists require no global balancing operations.
>
>I'm not clear at the "global balancing operations" statement:
>what are the operations and why skip lists don't need those.
>
>Any help is appreciated.
>
>
>Regards,
>Verdi
>
>_______________________________________________
>p2p-hackers mailing list
>p2p-hackers@zgp.org
>http://zgp.org/mailman/listinfo/p2p-hackers
>_______________________________________________
>Here is a web page listing P2P Conferences:
>http://www.neurogrid.net/twiki/bin/view/Main/PeerToPeerConferences
>  
>


From kerry at vscape.com  Tue May 17 18:53:57 2005
From: kerry at vscape.com (Kerry L. Bonin)
Date: Sat Dec  9 22:12:53 2006
Subject: [p2p-hackers] Multiple value per key query/response syntax?
Message-ID: <428A3DC5.6090106@vscape.com>

Hello!

I'm looking for any good discussions or examples of reasonably robust 
request/response formats for multiple transaction multimap queries in 
general, and multiple-value per key DHT in particular. 

I was considering using expiration timestamp windowing in the query as a 
sort of crude iterator filter for values that expire at a constant 
interval after storage or refresh, but I'm wondering about variable TTL 
vs. variable refresh intervals, vs. classic remote iterator design.

Any pointers would be greatly appreciated!

Kerry


From tangli03 at mails.tsinghua.edu.cn  Wed May 18 09:55:42 2005
From: tangli03 at mails.tsinghua.edu.cn (Li Tang)
Date: Sat Dec  9 22:12:53 2006
Subject: [p2p-hackers] Has CAN been implemented by any open source project?
In-Reply-To: <316355152.11145@mails.tsinghua.edu.cn>
Message-ID: <20050518094753.CB1863FC2A@capsicum.zgp.org>

Hello, guys!

   I am looking for some open source resources of DHT algorithms. While I
have found projects related to Chord, Pastry, and Tapestry, I got little
implementation of CAN. 
   Is anybody aware of related information? Any pointers, not limited to
CAN, would be greatly appreciated!

Sincerely,
Li



From baford at mit.edu  Wed May 18 18:59:21 2005
From: baford at mit.edu (Bryan Ford)
Date: Sat Dec  9 22:12:53 2006
Subject: [p2p-hackers] Official IETF behavior recommendations for NAT
	relevant to P2P
Message-ID: <200505181459.21722.baford@mit.edu>

Dear p2p-hackers,

A discussion is currently taking place on the IETF BEHAVE working group 
mailing list (see http://www1.ietf.org/html.charters/behave-charter.html) 
that is highly relevant to P2P application writers.  For those of you not 
already familiar with it, this IETF working group is defining standards for 
how NATs should behave so as to better support applications including P2P 
apps.  The chair has made a "Last Call" for comments on the current draft 
document specifying NAT behavior for UDP, so this may be the last chance we 
have for a while to influence how well the next generation of NATs support 
UDP-based P2P applications.

** Address-Restricted vs Full Cone NAT **

There's one issue in particular on which I personally fear the working group 
is currently headed in the wrong direction, and over which you folks on this 
list could perhaps exert considerable influence if you're willing to take a 
little time and make your voice heard on the BEHAVE list.  The current UDP 
draft, draft-ietf-behave-nat-udp-01.txt, recommends that NATs implement 
"Endpoint address-dependent filtering behavior", otherwise known as 
"Address-Restricted Cone NAT".  In other words, once a NATted UDP application 
opens a UDP session through a NAT to some external host, giving the 
application a public (NATted) port number, the NAT only accepts and forwards 
subsequent incoming UDP packets at that public port if they come from a 
source IP address that the application has previously sent UDP packets to.  
In other words, the NAT must see outgoing UDP traffic to a remote host before 
it will accept incoming traffic from that host on the application's public 
port.  In the proposed alternative, "Endpoint-independent filtering" or "Full 
Cone NAT", the NAT forwards any incoming traffic to the correct public port 
regardless of source IP address or port number.

Address-Restricted Cone NAT is moderately "P2P-friendly" in that it at least 
allows NATted applications to "hole punch" P2P connections with the help of a 
well-known (non-NATted) rendezvous server.  Address-Restricted Cone NAT still 
does not, however, allow the application to have a full "first-class" 
Internet presence (i.e., public IP address and UDP port number) at which it 
can be contacted proactively by any other Internet-connected host.  A P2P 
application behind an Address-Restricted Cone NAT is still forever dependent 
on other public (first-class) rendezvous servers to help establish each P2P 
communication session it needs.  A P2P app behind a Full Cone NAT, in 
contrast, is a full "first-class" network citizen itself, and can even _be_ a 
public rendezvous server for other less fortunate hosts once it learns its 
own public IP address and UDP port number.  This is an important benefit to 
applications, especially P2P applications that are trying to be highly 
decentralized.

The main argument for Restricted Cone NAT, of course, is security.  Until 
recently I believed this argument myself, but after further and more careful 
analysis I have come to the conclusion that Restricted filtering provides a 
very week security benefit in comparison with the fundamental loss in 
first-class application connectivity it entails.  My current proposal on the 
BEHAVE list does NOT mandate Full Cone NAT, but it does specify that NATs 
should at least be Full Cone in their default, "out of the box" configuration 
as provided by NAT vendors, and recommends that NATs implement Restricted 
Cone NAT as an administrator-configurable security option.  The current 
proposed text reads as follows:

   REQ-7: NATs MUST implement Endpoint-Independent
          Filtering in their default configuration.
          NATs SHOULD be configurable to support
          Address-Specific Filtering or Endpoint-Specific Filtering
          at the option of the system administrator.

If you have thoughts or opinions on this topic, I would greatly appreciate 
hearing them (and having other participants hear them) on the BEHAVE list.

** Why the Security Benefit of Restricted Cone NAT is Minimal **

Restrictive filtering ostensibly has the benefit of protecting applications on 
an internal host from traffic the internal host is not expecting, i.e., from 
traffic from previously unknown external hosts or endpoints.  This behavior 
can only possibly benefit the application if the application only ever 
initiates outgoing sessions from its internal port, and is obviously a curse 
if the application ever wants to accept incoming sessions from external 
hosts.  The NAT has no way of knowing which is the case.  But suppose the 
application is one that only initiates outgoing connections (i.e., a 
"traditional client/server app"), and thus might "potentially" benefit from a 
NAT's restrictive filtering behavior.  Consider TCP and UDP separately:

        ** TCP: When a client/server app makes an outgoing TCP connection, it 
typically does a "connect()" call to the host OS's sockets API without 
bothering to bind() the socket, and the OS's TCP stack allocates a local TCP 
port exclusively for the use of that connection.  Any TCP packets the host OS 
subsequently received on that port from endpoints other than the one it 
connect()ed to will be summarily rejected with TCP RSTs by the host OS, and 
the application never even sees any hint that they ever arrived.  Even if the 
application does a bind() before its connect() in order to use a specific 
source port (e.g., a <1024 "privileged" port), it still almost always obtains 
exclusive use of that port.  An application indeed has to work really bloody 
hard to set things up so that it can accept incoming TCP connections on the 
same port it is using for an outgoing connection, using SO_REUSEADDR and 
multiple sockets.  No traditional client/server application is going to go to 
all this effort just in order to create a security hole. :)  In other words, 
for TCP, restrictive filtering in practice provides absolutely zero 
protection from the _application_: the only entity that the restrictive 
filtering even affects is the host OS's protocol stack.  Restrictive 
filtering might for example benefit a host OS with a TCP stack that is so 
broken that it fails to check the source IP address and port number on 
incoming TCP packets at all, but if the host OS is _that_ broken then it 
undoubtedly has many other problems that the NAT can't possibly guard 
against.

        ** UDP: When a client/server UDP app uses a UDP port for communication 
with just one remote host, for convenience it typically does a connect() to 
the appropriate remote UDP endpoint, so that it can just send messages with 
send() or write() without having to specify a destination address for each 
datagram, and so that it can receive messages with read() or recv() without 
having to check the source address and port number in each message received.  
In this case, the host OS effectively provides the external endpoint 
filtering functionality as for TCP, and any restrictive filtering implemented 
by the NAT again provides no security benefit at all to the application.  A 
UDP application that uses a single UDP port for communication with multiple 
remote hosts, in contrast, must generally check the source IP addresses and 
port numbers in the datagrams it receives just in order to distinguish 
between the multiple communication sessions it is involved in, and hence it 
will inherently provide its own external endpoint filtering.  It's 
conceivable that some abysmally buggy UDP applications might benefit from the 
NAT's external filtering - e.g., a UDP application that only talks to one 
external endpoint but doesn't use connect() and doesn't check the source 
endpoint on incoming datagrams - but an application that is so seriously 
broken probably has many other security vulnerabilities that can be exploited 
despite the NAT's external filtering behavior, e.g., by finding out or 
guessing the public server the application is talking to (which is easy in 
many cases such as commercial games or services where there are only a few 
well-known servers) and sending packets with spoofed source IP address and 
UDP port numbers.  Besides, many, perhaps the majority, of UDP applications 
use UDP _because_ they need P2P-style communication, and so external 
filtering by the NAT provides no practical security benefit and only causes 
interference with the application.

Another common argument for Restricted Cone NAT (Address-dependent filtering) 
is that it protects the resources on the internal network from being consumed 
forwarding bogus packets even if the eventual end application will correctly 
discard them.  This argument is also weak, however:

	** Even with more permissive Full Cone NAT (endpoint-independent filtering), 
the NAT will still in practice filter out the vast majority of bogus traffic 
that reaches it, because most of the NAT's ports are typically not in active 
use by applications behind the NAT.  The only bogus traffic a Full Cone NAT 
will let through is bogus traffic that happens to hit the _specific_ external 
port numbers that correspond to active port bindings.  These port numbers are 
assigned more-or-less randomly (i.e., unpredictably) by the NAT, and they only 
ever appear in the high, non-well-known port number space, so most of the 
Internet "background noise" caused by viruses or other attacks on well-known 
ports will never get through in any case, and purely random "port guessing" 
attacks would have to send packets to a great many NAT ports in order to get 
a few packets through.  Thus, the incremental benefit of restrictive 
filtering over Full Cone NAT with regards to general bogus Internet traffic 
is very small.

	** An attacker might deliberately send bogus packets to a bound NAT port 
known to be in use.  But if the attacker can find out which NAT port the 
application is using, then the attacker probably also knows (or can easily 
find out or guess) the address and port number of the _remote_ host that the 
application is talking to and just spoof that source IP address and port 
number in its attack packets, getting around Restricted filtering as well.  
In fact the remote host's endpoint is in practice likely to be much easier 
for the attacker to find out or guess than the application's public NAT port, 
because of the overwhelming prevalence of well-known servers and ports on the 
Internet in comparison to the randomness and relative obscurity of temporary 
NAT-assigned public ports.  So the practical benefit that Restricted 
filtering provides even against directed attacks is still minimal.

	** Finally, the goal of "protecting the resources on the internal network 
from being consumed forwarding bogus packets" is really only a consideration 
on large corporate or ISP networks with substantial internal traffic load, 
and if this minimization of resource use is a concern then the network 
administrator can certainly be expected to know how to configure the 
company's NAT to enable Restricted filtering.  As I pointed out above, under 
my proposal a NAT can use Restricted filtering and still be BEHAVE-compliant 
as long as it is the administrator (not the NAT vendor) that enables this 
firewall behavior.

	** For home NATs, in contrast, minimizing resource load on the internal 
network due to bogus traffic is completely a non-issue because the home 
network (Ethernet or 802.11 or whatever) is typically so much faster than the 
Internet uplink anyway.  In this case it's much more important that NATs 
consistently support applications well, which my proposal ensures by 
requiring that the NAT's "out of the box" configuration is 
Endpoint-independent Filtering.  Joe Clueless who buys a home NAT router 
probably doesn't have any idea what NAT actually is, but he wants his 
applications to work.

** Conclusion **

In summary, restrictive filtering in the NAT interferes with applications 
substantially and provides very little real security benefit in 
practice, considering the way application communication and NAT port 
assignment actually works.  As such, especially considering the purpose of 
the BEHAVE working group is to increase the predictability and compatibility 
of NATs with applications and not to specify standards for firewall 
functionality, it seems highly inappropriate for the WG's documents to 
recommend firewall functionality that interferes with applications while 
providing dubious security benefits.  On the contrary, I think the WG should 
at least recommend, if not mandate, that NATs provide Endpoint-Independent 
Filtering (Full Cone NAT) by default.  NAT vendors or system administrators 
that are particularly paranoid about security and perceive the security 
benefits to be worth the interference it causes with applications can still 
enable more restrictive filtering policies, but that's a firewall deployment 
issue, not a NAT functionality issue.  We need to steer clear from 
recommending firewall functionality.

Thanks very much for your attention.  Please direct followups to the BEHAVE 
working group list, "ietf-behave@list.sipfoundry.org".  Thanks!

Bryan

From larytet.8753341 at bloglines.com  Wed May 18 21:07:56 2005
From: larytet.8753341 at bloglines.com (larytet.8753341@bloglines.com)
Date: Sat Dec  9 22:12:53 2006
Subject: [p2p-hackers] Official IETF behavior recommendations for NAT
	relevant to P2P
Message-ID: <1116450476.3435399312.6473.sendItem@bloglines.com>

dear Bryan, feel free to forward this email to IETF.

best regrads, Arkady



----------------------------------------------
hi,
i am a principle
developer
of Rodi project http://larytet.sourceforge.net/btRat.shtml 

i fully support
opinion expressed in this email http://zgp.org/pipermail/p2p-hackers/2005-May/002611.html



i think that it's a right thing to do to let (by default) packets from

any peer in the outside world to go through "hole" in NAT in case of UDP.




Arkady



--- Peer-to-peer development." <p2p-hackers@zgp.org wrote:

Dear p2p-hackers,
> 
> A discussion is currently taking place on the IETF
BEHAVE working group 
> mailing list (see http://www1.ietf.org/html.charters/behave-charter.html)

> that is highly relevant to P2P application writers.  For those of you
not 
> already familiar with it, this IETF working group is defining standards
for 
> how NATs should behave so as to better support applications including
P2P 
> apps.  The chair has made a "Last Call" for comments on the current
draft 
> document specifying NAT behavior for UDP, so this may be the last
chance we 
> have for a while to influence how well the next generation of
NATs support 
> UDP-based P2P applications.
> 
> ** Address-Restricted
vs Full Cone NAT **
> 
> There's one issue in particular on which I personally
fear the working group 
> is currently headed in the wrong direction, and
over which you folks on this 
> list could perhaps exert considerable influence
if you're willing to take a 
> little time and make your voice heard on the
BEHAVE list.  The current UDP 
> draft, draft-ietf-behave-nat-udp-01.txt,
recommends that NATs implement 
> "Endpoint address-dependent filtering behavior",
otherwise known as 
> "Address-Restricted Cone NAT".  In other words, once
a NATted UDP application 
> opens a UDP session through a NAT to some external
host, giving the 
> application a public (NATted) port number, the NAT only
accepts and forwards 
> subsequent incoming UDP packets at that public port
if they come from a 
> source IP address that the application has previously
sent UDP packets to.  
> In other words, the NAT must see outgoing UDP traffic
to a remote host before 
> it will accept incoming traffic from that host
on the application's public 
> port.  In the proposed alternative, "Endpoint-independent
filtering" or "Full 
> Cone NAT", the NAT forwards any incoming traffic to
the correct public port 
> regardless of source IP address or port number.

> 
> Address-Restricted Cone NAT is moderately "P2P-friendly" in that it
at least 
> allows NATted applications to "hole punch" P2P connections with
the help of a 
> well-known (non-NATted) rendezvous server.  Address-Restricted
Cone NAT still 
> does not, however, allow the application to have a full
"first-class" 
> Internet presence (i.e., public IP address and UDP port
number) at which it 
> can be contacted proactively by any other Internet-connected
host.  A P2P 
> application behind an Address-Restricted Cone NAT is still
forever dependent 
> on other public (first-class) rendezvous servers to
help establish each P2P 
> communication session it needs.  A P2P app behind
a Full Cone NAT, in 
> contrast, is a full "first-class" network citizen
itself, and can even _be_ a 
> public rendezvous server for other less fortunate
hosts once it learns its 
> own public IP address and UDP port number.  This
is an important benefit to 
> applications, especially P2P applications that
are trying to be highly 
> decentralized.
> 
> The main argument for Restricted
Cone NAT, of course, is security.  Until 
> recently I believed this argument
myself, but after further and more careful 
> analysis I have come to the
conclusion that Restricted filtering provides a 
> very week security benefit
in comparison with the fundamental loss in 
> first-class application connectivity
it entails.  My current proposal on the 
> BEHAVE list does NOT mandate Full
Cone NAT, but it does specify that NATs 
> should at least be Full Cone in
their default, "out of the box" configuration 
> as provided by NAT vendors,
and recommends that NATs implement Restricted 
> Cone NAT as an administrator-configurable
security option.  The current 
> proposed text reads as follows:
> 
> 
  REQ-7: NATs MUST implement Endpoint-Independent
>           Filtering in
their default configuration.
>           NATs SHOULD be configurable to support

>           Address-Specific Filtering or Endpoint-Specific Filtering
>
          at the option of the system administrator.
> 
> If you have thoughts
or opinions on this topic, I would greatly appreciate 
> hearing them (and
having other participants hear them) on the BEHAVE list.
> 
> ** Why the
Security Benefit of Restricted Cone NAT is Minimal **
> 
> Restrictive filtering
ostensibly has the benefit of protecting applications on 
> an internal host
from traffic the internal host is not expecting, i.e., from 
> traffic from
previously unknown external hosts or endpoints.  This behavior 
> can only
possibly benefit the application if the application only ever 
> initiates
outgoing sessions from its internal port, and is obviously a curse 
> if
the application ever wants to accept incoming sessions from external 
> hosts.
 The NAT has no way of knowing which is the case.  But suppose the 
> application
is one that only initiates outgoing connections (i.e., a 
> "traditional
client/server app"), and thus might "potentially" benefit from a 
> NAT's
restrictive filtering behavior.  Consider TCP and UDP separately:
> 
> 
       ** TCP: When a client/server app makes an outgoing TCP connection,
it 
> typically does a "connect()" call to the host OS's sockets API without

> bothering to bind() the socket, and the OS's TCP stack allocates a local
TCP 
> port exclusively for the use of that connection.  Any TCP packets
the host OS 
> subsequently received on that port from endpoints other than
the one it 
> connect()ed to will be summarily rejected with TCP RSTs by
the host OS, and 
> the application never even sees any hint that they ever
arrived.  Even if the 
> application does a bind() before its connect() in
order to use a specific 
> source port (e.g., a <1024 "privileged" port),
it still almost always obtains 
> exclusive use of that port.  An application
indeed has to work really bloody 
> hard to set things up so that it can
accept incoming TCP connections on the 
> same port it is using for an outgoing
connection, using SO_REUSEADDR and 
> multiple sockets.  No traditional client/server
application is going to go to 
> all this effort just in order to create
a security hole. :)  In other words, 
> for TCP, restrictive filtering in
practice provides absolutely zero 
> protection from the _application_: the
only entity that the restrictive 
> filtering even affects is the host OS's
protocol stack.  Restrictive 
> filtering might for example benefit a host
OS with a TCP stack that is so 
> broken that it fails to check the source
IP address and port number on 
> incoming TCP packets at all, but if the
host OS is _that_ broken then it 
> undoubtedly has many other problems that
the NAT can't possibly guard 
> against.
> 
>         ** UDP: When a client/server
UDP app uses a UDP port for communication 
> with just one remote host, for
convenience it typically does a connect() to 
> the appropriate remote UDP
endpoint, so that it can just send messages with 
> send() or write() without
having to specify a destination address for each 
> datagram, and so that
it can receive messages with read() or recv() without 
> having to check
the source address and port number in each message received.  
> In this
case, the host OS effectively provides the external endpoint 
> filtering
functionality as for TCP, and any restrictive filtering implemented 
> by
the NAT again provides no security benefit at all to the application.  A 

> UDP application that uses a single UDP port for communication with multiple

> remote hosts, in contrast, must generally check the source IP addresses
and 
> port numbers in the datagrams it receives just in order to distinguish

> between the multiple communication sessions it is involved in, and hence
it 
> will inherently provide its own external endpoint filtering.  It's

> conceivable that some abysmally buggy UDP applications might benefit from
the 
> NAT's external filtering - e.g., a UDP application that only talks
to one 
> external endpoint but doesn't use connect() and doesn't check the
source 
> endpoint on incoming datagrams - but an application that is so
seriously 
> broken probably has many other security vulnerabilities that
can be exploited 
> despite the NAT's external filtering behavior, e.g.,
by finding out or 
> guessing the public server the application is talking
to (which is easy in 
> many cases such as commercial games or services where
there are only a few 
> well-known servers) and sending packets with spoofed
source IP address and 
> UDP port numbers.  Besides, many, perhaps the majority,
of UDP applications 
> use UDP _because_ they need P2P-style communication,
and so external 
> filtering by the NAT provides no practical security benefit
and only causes 
> interference with the application.
> 
> Another common
argument for Restricted Cone NAT (Address-dependent filtering) 
> is that
it protects the resources on the internal network from being consumed 
>
forwarding bogus packets even if the eventual end application will correctly

> discard them.  This argument is also weak, however:
> 
> 	** Even with
more permissive Full Cone NAT (endpoint-independent filtering), 
> the NAT
will still in practice filter out the vast majority of bogus traffic 
> that
reaches it, because most of the NAT's ports are typically not in active 
> use by applications behind the NAT.  The only bogus traffic a Full Cone
NAT 
> will let through is bogus traffic that happens to hit the _specific_
external 
> port numbers that correspond to active port bindings.  These
port numbers are 
> assigned more-or-less randomly (i.e., unpredictably)
by the NAT, and they only 
> ever appear in the high, non-well-known port
number space, so most of the 
> Internet "background noise" caused by viruses
or other attacks on well-known 
> ports will never get through in any case,
and purely random "port guessing" 
> attacks would have to send packets to
a great many NAT ports in order to get 
> a few packets through.  Thus, the
incremental benefit of restrictive 
> filtering over Full Cone NAT with regards
to general bogus Internet traffic 
> is very small.
> 
> 	** An attacker
might deliberately send bogus packets to a bound NAT port 
> known to be
in use.  But if the attacker can find out which NAT port the 
> application
is using, then the attacker probably also knows (or can easily 
> find out
or guess) the address and port number of the _remote_ host that the 
> application
is talking to and just spoof that source IP address and port 
> number in
its attack packets, getting around Restricted filtering as well.  
> In fact
the remote host's endpoint is in practice likely to be much easier 
> for
the attacker to find out or guess than the application's public NAT port,

> because of the overwhelming prevalence of well-known servers and ports
on the 
> Internet in comparison to the randomness and relative obscurity
of temporary 
> NAT-assigned public ports.  So the practical benefit that
Restricted 
> filtering provides even against directed attacks is still minimal.

> 
> 	** Finally, the goal of "protecting the resources on the internal
network 
> from being consumed forwarding bogus packets" is really only a
consideration 
> on large corporate or ISP networks with substantial internal
traffic load, 
> and if this minimization of resource use is a concern then
the network 
> administrator can certainly be expected to know how to configure
the 
> company's NAT to enable Restricted filtering.  As I pointed out above,
under 
> my proposal a NAT can use Restricted filtering and still be BEHAVE-compliant

> as long as it is the administrator (not the NAT vendor) that enables this

> firewall behavior.
> 
> 	** For home NATs, in contrast, minimizing resource
load on the internal 
> network due to bogus traffic is completely a non-issue
because the home 
> network (Ethernet or 802.11 or whatever) is typically
so much faster than the 
> Internet uplink anyway.  In this case it's much
more important that NATs 
> consistently support applications well, which
my proposal ensures by 
> requiring that the NAT's "out of the box" configuration
is 
> Endpoint-independent Filtering.  Joe Clueless who buys a home NAT router

> probably doesn't have any idea what NAT actually is, but he wants his

> applications to work.
> 
> ** Conclusion **
> 
> In summary, restrictive
filtering in the NAT interferes with applications 
> substantially and provides
very little real security benefit in 
> practice, considering the way application
communication and NAT port 
> assignment actually works.  As such, especially
considering the purpose of 
> the BEHAVE working group is to increase the
predictability and compatibility 
> of NATs with applications and not to
specify standards for firewall 
> functionality, it seems highly inappropriate
for the WG's documents to 
> recommend firewall functionality that interferes
with applications while 
> providing dubious security benefits.  On the contrary,
I think the WG should 
> at least recommend, if not mandate, that NATs provide
Endpoint-Independent 
> Filtering (Full Cone NAT) by default.  NAT vendors
or system administrators 
> that are particularly paranoid about security
and perceive the security 
> benefits to be worth the interference it causes
with applications can still 
> enable more restrictive filtering policies,
but that's a firewall deployment 
> issue, not a NAT functionality issue.
 We need to steer clear from 
> recommending firewall functionality.
> 

> Thanks very much for your attention.  Please direct followups to the BEHAVE

> working group list, "ietf-behave@list.sipfoundry.org".  Thanks!
> 
>
Bryan
> _______________________________________________
> p2p-hackers mailing
list
> p2p-hackers@zgp.org
> http://zgp.org/mailman/listinfo/p2p-hackers

> _______________________________________________
> Here is a web page listing
P2P Conferences:
> http://www.neurogrid.net/twiki/bin/view/Main/PeerToPeerConferences

> 

From justin at specialbusservice.com  Wed May 18 22:59:34 2005
From: justin at specialbusservice.com (Justin Cormack)
Date: Sat Dec  9 22:12:53 2006
Subject: [p2p-hackers] Official IETF behavior recommendations for NAT
	relevant to P2P
In-Reply-To: <200505181459.21722.baford@mit.edu>
References: <200505181459.21722.baford@mit.edu>
Message-ID: <2D6F41F6-AA3A-438F-9764-0E54FB18E62E@specialbusservice.com>


On 18 May 2005, at 19:59, Bryan Ford wrote:

> Dear p2p-hackers,
>
> A discussion is currently taking place on the IETF BEHAVE working  
> group
> mailing list (see http://www1.ietf.org/html.charters/behave- 
> charter.html)


(snip)

How current is this? These are the people who loooked at STUn and  
decided it wasnt that much use, and are due to wind up next January.

(not on the list but just asking)

Not sure what the issue is with TCP - surely it just doesnt work, you  
cant get the sequence numbers to work.

For UDP, well its easy to spoof anyway, The main problem with not  
using full cone is that it restricts the number of connections that  
you can NAT (and given the lack of timeouts on UDP this does impact  
many applications). Asking for non full cone to be default is thus a  
bit broken. But all the makers of dumb boxes are doing it anyway.  
Since when have NAT vendors cared about the IETF? Isnt it only Linux/ 
BSD based boxes that do full cone anyway?

Given that l33t hackers are already giving owned machines ipv6  
addresses anyway I am not ure it is an issue...

j


From travis at redswoosh.net  Wed May 18 23:05:50 2005
From: travis at redswoosh.net (Travis Kalanick)
Date: Sat Dec  9 22:12:53 2006
Subject: [p2p-hackers] Official IETF behavior recommendations for
	NATrelevant to P2P
In-Reply-To: <2D6F41F6-AA3A-438F-9764-0E54FB18E62E@specialbusservice.com>
Message-ID: <200505182310.j4INADaL001505@be9.noc0.redswoosh.com>

Couldn't have said it better myself. . .

The NAT situation is something we can all build around today, and it will
only get easier to deal with over time by the forces already in play.

Travis


-----Original Message-----
From: p2p-hackers-bounces@zgp.org [mailto:p2p-hackers-bounces@zgp.org] On
Behalf Of Justin Cormack
Sent: Wednesday, May 18, 2005 4:00 PM
To: Peer-to-peer development.
Subject: Re: [p2p-hackers] Official IETF behavior recommendations for
NATrelevant to P2P


On 18 May 2005, at 19:59, Bryan Ford wrote:

> Dear p2p-hackers,
>
> A discussion is currently taking place on the IETF BEHAVE working  
> group
> mailing list (see http://www1.ietf.org/html.charters/behave- 
> charter.html)


(snip)

How current is this? These are the people who loooked at STUn and  
decided it wasnt that much use, and are due to wind up next January.

(not on the list but just asking)

Not sure what the issue is with TCP - surely it just doesnt work, you  
cant get the sequence numbers to work.

For UDP, well its easy to spoof anyway, The main problem with not  
using full cone is that it restricts the number of connections that  
you can NAT (and given the lack of timeouts on UDP this does impact  
many applications). Asking for non full cone to be default is thus a  
bit broken. But all the makers of dumb boxes are doing it anyway.  
Since when have NAT vendors cared about the IETF? Isnt it only Linux/ 
BSD based boxes that do full cone anyway?

Given that l33t hackers are already giving owned machines ipv6  
addresses anyway I am not ure it is an issue...

j

_______________________________________________
p2p-hackers mailing list
p2p-hackers@zgp.org
http://zgp.org/mailman/listinfo/p2p-hackers
_______________________________________________
Here is a web page listing P2P Conferences:
http://www.neurogrid.net/twiki/bin/view/Main/PeerToPeerConferences


From srhea at cs.berkeley.edu  Wed May 18 23:30:31 2005
From: srhea at cs.berkeley.edu (Sean C. Rhea)
Date: Sat Dec  9 22:12:53 2006
Subject: [p2p-hackers] Multiple value per key query/response syntax?
In-Reply-To: <428A3DC5.6090106@vscape.com>
References: <428A3DC5.6090106@vscape.com>
Message-ID: <7f4192482ce8d14c05a8d220c5026b69@cs.berkeley.edu>

On May 17, 2005, at 11:53 AM, Kerry L. Bonin wrote:
> I'm looking for any good discussions or examples of reasonably robust 
> request/response formats for multiple transaction multimap queries in 
> general, and multiple-value per key DHT in particular.
> I was considering using expiration timestamp windowing in the query as 
> a sort of crude iterator filter for values that expire at a constant 
> interval after storage or refresh, but I'm wondering about variable 
> TTL vs. variable refresh intervals, vs. classic remote iterator 
> design.

In OpenDHT, we allow multiple values per key.  If two puts come in with 
the same key, we store them both; a get returns all values.  If all 
values don't fit in a single packet, we return only a packet's worth, 
along with a "placemark" that can be used to retrieve the next set.  
This placemark works just like an iterator.

The way this is implemented is that we return values for gets in the 
order of their SHA-1 hashes.  The placemark returned is just the SHA of 
the last value returned.  To get the next set, we just skip over all 
values whose SHA is less than or equal to the placemark in the get 
handling code.

If a put occurs during iteration, it just falls somewhere in this 
sequence, and may or may not be returned to the client.  If the client 
starts a new get with placemark 0, however, it will be returned.

I hope that made sense,
Sean
-- 
   To announce that there must be no criticism of the president or that 
we
  are to stand by the president right or wrong is not only unpatriotic 
and
         servile but is morally treasonable to the American public.
                            -- Theodore Roosevelt

-------------- next part --------------
A non-text attachment was scrubbed...
Name: PGP.sig
Type: application/pgp-signature
Size: 186 bytes
Desc: This is a digitally signed message part
Url : http://zgp.org/pipermail/p2p-hackers/attachments/20050518/1fff95c8/PGP.pgp
From srhea at cs.berkeley.edu  Wed May 18 23:35:35 2005
From: srhea at cs.berkeley.edu (Sean C. Rhea)
Date: Sat Dec  9 22:12:53 2006
Subject: [p2p-hackers] Has CAN been implemented by any open source project?
In-Reply-To: <20050518094753.CB1863FC2A@capsicum.zgp.org>
References: <20050518094753.CB1863FC2A@capsicum.zgp.org>
Message-ID: <4595e3677475ae9fa1de7d7a1c3979a3@cs.berkeley.edu>

On May 18, 2005, at 2:55 AM, Li Tang wrote:
>    I am looking for some open source resources of DHT algorithms. 
> While I
> have found projects related to Chord, Pastry, and Tapestry, I got 
> little
> implementation of CAN.
>    Is anybody aware of related information? Any pointers, not limited 
> to
> CAN, would be greatly appreciated!

There was never a CAN implementation (outside of simulation) by the CAN 
authors.  Several other groups produced implementations of CAN for the 
purposes of writing papers, but no one ever produced a full 
implementation AFAIK.  (None of them handled node failure, for 
example.)

Sean
-- 
     Once, someone asked me what pleasure I took in riding for so long.
       'Pleasure?' I said. 'I don't understand the question.' I didn't
                   do it for pleasure. I did it for pain.
                             -- Lance Armstrong

-------------- next part --------------
A non-text attachment was scrubbed...
Name: PGP.sig
Type: application/pgp-signature
Size: 186 bytes
Desc: This is a digitally signed message part
Url : http://zgp.org/pipermail/p2p-hackers/attachments/20050518/3439695c/PGP.pgp
From lemonobrien at yahoo.com  Wed May 18 23:46:12 2005
From: lemonobrien at yahoo.com (Lemon Obrien)
Date: Sat Dec  9 22:12:53 2006
Subject: [p2p-hackers] Has CAN been implemented by any open source project?
In-Reply-To: <4595e3677475ae9fa1de7d7a1c3979a3@cs.berkeley.edu>
Message-ID: <20050518234612.76482.qmail@web53601.mail.yahoo.com>

i making a peer to peer system and did research on various systems of search; how best to do it. I may have implemented CAN; can someone decribe what CAN means and how its alogorythm.
 
thanks
lemon

"Sean C. Rhea" <srhea@cs.berkeley.edu> wrote:
On May 18, 2005, at 2:55 AM, Li Tang wrote:
> I am looking for some open source resources of DHT algorithms. 
> While I
> have found projects related to Chord, Pastry, and Tapestry, I got 
> little
> implementation of CAN.
> Is anybody aware of related information? Any pointers, not limited 
> to
> CAN, would be greatly appreciated!

There was never a CAN implementation (outside of simulation) by the CAN 
authors. Several other groups produced implementations of CAN for the 
purposes of writing papers, but no one ever produced a full 
implementation AFAIK. (None of them handled node failure, for 
example.)

Sean
-- 
Once, someone asked me what pleasure I took in riding for so long.
'Pleasure?' I said. 'I don't understand the question.' I didn't
do it for pleasure. I did it for pain.
-- Lance Armstrong

_______________________________________________
p2p-hackers mailing list
p2p-hackers@zgp.org
http://zgp.org/mailman/listinfo/p2p-hackers
_______________________________________________
Here is a web page listing P2P Conferences:
http://www.neurogrid.net/twiki/bin/view/Main/PeerToPeerConferences


You don't get no juice unless you squeeze
Lemon Obrien, the Third.
		
---------------------------------
Do you Yahoo!?
 Make Yahoo! your home page   
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://zgp.org/pipermail/p2p-hackers/attachments/20050518/339e8c57/attachment.html
From ch_407 at rediffmail.com  Thu May 19 00:33:42 2005
From: ch_407 at rediffmail.com (chinmaya narayan pandey)
Date: Sat Dec  9 22:12:53 2006
Subject: [p2p-hackers] Chord
Message-ID: <20050519003342.11538.qmail@webmail26.rediffmail.com>

 ?
hi 
is there any open source implementation of chord.



CHINMAYA
       PANDEY


-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://zgp.org/pipermail/p2p-hackers/attachments/20050519/d692f5db/attachment.htm
From davidopp at gmail.com  Thu May 19 00:54:09 2005
From: davidopp at gmail.com (David Oppenheimer)
Date: Sat Dec  9 22:12:53 2006
Subject: [p2p-hackers] Chord
In-Reply-To: <20050519003342.11538.qmail@webmail26.rediffmail.com>
References: <20050519003342.11538.qmail@webmail26.rediffmail.com>
Message-ID: <e041bec050518175439366db4@mail.gmail.com>

http://pdos.csail.mit.edu/chord/

click on "Downloads"

David

On 19 May 2005 00:33:42 -0000, chinmaya narayan pandey
<ch_407@rediffmail.com> wrote:
> 
> 
>   
> hi 
> is there any open source implementation of chord.
> 
> CHINMAYA
>       PANDEY
> 
> 
> 
> 
>  
> _______________________________________________
> p2p-hackers mailing list
> p2p-hackers@zgp.org
> http://zgp.org/mailman/listinfo/p2p-hackers
> _______________________________________________
> Here is a web page listing P2P Conferences:
> http://www.neurogrid.net/twiki/bin/view/Main/PeerToPeerConferences
> 
> 
>

From toscano_moura at hotmail.com  Thu May 19 01:46:06 2005
From: toscano_moura at hotmail.com (Hermano Toscano Moura)
Date: Sat Dec  9 22:12:53 2006
Subject: [p2p-hackers] Chord
In-Reply-To: <e041bec050518175439366db4@mail.gmail.com>
Message-ID: <BAY12-F155E6BBB31A1E1A36B3BC9F080@phx.gbl>

An HTML attachment was scrubbed...
URL: http://zgp.org/pipermail/p2p-hackers/attachments/20050519/a94649dc/attachment.html
From dbarrett at quinthar.com  Fri May 20 01:05:38 2005
From: dbarrett at quinthar.com (David Barrett)
Date: Sat Dec  9 22:12:53 2006
Subject: [p2p-hackers] Official IETF behavior recommendations for
	NATrelevant to P2P
In-Reply-To: <200505181459.21722.baford@mit.edu>
Message-ID: <20050520010540.D91873FC99@capsicum.zgp.org>

Hi Bryan, I think your work in this area is simply the best, and I entirely
agree with your judgment here.

I'm a P2P developer working in the VoIP space, and I can't stress enough the
annoyance of address-restricted NATs.  Were I working in a "closed" space
(ala Skype) it'd be merely a nuisance.  However, due to the limitations of
open standards (notably SIP and RTP), working in the presence of NATs is
quite challenging.  I would enthusiastically welcome more standardized
behavior in terms of how to "punch holes" and how long I can count on these
holes remaining open.  Let me explain by way of example:


First, as you well know, before anyone can hope to contact me behind a NAT,
I must first figure out what the global (NAT'd) address is for each of my
local ports.  I do this with the help of some central server, such as with
STUN.  Once the hole has been "punched" I have no idea how long it will stay
open, so I need to send a "keep-alive" to my central server.  The net result
is that there is a very high cost to open even one UDP port behind a NAT,
and an ongoing cost to keep it open.  

So even with a full-cone NAT, there is an up-front and ongoing cost to
opening a UDP port on which I intend to receive data.  However, with a
full-cone NAT, the cost is purely spent by my client -- I needn't involve
anyone else (excepting the central server) in preparing to receive data than
my own instance of my own application.

In the case of an address-restricted NAT, however, not only must I pay the
cost of being behind a NAT (and pay not once, but again for each client who
communicates with me), everyone else who wishes to communicate with me must
pay as well.  

For the full-cone NAT, once I know my global (NAT'd) IP/port, I can use that
as my own -- anybody can contact me via that address, without special
behavior to account for me being behind a NAT.  This means NAT-ignorant
protocols (such as RTP, for example) can be received from behind NATs,
without any special programming on the part of the sender.

However, for address-restricted NATs, this is not the case.  Imagine you
want to send me an RTP stream, and I am behind a NAT.  With a full-cone NAT,
I could send you my global IP/port (via SIP and SDP), and you could open an
RTP stream direct back to me.  But with an address-restricted NAT, this is
not enough.  The reason is a bit involved, but please bear with me.


First you need to send me the SIP INVITE via my proxy server.  SIP allows
the proxy server to either redirect you to contact me directly, or forward
on my request.  If I'm behind a full-cone NAT, either works fine.  If I'm
behind an address-restricted NAT, I can only hope that the proxy forwards
your request on to me, because I have no way of punching a hole to receive
your direct SIP request.  Furthermore, I have no way of indicating to the
SIP proxy that I need it to forward to me.  So right off the bat, receiving
SIP INVITE requests behind an address-restricted NAT is more complicated
than from behind a full-cone NAT.

Assuming I'm lucky and the SIP proxy forwards the request to me (via the
hole I've already punched for it), I can receive and respond to the INVITE
request indicating the port to which you should broadcast your RTP.
However, I have a complication that shared by all NATs, and that is getting
a port that complies with RTP.  The RTP standard indicates RTP ports should
be even-numbered, and the next port (odd-numbered) is implied to be open to
receive RTCP.  Unfortunately, I as a client have absolutely no control over
which ports my NAT assigns me, so it's very difficult for me to receive RTP
in a fully compliant fashion.  So for all NATs I must put in extra effort to
communicate the real RTP/RTCP ports I wish to receive on, possibly by
bending the RTP spec.  But a further complication of address-restricted NATs
is that I also need to somehow figure out the exact address you intend to
broadcast from, even though I would otherwise not care.  This means I must
bend the SIP/RTP framework even more (or dig into the details more than I
might otherwise) just to support address-restricted NATs.

In the meantime, while I am playing these extra games to punch holes to
receive your data, you have already begun to broadcast.  For full-cone NATs,
this is fine and expected.  But for address-restricted NATs, I can't receive
data despite having told you the correct IP/port to which you should
broadcast, because I haven't punched a hole for you.  While I'm playing
games trying to punch the hole, you're broadcasting data that I'm not
receiving.  Depending on the codec, that might be very important header
data.  Naturally UDP is an unreliable transport mechanism and thus there are
fallbacks if the occasional packet is lost, but there's a big difference
between "some packets might be lost" and "an unknown number of initial
packets *will* be lost".  More work must be put into supporting the
address-restricted case, and more deviation from the "mainstream" SIP/RTP
best practices.

Furthermore, just punching the holes themselves involve violating standards,
hopefully in benign fashion.  In the case of unidirectional RTP, there is
nowhere in the standard that I as a listener should send to your RTP
broadcast port -- thus there is no fully-compliant way to punch a hole to
receive your data.  (Yes I can send to your RTCP port, but that's a separate
port.)  The best I can do is send a malformed packet your way and hope it
doesn't cause you harm.  With full-cone NATs, I have no such problem.

And finally, every hole I punch requires some maintenance.  The less holes,
the better.  Generally, with full-cone, the number of holes I need punch is
proportional to the number of services I wish to offer (and the same holes
are used for all users).  But with address-restricted, it's proportional to
the product of the number of services, and the number of simultaneous users.
This means I have a much higher overhead to working with address-restricted
NATs.  But more importantly, I have so many more things that can go wrong.
This breeds the absolute worst kind of "partial failure" bug such as:

- Why can I hear Bob, but not Alice?
- Why can I hear Bob, but he can't hear me?
- Why can I receive Bob's RTP, but not Bob's RTCP?
- And so on...

And if I walk across the room and pick a different wi-fi hub, thus changing
IP addresses, with active sessions?  Oye, it's a ton of work to re-establish
all of these crazy holes and sessions in a bidirectional manner.  A ton of
work that I needn't do with full-cone NATs.


So, to make a long story short, NAT traversal is a hard problem, and it's
made especially hard by address-restricted NATs.  If I could count on
full-cone NAT behavior, my life as a programmer would be easier, and the
applications I produce would function better, faster, and more reliably.

In other words, I fully endorse Bryan's recommendation, and hope to see it
included in the final draft.

-david


> -----Original Message-----
> From: p2p-hackers-bounces@zgp.org [mailto:p2p-hackers-bounces@zgp.org] On
> Behalf Of Bryan Ford
> Sent: Wednesday, May 18, 2005 11:59 AM
> To: p2p-hackers@zgp.org
> Subject: [p2p-hackers] Official IETF behavior recommendations for
> NATrelevant to P2P
> 
> Dear p2p-hackers,
> 
> A discussion is currently taking place on the IETF BEHAVE working group
> mailing list (see http://www1.ietf.org/html.charters/behave-charter.html)
> that is highly relevant to P2P application writers.  For those of you not
> already familiar with it, this IETF working group is defining standards
> for
> how NATs should behave so as to better support applications including P2P
> apps.  The chair has made a "Last Call" for comments on the current draft
> document specifying NAT behavior for UDP, so this may be the last chance
> we
> have for a while to influence how well the next generation of NATs support
> UDP-based P2P applications.
> 
> ** Address-Restricted vs Full Cone NAT **
> 
> There's one issue in particular on which I personally fear the working
> group
> is currently headed in the wrong direction, and over which you folks on
> this
> list could perhaps exert considerable influence if you're willing to take
> a
> little time and make your voice heard on the BEHAVE list.  The current UDP
> draft, draft-ietf-behave-nat-udp-01.txt, recommends that NATs implement
> "Endpoint address-dependent filtering behavior", otherwise known as
> "Address-Restricted Cone NAT".  In other words, once a NATted UDP
> application
> opens a UDP session through a NAT to some external host, giving the
> application a public (NATted) port number, the NAT only accepts and
> forwards
> subsequent incoming UDP packets at that public port if they come from a
> source IP address that the application has previously sent UDP packets to.
> In other words, the NAT must see outgoing UDP traffic to a remote host
> before
> it will accept incoming traffic from that host on the application's public
> port.  In the proposed alternative, "Endpoint-independent filtering" or
> "Full
> Cone NAT", the NAT forwards any incoming traffic to the correct public
> port
> regardless of source IP address or port number.
> 
> Address-Restricted Cone NAT is moderately "P2P-friendly" in that it at
> least
> allows NATted applications to "hole punch" P2P connections with the help
> of a
> well-known (non-NATted) rendezvous server.  Address-Restricted Cone NAT
> still
> does not, however, allow the application to have a full "first-class"
> Internet presence (i.e., public IP address and UDP port number) at which
> it
> can be contacted proactively by any other Internet-connected host.  A P2P
> application behind an Address-Restricted Cone NAT is still forever
> dependent
> on other public (first-class) rendezvous servers to help establish each
> P2P
> communication session it needs.  A P2P app behind a Full Cone NAT, in
> contrast, is a full "first-class" network citizen itself, and can even
> _be_ a
> public rendezvous server for other less fortunate hosts once it learns its
> own public IP address and UDP port number.  This is an important benefit
> to
> applications, especially P2P applications that are trying to be highly
> decentralized.
> 
> The main argument for Restricted Cone NAT, of course, is security.  Until
> recently I believed this argument myself, but after further and more
> careful
> analysis I have come to the conclusion that Restricted filtering provides
> a
> very week security benefit in comparison with the fundamental loss in
> first-class application connectivity it entails.  My current proposal on
> the
> BEHAVE list does NOT mandate Full Cone NAT, but it does specify that NATs
> should at least be Full Cone in their default, "out of the box"
> configuration
> as provided by NAT vendors, and recommends that NATs implement Restricted
> Cone NAT as an administrator-configurable security option.  The current
> proposed text reads as follows:
> 
>    REQ-7: NATs MUST implement Endpoint-Independent
>           Filtering in their default configuration.
>           NATs SHOULD be configurable to support
>           Address-Specific Filtering or Endpoint-Specific Filtering
>           at the option of the system administrator.
> 
> If you have thoughts or opinions on this topic, I would greatly appreciate
> hearing them (and having other participants hear them) on the BEHAVE list.
> 
> ** Why the Security Benefit of Restricted Cone NAT is Minimal **
> 
> Restrictive filtering ostensibly has the benefit of protecting
> applications on
> an internal host from traffic the internal host is not expecting, i.e.,
> from
> traffic from previously unknown external hosts or endpoints.  This
> behavior
> can only possibly benefit the application if the application only ever
> initiates outgoing sessions from its internal port, and is obviously a
> curse
> if the application ever wants to accept incoming sessions from external
> hosts.  The NAT has no way of knowing which is the case.  But suppose the
> application is one that only initiates outgoing connections (i.e., a
> "traditional client/server app"), and thus might "potentially" benefit
> from a
> NAT's restrictive filtering behavior.  Consider TCP and UDP separately:
> 
>         ** TCP: When a client/server app makes an outgoing TCP connection,
> it
> typically does a "connect()" call to the host OS's sockets API without
> bothering to bind() the socket, and the OS's TCP stack allocates a local
> TCP
> port exclusively for the use of that connection.  Any TCP packets the host
> OS
> subsequently received on that port from endpoints other than the one it
> connect()ed to will be summarily rejected with TCP RSTs by the host OS,
> and
> the application never even sees any hint that they ever arrived.  Even if
> the
> application does a bind() before its connect() in order to use a specific
> source port (e.g., a <1024 "privileged" port), it still almost always
> obtains
> exclusive use of that port.  An application indeed has to work really
> bloody
> hard to set things up so that it can accept incoming TCP connections on
> the
> same port it is using for an outgoing connection, using SO_REUSEADDR and
> multiple sockets.  No traditional client/server application is going to go
> to
> all this effort just in order to create a security hole. :)  In other
> words,
> for TCP, restrictive filtering in practice provides absolutely zero
> protection from the _application_: the only entity that the restrictive
> filtering even affects is the host OS's protocol stack.  Restrictive
> filtering might for example benefit a host OS with a TCP stack that is so
> broken that it fails to check the source IP address and port number on
> incoming TCP packets at all, but if the host OS is _that_ broken then it
> undoubtedly has many other problems that the NAT can't possibly guard
> against.
> 
>         ** UDP: When a client/server UDP app uses a UDP port for
> communication
> with just one remote host, for convenience it typically does a connect()
> to
> the appropriate remote UDP endpoint, so that it can just send messages
> with
> send() or write() without having to specify a destination address for each
> datagram, and so that it can receive messages with read() or recv()
> without
> having to check the source address and port number in each message
> received.
> In this case, the host OS effectively provides the external endpoint
> filtering functionality as for TCP, and any restrictive filtering
> implemented
> by the NAT again provides no security benefit at all to the application.
> A
> UDP application that uses a single UDP port for communication with
> multiple
> remote hosts, in contrast, must generally check the source IP addresses
> and
> port numbers in the datagrams it receives just in order to distinguish
> between the multiple communication sessions it is involved in, and hence
> it
> will inherently provide its own external endpoint filtering.  It's
> conceivable that some abysmally buggy UDP applications might benefit from
> the
> NAT's external filtering - e.g., a UDP application that only talks to one
> external endpoint but doesn't use connect() and doesn't check the source
> endpoint on incoming datagrams - but an application that is so seriously
> broken probably has many other security vulnerabilities that can be
> exploited
> despite the NAT's external filtering behavior, e.g., by finding out or
> guessing the public server the application is talking to (which is easy in
> many cases such as commercial games or services where there are only a few
> well-known servers) and sending packets with spoofed source IP address and
> UDP port numbers.  Besides, many, perhaps the majority, of UDP
> applications
> use UDP _because_ they need P2P-style communication, and so external
> filtering by the NAT provides no practical security benefit and only
> causes
> interference with the application.
> 
> Another common argument for Restricted Cone NAT (Address-dependent
> filtering)
> is that it protects the resources on the internal network from being
> consumed
> forwarding bogus packets even if the eventual end application will
> correctly
> discard them.  This argument is also weak, however:
> 
> 	** Even with more permissive Full Cone NAT (endpoint-independent
> filtering),
> the NAT will still in practice filter out the vast majority of bogus
> traffic
> that reaches it, because most of the NAT's ports are typically not in
> active
> use by applications behind the NAT.  The only bogus traffic a Full Cone
> NAT
> will let through is bogus traffic that happens to hit the _specific_
> external
> port numbers that correspond to active port bindings.  These port numbers
> are
> assigned more-or-less randomly (i.e., unpredictably) by the NAT, and they
> only
> ever appear in the high, non-well-known port number space, so most of the
> Internet "background noise" caused by viruses or other attacks on well-
> known
> ports will never get through in any case, and purely random "port
> guessing"
> attacks would have to send packets to a great many NAT ports in order to
> get
> a few packets through.  Thus, the incremental benefit of restrictive
> filtering over Full Cone NAT with regards to general bogus Internet
> traffic
> is very small.
> 
> 	** An attacker might deliberately send bogus packets to a bound NAT
> port
> known to be in use.  But if the attacker can find out which NAT port the
> application is using, then the attacker probably also knows (or can easily
> find out or guess) the address and port number of the _remote_ host that
> the
> application is talking to and just spoof that source IP address and port
> number in its attack packets, getting around Restricted filtering as well.
> In fact the remote host's endpoint is in practice likely to be much easier
> for the attacker to find out or guess than the application's public NAT
> port,
> because of the overwhelming prevalence of well-known servers and ports on
> the
> Internet in comparison to the randomness and relative obscurity of
> temporary
> NAT-assigned public ports.  So the practical benefit that Restricted
> filtering provides even against directed attacks is still minimal.
> 
> 	** Finally, the goal of "protecting the resources on the internal
> network
> from being consumed forwarding bogus packets" is really only a
> consideration
> on large corporate or ISP networks with substantial internal traffic load,
> and if this minimization of resource use is a concern then the network
> administrator can certainly be expected to know how to configure the
> company's NAT to enable Restricted filtering.  As I pointed out above,
> under
> my proposal a NAT can use Restricted filtering and still be BEHAVE-
> compliant
> as long as it is the administrator (not the NAT vendor) that enables this
> firewall behavior.
> 
> 	** For home NATs, in contrast, minimizing resource load on the
> internal
> network due to bogus traffic is completely a non-issue because the home
> network (Ethernet or 802.11 or whatever) is typically so much faster than
> the
> Internet uplink anyway.  In this case it's much more important that NATs
> consistently support applications well, which my proposal ensures by
> requiring that the NAT's "out of the box" configuration is
> Endpoint-independent Filtering.  Joe Clueless who buys a home NAT router
> probably doesn't have any idea what NAT actually is, but he wants his
> applications to work.
> 
> ** Conclusion **
> 
> In summary, restrictive filtering in the NAT interferes with applications
> substantially and provides very little real security benefit in
> practice, considering the way application communication and NAT port
> assignment actually works.  As such, especially considering the purpose of
> the BEHAVE working group is to increase the predictability and
> compatibility
> of NATs with applications and not to specify standards for firewall
> functionality, it seems highly inappropriate for the WG's documents to
> recommend firewall functionality that interferes with applications while
> providing dubious security benefits.  On the contrary, I think the WG
> should
> at least recommend, if not mandate, that NATs provide Endpoint-Independent
> Filtering (Full Cone NAT) by default.  NAT vendors or system
> administrators
> that are particularly paranoid about security and perceive the security
> benefits to be worth the interference it causes with applications can
> still
> enable more restrictive filtering policies, but that's a firewall
> deployment
> issue, not a NAT functionality issue.  We need to steer clear from
> recommending firewall functionality.
> 
> Thanks very much for your attention.  Please direct followups to the
> BEHAVE
> working group list, "ietf-behave@list.sipfoundry.org".  Thanks!
> 
> Bryan
> _______________________________________________
> p2p-hackers mailing list
> p2p-hackers@zgp.org
> http://zgp.org/mailman/listinfo/p2p-hackers
> _______________________________________________
> Here is a web page listing P2P Conferences:
> http://www.neurogrid.net/twiki/bin/view/Main/PeerToPeerConferences


From wgallagher at gridironsoftware.com  Fri May 20 01:33:18 2005
From: wgallagher at gridironsoftware.com (Warren Gallagher)
Date: Sat Dec  9 22:12:53 2006
Subject: [p2p-hackers] Official IETF behavior recommendations for
	NATrelevant to P2P
In-Reply-To: <200505181459.21722.baford@mit.edu>
Message-ID: <200505200133.j4K1XBD8011939@mail4.magma.ca>

Bryan,

I agree with your assessment. I would further argue that NATs are not
security devices, they are address translators. Security is the job of the
firewall. As long as firewalls do not restrict a sysadmin's ability to
control the behaviour then everything is good.

Thanks for the good work
Warren Gallagher
Chief Software Architect, Gridiron Software

-----Original Message-----
From: p2p-hackers-bounces@zgp.org [mailto:p2p-hackers-bounces@zgp.org] On
Behalf Of Bryan Ford
Sent: May 18, 2005 2:59 PM
To: p2p-hackers@zgp.org
Subject: [p2p-hackers] Official IETF behavior recommendations for
NATrelevant to P2P

Dear p2p-hackers,

A discussion is currently taking place on the IETF BEHAVE working group 
mailing list (see http://www1.ietf.org/html.charters/behave-charter.html) 
that is highly relevant to P2P application writers.  For those of you not 
already familiar with it, this IETF working group is defining standards for 
how NATs should behave so as to better support applications including P2P 
apps.  The chair has made a "Last Call" for comments on the current draft 
document specifying NAT behavior for UDP, so this may be the last chance we 
have for a while to influence how well the next generation of NATs support 
UDP-based P2P applications.

** Address-Restricted vs Full Cone NAT **

There's one issue in particular on which I personally fear the working group

is currently headed in the wrong direction, and over which you folks on this

list could perhaps exert considerable influence if you're willing to take a 
little time and make your voice heard on the BEHAVE list.  The current UDP 
draft, draft-ietf-behave-nat-udp-01.txt, recommends that NATs implement 
"Endpoint address-dependent filtering behavior", otherwise known as 
"Address-Restricted Cone NAT".  In other words, once a NATted UDP
application 
opens a UDP session through a NAT to some external host, giving the 
application a public (NATted) port number, the NAT only accepts and forwards

subsequent incoming UDP packets at that public port if they come from a 
source IP address that the application has previously sent UDP packets to.  
In other words, the NAT must see outgoing UDP traffic to a remote host
before 
it will accept incoming traffic from that host on the application's public 
port.  In the proposed alternative, "Endpoint-independent filtering" or
"Full 
Cone NAT", the NAT forwards any incoming traffic to the correct public port 
regardless of source IP address or port number.

Address-Restricted Cone NAT is moderately "P2P-friendly" in that it at least

allows NATted applications to "hole punch" P2P connections with the help of
a 
well-known (non-NATted) rendezvous server.  Address-Restricted Cone NAT
still 
does not, however, allow the application to have a full "first-class" 
Internet presence (i.e., public IP address and UDP port number) at which it 
can be contacted proactively by any other Internet-connected host.  A P2P 
application behind an Address-Restricted Cone NAT is still forever dependent

on other public (first-class) rendezvous servers to help establish each P2P 
communication session it needs.  A P2P app behind a Full Cone NAT, in 
contrast, is a full "first-class" network citizen itself, and can even _be_
a 
public rendezvous server for other less fortunate hosts once it learns its 
own public IP address and UDP port number.  This is an important benefit to 
applications, especially P2P applications that are trying to be highly 
decentralized.

The main argument for Restricted Cone NAT, of course, is security.  Until 
recently I believed this argument myself, but after further and more careful

analysis I have come to the conclusion that Restricted filtering provides a 
very week security benefit in comparison with the fundamental loss in 
first-class application connectivity it entails.  My current proposal on the

BEHAVE list does NOT mandate Full Cone NAT, but it does specify that NATs 
should at least be Full Cone in their default, "out of the box"
configuration 
as provided by NAT vendors, and recommends that NATs implement Restricted 
Cone NAT as an administrator-configurable security option.  The current 
proposed text reads as follows:

   REQ-7: NATs MUST implement Endpoint-Independent
          Filtering in their default configuration.
          NATs SHOULD be configurable to support
          Address-Specific Filtering or Endpoint-Specific Filtering
          at the option of the system administrator.

If you have thoughts or opinions on this topic, I would greatly appreciate 
hearing them (and having other participants hear them) on the BEHAVE list.

** Why the Security Benefit of Restricted Cone NAT is Minimal **

Restrictive filtering ostensibly has the benefit of protecting applications
on 
an internal host from traffic the internal host is not expecting, i.e., from

traffic from previously unknown external hosts or endpoints.  This behavior 
can only possibly benefit the application if the application only ever 
initiates outgoing sessions from its internal port, and is obviously a curse

if the application ever wants to accept incoming sessions from external 
hosts.  The NAT has no way of knowing which is the case.  But suppose the 
application is one that only initiates outgoing connections (i.e., a 
"traditional client/server app