[p2p-hackers] avoiding uplink saturation / co-existing with other
coderman at gmail.com
Mon May 8 09:35:39 UTC 2006
On 5/7/06, Adam Back <adam at cypherspace.org> wrote:
> Is this something one should be worrying about, or will tcp "take care
> of it" gracefully?
unless the application performs throttling in userspace (which is
often poor; tricks like send/recv buffer sizing, nagle, etc can be
used to improve accuracy) the TCP stack will try and optimize across
all tcp sessions fairly. when many concurrent recv streams are active
this implies not only latency and low bandwidth per stream but also
likely invokes queuing at the DSL/cable ISP which can be as high as 2
seconds round trip. [this is a highly simplified example but common]
overall your link is being well used, but per application / tcp
session throughput is sluggish and constrained.
> It seemed to me that at least for a while bittorrent suffered from
> this problem.
losing your ssh / vpn session in a flood of bittorrent neighbors is
definitely painful without shaping. :)
> Anyone have a tried and tested algorithm/method to address this?
traffic shaping in kernel / upstream before the ISP works best in my
experience but does require a decent kernel (linux/bsd/*nix/cisco/*)
for shaping at your host/router.
if you need to rely on throttling at the application layer the use of
small send buffers and disabled nagle gives you better
consistency/accuracy when writing data at a regular interval on the
there are variations on this theme like LD_PRELOAD override of socket
api and DiffServ/DSCP tagging which have their own particular
> Generally what you'd like I think is QoS and to give the p2p app
> "bulk" precedence for other peoples traffic.
i suppose this is an open ended question (the ideal distribution of
bandwidth among services on a link) but the following is my favorite
- shape egress traffic such that buffering latency is avoided upstream to ISP
- prioritize interactive traffic (ssh, not scp; vpn; rtp; chat; etc)
- group bulk traffic into classes (p2p, backup, ftp, torrent, ?)
utilizing spare capacity above an optional minimum throughput
- leave large packets and other unclassified payload in a middle /
i prefer iptables classifiers with the MARK target which is used by tc
to configure the shaped queues and thresholds associated with each
regarding user interaction, i'd love to see a learning mode applied to
common/conservative traffic classification and shaping so that real
world link capability as monitored over time was used to set more
relevant thresholds / classification. this has the same limitations
as explicit shaping but would be much more usable and let p2p /
network application clients focus on features instead of poor traffic
you can find specific examples of "traffic shaping" for p2p in the
archives and via google. citeseer has a huge number of papers on
these subjects as well.
More information about the P2p-hackers