[p2p-hackers] Optimal UDP Message size for data streaming...

[Daniel Stutzbach]

On Sat, Feb 25, 2006 at 05:08:39PM -0800, David Barrett wrote:
> Has anyone on this list gotten MTU discovery over UDP to work in the real
> world?  And if so, is there truly such variance that it's worth it?  If the
> range is 1472-1500 bytes, it seems like a pointless exercise.  If it's
> 1472-65536 bytes, then it's more interesting.
 
Most connections will have a path-MTU of 1500 bytes.  Always using
larger datagrams is definitely a Bad Idea.  You might find larger MTUs
in parts of the Internet backbone core or on finely-tuned
gigabit-ethernet/FDDI/ATM networks, but that's about it.  Regular
users have ethernet (wired or wireless) in their path, which means
their MTU is at most 1500 bytes.

A significant minority of users will have a path-MTU slightly less
than 1500 bytes.  Basically, these users have a real MTU of 1500, but
they're doing some kind of tunneling (PPP, PPPoE, VPN, etc.) which
adds a few headers and eats into the MTU available to your
application.  PPPoE is an important one because some broadband access
providers use it to deliver service to their users (I'm not sure what
percentage, though).

For a very tiny minority of users, the MTU will be even lower because
they're using some funky access technology or doing something else
strange.

If you send datagrams that are slightly too big, they'll be fragmented
and the performance hit is pretty bad.  

If you send datagrams that are slightly too small, you lose a little
efficiency because a larger fraction of your throughput is being eaten
up by packet headers.

(As an aside: remember that for a datagram of x bytes, you only get
x-28 bytes of payload since a 20-byte IP header and an 8 byte UDP
header need to fit in there, too)

> Basically, given that you've got to make your application work well on
> small-MTU paths, is it worth the effort to find and use big MTUs or just
> more practical to find the lowest common denominator?

Well, the problem is that "the lowest common denominator" is not well
defined.  There may be one guy somewhere who is still using a SLIP
link with a 256-byte MTU.  If you use that as your baseline, all your
other users are going to take a performance hit.

On the other hand, if you use a fixed MTU of around 1400--1500, anyone
with a lower MTU is going to have especially bad performance.  Anyone
with a higher MTU is going to have only slightly worse performance
(compared to doing Path-MTU discovery).  Unfortunately, you always
risk somebody having a slightly lower MTU than the line you draw in
the sand.

I suggest just picking a reasonable value that works for you and your
users.  You can always add Path-MTU discovery latter if you want to
squeeze out a small performance improvement and/or cater to the small
fraction of users with a lower MTU and/or if the networks of the
future have a more diverse set of commonly-used MTUs than today's
Internet.

-- 
Daniel Stutzbach                           Computer Science Ph.D Student
http://www.barsoom.org/~agthorr                     University of Oregon