It's the Latency, Stupid

Fact One: Making more bandwidth is easy.

• People with ISDN lines can already do this. It’s called “bonding” and it uses two 56 (or 64) kbit/sec ISDN channels in parallel to give you a combined throughput of 112 (or 128) kbit/sec.

Fact Two: Once you have bad latency you’re stuck with it.

• What most end-users don’t know is that in the process of transferring those big files their computers have to send back and forth hundreds of little control messages, so the performance of small data packets directly affects the performance of everything else they do on the network.
• Once you’ve got yourself a device with bad latency there’s absolutely nothing you can do about it (except through out the device and get something else).

Fact Three: Current consumer devices have appallingly bad latency.

• Here’s a real-world example:
  • The distance from Stanford to Boston is 4320km.
  • The speed of light in vacuum is 300 x 10^16 m/s.
  • The speed of light in fibre is roughly 66% of the speed of light in vacuum.
  • The speed of light in fibre is 300 x 10^6 m/s * 0.66 = 200 x 10^6 m/s.
  • The one-way delay to Boston is 4320 km/ 20 x 10^6 m/s = 21.6ms.
  • The round-trip time to Boston and back is 43.2ms.
  • The current ping time from Stanford to Boston over today’s Internet is about 85ms.
  • So: the hardware of the Internet can currently achieve within a factor of two of the speed of light.
• We’re already within a factor of two of the theoretical optimum. I think that’s pretty good. Not many technologies can make that claim.
• The latency over your modem [to your ISP, which would be about 0.1ms if it’s 18km away] is actually over 100ms. Modems are currently operating at a level that’s 1000 times worse than the speed of light. They have a long way to go before they get close to what the rest of the Internet is achieving.
• Of course no modem link is every going to have a latency of 0.1ms. I’m not expecting that.
• The important issue is the total end-to-end transmission delay for a packet [which] is made up of fixed latency (including the speed-of-light propagation delay), plus the transmission time.
• For a 36 byte packet the transmission time is 10ms (the time it takes to send 288 bits at a rate of 28800 bits per second). When the actual transmission time is about 10ms, working to make the latency 0.1ms would be silly. All that’s needed is that the latency should not be so huge that it completely overshadows the transmission time.

Fact Four: Making limited bandwidth go further is easy.

Compression

• compression techniques trade off use of CPU power in exchange for lower bandwidth requirements. There’s no equivalent way to trade off use of extra CPU power to make up for poor latency.

Bandwidth-conscious code

• Another way to cope with limited bandwidth is to write programs that take care not to waste bandwidth.

Send less data

• If you don’t have enough bandwidth to send high resolution pictures, you can use lower resolution.

Caching

• One of the most effective techniques throughout all areas of computer science is caching, and that is just as true in networking.
• Checking the date and time a file was last modified is a tiny request to send across the network. This kind of request is so small that the throughput of your modem makes no difference–latency is all that matters.
• Since for the most part the Web browser is only doing small modification date queries to the Web server, the performance the user experiences is entirely dominated by the latency of the connection, and the throughput is virtually irrelevant.

Another analogy

• Part of the problem […] is misleading use of the word “faster”.
• Would you say that a Boeing 747 is three times “faster” than a Boeing 737? Of course not. They both cruise at around 500 miles per hour. The difference is that the 747 carries 500 passengers where as the 737 only carries 150. The Boeing 747 is three times bigger than the Boeing 737, not faster.
• if you were really in a hurry to get to London quickly, you’d take Concorde, which cruises around 1350 miles per hour. It only seats 100 passengers though, so it’s actually the smallest of the three. Size and speed are not the same thing.
• On the other hand, if you had to transport 1500 people and you only had one aeroplane to do it, the 747 could do it in three trips where the 737 would take ten
• That’s the problem with communications devices today. Manufacturers say “speed” when they mean “capacity”.
• The phrase “high-capacity slow link” that I used above probably looked very odd to you. Even to me it looks odd. We’ve been used to wrong thinking for so long that correct thinking looks odd now.
• If someone talks about a “high-capacity” oil tanker, do you immediately assume it’s a very fast ship? I doubt it. If someone talks about a “large-capacity” truck, do you immediately assume it’s faster than a small sports car?

Lessons to learn

Bandwidth Still Matters

• Many people believe that having a private 64kb/sec ISDN connection is just as good, or even better than having a 1/150 share of a 10MB/sec Ethernet.
• This idea, that you can average packets as if they were a fluid in a pipe, is flawed, as the following example will show:
  • Say we have a game where the stage of the virtual world amounts to 40K of data. We have a game server, and in this simple example, the game server transmits the entire current game state to the player once every 10 seconds. That’s 40K every 10 seconds, or an average of 4K/sec, or 32kb/sec. That’s only half the capacity of a 64kb/sec ISDN line, and 150 users doing this on an Ethernet is only half the capacity of the Ethernet. So far so good. Both links are running at only 50% capacity, so the performance should be the same, right? Wrong.
  • On the Ethernet, when the server sends the 40K to the player, the player can receive that data as little as 32ms later (320kb / 10Mb/sec). If the server is not the only machine sending packets on the Ethernet, then there could be contention for the shared medium, but even in that case the average delay before the player receives the data is only 64ms. On the ISDN line, when the server sends the 40K to a player, the player receives that data 5 seconds later (320kb / 64kb/sec). In both cases the users have the same average bandwidth, but the actual performance is very different.
  • The standard mistake is to assume that a 40K chunk every ten seconds and a uniform rate of 4K/second are the same thing. They’re not.
• The telephone companies assume that all communications are like the flow of fluid in a pipe. You just tell them the rate of flow you need, and they tell you how big the pipe has to be.
• Just because you don’t send data very often doesn’t mean you want it delivered late. You may only write to your aunt once a year, but that doesn’t mean that on the occasions when you do write her a letter you’d like it to take a year to be delivered.

Survey

Are we there yet?

• One major modem manufacturer has contacted me, and we’ve been investigating where the time goes. It seems that there is room for improvement, but unfortunately modems will never be able to match ISDN. The problem is that over a telephone line, electrical signals get “blurred” out. In order to decode just one single bit, a 33.6kb/s modem needs to take not just a single reading of the voltage on the phone line at that instant, but that single reading plus another 79 like it, spaced 1/6000 of a second apart. A mathematical function of those 80 readings give the actual results. This process is called “line equalization”. Better line equalization allows higher data rates, but the more “taps” the equalizer has the more delay it adds. The V.34 standard also specifies particular scrambling and descrambling of the data, which also take time. According to this company, the theoretical best round-trip delay for a 14.4kb/s modem (no compression or error recovery) should be 40ms, and for a 33.6kb/s modem 64ms. The irony here is that as the capacity goes up, the best-case latency gets worse instead of better. For a small packet, it would be faster for your modem to send it at 9.6kb/s than at 33.6kb/s!