Latency
Latency is, in HPC and networking terms, the period of time a packet spends traveling from its source to its destination. That time encompasses the time necessary to encode, send the packet, receive a packet, and decode it to be made available to the higher level software stack. Together, latency and bandwidth define the speed and capacity of a network. However, latency can be much more important for most parallel HPC applications than you might think. In fact, latency is often much more critical than bandwidth.
Latency is relative. In human terms, anything that takes place faster than about 200 milliseconds <2/10ths of a second) is not extremely noticeable. However, in HPC, that same 200 millisecond latency would be totally unacceptable, and would result in extremely slow code execution and unacceptably long job times. Milliseconds are abbreviated as (ms).
Most network technology latencies are measured in microseconds (one millionth of a second, abbreviated as us). Now this might seem quite fast, but the number of operations, and packets, per second that the average computer performs, much less a tuned HPC cluster and application, is quite high, and even this very small amount of time spent waiting for one transaction to occur prior to another transaction being processed can significantly impact a distributed or parallel HPC application, causing major delays in code execution.
Common latencies for different network technologies are shown below
- Gigabit Ethernet: ~40 to ~300us
- Infiniband: ~1us to ~5us
- Myrinet: ~2.3us
As can be seen from the numbers above, a low latency interconnect such as Infiniband or Myrinet can be 40 to 100 times faster than standard Gigabit Ethernet. 10 Gigabit Ethernet normally has about the same latencies as Gigabit Ethernet, but low latency Gigabit and 10 Gigabit Ethernet switches do exist which rival the latencies achieved by other technologies. It is unclear at this point that port costs on these lower latency switches will be low enough to provide a suitable alternative or replacement for Myrinet or Infiniband in HPC clusters.
As processor, memory, and even disk speed increases, along with the ever increasing range of parallel applications which take advantage of more and cheaper cores, latency will only become more critical as a factor in your HPC code speed of execution.
