Low Latency Messages on Distributed Memory Multiprocessors

This article describes many of the issues in developing an efficient interface for communication on distributed memory machines. Although the hardware component of message latency is less than 1 μs on many distributed memory machines, the software latency associated with sending and receiving typed messages is on the order of 50 is. The reason for this imbalance is that the software interface does not match the hardware. By changing the interface to match the hardware more closely, applications with fine grained communication can be put on these machines. This article describes several tests performed and many of the issues involved in supporting low latency messages on distributed memory machines.