Messaging on gigabit ethernet: some experiments with GAMMA and other systems

Abstract

The Genoa Active Message MAchine (GAMMA) is a lightweight communication system based on the Active Ports paradigm, originally designed for efficient implementation over low-cost Fast Ethernet interconnects. In this paper we report about the recently completed porting of GAMMA to the Packet Engines GNIC-II and the Netgear GA620 Gigabit Ethernet adapters, and provide a comparison among GAMMA, MPI/GAMMA, TCP/IP, and MPICH, on such commodity interconnects, using different performance metrics. With a combination of low end-to-end latency (9.5 μs with GNIC-II, 32 μs with GA620) and high transmission throughput (almost 97 MByte/s with GNIC-II and 125 MByte/s with GA620, the latter obtained without changing the firmware of the adapter), GAMMA demonstrates the potential for Gigabit Ethernet lightweight protocols to yield messaging performance comparable to the best Myrinet-based messaging systems. This result is of interest, given the envisaged drop in cost of Gigabit Ethernet due to the transition from fiber optic to UTP cabling and ever increasing mass market production of such standard interconnect. We also reports about a technique for message fragmentation that is commonly exploited to increase the throughput with short message. When a different, though more widely used, performance metrics is considered, such a technique results into a performance loss rather than improvement.