Design and implementation of efficient communication abstractions on the Virtual Interface Architecture: Stream sockets and RPC experience

Abstract

AbstractThe emergence and standardization of system area networks (SANs) has provided distributed applications with a medium for high‐bandwidth, low‐latency communication. Standard user‐level networking architecture such as the Virtual Interface (VI) Architecture enables distributed applications to perform low overhead communication over SANs. The VI Architecture significantly reduces system processing overheads and provides each consumer process with a protected, directly accessible interface to the network hardware. Developing distributed applications using low‐level primitives provided by user‐level networking architecture like the VI Architecture is complex and requires significant effort.This paper describes how high‐level communication paradigms like stream sockets and remote procedure call (RPC) can be efficiently built over the VI Architecture. To evaluate performance benefits for standard client–server and multi‐threaded environments, our focus is on off‐the‐shelf sockets and RPC interfaces and commercially available VI Architecture‐based SANs. The key design techniques developed in this paper include credit‐based flow control, decentralized user‐level protocol processing, caching of pinned communication buffers, and deferred processing of completed send operations.In the experimental evaluation, the one‐way bandwidth achieved by stream sockets over VI Architecture was three to four times better than the bandwidth achieved by running legacy protocols over the same interconnect. On the same SAN, high‐performance stream sockets and RPC over VI Architecture achieve significantly better (between 2× and 3× less) latency than conventional stream sockets and RPC over standard networking protocols in a Windows NT™ 4.0 environment. Furthermore, our high‐performance RPC transparently improved the network performance of the distributed component object model (DCOM) by a factor of two to three. Copyright © 2001 John Wiley & Sons, Ltd.