Virtual Interface Architecture Research Articles

Influence of high performance sockets on data processing intensity

Performance and limitations on the Virtual Interface Architecture Sockets (VIAS) are studied, using a component framework designed to provide runtime support for data intensive applications. The experimental results show that by reorganizing certain components, the significant performance applications improvements are achieving and performance characteristics of VIAS allow a more efficient data partitioning on the nodes

Comparing Ethernet and Soft RoCE for MPI Communication

RDMA has pleasant consideration in the late 1990 when the Virtual Interface Architecture was introduced. This growth has accelerated with the introduction of Open Fabrics Alliance’s (OFA’s) Verb Interface. The stability & independence of OFA verb interface facilitated significant growth of software applications that exploit the benefits of RDMA. RoCE is an emerging trend that can be made to work on the Ethernet infrastructures. In this paper, we evaluate the Linux cluster, having multi nodes with fast interconnects i.e. Gigabit Ethernet & Soft RoCE and evaluates its performance using IMB and OSU Micro Benchmarks. A comparison between the IMB and OSU Micro benchmark is done and our results shows that IMB Benchmark is performing better in case of collective Benchmark class as OSU Micro benchmark is performing better in other classes of benchmark.

An SSL Back-End Forwarding Scheme in Cluster-Based Web Servers

State-of-the-art cluster-based data centers consisting of three tiers (Web server, application server, and database server) are being used to host complex Web services such as e-commerce applications. The application server handles dynamic and sensitive Web contents that need protection from eavesdropping, tampering, and forgery. Although the secure sockets layer (SSL) is the most popular protocol to provide a secure channel between a client and a cluster-based network server, its high overhead degrades the server performance considerably and, thus, affects the server scalability. Therefore, improving the performance of SSL-enabled network servers is critical for designing scalable and high-performance data centers. In this paper, we examine the impact of SSL offering and SSL-session-aware distribution in cluster-based network servers. We propose a back-end forwarding scheme, called ssl_with_bf, that employs a low-overhead user-level communication mechanism like virtual interface architecture (VIA) to achieve a good load balance among server nodes. We compare three distribution models for network servers, round robin (RR), ssl_with_session, and ssl_with_bf, through simulation. The experimental results with 16-node and 32-node cluster configurations show that, although the session reuse of ssl_with_session is critical to improve the performance of application servers, the proposed back-end forwarding scheme can further enhance the performance due to better load balancing. The ssl_with_bf scheme can minimize the average latency by about 40 percent and improve throughput across a variety of workloads.

The logic description of the system of embedded hardware logic task

A new task mode, hardware logic task mode, is presented. Its architecture, schedule and implementation are described with HDL(Hardware Description Language), and the validity of the system has been proved by logic simulation. It has advantage for real-time applications and overheadsaving for operating system, so it is profitable for the controller in the embedded system. The relationship among RTOS (Real-Time Operating System), SoC (System on Chip), VIA (Virtual Interface Architecture) and hardware logic task is also discussed in the paper.

Distributed smart disks for I/O-intensive workloads on switched interconnects

Disk-based smart storage represents processor-embedded active I/O devices that are equipped with a dedicated network interface controller and on-disk storage (memory and disk). Today’s I/O-intensive workloads require architectures whose performance scales with increasing storage capacity. The advent of switch-based, low-latency and high-bandwidth network protocols, such as Virtual Interface Architecture and InfiniBand, promises to improve both performance and scalability beyond current state-of-the-art I/O interconnect topologies. This paper evaluates a disk-based distributed smart storage architecture composed of fully distributed processor-embedded disks that are interconnected by a high-performance switched network, and investigates the performance of the proposed processing model on different storage interconnects with representative workloads, including TPC-H queries, association rules mining, high-dimensional data clustering, and two-dimensional fast Fourier transform (FFT).

Anatomy of UDP and M-VIA for cluster communication

High interprocess communication latency is detrimental to parallel and grid computing. Over the years, the network bandwidth has increased rapidly while the end-to-end latency has not decreased much. This is because the latency is dominated by the protocol software execution time in the kernel instead of the raw transmission time over the link. In this paper, we perform an anatomical analysis of the complete communication path between a sender and a receiver through measurements. We present an in-depth evaluation of various components of the UDP protocol over Fast Ethernet. Virtual Interface Architecture (VIA) protocol has been recently proposed to overcome the software overhead of the TCP/UDP/IP protocol. We analyze M-VIA, a modular VIA implementation for Linux over Ethernet, and compare its performance with UDP. The aim of our experiments is to present the protocol overheads in details rather than to suggest new techniques to reduce overheads.

Software Distributed Shared Memory: a VIA-based implementation and comparison of sequential consistency with home-based lazy release consistency

A Distributed Shared Memory (DSM) system provides a distributed application with a shared virtual address space. This article proposes a design for implementing the DSM communication layer on top of the Virtual Interface Architecture (VIA), an industry standard for user-level networking protocols on high-speed clusters. User-level communication protocols operate in user mode, thus removing the operating system kernel's overhead from the critical communication pass, and significantly diminishing communication overhead as a result. We analyze VIA's facilities and limitations in order to ascertain which implementation trade-offs can be best applied to our development of an efficient communication substrate optimized for DSM requirements. We then implement a multithreaded version of the Home-based Lazy Release Consistency (HLRC) protocol on top of this substrate. In addition, we compare the performance of this HLRC protocol with that of the Sequential Consistency (SC) protocol in which a MultiView (MV) memory mapping technique was used. This technique enables a fine-grained access to shared memory, while still relying on the virtual memory hardware to track memory accesses. We perform an ‘apple-to-apple’ comparison on the same testbed environment and benchmark suite, and investigate the effectiveness and scalability of both protocols. Copyright © 2005 John Wiley & Sons, Ltd.

Integrated smart sensor networking framework for sensor-based appliances

This paper presents an integrated, sensor networking framework stemming from the IEEE 1451 smart transducer interface standard. This object-based networking model is complemented by the Virtual Interface Architecture (VIA), a Compaq-Intel-Microsoft approach to Internet messaging, and by the Internet Protocol (IP) multicast communication, mediating efficient and unified access from Internet to smart sensors. An application of networked sensors that offers a reusable design pattern for a class of sensor-based appliances is analyzed. The kernel of the paper focuses utilization of this framework for a computer-based pressure measurement systems developing environment as a real-world project while stressing smart pressure sensor and Internet connectivity architectures.

Design and implementation of a user‐level Sockets layer over Virtual Interface Architecture

AbstractThe Virtual Interface Architecture (VIA) is an industry standard user‐level communication architecture for system area networks. The VIA provides a protected, directly‐accessible interface to a network hardware, removing the operating system from the critical communication path. In this paper, we design and implement a user‐level Sockets layer over VIA, named SOVIA (Sockets Over VIA). Our objective is to use the SOVIA layer to accelerate the existing Sockets‐based applications with a reasonable effort and to provide a portable and high‐performance communication library based on VIA to application developers.SOVIA realizes comparable performance to native VIA, showing a minimum one‐way latency of 10.5 $\mu$s and a peak bandwidth of 814 Mbps on Giganet's cLAN. We have shown the functional compatibility with the existing Sockets API by porting File Transfer Protocol (FTP) and Remote Procedure Call (RPC) applications over the SOVIA layer. Compared to the Giganet's LAN Emulation (LANE) driver which emulates TCP/IP inside the kernel, SOVIA easily doubles the file transfer bandwidth in FTP and reduces the latency of calling an empty remote procedure by 77% in RPC applications. Copyright © 2003 John Wiley & Sons, Ltd.

병렬 파일 시스템에서의 하이브리드 채널 모델

병렬 파일 시스템은 고속의 네트웍으로 여러 대의 컴퓨터들을 서로 연결하여 컴퓨터들 간에 메시지를 주고받으면서 파일을 분산 저장하고 병렬로 읽어오는 방식으로 파일 입출력 장치의 병목현상을 해결한다. 그러나 대부분의 병렬 파일 시스템은 전달하려는 메시지의 특성을 고려하지 않은 프로토콜의 사용으로 성능저하의 문제를 가지고 있다. 이에 따라 본 논문에서 메시지 처리 방법으로 HCM(Hybrid Channel Model)을 제안한다. 본 논문에서 제안하는 HCM은 병렬 파일 시스템에서 전달되는 메시지를 그 특성에 따라 분리해서 별개의 프로토콜을 이용하여 제어 메시지와 파일 데이터 블록을 전송한다. 안정성이 검증된 TCP/IP를 이용하여 구현한 메시지 채널을 통해 제어 메시지를 고속의 데이터 전송이 가능한 VIA를 이용하여 구현한 데이터 채널을 통해 파일 데이터 블록을 각각 분리하여 처리하도록 하고 있다 HCM을 병렬 파일 시스템에 구현하고 실험해본 결과 본 논문에서 제안한 채널 모델이 상당한 성능향상을 보였다. Parallel file system solves I/O bottleneck to store a file distributedly and read it parallel exchanging messages among computers that is connected multiple computers with high speed networks. However, they do not consider the message characteristics and performances are decreased. Accordingly, the current study proposes the Hybrid Channel model (HCM) as a message-management method, whereby the messages of a parallel file system are classified by a message characteristic between control messages and file data blocks, and the communication channel is divided into a message channel and data channel. The message channel then transfers the control messages through TCP/IP with reliability, while the data channel that is implemented by Virtual Interface Architecture (VIA) transfers the file data blocks at high speed. In tests, the proposed parallel file system that is implemented by HCM exhibited a considerably improved performance.

AN EFFICIENT IMPLEMENTATION OF THE BSP PROGRAMMING LIBRARY FOR VIA

Virtual Interface Architecture(VIA) is a light-weight protocol for protected user-level zero-copy communication. In spite of the promised high performance of VIA, previous MPI implementations for GigaNet's cLAN revealed low communication performance. Two main sources of such low performance are the discrepancy in the communication model between MPI and VIA and the multi-threading overhead. In this paper, we propose a new implementation of the Bulk Synchronous Parallel(BSP) programming library for VIA called xBSP to overcome such problems. To the best of our knowledge, xBSP is the first implementation of the BSP library for VIA. xBSP demonstrates that the selection of a proper library is important to exploit the features of light-weight protocols. Intensive use of Remote Direct Memory Access(RDMA) operations leads to high performance close to the native VIA performance with respect to round trip delay and bandwidth. Considering the effects of multi-threading, memory registration, and completion policy on performance, we could obtain an efficient BSP implementation for cLAN, which was confirmed by experimental results.

An analysis of VI Architecture primitives in support of parallel and distributed communication

AbstractWe present the results of a detailed study of the Virtual Interface (VI) paradigm as a communication foundation for a distributed computing environment. Using Active Messages and the Split‐C global memory model, we analyze the inherent costs of using VI primitives to implement these high‐level communication abstractions. We demonstrate a minimum mapping cost (i.e. the host processing required to map one abstraction to a lower abstraction) of 5.4 μs for both Active Messages and Split‐C using four‐way 550 MHz Pentium III SMPs and the Myrinet network. We break down this cost to the use of individual VI primitives in supporting flow control, buffer management and event processing and identify the completion queue as the source of the highest overhead. Bulk transfer performance plateaus at 44 Mbytes/s for both implementations are due to the addition of fragmentation requirements. Based on this analysis, we present the implications for the VI successor, Infiniband. Copyright © 2002 John Wiley & Sons, Ltd.

Design Alternatives for Virtual Interface Architecture and an Implementation on IBM Netfinity NT Cluster

The Virtual Interface Architecture (VIA) specification has been developed to standardize user-level network interfaces that provide low-latency, high-bandwidth communications. Few hardware and software implementations of VIA exist. Since the VIA specification is flexible, different choices exist for implementing various components of VIA such as doorbells, address translation methods, and completion queues. Although previous studies have evaluated the overall performance of different VIA implementations, there has not been a comparative study on the performance of VIA components. In this paper, we evaluate and compare the performance of different implementations of essential VIA components. We discuss the pros and cons of each design approach and describe the required support for implementing each of them. Then, we discuss an experimental implementation of the Virtual Interface Architecture for the IBM SP Switch-Connected NT cluster, one of the newest clustering platforms available. We discuss different design issues involved in this implementation. In particular, we explain how the virtual-to-physical address translation is implemented efficiently with a minimum Network Interface Card (NIC) memory requirement. We show how caching the VIA descriptors on the NIC can reduce the communication latency. We also present an efficient scheme for implementing the VIA doorbells without any hardware support. We provide a comprehensive performance evaluation study and discuss the impact of several hardware improvements on the performance of our implementation. The performance of the implemented VIA surpasses that of other existing software implementations of the VIA and is comparable to that of a hardware VIA implementation. The peak measured bandwidth for our system is 101.4MBytes/s and the one-way latency for short messages is 18.2μs.

VI architecture communication features and performance on the Giganet cluster LAN

The virtual interface (VI) architecture standard was developed to satisfy the need for a high throughput, low latency communication system required for cluster computing. VI architecture aims to close the performance gap between the bandwidths and latencies provided by the communication hardware and visible to the application, respectively, by minimizing the software overhead on the critical path of the communication. This paper presents the results of a performance study of one VI architecture hardware implementation, the Giganet cLAN (cluster LAN). The focus of the study is to assess and compare the performance of different VI architecture data transfer modes and specific features that are available to higher-level communication software like MPI in order to aid the implementor to decide which VI architecture options to employ for various communication scenarios. Examples of such options include the use of send/receive vs. RDMA data transfers, polling vs. blocking to check completion of communication operations, multiple VIs, completion queues and scatter capabilities of VI architecture.

Design and implementation of efficient communication abstractions on the Virtual Interface Architecture: Stream sockets and RPC experience

AbstractThe emergence and standardization of system area networks (SANs) has provided distributed applications with a medium for high‐bandwidth, low‐latency communication. Standard user‐level networking architecture such as the Virtual Interface (VI) Architecture enables distributed applications to perform low overhead communication over SANs. The VI Architecture significantly reduces system processing overheads and provides each consumer process with a protected, directly accessible interface to the network hardware. Developing distributed applications using low‐level primitives provided by user‐level networking architecture like the VI Architecture is complex and requires significant effort.This paper describes how high‐level communication paradigms like stream sockets and remote procedure call (RPC) can be efficiently built over the VI Architecture. To evaluate performance benefits for standard client–server and multi‐threaded environments, our focus is on off‐the‐shelf sockets and RPC interfaces and commercially available VI Architecture‐based SANs. The key design techniques developed in this paper include credit‐based flow control, decentralized user‐level protocol processing, caching of pinned communication buffers, and deferred processing of completed send operations.In the experimental evaluation, the one‐way bandwidth achieved by stream sockets over VI Architecture was three to four times better than the bandwidth achieved by running legacy protocols over the same interconnect. On the same SAN, high‐performance stream sockets and RPC over VI Architecture achieve significantly better (between 2× and 3× less) latency than conventional stream sockets and RPC over standard networking protocols in a Windows NT™ 4.0 environment. Furthermore, our high‐performance RPC transparently improved the network performance of the distributed component object model (DCOM) by a factor of two to three. Copyright © 2001 John Wiley & Sons, Ltd.

QoS provisioning in clusters

Design of high performance cluster networks (routers) with Quality-of-Service (QoS) guarantees is becoming increasingly important to support a variety of multimedia applications, many of which have real-time constraints. Most commercial routers, which are based on the wormhole-switching paradigm, can deliver high performance, but lack QoS provisioning. In this paper, we present a pipelined wormhole router architecture that can provide high and predictable performance for integrated traffic in clusters. We consider two different implementations—a non-preemptive model and a more aggressive preemptive model. We also present the design of a network interface card (NIC) based on the Virtual Interface Architecture (VIA) design paradigm to support QoS in the NIC. The QoS capable router and NIC designs are evaluated with a mixed workload consisting of best-effort traffic, multimedia streams, and control traffic. Simulation results of an 8-port router and a (2 × 2) mesh network indicate that the preemptive router can provide better performance than the non-preemptive router for dynamically changing workloads. Co-evaluation of the QoS-aware NIC with the proposed router models shows significant performance improvement compared to that with a traditional NIC without any QoS support.

Javia: A Java interface to the virtual interface architecture

The Virtual Interface (VI) architecture has become the industry standard for user-level network interfaces. This paper presents the implementation and evaluation of Javia, a Java interface to the VI architecture. Javia explores two points in the design space. The first approach manages buffers in C and requires data copies between the Java heap and native buffers. The second approach relies on a Java-level buffer abstraction that eliminates the copies in the first approach. Javia achieves an effective bandwidth of 80 Mbytes s−1 for 8 kbyte messages, which is within 1% of those achieved by C programs. Performance evaluations of parallel matrix multiplication and of the active messages communication protocol show that Javia can serve as an efficient building block for Java cluster applications. Copyright © 2000 John Wiley & Sons, Ltd.

Server capacity planning for Web traffic workload

The goal of the paper is to provide a methodology for determining bandwidth requirements for various hardware components of a World Wide Web server. The paper assumes a traditional symmetric multiprocessor (SMP) architecture for the Web server, although the same analysis applies to an SMP node in a cluster. The paper derives formulae for bandwidth demands for memory, processor data bus, network adapters, disk adapters, I/O memory paths, and I/O buses. Since the Web workload characteristics vary widely, three sample workloads are considered for illustrative purposes: 1) standard SPECweb96; 2) a SPECweb96-like workload that assumes dynamic data and retransmissions; and 3) WebProxy, which models a Web proxy server that does not do much caching, and thus has rather severe requirements. The results point to a few general conclusions regarding Web workloads. In particular, reduction in memory/data bus bandwidth by using the virtual interface architecture (VIA) is very desirable, and the connectivity needs may go well beyond the capabilities of traditional systems based on the traditional PCI-bus. Web workloads also demand a significantly higher memory bandwidth than data bus bandwidth and this disparity is expected to increase with the use of VIA. Also, the current efforts to offload TCP/IP processing may require a larger headroom in I/O subsystem bandwidth than in the processor-memory subsystem.

Evolution of the virtual interface architecture

To provide a faster path between applications and the network, researchers have advocated removing the operating system kernel and its centralized networking stack from the critical path and creating a user level network interface. With these interfaces, designers can tailor the communication layers each process uses to the demands of that process. Consequently, applications can send and receive network packets without operating system intervention, which greatly decreases communication latency and increases network throughput. Unfortunately, the diversity of approaches and lack of consensus has stalled progress in refining research results into products-a prerequisite to the widespread adoption of these interfaces. Recently, however, Intel, Microsoft, and Compaq have introduced the Virtual Interface Architecture, an emerging standard for cluster or system area networks. Products based on the VIA have already surfaced, notably GigaNet's GNN1000 network interface. As more products appear, research into application level issues can proceed and the technology of user level network interfaces should mature. Several prototypes-among them Cornell University's U-Net2-have heavily influenced the VIA. We describe the architectural issues and design trade-offs at the core of these prototype designs.