Low-latency Interconnection Research Articles

Gateway: New high performance computing facility for EFDA task force on integrated Tokamak modelling

The EFDA task force on Integrated Tokamak Modelling (ITM-TF) is providing the EU fusion community with a complete and flexible suite of reliable software tools and codes able to simulate the next ITER and DEMO plasma discharges. EFDA has launched the Gateway project as the first computing facility to be jointly used by EU fusion associations. The Gateway has been designed to allow the ITM-TF members to work together on a common platform and share their codes, developments tools and technologies as well as to make able the inter-operation with tera-scale supercomputer facilities. Technically the Gateway is a rather small high computing facility in operation since 2008, with 1 Teraflops of theoretical peak and 100 Terabytes shared storage area for experimental and simulation data. This paper describes the information technologies involved in the Gateway facility, particularly as regards the low latency interconnect network for multi-core platform as well as fast I/O solutions implemented with storage over Infiniband and high performance parallel file systems. The solutions adopted on the Gateway facility had undergone to specific benchmarks showing excellent performances in typical fusion data handling issues. Finally the interoperability between Gateway and ENEA CRESCO supercomputer will be described showing the access solutions that allow the exploitation of tera-scale supercomputer facilities.

Mesh-of-Trees and Alternative Interconnection Networks for Single-Chip Parallelism

In single-chip parallel processors, it is crucial to implement a high-throughput low-latency interconnection network to connect the on-chip components, especially the processing units and the memory units. In this paper, we propose a new mesh of trees (MoT) implementation of the interconnection network and evaluate it relative to metrics such as wire complexity, total register count, single switch delay, maximum throughput, tradeoffs between throughput and latency, and post-layout performance. We show that on-chip interconnection networks can provide higher bandwidth between processors and shared first-level cache than previously considered possible, facilitating greater scalability of memory architectures that require that. MoT is also compared, both analytically and experimentally, to some other traditional network topologies, such as hypercube, butterfly, fat trees and butterfly fat trees. When we evaluate a 64-terminal MoT network at 90-nm technology, concrete results show that MoT provides higher throughput and lower latency especially when the input traffic (or the on-chip parallelism) is high, at comparable area. A recurring problem in networking and communication is that of achieving good sustained throughput in contrast to just high theoretical peak performance that does not materialize for typical work loads. Our quantitative results demonstrate a clear advantage of the proposed MoT network in the context of single-chip parallel processing.

Long-Time Simulations on High Resolution Meshes to Model Calcium Waves in a Heart Cell

A model for the flow of calcium on the scale of one heart cell is given by a system of time-dependent reaction-diffusion equations coupled by nonlinear reaction terms. Calcium ions enter into the cell at release units distributed throughout the cell and then diffuse. At each release unit, the probability for calcium to be released increases along with the concentration of calcium, thus creating a feedback loop of waves regenerating themselves repeatedly. The validation of this model requires simulations on the time scale of several repeated waves and on the spatial scale of the entire cell. This requires long-time studies on spatial meshes that need to have a high resolution to resolve the positions of the calcium release units throughout the entire cell. We detail the development of a special-purpose numerical method and parallel implementation for this problem. Parallel performance studies demonstrate the scalability of the implementation on a distributed-memory cluster with low-latency interconnect. Convergence studies verify convergence to analytical expectations and confirm the appropriateness of all numerical parameters. Application studies on the desired time and length scales confirm that the model exhibits the desired feedback mechanism for calcium currents through the release units at suitable high levels, but the long-time studies demonstrate also that the current model with its present parameters leads to excessive calcium concentrations over time. This phenomenon could only be observed using a computational method able to reach laboratory scale final times for a domain on the scale of a complete cell.

A unified real-time control and data acquisition hardware platform

This paper describes the implementation aspects of a real-time control and data acquisition hardware platform. The goal is to provide real-time signal processing at the signal acquisition level as well as low-latency interconnection between modules and units. In addition it will simultaneously perform the synchronism of all digitizer and generator endpoints. An analysis of the currently available low-latency data links and hardware processors is presented as well as an overview of the available tools for programming the platform. A prototype of this platform is expected for the first quarter of 2006 and is targeted to be used on neutron and gamma diagnostics. The platform will initially include the following modules: interconnect switch module, three types of transient recorders sampling at a rate of up to 400 MSPS@12-bit, 200 MSPS@14-bit and 2 GSPS@8-bit; one signal generator sourcing at a rate of up to 1 GSPS@12-bit, and a 400 ps time digitizer with six channels having a resolution of 400 ps and a throughput of 20 Mevents/s.

Performances of the Data Vortex switch architecture under nonuniform and bursty traffic

The Data Vortex switch architecture has been proposed as a scalable low-latency interconnection fabric for optical packet switches. This self-routed hierarchical architecture employs synchronous timing and distributed traffic-control signaling to eliminate optical buffering and to reduce the required routing logic, greatly facilitating a photonic implementation. In previous work, we have shown the efficient scalability of the architecture under uniform and random traffic conditions while maintaining high throughput and low-latency performance. This paper reports on the performance of the Data Vortex architecture under nonuniform and bursty traffic conditions. The results show that the switch architecture performs well under modest nonuniform traffic, but an excessive degree of nonuniformity will severely limit the scalability. As long as a modest degree of asymmetry between the number of input and output ports is provided, the Data Vortex switch is shown to handle very bursty traffic with little performance degradation.

SerialExpress-a high performance workstation interconnect

As part of the IEEE P2100 working group, we produced a SerialExpress draft standard that specifies a high-performance interconnect for use with workstations. Our goal was to develop an inexpensive, efficient, robust, scalable protocol for use in system area network (SAN) applications. Within this environment, we used buslike read and write transactions to provide an efficient mechanism for transferring data and controlling information between processor and I/O nodes. Our bandwidth, distance, and latency goals mandated the concurrent transmission of small (64-byte) packets to avoid the arbitration delay constraints of a broadcast bus protocol. We also wanted our solution to scale over a wide range of technologies. This article discusses the resulting draft standard, focusing on the technical aspects. In the future, we plan to address its implementation technologies and implications. This draft defines a protocol for low-cost, technology-independent, high-bandwidth, low-latency, and distance-insensitive interconnections for system area network applications.

Digital's clusters and scientific parallel applications

Traditional implementations of message passing libraries such as PVM [1] have incurred high latencies. The use of a low latency interconnect [2] and well-tuned message passing software greatly reduces this latency. For an interesting class of applications, this reduced latency has a great impact on execution time. This article describes such a low latency interconnect, Memory Channel™, from a low-level programmer's perspective. It then gives an overview of the implementation of message passing libraries on top of Memory Channel, and finally describes the impact of such an optimized message passing facility on an application

Multicasting protocols for high-speed, wormhole-routing local area networks

Wormhole routing LANs are emerging as an effective solution for high-bandwidth, low-latency interconnects in distributed computing and cluster computing applications. An important example is the 640 Mb/s crossbar-based Myrinet. A key property of conventional LANs, which is valuable for many distributed applications, is transparent, reliable network-level multicast. It is desirable to retain this property also in wormhole LANs. Unfortunately, efficient, reliable multicasting in wormhole LANs is problematic because of the potential for deadlocks. As a consequence, current multicasting implementations typically consist of repeated unicast or assume a priori buffer reservations. These solutions, however, tend to increase latency and do not scale well.In this paper we address the problem of providing transparent, reliable, efficient network level multicasting in the wormhole LAN. We describe several protocols for achieving deadlock-free, reliable multicasting using restricted routing and fast buffer reservation techniques. Tradeoffs involving complexity and performance of various solutions are discussed, and are illustrated using simulation. A simple multicast implementation for Myrinet has been carried out, and experimental results are presented.

VCSEL-array-based angle-multiplexed optoelectronic crossbar interconnects

A large crossbar switch, which is a desirable building block for any low-latency interconnect network, is difficult to implement because of many practical problems associated with digital electronics. We propose a new method for implementing a large optoelectronic crossbar interconnect to take advantage of a unique principle of optics. Based on an emerging vertical-cavity surface-emitting laser (VCSEL) technology, a passive angle-multiplexed beam-steering architecture is proposed as a key component of the optoelectronic crossbar. Various optical system parameters are evaluated. Because there is no optical fan-out power loss, the interconnect capacity of the proposed system is determined by the diffraction-limited receiver power cutoff, and therefore interconnection of more than 1000 nodes with a per node bandwidth of 1 GHz is possible with today's technology. A 64-element VCSEL-array-based proof-of-principle optical system for studying the interconnect scalability has been built. Details of the features of the proposed system, its advantages and limitations, demonstration experimental results, and their analyses are presented.

Technologies for low latency interconnection switches

This paper presents an engineering design for a low latency high bandwidth interconnection network which will form the switching substrate for a multi-model parallel processing system. The performance is enhanced with a variety of approaches covering interconnection protocols, routing, fault tolerance, advanced packaging, and electrical interconnection techniques. The synergistic application of these technologies leads to a high performance design.