High Bisection Bandwidth Research Articles

Large switches or blocking multi-stage networks? An evaluation of routing strategies for datacenter fabrics

Cloud computing clusters require efficient interconnects to deal with the increasing volume of inter-server (east–west) traffic. To cope with these new traffic patterns, datacenter networks are abandoning the oversubscribed topologies of the past, and adopt fat-tree fabrics with high bisection bandwidth. However, these fabrics typically employ either single-path or coarse-grained (flow-level) multipath routing.In this paper, we characterize the waste of bandwidth due to routing inefficiencies. Our analysis, confirmed by computer simulations, demonstrates that under a randomly selected permutation the expected throughputs of d-mod-k routing and of Equal-Cost-Multi-Pathing (ECMP) (or flow-level multipath routing) (Thaler and Hopps, 2000) [1] are close to 63% and 47%, respectively. Furthermore, nearly 30% of the flows are expected to undergo an unnecessary 3-fold slowdown. In contrast, packet-level multipath routing consistently delivers full throughput to all flows, thus serving the growing demands of inter-server (east–west) traffic better.Using unmodified TCP stacks, we also demonstrate that under typical traffic conditions and system configurations flow-level multi-path routing can abruptly increase the completion time of latency-critical flows by more than one order of magnitude. In contrast, packet-level multipath routing, proactively avoids in-fabric backlogs, and minimizes the flow completion time across the full range of configurations that we examine. Finally, we present the design of a cost-efficient switch node performing adaptive packet-level spraying.

Congestion-aware adaptive forwarding in datacenter networks

Datacenters employ the scale-out model to achieve scalability. This model requires parallelism in the underlying workload. Therefore, high bisection bandwidth is required to support intensive communications between servers. Several new datacenter architectures have been designed to provide redundant bandwidth. Currently, it is critical to design a mechanism to efficiently utilize the abundant bandwidth. In this paper, we propose a distributed Congestion-Aware Adaptive foRwording (CAAR) protocol to balance traffic load only depending on the local queue length information. CAAR allows flows to select under-utilized paths to forward packets. It is theoretically proved to be stable if the arrival rates are within the network throughput region. Simulation results under diverse datacenter topologies and communication patterns validate that CAAR achieves higher aggregate goodput compared with random and static routing protocols.

Minimizing Energy Consumption of FatTree Data Center Networks

Many data centers are built using a fat-tree network topology because of its high bisection bandwidth. Therefore thereis a need to develop analytical models for the energy behavior of fat-tree networks and examine strategies to reduce energy consumption. The most effective strategy is to power off entire switches, if possible. In this paper, we derive formulas for the minimum number of active switches needed in a fat-tree data center network for arbitrary types of loading. We also derive expressions for the expected traffic loss when these networks are overloaded with external (Internet) traffic. Results of detailed simulations conducted using well-known traffic models for data center networks [4] closely match our derived formulas. We show that a fat-tree network incurs significant energy cost even when very lightly loaded. In order to further reduce energy consumption, we consolidate traffic into fewer switches and derive expressions for energy cost versus load assuming traffic consolidation and show linear scaling. Finally, we observe that traffic patterns have a significant impact on energy consumption and this fact is evident in the analytical formulas.

OSA: An Optical Switching Architecture for Data Center Networks With Unprecedented Flexibility

A detailed examination of evolving traffic characteristics, operator requirements, and network technology trends suggests a move away from nonblocking interconnects in data center networks (DCNs). As a result, recent efforts have advocated oversubscribed networks with the capability to adapt to traffic requirements on-demand. In this paper, we present the design, implementation, and evaluation of OSA, a novel Optical Switching Architecture for DCNs. Leveraging runtime reconfigurable optical devices, OSA dynamically changes its topology and link capacities, thereby achieving unprecedented flexibility to adapt to dynamic traffic patterns. Extensive analytical simulations using both real and synthetic traffic patterns demonstrate that OSA can deliver high bisection bandwidth (60%-100% of the nonblocking architecture). Implementation and evaluation of a small-scale functional prototype further demonstrate the feasibility of OSA.

On-Line Multicast Scheduling with Bounded Congestion in Fat-Tree Data Center Networks

Multicast benefits numerous data center applications that require group communication by eliminating sending unnecessary duplicated packets in the network, thus significantly reduces network traffic and improves application throughput. Meanwhile, most data center networks (DCNs) today adopt a multi-rooted tree structure called fat-tree, which utilizes rich path multiplicity to deliver high bisection bandwidth. However, without an efficient flow scheduling algorithm that appropriately routes multicast flows to achieve traffic load balance, heavy congestion may occur throughout the network, which prevents full utilization of such high degree of link parallelism and causes unpredictable network performance. Hence, in this paper we study multicast flow scheduling in fat-tree DCNs, where multicast flow requests arrive one by one without a priori knowledge of future traffic. To address the drastic traffic fluctuation in data centers, we consider a very general traffic model called hose traffic model, where the only assumption is that the total bandwidth demand of traffic that enters (leaves) an ingress (egress) link of each server at any time is bounded by the capacity of its network interface card. We present a low-complexity on-line multicast flow scheduling algorithm for fat-tree DCNs. The algorithm can achieve bounded congestion and efficient bandwidth utilization under any arbitrary sequence of multicast flow requests that satisfy the hose model. We also derive the bound on congestion that the algorithm can achieve in a fat-tree DCN. Finally, we evaluate the algorithm by an event-driven DCN simulator under various types of traffic patterns, and show that the algorithm achieves superior performance in terms of network throughput and evenness of traffic load distribution.

Overview of the SpiNNaker System Architecture

SpiNNaker (a contraction of Spiking Neural Network Architecture) is a million-core computing engine whose flagship goal is to be able to simulate the behavior of aggregates of up to a billion neurons in real time. It consists of an array of ARM9 cores, communicating via packets carried by a custom interconnect fabric. The packets are small (40 or 72 bits), and their transmission is brokered entirely by hardware, giving the overall engine an extremely high bisection bandwidth of over 5 billion packets/s. Three of the principal axioms of parallel machine design (memory coherence, synchronicity, and determinism) have been discarded in the design without, surprisingly, compromising the ability to perform meaningful computations. A further attribute of the system is the acknowledgment, from the initial design stages, that the sheer size of the implementation will make component failures an inevitable aspect of day-to-day operation, and fault detection and recovery mechanisms have been built into the system at many levels of abstraction. This paper describes the architecture of the machine and outlines the underlying design philosophy; software and applications are to be described in detail elsewhere, and only introduced in passing here as necessary to illuminate the description.

Scalable high-radix router microarchitecture using a network switch organization

As the system size of supercomputers and datacenters increases, cost-efficient networks become critical in achieving good scalability on those systems. High -radix routers reduce network cost by lowering the network diameter while providing a high bisection bandwidth and path diversity. The building blocks of these large-scale networks are the routers or the switches and they need to scale accordingly to the increasing port count and increasing pin bandwidth. However, as the port count increases, the high-radix router microarchitecture itself needs to scale efficiently. Hierarchical crossbar switch organization has been proposed where a single large crossbar used for a router switch is partitioned into many small crossbars and overcomes the limitations of conventional router microarchitecture. Although the organization provides high performance, it has limited scalability due to excessive power and area overheads by the wires and intermediate buffers. In this article, we propose scalable router microarchitectures that leverage a network within the switch design of the high-radix routers themselves. These alternative designs lower the wiring complexity and buffer requirements. For example, when a folded-Clos switch is used instead of the hierarchical crossbar switch for a radix-64 router, it provides up to 73%, 58%, and 87% reduction in area, energy-delay product, and energy-delay-area product, respectively. We also explore more efficient switch designs by exploiting the traffic-pattern characteristics of the global network and its impact on the local network design within the switch for both folded-Clos and flattened butterfly networks. In particular, we propose a bilateral butterfly switch organization that has fewer crossbars and global wires compared to the topology-agnostic folded-Clos switch while achieving better low-load latency and equivalent saturation throughput.

Free-scaling your data center

The increasing popularity of both small and large private clouds and expanding public clouds poses new requirements to data center (DC) architectures. First, DC architectures should be incrementally scalable allowing the creation of DCs of arbitrary size with consistent performance characteristics. Second, initial DC deployments should be incrementally expandable supporting small-scale upgrades without decreasing operation efficiency. A DC architecture possessing both properties satisfies the requirement of free-scaling.Recent work in DC design focuses on traditional performance and scalability characteristics, therefore resulting in symmetric topologies whose upgradability is coarse-grained at best. In our earlier work we proposed Scafida, an asymmetric, scale-free network inspired DC topology which scales incrementally and has favorable structural characteristics. In this paper, we build on Scafida and propose a full-fledged DC architecture achieving free-scaling called FScafida. Our main contribution is threefold. First, we propose an organic expansion algorithm for FScafida; this combined with Scafida’s flexible original design results in a freely scalable architecture. Second, we introduce the Effective Source Routing mechanism that provides near-shortest paths, multi-path and multicast capability, and low signaling overhead by exploiting the benefits of the FScafida topology. Third, we show based on extensive simulations and a prototype implementation that FScafida is capable of handling the traffic patterns characteristic of both enterprise and cloud data centers, tolerates network equipment failures to a high degree, and allows for high bisection bandwidth.

Incrementally upgradable data center architecture using hyperbolic tessellations

Current trends in cloud computing suggest that both large, public clouds and small, private clouds will proliferate in the near future. Operational requirements, such as high bandwidth, dependability and smooth manageability, are similar for both types of clouds and their underlying data center architecture. Such requirements can be satisfied with utilizing fully distributed, low-overhead mechanisms at the algorithm level, and an efficient layer 2 implementation at the practical level. On the other hand, owners of evolving private data centers are in dire need of an incrementally upgradeable architecture which supports a small roll-out and continuous expansion in small quanta. In order to satisfy both requirements, we propose Poincaré, a data center architecture inspired by hyperbolic tessellations, which utilizes low-overhead, greedy routing. On one hand, Poincaré scales to support large data centers with low diameter, high bisection bandwidth, inherent multipath and multicast capabilities, and efficient error recovery. On the other hand, Poincaré supports incremental plug & play upgradability with regard to both servers and switches. We evaluate Poincaré using analysis, extensive simulations and a prototype implementation.

A Novel Approach to Optical Switching for Intradatacenter Networking

In this paper, we propose to apply a novel paradigm called labeled optical burst switching with home circuit (LOBS-HC) for intradatacenter networking to provide a high bisection bandwidth and significantly reduce the cost and energy consumption associated with electronic packet switching. The unique features of LOBS-HC that make it more suitable than either optical circuit switching (OCS) or optical packet/burst switching are exploited to enable all-to-all communications with a guaranteed lossless transmission bandwidth between any given pair of pods, while also supporting bursty transmissions through wavelength-sharing among home circuits (HCs) and statistical multiplexing. As a case study, hypercube-like topologies are considered for the interconnection among the pods within a datacenter. In particular, we first propose a simple but efficient HC assignment scheme called complementary HC for 2-D cube or ring, and then extend our works to n-cube and generalized hypercube by applying the concept of spanning balanced tree (SBT) for their HC assignment. Our analysis results show that with such datacenters, the minimum number of wavelengths needed in each case is significantly reduced from that needed with OCS and also, the network cost in terms of wires and transceivers needed is considerably reduced from that incurs with datacenters using electronic packet switching. We then evaluate the traffic performance of such hypercube-based datacenters using LOBS-HC through simulation experiments via the OPNET simulator. The performance results obtained for a variety of communication patterns and traffic models within a datacenter demonstrate the feasibility of the proposed approach.

Embedded Signal Processing Using Free-Space Optical Hypercube Interconnects

The speed and complexity of integrated circuits are increasing rapidly. For instance, today's mainstream processors have already surpassed gigahertz global clock frequencies on-chip. As a consequence, many algorithms proposed for applications in embedded signal-processing (ESP) systems, e.g. radar and sonar systems, can be implemented with a reasonable number (less than 1000) of processors, at least in terms of computational power. An extreme inter-processor network is required, however, to completely implement those algorithms. The demands are such that completely new interconnection architectures must be considered. In the search for new architectures, developers of parallel computer systems can actually take advantage of optical interconnects. The main reason for introducing optics from a system point of view is the strength in using benefits that enable new architecture concepts, e.g. free-space propagation and easy fan-out, together with benefits that can actually be exploited by simply replacing the electrical links with optical ones without changing the architecture, e.g. high bandwidth and complete galvanic isolation. In this paper, we propose a system suitable for embedded signal processing with extreme performance demands. The system consists of several computational modules that work independently and send data simultaneously in order to achieve high throughput. Each computational module is composed of multiple processors connected in a hypercube topology to meet scalability and high bisection bandwidth requirements. Free-space optical interconnects and planar packaging technology make it possible to arrange the hypercubes as planes with an associated three-dimensional communications space and to take advantage of many optical properties. For instance, optical fan-out reduces hardware cost. Altogether, this makes the system capable of meeting high performance demands in, for example, massively parallel signal processing. One 64-channel airborne radar system with nine computational modules and a sustained computational speed of more than 1.6 Tera floating point operations per second (TFLOPS) is presented. The effective inter-module bandwidth in this configuration is 1 024 Gbit/s.

FAST-Net optical interconnection prototype demonstration

The first prototype of the free-space accelerator for switching terabit networks (FAST-Net) program is described. The optical interconnection module consists of an array of matched high-resolution, wide-angle (f/1.1) imaging lenses that effect a global point-to-point shuffle link pattern across a multichip smart pixel array. A mirror is used to fold the shuffled links back onto the array to facilitate single-plane multichip packaging and simplified opto-mechanical alignment. The test system incorporates several monolithically integrated two-dimensional arrays of vertical-cavity surface-emitting lasers (VCSELs) and photodetectors (PDs) in the multichip plane. In initial experiments the prototype showed good overall optical efficiency and sufficient channel separation for VCSEL/PD arrays with pitches as small as 100 m. These results suggest that high bisection bandwidth optical free-space interconnection fabrics are feasible for high-throughput switching and multiprocessor applications.

Performance scaling comparison for free-space optical and electrical interconnection approaches

Projected performance metrics of free-space optical and electrical interconnections are estimated and compared in terms of smart-pixel input-output bandwidth density and practical geometric packaging constraints. The results suggest that three-dimensional optical interconnects based on smart pixels provide the highest volume, latency, and power-consumption benefits for applications in which globally interconnected networks are required to implement links across many integrated-circuit chips. It is further shown that interconnection approaches based on macro-optical elements achieve better scaling than those based on micro-optical elements. The scaling limits of micro-optical-based architectures stem from the need for repeaters to overcome diffraction losses in multichip architectures with high bisection bandwidth. The overall results provide guidance in determining whether and how strongly a free-space optical interconnection approach can be applied to a given multiprocessor problem.