High Bisection Bandwidth Research Articles

A smart and novel approach for managing incast and in-network congestion through adaptive routing

High-Performance Computing and Datacenter systems, with numerous endnodes, demand an efficient interconnection network to prevent performance bottlenecks. Fat-Tree topologies are preferred for their high bisection bandwidth and multiple shortest-path routes. While existing adaptive routing excels in light or in-network congestion, it struggles with incast congestion. This paper proposes a new technique, called Congestion-Aware Adaptive Routing (SCAR), which addresses both in-network and incast congestion. SCAR limits adaptivity for incast congestion, using deterministic routing, while employing adaptive routing for non-congesting flows. It also resolves in-network congestion by routing traffic flows through alternative routes. Simulation experiments on large Fat-Trees using synthetic and trace-based traffic patterns modeling realistic applications demonstrate SCAR’s immediate reaction on mitigating in-network congestion, and a reasonable delay during incast situations, while other state-of-the-art solutions are not able to cope with incast and in-network situations at the same time.

Edge-Based Congestion-Aware Datacenter Load Balancing with Smart Probing

Abstract Data center networks employ multipath topologies for delivering high bisection bandwidth. Load balancing through these multiple paths reduces latency and improves throughput. Network fabric-based schemes need customized hardware and cannot be implemented on commodity hardware. The existing end-host-based solutions have shown performance better than equal-cost multi-path (ECMP) but they use active probing and uneven congestion signal (e.g. coarse-grained RTT or ECN) that degrades performance especially in asymmetric topologies. Therefore, most of these end-host solutions cannot accurately sense congestion. End-to-end latency can effectively provide accurate path congestion information and smart probing with the help of congestion window can reduce network overhead. In this paper, we propose an edge-based congestion-aware load-balancing solution with smart probing. We combine ECN and end-to-end latency to get more accurate congestion metric and with probabilistic approach we avoid congestion at the best available path. Furthermore, we employ congestion window trend to perform smart probing that will reduce network overhead by performing probing only when required. The proposed scheme is implemented in network simulator ns-3 and flow completion time is taken as the key performance metric. The results show that the proposed scheme not only outperformed ECMP, Clove and LetFlow but also improved the performance over Hermes and CONGA at least 10–30% by employing (i) an accurate congestion signal that effectively detects congestion and asymmetries, and (ii) smart probing to reduce network overhead.

Toward higher-radix switches with co-packaged optics for improved network locality in data center and HPC networks [Invited

We study the network locality improvements that can be achieved by using co-packaged optics in data center and high-performance computing (HPC) networks. The increased escape bandwidth offered by co-packaged optics can enable switches with speeds of 51.2 Tb/s and beyond. From a network architecture perspective, the key advantages of introducing co-packaged optics at the switch points include the implementation of large-scale topologies of > 12,000 end points with 4 × higher bisection bandwidth and the reduction of the required number of switches by > 40 % compared with state-of-the-art approaches. From a network operation perspective, improved network locality and faster operation can be achieved since the higher-radix switches can mitigate the impact of network contention. Placing applications under fewer leaf switches reduces the number of packets that cross the spine switches in a leaf-spine topology. The proposed scheme is evaluated via discrete-event simulations: we initially evaluate the network locality properties of the system by using virtual-machine traces from a production data center, and we subsequently quantify the performance improvements by simulating an all-to-all pattern for a variety of message sizes over a number of nodes. The results suggest that co-packaged optics form a promising solution for keeping up with bandwidth scaling in future networks. The virtual-machine analysis shows that large-scale applications can be placed under up to 50% fewer first-level switches, while the network analysis shows speedups of up to 7.1, which translates to execution time reductions of up to 26% and 42.7% for applications with communication ratios of 0.3 and 0.5, respectively.

RMC: Reordering Marking and Coding for Fine-Grained Load Balancing in Data Centers

Data center networks typically adopt multi-rooted tree topologies to provide high bisection bandwidth. Various fine-grained load balancing schemes have been proposed to split flows across multiple paths. However, data center networks suffer from many uncertainties such as highly dynamic traffic. These uncertainties easily make network become asymmetric, resulting in significant packet reordering. Unfortunately, existing solutions passively deal with packet reordering based on a threshold and hardly adapt to asymmetric networks because of lacking the explicit reordering feedback. These solutions either fail to quickly respond to packet loss or cause unnecessary fast retransmission, which reduces link utilization and increases flow completion time. In this paper, we propose a fine-grained load balancing scheme RMC to eliminate the impact of packet reordering and handle uncertainties in asymmetric networks. To avoid unnecessary fast retransmission, the switch proactively identifies reordered packet according to local queue length and global path latency. Furthermore, we employ a coding technique with redundancy optimization to reduce long-tailed flow completion time under network asymmetry. Through a series of large-scale NS2 simulations and testbed experiments, we demonstrate that RMC effectively avoids unnecessary fast retransmission under different network scenarios and reduces flow completion time by up to 72% compared with state-of-the-art schemes.

HTPC: heterogeneous traffic-aware partition coding for random packet spraying in data center networks

Modern data center networks typically adopt multi-rooted tree topologies such leaf-spine and fat-tree to provide high bisection bandwidth. Load balancing is critical to achieve low latency and high throughput. Although the per-packet schemes such as Random Packet Spraying (RPS) can achieve high network utilization and near-optimal tail latency in symmetric topologies, they are prone to cause significant packet reordering and degrade the network performance. Moreover, some coding-based schemes are proposed to alleviate the problem of packet reordering and loss. Unfortunately, these schemes ignore the traffic characteristics of data center network and cannot achieve good network performance. In this paper, we propose a Heterogeneous Traffic-aware Partition Coding named HTPC to eliminate the impact of packet reordering and improve the performance of short and long flows. HTPC smoothly adjusts the number of redundant packets based on the multi-path congestion information and the traffic characteristics so that the tailing probability of short flows and the timeout probability of long flows can be reduced. Through a series of large-scale NS2 simulations, we demonstrate that HTPC reduces average flow completion time by up to 60% compared with the state-of-the-art mechanisms.

SRFabric: A Semi-Reconfigurable Rack Scale Topology

Rack scale design is a promising trend towards customized hardware design, where high density clusters of SoCs are integrated in the rack. One of the biggest challenges for rack scale computing is the interconnection network. Traditional data center topologies require too many ToR switches to support hundreds of SoCs, while distributed fabrics deliver a considerably high end-to-end latency and network oversubscription. Since no one topology fits all kinds of workloads, a flexible in-rack topology requires a careful redesign to dynamically adapt to diverse data center traffic within tight cost and space constraints in the rack. SRFabric is a semi-reconfigurable rack scale network topology that exploits the high path diversity, the cost-effectiveness of distributed fabrics, and the dynamic reconfigurability of circuit switches. This is accomplished by enabling multiple static ports and dynamic ports for each SoC. Leveraging the partial link reconfigurability, SRFabric is able to optimize its topology to dynamically adapt to various workload patterns. We further propose the design of SRFabric to decide the nearly optimal number of dynamic ports and static ports for expected communication density and performance. Extensive evaluations demonstrate that SRFabric can deliver lower average path length, i.e., 2.21 hops on average, and higher bisection bandwidth, i.e., up to 77% nonblocking bandwidth, and provide comparable performance with state-of-the-art strategy XFabric at a lower cost, i.e., XFabric costs up to 3 times more than that of SRFabric.

AAC: Adaptively Adjusting Concurrency by Exploiting Path Diversity in Datacenter Networks

Recent datacenter load balancing designs make full use of all available multiple paths to achieve high bisection bandwidth and support the increasing traffic demands. However, a multitude of uncertainties, such as congestion and asymmetry, easily leads to long tailed latencies for unlucky flows on bad paths. In this paper, we aim at adjusting the maximum number of multiple paths used by existing load balancing designs to achieve good tradeoff between the tailed latency and link utilization. Specifically, we propose a packet-level load balancing called scheme Adaptively Adjusting Concurrency (AAC) to spread packets across the multiple paths, which are adaptively selected according to path diversity. AAC is deployed at switch, without any modifications on end-hosts. The experimental results of NS2 simulation and Mininet implementation show that AAC significantly reduces the flow completion time by $$\sim$$ 21–56% over the state-of-the-art datacenter load balancing designs including MPTCP, LetFlow and RPS.

Coding-Based Distributed Congestion-Aware Packet Spraying to Avoid Reordering in Data Center Networks

Modern Data Center Networks (DCNs) are commonly based on Clos topologies with a large number of equal-cost multiple paths to provide high bisection bandwidth. The existing Random Packet Spraying (RPS) scheme spreads each flow of packets to all available parallel paths in order to achieve good load balancing under symmetric topologies. However, under asymmetric topologies caused by traffic dynamics or link failures, RPS potentially suffers from serious out-of-order problem. Therefore, to avoid packet reordering, we propose a Coding-based Distributed Congestion-aware Packet Spraying mechanism called CDCPS. At the sender end, CDCPS encodes packets using forward error correction (FEC) technology and adaptively adjusts the coding redundancy according to the asymmetric degree of multiple equal-cost paths. To make full use of link bandwidth, CDCPS randomly spreads encoded packets to all available paths at the switches. The original packets can be recovered immediately once enough encoded packets from uncongested paths arrive at the receiver, even if some encoded packets are blocked on congested paths. The test results of NS2 simulation showed that CDCPS eliminates out-of-order packets completely and effectively reduces the average and $99^{th}$ flow completion time by up to 73% and 78% over the state-of-the-art load balancing scheme.

An adaptive, fault tolerant, flow-level routing scheme for data center networks

Recent Data center network topologies put forward multiple paths between the end hosts to provide high bisection bandwidth. For the sake of utilizing the multiple paths efficiently many load-balancing techniques are proposed in literature. These load-balancing proposals mainly deal with distribution of traffic among multiple paths. Further, a good load-balancing technique scales down the latency of flows. While balancing the traffic among multiple paths, few proposals also take care of congestion. Flow-level solutions avoid flow splitting to prevent the packet re-ordering at the destination. Such adaptive solutions try to re-route (change the path) flows dynamically based on the feedback received from the network. In case of link failures, re-routing of the flow based on the queue size of the remote switch alone may be sub-optimal. The path to be chosen among multiple paths for re-routing, also poses a valid question. This paper proposes FlowFurl, an adaptive, distributed, end host driven and flow-level load-balancing technique. This adaptive routing technique supports dynamic flow-to-path assignment. This flow-to-path assignment is performed not only based on the congestion at the switch, but also the status of the links along the path. FlowFurl is the first technique to combine link failure and congestion information parameter for re-routing of flows. The proposed technique is deployment friendly, however it incurs small amount of overhead (small amount of memory at each switch to maintain the link state). We show that our proposed scheme FlowFurl performs better than considered schemes in terms of average flow latency, percentage of flows that achieve deadline and average flow throughput, by utilizing local information such as connectivity information with congestion information.

Combining Source-adaptive and Oblivious Routing with Congestion Control in High-performance Interconnects using Hybrid and Direct Topologies

Hybrid and direct topologies are cost-efficient and scalable options to interconnect thousands of end nodes in high-performance computing (HPC) systems. They offer a rich path diversity, high bisection bandwidth, and a reduced diameter guaranteeing low latency. In these topologies, efficient deterministic routing algorithms can be used to balance smartly the traffic flows among the available routes. Unfortunately, congestion leads these networks to saturation, where the HoL blocking effect degrades their performance dramatically. Among the proposed solutions to deal with HoL blocking, the routing algorithms selecting alternative routes, such as adaptive and oblivious, can mitigate the congestion effects. Other techniques use queues to separate congested flows from non-congested ones, thus reducing the HoL blocking. In this article, we propose a new approach that reduces HoL blocking in hybrid and direct topologies using source-adaptive and oblivious routing. This approach also guarantees deadlock-freedom as it uses virtual networks to break potential cycles generated by the routing policy in the topology. Specifically, we propose two techniques, called Source-Adaptive Solution for Head-of-Line Blocking Avoidance (SASHA) and Oblivious Solution for Head-of-Line Blocking Avoidance (OSHA). Experiment results, carried out through simulations under different traffic scenarios, show that SASHA and OSHA can significantly reduce the HoL blocking.

A cost-effective low-latency overlaid torus-based data center network architecture

In this paper, we present the design, analysis, and implementation of a novel data center network architecture named CLOT, which delivers significant reduction in the network diameter, network latency, and infrastructure cost. CLOT is built based on a switchless torus topology by adding only a number of most beneficial low-end switches in a proper way. Forming the servers in close proximity of each other in torus topology well implements the network locality. The extra layer of switches largely shortens the average routing path length of torus network, which increases the communication efficiency. We show that CLOT can achieve lower latency, smaller routing path length, higher bisection bandwidth and throughput, and better fault tolerance compared to both conventional hierarchical data center networks as well as the recently proposed CamCube network. Coupled with the coordinate based geographical addresses and credit based flow control, the specially designed POW routing algorithm helps CLOT achieve its maximum theoretical performance. Besides, an automatic address configuration mechanism and malfunction detection mechanism are provided to facilitate the network deployment and configuration. The sufficient mathematical analysis and theoretical derivation prove both guaranteed and ideal performance of CLOT.

Efficient load balancing over asymmetric datacenter topologies

Datacenter networks are often structured as multi-rooted trees to provide high bisection bandwidth at low cost. To utilize the available bisection bandwidth, an efficient load balancing algorithm is required. Under symmetric network conditions, packet spraying is known to perform well, as it enables fine-grained (packet-level) load balancing over equal cost paths. However, packet spraying performs poorly in asymmetric topologies. To make packet spraying effective under asymmetry while retaining its simplicity, we propose SAPS, “Symmetric Adaptive Packet Spraying”, a Software-Defined Networking (SDN) based scheme that uses packet spraying over symmetric virtual topologies. SAPS is based on the key insight that if we provide each flow with a symmetric view of the network fabric, then packet spraying can produce near-optimal performance. Through simulations and testbed experiments, we evaluate SAPS. Over a variety of application workloads and asymmetric network scenarios, including single and multiple link failures, results indicate that SAPS performs well, e.g., under single link failure, outperforming state-of-the-art load balancing schemes by up to 61% for average flow completion times.

Load Balancing in Data Center Networks: A Survey

Data center networks usually employ the scale-out model to provide high bisection bandwidth for applications. A large amount of data is required to be transferred frequently between servers across multiple paths. However, traditional load balancing algorithms like equal-cost multi-path routing are not suitable for rapidly varying traffic in data center networks. Based on the special data center topologies and traffic characteristics, researchers have recently proposed some novel traffic scheduling mechanisms to balance traffic. In this paper, we present a comprehensive survey of recent solutions for load balancing in data center networks. First, recently proposed data center network topologies and the studies of traffic characteristics are introduced. Second, the definition of the load-balancing problem is described. Third, we analyze the differences between data center load balancing mechanisms and traditional Internet traffic scheduling. Then, we present an in-depth overview of recent data center load balancing mechanisms. Finally, we analyze the performance of these solutions and discuss future research directions.

Non-SPF routing algorithm based on ordered semi-group preference algebra

Layer 2 network technology is extending beyond its traditional local area implementation and finding wider acceptance in provider's metropolitan area networks and large-scale cloud data center networks. This is mainly due to its plug-and-play capability and native mobility support. Many efforts have been put to increase the bisection bandwidth in layer 2 network, which has been constrained by the spanning tree protocol (STP) that layer 2 network uses for preventing looping. The recent trend is to incorporate layer 3's routing approach into layer 2 network so that multiple paths can be used for forwarding traffic between any source-destination (S-D) node pair. Equal cost multipath (ECMP) is one such example. However, ECMP may still be limited in generating multiple paths due to its shortest path (lowest cost) requirement. In this paper, we consider a non-shortest-path routing approach, called equal preference multipath (EPMP) based on ordered semi group theory, which can generate more paths than ECMP. In EPMP routing, all the paths with different traditionally-defined costs, such as hops, bandwidth, etc., can be determined equally now and thus they become equal candidate paths. By the comparative tests with ECMP, EPMP routing not only generates more paths, provides 15% higher bisection bandwidth, but also identifies bottleneck links in a hierarchical network when different traffic patterns are applied. EPMP is more flexible in controlling the number and length of multipath generation. Simulation results indicate the effectiveness of the proposed algorithm. It is a good reference for non-blocking running of big datacenter networks.

Dynamic Flow Scheduling With Uncertain Flow Duration in Optical Data Centers

Optical switching based on wavelength division multiplexing has become a promising network technology to scale the performance of data centers. It provides high bisection bandwidth with low power consumption and low complexity of network wiring. However, it raises new challenges for the flow scheduling problem due to the dynamic arrival of traffic flows with unknown service duration combined with the circuit-switched nature of optical networks and wavelength continuity constraint. While the knowledge of flow service time helps to use resources in a better way to increase the revenue, in practice, the service time cannot be accurately specified. In this paper, we address the problem of flow scheduling in optical data centers considering the above challenges. We first develop an optimization formulation using Markov decision process that can estimate the flow termination time and revenue for cloud providers in a long run under the uncertainty in flow service time. Since solving the optimization formulation is mathematically intractable, we then develop heuristic scheduling algorithms for both scenarios: with known and with unknown flow service time. We use a probabilistic model to address the uncertainty due to unknown flow service time. We design a flow scheduling framework that integrates the proposed algorithms to perform flow scheduling in optical data center networks. We evaluate the proposed algorithms through comprehensive simulations and compare their performance against that of a baseline algorithm. The results show that the proposed algorithms achieve significant performance improvement compared with the baseline algorithm.

OFPT: OpenFlow based Parallel Transport in Datacenters

Although the dense interconnection datacenter networks (DCNs) (e.g. FatTree) provide multiple paths and high bisection bandwidth for each server pair, the single-path TCP (SPT) and ECMP which are widely used currently neither achieve high bandwidth utilization nor have good load balancing. Due to only one available transmission path, SPT cannot make full use of all available bandwidth, while ECMP`s random hashing results in many collisions. In this paper, we present OFPT, an OpenFlow based Parallel Transport framework, which integrates precise routing and scheduling for better load balancing and higher network throughput. By adopting OpenFlow based centralized control mechanism, OFPT computes the optimal path and bandwidth provision for each flow according to the global network view. To guarantee high throughput, OFPT dynamically schedules flows with Seamless Flow Migration Mechanism (SFMM), which can avoid packet loss in flow rerouting. Finally, we test OFPT on Mininet and implement it in a real testbed. The experimental results show that the average network throughput in OFPT is up to 97.5% of bisection bandwidth, which is higher than ECMP by 36%. Besides, OFPT decreases the average flow completion time (AFCT) and achieves better scalability.

DiFS: Distributed Flow Scheduling for adaptive switching in FatTree data center networks

Data center networks leverage multiple parallel paths connecting end host pairs to offer high bisection bandwidth for cluster computing applications. However, the state-of-the-art routing protocols such as Equal Cost Multipath (ECMP) is load-oblivious due to the static flow-to-link assignment. They may cause bandwidth loss due to flow collisions. Recently proposed centralized scheduling algorithm or host based adaptive routing that requires network-wide state information may suffer from scalability problems. In this paper, we present Distributed Flow Scheduling (DiFS), a new adaptive switching method, for FatTree data center networks, which is a localized and switch-only solution. DiFS allows switches to cooperate to avoid over-utilized links and find available paths without centralized control. DiFS is scalable and can react quickly to dynamic traffic because it is independently executed on switches and requires no synchronization. DiFS provides global bounds of flow balance based on local optimization. Extensive simulations show that the aggregate throughput of DiFS using various traffic patterns is much better than that of ECMP, and is similar to or higher than those of two representative protocols that use network-wide optimization.

HHS: an efficient network topology for large-scale data centers

Designing an efficient topology is a critical challenge for large-scale data center networks. Although the current switch-centric network topologies have high bisection bandwidth, they bear the disadvantage of high network cost. In this paper, we propose a novel switch-centric data center network topology called Hyper Hoffman---Singleton (HHS). HHS is a symmetric multi-dimensional topology, in which switches form a Hoffman---Singleton graph in each dimension. The proposed topology can accommodate a large number of servers with small network diameter, low cost and high bisection bandwidth. We also present a multipath routing algorithm for HHS. Our simulation results show that the HHS network offers low latency and high throughput under different workloads. By comparing with the existing data center network topologies, we show that HHS is a promising candidate for large-scale data centers because of its ability to achieve a desirable trade-off between performance and cost, without introducing any overheads on servers.

Equal Preference Multi-Path Routing for L2 Hierarchical Networks

The layer 2 network technology is extending beyond its traditional local area implementation and finding wider acceptance in provider’s metropolitan area networks and large-scale cloud data center networks. This is mainly due to its plug-and-play capability and native mobility support. Many efforts have been put to increase the bisection bandwidth in a layer 2 network, which has been constrained by the spanning tree protocol that a layer 2 network uses for preventing looping. The recent trend is to incorporate layer 3’s routing approach into a layer 2 network so that multiple paths can be used for forwarding traffic between any source-destination (S-D) node pair. ECMP (equal cost multipath) is one such example. However, ECMP may still be limited in generating multiple paths due to its shortest path (lowest cost) requirement. In this paper, we consider a non-shortest-path routing approach, called EPMP (Equal Preference Multi-Path) that can generate more paths than ECMP. The EPMP is based on the ordered semi-group algebra. In the EPMP routing, paths that differ in traditionally-defined costs, such as hops, bandwidth, etc., can be made equally preferred and thus become candidate paths. We found that, in comparison with ECMP, EPMP routing not only generates more paths, provides higher bisection bandwidth, but also allows bottleneck links in a hierarchical network to be identified when different traffic patterns are applied. EPMP is also versatile in that it can use various ways of path preference calculations to control the number and the length of paths, making it flexible (like policy-based routing) but also objective (like shortest path first routing) in calculating preferred paths.

A Cached Middleware to Improve I/O Performance Using POSIX Interface

The performance of computing devices is increasing day by day, the storage devices are not capable to reach with this benchmark. The compute and storage resources are forming two parts of the shared network. This paper is presenting a storage middleware called HyCache+ to reduce the high bi-section bandwidth of the parallel computing systems. The middleware uses Programmable Operating System (POSIX) interface to the end users to swap the data with the high capacity network attached storage. The system uses a 2-layer scheduling approach. The HyCache+ can be deployed on the IBM Blue Gene / P supercomputers for the performance comparison.