Data Center Network Topologies Research Articles

Multiple Congestion Points and Congestion Reaction Mechanisms for Improving DCTCP Performance in Data Center Networks

For addressing problems such as long delays, latency fluctuations, and frequent timeouts in conventional Transmission Control Protocol (TCP) in a data center environment, Data Center TCP (DCTCP) has been proposed as a TCP replacement to satisfy the requirements of data center networks. It is gaining more popularity in academic as well as industry areas due to its performance in terms of high throughput and low latency, and is widely deployed in data centers. However, according to the recent research about the performance of DCTCP, authors have found that most times the sender’s congestion window reduces to one segment, which results in timeouts. In addition, the authors observed that the nonlinear marking mechanism of DCTCP causes severe queue oscillation, which results in low throughput. To address the above issues of DCTCP, we propose multiple congestion points using double threshold and congestion reaction using window adjustment (DT-CWA) mechanisms for improving the performance of DCTCP by reducing the number of timeouts. The results of a series of simulations in a typical data center network topology using Qualnet network simulator, the most widely used network simulator, demonstrate that the proposed window-based solution can significantly reduce the timeouts and noticeably improves the throughput compared to DCTCP under various network conditions.

Space-Division Multiplexing in Data Center Networks: On Multi-Core Fiber Solutions and Crosstalk-Suppressed Resource Allocation

The rapid growth of traffic inside data centers caused by the increasing adoption of cloud services necessitates a scalable and cost-efficient networking infrastructure. Space-division multiplexing (SDM) is considered as a promising solution to overcome the optical network capacity crunch and support cost-effective network capacity scaling. Multi-core fiber (MCF) is regarded as the most feasible and efficient way to realize SDM networks, and its deployment inside data centers seems very likely as the issue of inter-core crosstalk (XT) is not severe over short link spans (<1 km) compared to that in long-haul transmission. However, XT can still have a considerable effect in MCF over short distances,which can limit the transmission reach and in turn the data center’s size. XT can be further reduced by bi-directional transmission of optical signals in adjacent MCF cores. This paper evaluates the benefits of MCF-based SDM solutions in terms of maximizing the capacity and spatial efficiency of data center networks. To this end, we present an analytical model for XT in bi-directional normal step-index and trench-assisted MCFs and propose corresponding XT-aware core prioritization schemes. We further develop XT-aware spectrum resource allocation strategies aimed at relieving the complexity of online XT computation. These strategies divide the available spectrum into disjoint bands and incrementally add them to the pool of accessible resources based on the network conditions. Several combinations of core mapping and spectrum resource allocation algorithms are investigated for eight types of homogeneous MCFs comprising 7–61 cores, three different multiplexing schemes, and three data center network topologies with two traffic scenarios. Extensive simulation results showthat combining bi-directional transmission in dense core fibers with tailored resource allocation schemes significantly increases the network capacity. Moreover, a multiplexing scheme that combines SDM and WDM can achieve up to 33 times higher link spatial efficiency and up to 300 times greater capacity compared to a WDM solution.

WMGR: A Generic and Compact Routing Scheme for Data Center Networks

Data center networks (DCNs) connect hundreds and thousands of computers and, as a result of the exponential growth in their number of nodes, the design of scalable (compact) routing schemes plays a pivotal role in the optimal operation of the DCN. Traditional trends in the design of DCN architectures have led to solutions, where routing schemes and network topologies are interdependent, i.e., specialized routing schemes. Unlike these, we propose a routing scheme that is compact and generic, i.e., independent of the DCN topology, the word-metric-based greedy routing. In this scheme, each node is assigned to a coordinate (or label) in the word-metric space (WMS) of an algebraic group and then nodes forward packets to the closest neighbor to the destination in this WMS. We evaluate our scheme and compare it with other routing schemes in several topologies. We prove that the memory space requirements in nodes and the forwarding decision time grow sub-linearly (with respect to n, the number of nodes) in all of these topologies. The scheme finds the shortest paths in topologies based on Cayley graphs and trees (e.g. Fat tree), while in the rest of topologies, the length of any path is stretched by a factor that grows logarithmically (with respect to n). Moreover, the simulation results show that many of the paths remain far below this upper bound.

Switch hierarchical provider (SHP) networks for data network virtualization

Data network virtualization (DNV) is a well-emerging data infrastructure environment, consisting of data center network topology, virtualized data center and traditional ISP. DNV is described by the network topology, routing and protocols, is a collection of virtual hardware networking resources. DNV is organized in a tree-like topology connecting to consecutive k-pods for k/2 edge switches: making complex aggregation switch topology, provides virtual networks consisting virtual nodes and virtual links: the propagation delay between virtual nodes becomes valuable when data virtual networks is congested with data, multi-packet forwarding schemes are used to spread the data among different virtual network paths: presence of fault-packet forwarding between network nodes are intolerance and service migration is required to balance the data load among the data virtual nodes. This paper, presents a switch hierarchical provider network which allows each data virtual nodes providing a separate logical DN routine to reduce the propagation delay through virtual-hierarchical topology utilising complex multi-path topology reducing switching complexity, and proposed traffic congestion-less schemes to reduce the fault-tolerance and load sharing. Our model evaluates a network virtualization infrastructure addressing the real word DNV, critical analysis of network level routing algorithm and data virtualization solution to deploy DNV platforms with different platforms.

Load Balancing in Data Center Networks: A Survey

Data center networks usually employ the scale-out model to provide high bisection bandwidth for applications. A large amount of data is required to be transferred frequently between servers across multiple paths. However, traditional load balancing algorithms like equal-cost multi-path routing are not suitable for rapidly varying traffic in data center networks. Based on the special data center topologies and traffic characteristics, researchers have recently proposed some novel traffic scheduling mechanisms to balance traffic. In this paper, we present a comprehensive survey of recent solutions for load balancing in data center networks. First, recently proposed data center network topologies and the studies of traffic characteristics are introduced. Second, the definition of the load-balancing problem is described. Third, we analyze the differences between data center load balancing mechanisms and traditional Internet traffic scheduling. Then, we present an in-depth overview of recent data center load balancing mechanisms. Finally, we analyze the performance of these solutions and discuss future research directions.

Scalable topological forwarding and routing policies in RINA‐enabled programmable data centers

AbstractGiven the current expansion of cloud computing, the expected advent of the Internet of Things, and the requirements of future fifth‐generation network infrastructures, significantly larger pools of computational and storage resources will soon be required. This emphasizes the need for more scalable data centers that are capable of providing such an amount of resources in a cost‐effective way. A quick look into today's commercial data centers shows that they tend to rely on variations of well‐defined leaf‐spine/Clos data center network (DCN) topologies, offering low latency, ultrahigh bisectional bandwidth, and enhanced reliability against concurrent failures. However, DCNs are typically restricted by the use of the Transmission Control Protocol/Internet Protocol (TCP/IP) suite, thus suffering limited routing scalability. In this work, we study the benefits that replacing TCP/IP with the recursive internetwork architecture (RINA) can bring into commercial DCNs, focusing on forwarding and routing scalability. We quantitatively evaluate the benefits that RINA solutions can yield against those based on TCP/IP and highlight how, by deploying RINA, topological routing solutions can improve even more the efficiency of the network. To this goal, we propose a rule‐and‐exception forwarding policy tailored to the characteristics of several DCN variants, enabling fast forwarding decisions with merely neighbors' information. Upon failures, few exceptions are necessary, whose computation can also profit from the known topology. Extensive numerical results show that the proposed policy requirements depend mainly on the number of neighbors and concurrent failures in the DCN rather than its size, dramatically reducing the amount of forwarding and routing information stored at DCN nodes.

On energy consumption of switch-centric data center networks

Data center network (DCN) is the core of cloud computing and accounts for 40% energy spend when compared to cooling system, power distribution and conversion of the whole data center (DC) facility. It is essential to reduce the energy consumption of DCN to ensure energy-efficient (green) data center can be achieved. An analysis of DC performance and efficiency emphasizing the effect of bandwidth provisioning and throughput on energy proportionality of two most common switch-centric DCN topologies: three-tier (3T) and fat tree (FT) based on the amount of actual energy that is turned into computing power are presented. Energy consumption of switch-centric DCNs by realistic simulations is analyzed using GreenCloud simulator. Power-related metrics were derived and adapted for the information technology equipment processes within the DCN. These metrics are acknowledged as subset of the major metrics of power usage effectiveness and data center infrastructure efficiency, known to DCs. This study suggests that although in overall FT consumes more energy, it spends less energy for transmission of a single bit of information, outperforming 3T.

PowerNetS: Coordinating Data Center Network With Servers and Cooling for Power Optimization

Recently, a lot of research efforts have been made to optimize the large amounts of energy consumed by different devices in data centers, including servers, cooling, and the data center network (DCN). Unfortunately, current research addresses these devices mostly in a separate manner, leading to inferior optimization results. This paper proposes PowerNetS, a power optimization framework that coordinates servers and DCN, as well as cooling, for minimized power consumption of a data center. PowerNetS leverages workload correlation analysis for more energy savings during server and traffic consolidations. More importantly, PowerNetS tries to change the DCN topology during server consolidation, in order to have more intra-server traffic and shorter flows that go through fewer switches. For example, two virtual machines previously located on two different servers can now be migrated to the same server, so that the flow between them no longer needs to use switches, which allows more devices to sleep for energy savings without network performance degradation. PowerNetS has been implemented on a physical testbed with 6 servers and 10 virtual switches that are configured using a production 48-port OpenFlow switch. Our evaluation with Wikipedia, Yahoo!, and IBM traces shows that PowerNetS can save up to 51.6% of energy by coordinating servers and DCN, which is 44.3% and 15.8% more than two state-of-the-art baselines, respectively. By further coordinating with cooling to utilize different cooling efficiencies at different locations within a data center, PowerNetS can achieve 8.8%–14.6% additional energy savings.

Optical switching based small-world data center network

Redesigning data center networks, particularly taking advantages of optical switching technologies to reduce cost-per-bandwidth and energy consumption, has spawned interests of researchers worldwide in recent years. In this paper, we introduce and analysis our recently proposed optical switching technology powered data center network architecture, which is inspired from the small world data center network topology yet has additional flexibility brought by optical lightpath's reconfigurability. Specifically, logically full-meshed optical burst switching rings are employed to compose a lattice substrate, which provide dense connectivity for small groups of ToR switches to accommodate highly clustered and dynamic regional data traffic, while a reconfigurable wavelength circuit switching plane offers direct connections among the racks to effectively reduce overall network diameter. A centralized control plane is employed to realize traffic-adaptive optical lightpath scheduling mechanisms. We thoroughly study the throughput and latency performance of the proposed architecture through numerical simulations, and particularly reveal the impact of traffic patterns. It turns out that, the proposed network architecture can achieve high throughput with significantly reduced cost, and exploiting its traffic-adaptive reconfiguration capability can effectively improve throughput and reduce latency under varying traffic distributions.

SDN-Based Data Center Networking With Collaboration of Multipath TCP and Segment Routing

Large-scale data centers are major infrastructures in the big data era. Therefore, a stable and optimized architecture is required for data center networks (DCNs) to provide services to the applications. Many studies use software-defined network (SDN)-based multipath TCP (MPTCP) implementation to utilize the entire DCN’s performance and achieve good results. However, the deployment cost is high. In SDN-based MPTCP solutions, the flow allocation mechanism leads to a large number of forwarding rules, which may lead to storage consumption. Considering the advantages and limitations of the SDN-based MPTCP solution, we aim to reduce the deployment cost due to the use of an extremely expensive storage resource—ternary content addressable memory (TCAM). We combine MPTCP and segment routing (SR) for traffic management to limit the storage requirements. And to the best of our knowledge, we are among the first to use the collaboration of MPTCP and SR in multi-rooted DCN topologies. To explain how MPTCP and SR work together, we use four-layer DCN architecture for better description, which contains physical topology, SR over the topology, multiple path selection supplied by MPTCP, and traffic scheduling on the selected paths. Finally, we implement the proposed design in a simulated SDN-based DCN environment. The simulation results reveal the great benefits of such a collaborative approach.

The stellar transformation: From interconnection networks to datacenter networks

The first dual-port server-centric datacenter network, FiConn, was introduced in 2009 and there are several others now in existence; however, the pool of topologies to choose from remains small. We propose a new generic construction, the stellar transformation, that dramatically increases the size of this pool by facilitating the transformation of well-studied topologies from interconnection networks, along with their networking properties and routing algorithms, into viable dual-port server-centric datacenter network topologies. We demonstrate that under our transformation, numerous interconnection networks yield datacenter network topologies with potentially good, and easily computable, baseline properties. We instantiate our construction so as to apply it to generalized hypercubes and obtain the datacenter networks GQ⋆. Our construction automatically yields routing algorithms for GQ⋆ and we empirically compare GQ⋆ (and its routing algorithms) with the established datacenter networks FiConn and DPillar (and their routing algorithms); this comparison is with respect to network throughput, latency, load balancing, fault-tolerance, and cost to build, and is with regard to all-to-all, many all-to-all, butterfly, random, hot-region, and hot-spot traffic patterns. We find that GQ⋆ outperforms both FiConn and DPillar (sometimes significantly so) and that there is substantial scope for our stellar transformation to yield new dual-port server-centric datacenter networks that are a considerable improvement on existing ones.

BCCC: An Expandable Network for Data Centers

Designing a cost-effective network topology for data centers that can deliver sufficient bandwidth and consistent latency performance to a large number of servers has been an important and challenging problem. Many server-centric data center network topologies have been proposed recently due to their significant advantage in cost efficiency and data center agility, such as BCube, FiConn, and Bidimensional Compound Network (BCN). However, existing server-centric topologies are either not expandable or demanding prohibitive expansion cost. As the scale of data centers increases rapidly, the lack of expandability in existing server-centric data center networks imposes a severe obstacle for data center upgrade. In this paper, we present a novel server-centric data center network topology called BCube connected crossbars (BCCCs), which can provide good network performance using inexpensive commodity off-the-shelf switches and commodity servers with only two network interface card (NIC) ports. A significant advantage of BCCC is its good expandability. When there is a need for expansion, we can easily add new servers and switches into the existing BCCC with little alteration of the existing structure. Meanwhile, BCCC can accommodate a large number of servers while keeping a very small network diameter. A desirable property of BCCC is that its diameter increases only linearly to the network order (i.e., the number of dimensions), which is superior to most of the existing server-centric networks, such as FiConn and BCN, whose diameters increase exponentially with network order. In addition, there are a rich set of parallel paths with similar length between any pair of servers in BCCC, which enables BCCC to not only deliver sufficient bandwidth capacity and predictable latency to end hosts, but also provide graceful performance degradation in case of component failure. We conduct comprehensive comparisons between BCCC with other popular server-centric network topologies, such as FiConn and BCN. We also propose an effective addressing scheme and routing algorithms for BCCC. We show that BCCC has significant advantages over the existing server-centric topologies in many important metrics, such as expandability, server port utilization, and network diameter.

Performance analysis of modified BCube topologies for virtualized data center networks

Network virtualization enables the computing networks and data center (DC) providers to manage their networking resources in a flexible manner using software running on physical computers. Thus, using DC topology with high flexibility to the required service-level changes is very important to this purpose. In this paper, we address the existing issues with the classic DC network topology in virtualized DC networks and investigate the performance of a set of modified BCube topologies with the capability of providing dynamic structures according to the bandwidth requirement in a virtual DC network. In particular, we propose three main approaches to modify the structure of a classic BCube topology as a topology benchmark and investigate their associated structural features and maximum achievable interconnected bandwidth for different routing scenarios. Finally, we run an extensive simulation program to check the performance of the proposed modified topologies in a simulation environment which considers the failure analysis and also traffic congestion. Our simulation experiments, which are consistent to our design goals, show the efficiency of the proposed modified topologies comparing to the classic BCube in terms of bandwidth availability and failure resiliency.

A flexible and high-performance data center network topology

In order to meet the insatiable demands for cloud-based services, today's data centers may have hundreds of thousands of servers. During the past decade, many topologies have been proposed to inter...

Optimal Polynomial Time Algorithm for Restoring Multicast Cloud Services

The failure-prone nature of data center networks has evoked countless contributions to develop proactive and reactive countermeasures. Yet, most of these techniques were developed with unicast services in mind. When in fact, multiple services hosted in data center networks today rely on multicast communication to disseminate traffic. Hence, the existing survivability schemes fail to cater to the distinctive properties and quality of service requirements that multicast services entail. This letter is devoted to understanding the ramifications of facility node or substrate link failure on multicast services residing in cloud networks. We formally define the multicast virtual network restoration problem and prove its NP-complete nature in arbitrary graphs. Furthermore, we prove that the problem can be solved in polynomial-time in multi-rooted treelike data center network topologies.

On implementation of DCTCP on three-tier and fat-tree data center network topologies.

A data center is a facility for housing computational and storage systems interconnected through a communication network called data center network (DCN). Due to a tremendous growth in the computational power, storage capacity and the number of inter-connected servers, the DCN faces challenges concerning efficiency, reliability and scalability. Although transmission control protocol (TCP) is a time-tested transport protocol in the Internet, DCN challenges such as inadequate buffer space in switches and bandwidth limitations have prompted the researchers to propose techniques to improve TCP performance or design new transport protocols for DCN. Data center TCP (DCTCP) emerge as one of the most promising solutions in this domain which employs the explicit congestion notification feature of TCP to enhance the TCP congestion control algorithm. While DCTCP has been analyzed for two-tier tree-based DCN topology for traffic between servers in the same rack which is common in cloud applications, it remains oblivious to the traffic patterns common in university and private enterprise networks which traverse the complete network interconnect spanning upper tier layers. We also recognize that DCTCP performance cannot remain unaffected by the underlying DCN architecture hence there is a need to test and compare DCTCP performance when implemented over diverse DCN architectures. Some of the most notable DCN architectures are the legacy three-tier, fat-tree, BCube, DCell, VL2, and CamCube. In this research, we simulate the two switch-centric DCN architectures; the widely deployed legacy three-tier architecture and the promising fat-tree architecture using network simulator and analyze the performance of DCTCP in terms of throughput and delay for realistic traffic patterns. We also examine how DCTCP prevents incast and outcast congestion when realistic DCN traffic patterns are employed in above mentioned topologies. Our results show that the underlying DCN architecture significantly impacts DCTCP performance. We find that DCTCP gives optimal performance in fat-tree topology and is most suitable for large networks.

Correlation-Aware Traffic Consolidation for Power Optimization of Data Center Networks

Power optimization has become a key challenge in the design of large-scale enterprise data centers. Existing research efforts focus mainly on computer servers to lower their energy consumption, while only few studies have tried to address data center networks (DCNs), which can account for 10-20 percent of the total energy consumption of a data center. In this paper, we propose CARPO, a correlation-aware power optimization algorithm that dynamically consolidates traffic flows onto a small set of links and switches in a DCN and then shuts down unused network devices for energy savings. In sharp contrast to existing work, CARPO is designed based on a key observation from the analysis of real DCN traces that the bandwidth demands of different flows do not peak at exactly the same time. As a result, if the correlations among flows are considered in consolidation, more energy savings can be achieved. In addition, CARPO integrates traffic consolidation with link rate adaptation for maximized energy savings. We implement CARPO on a hardware testbed composed of 10 virtual switches configured with a production 48-port OpenFlow switch and 8 servers. Our empirical results with traces from Wikipedia and Yahoo! data centers demonstrate that CARPO can save up to 50 percent of network energy for a DCN, while having only negligible delay increases. CARPO also outperforms two state-of-the-art baselines by 19.6 and 95 percent on energy savings, respectively. Our simulation results with a large-scale DCN also show that CARPO can achieve more energy savings than the baselines for typical DCN topologies, such as fat tree and BCube.

Adaptive Path Isolation for Elephant and Mice Flows by Exploiting Path Diversity in Datacenters

Resource competition and conflicts in datacenter networks (DCNs) are frequent and intense. They become inevitable when mixing elephant and mice flows on shared transmission paths, resulting in arbitration between throughput and latency and performance degradation. We propose a novel flow scheduling scheme, Freeway, that leverages on path diversity in the DCN topology to guarantee, simultaneously, mice flow completion within deadline and high network utilization. Freeway adaptively partitions the available paths into low latency and high throughput paths and provides different transmission services for each category. A M/G/1-based model is developed to theoretically obtain the highest value of average delay over the path that will guarantee for 99% of mice flows their completion time before the deadline. Based on this bound, Freeway proposes a dynamic path partitioning algorithm to adjust dynamically with varying traffic load the number of low latency and high throughput paths. While mice flows are transmitted over low latency paths using a simple equal cost multiple path (ECMP) scheduling, Freeway load balances elephant flows on different high-throughput paths. We evaluate Freeway in a series of simulation on a large scale topology and use real traces. Our evaluation results show that Freeway significantly reduces the mice flows completion time within deadlines, while achieving remarkable throughput compared with current schemes. It is remarkable that Freeway does not need any change of DCN switch fabrics or scheduling algorithms and can be deployed easily on any generic datacenter network with switches implementing VLANs and trunking.

HHS: an efficient network topology for large-scale data centers

Designing an efficient topology is a critical challenge for large-scale data center networks. Although the current switch-centric network topologies have high bisection bandwidth, they bear the disadvantage of high network cost. In this paper, we propose a novel switch-centric data center network topology called Hyper Hoffman---Singleton (HHS). HHS is a symmetric multi-dimensional topology, in which switches form a Hoffman---Singleton graph in each dimension. The proposed topology can accommodate a large number of servers with small network diameter, low cost and high bisection bandwidth. We also present a multipath routing algorithm for HHS. Our simulation results show that the HHS network offers low latency and high throughput under different workloads. By comparing with the existing data center network topologies, we show that HHS is a promising candidate for large-scale data centers because of its ability to achieve a desirable trade-off between performance and cost, without introducing any overheads on servers.

Reliability and Survivability Analysis of Data Center Network Topologies

The architecture of several data centers have been proposed as alternatives to the conventional three-layer one.Most of them employ commodity equipment for cost reduction. Thus, robustness to failures becomes even more important, because commodity equipment is more failure-prone. Each architecture has a different network topology design with a specific level of redundancy. In this work, we aim at analyzing the benefits of different data center topologies taking the reliability and survivability requirements into account. We consider the topologies of three alternative data center architecture: Fat-tree, BCube, and DCell. Also, we compare these topologies with a conventional three-layer data center topology. Our analysis is independent of specific equipment, traffic patterns, or network protocols, for the sake of generality. We derive closed-form formulas for the Mean Time To Failure of each topology. The results allow us to indicate the best topology for each failure scenario. In particular, we conclude that BCube is more robust to link failures than the other topologies, whereas DCell has the most robust topology when considering switch failures. Additionally, we show that all considered alternative topologies outperform a three-layer topology for both types of failures. We also determine to which extent the robustness of BCube and DCell is influenced by the number of network interfaces per server.