Fat-tree Network Topology Research Articles

Deep Reinforcement Learning-Based Network Routing Technology for Data Recovery in Exa-Scale Cloud Distributed Clustering Systems

Research has been conducted to efficiently transfer blocks and reduce network costs when decoding and recovering data from an erasure coding-based distributed file system. Technologies using software-defined network (SDN) controllers can collect and more efficiently manage network data. However, the bandwidth depends dynamically on the number of data transmitted on the network, and the data transfer time is inefficient owing to the longer latency of existing routing paths when nodes and switches fail. We propose deep Q-network erasure coding (DQN-EC) to solve routing problems by converging erasure coding with DQN to learn dynamically changing network elements. Using the SDN controller, DQN-EC collects the status, number, and block size of nodes possessing stored blocks during erasure coding. The fat-tree network topology used for experimental evaluation collects elements of typical network packets, the bandwidth of the nodes and switches, and other information. The data collected undergo deep reinforcement learning to avoid node and switch failures and provide optimized routing paths by selecting switches that efficiently conduct block transfers. DQN-EC achieves a 2.5-times-faster block transmission time and 0.4-times-higher network throughput than open shortest path first (OSPF) routing algorithms. The bottleneck bandwidth and transmission link cost can be reduced, improving the recovery time approximately twofold.

Load balancing using openday light SDN controller: Case study

Traditional networking architectures have many limitations that need to be overcome to meet modern IT requirements. To overcome these limitations; Software Defined Networking (SDN) is taking place as the new networking approach. As traditional networking uses static switches, resource utilization is poor. There are packet loss and delay during switch breakdown. This paper proposes an implementation of a load balancing algorithm for an SDN based network to overcome the stated issues. To test the algorithm, a network is emulated using the Mininet and OpenDaylight platform (ODL) is used as an SDN controller. Python coding language is used to create fat-tree network topology and to write a load balancing algorithm. Finally, iPerf and Wireshark is used to test network performance. The network was tested before and after running the load balancing algorithm. The testing focused on some of the Quality of Service (QoS) parameters such as bandwidth and transfer rate in the fat-tree network. The algorithm increased bandwidth with at least more than 50%, and improved network utilization

Data privacy‐based coordinated placement method of workflows and data

With the rapid development of data acquisition technology, many industries data already have the characteristics of big data and cloud technology has provided strong support for the storage and complex calculations of these massive data. The meteorological department established the cloud data centre based on the existing storage and computing resources and re-arranged the historical data to reduce the historical data access time of applications. However, the placement of each workflow and input data also affects the average data access time, which in turn affects the computing efficiency of the cloud data centre. At the same time, because of the collaborative processing of multiple nodes, the resource utilisation of cloud data centre has also been paid more and more attention. In addition, with the increase of data security requirements, some privacy conflict data should avoid being placed on the same or neighbouring nodes. In response to this challenge, based on the fat-tree network topology, this study proposes a data privacy protection-based collaborative placement strategy of workflow and data to jointly optimise the average data access time, the average resource utilisation, and the data conflict degree. Finally, a large number of experimental evaluations and comparative analyses verify the efficiency of the proposed method.

Towards Network-Aware Service Composition in the Cloud

Composing several API-defined services into one composite service per user requirements has become an important service creation approach in the cloud-enabled API economy. Various service selection approaches in support of service composition on demand have been proposed. They usually assume that networking resources are over-provisioned and their usage needs not be considered when making quality-aware service composition decisions. In practice, these approaches often lead to wasteful network resource consumption and impractical end-to-end QoS optimality for cloud-based services. This paper proposes a network-aware cloud service composition approach, named NetMIP, with comparative experimental evaluations for the clouds that adopt the widely deployed fat-tree network topology. By formalizing the service composition goal as a multi-objective constraint optimization problem, we have validated the proposed approach can be used to effectively reduce network resource consumption and deliver QoS optimality while satisfying the end-to-end QoS constraints for the candidate composite services in the cloud. The comparative experimental evaluations are done via a credible cloud infrastructure simulation system, named WebCloudSim. Extensive evaluation results show that NetMIP outperforms several representative cloud service composition approaches in terms of network resource consumption, QoS optimality, and computation time under various service selection workloads and fat-tree network topology settings.

The high-speed networks of the Summit and Sierra supercomputers

Oak Ridge National Laboratory's Summit supercomputer and Lawrence Livermore National Laboratory's Sierra supercomputer utilize InfiniBand interconnect in a Fat-tree network topology, interconnecting all compute nodes, storage nodes, administration, and management nodes into one linearly scalable network. These networks are based on Mellanox 100-Gb/s EDR InfiniBand ConnectX-5 adapters and Switch-IB2 switches, with compute-rack packaging and cooling contributions from IBM. These devices support in-network computing acceleration engines such as Mellanox Scalable Hierarchical Aggregation and Reduction Protocol, graphics processor unit (GPU) Direct RDMA, advanced adaptive routing, Quality of Service, and other network and application acceleration. The overall IBM Spectrum Message Passing Interface (MPI) messaging software stack implements Open MPI, and was a collaboration between IBM, Mellanox, and NVIDIA to optimize direct communication between endpoints, whether compute nodes (with IBM POWER CPUs, NVIDIA GPUs, and flash memory devices), or POWER-hosted storage nodes. The Fat-tree network can isolate traffic among the compute partitions and to/from the storage subsystem, providing more predictable application performance. In addition, the high level of redundancy of this network and its reconfiguration capability ensures reliable high performance even after network component failures. This article details the hardware and software architecture and performance of the networks and describes a number of the high-performance computing (HPC) enhancements engineered into this generation of InfiniBand.

Path2SL: Leveraging InfiniBand Resources to Reduce Head-of-Line Blocking in Fat Trees

The number of endnodes in high-performance computing and datacenter systems is constantly increasing. Hence, it is crucial to minimize the impact of network congestion to guarantee a suitable network performance. InfiniBand is a prominent interconnect technology that allows implementing efficient topologies and routing algorithms, as well as queuing schemes that reduce the head-of-line (HoL) blocking effect derived from congestion situations. Here, we explain and evaluate thoroughly a queuing scheme called Path2SL that optimizes the use of the InfiniBand Virtual Lanes to reduce the HoL blocking in fat-tree network topologies.

Scheduling Live Migration of Virtual Machines

Every day, numerous VMs are migrated inside a datacenter to balance the load, save energy or prepare production servers for maintenance. Although VM placement problems are carefully studied, the underlying migration schedulers rely on vague adhoc models. This leads to unnecessarily long and energy-intensive migrations. We present mVM, a new and extensible migration scheduler. To provide schedules with minimal completion times, mVM parallelizes and sequentializes the migrations with regards to the memory workload and the network topology. mVM is implemented as a plugin of BtrPlace and its current library allows administrators to address temporal and energy concerns. Experiments on a real testbed shows mVM outperforms state-of-the-art migration schedulers. Compared to schedulers that cap the migration parallelism, mVM reduces the individual migration duration by 20.4 percent on average and the schedule completion time by 28.1 percent. In a maintenance operation involving 96 VMs migrated between 72 servers, mVM saves 21.5 percent Joules against BtrPlace. Compared to the migration model inside the cloud simulator CloudSim, the prediction error of the migrations duration is about 5 times lower with mVM. By computing schedules involving thousands of migrations performed over various fat-tree network topologies, we observed that the mVM solving time accounts for about 1 percent of the schedule execution time.

Minimizing Energy Consumption of FatTree Data Center Networks

Many data centers are built using a fat-tree network topology because of its high bisection bandwidth. Therefore thereis a need to develop analytical models for the energy behavior of fat-tree networks and examine strategies to reduce energy consumption. The most effective strategy is to power off entire switches, if possible. In this paper, we derive formulas for the minimum number of active switches needed in a fat-tree data center network for arbitrary types of loading. We also derive expressions for the expected traffic loss when these networks are overloaded with external (Internet) traffic. Results of detailed simulations conducted using well-known traffic models for data center networks [4] closely match our derived formulas. We show that a fat-tree network incurs significant energy cost even when very lightly loaded. In order to further reduce energy consumption, we consolidate traffic into fewer switches and derive expressions for energy cost versus load assuming traffic consolidation and show linear scaling. Finally, we observe that traffic patterns have a significant impact on energy consumption and this fact is evident in the analytical formulas.