High Performance Computing Networks Research Articles

Enabling high reliability via matroidal connectivity and conditional matroidal connectivity on arrangement graph networks

Connectivity indicators are commonly used to evaluate system fault tolerance and reliability. However, with the high demand for multi-processor systems in high-performance computing and data center networks, the number of processors is getting larger, and the network is getting more complex. Thus, traditional connectivity and other indicators are hardly competent in assessing the reliability of complex networks. The matroidal connectivity and conditional matroidal connectivity are novel connectivity metrics that measure the actual fault-tolerant capability based on the constraints of each network dimension. In this paper, we study matroidal connectivity and conditional matroidal connectivity of the (n,k)-arrangement graph network An,k from the natural perspective of the partition of edge dimension and obtain their theoretically accurate values. Moreover, we conduct numerical analysis to compare matroidal connectivity with other conditional edge connectivities in An,k. Additionally, we explore the distribution pattern of edge failure through simulation experiments in An,k and attain the relation of conditional matroidal connectivity related to network scales. Our investigations include two famous network classes: alternating group graphs and star graphs.

Application of high performance computing and deep neural network learning in intelligent city mechanical product recommendation

Intelligent city is a product of the deep integration of information technology, industrialization and urbanization, which has a large number of intelligent mechanical products. The users widely evaluate their application characteristics, and the selection of mechanical products based on user evaluation has become a trend. Nowadays, personalized mechanical product recommendation based on user evaluation is more and more widely used. However, due to the sparse evaluation data, the recommendation accuracy needs to be improved. In this paper, the principle of matrix decomposition is deeply analyzed in order to provide useful ideas for solving this problem. The bias weight hybrid recommendation model of user preference and rating object characteristics is proposed, and the corresponding hybrid recommendation algorithm is designed. First, estimated data obtained using the matrix decomposition principle is supplemented to the sparse data matrix. Secondly, according to the characteristics of users and ratings, initial positions were set based on the statistical distribution of high-performance computing data, and bias weights were set by incorporating each feature. Finally, the nonlinear learning ability of deep neural network learning is used to enhance the classification effectiveness. Practice has proved that the constructed model is reasonable, the designed algorithm converges fast, the recommendation accuracy is improved by about 10%, and the model better alleviates the problem of sparse scoring data. The practical application is simple and convenient, and has good application value.

Hybrid PDES Simulation of HPC Networks Using Zombie Packets

Although high-fidelity network simulations have proven to be reliable and cost-effective tools to peer into architectural questions for high-performance computing (HPC) networks, they incur a high resource cost. The time spent in simulating a single millisecond of network traffic in the highest detail can take hours, even for static, well-behaved traffic patterns such as uniform random. Surrogate models offer a significant reduction in runtime, yet they cannot serve as complete replacements and should only be used when appropriate. Thus, there is a need for hybrid modeling, where high-fidelity simulation and surrogates run side-by-side. We present a surrogate model for HPC networks in which: packets bypass the network, while the network state is left untouched, i.e , suspended. To bypass the network, we use historical data to estimate the arrival time at which every packet should be scheduled at; to suspend the network, all in-flight packets are scheduled to arrive at their destinations, and are kept in the system to awaken as zombies when switching back to high-fidelity. Speedup for a hybrid model is relative to the proportion of surrogate to high-fidelity. This light-weight surrogate obtained up to 76 × speedup. Keeping the zombies in the network showed an increase in the accuracy of the high-fidelity simulation on restart when compared to restarting the network from an empty state.

Research on High-performance Computing Network Search System based on Computer Big Data

An efficient computing system for massively parallel systems is established. The stochastic Petri net abstracts and models the high-performance computer work scheduling system. The IB switch is modeled. Stochastic Petri nets are used for performance analysis. Finally, the proposed method is combined with InfiniBand interconnection architecture to evaluate the system's delay. The experimental results prove the feasibility of this algorithm.

Investigating and Reducing the Architectural Impact of Transient Faults in Special Function Units for GPUs

Ensuring the reliability of GPUs and their internal components is paramount, especially in safety-critical domains like autonomous machines and self-driving cars. These cutting-edge applications heavily rely on GPUs to implement complex algorithms due to their implicit programming flexibility and parallelism, which is crucial for efficient operation. However, as integration technologies advance, there is a growing concern regarding the potential increase in fault sensitivity of the internal components of current GPU generations. In particular, Special Function Unit (SFU) cores inside GPUs are used in multimedia, High-Performance Computing, and neural network training. Despite their frequent usage and critical role in several domains, reliability evaluations on SFUs and the development of effective mitigation solutions have yet to be studied and remain unexplored. This work evaluates the impact of transient faults in the main hardware structures of SFUs in GPUs. In addition, we analyze the main overhead costs and benefits of developing selective-hardening mechanisms for SFUs. We focus on evaluating and analyzing two SFU architectures for GPUs (’fused’ and ’modular’) and their relations to energy, area, and reliability impact on parallel applications. The experiments resort to fine-grain fault injection campaigns on an RTL GPU model (FlexGripPlus) instrumented with both SFUs. The results on both SFU architectures indicate that fused SFUs (in commercial-grade devices) require lower area overhead (about 27%) for their integration in GPUs but are more vulnerable to transient faults (in up to 47% for the analyzed cases) and less power efficient (in up to 36.6%) than modular SFUs. Moreover, the reliability estimation shows that Modular SFUs are structurally more resilient than Fused ones in up to one order of magnitude. Similarly, selective-hardening mechanism based on Triple-Modular Redundancy (TMR) shows that coarse-grain strategies might increase the reliability of the overall SFUs under feasible overhead costs.

Beyond matchings: Dynamic multi-hop topology for demand-aware datacenters

State-of-the-art topologies for datacenters (DC) and high-performance computing (HPC) networks are demand-oblivious and static. Therefore, such network topologies are optimized for the worst-case traffic scenarios. However, recent optical circuit-switching technologies enable real-time dynamic topologies that change in microseconds or less. This capability opens the door for the design of self-adjusting networks: networks with demand-aware and reconfigurable topologies in which links can be re-adjusted online and in response to evolving traffic patterns.In this paper, we study self-adjusting networks using a recently proposed model of reconfigurable networks and present a novel algorithm, GreedyEgoTrees (GET), that dynamically changes the network topology. While previous algorithms used a local perspective, GET takes a global view and greedily builds ego-trees for nodes in the network, where nodes cooperate to help each other. In contrast to recent proposals, GET is optimized for multi-hop routing, and we show that it has nice theoretical guarantees as a function of the demand’s entropy. Empirical results also show that GET outperforms recently proposed algorithms (like static expander and greedy dynamic matching) and can significantly improve the average path length by up-to 65% for real DC, HPC, and other communication traces.

Guest Editorial JQE Special Virtual Issue Dedicated to the 23rd European Conference on Integrated Optics

This Special Issue is associated with the European Conference on Integrated Optics (ECIO), held on May 4–6, 2022, in Milan, Italy. This conference was the 23rd in a series that started in London in 1981. After the virtual edition of 2020 and the cancellation of the 2021 edition due to the COVID-19 pandemic emergency, the 2022 ECIO conference was hosted fully in presence by Politecnico di Milano and was attended by about 250 participants. The scientific sessions of the conference were opened by two plenary speakers, Prof. Volker Sorger of George Washington University, USA, and Prof. Paul Prucnal of Princeton University, USA, who presented recent developments on integrated photonics for machine learning, high-performance computing, and neural networks.

Monitoring InfiniBand Networks to React Efficiently to Congestion

Current high-performance interconnection networks for high-performance computing and data-center systems incorporate mechanisms to prevent congestion from degrading network performance. Specifically, the popular InfiniBand specification defines a mechanism to reduce the injection rate of the traffic flows contributing to congestion. However, the efficiency of this mechanism depends on the values configured for certain parameters, that may be suitable for some congestion situations but not for others. Therefore, we think that these parameters should be reconfigured dynamically, based on accurate and updated information about the actual status of congestion. For that purpose, we have combined a light-weight platform monitoring tool (LIMITLESS) with the InfiniBand control software (OpenSM), so that the former provides the latter with enhanced knowledge about congestion to appropriately reconfigure the parameters driving the behavior of the congestion-control mechanism. Experiments performed in a real InfiniBand-based cluster confirm that this approach significantly reduces the number of wrong reactions to the congestion-control mechanism.

Femtosecond laser welding for robust and low loss optical fiber bonding.

Driven by the increasing demand for faster high-performance computing (HPC) networks and higher data center fabric transmission bandwidth, to favorite the needs of machine learning, data training, and computing, the adoption of co-packaged optics (CPO) and near-packaged optics (NPO) is one of the innovations to mitigate the slowing down of Moore's law. Because of the high temperature generated by the next generation of high-speed chips like switch ASICs, CPUs, and GPUs, coupling fibers to photonic integrated circuit (PIC) with traditional epoxy-based fiber arrays is becoming more challenging and problematic. Therefore, an epoxy-free bonding method using femtosecond laser welding borosilicate glass 3.3 and optical fibers is proposed and demonstrated. Then, a low loss and polarization independent fiber to fiber coupling was demonstrated to show the reliability of bonding. In the experiment, a V groove is used for aligning and positioning two fibers. After welding, the minimum coupling loss and polarization dependent loss is 0.347 dB and below 0.1 dB respectively. The average shear force limit of the welded samples with 0.5 mm welding length is measured to be as high as ∼0.719 N. This technology could be used for epoxy-free based edge coupling the high density multi-fibers with PIC and has potential of scalable manufacturability through automation.

A Low BER DB-PAM4 Adaptive Equalizer for Large Channel Loss in Wireline Receivers

A high-speed serial interface is the core IP of a high-performance computer, data center and interconnection network; its bandwidth and bit error performance restrict the development of the system. With the evolution of high-speed serial interface line rates from 56 G to 112 G in high-end information systems, their bit errors increase sharply, limiting system performance. In order to solve the high-bit-error problem of the 112 Gb/s high-speed serial interface, this paper proposes a low-error Duo-binary PAM4 (DB-PAM) receive equalization technology. This technology utilizes the Duo-binary (DB) signal in which the channel and the transceiving equalizer work together to realize the low-error reception of the 112 Gb/s signal in the high-attenuation channel. To solve the problem of difficulty in generating high-speed DB-PAM4 signals and the complex adjustment of equalization parameters, this paper proposes a two-step progressive equalization technique. In the first step, the technology transmits not-return-to-zero (NRZ) signals at the transmitter (TX) and generates a Duo-binary (DB) signal path at the receiver using the least mean square error (LMS) algorithm. In the second step, the technology sends a precode-PAM4 (pre-PAM4) signal at TX; at the receiving end, the adaptive equalization algorithm is used to adjust the DSP equalization parameters to generate the optimal equalization parameters of the DB-PAM4 signal. This paper uses Cadence’s AMS simulation platform to verify the receive equalizer of DB-PAM4. Simulation results show: when a 112 Gb/s pre-PAM4 signal passes through the 35 dB@28 GHz channel, the receiver (RX) utilizes the adaptive equalizer to generate a 112 Gb/s DB-PAM4 signal, and the receiver bit error rate (BER) is less than 3e-9.

Wide-Sense Nonblocking Multicast in WDM Optical Linear Array and Ring Networks with 3-Length Extension Under Longest Link First Routing

The modern scientific growths in optical networks based on wavelength division multiplexing (WDM) are more attractive to satisfy the high bandwidth requirements of the modern internet infrastructure. Moreover, they potential to satisfy the future bandwidth requirements.WDM optical networks act as the backbone for telecommunication and high-performance communication networks. Multicast communication is the simultaneous transmission of data from one source node to many destination nodes available in the network over wavelength division multiplexing. It is extensively deployed in high performance computing and communication networks. In this article, a linear array and ring networks are extended by directly linking all nodes which are separated by two intermediate nodes with additional fibers which is referred as linear array and ring network with 3-length extension. The wavelength allotment methods are proposed to realize one-to-many communication over wavelength division multiplexing optical linear array and ring with 2-length extension under longest link first routing and the minimum wavelength number needed is determined. The minimum wavelength number needed to support for the extended linear array is reduced approximately by 20% when compared with that of a linear array network and for the extended unidirectional ring is reduced approximately by 25% when compared with that of a unidirectional ring network. And also, for the extended bidirectional ring the minimum wavelength number is reduced approximately by 33% when compared with that of bidirectional ring network.

All-to-all Broadcast in Optical WDM Ring with 2-length Extension and 3-length Extension

All-to-all broadcast communication is to transmit a distinctive data from each node to every other node available in the network. This is a fundamental prerequisite in high performance computing and telecommunication networks including enterprises operating datacenters which may spread across thousands of nodes by means of WDM optical networks. Wavelengths are scarce resources in WDM optical networks. Reducing the wavelengths usage is essential to decrease the price and complication of the network. In this article, a ring network is extended by additionally connecting alternate nodes in order to provide alternate paths so as to reduce the effective number of hops between the communicating nodes and also to reduce the wavelength requirements and this network is referred as a ring network with 2-length extension. Similarly, a ring network is extended by directly linking all nodes which are separated by two intermediate nodes with additional fibers and this network is referred as a ring network with 3-length extension. For the ring network with 2-length extension the optimum wavelength number necessary to establish all-to-all broadcast under unidirectional routing is derived by grouping nonoverlapping connections on a common wavelength. The wavelength number necessary atmost to establish all-to-all broadcast in a unidirectional ring with 3-length extension is derived using longest link first routing algorithm.

Performance trade-offs in reconfigurable networks for HPC

Designing efficient interconnects to support high-bandwidth and low-latency communication is critical toward realizing high performance computing (HPC) and data center (DC) systems in the exascale era. At extreme computing scales, providing the requisite bandwidth through overprovisioning becomes impractical. These challenges have motivated studies exploring reconfigurable network architectures that can adapt to traffic patterns at runtime using optical circuit switching. Despite the plethora of proposed architectures, surprisingly little is known about the relative performances and trade-offs among different reconfigurable network designs. We aim to bridge this gap by tackling two key issues in reconfigurable network design. First, we study how cost, power consumption, network performance, and scalability vary based on optical circuit switch (OCS) placement in the physical topology. Specifically, we consider two classes of reconfigurable architectures: one that places OCSs between top-of-rack (ToR) switches—ToR-reconfigurable networks (TRNs)—and one that places OCSs between pods of racks—pod-reconfigurable networks (PRNs). Second, we tackle the effects of reconfiguration frequency on network performance. Our results, based on network simulations driven by real HPC and DC workloads, show that while TRNs are optimized for low fan-out communication patterns, they are less suited for carrying high fan-out workloads. PRNs exhibit better overall trade-off, capable of performing comparably to a fully non-blocking fat tree for low fan-out workloads, and significantly outperform TRNs for high fan-out communication patterns.

Highly Efficient Multicast over Surface Wave in Hybrid Wireless-Optical On-Chip Networks for IoT HPC

As an ideal platform for Internet of Things (IoT) High Performance Computing (HPC), the Network-on-Chip (NoC) implements the parallelized task processing of Intellectual Property (IP) cores. However, the data interaction of IoT HPC has increased exponentially, resulting in highly consumed energy and bandwidth of the electrical NoC. This problem can be solved by the Optical NoC (ONoC), but the flexibility should be further improved for the multicast message transmission. We design a novel hybrid wireless-optical on-chip network with an effective routing mechanism for the message multicast transmission of IoT HPC. We utilize the wireless single-hop transmission to reduce on-chip communication distance and to forward multicast messages. For the high attenuation on-chip wireless signal, we design an on-chip surface waveguide structure for the surface wave as a more reliable wireless transmission medium. A flexible routing mechanism is also proposed to ensure efficient coordination between optical and wireless layers, thus relieving the pressure of the single-layer message transmission. The simulation results show that, compared with the traditional ONoC using mesh topology, our scheme performs better in terms of average delay, average energy consumption, and network throughput.

Revisiting network congestion avoidance through adaptive packet-chaining reservation

Endpoint congestion is a bottleneck in high-performance computing (HPC) networks, which severely impacts system performance, especially for latency-sensitive applications. When the long messages (or flows) has a far larger duration than the round-trip time (RTT), the proactive or reactive countermeasures, an effective solution to endpoint solution, can control the injection rate within a proper range dynamically. However, many HPC applications produce hybrid traffic (a mix of short and long messages) and are dominated by short messages. Existing proactive congestion avoidance methods face the great challenge of scheduling the rapidly changing traffic caused by these short messages. In this paper, we first propose the Packet-Chaining Reservation Protocol (PCRP), that is, a novel congestion management strategy which leverages the advantages of proactive (scheduling the whole flow) and reactive (scheduling the single packet) congestion avoidance techniques. In fact, it is an eclectic method of scheduling. We select the chaining of packets as a flexible reservation granularity between the whole flow and a single packet. The PCRP allows small flows to be speculatively transmitted without being discarded. It also gives the small flows an appropriate priority based on the detected traffic conditions. The PCRP can make a quick respond to network conditions, effectively avoiding endpoint congestion and reducing the average flow delay. However, PCRP is only suitable for short-flow-dominant traffic, as it performs poorly when facing other traffic. This is because PCRP can starve longer flows and thus introduces the unbearable tail delay. Therefore, we further propose the Packet-Chaining Reservation Protocol with Adaptive Framework (PCRP+), which is a reservation protocol that can flexibly adjust the scheduling strategy according to the network load. The PCRP+ can achieve and maintain low tail delay and desirable universality by adopting the adaptive framework. We conduct extensive experiments to evaluate the PCRP+ and compare it with the Speculative Reservation Protocol (SRP) and Bilateral Flow Reservation Protocol (BFRP), the two most typical proactive reservation-based protocols. Evaluation results demonstrate that our design can reduce the flow latency by an average of 25.71%, 21.21%, and 29.01% for hotspot traffic, uniform traffic, and GPCNeT traffic, respectively.

A Survey of High-Performance Interconnection Networks in High-Performance Computer Systems

High-performance interconnection network is the key to realizing high-speed, collaborative, parallel computing at each node in a high-performance computer system. Its performance and scalability directly affect the performance and scalability of the whole system. With continuous improvements in the performance of high-performance computer systems, the trend in the development of high-performance interconnection networks is mainly reflected in network sizes and network bandwidths. With the slowdown of Moore’s Law, it is necessary to adopt new packaging design technologies to implement high-performance interconnection networks for high-performance computing. This article analyzes the main interconnection networks used by high-performance computer systems in the Top500 list of November 2021, and it elaborates the design of representative, state-of-the-art, high-performance interconnection networks, including NVIDIA InfiniBand, Intel Omni-Path, Cray Slingshot/Aries, and custom or proprietary networks, including Fugaku Tofu, Bull BXI, TH Express, and so forth. This article also comprehensively discusses the latest technologies and trends in this field. In addition, based on the analysis of the challenges faced by high-performance interconnection network design in the post-Moore era and the exascale computing era, this article presents a perspective on high-performance interconnection networks.

Toward higher-radix switches with co-packaged optics for improved network locality in data center and HPC networks [Invited

We study the network locality improvements that can be achieved by using co-packaged optics in data center and high-performance computing (HPC) networks. The increased escape bandwidth offered by co-packaged optics can enable switches with speeds of 51.2 Tb/s and beyond. From a network architecture perspective, the key advantages of introducing co-packaged optics at the switch points include the implementation of large-scale topologies of > 12,000 end points with 4 × higher bisection bandwidth and the reduction of the required number of switches by > 40 % compared with state-of-the-art approaches. From a network operation perspective, improved network locality and faster operation can be achieved since the higher-radix switches can mitigate the impact of network contention. Placing applications under fewer leaf switches reduces the number of packets that cross the spine switches in a leaf-spine topology. The proposed scheme is evaluated via discrete-event simulations: we initially evaluate the network locality properties of the system by using virtual-machine traces from a production data center, and we subsequently quantify the performance improvements by simulating an all-to-all pattern for a variety of message sizes over a number of nodes. The results suggest that co-packaged optics form a promising solution for keeping up with bandwidth scaling in future networks. The virtual-machine analysis shows that large-scale applications can be placed under up to 50% fewer first-level switches, while the network analysis shows speedups of up to 7.1, which translates to execution time reductions of up to 26% and 42.7% for applications with communication ratios of 0.3 and 0.5, respectively.

AMLR: An Adaptive Multi-Level Routing Algorithm for Dragonfly Network

High-radix hierarchical structures, such as the dragonfly, fat-tree, and torus, are cost-effective topologies for high-performance computer (HPC) networks. In these networks, dragonfly outperforms traditional topologies such as fat-tree and torus in cost and scalability. However, network congestion occurs due to the imbalanced traffic pattern, which can lead to degraded performance. The routing algorithm influences the performance of the dragonfly topology in many ways. Routing algorithm can be designed to avoid saturating global or local links, and to avoid deadlock in the network. In this letter, we introduce an adaptive multi-level routing (AMLR) for dragonfly networks. AMLR has three-level routes. By dividing these routes meticulously, all paths of the network can be used more effectively. Traffic between groups will be more balanced. In particular, we propose a congestion control scheme to cooperate with AMLR in the data transmission process. Furthermore, congestion detection and notification are leveraged to identify congested packet and inform the network. Evaluations show that the proposed adaptive multi-level routing and congestion control mechanism can relieve the imbalance between groups in the 100-node dragonfly topology. As a result, AMLR provides 26%, 98%, 78%, and 99% lower latencies, and 13%, 87%, 13%, and 128% higher throughputs compared to the shortest routing under uniform, adv+i, hotspot, and permutation traffic, respectively.

On the performance investigation of a fast optical switches based optical high performance computing infrastructure

Optical networks based on fast optical switches (FOSes) could potentially solve the latency, bandwidth, cost and power consumption challenges in current electrical switches (ESes) based high performance computing (HPC) networks. In this work, we present a novel HPC network which employs distributed FOS interconnecting by removing ES (Firefly). In Firefly, Dragonfly topology is adopted for the inter-group connection of blades, while the intra-group connection of blades is implemented by FOS with fast optical flow control. The Firefly exploits the wavelength, space, and time switching domain with nanoseconds reconfiguration time of the FOS to achieve efficient statistical multiplexing operation. We numerically investigate the Firefly performance with real HPC traffic traces collected by running multiple computing applications in MareNostrum 3 HPC infrastructure with Leaf-Spine architecture. Compared with Leaf Spine architecture, results show that Firefly performs 62.4%, 54%, 68.6%, and 71.8% less latency for the applications Conjugate Gradient (CG), Multi-Grid (MG), Multiple Instruction Lattice Computation (MILC), and Miniature Molecular Dynamics (MINI_MD), respectively. Moreover, Firefly can save 56.4% cost and 65.7% power consumption, respectively, with respect to Leaf-Spine when both support around 10,000 blades.

Analog memristive synapse based on topotactic phase transition for high-performance neuromorphic computing and neural network pruning.

Inspired by the human brain, nonvolatile memories (NVMs)-based neuromorphic computing emerges as a promising paradigm to build power-efficient computing hardware for artificial intelligence. However, existing NVMs still suffer from physically imperfect device characteristics. In this work, a topotactic phase transition random-access memory (TPT-RAM) with a unique diffusive nonvolatile dual mode based on SrCoO x is demonstrated. The reversible phase transition of SrCoO x is well controlled by oxygen ion migrations along the highly ordered oxygen vacancy channels, enabling reproducible analog switching characteristics with reduced variability. Combining density functional theory and kinetic Monte Carlo simulations, the orientation-dependent switching mechanism of TPT-RAM is investigated synergistically. Furthermore, the dual-mode TPT-RAM is used to mimic the selective stabilization of developing synapses and implement neural network pruning, reducing ~84.2% of redundant synapses while improving the image classification accuracy to 99%. Our work points out a new direction to design bioplausible memristive synapses for neuromorphic computing.