High Performance Interconnects Research Articles

DISTRIBUTED HIGH-PERFORMANCE COMPUTING METHODS FOR ACCELERATING DEEP LEARNING TRAINING

This paper comprehensively analyzes distributed high-performance computing methods for accelerating deep learning training. We explore the evolution of distributed computing architectures, including data parallelism, model parallelism, and pipeline parallelism, and their hybrid implementations. The study delves into optimization techniques crucial for large-scale training, such as distributed optimization algorithms, gradient compression, and adaptive learning rate methods. We investigate communication-efficient algorithms, including Ring All Reduce variants and decentralized training approaches, which address the scalability challenges in distributed systems. The research examines hardware acceleration and specialized systems, focusing on GPU clusters, custom AI accelerators, high-performance interconnects, and distributed storage systems optimized for deep learning workloads. Finally, we discuss this field's challenges and future directions, including scalability-efficiency trade-offs, fault tolerance, energy efficiency in large-scale training, and emerging trends like federated learning and neuromorphic computing. Our findings highlight the synergy between advanced algorithms, specialized hardware, and optimized system designs in pushing the boundaries of large-scale deep learning, paving the way for future breakthroughs in artificial intelligence.

Design and conductivity optimization of La0.3Sr0.7TiO3/La0.8Sr0.2MnO3 bi-layer structures as tubular segmented-in-series solid oxide fuel cell interconnect☆

This study focused on meeting the stringent stability requirements of tubular segmented-in-series solid oxide fuel cells (SOFCs) in reducing and oxidizing atmospheres. To address this challenge, a bi-layer perovskite ceramic interconnect was designed by controlling the oxygen partial pressure, because of the strong correlation between the conductivity of strontium-doped lanthanum titanate (LST) and the oxygen partial pressure. The LST powder was prepared using solid-phase and sol-gel methods, and their influence on particle size and sintering behavior was compared. LST/lanthanum strontium manganite (LSM) bi-layer ceramic interconnects with varying thicknesses were fabricated through screen printing and co-sintering. The results demonstrate favorable interfacial bonding and excellent chemical compatibility between the ceramic layers. The conductivity of the bi-layer interconnect exhibits a temperature-dependent behavior, peaking at 550 °C. Simulation calculations and research findings validate that the conductivity of the bi-layer interconnect is determined by the thickness of the LSM layer and the oxygen partial pressure at the interconnect interface. Optimal conductivity is achieved with a bi-layer interconnect consisting of approximately 15 μm of LST and 4 μm of LSM. This can be attributed to the efficient regulation of oxygen partial pressure at the interface, effectively mitigating LSM decomposition caused by low oxygen partial pressure and the subsequent reduction in conductivity. These results provide valuable fundamental data and methodology for the development of high-performance interconnects for tubular segmented-in-series SOFCs.

Methods for Constructing High-Performance Interconnects in System-on-Chip Designs

Methods for Constructing High-Performance Interconnects in System-on-Chip Designs

Effect of the solder conductive particles and substrate widths on the current carrying capability for flex-on-board (FOB) assembly

PurposeThe study aims to ascertain the influence of solder conductive particle types and substrate widths on the current carrying capability of flex-on-board (FOB) assemblies. By comparing Sn58Bi and SAC305 particles and varying substrate widths, the research sought to provide insights into the stability and performance of solder joints under different scenarios, particularly in high-power applications.Design/methodology/approachThe study used a comprehensive design/methodology, encompassing the investigation of solder conductive particle types (Sn58Bi and SAC305) and substrate widths on the current carrying capability of FOB assembly. Stable solder joints were obtained by manipulating the curing speed of anisotropic conductive films for both particle types. Various tests were conducted, including current carrying capability assessments under differing conditions.FindingsThe study revealed that larger substrate widths yielded higher current carrying capability due to increased contact area and reduced contact resistance. Notably, solder joints remained stable beyond the solder melting temperature due to encapsulation by cured epoxy resin. SAC305 solder joints exhibited superior current carrying capability over Sn58Bi in continuous high-voltage conditions. The results emphasized the stability of SAC305 solder joints and their suitability for robust interconnections in high-power FOB assemblies.Originality/valueThis study contributes by offering a comprehensive assessment of the impact of solder particle types and substrate widths on solder joint performance in FOB assemblies. The finding that SAC305 joints outperform Sn58Bi under continuous high-voltage conditions adds significant value. Moreover, the observation of stable solder joints beyond solder melting temperature due to resin encapsulation introduces a novel aspect to solder joint reliability. These insights provide valuable guidance for designing robust and high-performance interconnections in demanding applications.

Efficient Thermal-Stress Coupling Design of Chiplet-Based System with Coaxial TSV Array.

In this research, an efficient thermal-stress coupling design method for a Chiplet-based system with a coaxial through silicon via (CTSV) array is developed by combining the support vector machine (SVM) model and particle swarm optimization algorithm with linear decreasing inertia weight (PSO-LDIW). The complex and irregular relationship between the structural parameters and critical indexes is analyzed by finite element simulation. According to the simulation data, the SVM model is adopted to characterize the relationship between structural parameters and critical indexes of the CTSV array. Based on the desired critical indexes of the CTSV array, the multi-objective evaluation function is established. Afterwards, the structural parameters of the CTSV array are optimized through the PSO-LDIW algorithm. Finally, the effectiveness of the developed method is verified by the finite element simulation. The simulated peak temperature, peak stress of the Chiplet-based system, and peak stress of the copper column (306.16 K, 28.48 MPa, and 25.76 MPa) well agree with the desired targets (310 K, 30 MPa, and 25 MPa). Therefore, the developed thermal-stress coupling design method can effectively design CTSV arrays for manufacturing high-performance interconnect structures applied in Chiplet-based systems.

High-Performance Interconnects With Reduced Far-End Crosstalk for High-Speed ICs and Communication Systems

High-Performance Interconnects With Reduced Far-End Crosstalk for High-Speed ICs and Communication Systems

Monitoring InfiniBand Networks to React Efficiently to Congestion

Current high-performance interconnection networks for high-performance computing and data-center systems incorporate mechanisms to prevent congestion from degrading network performance. Specifically, the popular InfiniBand specification defines a mechanism to reduce the injection rate of the traffic flows contributing to congestion. However, the efficiency of this mechanism depends on the values configured for certain parameters, that may be suitable for some congestion situations but not for others. Therefore, we think that these parameters should be reconfigured dynamically, based on accurate and updated information about the actual status of congestion. For that purpose, we have combined a light-weight platform monitoring tool (LIMITLESS) with the InfiniBand control software (OpenSM), so that the former provides the latter with enhanced knowledge about congestion to appropriately reconfigure the parameters driving the behavior of the congestion-control mechanism. Experiments performed in a real InfiniBand-based cluster confirm that this approach significantly reduces the number of wrong reactions to the congestion-control mechanism.

Fusion splicing of hollow-core to standard single-mode fibers using a gradient-index bridge fiber

High-performance interconnection between hollow-core fiber and conventional solid-core fiber is of great significance for a lot of promising applications of hollow-core fibers. The current problems for high-performance hollow-core fiber/solid-core fiber interconnection mainly involve mode field mismatch of the two fibers and the complex splicing process. Subject of study. We study fusion splicing of anti-resonant hollow-core fiber with low loss (0.52 dB) and conventional single-mode fiber using a convenient graded-index bridge fiber. Purpose of the work. We aim to eliminate the mode field diameter mismatch between anti-resonant hollow-core fiber and single-mode fiber and simplify the mode field diameter adaptation method and process. Method. We employ a convenient graded-index bridge fiber approach, where a gradient index fiber is first spliced with the single-mode fiber, and the mode interference within the graded-index bridge fiber causes periodic enlargement and shrinkage of the mode field diameter. By precise cutting of the graded-index bridge fiber, the mode field diameter can be well controlled and made approximately equal to the mode field diameter of the anti-resonant hollow-core fiber. After mode field diameter adaptation, an optimized arc discharge fusion splicing procedure is applied for the anti-resonant hollow-core fiber/single-mode fiber fusion splicing. Main results. A fusion splicing loss of about 0.52 dB is achieved, which involves nearly 0.42 dB mode-field-diameter-mismatch-induced loss and about 0.1 dB fusion splicing loss that is caused by the deformation of the anti-resonant hollow-core fiber. Practical significance. The developed approach is convenient and cost-effective, which can benefit a lot of applications of anti-resonant hollow-core fibers in future telecommunications, fiber gas lasers as well as fiber-optic sensing.

Combined Distributed Shared-Buffered and Diagonally-Linked Mesh Topology for High-Performance Interconnect.

Networks-on-Chip (NoCs) have become the de-facto on-chip interconnect for multi/manycore systems. A typical NoC router is made up of buffers used to store packets that are unable to advance to their desired destination. However, buffers consume significant power/area and are often underutilized, especially in cases of applications with non-uniform traffic patterns thus leading to performance degradation for such applications. To improve network performance, the Roundabout NoC (R-NoC) concept is considered. R-NoC is inspired by real-life multi-lane traffic roundabouts and consists of lanes that are shared by multiple input/output ports to maximize buffering resource utilization. R-NoC relies on router-internal adaptive routing that decides the lane path based on back pressure. Back pressure makes it possible to assess lane utilization and route packets accordingly. This is made possible thanks to the use of elastic buffers for control flow, a clever type of handshaking in a way similar to asynchronous circuits. Another prominent feature of R-NoC is that internal routing and arbitration are completely distributed which allows for significant freedom in deciding internal router topology and parameters. This work leverages this property and proposes novel yet unexplored configurations for which an in-depth evaluation of corresponding implementations on 45 nm CMOS technology is given. Each configuration is evaluated performance and power-wise on both synthetic and real application traffic. Several R-NoC configurations are identified and demonstrated to provide very significant performance improvements over standard mesh configurations and a typical input-buffered router, without compromising area and power consumption. Exploiting the distributed nature of R-NoC routers, a diagonally-linked configuration is then proposed which incurs moderate area overhead and features yet better performance and energy efficiency.

Congestion management in high-performance interconnection networks using adaptive routing notifications

Congestion management in high-performance interconnection networks using adaptive routing notifications

A Survey of High-Performance Interconnection Networks in High-Performance Computer Systems

High-performance interconnection network is the key to realizing high-speed, collaborative, parallel computing at each node in a high-performance computer system. Its performance and scalability directly affect the performance and scalability of the whole system. With continuous improvements in the performance of high-performance computer systems, the trend in the development of high-performance interconnection networks is mainly reflected in network sizes and network bandwidths. With the slowdown of Moore’s Law, it is necessary to adopt new packaging design technologies to implement high-performance interconnection networks for high-performance computing. This article analyzes the main interconnection networks used by high-performance computer systems in the Top500 list of November 2021, and it elaborates the design of representative, state-of-the-art, high-performance interconnection networks, including NVIDIA InfiniBand, Intel Omni-Path, Cray Slingshot/Aries, and custom or proprietary networks, including Fugaku Tofu, Bull BXI, TH Express, and so forth. This article also comprehensively discusses the latest technologies and trends in this field. In addition, based on the analysis of the challenges faced by high-performance interconnection network design in the post-Moore era and the exascale computing era, this article presents a perspective on high-performance interconnection networks.

Digital Copyright Protection: Focus on Some Relevant Solutions

Copyright protection of digital content is considered a relevant problem of the current Internet since content digitalization and high performance interconnection networks have greatly increased the possibilities to reproduce and distribute digital content. Digital Rights Management (DRM) systems try to prevent the inappropriate or illegal use of copyrighted digital content. They are promoted by the major global media players, but they are also perceived as proprietary solutions that give rise to classic problems of privacy and fair use. On the other hand, watermarking protocols have become a possible solution to the problem of copyright protection. They have evolved during the last decade, and interesting proposals have been designed. This paper first presents current trends concerning the most significant solutions to the problem of copyright protection based on DRM systems and then focuses on the most promising approaches in the field of watermarking protocols. In this regard, the examined protocols are discussed in order to individuate which of them can better represent the right trade-off between opposite goals, such as, for example, security and easy of use, so as to prove that it is possible to implement open solutions compatible with the current web context without resorting to proprietary architectures or impairing the protection of copyrighted digital content.

Providing quality of service in omni-path networks

New hierarchical crossbar switch architectures, such as Omni-Path (OPA) and Cray X2, have appeared to improve packet latency, reduce overall cost and increase fault tolerance of the high-performance interconnection networks in supercomputing and data center systems. These and other interconnect technologies (Infiniband or 40/100 Gigabit Ethernet) include support to provide quality of service (QoS) to the applications. In this paper, we show how this QoS support can be enabled to achieve bandwidth and/or latency differentiation in Omni-Path interconnection networks, as a representative case of hierarchical switches. To do that, three different table-based schedulers are used. We include the description of these schedulers and a comparative study by using the results obtained when we evaluate them with Hiperion, a simulation tool that implements an OPA model.

Heat dissipation of underlying multilayered graphene layers grown on Cu–Ni alloys for high-performance interconnects

Electromigration with Joule heating under high current results in the failure of metal interconnect, which affects the performance and lifetime of electronic devices. Furthermore, advanced semiconductor technologies, such as three-dimensional integrated chips and power devices, which include insulated-gate bipolar transistors and compound semiconductors, require an electrical conducting line with high charge-carrying capacity for effective transmission of electrical signal. In this study, we investigate the effect of underlying graphene layers on the electrical properties of Cu interconnects. Multilayered graphene with a thickness of a few nanometers and a flat surface is synthesized using Cu–Ni alloys. The maximum current capacity of the Cu interconnect is improved by increasing the graphene width to 1.60 × 108 A/cm2, which is approximately seven times greater than that of only Cu. In addition, the wide graphene layer prevents the sudden current drop due to breakage of the Cu interconnect. The results of thermal imaging and simulation reveal the importance of effective heat dissipation for realizing high performance of the interconnect.

Novel coupled modular steel structure and seismic tests on high-performance interconnection

Modular building registers the advanced stage of construction industrialization and the corner-supported steel modular building is the most widely applied. However, its structure, which is integrated by corner interconnections only, incurs the problems of structural redundancy and interconnection-assembly complexity. To this gap, this paper proposes the novel coupled modular steel structure and suitable interconnection, which significantly affects the overall structural response and assembly efficiency. Pseudo static tests of three full-scale cruciform joint specimens are implemented to study the seismic performance of interconnection. The results show that the currently proposed coupled modular steel structure could replace the incompletely-combined group-columns and laminated beams with single structural components to avoid the structural redundancy and reduce the assembly difficulty of interconnections. The proposed interconnection exhibits excellent seismic performance. Hysteretic loops of coupled beam and module beam are “bow” shape and “shuttle” shape, respectively, indicating their good hysteretic performance. Ultimate storey drift angles of coupled beam and module beam could reach 0.08–0.09 rad and 0.06–0.08 rad, respectively, which satisfy the requirement for SMF in ANSI/AISC 341-16 and indicate their excellent rotation capacity. The ductility coefficients of the specimens are 2.27–3.64 except for MC2, indicating their good ductility capacity. Adopting the flush end plate for coupled beam and high strength bolts for vertical splice is recommended for the interconnections to obtain better hysteretic performance and bearing capacity. This research provides brand new design for modular construction and references for future research on global response of coupled modular steel structure.

Evaluation of HPC Acceleration and Interconnect Technologies for High-Throughput Data Acquisition.

Efficient data movement in multi-node systems is a crucial issue at the crossroads of scientific computing, big data, and high-performance computing, impacting demanding data acquisition applications from high-energy physics to astronomy, where dedicated accelerators such as FPGA devices play a key role coupled with high-performance interconnect technologies. Building on the outcome of the RECIPE Horizon 2020 research project, this work evaluates the use of high-bandwidth interconnect standards, namely InfiniBand EDR and HDR, along with remote direct memory access functions for direct exposure of FPGA accelerator memory across a multi-node system. The prototype we present aims at avoiding dedicated network interfaces built in the FPGA accelerator itself, leaving most of the resources for user acceleration and supporting state-of-the-art interconnect technologies. We present the detail of the proposed system and a quantitative evaluation in terms of end-to-end bandwidth as concretely measured with a real-world FPGA-based multi-node HPC workload.

Performance and Area Trade-Off of 3D-Stacked DRAM Based Chip Multiprocessor with Hybrid Interconnect

Nowadays, the number of cores on a chip multiprocessor is growing to increase system performance. However, inadequate on-chip interconnection and memory bandwidth have diminished the potential of these chip multiprocessors. High performance interconnects, 3D-stacked main memory, and large on-chip caches are the architectural parameters used to tackle this issue. For a fixed die size, high performance interconnects, and the 3D-stacked memory fosters the increasing rate of the number of cores on a chip multiprocessor whereas increasing the size of on-chip cache poses a restriction. In this work, we study the trade-off between the performance and overall chip area (evaluated using the number of cores, their types and cache size per core) of the chip multiprocessor for different combinations of the interconnection network, DRAM memory (off-chip or on-chip) and self-adaptive page mapping. Our experiments show that for the base-case (chip multiprocessor with the off-chip DRAM and without hybrid interconnect) architecture and without page mapping technique, to increase the core count for a fixed die size, merely shrinking the cache size degrades the performance. Whereas, reducing the cache size and increasing the chip multiprocessor core count along with our considered target architecture and page mapping technique scale up the performance.

The Case for Replication-Aware Memory-Error Protection in Disaggregated Memory

Disaggregated memory leverages recent technology advances in high-density, byte-addressable non-volatile memory and high-performance interconnects to provide a large memory pool shared across multiple compute nodes. Due to higher memory density, memory errors may become more frequent. Unfortunately, tolerating memory errors through existing memory-error protection techniques becomes impractical due to increasing storage cost. This letter proposes replication-aware memory-error protection to improve storage efficiency of protection in data-centric applications that already rely on memory replication for performance and availability. It lets such applications lower protection storage cost by weakening the protection of each individual replica, but still realize a strong protection target by relying on the collective protection conferred by multiple replicas.

Electrochemical Atomic Layer Deposition and Etching – Enabling Technologies for Atomically-Precise Fabrication of High-Performance Interconnects

Moore’s law drives miniaturization of transistors and interconnects in nano-electronics circuits. As critical interconnect dimensions approach the 1 nm length scale, the semiconductor industry will need revolutionary advances in: (i) novel materials and integrated approaches to fabricate interconnects with lower resistance; and (ii) processing technologies for atomically-precise fabrication of novel materials and structures. In the present talk, we will outline the fundamental aspects of electrochemical atomic layer deposition (e-ALD) and etching (e-ALE) processes that enable atomically-precise fabrication of advanced interconnects. We will highlight the key uniqueness of our e-ALD approach, namely the utilization of underpotentially-deposited Zn adlayers to facilitate e-ALD of metals. This uniqueness extends considerably the range of materials accessible via e-ALD to include emerging interconnect materials beyond Cu, i.e., Pt-group metals, non-noble metals such as Co and its alloys, as well as non-metals. Furthermore, our e-ALD approach enables fabrication of novel Zn-silicate based self-forming barriers, which allow for integrated dual-damascene fabrication approaches with significant reductions in interconnect resistance. Void-free metallization of sub-10 nm features utilizing e-ALD will be demonstrated. Finally, efforts to build a library of materials accessible via e-ALD and its etching analog (e-ALE) will be described, and recent progress in building a multi-scale predictive e-ALD process model will be discussed.

Low loss and high performance interconnection between standard single-mode fiber and antiresonant hollow-core fiber

We demonstrate halving the record-low loss of interconnection between a nested antiresonant nodeless type hollow-core fiber (NANF) and standard single-mode fiber (SMF). The achieved interconnection loss of 0.15 dB is only 0.07 dB above the theoretically-expected minimum loss. We also optimized the interconnection in terms of unwanted cross-coupling into the higher-order modes of the NANF. We achieved cross-coupling as low as −35 dB into the LP_{11} mode (the lowest-loss higher-order mode and thus the most important to eliminate). With the help of simulations, we show that the measured LP_{11} mode coupling is most likely limited by the slightly imperfect symmetry of the manufactured NANF. The coupling cross-talk into the highly-lossy LP_{02} mode (>2000 dB/km in our fiber) was measured to be below −22 dB. Furthermore, we show experimentally that the anti-reflective coating applied to the interconnect interface reduces the insertion loss by 0.15 dB while simultaneously reducing the back-reflection below −40 dB over a 60 nm bandwidth. Finally, we also demonstrated an alternative mode-field adapter to adapt the mode-field size between SMF and NANF, based on thermally-expanded core fibers. This approach enabled us to achieve an interconnection loss of 0.21 dB and cross-coupling of −35 dB into the LP_{11} mode.