Future High-performance Computing Research Articles

Solving Boltzmann optimization problems with deep learning

Decades of exponential scaling in high-performance computing (HPC) efficiency is coming to an end. Transistor-based logic in complementary metal-oxide semiconductor (CMOS) technology is approaching physical limits beyond which further miniaturization will be impossible. Future HPC efficiency gains will necessarily rely on new technologies and paradigms of computing. The Ising model shows particular promise as a future framework for highly energy-efficient computation. Ising systems are able to operate at energies approaching thermodynamic limits for energy consumption of computation. Ising systems can function as both logic and memory. Thus, they have the potential to significantly reduce energy costs inherent to CMOS computing by eliminating costly data movement. The challenge in creating Ising-based hardware is in optimizing useful circuits that produce correct results on fundamentally nondeterministic hardware. The contribution of this paper is a novel machine learning approach, a combination of deep neural networks and random forests, for efficiently solving optimization problems that minimize sources of error in the Ising model. In addition, we provide a process to express a Boltzmann probability optimization problem as a supervised machine learning problem.

Implementation and evaluation of diabatic advection in the Lagrangian transport model MPTRAC 2.6

Abstract. Diabatic transport schemes with hybrid zeta coordinates, which follow isentropes in the stratosphere, are known to greatly improve Lagrangian transport calculations compared to the kinematic approach. However, some Lagrangian transport calculations with a diabatic approach, such as the Chemical Lagrangian Transport Model of the Stratosphere (CLaMS), are not well prepared to run on modern high-performance computing (HPC) architectures. Here, we implemented and evaluated a new diabatic transport scheme in the Massive-Parallel Trajectory Calculations (MPTRAC) model. While MPTRAC can be used either with shared-memory multiprocessing on CPUs or with GPUs to offload computationally intensive calculations, making it flexible for many HPC applications, it has been limited to kinematic trajectories in pressure coordinates. The extended modelling approach now enables the use of either kinematic or diabatic vertical velocities and the coupling of different MPTRAC modules based on pressure or hybrid zeta coordinates. This study focus on the accuracy of the implementation in comparison to the CLaMS model. The evaluation of the new transport scheme in MPTRAC shows that, after 90 d of forward calculations, distributions of air parcels in the upper troposphere and lower stratosphere (UTLS) are almost identical for MPTRAC and CLaMS. No significant bias between the two Lagrangian models was found. Furthermore, after 1 d, internal uncertainties (e.g. due to interpolation or the numerical integration method) in the Lagrangian transport calculations are at least 1 order of magnitude smaller than external uncertainties (e.g. from reanalysis selection or downsampling of ERA5). Differences between trajectories using either CLaMS or MPTRAC are on the order of the combined internal uncertainties within MPTRAC. Since the largest systematic differences are caused by the reanalysis and the vertical velocity (diabatic vs. kinematic), the results support the development efforts for trajectory codes that can access the full resolution of ERA5 in combination with diabatic vertical velocities. This work is part of a larger effort to adapt Lagrangian transport in state-of-the-art models such as CLaMS and MPTRAC to current and future HPC architectures and exascale applications.

EChem: A Notebook Exploration of Quantum Chemistry

The eChem project features an e-book published as a web page (10.30746/978-91-988114-0-7), collecting a repository of Jupyter notebooks developed for the dual purpose of explaining and exploring the theory underlying computational chemistry in a highly interactive manner as well as providing a tutorial-based presentation of the complex workflows needed to simulate embedded molecular systems of real biochemical and/or technical interest. For students ranging from beginners to advanced users, the eChem book is well suited for self-directed learning, but workshops led by experienced instructors and targeting student bodies with specific needs and interests can readily be formed from its components. This has been done by using eChem as the base for a workshop directed toward graduate students learning the theory and practices of quantum chemistry, resulting in very positive assessment of the interactive nature of this framework. The members of the eChem team are engaged in both education and research, and as a mirroring activity, we develop the open-source software upon which this e-book is predominantly based. The overarching vision and goal of our work is to provide a science- and education-enabling software platform for quantum molecular modeling on contemporary and future high-performance computing systems, and to document the resulting development and workflows in the eChem book.

Codesign for Extreme Heterogeneity: Integrating Custom Hardware With Commodity Computing Technology to Support Next-Generation HPC Converged Workloads

The future of high-performance computing (HPC) will be driven by the convergence of physical simulation, artificial intelligence, machine learning, and data science computing capabilities. While computational performance gains afforded by technology scaling, as predicted by Moore's Law, have enabled large-scale HPC system design and deployment using commodity CPU and GPU processing components, emerging technologies will be required to effectively support such converged workloads. These emerging technologies will integrate commodity computing components with custom processing and networking accelerators into ever-more heterogeneous architectures resulting in a diverse ecosystem of industry technology developers, university, and U.S. Government researchers. In this article, we describe efforts at the U.S. Department of Energy's Pacific Northwest National Laboratory to construct an end-to-end codesign framework that lays a groundwork for such an ecosystem, including notable outcomes, remaining challenges, and future opportunities.

Data Locality in High Performance Computing, Big Data, and Converged Systems: An Analysis of the Cutting Edge and a Future System Architecture

Big data has revolutionized science and technology leading to the transformation of our societies. High-performance computing (HPC) provides the necessary computational power for big data analysis using artificial intelligence and methods. Traditionally, HPC and big data had focused on different problem domains and had grown into two different ecosystems. Efforts have been underway for the last few years on bringing the best of both paradigms into HPC and big converged architectures. Designing HPC and big data converged systems is a hard task requiring careful placement of data, analytics, and other computational tasks such that the desired performance is achieved with the least amount of resources. Energy efficiency has become the biggest hurdle in the realization of HPC, big data, and converged systems capable of delivering exascale and beyond performance. Data locality is a key parameter of HPDA system design as moving even a byte costs heavily both in time and energy with an increase in the size of the system. Performance in terms of time and energy are the most important factors for users, particularly energy, due to it being the major hurdle in high-performance system design and the increasing focus on green energy systems due to environmental sustainability. Data locality is a broad term that encapsulates different aspects including bringing computations to data, minimizing data movement by efficient exploitation of cache hierarchies, reducing intra- and inter-node communications, locality-aware process and thread mapping, and in situ and transit data analysis. This paper provides an extensive review of cutting-edge research on data locality in HPC, big data, and converged systems. We review the literature on data locality in HPC, big data, and converged environments and discuss challenges, opportunities, and future directions. Subsequently, using the knowledge gained from this extensive review, we propose a system architecture for future HPC and big data converged systems. To the best of our knowledge, there is no such review on data locality in converged HPC and big data systems.

Computational Challenges for Multi-loop Collider Phenomenology

Precision measurements at the LHC and future colliders require theory predictions with uncertainties at the percent level for many observables. Theory uncertainties due to the perturbative truncation are particularly relevant and must be reduced to fully exploit the physics potential of collider experiments. In recent years the theoretical high energy physics community has made tremendous analytical and numerical advances to address this challenge. In this white paper, we survey state-of-the-art calculations in perturbative quantum field theory for collider phenomenology with a particular focus on the computational requirements at high perturbative orders. We show that these calculations can have specific high-performance-computing (HPC) profiles that should to be taken into account in future HPC resource planning.

Assessing the robustness and scalability of the accelerated pseudo-transient method

Abstract. The development of highly efficient, robust and scalable numerical algorithms lags behind the rapid increase in massive parallelism of modern hardware. We address this challenge with the accelerated pseudo-transient (PT) iterative method and present a physically motivated derivation. We analytically determine optimal iteration parameters for a variety of basic physical processes and confirm the validity of theoretical predictions with numerical experiments. We provide an efficient numerical implementation of PT solvers on graphical processing units (GPUs) using the Julia language. We achieve a parallel efficiency of more than 96 % on 2197 GPUs in distributed-memory parallelisation weak-scaling benchmarks. The 2197 GPUs allow for unprecedented tera-scale solutions of 3D variable viscosity Stokes flow on 49953 grid cells involving over 1.2 trillion degrees of freedom (DoFs). We verify the robustness of the method by handling contrasts up to 9 orders of magnitude in material parameters such as viscosity and arbitrary distribution of viscous inclusions for different flow configurations. Moreover, we show that this method is well suited to tackle strongly nonlinear problems such as shear-banding in a visco-elasto-plastic medium. A GPU-based implementation can outperform direct-iterative solvers based on central processing units (CPUs) in terms of wall time, even at relatively low spatial resolution. We additionally motivate the accessibility of the method by its conciseness, flexibility, physically motivated derivation and ease of implementation. This solution strategy thus has a great potential for future high-performance computing (HPC) applications, and for paving the road to exascale in the geosciences and beyond.

A 16-Bit Parallel Prefix Carry Look-Ahead Kogge-Stone Adder Implemented in Adiabatic Quantum-Flux-Parametron Logic

Extremely energy-efficient logic devices are required for future low-power high-performance computing systems. Superconductor electronic technology has a number of energy-efficient logic families. Among them is the adiabatic quantum-flux-parametron (AQFP) logic family, which adiabatically switches the quantum-flux-parametron (QFP) circuit when it is excited by an AC power-clock. When compared to state-of-the-art CMOS technology, AQFP logic circuits have the advantage of relatively fast clock rates (5 GHz to 10 GHz) and 5 - 6 orders of magnitude reduction in energy before cooling overhead. We have been developing extremely energy-efficient computing processor components using the AQFP. The adder is the most basic computational unit and is important in the development of a processor. In this work, we designed and measured a 16-bit parallel prefix carry look-ahead Kogge-Stone adder (KSA). We fabricated the circuit using the AIST 10 kA/cm2 High-speed STandard Process (HSTP). Due to a malfunction in the measurement system, we were not able to confirm the complete operation of the circuit at the low frequency of 100 kHz in liquid He, but we confirmed that the outputs that we did observe are correct for two types of tests: (1) critical tests and (2) 110 random input tests in total. The operation margin of the circuit is wide, and we did not observe any calculation errors during measurement.

A scalability study of the Ice-sheet and Sea-level System Model (ISSM, version 4.18)

Abstract. Accurately modelling the contribution of Greenland and Antarctica to sea level rise requires solving partial differential equations at a high spatial resolution. In this paper, we discuss the scaling of the Ice-sheet and Sea-level System Model (ISSM) applied to the Greenland Ice Sheet with horizontal grid resolutions varying between 10 and 0.25 km. The model setup used as benchmark problem comprises a variety of modules with different levels of complexity and computational demands. The core builds the so-called stress balance module, which uses the higher-order approximation (or Blatter–Pattyn) of the Stokes equations, including free surface and ice-front evolution as well as thermodynamics in form of an enthalpy balance, and a mesh of linear prismatic finite elements, to compute the ice flow. We develop a detailed user-oriented, yet low-overhead, performance instrumentation tailored to the requirements of Earth system models and run scaling tests up to 6144 Message Passing Interface (MPI) processes. The results show that the computation of the Greenland model scales overall well up to 3072 MPI processes but is eventually slowed down by matrix assembly, the output handling and lower-dimensional problems that employ lower numbers of unknowns per MPI process. We also discuss improvements of the scaling and identify further improvements needed for climate research. The instrumented version of ISSM thus not only identifies potential performance bottlenecks that were not present at lower core counts but also provides the capability to continually monitor the performance of ISSM code basis. This is of long-term significance as the overall performance of ISSM model depends on the subtle interplay between algorithms, their implementation, underlying libraries, compilers, runtime systems and hardware characteristics, all of which are in a constant state of flux. We believe that future large-scale high-performance computing (HPC) systems will continue to employ the MPI-based programming paradigm on the road to exascale. Our scaling study pertains to a particular modelling setup available within ISSM and does not address accelerator techniques such as the use of vector units or GPUs. However, with 6144 MPI processes, we identified issues that need to be addressed in order to improve the ability of the ISSM code base to take advantage of upcoming systems that will require scaling to even higher numbers of MPI processes.

A power constrained approximate multiplier with a high level of configurability

Approximate computing helps tackle the challenges of future embedded and high-performance computing by using various methodologies. By increasing the volume of computations and limiting the power consumption, approximate computing can address these challenges in various demands. Approximate computing aims to achieve acceptable accuracy, rather than exact and correct results and can be used in error-tolerant applications to reduce hardware resources, delay, and most importantly power consumption. This paper introduces a new approximate multiplier, with a high degree of configurability, for unsigned numbers. It aims to reduce all hardware metrics besides keeping high accuracy. The proposed method offers well-optimized options in a power-accuracy tradeoff compared to the other configurable algorithm instances. In addition, it provides a wide range of options with power saving from 35% to 85% to satisfy most applications with a desirable power budget. Furthermore, the proposed approximate multiplier has been employed in Discrete Cosine Transform (DCT) applications.

Estimating the overhead and coupling of scientific computing clusters

Since simulation became the third pillar of scientific research, several forms of computers have become available to drive computer aided simulations, and nowadays, clusters are the most popular type of computers supporting these tasks. For instance, cluster settings, such as the so-called supercomputers, cluster of workstations (COW), cluster of desktops (COD), and cluster of virtual machines (COV) have been considered in literature to embrace a variety of scientific applications. However, those scientific applications categorized as high-performance computing (HPC) are conceptually restricted to be addressed only by supercomputers. In this aspect, we introduce the notions of cluster overhead and cluster coupling to assess the capacity of non-HPC systems to handle HPC applications. We also compare the cluster overhead with an existing measure of overhead in computing systems, the total parallel overhead, to explain the correctness of our methodology. The evaluation of capacity considers the seven dwarfs of scientific computing, which are well-known, scientific computing building blocks considered in the development of HPC applications. The evaluation of these building blocks provides insights regarding the strengths and weaknesses of non-HPC systems to deal with future HPC applications developed with one or a combination of these algorithmic building blocks.

Future of HPC: The Internet of Workflows

Driven by convergence with artificial intelligence and data analytics, increased heterogeneity, and a hybrid cloud/on-premise delivery model, dynamic composition of workflows will be a key design criteria of future high-performance computing (HPC) systems. While tightly coupled HPC workloads will continue to execute on dedicated supercomputers, other jobs will run elsewhere, including public clouds, and at the edge. Connecting these distributed computing tasks into coherent applications that can perform at scale is what we call “Internet of Workflows.”

ARENA: Asynchronous Reconfigurable Accelerator Ring to Enable Data-Centric Parallel Computing

The next generation HPC and data centers are likely to be reconfigurable and data-centric due to the trend of hardware specialization and the emergence of data-driven applications. In this article, we propose ARENA - an asynchronous reconfigurable accelerator ring architecture as a potential scenario on how the future HPC and data centers will be like. Despite using the coarse-grained reconfigurable arrays (CGRAs) as the substrate platform, our key contribution is not only the CGRA-cluster design itself, but also the ensemble of a new architecture and programming model that enables asynchronous tasking across a cluster of reconfigurable nodes, so as to bring specialized computation to the data rather than the reverse. We presume distributed data storage without asserting any prior knowledge on the data distribution. Hardware specialization occurs at runtime when a task finds the majority of data it requires are available at the present node. In other words, we dynamically generate specialized CGRA accelerators where the data reside. The asynchronous tasking for bringing computation to data is achieved by circulating the task token, which describes the dataflow graphs to be executed for a task, among the CGRA cluster connected by a fast ring network. Evaluations on a set of HPC and data-driven applications across different domains show that ARENA can provide better parallel scalability with reduced data movement (53.9 percent). Compared with contemporary compute-centric parallel models, ARENA can bring on average 4.37× speedup. The synthesized CGRAs and their task-dispatchers only occupy 2.93mm$^2$2<; inline-graphic xlink:href="tan-ieq1-3081074.gif"/> chip area under 45nm process technology and can run at 800MHz with on average 759.8mW power consumption. ARENA also supports the concurrent execution of multi-applications, offering ideal architectural support for future high-performance parallel computing and data analytics systems.

ENABLE HIGH PERFORMANCE COMPUTING IN CLOUD: A REVIEW

High Performance Computing (HPC) resulting whole computing power in a way that delivers much higher performance than it could get in typical desktop computer or workstation. High Performance Computing (HPC) allows scientists and engineers to solve complex science, engineering, and business problems using applications that require high bandwidth, enhanced networking, and very high compute capabilities. HPC's democratization has been driven particularly by cloud computing, which has given scientists access to supercomputing-like features as the pay as you go. This paper will provide an overview on benets, challenges and future of HPC in cloud.

Energy Efficiency and Yield Optimization for Optical Interconnects via Transceiver Grouping

Optical interconnects enabled by silicon microring-based transceivers offer great potential for short-reach data communication in future high-performance computing systems. However, microring resonators are prone to process variations that harm both the energy efficiency and the yield of the fabricated transceivers. Especially in the application scenario where a batch of transceivers are fabricated for assembling multiple optical networks, how the transceivers are mixed and matched can directly impact the average energy efficiency and the yield of the networks assembled. In this study, we propose transceiver grouping for assembling communication networks from a pool of fabricated transceivers, aiming to optimize the network energy efficiency and the yield. We evaluated our grouping algorithms by wafer-scale measurement data of microring-based transceivers, as well as synthetic data generated based on an experimentally validated variation model. Our experimental results demonstrate that optimized grouping achieves significant improvement in the network energy efficiency and the yield across a wide range of network configurations, compared to a baseline strategy that randomly groups the transceivers.

Empirical investigation: performance and power‐consumption based dual‐level model for exascale computing systems

Exascale computing systems (ECS) are anticipated to perform at Exaflop speed (1018 operations per second) using power consumption <20 MW. This ultrascale performance requires the speedup in the system by thousand-fold enhancement in current Petascale. For future high-performance computing (HPC), power consumption is one of the vital challenges faced to achieve Exaflops through the traditional way of increasing clock-speed. One standard way to attain such significant performance is through massive parallelism. In the early stages, it is hard to decide the promising parallel programming approach that can provide massive parallelism to attain ExaFlops. This article commences with a short description and implementation of algorithms of various hybrid parallel programming models (PPMs) for homogeneous and heterogeneous cluster systems. Furthermore, the authors evaluated performance and power consumption in these hybrid models by implementing in two HPC benchmarking applications such as square matrix multiplication and Jacobi iterative solver for two-dimensional Laplace equation. The results demonstrated that the hybrid of heterogeneous (MPI + X) outperformed to homogeneous parallel programming (MPI + OpenMP) model. This empirical investigation of hybrid PPMs is a leading step for researchers and development communities to select a promising model for emerging ECS.

Pass-efficient methods for compression of high-dimensional turbulent flow data

The future of high-performance computing, specifically on future Exascale computers, will presumably see memory capacity and bandwidth fail to keep pace with data generated, for instance, from massively parallel partial differential equation (PDE) systems. Current strategies proposed to address this bottleneck entail the omission of large fractions of data, as well as the incorporation of in situ compression algorithms to avoid overuse of memory. To ensure that post-processing operations are successful, this must be done in a way that a sufficiently accurate representation of the solution is stored. Moreover, in situations where the input/output system becomes a bottleneck in analysis, visualization, etc., or the execution of the PDE solver is expensive, the number of passes made over the data must be minimized. In the interest of addressing this problem, this work focuses on the utility of pass-efficient, parallelizable, low-rank, matrix decomposition methods in compressing high-dimensional simulation data from turbulent flows. A particular emphasis is placed on using coarse representation of the data – compatible with the PDE discretization grid – to accelerate the construction of the low-rank factorization. This includes the presentation of a novel single-pass matrix decomposition algorithm for computing the so-called interpolative decomposition. The methods are described extensively and numerical experiments on two turbulent channel flow data are performed. In the first (unladen) channel flow case, compression factors exceeding 400 are achieved while maintaining accuracy with respect to first- and second-order flow statistics. In the particle-laden case, compression factors of 100 are achieved and the compressed data is used to recover particle velocities. These results show that these compression methods can enable efficient computation of various quantities of interest in both the carrier and disperse phases.

All-Optical 2 $ \times $ 2-Bit Multiplier at 40 Gb/s Based on Canonical Logic Units-based Programmable Logic Array (CLUs-PLA)

Multiplication is an important function of logic operation, and all-optical high-speed multiplication logic operation will lay the foundation for future high-speed optical computing and optical logic processing chip. In this article, by introducing the structure of canonical logic units-based programmable logic array (CLUs-PLA), we propose a scheme to realize all-optical 2 × 2-bit multiplier. In our scheme, different types of CLUs are generated using bidirectional multichannel four-wave mixing (FWM), then the results of multiplier at the operation of 40 Gb/s can be obtained by simple power coupling of corresponding CLUs. Eye diagrams of logic results are widely open, and the extinction ratios are more than 9.4 dB. Comparing with multiplier based on traditional hierarchical computing, multiplier based on parallel computing in our scheme can reduce the number of AND gate by 4, and avoid further deterioration of signal quality due to three-order cascade of AND gate. Moreover, the scheme has the potential to realize m × n-bit (m + n ≤ 9, m and n are positive integers) multiplier at higher operation rate in the integrated platform, paving the way towards multi-bit high-speed compact complex logic devices for future high-performance optical computing and optical logic processing chip.

High performance reconfigurable computing for numerical simulation and deep learning

Due to their customizable on-chip resources, reconfigurable computing platforms such as FPGAs are able to achieve better time-to-solution and energy-to-solution than general-purpose processors. They have been widely adopted in many important applications, from traditional numerical processing to emerging deep learning systems. Since FPGAs have become promising options for current and future high performance computing, this report summarises and analyses recent FPGA-related efforts, including the latest industrial approaches, the state-of-the-art reconfigurable solutions, and various issues such as on-chip resources and development productivity.

Applying the swept rule for solving explicit partial differential equations on heterogeneous computing systems

Applications that exploit the architectural details of high-performance computing (HPC) systems have become increasingly invaluable in academia and industry over the past two decades. The most important hardware development of the last decade in HPC has been the general purpose graphics processing unit (GPGPU), a class of massively parallel devices that now contributes the majority of computational power in the top 500 supercomputers. As these systems grow, small costs such as latency—due to the fixed cost of memory accesses and communication—accumulate in a large simulation and become a significant barrier to performance. The swept time-space decomposition rule is a communication-avoiding technique for time-stepping stencil update formulas that attempts to reduce latency costs. This work extends the swept rule by targeting heterogeneous, CPU/GPU architectures representing current and future HPC systems. We compare our approach to a naive decomposition scheme with two test equations using an MPI+CUDA pattern on 40 processes over two nodes containing one GPU. The swept rule produces a factor of 1.9 to 23 speedup for the heat equation and a factor of 1.1 to 2.0 speedup for the Euler equations, using the same processors and work distribution, and with the best possible configurations. These results show the potential effectiveness of the swept rule for different equations and numerical schemes on massively parallel compute systems that incur substantial latency costs.