Supercomputer Architectures Research Articles

Scalability as a Key Property of Mapping Computational Tasks to Supercomputer Architecture

When solving complex computational problems on modern supercomputers, an increasingly important role is played by the scalability property, which characterizes the ability of applications to adapt to various degrees of parallelism of computing systems.

JUWELS Cluster and Booster: Exascale Pathfinder with Modular Supercomputing Architecture at Juelich Supercomputing Centre

JUWELS is a multi-petaflop modular supercomputer operated by Juelich Supercomputing Centre at Forschungszentrum Juelich as a European and national supercomputing resource for the Gauss Centre for Supercomputing. In addition, JUWELS serves the Earth system modeling community and the AI community within the Helmholtz Association as well. JUWELS currently consists of two modules. The first module deployed in 2018 is the so-called Cluster module. The Cluster is a BullSequana X1000 system with Intel Xeon Skylake-SP processors and Mellanox EDR InfiniBand. The second module deployed in 2020 is the so-called Booster module. The Booster is a BullSequana XH2000 system with 2nd generation AMD EPYC processors, NVIDIA Ampere GPUs and NVIDIA/Mellanox HDR Infiniband. This paper describes in detail the architecture of the system from a users perspective, and additionally provides further insights into the administrative infrastructure used to operate the supercomputer.

Redesigning Vina@QNLM for Ultra-Large-Scale Molecular Docking and Screening on a Sunway Supercomputer.

Ultra-large-scale molecular docking can improve the accuracy of lead compounds in drug discovery. In this study, we developed a molecular docking piece of software, Vina@QNLM, which can use more than 4,80,000 parallel processes to search for potential lead compounds from hundreds of millions of compounds. We proposed a task scheduling mechanism for large-scale parallelism based on Vinardo and Sunway supercomputer architecture. Then, we readopted the core docking algorithm to incorporate the full advantage of the heterogeneous multicore processor architecture in intensive computing. We successfully expanded it to 10, 465, 065 cores (1,61,001 management process elements and 0, 465, 065 computing process elements), with a strong scalability of 55.92%. To the best of our knowledge, this is the first time that 10 million cores are used for molecular docking on Sunway. The introduction of the heterogeneous multicore processor architecture achieved the best speedup, which is 11x more than that of the management process element of Sunway. The performance of Vina@QNLM was comprehensively evaluated using the CASF-2013 and CASF-2016 protein–ligand benchmarks, and the screening power was the highest out of the 27 pieces of software tested in the CASF-2013 benchmark. In some existing applications, we used Vina@QNLM to dock more than 10 million molecules to nine rigid proteins related to SARS-CoV-2 within 8.5 h on 10 million cores. We also developed a platform for the general public to use the software.

Leveraging HPC accelerator architectures with modern techniques \u2014 hydrologic modeling on GPUs with ParFlow

Rapidly changing heterogeneous supercomputer architectures pose a great challenge to many scientific communities trying to leverage the latest technology in high-performance computing. Many existing projects with a long development history have resulted in a large amount of code that is not directly compatible with the latest accelerator architectures. Furthermore, due to limited resources of scientific institutions, developing and maintaining architecture-specific ports is generally unsustainable. In order to adapt to modern accelerator architectures, many projects rely on directive-based programming models or build the codebase tightly around a third-party domain-specific language or library. This introduces external dependencies out of control of the project. The presented paper tackles the issue by proposing a lightweight application-side adaptor layer for compute kernels and memory management resulting in a versatile and inexpensive adaptation of new accelerator architectures with little draw backs. A widely used hydrologic model demonstrates that such an approach pursued more than 20 years ago is still paying off with modern accelerator architectures as demonstrated by a very significant performance gain from NVIDIA A100 GPUs, high developer productivity, and minimally invasive implementation; all while the codebase is kept well maintainable in the long-term.

MFIX-Exa: A path toward exascale CFD-DEM simulations

MFIX-Exa is a computational fluid dynamics–discrete element model (CFD-DEM) code designed to run efficiently on current and next-generation supercomputing architectures. MFIX-Exa combines the CFD-DEM expertise embodied in the MFIX code—which was developed at NETL and is used widely in academia and industry—with the modern software framework, AMReX, developed at LBNL. The fundamental physics models follow those of the original MFIX, but the combination of new algorithmic approaches and a new software infrastructure will enable MFIX-Exa to leverage future exascale machines to optimize the modeling and design of multiphase chemical reactors.

Sunway supercomputer architecture towards exascale computing: analysis and practice

Sunway supercomputer architecture towards exascale computing: analysis and practice

VGL: a high-performance graph processing framework for the NEC SX-Aurora TSUBASA vector architecture

Developing efficient graph algorithms implementations is an extremely important problem of modern computer science, since graphs are frequently used in various real-world applications. Graph algorithms typically belong to the data-intensive class, and thus using architectures with high-bandwidth memory potentially allows to solve many graph problems significantly faster compared to modern multicore CPUs. Among other supercomputer architectures, vector systems, such as the SX family of NEC vector supercomputers, are equipped with high-bandwidth memory. However, the highly irregular structure of many real-world graphs makes it extremely challenging to implement graph algorithms on vector systems, since these implementations are usually bulky and complicated, and a deep understanding of vector architectures hardware features is required. This paper presents the world first attempt to develop an efficient and simultaneously simple graph processing framework for modern vector systems. Our vector graph library (VGL) framework targets NEC SX-Aurora TSUBASA as a primary vector architecture and provides relatively simple computational and data abstractions. These abstractions incorporate many vector-oriented optimization strategies into a high-level programming model, allowing quick implementation of new graph algorithms with a small amount of code and minimal knowledge about features of vector systems. In this paper, we evaluate the VGL performance on four widely used graph processing problems: breadth-first search, single source shortest paths, connected components, and page rank. The provided comparative performance analysis demonstrates that the VGL-based implementations achieve significant acceleration over the existing high-performance frameworks and libraries: up to 14 times speedup over multicore CPUs (Ligra, Galois, GAPBS) and up to 3 times speedup compared to NVIDIA GPU (Gunrock, NVGRAPH) implementations.

A New Optoelectronic Hybrid Network Based on Scheduling Optimization of Optical Links

The emergence of exascale computers will represent a milestone in high-performance computing (HPC). Optoelectronic interconnections and configurable switches will change the traditional supercomputer architecture. However, new hardware is not easily adapted to dynamic running conditions. Based on scheduling optimization of optical links, we propose a new optoelectronic hybrid network, the software-defined network accelerator (sDNA), for an exascale computer. Our scheduling optimization contains an optical interconnection method and an adaptive routing method. The main contribution of our work is an extended edge forwarding index (E-EFI) optical interconnection method based on slow-switching optical devices. The optical link connections are established by evaluating the traffic offloading revenue for each optical link candidate. To support optical interconnection, sDNA selects a suitable routing strategy according to the job-schedule information and prior HPC application knowledge. We tested sDNA in a network simulator and a prototype exascale computer system using both the US Department of Energy (DOE) application and real-world communication benchmarks. The verification results for traffic offloading reveal that our optical interconnection method not only offloads traffic from electrical links to optical links but also avoids the congestion inherent to electrical links. sDNA maintains a throughput of more than 80 percent bandwidth and reduces the communication delay by 10 percent in our real prototype system and simulator. Thus, sDNA is an ideal candidate for accelerating the communication performance of exascale computers.

Building ontologies for solving compute-intensive problems

The aim of the study, results of which are presented in this paper, is to analyse methods and tools for constructing an ontology related to solving compute-intensive problems and to form algorithms of its use. This problem arises from the need to solve problems using modern and future supercomputers, containing millions and, in the long term, billions of simultaneously operating computing cores and having a huge degree of parallelism. In solving such problems, the researcher should be well versed in both computational methods for solving the problem and modern supercomputer technologies, which is not always the case. One of the solutions to this problem is the creation of a knowledge base that includes ontological descriptions of methods for solving compute-intensive problems and architectures of supercomputers that can be used to solve them. The development of ontologies for a given subject area is one of the most important stage in creating an intelligent support system for solving specific compute-intensive problems. The paper discusses the methods and tools that are used to build the ontology. The paper also presents examples of the development of ontologies for astrophysics and geophysics problems.

Algorithm for replica redistribution in an implementation of the population annealing method on a hybrid supercomputer architecture

The population annealing method is a promising approach for large-scale simulations because it is potentially scalable on any parallel architecture. We present an implementation of the algorithm on a hybrid program architecture combining CUDA and MPI. The problem is to keep all general-purpose graphics processing unit devices as busy as possible by efficiently redistributing replicas. We provide details of testing on hardware based the Intel Skylake/Nvidia V100 running more than two million replicas of the Ising model sample in parallel. The results are quite encouraging because the acceleration grows toward the perfect line as the complexity of the simulated system increases.

ArborX

Searching for geometric objects that are close in space is a fundamental component of many applications. The performance of search algorithms comes to the forefront as the size of a problem increases both in terms of total object count as well as in the total number of search queries performed. Scientific applications requiring modern leadership-class supercomputers also pose an additional requirement of performance portability, i.e., being able to efficiently utilize a variety of hardware architectures. In this article, we introduce a new open-source C++ search library, ArborX, which we have designed for modern supercomputing architectures. We examine scalable search algorithms with a focus on performance, including a highly efficient parallel bounding volume hierarchy implementation, and propose a flexible interface making it easy to integrate with existing applications. We demonstrate the performance portability of ArborX on multi-core CPUs and GPUs and compare it to the state-of-the-art libraries such as Boost.Geometry.Index and nanoflann.

On improving the efficiency of mathematical modeling of the problem of stability of construction

Algorithmic software for mathematical modeling of structural stability is considered, which is reduced to solving a partial generalized eigenvalues problem of sparse matrices, with automatic parallelization of calculations on modern parallel computers with graphics processors. Peculiarities of realization of parallel algorithms for different structures of sparse matrices are presented. The times of solving the problem of stability of composite materialsusing a three-dimensional model of "finite size fibers" on computers of different architectures are given. In mathematical modeling of physical and technical processes in many cases there is a need to solve problems of algebraic problem of eigenvalues (APVZ) with sparse matrices of large volumes. In particular, such problems arise in the analysis of the strength of structures in civil and industrial construction, aircraft construction, electric welding, etc. The solving to these problems is to determine the eigenvalues and eigenvectors of sparse matrices of different structure. The efficiency of solving these problems largely depends on the effectiveness of mathematical modeling of the problem as a whole. Continuous growth of task parameters, calculation of more complete models of objects and processes on computers require an increase in computer productivity. High-performance computing requirements are far ahead of traditional parallel computing, even with multicore processors. High-performance computing requirements are far ahead of traditional parallel computing, even with multicore processors. Today, this problem is solved by using powerful supercomputers of hybrid architecture, such as computers with multicore processors (CPUs) and graphics processors (GPUs), which combine MIMD and SIMD architectures. But the potential of high-performance computers can be used to the fullest only with algorithmic software that takes into account both the properties of the task and the features of the hybrid architecture. Complicating the architecture of modern high-performance supercomputers of hybrid architecture, which are actively used for mathematical modeling (increasing the number of computer processors and cores, different types of computer memory, different programming technologies, etc.) means a significant complication of efficient use of these resources in creating parallel algorithms and programs. here are problems with the creation of algorithmic software with automatic execution of stages of work, which are associated with the efficient use of computing resources, ways to store and process sparse matrices, analysis of the reliability of computer results. This makes it possible to significantly increase the efficiency of mathematical modeling of practical problems on modern high-performance computers, as well as free users from the problems of parallelization of complex problems. he developed algorithmic software automatically implements all stages of parallel computing and processing of sparse matrices on a hybrid computer. It was used at the Institute of Mechanics named after S.P. Tymoshenko NAS of Ukraine in modeling the strength problems of composite material. A significant improvement in the time characteristics of mathematical modeling was obtained. Problems of mathematical modeling of the properties of composite materials has an important role in designing the processes of deformation and destruction of products in various subject areas. Algorithmic software for mathematical modeling of structural stability is considered, which is reduced to solving a partial generalized problem of eigen values of sparse matrices of different structure of large orders, with automatic parallelization of calculations on modern parallel computers with graphics processors. The main methodological principles and features of implementation of parallel algorithms for different structures of sparse matrices are presented, which ensure effective implementation of multilevel parallelism of a hybrid system and reduce data exchange time during the computational process. As an example of these approaches, a hybrid algorithm of the iteration method in subspace for tape and block-diagonal matrices with a frame for computers of hybrid architecture is given. Peculiarities of data decomposition for matrices of profile structure at realization of parallel algorithms are considered. The proposed approach provides automatic determination of the required topology of the hybrid computer and the optimal amount of resources for the organization of an efficient computational process. The results of testing the developed algorithmic software for problems from the collection of the University of Florida, as well as the times of solving the problem of stability of composite materials using a three-dimensional model of "finite size fibers" on computers of different architectures. The results show a significant improvement in the time characteristics of solving problems.

GPGPU-based randomized visual secret sharing (GRVSS) for grayscale and colour images

Visual Secret Sharing (VSS) is a technique used for sharing secret images between users. The existing VSS schemes reconstruct the original secret image as a halftone image with only a 50% contrast. The Randomized Visual Secret Sharing (RVSS) scheme overcomes the disadvantages of existing VSS schemes. Although RVSS extracts the secret image with better contrast, it is computationally expensive. This paper proposes a General Purpose Graphics Processing Unit (GPGPU)-based Randomized Visual Secret Sharing (GRVSS) technique that leverages data parallelism in the RVSS pipeline. The performance of the GRVSS is compared with the RVSS in a generic and PARAM Shavak supercomputer architecture. The GRVSS outperforms the RVSS in both architectures.

Content of ontology for solving compute-intensive problems of the cosmic plasma hydrodynamics

Modern mathematical modeling on supercomputer systems requires that researchers have both a view of the main aspects of a studied phenomena and a knowledge base of numerical methods and supercomputer technologies for solving natural science problems. Development of intelligent support systems for solving such problems can greatly facilitate the researcher work by providing convenient tools for constructing an efficient scheme of problem solution on a target computational architecture and, if possible, implementing it using already developed program codes.This paper describes an implementation of this idea for the compute-intensive problems of the cosmic plasma hydrodynamics. A knowledge base has been developed, including ontological descriptions of studied astrophysical phenomena, their mathematical representations, numerical methods for solving set problems and supercomputer architectures that can be used for calculations. The paper presents patterns describing the main classes of the constructed ontology and their instances. The ontology can be expanded by a generation system of computational codes for the selected schemes of a problem solution.

Toward a scalable robust security-constrained optimal power flow using a proximal projection bundle method

Robust security-constrained optimal power flow (rSCOPF) aims to find the worst-case contingencies of alternating current optimal power flow (ACOPF) in power systems. With the rise of GPU architectures on the upcoming supercomputer architectures, optimization algorithms that rely on sparse linear algebra and indefinite linear systems are becoming increasingly hard to solve efficiently (e.g. interior-point method). To address this we revisit a maximin optimization formulation of the rSCOPF and the single-level mixed-integer semidefinite programming (MISDP) reformulation, which is obtained by taking the Lagrangian relaxation of the inner minimization ACOPF problem. In this paper, we focus on the development of a proximal projection bundle method (PPBM) for solving continuous relaxation node subproblems of the MISDP problem, based primarily on the well-known alternating direction method of multipliers. Cutting planes reminiscent of bundle method ideas are also applied in coordination with updates of the proximal parameter. The cutting-plane method can generate a large number of linear inequalities, leading to a large scale but decomposable quadratic programming (QP) subproblem that is amenable to GPUs. We present the numerical results on the IEEE 30, 57, 118, and 300-bus systems by using our PBMM method. We discuss the main computational bottleneck of our method, which is the time taken to solve each iteration of a QP subproblem instance of the PPBM, and how GPU architectures can accelerate this solution process.

A new code for the numerical simulation of relativistic flows on supercomputers by means of a low-dissipation scheme

A new code to simulate special relativistic hydrodynamic flows on supercomputer architectures with distributed memory is described. The code is based on a combination of Godunov’s method and a piecewise parabolic method with a local stencil. This approach has good conservation properties, correctly reproduces shock waves, and ensures high accuracy on smooth solutions and low dissipation on discontinuities. Only a local computation stencil is needed for the piecewise parabolic reconstruction of the solution. The code scalability is 94% on a cluster, Intel Xeon X5670 NKS-30T, with 768 cores. The results of code verification using a relativistic jet problem and computational experiments on the evolution of a galactic jet are presented. Program summaryProgram title: RHydroBox3DCPC Library link to program files:http://dx.doi.org/10.17632/xskchgynfy.1Licensing provisions: GNU General Public License 3Programming language: C++Nature of problem: The special relativistic hydrodynamic flows.Solution method: The combination of Godunov’s method and a piecewise parabolic method on a local stencil.Additional comments including restrictions and unusual features: External library – MPI.

OHTMA: an optimized heuristic topology-aware mapping algorithm on the Tianhe-3 exascale supercomputer prototype

With the rapid increase of the size of applications and the complexity of the supercomputer architecture, topology-aware process mapping becomes increasingly important. High communication cost has become a dominant constraint of the performance of applications running on the supercomputer. To avoid a bad mapping strategy which can lead to terrible communication performance, we propose an optimized heuristic topology-aware mapping algorithm (OHTMA). The algorithm attempts to minimize the hop-byte metric that we use to measure the mapping results. OHTMA incorporates a new greedy heuristic method and pair-exchange-based optimization. It reduces the number of long-distance communications and effectively enhances the locality of the communication. Experimental results on the Tianhe-3 exascale supercomputer prototype indicate that OHTMA can significantly reduce the communication costs.

On the Monotonic Lagrangian Grid as Antecedent to the Neighborhood Grid

We note that a recently described data structure, the Neighborhood Grid, is equivalent to a data structure developed in the mid-1980s called the Monotonic Lagrangian Grid (MLG). The MLG was originally developed to support high-performance molecular and fluid dynamics simulations on both supercomputer and vector processing architectures and still finds use in those and other areas. In this paper we emphasize that the rediscovery of the MLG offers benefits to users of the Neighborhood Grid in the form of an existing literature with results relevant to its efficient implementation in various contexts while users of the MLG similarly benefit from new theoretical results obtained for the Neighborhood grid.

Computational functionality‐driven design of semiconductors for optoelectronic applications

AbstractThe rapid development of the semiconductor industry has motivated researchers passion for accelerating the discovery of advanced optoelectronic materials. Computational functionality‐driven design is an emerging branch of material science that has become effective at making material predictions. By combining advanced solid‐state knowledge and high‐throughput first‐principles computational approaches with intelligent algorithms plus database development, experts can now efficiently explore many novel materials by taking advantage of the power of supercomputer architectures. Here, we discuss a set of typical design strategies that can be used to accelerate inorganic optoelectronic materials discovery from computer simulations: In silico computational screening; knowledge‐based inverse design; and algorithm‐based searching. A few representative examples in optoelectronic materials design are discussed to illustrate these computational functionality‐driven modalities. Challenges and prospects for the computational functionality‐driven design of materials are further highlighted at the end of the review.image

Supercomputer technologies for solving problems of computational physics

The issues of using supercomputer technologies developed by the authors to solve compute-intensive problems of mathematical physics are discussed. The technology of developing algorithms and software for supercomputers contains the three related stages: co-design, by which we understand the adaptation of the problem statement, the mathematical method and the computational algorithm to the parallel architecture of the supercomputer at all stages of the problem solution; studying the scalability of computational algorithms for the most promising supercomputers based on simulation modeling; evaluation of the energy efficiency of algorithms for various implementations on a given supercomputer architecture. It is proposed to further develop this approach with the use of the intelligence support for solving compute-intensive problems using the ontology of computational methods and algorithms for solving the problems of computational physics, the ontology of computational heterogeneous architectures and decision rules.