High Performance Computing Platforms Research Articles

ExaWorks software development kit: a robust and scalable collection of interoperable workflows technologies

Scientific discovery increasingly requires executing heterogeneous scientific workflows on high-performance computing (HPC) platforms. Heterogeneous workflows contain different types of tasks (e.g., simulation, analysis, and learning) that need to be mapped, scheduled, and launched on different computing. That requires a software stack that enables users to code their workflows and automate resource management and workflow execution. Currently, there are many workflow technologies with diverse levels of robustness and capabilities, and users face difficult choices of software that can effectively and efficiently support their use cases on HPC machines, especially when considering the latest exascale platforms. We contributed to addressing this issue by developing the ExaWorks Software Development Kit (SDK). The SDK is a curated collection of workflow technologies engineered following current best practices and specifically designed to work on HPC platforms. We present our experience with (1) curating those technologies, (2) integrating them to provide users with new capabilities, (3) developing a continuous integration platform to test the SDK on DOE HPC platforms, (4) designing a dashboard to publish the results of those tests, and (5) devising an innovative documentation platform to help users to use those technologies. Our experience details the requirements and the best practices needed to curate workflow technologies, and it also serves as a blueprint for the capabilities and services that DOE will have to offer to support a variety of scientific heterogeneous workflows on the newly available exascale HPC platforms.

Harmonizing Power and Economy: A Comprehensive Survey On Optimizing Heterogeneous Platforms For High-Performance Computing

Harmonizing Power and Economy: A Comprehensive Survey On Optimizing Heterogeneous Platforms For High-Performance Computing

Advancing Climate Modeling through High-Performance Computing: Towards More Accurate and Efficient Simulations

A crucial branch of science called climate modeling uses mathematical equations and computer simulations to study and forecast the Earth's climate sys- tem. The main elements of climate modeling, such as general circulation models (GCMs), data assimilation methods, and numerical formulations, are outlined in this paper. GCMs, which include grid point and spectral models, are effective instruments for examining the behavior of the climate. Four-Dimensional Data Assimilation (4D-Var) is one example of a data assimilation technique that in- corporates observational data into models to improve their correctness. Numeri cal methods, ocean dynamics, heat transport, radiative transfer, and atmospheric dynamics are all included in numerical formulations. The simulation of different climate processes is possible because to these mathematical representations. Fur thermore, the detection of precipitation patterns within climate modeling is using machine learning techniques like Random Forest more frequently. This paper highlights the importance of high-performance computing (HPC) in climate modeling, boosting efficiency and simulations, in the context of research technique. Advanced data assimilation and validation techniques are also examined, as well as the influence of high-resolution modeling on small-scale climatic processes. On HPC platforms, accessibility to climate modeling is addressed. It is shown how climate modeling crosses physics, mathematics, computer science, and engineering to be interdisciplinary. A comprehensive understanding of the Earth's intricate climate system gains from the integration of all its parts, from atmospheric dynamics to data assimilation. We explore the consequences of these research approaches, their contribution to enhancing climate prediction models, and the influence of various factors on climatic variables in the debate. Climate modeling becomes an essential tool for studying precipitation patterns and climate change, ultimately improving our comprehension of the complex cli- mate system on Earth.

Leonhard Med, a trusted research environment for processing sensitive research data.

This paper provides an overview of the development and operation of the Leonhard Med Trusted Research Environment (TRE) at ETH Zurich. Leonhard Med gives scientific researchers the ability to securely work on sensitive research data. We give an overview of the user perspective, the legal framework for processing sensitive data, design history, current status, and operations. Leonhard Med is an efficient, highly secure Trusted Research Environment for data processing, hosted at ETH Zurich and operated by the Scientific IT Services (SIS) of ETH. It provides a full stack of security controls that allow researchers to store, access, manage, and process sensitive data according to Swiss legislation and ETH Zurich Data Protection policies. In addition, Leonhard Med fulfills the BioMedIT Information Security Policies and is compatible with international data protection laws and therefore can be utilized within the scope of national and international collaboration research projects. Initially designed as a "bare-metal" High-Performance Computing (HPC) platform to achieve maximum performance, Leonhard Med was later re-designed as a virtualized, private cloud platform to offer more flexibility to its customers. Sensitive data can be analyzed in secure, segregated spaces called tenants. Technical and Organizational Measures (TOMs) are in place to assure the confidentiality, integrity, and availability of sensitive data. At the same time, Leonhard Med ensures broad access to cutting-edge research software, especially for the analysis of human -omics data and other personalized health applications.

FLEW: A DNS Solver for Compressible Flows in Generalized Curvilinear Coordinates

We present FLEW, an in-house high-fidelity solver for direct numerical simulation (DNS) of turbulent compressible flows over arbitrary shaped geometries. FLEW solves the Navier–Stokes equations written in a generalized curvilinear coordinate system, in which the surface coordinates are non-orthogonal, whereas the third axis is normal to the surface. Spatial discretization relies on high-order finite-difference schemes. The convective terms are discretized using an hybrid approach, combining the near-zero numerical dissipation provided by central approximations with the robustness of weighted essentially non-oscillatory (WENO) schemes, required to capture shock waves. Central schemes are stabilized using a skew-symmetric-like splitting of convective derivatives, endowing the solver with the energy-preserving property in the inviscid limit. The maximum order of accuracy is eighth for central schemes (also used for viscous terms discretization) and seventh for WENO. The code is oriented to modern high-performance computing (HPC) platforms thanks to message passing interface (MPI) parallelization and the ability to run on graphics processing unit (GPU) architectures. Reliability, accuracy and robustness of the code are assessed in the low-subsonic, transonic and supersonic regimes. We present the results of several benchmarks, namely the inviscid Taylor–Green vortex, turbulent curved channel flow, transonic laminar flow over a NACA 0012 airfoil and turbulent supersonic ramp flow. The results for all configurations proved to be in excellent agreement with previous studies.

Coupled neutronics and thermal-hydraulics using TRUST-NK for high performance computing in molten salt reactor simulation

Molten Salt Reactors display a strong coupling between neutronics and thermal-hydraulics for which various simulation tools have been developed. In this paper we describe TRUST-NK, a neutronics code based on TRUST, CEA’s High Performance Computing (HPC) platform. TRUST-NK provides a multi-group diffusion solver with transport of delayed neutron precursors, for both steady-state and transient simulations. Being based on TRUST allows TRUST-NK to take advantage of the platform’s HPC capabilities and to easily couple to CFD solvers provided by other derivatives of TRUST. After a brief description of the code, we present an example of application to a simplified MSR model.

WFA-GPU: gap-affine pairwise read-alignment using GPUs.

Advances in genomics and sequencing technologies demand faster and more scalable analysis methods that can process longer sequences with higher accuracy. However, classical pairwise alignment methods, based on dynamic programming (DP), impose impractical computational requirements to align long and noisy sequences like those produced by PacBio and Nanopore technologies. The recently proposed wavefront alignment (WFA) algorithm paves the way for more efficient alignment tools, improving time and memory complexity over previous methods. However, high-performance computing (HPC) platforms require efficient parallel algorithms and tools to exploit the computing resources available on modern accelerator-based architectures. This paper presents WFA-GPU, a GPU (graphics processing unit)-accelerated tool to compute exact gap-affine alignments based on the WFA algorithm. We present the algorithmic adaptations and performance optimizations that allow exploiting the massively parallel capabilities of modern GPU devices to accelerate the alignment computations. In particular, we propose a CPU-GPU co-design capable of performing inter-sequence and intra-sequence parallel sequence alignment, combining a succinct WFA-data representation with an efficient GPU implementation. As a result, we demonstrate that our implementation outperforms the original multi-threaded WFA implementation by up to 4.3× and up to 18.2× when using heuristic methods on long and noisy sequences. Compared to other state-of-the-art tools and libraries, the WFA-GPU is up to 29× faster than other GPU implementations and up to four orders of magnitude faster than other CPU implementations. Furthermore, WFA-GPU is the only GPU solution capable of correctly aligning long reads using a commodity GPU. WFA-GPU code and documentation are publicly available at https://github.com/quim0/WFA-GPU.

Edge, Fog, and Cloud Against Disease: The Potential of High-Performance Cloud Computing for Pharma Drug Discovery.

The high-performance computing (HPC) platform for large-scale drug discovery simulation demands significant investment in speciality hardware, maintenance, resource management, and running costs. The rapid growth in computing hardware has made it possible to provide cost-effective, robust, secure, and scalable alternatives to the on-premise (on-prem) HPC via Cloud, Fog, and Edge computing. It has enabled recent state-of-the-art machine learning (ML) and artificial intelligence (AI)-based tools for drug discovery, such as BERT, BARD, AlphaFold2, and GPT. This chapter attempts to overview types of software architectures for developing scientific software or application with deployment agnostic (on-prem to cloud and hybrid) use cases. Furthermore, the chapter aims to outline howthe innovation is disrupting the orthodox mindset of monolithic software running on on-prem HPCand provide the paradigm shift landscape tomicroservices driven application programming (API) and message parsing interface (MPI)-based scientific computing across the distributed, high-available infrastructure. This is coupledwith agileDevOps, and good coding practices, low code and no-code application development frameworks for cost-efficient, secure, automated, and robust scientific application life cycle management.

Fully Parallel Optimization of Coordinated Electricity and Natural Gas Systems on High-Performance Computing

Intensified interactions between electric power and natural gas infrastructures raise significant demands to coordinate their system operations. This paper proposes a fully parallel optimization method that can achieve a rapid decision-making on the day-ahead coordinated operation of electricity and natural gas systems on the high-performance computing (HPC) platform. The proposed method can flexibly tailor decomposition strategies to solve the optimization problem according to unique features of problem models in both electric power and natural gas systems. Particularly, the operation problem of power system is split by function and time into numbers of singe-unit subproblems and single-period network subproblems, while the operation problem of natural gas system is decomposed into multiple area subproblems. All the scalable subproblems are solved and coordinated quickly in a fully parallel manner on HPC for improving computational efficiency and tractability of complex electricity-gas co-optimization problem. By optimally coordinate electricity and natural gas systems under different operating conditions, the proposed method could improve the energy economics as well as the system resilience to various outages due to extreme events. Numerical results demonstrate the effectiveness and efficiency of our proposed co-optimization method and its HPC implementation.

BioLegato: a programmable, object-oriented graphic user interface

BackgroundBiologists are faced with an ever-changing array of complex software tools with steep learning curves, often run on High Performance Computing platforms. To resolve the tradeoff between analytical sophistication and usability, we have designed BioLegato, a programmable graphical user interface (GUI) for running external programs.ResultsBioLegato can run any program or pipeline that can be launched as a command. BioLegato reads specifications for each tool from files written in PCD, a simple language for specifying GUI components that set parameters for calling external programs. Thus, adding new tools to BioLegato can be done without changing the BioLegato Java code itself. The process is as simple as copying an existing PCD file and modifying it for the new program, which is more like filling in a form than writing code. PCD thus facilitates rapid development of new applications using existing programs as building blocks, and getting them to work together seamlessly.ConclusionBioLegato applies Object-Oriented concepts to the user experience by organizing applications based on discrete data types and the methods relevant to that data. PCD makes it easier for BioLegato applications to evolve with the succession of analytical tools for bioinformatics. BioLegato is applicable not only in biology, but in almost any field in which disparate software tools need to work as an integrated system.

Survey of Novel Architectures for Energy Efficient High-Performance Mobile Computing Platforms

There are many real-world applications that require high-performance mobile computing systems for onboard, real-time processing of gathered data due to latency, reliability, security, or other application constraints. Unfortunately, most existing high-performance mobile computing systems require a prohibitively high power consumption in the face of the limited power available from the batteries typically used in these applications. For high-performance mobile computing to be practical, alternative hardware designs are needed to increase the computing performance while minimizing the required power consumption. This article surveys the state-of-the-art in high-efficiency, high-performance onboard mobile computing, focusing on the latest developments. It was found that more research is needed to design high-performance mobile computing systems while minimizing the required power consumption to meet the needs of these applications.

HPC Platform for Railway Safety-Critical Functionalities Based on Artificial Intelligence

The automation of railroad operations is a rapidly growing industry. In 2023, a new European standard for the automated Grade of Automation (GoA) 2 over European Train Control System (ETCS) driving is anticipated. Meanwhile, railway stakeholders are already planning their research initiatives for driverless and unattended autonomous driving systems. As a result, the industry is particularly active in research regarding perception technologies based on Computer Vision (CV) and Artificial Intelligence (AI), with outstanding results at the application level. However, executing high-performance and safety-critical applications on embedded systems and in real-time is a challenge. There are not many commercially available solutions, since High-Performance Computing (HPC) platforms are typically seen as being beyond the business of safety-critical systems. This work proposes a novel safety-critical and high-performance computing platform for CV- and AI-enhanced technology execution used for automatic accurate stopping and safe passenger transfer railway functionalities. The resulting computing platform is compatible with the majority of widely-used AI inference methodologies, AI model architectures, and AI model formats thanks to its design, which enables process separation, redundant execution, and HW acceleration in a transparent manner. The proposed technology increases the portability of railway applications into embedded systems, isolates crucial operations, and effectively and securely maintains system resources.

Uncovering I/O demands on HPC platforms: Peeking under the hood of Santos Dumont

High-Performance Computing (HPC) platforms are required to solve the most diverse large-scale scientific problems in various research areas, such as biology, chemistry, physics, and health sciences. Researchers use a multitude of scientific softwares, which have different requirements. These include input and output operations, which directly impact performance due to the existing difference in processing and data access speeds. Thus, supercomputers must efficiently handle mixed workload when storing data from the applications. Understanding the set of applications and their performance running in a supercomputer is paramount to understanding the storage system's usage, pinpointing possible bottlenecks, and guiding optimization techniques. This research proposes a methodology and visualization tool to evaluate a supercomputer's data storage infrastructure's performance, taking into account the diverse workload and demands of the system over a long period of operation. As a study case, we focus on the Santos Dumont supercomputer, identifying inefficient usage, problematic performance factors, and providing guidelines on how to tackle those issues.

Mixed precision support in HPC applications: What about reliability?

In their quest for exascale and beyond, High-Performance Computing (HPC) systems continue becoming ever larger and more complex. Application developers, on the other hand, leverage novel methods to improve the efficiency of their own codes: a recent trend is the use of floating-point mixed precision, or the careful interlocking of single- and double-precision arithmetic, as a tool to improve performance as well as reduce network and memory boundedness. However, while it is known that modern HPC systems suffer hardware faults at daily rates, the impact of reduced precision on application reliability is yet to be explored. In this work we aim to fill this gap: first, we propose a qualitative survey to identify the branches of HPC where mixed precision is most popular. Second, we show the results of instruction-level fault injection experiments on a variety of representative HPC workloads, comparing vulnerability to Silent Data Errors (SDEs) under different numerical configurations. Our experiments indicate that use of single and mixed precision leads to comparatively more frequent and more severe SDEs, with concerning implications regarding their use on extreme-scale, fault-prone HPC platforms.

An Optimal Network-Aware Scheduling Technique for Distributed Deep Learning in Distributed HPC Platforms

Deep learning is a growing technique used to solve complex artificial intelligence (AI) problems. Large-scale deep learning has become a significant issue as a result of the expansion of datasets and the complexity of deep learning models. For training large-scale models, the cloud can be used as a distributed HPC (high-performance computing) tool with benefits in cost and flexibility. However, one of the major performance barriers in distributed deep learning in a distributed HPC environment is the network. The performance is often limited by heavy traffic like many stochastic gradient descent transfers for distributed communication. There are many network studies in distributed deep learning to solve these problems, but most research only focuses on improving communication performance and applying new methods or algorithms like overlapping parameter synchronization to minimize communication delay rather than considering the actual network. In this paper, we are focusing on the actual network, especially in a distributed HPC environment. In such an environment, if cluster nodes are assigned to different zones/regions which means a set of an appropriate number of distributed HPC nodes when performing distributed deep learning tasks, performance degradation due to network delay may occur. The proposed network optimization algorithm ensures that distributed work is placed in the same zone as much as possible to reduce network delay. Furthermore, scoring using network monitoring tools like loss, delay, and throughput is applied to select the optimal node within the zone. Our proposal has been validated on the Kubernetes platform, an open source orchestrator for the automatic management and deployment of micro-services. The performance of distributed deep learning is improved through the proposed scheduler.

NOAA’s Global Forecast System Data in the Cloud for Community Air Quality Modeling

Herein, we present the initial application of the NOAA-EPA Atmosphere-Chemistry Coupler (NACC) in the cloud (“NACC-Cloud”, version 1), which processes NOAA’s operational Global Forecast System version 16 (GFSv16) meteorology on-demand and produces model-ready meteorological files needed to drive U.S. EPA’s Community Multiscale Air Quality (CMAQ) model. NACC is adapted from the U.S. EPA’s Meteorology-Chemistry Interface Processor version 5 (MCIPv5) and is used as the primary model coupler in the current operational NWS/NOAA air quality forecasting model. The development and use of NACC-Cloud in this work are critical to provide the scientific community streamlined access to NOAA’s operational GFSv16 data and user-defined processing and download of model-ready, meteorological input for any regional CMAQ domain worldwide. The NACC-Cloud system was implemented on the Amazon® Web Services High-Performance Computing platform, and results from this work show that the NACC-Cloud system is immediately beneficial to the air quality modeling community worldwide.

HPC+ in the medical field: Overview and current examples.

To say data is revolutionising the medical sector would be a vast understatement. The amount of medical data available today is unprecedented and has the potential to enable to date unseen forms of healthcare. To process this huge amount of data, an equally huge amount of computing power is required, which cannot be provided by regular desktop computers. These areas can be (and already are) supported by High-Performance-Computing (HPC), High-Performance Data Analytics (HPDA), and AI (together "HPC+"). This overview article aims to show state-of-the-art examples of studies supported by the National Competence Centres (NCCs) in HPC+ within the EuroCC project, employing HPC, HPDA and AI for medical applications. The included studies on different applications of HPC in the medical sector were sourced from the National Competence Centres in HPC and compiled into an overview article. Methods include the application of HPC+ for medical image processing, high-performance medical and pharmaceutical data analytics, an application for pediatric dosimetry, and a cloud-based HPC platform to support systemic pulmonary shunting procedures. This article showcases state-of-the-art applications and large-scale data analytics in the medical sector employing HPC+ within surgery, medical image processing in diagnostics, nutritional support of patients in hospitals, treating congenital heart diseases in children, and within basic research. HPC+ support scientific fields from research to industrial applications in the medical area, enabling researchers to run faster and more complex calculations, simulations and data analyses for the direct benefit of patients, doctors, clinicians and as an accelerator for medical research.

Sparse matrix multiplication in a record-low power self-rectifying memristor array for scientific computing.

Memristor-enabled in-memory computing provides an unconventional computing paradigm to surpass the energy efficiency of von Neumann computers. Owing to the limitation of the computing mechanism, while the crossbar structure is desirable for dense computation, the system's energy and area efficiency degrade substantially in performing sparse computation tasks, such as scientific computing. In this work, we report a high-efficiency in-memory sparse computing system based on a self-rectifying memristor array. This system originates from an analog computing mechanism that is motivated by the device's self-rectifying nature, which can achieve an overall performance of ~97 to ~11 TOPS/W for 2- to 8-bit sparse computation when processing practical scientific computing tasks. Compared to previous in-memory computing system, this work provides over 85 times improvement in energy efficiency with an approximately 340 times reduction in hardware overhead. This work can pave the road toward a highly efficient in-memory computing platform for high-performance computing.

Solving the electronic structure problem for over 100 000 atoms in real space

Using a real-space high-order finite-difference approach, we investigate the electronic structure of large spherical silicon nanoclusters. Within Kohn-Sham density functional theory and using pseudopotentials, we report the self-consistent field convergence of a system with over 100 000 atoms: a ${\mathrm{Si}}_{107,641}{\mathrm{H}}_{9,084}$ nanocluster with a diameter of 16 nm. Our approach uses Chebyshev-filtered subspace iteration to speed up the convergence of the eigenspace, and blockwise Hilbert space-filling curves to speed up sparse matrix-vector multiplications, all of which are implemented in the parsec code. For the largest system, we utilized 2048 nodes (114 688 cores) on the Frontera machine in the Texas Advanced Computing Center. Our quantitative analysis of the electronic structure shows how it gradually approaches its bulk counterpart as a function of nanocluster size. The band gap is enlarged due to quantum confinement in nanoclusters, but decreases as the system size increases, as expected. Our work serves as a proof of concept for the capacity of the real-space approach in efficiently parallelizing very large calculations using high-performance computer platforms, which can straightforwardly be replicated in other systems with more than ${10}^{5}$ atoms.

Energy-Efficient Parallel Computing: Challenges to Scaling

The energy consumption of Information and Communications Technology (ICT) presents a new grand technological challenge. The two main approaches to tackle the challenge include the development of energy-efficient hardware and software. The development of energy-efficient software employing application-level energy optimization techniques has become an important category owing to the paradigm shift in the composition of digital platforms from single-core processors to heterogeneous platforms integrating multicore CPUs and graphics processing units (GPUs). In this work, we present an overview of application-level bi-objective optimization methods for energy and performance that address two fundamental challenges, non-linearity and heterogeneity, inherent in modern high-performance computing (HPC) platforms. Applying the methods requires energy profiles of the application’s computational kernels executing on the different compute devices of the HPC platform. Therefore, we summarize the research innovations in the three mainstream component-level energy measurement methods and present their accuracy and performance tradeoffs. Finally, scaling the optimization methods for energy and performance is crucial to achieving energy efficiency objectives and meeting quality-of-service requirements in modern HPC platforms and cloud computing infrastructures. We introduce the building blocks needed to achieve this scaling and conclude with the challenges to scaling. Briefly, two significant challenges are described, namely fast optimization methods and accurate component-level energy runtime measurements, especially for components running on accelerators.