High Performance Computing Platforms Research Articles

Leonhard Med, a trusted research environment for processing sensitive research data.

This paper provides an overview of the development and operation of the Leonhard Med Trusted Research Environment (TRE) at ETH Zurich. Leonhard Med gives scientific researchers the ability to securely work on sensitive research data. We give an overview of the user perspective, the legal framework for processing sensitive data, design history, current status, and operations. Leonhard Med is an efficient, highly secure Trusted Research Environment for data processing, hosted at ETH Zurich and operated by the Scientific IT Services (SIS) of ETH. It provides a full stack of security controls that allow researchers to store, access, manage, and process sensitive data according to Swiss legislation and ETH Zurich Data Protection policies. In addition, Leonhard Med fulfills the BioMedIT Information Security Policies and is compatible with international data protection laws and therefore can be utilized within the scope of national and international collaboration research projects. Initially designed as a "bare-metal" High-Performance Computing (HPC) platform to achieve maximum performance, Leonhard Med was later re-designed as a virtualized, private cloud platform to offer more flexibility to its customers. Sensitive data can be analyzed in secure, segregated spaces called tenants. Technical and Organizational Measures (TOMs) are in place to assure the confidentiality, integrity, and availability of sensitive data. At the same time, Leonhard Med ensures broad access to cutting-edge research software, especially for the analysis of human -omics data and other personalized health applications.

FLEW: A DNS Solver for Compressible Flows in Generalized Curvilinear Coordinates

AbstractWe present FLEW, an in-house high-fidelity solver for direct numerical simulation (DNS) of turbulent compressible flows over arbitrary shaped geometries. FLEW solves the Navier–Stokes equations written in a generalized curvilinear coordinate system, in which the surface coordinates are non-orthogonal, whereas the third axis is normal to the surface. Spatial discretization relies on high-order finite-difference schemes. The convective terms are discretized using an hybrid approach, combining the near-zero numerical dissipation provided by central approximations with the robustness of weighted essentially non-oscillatory (WENO) schemes, required to capture shock waves. Central schemes are stabilized using a skew-symmetric-like splitting of convective derivatives, endowing the solver with the energy-preserving property in the inviscid limit. The maximum order of accuracy is eighth for central schemes (also used for viscous terms discretization) and seventh for WENO. The code is oriented to modern high-performance computing (HPC) platforms thanks to message passing interface (MPI) parallelization and the ability to run on graphics processing unit (GPU) architectures. Reliability, accuracy and robustness of the code are assessed in the low-subsonic, transonic and supersonic regimes. We present the results of several benchmarks, namely the inviscid Taylor–Green vortex, turbulent curved channel flow, transonic laminar flow over a NACA 0012 airfoil and turbulent supersonic ramp flow. The results for all configurations proved to be in excellent agreement with previous studies.

WFA-GPU: gap-affine pairwise read-alignment using GPUs.

Advances in genomics and sequencing technologies demand faster and more scalable analysis methods that can process longer sequences with higher accuracy. However, classical pairwise alignment methods, based on dynamic programming (DP), impose impractical computational requirements to align long and noisy sequences like those produced by PacBio and Nanopore technologies. The recently proposed wavefront alignment (WFA) algorithm paves the way for more efficient alignment tools, improving time and memory complexity over previous methods. However, high-performance computing (HPC) platforms require efficient parallel algorithms and tools to exploit the computing resources available on modern accelerator-based architectures. This paper presents WFA-GPU, a GPU (graphics processing unit)-accelerated tool to compute exact gap-affine alignments based on the WFA algorithm. We present the algorithmic adaptations and performance optimizations that allow exploiting the massively parallel capabilities of modern GPU devices to accelerate the alignment computations. In particular, we propose a CPU-GPU co-design capable of performing inter-sequence and intra-sequence parallel sequence alignment, combining a succinct WFA-data representation with an efficient GPU implementation. As a result, we demonstrate that our implementation outperforms the original multi-threaded WFA implementation by up to 4.3× and up to 18.2× when using heuristic methods on long and noisy sequences. Compared to other state-of-the-art tools and libraries, the WFA-GPU is up to 29× faster than other GPU implementations and up to four orders of magnitude faster than other CPU implementations. Furthermore, WFA-GPU is the only GPU solution capable of correctly aligning long reads using a commodity GPU. WFA-GPU code and documentation are publicly available at https://github.com/quim0/WFA-GPU.

Edge, Fog, and Cloud Against Disease: The Potential of High-Performance Cloud Computing for Pharma Drug Discovery.

The high-performance computing (HPC) platform for large-scale drug discovery simulation demands significant investment in speciality hardware, maintenance, resource management, and running costs. The rapid growth in computing hardware has made it possible to provide cost-effective, robust, secure, and scalable alternatives to the on-premise (on-prem) HPC via Cloud, Fog, and Edge computing. It has enabled recent state-of-the-art machine learning (ML) and artificial intelligence (AI)-based tools for drug discovery, such as BERT, BARD, AlphaFold2, and GPT. This chapter attempts to overview types of software architectures for developing scientific software or application with deployment agnostic (on-prem to cloud and hybrid) use cases. Furthermore, the chapter aims to outline howthe innovation is disrupting the orthodox mindset of monolithic software running on on-prem HPCand provide the paradigm shift landscape tomicroservices driven application programming (API) and message parsing interface (MPI)-based scientific computing across the distributed, high-available infrastructure. This is coupledwith agileDevOps, and good coding practices, low code and no-code application development frameworks for cost-efficient, secure, automated, and robust scientific application life cycle management.

Fully Parallel Optimization of Coordinated Electricity and Natural Gas Systems on High-Performance Computing

Intensified interactions between electric power and natural gas infrastructures raise significant demands to coordinate their system operations. This paper proposes a fully parallel optimization method that can achieve a rapid decision-making on the day-ahead coordinated operation of electricity and natural gas systems on the high-performance computing (HPC) platform. The proposed method can flexibly tailor decomposition strategies to solve the optimization problem according to unique features of problem models in both electric power and natural gas systems. Particularly, the operation problem of power system is split by function and time into numbers of singe-unit subproblems and single-period network subproblems, while the operation problem of natural gas system is decomposed into multiple area subproblems. All the scalable subproblems are solved and coordinated quickly in a fully parallel manner on HPC for improving computational efficiency and tractability of complex electricity-gas co-optimization problem. By optimally coordinate electricity and natural gas systems under different operating conditions, the proposed method could improve the energy economics as well as the system resilience to various outages due to extreme events. Numerical results demonstrate the effectiveness and efficiency of our proposed co-optimization method and its HPC implementation.

BioLegato: a programmable, object-oriented graphic user interface

BackgroundBiologists are faced with an ever-changing array of complex software tools with steep learning curves, often run on High Performance Computing platforms. To resolve the tradeoff between analytical sophistication and usability, we have designed BioLegato, a programmable graphical user interface (GUI) for running external programs.ResultsBioLegato can run any program or pipeline that can be launched as a command. BioLegato reads specifications for each tool from files written in PCD, a simple language for specifying GUI components that set parameters for calling external programs. Thus, adding new tools to BioLegato can be done without changing the BioLegato Java code itself. The process is as simple as copying an existing PCD file and modifying it for the new program, which is more like filling in a form than writing code. PCD thus facilitates rapid development of new applications using existing programs as building blocks, and getting them to work together seamlessly.ConclusionBioLegato applies Object-Oriented concepts to the user experience by organizing applications based on discrete data types and the methods relevant to that data. PCD makes it easier for BioLegato applications to evolve with the succession of analytical tools for bioinformatics. BioLegato is applicable not only in biology, but in almost any field in which disparate software tools need to work as an integrated system.

Survey of Novel Architectures for Energy Efficient High-Performance Mobile Computing Platforms

There are many real-world applications that require high-performance mobile computing systems for onboard, real-time processing of gathered data due to latency, reliability, security, or other application constraints. Unfortunately, most existing high-performance mobile computing systems require a prohibitively high power consumption in the face of the limited power available from the batteries typically used in these applications. For high-performance mobile computing to be practical, alternative hardware designs are needed to increase the computing performance while minimizing the required power consumption. This article surveys the state-of-the-art in high-efficiency, high-performance onboard mobile computing, focusing on the latest developments. It was found that more research is needed to design high-performance mobile computing systems while minimizing the required power consumption to meet the needs of these applications.

HPC Platform for Railway Safety-Critical Functionalities Based on Artificial Intelligence

The automation of railroad operations is a rapidly growing industry. In 2023, a new European standard for the automated Grade of Automation (GoA) 2 over European Train Control System (ETCS) driving is anticipated. Meanwhile, railway stakeholders are already planning their research initiatives for driverless and unattended autonomous driving systems. As a result, the industry is particularly active in research regarding perception technologies based on Computer Vision (CV) and Artificial Intelligence (AI), with outstanding results at the application level. However, executing high-performance and safety-critical applications on embedded systems and in real-time is a challenge. There are not many commercially available solutions, since High-Performance Computing (HPC) platforms are typically seen as being beyond the business of safety-critical systems. This work proposes a novel safety-critical and high-performance computing platform for CV- and AI-enhanced technology execution used for automatic accurate stopping and safe passenger transfer railway functionalities. The resulting computing platform is compatible with the majority of widely-used AI inference methodologies, AI model architectures, and AI model formats thanks to its design, which enables process separation, redundant execution, and HW acceleration in a transparent manner. The proposed technology increases the portability of railway applications into embedded systems, isolates crucial operations, and effectively and securely maintains system resources.

Uncovering I/O demands on HPC platforms: Peeking under the hood of Santos Dumont

High-Performance Computing (HPC) platforms are required to solve the most diverse large-scale scientific problems in various research areas, such as biology, chemistry, physics, and health sciences. Researchers use a multitude of scientific softwares, which have different requirements. These include input and output operations, which directly impact performance due to the existing difference in processing and data access speeds. Thus, supercomputers must efficiently handle mixed workload when storing data from the applications. Understanding the set of applications and their performance running in a supercomputer is paramount to understanding the storage system's usage, pinpointing possible bottlenecks, and guiding optimization techniques. This research proposes a methodology and visualization tool to evaluate a supercomputer's data storage infrastructure's performance, taking into account the diverse workload and demands of the system over a long period of operation. As a study case, we focus on the Santos Dumont supercomputer, identifying inefficient usage, problematic performance factors, and providing guidelines on how to tackle those issues.

Mixed precision support in HPC applications: What about reliability?

In their quest for exascale and beyond, High-Performance Computing (HPC) systems continue becoming ever larger and more complex. Application developers, on the other hand, leverage novel methods to improve the efficiency of their own codes: a recent trend is the use of floating-point mixed precision, or the careful interlocking of single- and double-precision arithmetic, as a tool to improve performance as well as reduce network and memory boundedness. However, while it is known that modern HPC systems suffer hardware faults at daily rates, the impact of reduced precision on application reliability is yet to be explored. In this work we aim to fill this gap: first, we propose a qualitative survey to identify the branches of HPC where mixed precision is most popular. Second, we show the results of instruction-level fault injection experiments on a variety of representative HPC workloads, comparing vulnerability to Silent Data Errors (SDEs) under different numerical configurations. Our experiments indicate that use of single and mixed precision leads to comparatively more frequent and more severe SDEs, with concerning implications regarding their use on extreme-scale, fault-prone HPC platforms.

An Optimal Network-Aware Scheduling Technique for Distributed Deep Learning in Distributed HPC Platforms

Deep learning is a growing technique used to solve complex artificial intelligence (AI) problems. Large-scale deep learning has become a significant issue as a result of the expansion of datasets and the complexity of deep learning models. For training large-scale models, the cloud can be used as a distributed HPC (high-performance computing) tool with benefits in cost and flexibility. However, one of the major performance barriers in distributed deep learning in a distributed HPC environment is the network. The performance is often limited by heavy traffic like many stochastic gradient descent transfers for distributed communication. There are many network studies in distributed deep learning to solve these problems, but most research only focuses on improving communication performance and applying new methods or algorithms like overlapping parameter synchronization to minimize communication delay rather than considering the actual network. In this paper, we are focusing on the actual network, especially in a distributed HPC environment. In such an environment, if cluster nodes are assigned to different zones/regions which means a set of an appropriate number of distributed HPC nodes when performing distributed deep learning tasks, performance degradation due to network delay may occur. The proposed network optimization algorithm ensures that distributed work is placed in the same zone as much as possible to reduce network delay. Furthermore, scoring using network monitoring tools like loss, delay, and throughput is applied to select the optimal node within the zone. Our proposal has been validated on the Kubernetes platform, an open source orchestrator for the automatic management and deployment of micro-services. The performance of distributed deep learning is improved through the proposed scheduler.

NOAA’s Global Forecast System Data in the Cloud for Community Air Quality Modeling

Herein, we present the initial application of the NOAA-EPA Atmosphere-Chemistry Coupler (NACC) in the cloud (“NACC-Cloud”, version 1), which processes NOAA’s operational Global Forecast System version 16 (GFSv16) meteorology on-demand and produces model-ready meteorological files needed to drive U.S. EPA’s Community Multiscale Air Quality (CMAQ) model. NACC is adapted from the U.S. EPA’s Meteorology-Chemistry Interface Processor version 5 (MCIPv5) and is used as the primary model coupler in the current operational NWS/NOAA air quality forecasting model. The development and use of NACC-Cloud in this work are critical to provide the scientific community streamlined access to NOAA’s operational GFSv16 data and user-defined processing and download of model-ready, meteorological input for any regional CMAQ domain worldwide. The NACC-Cloud system was implemented on the Amazon® Web Services High-Performance Computing platform, and results from this work show that the NACC-Cloud system is immediately beneficial to the air quality modeling community worldwide.

HPC+ in the medical field: Overview and current examples.

To say data is revolutionising the medical sector would be a vast understatement. The amount of medical data available today is unprecedented and has the potential to enable to date unseen forms of healthcare. To process this huge amount of data, an equally huge amount of computing power is required, which cannot be provided by regular desktop computers. These areas can be (and already are) supported by High-Performance-Computing (HPC), High-Performance Data Analytics (HPDA), and AI (together "HPC+"). This overview article aims to show state-of-the-art examples of studies supported by the National Competence Centres (NCCs) in HPC+ within the EuroCC project, employing HPC, HPDA and AI for medical applications. The included studies on different applications of HPC in the medical sector were sourced from the National Competence Centres in HPC and compiled into an overview article. Methods include the application of HPC+ for medical image processing, high-performance medical and pharmaceutical data analytics, an application for pediatric dosimetry, and a cloud-based HPC platform to support systemic pulmonary shunting procedures. This article showcases state-of-the-art applications and large-scale data analytics in the medical sector employing HPC+ within surgery, medical image processing in diagnostics, nutritional support of patients in hospitals, treating congenital heart diseases in children, and within basic research. HPC+ support scientific fields from research to industrial applications in the medical area, enabling researchers to run faster and more complex calculations, simulations and data analyses for the direct benefit of patients, doctors, clinicians and as an accelerator for medical research.

Sparse matrix multiplication in a record-low power self-rectifying memristor array for scientific computing.

Memristor-enabled in-memory computing provides an unconventional computing paradigm to surpass the energy efficiency of von Neumann computers. Owing to the limitation of the computing mechanism, while the crossbar structure is desirable for dense computation, the system's energy and area efficiency degrade substantially in performing sparse computation tasks, such as scientific computing. In this work, we report a high-efficiency in-memory sparse computing system based on a self-rectifying memristor array. This system originates from an analog computing mechanism that is motivated by the device's self-rectifying nature, which can achieve an overall performance of ~97 to ~11 TOPS/W for 2- to 8-bit sparse computation when processing practical scientific computing tasks. Compared to previous in-memory computing system, this work provides over 85 times improvement in energy efficiency with an approximately 340 times reduction in hardware overhead. This work can pave the road toward a highly efficient in-memory computing platform for high-performance computing.

Solving the electronic structure problem for over 100 000 atoms in real space

Using a real-space high-order finite-difference approach, we investigate the electronic structure of large spherical silicon nanoclusters. Within Kohn-Sham density functional theory and using pseudopotentials, we report the self-consistent field convergence of a system with over 100 000 atoms: a ${\mathrm{Si}}_{107,641}{\mathrm{H}}_{9,084}$ nanocluster with a diameter of 16 nm. Our approach uses Chebyshev-filtered subspace iteration to speed up the convergence of the eigenspace, and blockwise Hilbert space-filling curves to speed up sparse matrix-vector multiplications, all of which are implemented in the parsec code. For the largest system, we utilized 2048 nodes (114 688 cores) on the Frontera machine in the Texas Advanced Computing Center. Our quantitative analysis of the electronic structure shows how it gradually approaches its bulk counterpart as a function of nanocluster size. The band gap is enlarged due to quantum confinement in nanoclusters, but decreases as the system size increases, as expected. Our work serves as a proof of concept for the capacity of the real-space approach in efficiently parallelizing very large calculations using high-performance computer platforms, which can straightforwardly be replicated in other systems with more than ${10}^{5}$ atoms.

Energy-Efficient Parallel Computing: Challenges to Scaling

The energy consumption of Information and Communications Technology (ICT) presents a new grand technological challenge. The two main approaches to tackle the challenge include the development of energy-efficient hardware and software. The development of energy-efficient software employing application-level energy optimization techniques has become an important category owing to the paradigm shift in the composition of digital platforms from single-core processors to heterogeneous platforms integrating multicore CPUs and graphics processing units (GPUs). In this work, we present an overview of application-level bi-objective optimization methods for energy and performance that address two fundamental challenges, non-linearity and heterogeneity, inherent in modern high-performance computing (HPC) platforms. Applying the methods requires energy profiles of the application’s computational kernels executing on the different compute devices of the HPC platform. Therefore, we summarize the research innovations in the three mainstream component-level energy measurement methods and present their accuracy and performance tradeoffs. Finally, scaling the optimization methods for energy and performance is crucial to achieving energy efficiency objectives and meeting quality-of-service requirements in modern HPC platforms and cloud computing infrastructures. We introduce the building blocks needed to achieve this scaling and conclude with the challenges to scaling. Briefly, two significant challenges are described, namely fast optimization methods and accurate component-level energy runtime measurements, especially for components running on accelerators.

Scalable Data Processing Platform for Earth Observation Data Repositories

Earth observation (EO) satellite data is essential to environmental monitoring. At a national and regional level, the open data cubes harness the power of satellite data by providing application programming interfaces and services to the end-users. The volume and the complexity of satellite observations are increasing, demanding novel approaches for data storing, managing, and processing. High-performance computing (HPC) and cloud platforms may improve Big EO data processing performance. However, it is necessary to consider several vital aspects for efficient and flexible EO data processing, such as the interoperability from cloud-HPC and EO data repositories, automatic provisioning and scaling of cloud-HPC resources, cost-effectiveness, support of new EO data formats and open-source packages, or linkage with data cube platforms. The article proposes a scalable EO data processing platform interoperable from cloud-HPC and EO data repositories. The platform enables linking any data repository supporting web coverage service or SpatioTemporal Asset Catalog Application Programming Interfaces (STAC-API), and any cloud or HPC resource supporting scheduling system API for providing access to the cluster backends.

Framework for Performance Enhancement of MPI based Application on Cloud

Cloud technology is a major revolution that has happened in the computing era that has changed the way applications and resources are used. Elasticity is the key characteristic of the cloud, wherein the required number of resources are provided as a service with a pay-as-you-go principle. This reduces the huge cost involved in buying, installing, and maintaining the resources. Cloud computing, with its highly scalable resources, can be a good platform for High-Performance Computing (HPC). HPC application performance highly depends on the quantity of the resources, which makes the cloud a suitable candidate. But the HPC community is not very happy with cloud technology, and most of the users still think cloud technology is not suitable for HPC applications. But virtualization technology, which is the foundation of the cloud, degrades the performance of applications in the urge to improve utilization. The hypervisor layer and resource sharing by the virtual machines (VMs) hosted on the same node are the main reasons for performance degradation in the cloud. The majority of the HPC applications belong to the message passing (MPI) category, and for these applications, communication cost is the major stakeholder in deciding performance. If these applications are hosted on the cloud, it leads to further performance degradation as the process on a VM communicates through the virtual network interface, which in turn shares the network interface of the host machine with other VMs. MPI-based applications hosted on MPPs work in a bandwidth shared environment as multiple processes communicate over the same network. But in the cloud, as the number of VMs increases, per node bandwidth availability per communication reduces. To address the above issues, we have built a framework to enhance the performance of MPI-based HPC applications on VMs by considering proper VM placement strategy and resource reservation policies, with knowledge of resource availability and process communication patterns. A VM placement strategy for dynamic clustering of VMs with high priority for shared memory-based communication is proposed and tested. Results show that with a medium number of processes, there is an improvement of around 70% with our placement strategy for high data communicating processes. If there are fewer processes, and the single physical node can hold all the VMs, then the performance improvement is up to 500%.

射电脉冲星搜索的优化方法

The connection between astronomy and high-performance computing is becoming stronger with the development of innovative observing facilities such as the Square Kilometre Array (SKA) and the proposed innovative platform for big data and high-performance computing. Astronomical computation is characterized by a large data volume and massive parallelism, particularly for pulsar search, a leading scientific direction of the SKA. In this study, we present an approach for accelerating the pulsar search pipeline based on OpenMP and multiprocessing techniques. We propose a method for solving the load imbalance problem and have successfully installed the pipeline on x86 and ARM compute nodes on the China SKA Regional Centre prototype (CSRC-P). The performance evaluation from tests on the Murchison Widefield Array (MWA) VCS observations shows that our optimization method works well on the x86 and ARM nodes, improving the relative speedup by factors of 10.4–12.2 and 24.5–25.8, respectively, compared with the original single-thread approach. The ARM platform was 1.1–1.3 times faster than the x86 platform in the tested cases, showing its great potential for SKA data processing. This optimized pulsar search pipeline deployed on the CSRC-P will be used for the pulsar survey of the southern-sky MWA rapid two-meter program for various scientific goals, including pulsar timing arrays for gravitational wave detection.

Adaptive elasticity policies for staging-based in situ visualization

In situ processing aims to alleviate the growing gap between computation and I/O capabilities by performing data processing close to the data source. In situ processing is widely used to process data generated by multiple data sources, including observation data from edge devices or scientific observational facilities and the simulation data generated by scientific computation on a high-performance computing (HPC) platform. For a scientific workflow that is run on an HPC platform and composed of a simulation program and an in situ data analytics or visualization (abbreviated as ana/vis) task, there is an implicit assumption that the computing resources assigned to the workflow keep static during the workflow execution. However, with the converging trend between the HPC and cloud computing platform, running the in situ ana/vis task in an elastic way is promising to decrease its overhead and improve its resource utilization rate. Resource elasticity represents the ability to change resource configurations such as the number of computing nodes/processes during workflow execution. An elastic job may dynamically adjust resource configurations; it may use a few resources at the beginning and more resources toward the end of the job when interesting data appear. However, it is hard to predict a priori how many computing nodes/processes need to be added/removed during the workflow execution to adapt to changing workflow needs. How to efficiently guide elasticity operations, such as growing or shrinking the number of processes used for in situ analysis during workflow execution, is an open-ended research question. In this article, we present adaptive elasticity policies that adopt workflow runtime information collected during workflow execution to predict how to trigger the addition/removal of processes in order to minimize in situ processing overhead. Taking in situ visualization tasks as an example, we integrate the presented elasticity policies into a staging-based elastic workflow and evaluate its efficiency in multiple elasticity scenarios. Compared with the situation without elasticity or with a static elasticity policy that uses a fixed number of processes for each rescaling operation, the adaptive elasticity policy can save overhead in finding a proper resource configuration and improve resource utilization efficiency. One experiment illustrates that the adaptive elasticity policy saves 41% of core-hours compared with the situation without the resource elasticity.