HPC Platforms Research Articles

Viability Study of SYCL as a Unified Programming Model for Heterogeneous Systems Based on GPUs in Bioinformatics

Bioinformatics and Computational Biology are disciplines that have used GPUs for over two decades to accelerate data processing in computational applications, where CUDA has been the most utilized programming language. However, the exclusive use of CUDA presents a portability issue, as it is only compatible with NVIDIA GPUs and not with other heterogeneous architectures, such as AMD or Intel GPUs, or any other type of accelerator. To address this portability challenge, the Khronos Group recently introduced the SYCL standard, a multi-platform programming model that offers a high-level programming interface. This standard facilitates the development of portable applications that can efficiently leverage the capabilities of different hardware devices, such as NVIDIA, Intel, and AMD GPUs, without needing significant modifications to the source code. Therefore, this thesis set out as its general objective to evaluate the feasibility of SYCL as a unified, portable, and efficient heterogeneous programming model for the design and development of computationally demanding applications on heterogeneous GPU-based systems, specifically in the field of bioinformatics. Initially, a detailed investigation was conducted about heterogeneous programming models, performance metrics, and bioinformatics concepts, in order to establish the theoretical foundations of this thesis. Then, the SW# suite was chosen as the case study, as it represents a clear example of a CUDA-based bioinformatics application for biological sequence alignment. Using the SYCLomatic tool, a complete migration of the CUDA code to SYCL was carried out, which involved modifying the generated code and solving runtime errors. Additionally, the functionality was verified, optimizations were applied, and the resulting SYCL code was standardized to be compatible with other SYCL implementations. Subsequently, multiple experiments were conducted to evaluate the functionality and performance portability of the software migrated to SYCL. These experiments involved running the application on a wide variety of HPC platforms, including different CPUs and GPUs from various manufacturers. The results obtained demonstrated performance comparable to CUDA in most configurations, confirming the effectiveness of SYCL. Moreover, good performance portability across platforms was observed due to SYCL's ability to run on various hardware combinations. Furthermore, performance remained consistent when switching SYCL implementations. In conclusion, this study demonstrates that SYCL is a viable alternative as a unified, portable, and efficient programming model in the context of heterogeneous computing with GPUs for bioinformatics applications. The findings of this study lay the groundwork for the transition of legacy applications and the development of new solutions leveraging the capabilities of SYCL.

Critical collapse of massless scalar fields in asymptotically anti-de Sitter spacetime* *Supported by the National Natural Science Foundation of China (11925503) and the Guangdong Major project of Basic and Applied Basic Research (2019B030302001). The computation is completed on the HPC Platform of Huazhong University of Science and Technology

We conduct numerical investigations on the critical collapse of spherically symmetric massless scalar fields in asymptotically anti-de Sitter spacetime. Our primary focus is on the behavior of the critical amplitude under various initial configurations of the scalar field. Through our numerical results, we obtain a formula that determines critical amplitude in terms of cosmological constant Λ: , where σ denotes the initial width of the scalar field and is the initial position of the scalar field. Notably, we highlight that the slope of this linear relationship depends on the initial configuration of the scalar field.

Parallel quantum computing simulations via quantum accelerator platform virtualization

Quantum circuit execution is a central task in quantum computation. Due to inherent quantum-mechanical constraints, quantum computing workflows often involve a considerable number of independent measurements over a large set of slightly different quantum circuits. Here we discuss a simple model for parallelizing such quantum circuit executions that is based on introducing a large array of virtual quantum processing units (mapped to HPC nodes in our case) as a parallel quantum computing platform. Implemented within the XACC framework, the model can readily take advantage of its backend-agnostic features, enabling parallel quantum computing/simulation over any target backend supported by XACC. We illustrate the performance of this approach by demonstrating strong scaling in two pertinent domain science problems, namely in computing the gradients for the multi-contracted variational quantum eigensolver and in data-driven quantum circuit learning, where we vary the number of qubits and the number of circuit layers. The latter simulation leverages the cuQuantum library to run efficiently on GPU-accelerated HPC platforms.

Checkpointing Strategies to Tolerate Non-Memoryless Failures on HPC Platforms

This article studies checkpointing strategies for parallel applications subject to failures. The optimal strategy to minimize total execution time, or makespan, is well known when failure IATs obey an Exponential distribution, but it is unknown for non-memoryless failure distributions. We explain why the latter fact is misunderstood in recent literature. We propose a general strategy that maximizes the expected efficiency until the next failure, and we show that this strategy achieves an asymptotically optimal makespan, thereby establishing the first optimality result for arbitrary failure distributions. Through extensive simulations, we show that the new strategy is always at least as good as the Young/Daly strategy for various failure distributions. For distributions with high infant mortality (such as LogNormal with shape parameter k =2.51 or Weibull with shape parameter 0.5), the execution time is divided by a factor of 1.9 on average, and up to a factor 4.2 for recently deployed platforms.

Improving batch schedulers with node stealing for failed jobs

SummaryAfter a machine failure, batch schedulers typically re‐schedule the job that failed with a high priority. This is fair for the failed job but still requires that job to re‐enter the submission queue and to wait for enough resources to become available. The waiting time can be very long when the job is large and the platform highly loaded, as is the case with typical HPC platforms. We propose another strategy: when a job fails, if no platform node is available, we steal one node from another job , and use it to continue the execution of despite the failure. In this work, we give a detailed assessment of this node stealing strategy using traces from the Mira supercomputer at Argonne National Laboratory. The main conclusion is that node stealing improves the utilization of the platform and dramatically reduces the flow of large jobs, at the price of slightly increasing the flow of small jobs.

A high-performance computational workflow to accelerate GATK SNP detection across a 25-genome dataset

BackgroundSingle-nucleotide polymorphisms (SNPs) are the most widely used form of molecular genetic variation studies. As reference genomes and resequencing data sets expand exponentially, tools must be in place to call SNPs at a similar pace. The genome analysis toolkit (GATK) is one of the most widely used SNP calling software tools publicly available, but unfortunately, high-performance computing versions of this tool have yet to become widely available and affordable.ResultsHere we report an open-source high-performance computing genome variant calling workflow (HPC-GVCW) for GATK that can run on multiple computing platforms from supercomputers to desktop machines. We benchmarked HPC-GVCW on multiple crop species for performance and accuracy with comparable results with previously published reports (using GATK alone). Finally, we used HPC-GVCW in production mode to call SNPs on a “subpopulation aware” 16-genome rice reference panel with ~ 3000 resequenced rice accessions. The entire process took ~ 16 weeks and resulted in the identification of an average of 27.3 M SNPs/genome and the discovery of ~ 2.3 million novel SNPs that were not present in the flagship reference genome for rice (i.e., IRGSP RefSeq).ConclusionsThis study developed an open-source pipeline (HPC-GVCW) to run GATK on HPC platforms, which significantly improved the speed at which SNPs can be called. The workflow is widely applicable as demonstrated successfully for four major crop species with genomes ranging in size from 400 Mb to 2.4 Gb. Using HPC-GVCW in production mode to call SNPs on a 25 multi-crop-reference genome data set produced over 1.1 billion SNPs that were publicly released for functional and breeding studies. For rice, many novel SNPs were identified and were found to reside within genes and open chromatin regions that are predicted to have functional consequences. Combined, our results demonstrate the usefulness of combining a high-performance SNP calling architecture solution with a subpopulation-aware reference genome panel for rapid SNP discovery and public deployment.

Parallel Deployment and Performance Analysis of a Multi-Hop Routing Protocol for 5G Backhaul Networks Using Cloud and HPC Platforms

Parallel Deployment and Performance Analysis of a Multi-Hop Routing Protocol for 5G Backhaul Networks Using Cloud and HPC Platforms

Presenting the multi-phase solver implemented in the open source TrioCFD code based on the TRUST HPC platform

A new open source CFD-scale multi-phase flow environment has been developed in the TrioCFD code using the TRUST HPC platform. Mass, momentum and energy equations can be solved for an arbitrary number of phases. This environment is scalable, can use multiple numerical schemes and resolution algorithms, and enables easy modification of the physical closures. It has been verified on adiabatic and boiling two-phase simulations.

Performances of the CFD open-source HPC platform TRUST on GPUs

TRUST is a versatile open-source CFD tool developed by the CEA since 1993. Initially designed for nuclear applications, TRUST has evolved to tackle a range of thermohydraulic challenges, from one-phase to multiphase flows. It offers various numerical methods and supports different mesh types for efficient computation on diverse computing platforms, including high-performance computers. Recently, efforts have been made to integrate GPU computing libraries like AmgX, rocALUTION, and Kokkos aiming for a hybrid CPU/GPU code achieving better performance portability. This paper provides an overview of the TRUST platform, discusses its GPU computing strategy, and presents selected associated results.

Specification and Description Language Models Automatic Execution in a High-Performance Environment

Specification and Description Language (SDL) is a language that can represent the behavior and structure of a model completely and unambiguously. It allows the creation of frameworks that can run a model without the need to code it in a specific programming language. This automatic process simplifies the key phases of model building: validation and verification. SDLPS is a simulator that enables the definition and execution of models using SDL. In this paper, we present a new library that enables the execution of SDL models defined on SDLPS infrastructure on a HPC platform, such as a supercomputer, thus significantly speeding up simulation runtime. Moreover, we apply the SDL language to a social science use case, thus opening a new avenue for facilitating the use of HPC power to new groups of users. The tools presented here have the potential to increase the robustness of modeling software by improving the documentation, verification, and validation of the models.

Performance on HPC Platforms Is Possible Without C++

Performance on HPC Platforms Is Possible Without C++

A methodology for selecting a performance-convenient ABMS development framework on HPC platforms

Agent-based modeling and simulation (ABMS) is an approach for simulating the actions and interactions of autonomous agents. Such interactions occur within a defined environment to assess their effects on a system as a whole. Depending on the complexity of the model and the number of simulated agents, an ABMS application may require a significant amount of computational resources. It makes them good candidates to be parallelized on HPC systems. However, most developers of ABMS simulators are experts in the specific simulation domain, but they lack the expertise to develop parallel applications. Consequently, several frameworks for generating HPC ABMS applications have been developed, and it may now be challenging for these non-expert users to choose which of these frameworks would provide the best performing simulator for a particular model. This paper presents a methodology that uses a benchmark to help non-expert users to select the most suitable framework to generate the best performing parallel implementation for a given ABMS model. Such a benchmark considers the common characteristics of parallel ABMS applications and includes parameters for influencing their relevant performance aspects. The methodology is based on defining a set of problem classes that represent the majority of known ABMS models and systematically conducting a series of experiments to determine which framework offers the best performance for each class. Then, users only need to identify the class that closely aligns with their model to make an informed decision regarding the appropriate development framework. The methodology is used to assess well-known ABMS parallel development frameworks (FLAME, RepastHPC, and DMASON) on real HPC platforms. The obtained results are validated using a real application for infection and contact tracing modeling.

Critique of “A Parallel Framework for Constraint-Based Bayesian Network Learning via Markov Blanket Discovery” by SCC Team From ShanghaiTech University

In SC20, Srivastava <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">et al.</i> [6] proposed a Parallel F <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">ram</b> ework for <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">B</b> ayesian <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Le</b> arning, or ramBLe, for short, which is a highly parallel and efficient framework for learning the structure of Bayesian Networks (BNs) from samples, particularly large genome-scale networks. As part of our participation in the SC21 Student Cluster Competition, our task was to verify conclusions from the original work [6]. Here we present the outcome of our experiments, which were performed on a four-node cluster from the Oracle Cloud HPC platform. We reproduce the numerical results from [6], namely the algorithm's performance and scaling behavior using MPI and different Python and Boost libraries on the Oracle cloud.

Versatile software-defined HPC and cloud clusters on Alps supercomputer for diverse workflows

Supercomputers have been driving innovations for performance and scaling benefiting several scientific applications for the past few decades. Yet their ecosystems remain virtually unchanged when it comes to integrating distributed data-driven workflows, primarily due to rather rigid access methods and restricted configuration management options. X-as-a-Service model of cloud has introduced, among other features, a developer-centric DevOps approach empowering developers of infrastructure, platform to software artefacts, which, unfortunately contemporary supercomputers still lack. We introduce vClusters (versatile software-defined clusters), which is based on Infrastructure-as-code (IaC) technology. vClusters approach is a unique fusion of HPC and cloud technologies resulting in a software-defined, multi-tenant cluster on a supercomputing ecosystem, that, together with software-defined storage, enable DevOps for complex, data-driven workflows like grid middleware, alongside a classic HPC platform. IaC has been a commonplace in cloud computing, however, it lacked adoption within multi-Petascale ecosystems due to concerns related to performance and interoperability with classic HPC data centres’ ecosystems. We present an overview of the Swiss National Supercomputing Centre’s flagship Alps ecosystem as an implementation target for vClusters for HPC and data-driven workflows. Alps is based on the Cray-HPE Shasta EX supercomputing platform that includes an IaC compliant, microservices architecture (MSA) management system, which we leverage for demonstrating vClusters usage for our diverse operational workflows. We provide implementation details of two operational vClusters platforms: a classic HPC platform that is used predominantly by hundreds of users running thousands of large-scale numerical simulations batch jobs; and a widely used, data-intensive, Grid computing middleware platform used for CERN Worldwide LHC Computing Grid (WLCG) operations. The resulting solution showcases reuse and reduction of common configuration recipes across vCluster implementations, minimising operational change management overheads while introducing flexibility for managing artefacts for DevOps required by diverse workflows.

Towards practical and massively parallel quantum computing emulation for quantum chemistry

Quantum computing is moving beyond its early stage and seeking for commercial applications in chemical and biomedical sciences. In the current noisy intermediate-scale quantum computing era, the quantum resource is too scarce to support these explorations. Therefore, it is valuable to emulate quantum computing on classical computers for developing quantum algorithms and validating quantum hardware. However, existing simulators mostly suffer from the memory bottleneck so developing the approaches for large-scale quantum chemistry calculations remains challenging. Here we demonstrate a high-performance and massively parallel variational quantum eigensolver (VQE) simulator based on matrix product states, combined with embedding theory for solving large-scale quantum computing emulation for quantum chemistry on HPC platforms. We apply this method to study the torsional barrier of ethane and the quantification of the protein–ligand interactions. Our largest simulation reaches 1000 qubits, and a performance of 216.9 PFLOP/s is achieved on a new Sunway supercomputer, which sets the state-of-the-art for quantum computing emulation for quantum chemistry.

An OpenMP GPU-offload implementation of a non-equilibrium solidification cellular automata model for additive manufacturing

In this paper, performance strategies on GPU-based HPC platforms of a cellular automata (CA) simulation code for non-equilibrium solidification, including nucleation, grain growth, solute partitioning and transport for the metal additive manufacturing (AM) process are investigated using OpenMP 4.5. To accurately report the speed-up for multicore CPUs and GPUs, a rigorous performance analysis employed optimizations appropriate for both CPU-only code (baseline) and GPU offload codes for an isothermal test problem. The performance results on Summit at the Oak Ridge Leadership Computing Facility indicate that using a precomputed list of interface cells significantly decreased the wall-clock time on GPUs. The speedup due to GPU acceleration was evaluated for a full Summit node and measured to be 1.8X when comparing a 6 MPI tasks run with 6 GPUs versus 36 MPI tasks on the CPU only. That speed-up was found to be 7.9X when comparing 6 MPI tasks with 6 GPUs versus the 6 MPI tasks running on the CPU only. Performance measurements showed that system total time is almost constant for runs with more than 96 MPI tasks (or GPUs), indicating that the GPU-accelerated code showed an excellent weak scaling performance.Finally, a rapid directional solidification problem was considered to demonstrate the CA code capability on Summit. It was found that a mesh size of at least 0.05 μm is recommended for the AM-like simulations in order to obtain accurate elongated grain microstructure and elongated subgrain features, which are in qualitative good agreement with experimental data. The results presented in this study indicate that the performance strategies on GPU-based HPC platforms for the CA code are appropriate for novel HPC exascale platforms.

Design and Performance Characterization of RADICAL-Pilot on Leadership-Class Platforms

Many extreme scale scientific applications have workloads comprised of a large number of individual high-performance tasks. The Pilot abstraction decouples workload specification, resource management, and task execution via job placeholders and late-binding. As such, suitable implementations of the Pilot abstraction can support the collective execution of large number of tasks on supercomputers. We introduce RADICAL-Pilot (RP) as a portable, modular and extensible pilot-enabled runtime system. We describe RP's design, architecture and implementation. We characterize its performance and show its ability to scalably execute workloads comprised of tens of thousands heterogeneous tasks on DOE and NSF leadership-class HPC platforms. Specifically, we investigate RP's weak/strong scaling with CPU/GPU, single/multi core, (non)MPI tasks and Python functions when using most of ORNL Summit and TACC Frontera. RADICAL-Pilot can be used stand-alone, as well as the runtime for third-party workflow systems.

Improving the performance of batch schedulers using online job runtime classification

Job scheduling in high-performance computing platforms is a hard problem that involves uncertainties on both the job arrival process and their execution times. Users typically provide only loose upper bounds for job execution times, which are not so useful for scheduling heuristics based on processing times. Previous studies focused on applying regression techniques to obtain better execution time estimates, which worked reasonably well and improved scheduling metrics. However, these approaches require a long period of training data.In this work, we propose a simpler approach by classifying jobs as small or large and prioritizing the execution of small jobs over large ones. Indeed, small jobs are the most impacted by queuing delays, but they typically represent a light load and incur a small burden on the other jobs. The classifier operates online and learns by using data collected over the previous weeks, facilitating its deployment and enabling a fast adaptation to changes in the workload characteristics.We evaluate our approach using four scheduling policies on seven HPC platform workload traces. We show that: first, incorporating such classification reduces the average bounded slowdown of jobs in all scenarios, second, in most considered scenarios, the improvements are comparable to the ideal hypothetical situation where the scheduler would know in advance the exact running time of jobs.

Comparing Distributed Termination Detection Algorithms for Modern HPC Platforms

Comparing Distributed Termination Detection Algorithms for Modern HPC Platforms

Evaluation of Clustering Algorithms on HPC Platforms

Clustering algorithms are one of the most widely used kernels to generate knowledge from large datasets. These algorithms group a set of data elements (i.e., images, points, patterns, etc.) into clusters to identify patterns or common features of a sample. However, these algorithms are very computationally expensive as they often involve the computation of expensive fitness functions that must be evaluated for all points in the dataset. This computational cost is even higher for fuzzy methods, where each data point may belong to more than one cluster. In this paper, we evaluate different parallelisation strategies on different heterogeneous platforms for fuzzy clustering algorithms typically used in the state-of-the-art such as the Fuzzy C-means (FCM), the Gustafson–Kessel FCM (GK-FCM) and the Fuzzy Minimals (FM). The experimental evaluation includes performance and energy trade-offs. Our results show that depending on the computational pattern of each algorithm, their mathematical foundation and the amount of data to be processed, each algorithm performs better on a different platform.