High Performance Computing Applications Research Articles

Practicable live container migrations in high performance computing clouds: Diskless, iterative, and connection-persistent

Checkpoint/Restore techniques had been thoroughly used by the High Performance Computing (HPC) community in the context of failure recovery. Given the current trend in HPC to use containerization to obtain fast, customized, portable, flexible, and reproducible deployments of their workloads, as well as efficient and reliable sharing and management of HPC Cloud infrastructures, there is a need to integrate Checkpoint/Restore with containerization in such a way that the freeze time of the application is minimal and live migrations are practicable. Whereas current Checkpoint/Restore tools (such as CRIU) support several options to accomplish this, most of them are rarely exploited in HPC Clouds and, consequently, their potential impact on the performance is barely known. Therefore, this paper explores the use of CRIU’s advanced features to implement diskless, iterative (pre-copy and post-copy) migrations of containers with external network namespaces and established TCP connections, so that memory-intensive and connection-persistent HPC applications can live-migrate. Our extensive experiments to characterize the performance impact of those features demonstrate that properly-configured live migrations incur low application downtime and memory/disk usage and are indeed feasible in containerized HPC Clouds.

Ion-molecule collision cross-section calculations using trajectory parallelization in distributed systems

Ion Mobility coupled with Mass Spectrometry (IM-MS) stands as a strong analytical method for structurally characterizing complex molecules. In IM-MS, the sample under investigation is ionized and propelled by an electric field into a drift tube, which collides against a buffer gas. The separation of the ion gas phase is then measured through the differences in their rotationally averaged Collision Cross-Section (CCS) values. The effectiveness of the measured Collision Cross-Section (CCS) for structural characterization critically depends on the validation against theoretical calculations. This validation process relies on intensive molecular mechanics simulations, which can be computationally demanding, especially for large systems such as molecular assemblies and viruses. Therefore, reliable and fast CCS calculations are needed to help interpret IM-MS experimental data. This work presents the MassCCS software, which considerably increases the CCS simulation performance by implementing a linked-cell-based algorithm, incorporating High-Performance Computing (HPC) techniques. We performed extensive tests regarding the system size, shape, and number of CPU cores. Experimental results reveal speedups up to 3 orders of magnitude faster than Collision Simulator for Ion Mobility Spectrometry (CoSIMS) and High-Performance Collision Cross Section (HPCCS), optimized solutions for CCS simulations, for a single node execution. In addition, we extended MassCCS at the inter-node level by employing OpenMP Cluster (OMPC). OMPC is an innovative programming model designed for the development of HPC applications. It streamlines the development process and simplifies software maintenance using only OpenMP directives. Notably, OMPC delivers a performance level comparable to a pure MPI implementation. This enhancement enabled expensive CCS calculations using nitrogen buffer gas for large systems such as human adenovirus with ∼11 million atoms in just ∼4 min, making MassCCS the most performant software nowadays, to the best of our knowledge. MassCCS is available as free software for Academic use at https://github.com/cces-cepid/massccs.

SHREG: Mitigating register redundancy in GPUs

Graphics Processing Units (GPUs) have become dominant accelerators for Machine Learning (ML) and High-Performance Computing (HPC) applications due to their massive parallelism capabilities, through the utilization of general matrix-to-matrix multiplication (GEMM) kernels. However, GEMM kernels often suffer from duplicated memory requests, mainly caused by matrix tiling used for handling large matrices. While GPUs have adopted programmable shared memory to mitigate this issue by preserving frequently reused data in shared memory, GEMM still introduces duplication in register files. Our observations show that the matrix tiling issues memory requests to the same shared memory address for neighboring threads, and this results in a substantial increase in the number of duplicated data in the register files. Such duplication degrades GPU performance by limiting warp-level parallelism due to the register shortage and redundant memory requests to shared memory. We find that the data duplication can be categorized into two types that occur with fixed patterns during the matrix tiling. Based on these observations, we introduce SHREG, an architecture design that enables different threads to share registers for overlapped data from shared memory, effectively reducing duplicated data within the register files. By leveraging the duplication patterns, SHREG utilizes register sharing and improves performance with minimal hardware overhead. Our evaluation shows that SHREG improves performance by 31.4% on various ML applications over the baseline GPU.

Tarazu: An Adaptive End-to-end I/O Load-balancing Framework for Large-scale Parallel File Systems

The imbalanced I/O load on large parallel file systems affects the parallel I/O performance of high-performance computing (HPC) applications. One of the main reasons for I/O imbalances is the lack of a global view of system-wide resource consumption. While approaches to address the problem already exist, the diversity of HPC workloads combined with different file striping patterns prevents widespread adoption of these approaches. In addition, load-balancing techniques should be transparent to client applications. To address these issues, we propose Tarazu , an end-to-end control plane where clients transparently and adaptively write to a set of selected I/O servers to achieve balanced data placement. Our control plane leverages real-time load statistics for global data placement on distributed storage servers, while our design model employs trace-based optimization techniques to minimize latency for I/O load requests between clients and servers and to handle multiple striping patterns in files. We evaluate our proposed system on an experimental cluster for two common use cases: the synthetic I/O benchmark IOR and the scientific application I/O kernel HACC-I/O. We also use a discrete-time simulator with real HPC application traces from emerging workloads running on the Summit supercomputer to validate the effectiveness and scalability of Tarazu in large-scale storage environments. The results show improvements in load balancing and read performance of up to 33% and 43%, respectively, compared to the state-of-the-art.

Ferroelectrically tuned tunneling photodetector based on graphene/h-BN/In2Se3 heterojunction

Transition metal dichalcogenides (TMDs) show promise for photodetection across various wavelengths. However, photoconductive devices often face various limitations such as low photoresponsivity or high dark current. This study introduces a highly sensitive photodetector using a graphene/h-BN/In2Se3 heterostructure. A substantial electron barrier (2.72 eV) at the graphene/h-BN junction effectively suppresses dark current, while the small hole barrier (1.39 eV) at the h-BN/In2Se3 junction facilitates efficient tunneling of photocarriers under illumination. The photodetector demonstrates a fast response time (Tr = 15 ms, Td = 5 ms), outstanding responsivity (1.18 × 104 μA/W), and detectivity (1.74 × 1010 Jones) at 4 V bias and 637 nm light illumination. Spectral measurements reveal a broad detection range from visible to near-infrared (450–1310 nm). Furthermore, leveraging the non-volatile ferroelectric properties of In2Se3 allows modulation of the barrier height, enabling non-volatile tunable photoconductivity. This study represents a significant advancement in ferroelectric based optoelectronic detection devices, providing strong support for applications in high-performance computing, neural networks, and related domains.

High Speed Transmission Characteristics on Glass Based Interposers

Interposers for system in package will became more and more important for advanced electronic systems. As the industry moves toward HPC (High Performance Computing) for huge data transmission with low power consumption. The major engineering requirements for HPC application are highdensity, high speed data transmission, low loss, precision manufacturing and low cost. On the 2.5D heterogenous package structure, key technology is fine wiring connected with CPU chip and HBM chips. The fine wires needed high speed transmission. This paper presents the demonstration of Glass-based twin type interposers, Glass interposer with fine pitch metalized through via and RDL interposer with low loss dielectrics on Glass carrier. We compare twin types of Interposes fabrication process capability with panel size format 300x400mm. Finally, we compare the transmission characteristics using eye diagram for heterogenous integration application. Glass Interposer has low loss glass via, it is effective vertical interconnection. RDL interposer has low loss dielectric layers, excellent transmission characteristics obtained on fine pitch trace area.

Malleability techniques applications in high-performance computing

Malleability techniques applications in high-performance computing

FCBench: Cross-Domain Benchmarking of Lossless Compression for Floating-Point Data

While both the database and high-performance computing (HPC) communities utilize lossless compression methods to minimize floating-point data size, a disconnect persists between them. Each community designs and assesses methods in a domain-specific manner, making it unclear if HPC compression techniques can benefit database applications or vice versa. With the HPC community increasingly leaning towards in-situ analysis and visualization, more floating-point data from scientific simulations are being stored in databases like Key-Value Stores and queried using in-memory retrieval paradigms. This trend underscores the urgent need for a collective study of these compression methods' strengths and limitations, not only based on their performance in compressing data from various domains but also on their runtime characteristics. Our study extensively evaluates the performance of eight CPU-based and five GPU-based compression methods developed by both communities, using 33 real-world datasets assembled in the Floating-point Compressor Benchmark (FCBench). Additionally, we utilize the roofline model to profile their runtime bottlenecks. Our goal is to offer insights into these compression methods that could assist researchers in selecting existing methods or developing new ones for integrated database and HPC applications.

Redundancy-free integrated optical convolver for optical neural networks based on arrayed waveguide grating

Abstract Optical neural networks (ONNs) have gained significant attention due to their potential for high-speed and energy-efficient computation in artificial intelligence. The implementation of optical convolutions plays a vital role in ONNs, as they are fundamental operations within neural network architectures. However, state-of-the-art convolution architectures often suffer from redundant inputs, leading to substantial resource waste. Here, we demonstrate an integrated optical convolution architecture that leverages the inherent routing principles of arrayed waveguide grating (AWG) to execute the sliding of convolution kernel and summation of results. M × N multiply–accumulate (MAC) operations are facilitated by M + N units within a single clock cycle, thus eliminating the redundancy. In the experiment, we achieved 5 bit precision and 91.9 % accuracy in the handwritten digit recognition task confirming the reliability of our approach. Its redundancy-free architecture, low power consumption, high compute density (8.53 teraOP mm−1 s−1) and scalability make it a valuable contribution to the field of optical neural networks, thereby paving the way for future advancements in high-performance computing and artificial intelligence applications.

A Visual Comparison of Silent Error Propagation.

High-performance computing (HPC) systems play a critical role in facilitating scientific discoveries. Their scale and complexity (e.g., the number of computational units and software stack) continue to grow as new systems are expected to process increasingly more data and reduce computing time. However, with more processing elements, the probability that these systems will experience a random bit-flip error that corrupts a program's output also increases, which is often recognized as silent data corruption. Analyzing the resiliency of HPC applications in extreme-scale computing to silent data corruption is crucial but difficult. An HPC application often contains a large number of computation units that need to be tested, and error propagation caused by error corruption is complex and difficult to interpret. To accommodate this challenge, we propose an interactive visualization system that helps HPC researchers understand the resiliency of HPC applications and compare their error propagation. Our system models an application's error propagation to study a program's resiliency by constructing and visualizing its fault tolerance boundary. Coordinating with multiple interactive designs, our system enables domain experts to efficiently explore the complicated spatial and temporal correlation between error propagations. At the end, the system integrated a nonmonotonic error propagation analysis with an adjustable graph propagation visualization to help domain experts examine the details of error propagation and answer such questions as why an error is mitigated or amplified by program execution.

Acceleration of complex high-performance computing ensemble simulations with super-resolution-based subfilter models

Direct numerical simulation (DNS) of fluid flow problems has been one of the most important applications of high-performance computing (HPC) in the last decades. For example, turbulent flows require the simultaneous resolution of multiple spatial and temporal scales as all scales are coupled, resulting in very large simulations with enormous degrees of freedom. Another example is reactive flows, which typically result in a large system of coupled differential equations and multiple transport equations that must be solved simultaneously. In addition, many flows exhibit chaotic behavior, meaning that only statistical ensembles of results can be compared, further increasing the computational time. In this work, a combined HPC/deep learning (DL) workflow is presented that drastically reduces the overall computational time required while still providing acceptable accuracy.Traditionally, all the simulations required to compute ensemble statistics are performed using expensive DNS. The idea behind the combined HPC/DL workflow is to reduce the number of expensive DNSs by developing a DL-assisted large-eddy simulation (LES) approach that uses a sophisticated DL network, called PIESRGAN, as a subfilter model for all unclosed terms and is accurate enough to substitute DNSs. The remaining DNSs are thus used in two ways: first, as data contributing to the ensemble statistics, and second, as data used to train the DL network. It was found that in many cases two remaining DNSs are sufficient for training the LES approach. The cost of the DL-supported LES is usually more than one order of magnitude cheaper than the DNS, which drastically speeds up the workflow, even considering the overhead for training the DL network.

PARALiA: A Performance Aware Runtime for Auto-tuning Linear Algebra on Heterogeneous Systems

Dense linear algebra operations appear very frequently in high-performance computing (HPC) applications, rendering their performance crucial to achieve optimal scalability. As many modern HPC clusters contain multi-GPU nodes, BLAS operations are frequently offloaded on GPUs, necessitating the use of optimized libraries to ensure good performance. Unfortunately, multi-GPU systems are accompanied by two significant optimization challenges: data transfer bottlenecks as well as problem splitting and scheduling in multiple workers (GPUs) with distinct memories. We demonstrate that the current multi-GPU BLAS methods for tackling these challenges target very specific problem and data characteristics, resulting in serious performance degradation for any slightly deviating workload. Additionally, an even more critical decision is omitted because it cannot be addressed using current scheduler-based approaches: the determination of which devices should be used for a certain routine invocation. To address these issues we propose a model-based approach: using performance estimation to provide problem-specific autotuning during runtime. We integrate this autotuning into an end-to-end BLAS framework named PARALiA. This framework couples autotuning with an optimized task scheduler, leading to near-optimal data distribution and performance-aware resource utilization. We evaluate PARALiA in an HPC testbed with 8 NVIDIA-V100 GPUs, improving the average performance of GEMM by 1.7× and energy efficiency by 2.5× over the state-of-the-art in a large and diverse dataset and demonstrating the adaptability of our performance-aware approach to future heterogeneous systems.

Enabling the execution of HPC applications on public clouds with HPC@Cloud toolkit

AbstractThe advent of cloud computing has made access to computing infrastructure available to millions of users that face resource constraints. In the context of high performance computing (HPC), public cloud resources have emerged as a cost‐effective alternative to expensive on‐premises clusters. However, there are several challenges and limitations in adopting this approach. This paper proposes HPC@Cloud , a provider‐agnostic open‐source software toolkit that facilitates the migration, testing, and execution of HPC applications in public clouds. The toolkit takes advantage of various fault tolerance technologies to enable the use of inexpensive transient cloud infrastructure, commonly known as “spot” instances. Also, it features integration with singularity containers, allowing users to run complex applications on virtual HPC clusters in a portable and reproducible way. Finally, it provides a data‐based empirical approach to estimating cloud infrastructure costs for HPC workloads. The results obtained on two public cloud providers (AWS and Vultr) show that: (i) HPC@Cloud can efficiently build virtual HPC clusters on the cloud; (ii) the new adaptive fault tolerance strategy outperforms other existing strategies based on blocking restoration; (iii) the integration of singularity containers into HPC@Cloud improves the portability of HPC applications to public clouds with negligible performance penalty to the applications; (iv) the proposed cost prediction approach can estimate the cost of running the applications on AWS and Vultr with up to 93% accuracy on average.

Modeling Of 3D Integrated Circuits, Heat Transfer Solutions and Application

As transistor sizes approach the quantum limit, the cost of further shrinking them becomes prohibitively high. To overcome this limitation and surpass Moore's Law, 3D (Integrated Circuits) IC technology has emerged. However, while 3D ICs offer advantages like high integration, they also present challenges related to thermal management. This paper introduces and discusses partial differential equations and modeling methods for heat transfer in 3D ICs. It explores solutions to address thermal conduction problems and analyzes the potential application areas and prospects of 3D ICs. By utilizing different modeling methods, we can optimize the heat transfer problem during the design stage. To enhance the thermal conduction of 3D ICs, this study proposes the use of copper thermal conductive materials, graphene thermal conductive layers, and phase-change material cooling. As technology advances and costs decrease, 3D ICs are expected to find broader applications in high-performance computing, artificial intelligence, the Internet of Things, and other fields. Despite its numerous advantages, 3D integrated circuit technology still faces challenges such as cost, heat, and silicon vias. To address these issues, further technological innovations and updates to Computer Aided Design (CAD) tools are necessary. Overall, this study holds significant social and scientific importance as it promotes the development of 3D IC technology, improves electronic device performance, and advances scientific research.

Multi-resource scheduling of moldable workflows

Resource scheduling plays a vital role in High-Performance Computing (HPC) systems. Most scheduling research in HPC has focused on only a single type of resource (e.g., computing cores or I/O resource). With the advancement in hardware architectures and the increase in data-intensive HPC applications, there is a need to simultaneously consider a diverse set of resources (e.g., computing cores, cache, memory, I/O, and network resources) in the design of runtime schedulers for improving the overall application performance. In this paper, we study multi-resource scheduling to minimize the makespan of computational workflows comprised of moldable parallel jobs. Moldable jobs allow the scheduler to flexibly select a variable set of resources before execution, thus can adapt to the available system resources (as compared to rigid jobs) while staying easy to design and implement (as compared to malleable jobs). We propose a Multi-Resource Scheduling Algorithm (MRSA), which combines a novel resource allocation strategy and an extended list scheduling scheme to schedule the jobs. We prove that, on a system with d types of schedulable resources, MRSA achieves an approximation ratio of 1.619d+2.545d+1 for any d≥1, and a ratio of d+3d23+O(d3) when d is large (i.e., d≥22). We also present improved approximation results for workflows comprised of jobs with special precedence constraints (e.g., series-parallel graphs, trees, and independent jobs). Further, we prove a lower bound of d on the approximation ratio of any list-based scheduling algorithm with local priority considerations. Finally, we conduct a comprehensive set of simulations to evaluate the performance of the algorithm using synthetic workflows of different structures and moldable jobs following different speedup models. The results show that MRSA fares better than the theoretical bound predicts, and that it consistently outperforms two baseline heuristics under a variety of parameter settings, illustrating its robust practical performance.

Python-based social science applications’ profiling and optimization on HPC systems using task and data parallelism

This research addresses the pressing need to optimize Python-based social science applications for high-performance computing (HPC)systems, emphasizing the combined use of task and data parallelism techniques. The paper delves into a substantial body of research,recognizing Python’s interpreted nature as a challenge for efficient social science data processing. The paper introduces a Pythonprogram that exemplifies the proposed methodology. This program uses task parallelism with multi-processing and data parallelismwith dask to optimize data processing workflows. It showcases how researchers can effectively manage large datasets and intricatecomputations on HPC systems. The research offers a comprehensive framework for optimizing Python-based social science applicationson HPC systems. It addresses the challenges of Python’s performance limitations, data-intensive processing, and memory efficiency.Incorporating insights from a rich literature survey, it equips researchers with valuable tools and strategies for enhancing the efficiencyof their social science applications in HPC environments.

I/O Access Patterns in HPC Applications: A 360-Degree Survey

The high-performance computing I/O stack has been complex due to multiple software layers, the inter-dependencies among these layers, and the different performance tuning options for each layer. In this complex stack, the definition of an “I/O access pattern” has been reappropriated to describe what an application is doing to write or read data from the perspective of different layers of the stack, often comprising a different set of features. It has become common to have to redefine what is meant when discussing a pattern in every new study, as no assumption can be made. This survey aims to propose a baseline taxonomy, harnessing the I/O community’s knowledge over the past 20 years. This definition can serve as a common ground for high-performance computing I/O researchers and developers to apply known I/O tuning strategies and design new strategies for improving I/O performance. We seek to summarize and bring a consensus to the multiple ways to describe a pattern based on common features already used by the community over the years.

Variable Neighborhood Search for precedence-constrained tasks optimization on heterogeneous systems

Supercomputing power is one of the fundamental pillars of the digital society, which depends on the accurate scheduling of parallel applications in High-Performance Computing (HPC) centers to minimize computing times. However, precedence-constraint task scheduling is a well-known NP-Hard optimization problem, and no optimal polynomial-time algorithm exists to solve it. Therefore, as accurate as possible to the optimal values, heuristic algorithms are of relevant interest. Our new scheduling proposal name is EFT-GVNS, which stands for Earliest Finish Time - General Variable Neighborhood Search. EFT-GVNS uses a Composite Local Search (CLS), making our proposal more efficient than traditional GVNS. EFT-GVNS accuracy against four high-performance algorithms from the state-of-the-art (EDA, EFT-ILS, GRASP-CPA, MPQGA) and one reference algorithm in the literature (HEFT) is studied. Experimental results over four real-world applications (Fpppp, LIGO, Robot, Sparse) and 14 synthetic instances from the literature show that EFT-GVNS outperforms in terms of the median achieved results, with a global improvement of 37.6%, 27.4%, 17.8%, 6.1%, 2.2% to HEFT, EDA, EFT-ILS, GRASP-CPA, and MPQGA, respectively. EFT-GVNS achieves all the 14 optimal values of the synthetic benchmark.

Towards Effective Bulgarian Competence Centre in High Performance Computing – Service Portfolio and Competences

Three years ago a new European initiative started: establishing a network of Centres of Competence in HPC in the European countries through the project EuroCC (2020-2022) in its second phase EuroCC2 (2023-2025). The goal is to accelerate the improvement of national and thus European capabilities in the area of HPC+ technologies, where HPC+ means High Performance Computing (HPC) and HPC application in HPDA and AI. In order to form an effective National Competence Centre in HPC+ technologies, we performed extensive competence mapping and now devise an extensive service portfolio, open to users from academia, public administration and industry. In this paper we present our analysis of the situation in Bulgaria as well as our approaches to make NCC and effective focus point for those that can benefit from use of HPC+ technology in their research or business.

UrsaX: Integrating Block I/O and Message Transfer for Ultrafast Block Storage on Supercomputers

It is increasingly important for the next-generation exascale supercomputers to extend its applications beyond traditional HPC (high-performance computing) scenarios, so as to achieve high social and economic benefit. Similar to AWS (Amazon Web Services) and Alibaba Cloud, cloud-style virtual HPC service is a promising application scenario on supercomputers, for which remote block storage is the key to provide tenants with supercomputers extremely-high storage performance. Unfortunately, the state-of-the-art block storage software systems (such as URSA and Ceph) cannot adapt to the advanced hardware features of supercomputers. This paper presents UrsaX, an efficient block storage service for our next-generation Tianhe exascale supercomputer that is equipped with the high-performance GLEX network and NVMe (Non-Volatile Memory Express) SSDs. UrsaXs virtual disks, which can be mounted like normal physical ones, enable not only traditional HPC applications but also supercomputeroblivious POSIX applications to enjoy the high performance of supercomputers. At the core of UrsaX is with a novel design of the efficient integration of on-disk block I/O and in-network message transfer on supercomputers. UrsaX utilizes the NVMe Fabrics kernel module to expand the NVMe standard on the supercomputer network, and separates metadata I/O and data I/O of blocks respectively being handled over the MP (Mini Packet) and RDMA (Remote Direct Memory Access) protocols. We thoroughly explore the design space for remote block storage on supercomputers including parallelism, scalability, fault tolerance, and consistency. We conduct extensive evaluation on a subset of our exascale supercomputer consisting of 44 storage machines (each with four NVMe SSDs). The result shows that UrsaX achieves local-storage-level I/O latency (tens of microseconds) while being able to linearly increase the aggregate performance (IOPS and throughput) as the system scale increases, an order of magnitude higher than the state-of-the-art block storage systems.