Role In High Performance Computing Research Articles

The Egyptian national HPC grid (EN-HPCG): open-source Slurm implementation from cluster to grid approach

Recently, Egypt has recognized the pivotal role of High Performance Computing in advancing science and innovation. Additionally, Egypt realizes the importance of collaboration between different institutions and universities to consolidate their own computational and data resources into a unified platform to serve different disciplines (e.g., scientific, industrial, governmental). Otherwise, additional resources would be needed to be purchased with the associated cost, effort, and time difficulties (e.g., setup, administration, maintenance, etc.). Thus, this paper delves into the architecture and capabilities of the EN-HPCG grid using two different workload management systems: (i) Slurm (Open-Source) and (ii) PBS Pro (Licensed). This paper compares the performance of the grid between Slurm and PBS Pro in specific high-throughput computing (HTC) applications using the NAS Grid parallel benchmark (NGB) to determine which workload manager is more suitable for EN-HPCG. The evaluation includes grid-level performance metrics such as throughput, and the number of tasks completed as a function of time. Also, the presented methodology aims to assist potential partners in their decision-making process to join the EN-HPCG grid, with a focus on the site speed-up metric. Our results showed that, unless an open-source solution without cost and license problems is an obligation (in which case, Slurm is the viable solution), then it is not advisable to integrate a cluster with high-speed hardware with a cluster possessing outdated hardware when using the Slurm scheduler. In contrast, the PBS Pro scheduler takes into account online decision-making in a dynamic environment using a unified grid.

Multi-resource scheduling of moldable workflows

Resource scheduling plays a vital role in High-Performance Computing (HPC) systems. Most scheduling research in HPC has focused on only a single type of resource (e.g., computing cores or I/O resource). With the advancement in hardware architectures and the increase in data-intensive HPC applications, there is a need to simultaneously consider a diverse set of resources (e.g., computing cores, cache, memory, I/O, and network resources) in the design of runtime schedulers for improving the overall application performance. In this paper, we study multi-resource scheduling to minimize the makespan of computational workflows comprised of moldable parallel jobs. Moldable jobs allow the scheduler to flexibly select a variable set of resources before execution, thus can adapt to the available system resources (as compared to rigid jobs) while staying easy to design and implement (as compared to malleable jobs). We propose a Multi-Resource Scheduling Algorithm (MRSA), which combines a novel resource allocation strategy and an extended list scheduling scheme to schedule the jobs. We prove that, on a system with d types of schedulable resources, MRSA achieves an approximation ratio of 1.619d+2.545d+1 for any d≥1, and a ratio of d+3d23+O(d3) when d is large (i.e., d≥22). We also present improved approximation results for workflows comprised of jobs with special precedence constraints (e.g., series-parallel graphs, trees, and independent jobs). Further, we prove a lower bound of d on the approximation ratio of any list-based scheduling algorithm with local priority considerations. Finally, we conduct a comprehensive set of simulations to evaluate the performance of the algorithm using synthetic workflows of different structures and moldable jobs following different speedup models. The results show that MRSA fares better than the theoretical bound predicts, and that it consistently outperforms two baseline heuristics under a variety of parameter settings, illustrating its robust practical performance.

Link fault tolerance of BC networks and folded hypercubes on h-extra r-component edge-connectivity

Parallel and distributed systems play a significant role in high-performance computing, prompting us to investigate qualitative and quantitative metrics to indicate the fault tolerance and vulnerability of systems. Consider the setup where there are large-scale link malfunctions that disconnect the network and result in various components, each with multiple processors. In this paper, we propose and study the h-extra r-component edge-connectivity of a connected graph G, which is denoted by cλrh(G) and has not been addressed before. Let h and r be two positive integers with r≥2. An edge subset F⊆E(G) is said to be an h-extra r-component edge-cut of G, if any, G−F has at least r components and every component of G−F has at least h vertices. The cardinality of the minimum h-extra r-component edge-cut of G is the h-extra r-component edge-connectivity of G. Let h be a positive integer with the decomposition h=∑i=0t2ki, where ki>ki+1, 0≤i≤t−1. In this paper, we derive a lower bound for the exact value of h-extra 3-component edge-connectivity of BC networks Bn and demonstrate that it is tight for one member of Bn, hypercube Qn, in the interval 1≤h≤2⌊n2⌋−1−1, n≥4. This lower bound is also tight for the exact value of 2c-extra 3-component edge-connectivity of Bn with 0≤c≤n−2, n≥4. Specifically, cλ3h(Qn)=2nh−∑i=0tki2ki+1−∑i=0ti2ki+2−h for 1≤h≤2⌊n2⌋−1−1, n≥4 and cλ32c(Bn)=(2n−2c−1)2c for 0≤c≤n−2, n≥4. Exact value of h-extra 3-component edge-connectivity of n-dimensional folded hypercube FQn, cλ3h(FQn) is 2(n+1)h−∑i=0tki2ki+1−∑i=0ti2ki+2−h for 1≤h≤2⌈n2⌉−1−1, n≥4, and that of 2c-extra 3-component edge-connectivity of FQn, cλ32c(FQn), is (2n−2c+1)2c for 0≤c≤n−2, n≥4.

VCSN: Virtual Circuit-Switching Network for Flexible and Simple-to-Operate Communication in HPC FPGA Cluster

FPGA clusters promise to play a critical role in high-performance computing (HPC) systems in the near future due to their flexibility and high power efficiency. The operation of large-scale general-purpose FPGA clusters on which multiple users run diverse applications requires flexible network topology to be divided and reconfigured. This paper proposes Virtual Circuit-Switching Network (VCSN) that provides an arbitrarily reconfigurable network topology and simple-to-operate network system among FPGA nodes. With virtualization, user logic on FPGAs can communicate with each other as if a circuit-switching network was available. This paper demonstrates that VCSN with 100 Gbps Ethernet achieves highly-efficient point-to-point communication among FPGAs due to its unique and efficient communication protocol. We compare VCSN with a direct connection network (DCN) that connects FPGAs directly. We also show a concrete procedure to realize collective communication on an FPGA cluster with VCSN. We demonstrate that the flexible virtual topology provided by VCSN can accelerate collective communication with simple operations. Furthermore, based on experimental results, we model and estimate communication performance by DCN and VCSN in a large FPGA cluster. The result shows that VCSN has the potential to accelerate gather communication up to about 1.97 times more than DCN.

Novel high-performance QCA Fredkin gate and designing scalable QCA binary to gray and vice versa

In the design of digital logic circuits, QCA technology is an excellent alternative to CMOS technology. Its advantages over CMOS include low power consumption, fast circuit switching, and nanoscale design. Circuits that convert data between different formats are code converters. Code converters have an essential role in high-performance computing and signal processing. In this paper, first, we proposed a novel QCA structure for the quantum reversible Fredkin gate. Second, we proposed 4-bit and 8-bit QCA binary-to-gray converter and vice versa. For the second proposal, both reversible and irreversible structures are suggested. The proposed structures are scalable up to N bits. To change the conversion type from B2G to G2B, we use a 2:1 QCA multiplexer. The proposed QCA Fredkin is applied in the reversible design of QCA code converters as multiplexers. The suggested designs are simulated using the QCADesigner tool. Then we calculated figures of merit, including cell counts, occupied areas, and clock zones. Finally, we compare the proposed structures to existing research. Our proposed approach is the first quantum-dot cellular automata design to perform B2G conversion and G2B in a single QCA circuit. The proposed designs are scalable. Specifications are reported.

A Study of Failure Recovery and Logging of High-Performance Parallel File Systems

Large-scale parallel file systems (PFSs) play an essential role in high-performance computing (HPC). However, despite their importance, their reliability is much less studied or understood compared with that of local storage systems or cloud storage systems. Recent failure incidents at real HPC centers have exposed the latent defects in PFS clusters as well as the urgent need for a systematic analysis. To address the challenge, we perform a study of the failure recovery and logging mechanisms of PFSs in this article. First, to trigger the failure recovery and logging operations of the target PFS, we introduce a black-box fault injection tool called PFault , which is transparent to PFSs and easy to deploy in practice. PFault emulates the failure state of individual storage nodes in the PFS based on a set of pre-defined fault models and enables examining the PFS behavior under fault systematically. Next, we apply PFault to study two widely used PFSs: Lustre and BeeGFS. Our analysis reveals the unique failure recovery and logging patterns of the target PFSs and identifies multiple cases where the PFSs are imperfect in terms of failure handling. For example, Lustre includes a recovery component called LFSCK to detect and fix PFS-level inconsistencies, but we find that LFSCK itself may hang or trigger kernel panics when scanning a corrupted Lustre. Even after the recovery attempt of LFSCK, the subsequent workloads applied to Lustre may still behave abnormally (e.g., hang or report I/O errors). Similar issues have also been observed in BeeGFS and its recovery component BeeGFS-FSCK. We analyze the root causes of the abnormal symptoms observed in depth, which has led to a new patch set to be merged into the coming Lustre release. In addition, we characterize the extensive logs generated in the experiments in detail and identify the unique patterns and limitations of PFSs in terms of failure logging. We hope this study and the resulting tool and dataset can facilitate follow-up research in the communities and help improve PFSs for reliable high-performance computing.

The PINNs method discovery to the solution of coupled Wave- Klein-Gordon equations

Recently, the research of PDEs is regarded as one of the most important disciplines. Almost all scientific problems can be described by a differential equation, especially, many physical phenomena can be described by the system of coupled Wave-Klein-Gordon equations, which plays an important role in high-performance computing, control engineering, and electronic power system. Consequently, in our work, we use the PINNs to solve the numerical solution of coupled Wave-Klein-Gordon equations, to help us better understand the nonlinear physical phenomena, and to promote the rapid development of various fields such as in high-performance computing, control engineering, and electronic power system.

On-GPU Thread-Data Remapping for Branch Divergence Reduction

General Purpose GPU computing (GPGPU) plays an increasingly vital role in high performance computing and other areas like deep learning. However, arising from the SIMD execution model, the branch divergence issue lowers efficiency of conditional branching on GPUs, and hinders the development of GPGPU. To achieve runtime on-the-spot branch divergence reduction, we propose the first on-GPU thread-data remapping scheme. Before kernel launching, our solution inserts codes into GPU kernels immediately before each target branch so as to acquire actual runtime divergence information. GPU software threads can be remapped to datasets multiple times during single kernel execution. We propose two thread-data remapping algorithms that are tailored to the GPU architecture. Effective on two generations of GPUs from both NVIDIA and AMD, our solution achieves speedups up to 2.718 with third-party benchmarks. We also implement three GPGPU frontier benchmarks from areas including computer vision, algorithmic trading and data analytics. They are hindered by more complex divergence coupled with different memory access patterns, and our solution works better than the traditional thread-data remapping scheme in all cases. As a compiler-assisted runtime solution, it can better reduce divergence for divergent applications that gain little acceleration on GPUs for the time being.

Exploring the feasibility of lossy compression for PDE simulations

Checkpoint restart plays an important role in high-performance computing (HPC) applications, allowing simulation runtime to extend beyond a single job allocation and facilitating recovery from hardware failure. Yet, as machines grow in size and in complexity, traditional approaches to checkpoint restart are becoming prohibitive. Current methods store a subset of the application’s state and exploit the memory hierarchy in the machine. However, as the energy cost of data movement continues to dominate, further reductions in checkpoint size are needed. Lossy compression, which can significantly reduce checkpoint sizes, offers a potential to reduce computational cost in checkpoint restart. This article investigates the use of numerical properties of partial differential equation (PDE) simulations, such as bounds on the truncation error, to evaluate the feasibility of using lossy compression in checkpointing PDE simulations. Restart from a checkpoint with lossy compression is considered for a fail-stop error in two time-dependent HPC application codes: PlasComCM and Nek5000. Results show that error in application variables due to a restart from a lossy compressed checkpoint can be masked by the numerical error in the discretization, leading to increased efficiency in checkpoint restart without influencing overall accuracy in the simulation.

Energy aware scheduling model and online heuristics for stencil codes on heterogeneous computing architectures

Performance of high-end supercomputers will reach the exascale through the advent of core counts in billions. However, in the upcoming exascale computing era it is important not only to focus on the performance, but also on scalability of fine-grained parallel applications, data locality and energy aware scheduling within the parallel code. In fact, parallel applications need to change even now by redesigning algorithms and data structures respectively to take advantage of the recent improvements in energy efficiency of heterogeneous computing hardware, including multicore processors and GPU accelerators. Over the next few years one of the biggest challenges for exascale will be the ability of parallel applications to fully exploit locality which will, in turn, be required to achieve expected performance and energy efficiency. Future highly parallel applications will have to deal with deep memory hierarchies taking into account energy cost in moving data off-chip. Therefore, they will have to apply new coordinated scheduling approaches to balance energy aware resource utilization and minimize work starvation during runtime. As new constraints and limits on memory bandwidth and energy will play a key role in high performance computing (HPC) in the future, more sophisticated and dynamic scheduling techniques will be needed and applied within the parallel code. In this paper we focus on an energy-aware distribution of the stencil workload on heterogeneous processors. Our analysis of energy and performance models focused on relevant class of stencil computations to explore the relationship between task scheduling algorithms and energy constraints. More precisely, we search for a schedule which minimizes the energy usage within a specified computation’s deadline of the stencil workload on heterogeneous architectures. Since the problem is computationally intractable, we present an integer linear programming formulation for finding optimal schedules. As finding optimal schedules is time consuming we have developed four heuristics and tested them experimentally with respect to optimal solutions. In our work we focus on a single node configurations with heterogeneous processors. These configurations represent the state of the art multi- and many-core architectures.

Evaluation of redundant data storage in clusters based on multi-multicast and local storage

Cluster platforms have an important role in high performance computing (HPC). They execute cloud computing, data-intensive computing and data center applications, which are supported on distributed file systems. The implementation of data redundancy in these file systems provides a support for high availability and error tolerance. This work proposes an implementation of redundant data storage based on the storage included in the cluster nodes, instead of more expensive approaches with a dedicated storage and network, and on multi-multicast transfers, instead on unicast transfers, to perform the multiple simultaneous data diffusion required for implementing redundant data storage. The proposal applies a recently proposed congestion control scheme that adjusts the sender injection rate, taking into account control information from the receiver nodes and the storage technology available on the cluster nodes. The implementation takes full advantage of the switch diffusion hardware and of the IGMP snooping capability of current switches, which allows to multicast a packet just to the output links with receivers joined to a multicast group. It is made at the user level directly on the UDP interface. Evaluation tests with multiple simultaneous storage accesses were performed in a CentOS cluster. Test results show a more efficient use of the cluster storage. The global bandwidth improves by using hardware related to the storage (network and storage devices) more efficiently.

The Role of High Performance Computing and Communication for Real-Time Biofeedback in Sport

This paper studies the main technological challenges of real-time biofeedback in sport. We identified communication and processing as two main possible obstacles for high performance real-time biofeedback systems. We give special attention to the role of high performance computing with some details on possible usage of DataFlow computing paradigm. Motion tracking systems, in connection with the biomechanical biofeedback, help in accelerating motor learning. Requirements about various parameters important in real-time biofeedback applications are discussed. Inertial sensor tracking system accuracy is tested in comparison with a high performance optical tracking system. Special focus is given on feedback loop delays. Real-time sensor signal acquisitions and real-time processing challenges, in connection with biomechanical biofeedback, are presented. Despite the fact that local processing requires less energy consumption than remote processing, many other limitations, most often the insufficient local processing power, can lead to distributed system as the only possible option. A multiuser signal processing in football match is recognised as an example for high performance application that needs high-speed communication and high performance remote computing. DataFlow computing is found as a good choice for real-time biofeedback systems with large data streams.

MAPS

GPUs play an increasingly important role in high-performance computing. While developing naive code is straightforward, optimizing massively parallel applications requires deep understanding of the underlying architecture. The developer must struggle with complex index calculations and manual memory transfers. This article classifies memory access patterns used in most parallel algorithms, based on Berkeley’s Parallel “Dwarfs.” It then proposes the MAPS framework, a device-level memory abstraction that facilitates memory access on GPUs, alleviating complex indexing using on-device containers and iterators. This article presents an implementation of MAPS and shows that its performance is comparable to carefully optimized implementations of real-world applications.

A Checkpoint/Restart Scheme for CUDA Programs with Complex Computation States

Checkpoint/restart has been an effective mechanism to achieve fault tolerance for many long-running scientific applications. The common approach is to save computation states in memory and secondary storage for execution resumption. However, as the GPU plays a much bigger role in high performance computing, there is no effective checkpoint/restart scheme yet due to the difficulty of the GPU computation state handling. This paper proposes an application-level checkpoint/restart scheme to save and restore GPU computation states in annotated user programs. A pre-compiler and run-time support module are developed to construct and save states in CPU system memory dynamically, whereas secondary storage can be utilized for scalability and long-term fault tolerance. CUDA programs with complicated computation states are supported. State-related variables dissipated in various memory units are collected. Both stack and heap are duplicated at application level for state construction. Experimental results have demonstrated the effectiveness of the proposed scheme.

High Performance Computing: Infrastructure, Application, and Operation

The last decades have witnessed an increasingly indispensible role of high performance computing (HPC) in science, business and financial sectors, as well as military and national security areas. To introduce key aspects of HPC to a broader community, an HPC session was organized for the first time ever for the United States and Korea Conference (UKC) during 2012. This paper summarizes four invited talks that each covers scientific HPC applications, large-scale parallel file systems, administration/maintenance of supercomputers, and green technology towards building power efficient supercomputers of the next generation.

High performance computing for detection of electricity theft

Transmission and distribution of electricity involve technical as well as Non-Technical Losses (NTLs). Illegal consumption of electricity constitutes a major portion of the NTL at distribution feeder level. Considering the severity and devastating effects of the problem, illegal consumption of electricity has to be detected instantly in real-time. To this end, this paper investigates the possibility and role of High Performance Computing (HPC) algorithms in detection of illegal consumers. This paper designs and implements an encoding procedure to simplify and modify customer energy consumption data for quicker analysis without compromising the quality or uniqueness of the data. This paper parallelizes overall customer classification process. The parallelized algorithms have resulted in appreciable results as displayed in the results section of the paper.

A New Vectorization Technique for Expression Templates in C++

Vector operations play an important role in high performance computing and are typically provided by highly optimized libraries that implement the Basic Linear Algebra Subprograms (BLAS) interface. In C++ templates and operator overloading allow the implementation of these vector operations as expression templates which construct custom loops at compile time and providing a more abstract interface. Unfortunately existing expression template libraries lack the performance of fast BLAS implementations. This paper presents a new approach - Statically Accelerated Loop Templates (SALT) - to close this performance gap by combining expression templates with an aggressive loop unrolling technique. Benchmarks were conducted using the Intel C++ compiler and GNU Compiler Collection to assess the performance of our library relative to Intel's Math Kernel Library as well as the Eigen template library. The results show that the approach is able to provide optimization comparable to the fastest available BLAS implementations, while retaining the convenience and flexibility of a template library.

Performance Evaluation of A Infiniband-based Lustre Parallel File System

Abstract As one of the most widely used parallel file systems, Lustre plays an important role in High Performance Computing (HPC). In this paper, an infiniband-based Lustre parallel file system is built firstly. And then a series of experiments are executed to evaluate the performance of luster with infiniaband. Some parameters have great influence on the result, such as size record, client count, stripe count and stripe size. Experimental results indicate that the higher is the speed of network the higher aggregation bandwidth luster could provide. Moreover, Client count and stripe count have more impact on the performance of luster than the record size and stripe size. Appropriate record size and stripe size could improve the performance a little, but not significantly.

Virtualizing high performance computing

While virtualization is widely used in commercial enterprise environments, it has not to date played any significant role in High Performance Computing (HPC). However, with the rise of cloud computing and its promise of computing on demand, the HPC community's interest in virtualization (a key cloud enabler) is increasing. Beyond cloud computing, virtualization offers additional potential benefits for HPC, among them reactive and proactive application fault tolerance, secure and fault-isolated use of shared-resource clusters, dynamic provisioning, job migration, and support for heterogeneous HPC facilities. This paper describes both the promises and challenges in this new, emerging area, including a discussion of performance-related issues

HIERARCHICAL REPLICATION TECHNIQUES TO ENSURE CHECKPOINT STORAGE RELIABILITY IN GRID ENVIRONMENT

An efficient and reliable fault tolerance protocol plays a key role in High Performance Computing. Rollback recovery is the most common fault tolerance technique used in High Performance Computing and especially in MPI applications. This technique relies on the reliability of the checkpoint storage. Most of the rollback recovery protocols assume that the checkpoint servers machines are reliable. However, in a grid environment any unit can fail at any moment, including components used to connect different administrative domains. Such failures lead to the loss of a whole set of machines, including the more reliable machines used to store the checkpoints in this administrative domain. Thus it is not safe to rely on the high Mean Time Between Failures of specific machines to store the checkpoint images.This paper introduces a new coordinated checkpoint protocol, which tolerates checkpoint server failures and clusters failures, and ensures a checkpoint storage reliability in a grid environment. To provide this reliability the protocol is based on a replication process. We propose new hierarchical replication strategies that exploit the locality of checkpoint images in order to minimize inter-cluster communication.We evaluate the effectiveness of our two hierarchical replication strategies through simulations against several criteria such as topology and scalability.