World Of High-performance Computing Research Articles

On Hardware Flexibility and Heterogeneity: A Vision for Monte Carlo Codes on Incoming RISC-V Computing Devices with AI-based Cross Section

As an open-sourced instruction set and being flexible in hardware extension, RISC-V begins its pace to enter the world of high performance computing. One of the distinguished feature of processing units adopting RISC-V is its ability to add custom circuits with special purpose accelerators. As the artificial general intelligence becomes practical, AI accelerators become an indispensable part of computing devices, where RISC-V is a great fit for the CPU to glue accelerators together. A system of chip designed by Alibaba T-head is one of the early chip in the massive production adopting RISC-V CPU, where the CPU, named Xuantie-910, has a high performance design with 128-bit RISC-V vector processing units, which are designed for accelerating AI applications. OpenMC has been adapted to run on Xuantie-910. In the Monte Carlo method for reactor physics, fetching the neutron cross sections is the hotspot that takes the majority of the computational burden. The traditional point-wise cross sections are slow because of memory latency caused by accessing many nonconsecutive memory addresses. An AI model for cross section is hence proposed. With 2.2 KB of runtime size, the smallest in the published work, the data can be fetched entirely in the L1 cache during on-the-fly cross section evaluation through single memory read. The in-house AI model also covers the entire energy range, unlike only the resonance range is supported in previous work. So, the effects from memory latency is minimized. The average relative error in AI modeled U-238 elastic cross section is 0.6% from point-wise cross section. With a modified version of OpenMC on Apple M3 Max, for a VERA pin-cell problem, compared to the point-wise cross section, the adoption of AI modeled cross section reduces the total runtime by 7%, although the runtime for calculating U-238 elastic cross section causes 40% more runtime. The adoption of AI modeled U-238 elastic cross section leads to K-effective 302 pcm higher than the case of adoption of point-wise cross sections. Advantage of AI model has been verified. With AI modeled cross section, the neutron slowing down problems with pure elastic scattering on U-238 has been studied on Xuantie-910. The average relative error in 65,536 group fluxes is about 0.9% from using point-wise cross section. However, with accelerating with the 128-bit vector processing units, the performance degrades by 35%, because of the narrow 64-bit load and store interface to the vector register files. The performance with Al modeled cross section is about 1/4 of the case with point-wise cross sections. In addition, the 1,024-bit width Ara RISC-V vector processing has been used to study the cost of AI modeled cross section evaluation. Being able to access the open-sourced hardware design in SystemVerilog, cycle accurate circuit simulation is performed. Using the vector processing units, the cost is reduced to 65% of the case using scalar instructions. The 128-bit load and store interface to vector processing units is a major contributor to the speeding up. The width of the load and store interface to vector processing units should be the main optimization factor in chip design to accelerate the AI modeled cross section evaluation.

Introduction to High-Performance Computing.

Since the first general-purpose computing machines came up in the middle of the twentieth century, computer science's popularity has been growing steadily until our time. The first computers represented a significant leap forward in automating calculations, so that several theoretical methods could be taken from paper into practice. The continuous need for increased computing capacity made computers evolve and become more and more powerful. Nowadays, high-performance computing (HPC) is a crucial component of scientific and technological advancement. This book chapter introduces the field of HPC, covering key concepts and essential terminology to understand this complex and rapidly evolving area. The chapter begins with an overview of what HPC is and how it differs from conventional computing. It then explores the various components and configurations of supercomputers, including shared memory systems, distributed memory systems, and hybrid systems and the different programming models used in HPC, including message passing, shared memory, and data parallelism. Finally, the chapter discusses significant challenges and future directions in supercomputing. Overall, this chapter provides a comprehensive introduction to the world of HPC and is an essential resource for anyone interested in this fascinating field.

Critique of “Planetary Normal Mode Computation: Parallel Algorithms, Performance, and Reproducibility” by SCC Team From University of Warsaw

The Supercomputing conference holds a student cluster competition - A competition that aims to introduce undergraduate students to the world of High Performance Computing. One of the tasks at the SC19 conference was to reproduce the scaling results obtained in the article titled Computing Planetary Interior Normal Modes with a Highly Parallel Polynomial Filtering Eigensolver by Jia Shi et al. They introduced a new approach to the problem of calculating planetary normal modes. The developed method is a highly parallel algorithm that approximates the results via the mixed finite element method on unstructured tetrahedral meshes. The cluster used consisted of five 40-core Intel Xeon Cascade Lake nodes, equipped with 384 GB of RAM each. The original study was demonstrated on Stampede2 system on partitions ranging from 2 to 256 nodes with 48-core Intel Xeon Skylake and 192 GB of RAM each. Our cluster was equipped with Infiniband interconnect while Stampede2 used Intel Omni-Path network. We decided to work with HPC-X MPI library in place of Intel MPI used the reference study. The design - sizes and schedule - of experimental runs was the most challenging part of our study. We run both strong and weak scalability experiments in a one set of runs in order to fit in limited computing capacity and very tight time schedule. Due to limitations of our hardware, we had to split the weak scaling study into two separate smaller studies. We didn't manage to reproduce the exact results, however, we achieved similar scalability trend.

Approaches of enhancing interoperations among high performance computing and big data analytics via augmentation

The dawn of exascale computing and its convergence with big data analytics has greatly spurred research interests. The reasons are straightforward. Traditionally, high performance computing (HPC) systems have been used for scientific applications involving majority of compute-intensive tasks. At the same time, the proliferation of big data resulted into design of data-intensive processing paradigms like Apache big data stack. Big data generating at high pace necessitates faster processing mechanisms for getting insights at a real time. For this, the HPC systems may serve as panacea for solving the big data problems. Though the HPC systems have the capability to give the promising results for big data, directly integrating them with existing data-intensive frameworks like Apache big data stack is not straightforward due to challenges associated with them. This triggers a research on seamlessly integrating these two paradigms based on interoperable framework, programming model, and system architecture. The aim of this paper is to assess a progress made in HPC world as an effort to augment it with big data analytics support. As an outcome of this, the taxonomy showing the factors to be considered for augmenting HPC systems with big data support has been put forth. This paper sheds light upon how big data frameworks can be ported to HPC platforms as a preliminary step towards the convergence of big data and exascale computing ecosystem. The focus is given on research issues related to augmenting HPC paradigms with big data frameworks and corresponding approaches to address those issues. This paper also discusses data-intensive as well as compute-intensive processing paradigms, benchmark suites and workloads, and future directions in the domain of integrating HPC with big data analytics.

Accelerating genetic algorithms with GPU computing: A selective overview

The emergence of GPU-CPU heterogeneous architectures has led to a fundamental paradigm shift in parallel programming. Accelerating Genetic Algorithms (GAs) on these architectures has received significant attention from both practitioners and researchers ever since GPUs emerged. In the past decade we have witnessed many progresses on migrating parallel GAs from CPU to GPU (Graphical Processing Unit) architecture, which makes this research field truly enter into the world of High Performance Computing (HPC), and demonstrates a great potential to many research disciplines and industrial worlds that can benefit from the power of GPU accelerated stochastic global search to explore large and complex search spaces for better solutions.Designing a parallel algorithm on GPU is quite different from designing one on CPU. On CPU architecture, we typically consider how to distribute data across tens of CPU threads, while on GPU architecture, we have more than hundreds of thousands of GPU threads running simultaneously. Therefore, we should rethink the design approaches and implementation strategies of parallel algorithms to fully utilize the computing power of GPUs to accelerate the computation of GAs. The intention of this paper is to give an overview on selective works of parallel GAs designed for GPU architecture.In this survey paper, we first reexamine the concept of granularity of parallelism for GAs on GPU architecture, discuss how the aspect of data layout affect the kernel design to maximize memory bandwidth, and explain how to organize threads in grid and blocks to expose sufficient parallelism to GPU. The comprehensive overview on selective works since 2010 then follows. The focus is mainly on the perspective of GPU architecture: how to accelerate GAs with GPU computing. Performance issues are not touched in this review, because most of these works are conducted on very early GPU cards, which are out of date already.We finally discuss some future research suggestions in the last section, especially about how to build up an efficient implementation of parallel GAs for hyper-scale computing. Many industrial and academic disciplines will be benefited from the GPU accelerated parallel GAs, one of the promising area is to evolve better deep neural networks.

An Analytical Approach for Optimizing the Performance of Hadoop Map Reduce Over RoCE

Data intensive systems aim to efficiently process “big” data. Several data processing engines have evolved over past decade. These data processing engines are modeled around the MapReduce paradigm. This article explores Hadoop's MapReduce engine and propose techniques to obtain a higher level of optimization by borrowing concepts from the world of High Performance Computing. Consequently, power consumed and heat generated is lowered. This article designs a system with a pipelined dataflow in contrast to the existing unregulated “bursty” flow of network traffic, the ability to carry out both Map and Reduce tasks in parallel, and a system which incorporates modern high-performance computing concepts using Remote Direct Memory Access (RDMA). To establish the claim of an increased performance measure of the proposed system, the authors provide an algorithm for RoCE enabled MapReduce and a mathematical derivation contrasting the runtime of vanilla Hadoop. This article proves mathematically, that the proposed system functions 1.67 times faster than the vanilla version of Hadoop.

ADAMANT: Tools to Capture, Analyze, and Manage Data Movement

In the converging world of High Performance Computing and Big Data, moving data is becoming a critical aspect of performance and energy efficiency. In this paper we present the Advanced DAta Movement Analysis Toolkit (ADAMANT), a set of tools to capture and analyze data movement within an application, and to aid in understanding performance and energy efficiency in current and future systems. ADAMANT identifies all the data objects allocated by an application and uses instrumentation modules to monitor relevant events (e.g. cache misses). Finally, ADAMANT produces a per-object performance profile.In this paper we demonstrate the use of ADAMANT in analyzing three applications, BT, BFS, and Velvet, and evaluate the impact of different memory technology. With the information produced by ADAMANT we were able to model and compare different memory configurations and object placement solutions. In BFS we devised a placement which outperforms caching, while in the other two cases we were able to point out which data objects may be problematic for the configurations explored, and would require refactoring to improve performance.

A performance comparison of current HPC systems: Blue Gene/Q, Cray XE6 and InfiniBand systems

We present here a performance analysis of three of current architectures that have become commonplace in the High Performance Computing world. Blue Gene/Q is the third generation of systems from IBM that use modestly performing cores but at large-scale in order to achieve high performance. The XE6 is the latest in a long line of Cray systems that use a 3-D topology but the first to use its Gemini interconnection network. InfiniBand provides the flexibility of using compute nodes from many vendors that can be connected in many possible topologies. The performance characteristics of each vary vastly, and the way in which nodes are allocated in each type of system can significantly impact on achieved performance. In this work we compare these three systems using a combination of micro-benchmarks and a set of production applications. In addition we also examine the differences in performance variability observed on each system and quantify the lost performance using a combination of both empirical measurements and performance models. Our results show that significant performance can be lost in normal production operation of the Cray XE6 and InfiniBand Clusters in comparison to Blue Gene/Q.

From Silicon to Science

The field of high performance computing (HPC) currently abounds with excitement about the potential of a broad class of things called accelerators . And, yet, few accelerator based systems are being deployed in general purpose HPC environments. Why is that? This article explores the challenges that accelerators face in the HPC world, with a specific focus on FPGA based systems. We begin with an overview of the characteristics and challenges of typical HPC systems and applications and discuss why FPGAs have the potential to have a significant impact. The bulk of the article is focused on twelve specific areas where FPGA researchers can make contributions to hasten the adoption of FPGAs in HPC environments.