High Performance Computing Architectures Research Articles

Transmission and distribution co‐simulation: a review and propositions

With the growing trend of emerging new technologies in distribution networks, such as wind turbines, solar panels, electric vehicles, and distributed generations, the passive distribution systems may become ‘active’ which requires more study in the area of integrated transmission and distribution systems (ITDSs) and corresponding bilateral interactions. To solve this problem, most of the studies connect distinct simulators to create a novel co-simulation framework for ITDS. In this study, the authors present a literature survey of existing ITDS co-simulation frameworks along co-optimisation in ITDS. These frameworks are categorised on multiple characteristics, such as simulation tools, synchronisation methods, and research topics. Furthermore, they propose a software platform that is focused on the integrated generation, transmission, distribution, and customer systems (IGTDCSs). The proposed framework also comprises several technological dimensions such as stochastic optimisation, high-performance computing, and high-level design software architecture for planning integrated and flexible power networks and optimising their technological trajectories and operational functioning considering uncertainties. By developing a prototype informed with software engineering and complex system design approaches, they will demonstrate the relevance of a unified vision of IGTDCS simulation in a minute-by-minute horizon, a vision that may later benefit electromagnetic transient simulation or stability co-simulation tools, in horizons from the microsecond to the second.

ELIXIR-IT HPC@CINECA: high performance computing resources for the bioinformatics community

BackgroundThe advent of Next Generation Sequencing (NGS) technologies and the concomitant reduction in sequencing costs allows unprecedented high throughput profiling of biological systems in a cost-efficient manner. Modern biological experiments are increasingly becoming both data and computationally intensive and the wealth of publicly available biological data is introducing bioinformatics into the “Big Data” era. For these reasons, the effective application of High Performance Computing (HPC) architectures is becoming progressively more recognized also by bioinformaticians.Here we describe HPC resources provisioning pilot programs dedicated to bioinformaticians, run by the Italian Node of ELIXIR (ELIXIR-IT) in collaboration with CINECA, the main Italian supercomputing center.ResultsStarting from April 2016, CINECA and ELIXIR-IT launched the pilot Call “ELIXIR-IT HPC@CINECA”, offering streamlined access to HPC resources for bioinformatics. Resources are made available either through web front-ends to dedicated workflows developed at CINECA or by providing direct access to the High Performance Computing systems through a standard command-line interface tailored for bioinformatics data analysis. This allows to offer to the biomedical research community a production scale environment, continuously updated with the latest available versions of publicly available reference datasets and bioinformatic tools. Currently, 63 research projects have gained access to the HPC@CINECA program, for a total handout of ~ 8 Millions of CPU/hours and, for data storage, ~ 100 TB of permanent and ~ 300 TB of temporary space.ConclusionsThree years after the beginning of the ELIXIR-IT HPC@CINECA program, we can appreciate its impact over the Italian bioinformatics community and draw some considerations. Several Italian researchers who applied to the program have gained access to one of the top-ranking public scientific supercomputing facilities in Europe. Those investigators had the opportunity to sensibly reduce computational turnaround times in their research projects and to process massive amounts of data, pursuing research approaches that would have been otherwise difficult or impossible to undertake. Moreover, by taking advantage of the wealth of documentation and training material provided by CINECA, participants had the opportunity to improve their skills in the usage of HPC systems and be better positioned to apply to similar EU programs of greater scale, such as PRACE. To illustrate the effective usage and impact of the resources awarded by the program - in different research applications - we report five successful use cases, which have already published their findings in peer-reviewed journals.

Performance analysis of Volna-OP2 – massively parallel code for tsunami modelling

The software package Volna-OP2 is a robust and efficient code capable of simulating the complete life cycle of a tsunami whilst harnessing the latest High Performance Computing (HPC) architectures. In this paper, a comprehensive error analysis and scalability study of the GPU version of the code is presented. A novel decomposition of the numerical errors into the dispersion and dissipation components is explored. Most tsunami codes exhibit amplitude smearing and/or phase lagging/leading, so the decomposition shown here is a new approach and novel tool for explaining these occurrences.To date, Volna-OP2 has been widely used by the tsunami modelling community. In particular its computational efficiency has allowed various sensitivity analyses and uncertainty quantification studies. Due to the number of simulations required, there is always a trade-off between accuracy and runtime when carrying out these statistical studies. The analysis presented in this paper will guide the user towards an acceptable level of accuracy within a given runtime.

Performance and Energy Assessment of a Lattice Boltzmann Method Based Application on the Skylake Processor

This paper presents the performance analysis for both the computing performance and the energy efficiency of a Lattice Boltzmann Method (LBM) based application, used to simulate three-dimensional multicomponent turbulent systems on massively parallel architectures for high-performance computing. Extending results reported in previous works, the analysis is meant to demonstrate the impact of using optimized data layouts designed for LBM based applications on high-end computer platforms. A particular focus is given to the Intel Skylake processor and to compare the target architecture with other models of the Intel processor family. We introduce the main motivations of the presented work as well as the relevance of its scientific application. We analyse the measured performances of the implemented data layouts on the Skylake processor while scaling the number of threads per socket. We compare the results obtained on several CPU generations of the Intel processor family and we make an analysis of energy efficiency on the Skylake processor compared with the Intel Xeon Phi processor, finally adding our interpretation of the presented results.

Development of a CPU/GPU portable software library for Lagrangian–Eulerian simulations of liquid sprays

The Lagrangian–Eulerian method is widely used for simulations of fuel sprays in turbulent combustion because of the advantage of treating the spray droplets as discrete points. One challenge of the Lagrangian–Eulerian method is the intense computational requirement when tracking the large number of Lagrangian particles needed for high fidelity. We have developed a performance-portable library, Grit, to track the Lagrangian particles in parallel on central processing unit (CPU) and graphics processing unit (GPU) accelerated high performance computing (HPC) architectures. Grit is a C++ library which employs Message Passing Interface (MPI) for distributed memory parallelism and Kokkos programming model for on-node shared memory parallelism with performance portability across different architectures of GPUs and multi-core/manycore CPUs. The parallel algorithms, key parallel kernels, and their performances on the pre-exascale supercomputer, Summit, are presented. Grit is coupled with a direct numerical simulation (DNS) solver, S3D, for multiphase simulations. A conservative formulation has been developed and implemented in Grit for phase coupling with thermodynamic consistency. The formulation separates the conservation of mass, momentum and energy from the physical models to prevent accidental violation of conservation laws due to inconsistent models. The formulation also enforces consistent definitions of enthalpies of the fuel for both phases and the latent heat of evaporation. Simulations of turbulent particle-laden flow and the evaporation of dilute turbulent spray jet are performed to verify the software implementation, and to demonstrate the scalability of Grit for large-scale multiphase simulations.

Transformations of High-Level Synthesis Codes for High-Performance Computing

Spatial computing architectures promise a major stride in performance and energy efficiency over the traditional load/store devices currently employed in large scale computing systems. The adoption of high-level synthesis (HLS) from languages such as C++ and OpenCL has greatly increased programmer productivity when designing for such platforms. While this has enabled a wider audience to target spatial computing architectures, the optimization principles known from traditional software design are no longer sufficient to implement high-performance codes, due to fundamentally distinct aspects of hardware design, such as programming for deep pipelines, distributed memory resources, and scalable routing. To alleviate this, we present a collection of optimizing transformations for HLS, targeting scalable and efficient architectures for high-performance computing (HPC) applications. We systematically identify classes of transformations (pipelining, scalability, and memory), the characteristics of their effect on the HLS code and the resulting hardware (e.g., increasing data reuse or resource consumption), and the objectives that each transformation can target (e.g., resolve interface contention, or increase parallelism). We show how these can be used to efficiently exploit pipelining, on-chip distributed fast memory, and on-chip dataflow, allowing for massively parallel architectures. To quantify the effect of various transformations, we cover the optimization process of a sample set of HPC kernels, provided as open source reference codes. We aim to establish a common toolbox to guide both performance engineers and compiler engineers in tapping into the performance potential offered by spatial computing architectures using HLS.

Implementation of All 27 Possible Univariate Ternary Logics With a Single ZnO Memristor

Memristors with small size, fast speed, low power, CMOS compatibility and nonvolatile modulation of device resistance are promising candidates for the next-generation data storage and in-memory logic computing paradigm. In comparison to the binary logics enabled by memristor devices, multi-valued logics can provide higher computation efficiency with simple operation scheme, reduced circuit complexity, and smaller chip area. In this contribution, we demonstrate that all the 27 univariate ternary logic operations can be realized with a single ZnO three-state resistive switching memristor in at most three steps. The nonvolatile modulation characteristics of the memristor allow the read step to be independent of the logic operation and capacitate logic-in-memory applications. The present methodology could be beneficial for constructing future high-performance computation architectures.

GPU empowered pipelines for calculating genome-wide kinship matrices with ultra-high dimensional genetic variants and facilitating 1D and 2D GWAS.

Genome-wide association study (GWAS) is a powerful approach that has revolutionized the field of quantitative genetics. Two-dimensional GWAS that accounts for epistatic genetic effects needs to consider the effects of marker pairs, thus quadratic genetic variants, compared to one-dimensional GWAS that accounts for individual genetic variants. Calculating genome-wide kinship matrices in GWAS that account for relationships among individuals represented by ultra-high dimensional genetic variants is computationally challenging. Fortunately, kinship matrix calculation involves pure matrix operations and the algorithms can be parallelized, particular on graphics processing unit (GPU)-empowered high-performance computing (HPC) architectures. We have devised a new method and two pipelines: KMC1D and KMC2D for kinship matrix calculation with high-dimensional genetic variants, respectively, facilitating 1D and 2D GWAS analyses. We first divide the ultra-high-dimensional markers and marker pairs into successive blocks. We then calculate the kinship matrix for each block and merge together the block-wise kinship matrices to form the genome-wide kinship matrix. All the matrix operations have been parallelized using GPU kernels on our NVIDIA GPU-accelerated server platform. The performance analyses show that the calculation speed of KMC1D and KMC2D can be accelerated by 100–400 times over the conventional CPU-based computing.

PARPA: 고성능 컴퓨팅을 위한 이기종 아키텍처 동시 사용 프레임워크

PARPA: 고성능 컴퓨팅을 위한 이기종 아키텍처 동시 사용 프레임워크

Machine Learning Acceleration

The articles in this special section focus on machine learning acceleration. In recent years, machine learning (ML) has become one of the most important pillars of computing industry, driven by the remarkable advances in the ML algorithms coupled with innovation in high performance computer architecture, as typified by GPUs. These advances in tandem propelled ML to a new era of real-world applicability and impact in almost every aspect of the IT industry. As the importance of ML in the emerging applications increases, the ML accelerators have become a critical component of every modern computing system, from data centers to mobile/IoT devices.

High performance parallel KMP algorithm on a heterogeneous architecture

String matching algorithm is widely used in many application areas such as bio-informatics, network intrusion detection, computer virus scan, among many others. KMP (Knuth-Morris-Pratt) algorithm is commonly used for its fast execution time compared with many other string matching algorithms when applied to large input texts. However, the performance of the KMP algorithm is limited when the input text size increases significantly beyond a certain limit. In this paper, we propose a high performance parallel KMP algorithm on the Heterogeneous High Performance Computing (HPC) architecture based on the general-purpose multicore microprocessor and the Graphic Processing Unit (GPU) using OpenCL as the GPGPU (General Purpose computing using Graphic Processing Unit) platform. The proposed parallel KMP algorithm mainly focuses on optimizing the CPU-GPU memory hierarchy by overlapping the data transfer between the CPU memory and the GPU memory with the string matching operations on the GPU. It also optimizes the allocation of the work-groups and the work-items, and places the pattern data and the Failure Table to the on-chip shared-memory of the GPU. The experimental results show that the optimized parallel KMP algorithm leads up to ~8 times faster execution time.

High order parallelisation of an unstructured grid, discontinuous-Galerkin finite element solver for the Boltzmann–BGK equation

ABSTRACTThis paper outlines the implementation and performance of a parallelisation approach involving partitioning of both physical space and velocity space domains for finite element solution of the Boltzmann-BGK equation. The numerical solver is based on a discontinuous Taylor–Galerkin approach. To the authors' knowledge this is the first time a ‘high order’ parallelisation, or `phase space parallelisation', approach has been attempted in conjunction with a numerical solver of this type. Restrictions on scalability have been overcome with the implementation detailed in this paper. The developed algorithm has major advantages over continuum solvers in applications where strong discontinuities prevail and/or in rarefied flow applications where the Knudsen number is large. Previous work by the authors has outlined the range of applications that this solver is capable of tackling. The paper demonstrates that the high order parallelisation implemented is significantly more effective than previous implementations at exploiting High Performance Computing architectures.

MRT-Lattice Boltzmann Model for Multilayer Shallow Water Flow

The objectives of this study are to introduce a multiple-relaxation-time (MRT) lattice Boltzmann model (LBM) to simulate multilayer shallow water flows and to introduce graphics processing unit (GPU) computing to accelerate the lattice Boltzmann model. Using multiple relaxation times in the lattice Boltzmann model has an advantage of handling very low kinematic viscosity without causing a stability problem in the shallow water equations. This study develops a multilayer MRT-LBM to solve the multilayer Saint-Venant equations to obtain horizontal flow velocities in various depths. In the multilayer MRT-LBM, vertical kinematic viscosity forcing is the key term to couple adjacent layers. We implemented the multilayer MRT-LBM to a GPU-based high-performance computing (HPC) architecture. The multilayer MRT-LBM was verified by analytical solutions for cases of wind-driven, density-driven, and combined circulations with non-uniform bathymetry. The results show good speedup and scalability for large problems. Numerical solutions compared well to the analytical solutions. The multilayer MRT-LBM is promising for simulating lateral and vertical distributions of the horizontal velocities in shallow water flow.

Alleviating Scalability Limitation of Accelerator-Based Platforms

Accelerator-based chip multiprocessors (ACMPs), which combine application-specific HW accelerators (ACCs) with host processor core(s), are promising architectures for high-performance and power-efficient computing. However, ACMPs with many ACCs have scalability limitations. The ACCs’ performance benefits can be overshadowed by bottlenecks on shared resources of processor core(s), communication fabric/DMA, and on-chip memory. Primarily, this is rooted in the ACCs’ data access and the orchestration dependency. Due to very loosely defined ACC communication semantics, and relying on general architectures, the resources bottlenecks hamper performance. This paper explores and alleviates the scalability limitations of ACMPs. To this end, this paper first proposes ACMPerf , an analytical model to capture the impact of the resources bottlenecks on the achievable ACCs’ benefits. Then, this paper identifies and formalizes ACC communication semantics which paves the path toward a more scalable integration of ACCs. The semantics describe four primary aspects: 1) data access; 2) data granularity; 3) data marshalling; and 4) synchronization. Finally, this paper proposes a novel architecture of transparent self-synchronizing accelerators (TSS). TSS efficiently realizes our identified communication semantics of direct ACC-to-ACC connections often occurring in streaming applications. TSS delivers more of the ACCs’ benefits than conventional ACMP architectures. Given the same set of ACCs, TSS has up to $130 \times $ higher throughput and $78 \times $ lower energy consumption, mainly due to reducing the load on shared architectural resources by $78.3 \times $ .

Toward data-driven architectural support in improving the performance of future HPC architectures

We propose architectures based on Data-Driven Multithreading (DDM), a hybrid control-flow/data-flow model, to address the concurrency challenges faced by future High-Performance Computing (HPC) systems. We focus on the design and implementation of an optimized hardware Thread Scheduling Unit (TSU) and its integration into a multi-core system dubbed MiDAS. The TSU is the core of the DDM model and it orchestrates the execution of multiple threads on sequential processors based on data availability. MiDAS was prototyped on a Xilinx Virtex-6 FPGA and extensively evaluated using several micro-benchmarks, showing that it achieves linearly-growing performance as the processing core count increases even when running benchmarks comprising very small problem sizes. Under the largest problem size tested and with all 8 available cores being utilized, MiDAS achieves an average speedup of 7.91×, exhibiting 98.8% utilization efficiency. Further, several results pertaining to the proposed hardware TSU are provided, including FPGA real estate requirements, where it is found that MiDAS’s TSU demands relatively small overheads and reduced power consumption, while various TSU operations adhere to low latency responses. To back said claims, the proposed DDM-based TSU is compared with the Task Superscalar architecture that implements the StarSs programming framework in hardware. As such, comparison results show that the proposed TSU requires much less of both hardware investment and energy consumption to operate. Specifically, Task Superscalar is found to be 4.94 × larger than the DDM-supporting TSU in terms of slice register requirements and 11.34 × larger with respect to the slice look-up table count. Last, the hardware TSU is compared with a software TSU implementation offering identical functionalities, with both being run on an FPGA fabric under a synthetic application, where a detailed performance evaluation shows that MiDAS’s hardware-implemented TSU significantly outperforms its software-based TSU counterpart.

Foreword to the special issue of the workshop on high performance computing systems (XVIII Simpósio em Sistemas Computacionais de Alto Desempenho, WSCAD 2017)

Foreword to the special issue of the workshop on high performance computing systems (XVIII Simpósio em Sistemas Computacionais de Alto Desempenho, WSCAD 2017)

Runtime mechanisms to survive new HPC architectures: A use case in human respiratory simulations

Computational fluid and particle dynamics (CFPD) simulations are of paramount importance for studying and improving drug effectiveness. Computational requirements of CFPD codes demand high-performance computing (HPC) resources. For these reasons, we introduce and evaluate in this article system software techniques for improving performance and tolerating load imbalance on a state-of-the-art production CFPD code. We demonstrate benefits of these techniques on Intel-, IBM- and Arm-based HPC technologies ranked in the Top500 supercomputers, showing the importance of using mechanisms applied at runtime to improve the performance independently of the underlying architecture. We run a real CFPD simulation of particle tracking on the human respiratory system, showing performance improvements of up to 2×, across different architectures, while applying runtime techniques and keeping constant the computational resources.

Pointerchain: Tracing pointers to their roots – A case study in molecular dynamics simulations

As scientific frameworks become sophisticated, so do their data structures. A data structure typically includes pointers and arrays to other structures in order to preserve application’s state. In order to ensure data consistency from a scientific application on a modern high performance computing (HPC) architecture, the management of such pointers on the host and the device, has become complicated in terms of memory allocations because they occupy separate memory spaces. It becomes so severe that one must go through a chain of pointers to extract the effective address. In this paper, we propose to reduce the need of excessive data transfer by introducing the idea of pointerchain, a directive that replaces the pointer chains with their corresponding effective address inside the parallel region of a code. Based on our analysis, pointerchain leads to a 39% and 38% reduction in the amount of generated codes and the total executed instructions, respectively.With pointerchain, we have parallelized CoMD, a Molecular Dynamics (MD) proxy application on heterogeneous HPC architectures while maintaining a single portable codebase. This portable codebase utilizes OpenACC, an emerging directive-based programming model, to address the need of memory allocations from three computational kernels in CoMD. Two of the three embarrassingly parallel kernels highly benefit from OpenACC and perform better than the hand-written CUDA counterparts. The third kernel performed 61% of peak performance of its CUDA counterpart. The three kernels are common modules in any MD simulations. Our findings provides useful insights into parallelizing legacy MD software across heterogeneous platforms.

Design and Evaluation of a Scalable Engine for 3D-FFT Computation in an FPGA Cluster

The Three Dimensional Fast Fourier Transform (3D-FFT) is commonly used to solve the partial differential equations describing the system evolution in several physical phenomena, such as the motion of viscous fluids described by the Navier-Stokes equations. Simulation of such problems requires the use of a parallel High-Performance Computing architecture since the size of the problem grows with the cube of the FFT size, and the representation of the single point comprises several double precision floating- point complex numbers. Modern High-Performance Computing (HPC) systems are considering the inclusion of FPGAs as components of this computing architecture because they can combine effective hardware acceleration capabilities and dedicated communication facilities. Furthermore, the network topology can be optimized for the specific calculation that the cluster must perform, especially in the case of algorithms limited by the data exchange delay between the processors. In this paper, we explore an HPC design that uses FPGA accelerators to compute the 3DFFT. We devise a scalable FFT engine based on a custom radix-2 double-precision core that is used to implement the Decimation in Frequency version of the Cooley-Tukey FFT algorithm. The FFT engine can be adapted to different technology constraints and networking topologies by adjusting the number of cores and configuration parameters in order to minimize the overall calculation time. We compare the various possible configurations with the technological limits of available hardware. Finally, we evaluate the bandwidth required for continuous FFT execution in the APEnet toroidal mesh network.

The international race towards Exascale in Europe

In this article, we describe the context in which an international race towards Exascale computing has started. We cover the political and economic context and make a review of the recent history in high performance computing (HPC) architectures, with special emphasis on the recently announced European initiatives to reach Exascale computing in Europe. We conclude by describing current challenges and trends.