Modern High-performance Computing Research Articles

Exploiting OpenMP and OpenACC to accelerate a geometric approach to molecular docking in heterogeneous HPC nodes

In drug discovery, molecular docking is the task in charge of estimating the position of a molecule when interacting with the docking site. This task is usually used to perform screening of a large library of molecules, in the early phase of the process. Given the amount of candidate molecules and the complexity of the application, this task is usually performed using high-performance computing (HPC) platforms. In modern HPC systems, heterogeneous platforms provide a better throughput with respect to homogeneous platforms. In this work, we ported and optimized a molecular docking application to a heterogeneous system, with one or more GPU accelerators, leveraging a hybrid OpenMP and OpenACC approach. The target application focuses on the virtual screening phases in the drug discovery process, and it is based on geometric transformations of the target ligands. We prove that our approach has a better exploitation of the node compared to pure CPU/GPU data splitting approaches, reaching a throughput improvement up to 25% while considering the same computing node.

Algorithmic Aspects of the LES-PBE-PDF Method for Modeling Soot Particle Size Distributions in Turbulent Flames

ABSTRACTIn recent times, the LES-PBE-PDF framework has been developed to couple large eddy simulation (LES) and population balance models (PBE) for the description of soot formation in turbulent hydrocarbon flames. This approach is based on a modeled evolution equation for the LES-filtered probability density function (pdf) associated with the instantaneous gas composition and soot particle size distribution. Here, the interaction of turbulence with chemical reactions and soot formation can be represented without approximations on part of the chemical and soot formation kinetics, while effects due to turbulent transport and molecular diffusion require closure. In view of an efficient numerical solution scheme, we previously proposed to combine a statistically equivalent reformulation of the joint scalar-number density pdf based on Eulerian stochastic fields with a time-explicit adaptive grid discretization in particle size space and a fractional time stepping scheme. In this article, we present algorithmic aspects and relay implementational details for a consistent semi-discrete formulation of the PBE fractional step as well as an effective dynamic load balancing scheme for both the chemical reaction and PBE fractional steps. Considering soot formation in the Delft III turbulent diffusion flame as a test case, we show that the persisting load imbalance is almost negligible on average and give evidence of linear strong scaling on a modern high performance computer for moderate numbers of compute nodes.

Energy-Aware High-Performance Computing: Survey of State-of-the-Art Tools, Techniques, and Environments

The paper presents state of the art of energy-aware high-performance computing (HPC), in particular identification and classification of approaches by system and device types, optimization metrics, and energy/power control methods. System types include single device, clusters, grids, and clouds while considered device types include CPUs, GPUs, multiprocessor, and hybrid systems. Optimization goals include various combinations of metrics such as execution time, energy consumption, and temperature with consideration of imposed power limits. Control methods include scheduling, DVFS/DFS/DCT, power capping with programmatic APIs such as Intel RAPL, NVIDIA NVML, as well as application optimizations, and hybrid methods. We discuss tools and APIs for energy/power management as well as tools and environments for prediction and/or simulation of energy/power consumption in modern HPC systems. Finally, programming examples, i.e., applications and benchmarks used in particular works are discussed. Based on our review, we identified a set of open areas and important up-to-date problems concerning methods and tools for modern HPC systems allowing energy-aware processing.

Pointerchain: Tracing pointers to their roots – A case study in molecular dynamics simulations

As scientific frameworks become sophisticated, so do their data structures. A data structure typically includes pointers and arrays to other structures in order to preserve application’s state. In order to ensure data consistency from a scientific application on a modern high performance computing (HPC) architecture, the management of such pointers on the host and the device, has become complicated in terms of memory allocations because they occupy separate memory spaces. It becomes so severe that one must go through a chain of pointers to extract the effective address. In this paper, we propose to reduce the need of excessive data transfer by introducing the idea of pointerchain, a directive that replaces the pointer chains with their corresponding effective address inside the parallel region of a code. Based on our analysis, pointerchain leads to a 39% and 38% reduction in the amount of generated codes and the total executed instructions, respectively.With pointerchain, we have parallelized CoMD, a Molecular Dynamics (MD) proxy application on heterogeneous HPC architectures while maintaining a single portable codebase. This portable codebase utilizes OpenACC, an emerging directive-based programming model, to address the need of memory allocations from three computational kernels in CoMD. Two of the three embarrassingly parallel kernels highly benefit from OpenACC and perform better than the hand-written CUDA counterparts. The third kernel performed 61% of peak performance of its CUDA counterpart. The three kernels are common modules in any MD simulations. Our findings provides useful insights into parallelizing legacy MD software across heterogeneous platforms.

Design and Evaluation of a Scalable Engine for 3D-FFT Computation in an FPGA Cluster

The Three Dimensional Fast Fourier Transform (3D-FFT) is commonly used to solve the partial differential equations describing the system evolution in several physical phenomena, such as the motion of viscous fluids described by the Navier-Stokes equations. Simulation of such problems requires the use of a parallel High-Performance Computing architecture since the size of the problem grows with the cube of the FFT size, and the representation of the single point comprises several double precision floating- point complex numbers. Modern High-Performance Computing (HPC) systems are considering the inclusion of FPGAs as components of this computing architecture because they can combine effective hardware acceleration capabilities and dedicated communication facilities. Furthermore, the network topology can be optimized for the specific calculation that the cluster must perform, especially in the case of algorithms limited by the data exchange delay between the processors. In this paper, we explore an HPC design that uses FPGA accelerators to compute the 3DFFT. We devise a scalable FFT engine based on a custom radix-2 double-precision core that is used to implement the Decimation in Frequency version of the Cooley-Tukey FFT algorithm. The FFT engine can be adapted to different technology constraints and networking topologies by adjusting the number of cores and configuration parameters in order to minimize the overall calculation time. We compare the various possible configurations with the technological limits of available hardware. Finally, we evaluate the bandwidth required for continuous FFT execution in the APEnet toroidal mesh network.

Implementation of Feldman-Cousins corrections and oscillation calculations in the HPC environment for the NOvA Experiment

Analysis of neutrino oscillation data involves a combination of complex fitting procedures and statistical correction techniques that are used to determine the full three-flavor PMNS parameters and constraint contours. These techniques rely on computationally intensive “multi-universe” stochastic modeling. The process of calculating these contours and corrections can dominate final stages of the data analysis and become a bottleneck for examining the ef-fect of systematic variations on the final results. As part of the DOE SciDAC-4 sponsored research program, we present a new implementation of a neutrino oscillation fitting and framework to carry out calculations of Feldman-Cousins corrections. This implementation is specifically designed and optimized to operate on modern High-Performance Computing (HPC) facilities. We present the performance of the system in calculating allowed regions for the NOvA experiment based on 8.9 x 1020 and 6.9 x 1020 protons-on-target (POT) neutrino and antineutrino datasets, and compare the performance of this new implementation run at the NERSC supercomputing facility with that from methods used previously by the NOvA collaboration running on grid computing facilities.

Performance Improvement of 4-Bit Static CMOS Carry Look-Ahead Adder Using Modified Circuits for Carry Propagate and Generate Terms

Carry Look-Ahead Adder (CLA) is considered as one of the most widely used adder topologies which are used in high performance computing systems. In this research, an improved version of 4-bit CLA adder has been proposed. Performance improvement of 4-bit CLA adder has been made by using hybrid AND and XOR gates in the input side for generating carry propagate and carry generate terms. The CLA circuits are kept exactly the same as the conventional one. Performance of the proposed modified 4-bit CLA adder has been evaluated and compared with the conventional design using Cadence tools in 90 nm technology node. Performance has been evaluated and compared in terms of average power, propagation delay and power delay product. The proposed modified design exhibited significant improvement in performance while compared with the conventional one. Enhancement done by the proposed 4-bit CLA adder design in average power, propagation delay and PDP were 14.96%, 11.76% and 25.32% respectively. In addition to performance enhancement, transistor count require for the proposed design is quite less compared to the conventional design which result in less surface area on chip. Moreover, less transistor count accounts for less power dissipation. Hence, utilizing the proposed design in modern high-performance computing systems would bring about high-performance improvements.

Run-Time Resource Management Controller for Power Efficiency of GP-GPU Architecture

The demand for high-performance computing (HPC) has been increasing since the invention of computing technology. This led to the invocation of sophisticated multi/many-core processors with high performance. Graphical processing units (GPUs) have emerged as an important innovation in the many-core era as it features a high number of processors. The GPU acts as a computational accelerator that can significantly reduce the computational time of the HPC, as it can offer standout parallelism for high-end computing applications such as graphics designing. However, increasing the resources has resulted in higher power consumption and heat dissipation which has been a challenging problem for modern HPC Units. On the other hand, because of the dynamic nature of workload, a large amount of parallelism, offered by these many-core processors, is often underutilized. An ideal system would be smart enough to efficiently utilize resources and save power where less workload is available. Reducing the resources dynamically has direct implications on the performance of the system. However, if less workload is available, reducing the resources would not harm the performance, rather it would save power with less to no trade-off in overall throughput of the system. In this paper, a smart power and performance efficient resource management controller for general purpose-GPU architecture is presented. The proposed controller, based on a feedback mechanism, keeps on analyzing the current frequency of central processing unit (CPU) and GPU, number of active cores of the CPU and utilization of CPU and GPU. On the basis of collected data, the proposed controller which features fuzzy type-2 as an optimizing mechanism tries to create a balance between performance and power consumption. The results are evaluated against various benchmarks on NVIDIA TK1 GPU kit and by using dynamic voltage and frequency scaling and core gating, up to 47 % reduction in power consumption has been achieved.

Optimizing I/O Performance of HPC Applications with Autotuning

Parallel Input output is an essential component of modern high-performance computing (HPC). Obtaining good I/O performance for a broad range of applications on diverse HPC platforms is a major challenge, in part, because of complex inter dependencies between I/O middleware and hardware. The parallel file system and I/O middleware layers all offer optimization parameters that can, in theory, result in better I/O performance. Unfortunately, the right combination of parameters is highly dependent on the application, HPC platform, problem size, and concurrency. Scientific application developers do not have the time or expertise to take on the substantial burden of identifying good parameters for each problem configuration. They resort to using system defaults, a choice that frequently results in poor I/O performance. We expect this problem to be compounded on exascale-class machines, which will likely have a deeper software stack with hierarchically arranged hardware resources. We present as a solution to this problem an autotuning system for optimizing I/O performance, I/O performance modeling, I/O tuning, and I/O patterns. We demonstrate the value of this framework across several HPC platforms and applications at scale.

A Study on Cross-Architectural Modelling of Power Consumption Using Neural Networks

On the path to Exascale, the goal of High Performance Computing (HPC) to achieve maximum performance becomes the goal of achieving maximum performance under strict power constraint. Novel approaches to hardware and software co-design of modern HPC systems have to be developed to address such challenges. In this paper, we study prediction of power consumption of HPC systems using metrics obtained from hardware performance counters. We argue that this methodology is portable across different micro architecture implementations and compare results obtained on Intel 64, IBMR and Cavium ThunderXR ARMv8 microarchitectures.We discuss optimal number and type of hardware performance counters required to accurately predict power consumption.We compare accuracy of power predictions provided by models based on Linear Regression (LR) and Neural Networks (NN). We find that the NN-based model provides better accuracy of predictions than the LR model. We also find, that presently it is not yet possible to predict power consumption on a given microarchitecture using data obtained on a different microarchitecture. Results of our work can be used as a starting point for developing unified, cross-architectural models for predicting power consumption.

Contention-Aware Reliability Efficient Scheduling on Heterogeneous Computing Systems

Energy efficiency and system reliability are the two main measurements in modern high-performance computing. The majority of previous recent studies have focused on realizing parallel task scheduling with low energy consumption or fast execution time. These approaches were developed with the classic scheduling model. However, the contention model is gaining increasing recognition as a more practical tool to create accurate and efficient schedules. This study proposes a contention-aware reliability management with deadline and energy budget constraints (CARMEB) algorithm for parallel task scheduling in heterogeneous computing systems. CARMEB involves three phases, namely, task priority calculation, communication edge allocation, and slack reclaiming. Results are validated by conducting extensive experiments, including randomly generated task graphs and three types of task graphs in real-world applications. This study demonstrates that our algorithm significantly improves system reliability.

Managing high-performance computing applications as an on-demand service on federated clouds

There are several challenges (e.g., imbalance between supply and demand of hardware resources and software licenses, and usability) under modern High-Performance Computing (HPC) environment. As a means of providing an on-demand service for end users, we propose a Software-as-a-Service (SaaS) approach for managing commercial HPC applications as a Web-based service deployed on top of federated clouds. Some inter-trusted private or public clouds are federated to create a unified service platform with a large amount of hardware resources. In addition, an on-demand, pay-per-use model for Web-service-enabled HPC applications is proposed. Further, we provide an economic analysis of the proposed approach from the perspective of end users, cloud service providers, and Independent Software Vendors (ISVs). We conduct a simulation using two HPC application services on three federated clouds. A combined Quality of Service (QoS) and economic evaluation demonstrates a better effect of the proposed approach comparing with existing HPC platforms.

MR-Advisor: A comprehensive tuning, profiling, and prediction tool for MapReduce execution frameworks on HPC clusters

MapReduce is the most popular parallel computing framework for big data processing which allows massive scalability across distributed computing environment. Advanced RDMA-based design of Hadoop MapReduce has been proposed that alleviates the performance bottlenecks in default Hadoop MapReduce by leveraging the benefits from RDMA. On the other hand, data processing engine, Spark, provides fast execution of MapReduce applications through in-memory processing. Performance optimization for these contemporary big data processing frameworks on modern High-Performance Computing (HPC) systems is a formidable task because of the numerous configuration possibilities in each of them. In this paper, we propose MR-Advisor, a comprehensive tuning, profiling, and prediction tool for MapReduce. MR-Advisor is generalized to provide performance optimizations for Hadoop, Spark, and RDMA-enhanced Hadoop MapReduce designs over different file systems such as HDFS, Lustre, and Tachyon. Performance evaluations reveal that, with MR-Advisor’s suggested values, the job execution performance can be enhanced by a maximum of 58% over the current best-practice values for user-level configuration parameters. To the best of our knowledge, this is the first tool that supports tuning and prediction for both Apache Hadoop and Spark, as well as the RDMA and Lustre-based advanced designs.

ARRC: A random ray neutron transport code for nuclear reactor simulation

A massively parallel implementation of a recently developed technique for numerically integrating the transport equation, The Random Ray Method (TRRM) (Tramm et al., 2017), is applied to several large reactor benchmark problems. The implementation, which is part of a new development called The Advanced Random Ray Code (ARRC), is one of the first parallel implementations of TRRM. Our goal is to better understand the accuracy and performance characteristics of TRRM on massive scale problems, and to provide community software that facilitates further algorithmic development and potentially its application to a broader class of problems. Key features of ARRC include extreme memory efficiency, domain decomposition, a task based parallel structure, and the ability to efficiently utilize Single Instruction Multiple Data (SIMD) vector units. These attributes lead to efficient performance on modern high performance computer (HPC) architectures, enabling the detailed simulation of reactor cores in three dimensions.

BrainFrame: a node-level heterogeneous accelerator platform for neuron simulations

Objective. The advent of high-performance computing (HPC) in recent years has led to its increasing use in brain studies through computational models. The scale and complexity of such models are constantly increasing, leading to challenging computational requirements. Even though modern HPC platforms can often deal with such challenges, the vast diversity of the modeling field does not permit for a homogeneous acceleration platform to effectively address the complete array of modeling requirements. Approach. In this paper we propose and build BrainFrame, a heterogeneous acceleration platform that incorporates three distinct acceleration technologies, an Intel Xeon-Phi CPU, a NVidia GP-GPU and a Maxeler Dataflow Engine. The PyNN software framework is also integrated into the platform. As a challenging proof of concept, we analyze the performance of BrainFrame on different experiment instances of a state-of-the-art neuron model, representing the inferior-olivary nucleus using a biophysically-meaningful, extended Hodgkin–Huxley representation. The model instances take into account not only the neuronal-network dimensions but also different network-connectivity densities, which can drastically affect the workload’s performance characteristics. Main results. The combined use of different HPC technologies demonstrates that BrainFrame is better able to cope with the modeling diversity encountered in realistic experiments while at the same time running on significantly lower energy budgets. Our performance analysis clearly shows that the model directly affects performance and all three technologies are required to cope with all the model use cases. Significance. The BrainFrame framework is designed to transparently configure and select the appropriate back-end accelerator technology for use per simulation run. The PyNN integration provides a familiar bridge to the vast number of models already available. Additionally, it gives a clear roadmap for extending the platform support beyond the proof of concept, with improved usability and directly useful features to the computational-neuroscience community, paving the way for wider adoption.

Impacts of optimization strategies on performance, power/energy consumption of a GPU based parallel reduction

In the era of modern high performance computing, GPUs have been considered an excellent accelerator for general purpose data-intensive parallel applications. To achieve application speedup from GPUs, many of performance-oriented optimization techniques have been proposed. However, in order to satisfy the recent trend of power and energy consumptions, power/energy-aware optimization of GPUs needs to be investigated with detailed analysis in addition to the performance-oriented optimization. In this work, in order to explore the impact of various optimization strategies on GPU performance, power and energy consumptions, we evaluate performance and power/energy consumption of a well-known application running on different commercial GPU devices with the different optimization strategies. In particular, in order to see the more generalized performance and power consumption patterns of GPU based accelerations, our evaluations are performed with three different Nvdia GPU generations (Fermi, Kepler and Maxwell architectures), various core clock frequencies and memory clock frequencies. We analyze how a GPU kernel execution is affected by optimization and what GPU architectural factors have much impact on its performance and power/energy consumption. This paper also categorizes which optimization technique primarily improves which metric (i.e., performance, power or energy efficiency). Furthermore, voltage frequency scaling (VFS) is also applied to examine the effect of changing a clock frequency on these metrics. In general, our work shows that effective GPU optimization strategies can improve the application performance significantly without increasing power and energy consumption.

Introduction to the Special Issue on High Performance Computing Solutions for Complex Problems

Introduction to the Special Issue on High Performance Computing Solutions for Complex Problems

Proposal for a Nanoscale Superconductive Memory

Energy efficiency is a key issue for modern high performance computing. Superconductive digital electronics has already demonstrated superior performances in terms of speed and energy dissipations. However, there is still the open issue of the realization of effective submicron scale superconductive memories. Superconducting nanowires represent the state-of-the-art of single-photon detectors. Their technology has also been used to realize three-terminal active devices, where the output response is triggered by a current pulse. The combination of the electrothermal mechanism of the nanowire and the magnetic coupling with a suitable material can be used for the realization of a nanowire-based memory device scalable to nanoscale. The principle of operation and material requirements are presented here. In particular, the feasibility of the proposed device using EuS as magnetic material and NbN as nanowire is discussed. By using a physical model of the nanowire dynamics, in terms of the spatial distribution of electron and phonon temperatures, the feasibility of the proposed device has been verified, through numerical simulations. The device configurations considered have the specific goal of realizing reliable and high-speed read and write operations, with the possibility of scalability to nanoscale.

A multi-aspect online tuning framework for HPC applications

Developing software applications for high-performance computing (HPC) requires careful optimizations targeting a myriad of increasingly complex, highly interrelated software, hardware and system components. The demands placed on minimizing energy consumption on extreme-scale HPC systems and the associated shift towards hete rogeneous architectures add yet another level of complexity to program development and optimization. As a result, the software optimization process is often seen as daunting, cumbersome and time-consuming by software developers wishing to fully exploit HPC resources. To address these challenges, we have developed the Periscope Tuning Framework (PTF), an online automatic integrated tuning framework that combines both performance analysis and performance tuning with respect to the myriad of tuning parameters available to today’s software developer on modern HPC systems. This work introduces the architecture, tuning model and main infrastructure components of PTF as well as the main tuning plugins of PTF and their evaluation.

Comparative Implementation of High Performance Computing for Power System Dynamic Simulations

Dynamic simulation for transient stability assessment is one of the most important, but intensive, computational tasks for power system planning and operation. Several commercial software tools provide functionality for performing multiple dynamic simulations such as those in contingency analysis simultaneously on parallel computers. Nevertheless, a single dynamic simulation is still a time consuming process performed sequentially on one single computing core as the tools were originally designed. Modern high performance computing (HPC) holds the promise to accelerate a single dynamic simulation by parallelizing its kernel algorithms without compromising computational accuracy. Parallelizing a single dynamic simulation is a much more challenging problem than the contingency-type parallel computing. It requires a good match between simulation algorithms and computing hardware. This paper provides guidance for such a match so as to design and implement parallel dynamic simulation to maximize the utilization of computing hardware and the performance of the simulation. The guidance is derived through comparative implementation of four parallel dynamic simulation schemes in two state-of-the-art HPC environments: 1) message passing interface and 2) open multi-processing. The scalability and speedup performance of parallelized dynamic simulation are thoroughly studied to determine the impact of simulation algorithms and computing hardware configurations. Several testing cases are presented to illustrate the derived guidance.