OpenMP Implementation Research Articles

A GPU Implementation of the Correlation Technique for Real-time Fourier Domain Pulsar Acceleration Searches

Abstract The study of binary pulsars enables tests of general relativity. Orbital motion in binary systems causes the apparent pulsar spin frequency to drift, reducing the sensitivity of periodicity searches. Acceleration searches are methods that account for the effect of orbital acceleration. Existing methods are currently computationally expensive, and the vast amount of data that will be produced by next-generation instruments such as the Square Kilometre Array necessitates real-time acceleration searches, which in turn requires the use of high-performance computing (HPC) platforms. We present our implementation of the correlation technique for the Fourier Domain Acceleration Search (FDAS) algorithm on Graphics Processor Units (GPUs). The correlation technique is applied as a convolution with multiple finite impulse response (FIR) filters in the Fourier domain. Two approaches are compared: the first uses the NVIDIA cuFFT library for applying Fast Fourier transforms (FFTs) on the GPU, and the second contains a custom FFT implementation in GPU shared memory. We find that the FFT shared-memory implementation performs between 1.5 and 3.2 times faster than our cuFFT-based application for smaller but sufficient filter sizes. It is also 4–6 times faster than the existing GPU and OpenMP implementations of FDAS. This work is part of the AstroAccelerate project, a many-core accelerated time-domain signal-processing library for radio astronomy.

Unleashing Fine-Grained Parallelism on Embedded Many-Core Accelerators with Lightweight OpenMP Tasking

In recent years, programmable many-core accelerators (PMCAs) have been introduced in embedded systems to satisfy stringent performance/Watt requirements. This has increased the urge for programming models capable of effectively leveraging hundreds to thousands of processors. Task-based parallelism has the potential to provide such capabilities, offering high-level abstractions to outline abundant and irregular parallelism in embedded applications. However, efficiently supporting this programming paradigm on embedded PMCAs is challenging, due to the large time and space overheads it introduces. In this paper we describe a lightweight OpenMP tasking runtime environment (RTE) design for a state-of-the-art embedded PMCA, the Kalray MPPA 256. We provide an exhaustive characterization of the costs of our RTE, considering both synthetic workload and real programs, and we compare to several other tasking RTEs. Experimental results confirm that our solution achieves near-ideal parallelization speedups for tasks as small as 5K cycles, and an average speedup of 12 $\times$ for real benchmarks, which is $\approx 60\%$ higher than what we observe with the original Kalray OpenMP implementation.

Parallelisation of equation-based simulation programs on heterogeneous computing systems.

Numerical solutions of equation-based simulations require computationally intensive tasks such as evaluation of model equations, linear algebra operations and solution of systems of linear equations. The focus in this work is on parallel evaluation of model equations on shared memory systems such as general purpose processors (multi-core CPUs and manycore devices), streaming processors (Graphics Processing Units and Field Programmable Gate Arrays) and heterogeneous systems. The current approaches for evaluation of model equations are reviewed and their capabilities and shortcomings analysed. Since stream computing differs from traditional computing in that the system processes a sequential stream of elements, equations must be transformed into a data structure suitable for both types. The postfix notation expression stacks are recognised as a platform and programming language independent method to describe, store in computer memory and evaluate general systems of differential and algebraic equations of any size. Each mathematical operation and its operands are described by a specially designed data structure, and every equation is transformed into an array of these structures (a Compute Stack). Compute Stacks are evaluated by a stack machine using a Last In First Out queue. The stack machine is implemented in the DAE Tools modelling software in the C99 language using two Application Programming Interface (APIs)/frameworks for parallelism. The Open Multi-Processing (OpenMP) API is used for parallelisation on general purpose processors, and the Open Computing Language (OpenCL) framework is used for parallelisation on streaming processors and heterogeneous systems. The performance of the sequential Compute Stack approach is compared to the direct C++ implementation and to the previous approach that uses evaluation trees. The new approach is 45% slower than the C++ implementation and more than five times faster than the previous one. The OpenMP and OpenCL implementations are tested on three medium-scale models using a multi-core CPU, a discrete GPU, an integrated GPU and heterogeneous computing setups. Execution times are compared and analysed and the advantages of the OpenCL implementation running on a discrete GPU and heterogeneous systems are discussed. It is found that the evaluation of model equations using the parallel OpenCL implementation running on a discrete GPU is up to twelve times faster than the sequential version while the overall simulation speed-up gained is more than three times.

Numerical Fluid-Structure Interaction Study on the NREL 5MW HAWT

The development of reliable Fluid-Structure Interaction (FSI) simulation tools and models for the wind turbines is a critical step in the design procedure towards achieving optimized large wind turbine structures. Such approach will mitigate the aeroelastic instabilities like: torsional flutter, stall flutter and edgewise instability that introduce extra stresses to the turbine structure leading to reduced life time and substantial failures. In this study, FSI simulations were held using the commercial package Ansys v18.2 solvers as a preliminary step towards our on-going development of a reliable Open-Source solver. These simulations were applied to the full-scale rotor blades of the NREL 5MW reference horizontal axis wind turbine. The aerodynamic loads and structural responses computations were carried out using a steady-state FSI analysis. The computations were run on the Kyushu University multi-core Linux cluster using the public domain openMPI implementation of the standard message passing interface (MPI). Finally, the results were validated against the Technical University of Denmark’s (DTU) MIRAS aeroelastic code results as well as the widely used FLEX5-Q3UIC and FAST codes in different cases showing reasonable agreement.

GMMSP on GPU

Superpixel segmentation is a fundamental task in computer vision. Existing works contribute to superpixel segmentation either by improving segmentation accuracy or by reducing execution time. The former modifies existing models or develops new models to improve accuracy. The latter accelerates existing implementations or reduces algorithm complexity to improve execution rate. This work falls into the second category. Recently, a superpixel algorithm using Gaussian mixture model (GMMSP) achieves state-of-the-art performance in accuracy. After exploring this algorithm, we reached new conclusions on GMMSP that unlock potential concerning fine-grain parallelism implementation. We implement GMMSP with CUDA and make it run on GPUs. Experiments are conducted to validate the consistency between CPU and GPU implementations and to evaluate the performance of our implementation with respect to a serial and an OpenMP implementation. When we consider a full implementation with a postprocessing step executed on CPU to guarantee connectivity constraint, the proposed implementation achieves a speedup of 21× compared to the OpenMP implementation for images of size 240 × 320, using NVIDIA GTX 1080. It is also mentionable that we achieve a performance of over 1000 FPS on GTX 1080 (speedup of 77× compared to the OpenMP implementation) if the connectivity constraint is not included.

ParaCells: A GPU Architecture for Cell-Centered Models in Computational Biology.

In computational biology, the hierarchy of biological systems requires the development of flexible and powerful computational tools. Graphics processing unit (GPU) architecture has been a suitable device for parallel computing in simulating multi-cellular systems. However, in modeling complex biological systems, scientists often face two tasks, mathematical formulation and skillful programming. In particular, specific programming skills are needed for GPU programming. Therefore, the development of an easy-to-use computational architecture, which utilizes GPU for parallel computing and provides intuitive interfaces for simple implementation, is needed so that general scientists can perform GPU simulations without knowing much about the GPU architecture. Here, we introduce ParaCells, a cell-centered GPU simulation architecture for NVIDIA compute unified device architecture (CUDA). ParaCells was designed as a versatile architecture that connects the user logic (in C++) with NVIDIA CUDA runtime and is specific to the modeling of multi-cellular systems. An advantage of ParaCells is its object-oriented model declaration, which allows it to be widely applied to many biological systems through the combination of basic biological concepts. We test ParaCells with two applications. Both applications are significantly faster when compared with sequential as well as parallel OpenMP and OpenACC implementations. Moreover, the simulation programs based on ParaCells are cleaner and more readable than other versions.

High-performance multi-GPU solver for describing nonlinear acoustic waves in homogeneous thermoviscous media

A double-precision numerical solver to describe the propagation of high-intensity ultrasound fluctuations using a novel finite-amplitude compressible acoustic model working in multiple processing units (GPUs) is presented. The present solver is based on a conservative hyperbolic formulation derived from a variational analysis of the compressible Navier–Stokes equations and is implemented using an explicit high-order finite difference strategy. In this work, a WENO–Z reconstruction scheme along with a high-order finite-difference stencil are used to approximate the contributions of convective and diffusive spatial operators, respectively. The spatial operators are then associated to a low–storage Runge–Kutta scheme to integrate the system explicitly in time. The present multi-GPU implementation aims to make the best use of every single GPU and gain optimal performance of the algorithm on the per-node basis. To assess the performance of the present solver, a typical mini-server computer with 4 Tesla K80 dual GPU accelerators is used. The results show that the present formulation scales linearly for large domain problems. Moreover, when compared to an OpenMP implementation running with an i7 processor of 4.2 GHz, this is outperformed by our MPI-GPU implementation by a factor of 99. In this work, the present multi-GPU solver is illustrated with a three-dimensional simulation of a highly-intense focused ultrasound propagation.

On the adequacy of lightweight thread approaches for high-level parallel programming models

High-level parallel programming models (PMs) are becoming crucial in order to extract the computational power of current on-node multi-threaded parallelism. The most popular PMs, such as OpenMP or OmpSs, are directive-based: the complexity of the hardware is hidden by the underlying runtime system, improving coding productivity. The implementations of OpenMP usually rely on POSIX threads (pthreads), offering excellent performance for coarse-grained parallelism and a perfect match with the current hardware. OmpSs is a task oriented PM based on an ad hoc runtime solution called Nanos++; it is the precursor of the tasking parallelism in the OpenMP tasking specification. A recent trend in runtimes and applications points to leveraging massive on-node parallelism in conjunction with fine-grained and dynamic scheduling paradigms. In this paper we analyze the behavior of the OpenMP and OmpSs PMs on top of the recently emerged Generic Lightweight Threads (GLT) API. GLT exposes a common API for lightweight thread (LWT) libraries that offers the possibility of running the same application over different native LWT solutions. We describe the design details of those high-level PMs implemented on top of GLT and analyze different scenarios in order to assess where the use of LWTs may benefit application performance. Our work reveals those scenarios where LWTs overperform pthread-based solutions and compares the performance between an ad hoc solution and a generic implementation.

Towards distributed acceleration of image processing applications using reconfigurable active SSD clusters: a case study of seismic data analysis

In this work, we propose a high performance distributed system that consists of several middleware servers each connected to a number of FPGAs with extended solid state storage that we call reconfigurable active solid state device (RASSD) nodes. A full data communication solution between middleware and RASSD nodes is presented. We use seismic data analysis as a case study to quantify how and by how much RASSD nodes can accelerate computational throughput. Speedup of seismic data prediction time when both GLCM and Haralick features are accelerated is examined. The distributed system achieves 102× speedup compared to 4-thread openMP implementation and 265× speedup compared to single thread modern CPU performance. Performance is 5× better than previous work reporting speedup on GLCM and Haralick feature analysis when data is local to the FPGA and 20× better than an identical CUDA implementation using modern GPU.

GPU Framework for Change Detection in Multitemporal Hyperspectral Images

Nowadays, it is increasingly common to detect land cover changes using remote sensing multispectral images captured at different time-frames over the same area. A large part of the available change detection (CD) methods focus on pixel-based operations. The use of spectral–spatial techniques helps to improve the accuracy results but also implies a significant increase in processing time. In this paper, a Graphic Processor Unit (GPU) framework to perform object-based CD in multitemporal remote sensing hyperspectral data is presented. It is based on Change Vector Analysis with the Spectral Angle Mapper distance and Otsu’s thresholding. Spatial information is taken into account by considering watershed segmentation. The GPU implementation achieves real-time execution and speedups of up to 46.5 $$\times $$ with respect to an OpenMP implementation.

Orchestrating parallel detection of strongly connected components on GPUs

Abstract Detecting strongly connected components (SCC) is a practical graph analytics algorithm widely used in many application domains. To accelerate SCC detection, parallel algorithms have been proposed and implemented on GPUs. However, existing GPU implementations show unstable performance for various graphs, especially for real-world graphs, as these implementations do not have a clear understanding of the graph properties. In this paper, we analyze that graphs in SCC detection usually exhibit (1) skewed component sizes (the static property) and (2) dynamically changed graph structure (the dynamic property). To deal with these irregular graph properties, we propose a hybrid method that divides the algorithm into two phases and exploits different levels of parallelism for different-sized components. We also customize the graph traversal strategies for each phase to handle the dynamically changed graph structure. Our method is carefully implemented to take advantage of the GPU hardware. Evaluation with diverse synthetic and real-world graphs shows that our method substantially improves existing GPU implementations, both performance-wise and applicability-wise. It also achieves an average speedup of 5.6 × and 1.5 × over the sequential and OpenMP implementations on the CPU respectively.

A multithreaded OpenMP implementation of the LU-SGS method using the multilevel decomposition of the unstructured computational mesh

A new parallel version of the method LU-SGS (Lower-upper symmetric Gauss-Seidel) based on a multilevel decomposition of the unstructured computational mesh is proposed. The advantages of the proposed approach are demonstrated by computing the supersonic flow around the RAM-C geometry. The method is well scalable when a large number of threads are used on the processor Intel Xeon Phi.

Parallelization of large vector similarity computations in a hybrid CPU+GPU environment

The paper presents design, implementation and tuning of a hybrid parallel OpenMP+CUDA code for computation of similarity between pairs of a large number of multidimensional vectors. The problem has a wide range of applications, and consequently its optimization is of high importance, especially on currently widespread hybrid CPU+GPU systems targeted in the paper. The following are presented and tested for computation of all vector pairs: tuning of a GPU kernel with consideration of memory coalescing and using shared memory, minimization of GPU memory allocation costs, optimization of CPU–GPU communication in terms of size of data sent, overlapping CPU–GPU communication and kernel execution, concurrent kernel execution, determination of best sizes for data batches processed on CPUs and GPUs along with best GPU grid sizes. It is shown that all codes scale in hybrid environments with various relative performances of compute devices, even for a case when comparisons of various vector pairs take various amounts of time. Tests were performed on two high-performance hybrid systems with: 2 x Intel Xeon E5-2640 CPU + 2 x NVIDIA Tesla K20m and latest generation 2 x Intel Xeon CPU E5-2620 v4 + NVIDIA’s Pascal generation GTX 1070 cards. Results demonstrate expected improvements and beneficial optimizations important for users incorporating such types of computations into their parallel codes run on similar systems.

Evaluation of directive-based performance portable programming models

We present an extended exploration of the performance portability of directives provided by OpenMP 4 and OpenACC to program various types of node architectures with attached accelerators. To do this, we use examples of algorithms with varying computational intensities for our evaluation, as both compute and data access efficiency are important considerations for overall application performance. We implement the kernels of interest using various methods provided by newer OpenACC and OpenMP implementations, and we evaluate their performance on various platforms including both x86\_64 and Power8 with attached NVIDIA GPUs, X86\_64 multicores, self-hosted Intel Xeon Phi KNL, as well as an X86\_64 host system with Intel Xeon Phi coprocessors. Furthermore, we present in detail what factors affected the performance portability, including how to pick the right programming model, its programming style, its availability on different platforms, and how well compilers can optimise and target multiple platforms.

Research and implementation of a high performance parallel computing digital down converter on graphics processing unit

SummaryDigital down converter (DDC) is a time‐intensive and data‐intensive computing task and considered as the key technology in software defined radio. This paper proposes a high‐performance implementation of DDC on a graphics processing unit (GPU) using CUDA, which is composed of a numerically controlled oscillator stage, a cascaded integrator‐comb (CIC) decimation filter stage, and a finite impulse response (FIR) filter stage. The GPU implementation and optimizing of all the stages are studied in detail. Additionally, for handling a long‐duration signal, the signal data sequence is truncated into segments; the overlap‐save and overlap‐add mechanisms were applied in CIC stage and FIR stage, respectively. Finally, experiments were conducted to evaluate the performance of GPU‐based DDC with respect to a sequential version CPU implementation and an OpenMP implementation (16 threads). Experimental results demonstrate that the DDC achieves significant improvements on the GPU; the maximum speed ups in numerically controlled oscillator stage, CIC stage, and FIR stage can achieve more than 1242, 527, and 179 times, including data‐transfer, kernel execution, and other processing operations; the overall speed up of DDC can achieve more than 180. In the meantime, the speed ups of GPU implementation are far above the OpenMP implementation (about 2.5‐6.4 times).

Supporting Enhanced Exception Handling with OpenMP in Object-Oriented Languages

The proliferation of parallel processing in shared-memory applications has encouraged developing assistant frameworks such as OpenMP. OpenMP has become increasingly prevalent due to the simplicity it offers to elegantly and incrementally introduce parallelism. However, it still lacks some high-level language features that are essential in object-oriented programming. One such mechanism is that of exception handling. In languages such as Java, the concept of exception handling has been an integral aspect to the language since the first release. For OpenMP to be truly embraced within this object-oriented community, essential object-oriented concepts such as exception handling need to be given some attention. The official OpenMP standard has little specification on error recovery, as the challenges of supporting exception-based error recovery in OpenMP extends to both the semantic specifications and related runtime support. This paper proposes a systematic mechanism for exception handling with the co-use of OpenMP directives, which is based on a Java implementation of OpenMP. The concept of exception handling with OpenMP directives has been formalized and categorized. Hand in hand with this exception handling proposal, a flexible approach to thread cancellation is also proposed (as an extension on OpenMP directives) that supports this exception handling within parallel execution. The runtime support and its implementation are discussed. The evaluation shows that while there is no prominent overhead introduced, the new approach provides a more elegant coding style which increases the parallel development efficiency and software robustness.

Solving the maximum subsequence sum and related problems using BSP/CGM model and multi-GPU CUDA

The maximum subsequence problem finds a contiguous subsequence of the largest sum of a sequence of n numbers. Solutions to this problem are used in various branches of science, especially in applications of computational biology. The best sequential solution to the problem has an O(n) running time and uses dynamic programming. Although effective, this solution returns little information and disregards the existence of more than a maximum subsequence sum. Particularly in DNA analysis, if we find all maximum subsequence sums, we will also find all the possible pathogenicity islands, which are stretches with high possibility of causing some diseases. We present new Bulk Synchronous Parallel/Coarse-Grained Multicomputer (BSP/CGM) parallel algorithms, which consider the existence of more than one subsequence of maximum sum, and are able to find solutions to three problems: the longest maximum subsequence sum, the shortest maximum subsequence sum, and the number of disjoint subsequences of maximum sum. To the best of our knowledge, there are no parallel BSP/CGM algorithms for the related problems. Taking advantage of the advent of general purpose graphics processing unit (GPGPU), we implemented our algorithms on multi-GPU with Compute Unified Device Architecture (CUDA) and, for comparison purposes, MPI and OpenMP implementations have also been developed. The algorithms presented good speedups, as confirmed by experimental results. They use p processors and require O(n/p) parallel time with a constant number of communication rounds for the algorithm of the maximum subsequence sum and O(logp) communication rounds, with O(n/p) local computation per round, for the algorithms of the related problems. We concluded that our algorithms for the maximum subsequence sum and related problems are unique and effective. We also believe that the BSP/CGM model can guide parallel implementations in modern architectures such as GPGPU/CUDA. As future work, we intend to extend these results to arrays with higher dimensions and compute all maximal subsequences in a given interval.

Parallel implementation of approximate atomistic models of the AMOEBA polarizable model

In this work we present a replicated data hybrid OpenMP/MPI implementation of a hierarchical progression of approximate classical polarizable models that yields speedups of up to ∼10 compared to the standard OpenMP implementation of the exact parent AMOEBA polarizable model. In addition, our parallel implementation exhibits reasonable weak and strong scaling. The resulting parallel software will prove useful for those who are interested in how molecular properties converge in the condensed phase with respect to the MBE, it provides a fruitful test bed for exploring different electrostatic embedding schemes, and offers an interesting possibility for future exascale computing paradigms.

AQsort: Scalable Multi-Array In-Place Sorting with OpenMP

A new multi-threaded variant of the quicksort algorithm called AQsort and its C++/OpenMP implementation are presented. AQsort operates in place and was primarily designed for high-performance computing (HPC) runtime environments. It can work with multiple arrays at once; such a functionality is frequently required in HPC and cannot be accomplished with standard C pointer-based or C++ iterator-based approach. An extensive study is provided that evaluates AQsort experimentally and compares its performance with modern multi-threaded implementations of in-place and out-of-place sorting algorithms based on OpenMP, Cilk Plus, and Intel TBB. The measurements were conducted on several leading-edge HPC architectures, namely Cray XE6 nodes with AMD Bulldozer CPUs, Cray XC40 nodes with Intel Hasswell CPUs, IBM BlueGene/Q nodes, and Intel Xeon Phi coprocessors. The obtained results show that AQsort provides good scalability and sorting performance generally comparable to its competitors. In particular cases, the performance of AQsort may be slightly lower, which is the price for its universality and ability to work with substantially larger amounts of data.

FEREBUS: Highly parallelized engine for kriging training.

FFLUX is a novel force field based on quantum topological atoms, combining multipolar electrostatics with IQA intraatomic and interatomic energy terms. The program FEREBUS calculates the hyperparameters of models produced by the machine learning method kriging. Calculation of kriging hyperparameters (θ and p) requires the optimization of the concentrated log-likelihood L̂(θ,p). FEREBUS uses Particle Swarm Optimization (PSO) and Differential Evolution (DE) algorithms to find the maximum of L̂(θ,p). PSO and DE are two heuristic algorithms that each use a set of particles or vectors to explore the space in which L̂(θ,p) is defined, searching for the maximum. The log-likelihood is a computationally expensive function, which needs to be calculated several times during each optimization iteration. The cost scales quickly with the problem dimension and speed becomes critical in model generation. We present the strategy used to parallelize FEREBUS, and the optimization of L̂(θ,p) through PSO and DE. The code is parallelized in two ways. MPI parallelization distributes the particles or vectors among the different processes, whereas the OpenMP implementation takes care of the calculation of L̂(θ,p), which involves the calculation and inversion of a particular matrix, whose size increases quickly with the dimension of the problem. The run time shows a speed-up of 61 times going from single core to 90 cores with a saving, in one case, of ∼98% of the single core time. In fact, the parallelization scheme presented reduces computational time from 2871 s for a single core calculation, to 41 s for 90 cores calculation. © 2016 The Authors. Journal of Computational Chemistry Published by Wiley Periodicals, Inc.