OpenMP Programs Research Articles

An automated OpenMP mutation testing framework for performance optimization

Performance optimization continues to be a challenge in modern HPC software. Existing performance optimization techniques, including profiling-based and auto-tuning techniques, fail to indicate program modifications at the source level thus preventing their portability across compilers. This paper describes Muppet, a new approach that identifies program modifications called mutations aimed at improving program performance. Muppet’s mutations help developers reason about performance defects and missed opportunities to improve performance at the source code level. In contrast to compiler techniques that optimize code at intermediate representations (IR), Muppet uses the idea of source-level mutation testing to relax correctness constraints and automatically discover optimization opportunities that otherwise are not feasible using the IR. We demonstrate the Muppet’s concept in the OpenMP programming model. Muppet generates a list of OpenMP mutations that alter the program parallelism in various ways, and is capable of running a variety of optimization algorithms such as delta debugging, Bayesian Optimization and decision tree optimization to find a subset of mutations which, when applied to the original program, cause the most speedup while maintaining program correctness. When Muppet is evaluated against a diverse set of benchmark programs and proxy applications, it is capable of finding sets of mutations that induce speedup in 75.9% of the evaluated programs.

Multi-criteria Analysis and Optimization in the AMPERE Ecosystem

The AMPERE project is developing the next generation of high-performance and energy efficient Cyber-Physical Systems supporting multi-criteria optimization. This paper provides the general overview of the AMPERE approach to analyse and optimize parallel real-time applications in heterogeneous platforms. More specifically, it details how parallel OpenMP programs are generated from AMALTHEA models, and the multi-criteria optimization methodology, considering the fault-tolerance, time and energy properties of the targeted applications.

A parallel programming application of the A* algorithm in digital rock physics

Pore-scale analysis and characterization of reservoir rocks provide valuable information for the definition and management of underground hydrogen storage and CO2 storage or sequestration.This article presents an optimized implementation of the A* algorithm, the most popular pathfinding method in the presence of obstacles. In this context, the algorithm is applied to recognize the minimum length connected paths through each flow direction of 3D images of a porous medium representative of a reservoir rock. The identification of the main paths allows the characterization of the pore space and the calculation of fundamental petrophysical properties such as tortuosity and effective porosity, which can be used for permeability estimation. Compared to other algorithms available for pore-scale characterization, such as the pore centroid, A* provides a better approximation of the pore space available for the flow and, therefore, a reliable characterization of the petrophysical properties. On the other hand, path identification is significantly consuming in terms of time and memory. In this paper, an efficient and optimized implementation based on C++/OpenMP programming language is presented.The proposed implementation aims to the analysis of large-scale models profiting from parallelization, memory optimization, and enhanced managing of dead paths. Three test cases of increasing sizes are presented, to analyze the advantages and the disadvantages of the algorithm as the number of explored points increases. The 3D binary images analyzed are related to a synthetic domain (1 million voxels) and two actual sandstone samples (about 4 and 64 million voxels respectively). The code is validated against a Matlab serial implementation, showing a significant improvement in efficiency. Remarkable test cases of several millions of voxels were afforded, overcoming the memory and execution slowness issues. Moreover, the proposed implementation is suitable for large pore-scale models run in HPC environments.

Compute units in OpenMP: Extensions for heterogeneous parallel programming

SummaryThis article evaluates the current support for heterogeneous OpenMP 5.2 applications regarding the simultaneous activation of host and device computing units (e.g., CPUs, GPUs, or FPGAs). The article identifies limitations in the current OpenMP specification and describes the design and implementation of novel OpenMP extensions and runtime support for heterogeneous parallel programming. The Compute Unit (CUs) abstraction is introduced in the OpenMP programming model. The Compute Unit abstraction is defined in terms of an aggregation of computing elements (e.g., CPUs, GPUs, FPGAs). On top of CUs, the article describes dynamic work sharing constructs and schedulers that address the inherent differences in compute power of host and device CUs. New constructs and the corresponding runtime support are described for the new abstractions. The article evaluates the case of a hybrid multilevel parallelization of the NPB‐MZ benchmark suite. The implementation exploits both coarse‐grain and fine‐grain parallelism, mapped to CUs of different nature (GPUs and CPUs). All CUs are activated using the new extensions and runtime support. We compare hybrid and nonhybrid executions under two state‐of‐the‐art work‐distribution schemes (Static and Dynamic Task schedulers). On a computing node composed of one AMD EPYC 7742 @ 2.250GHz (64 cores and 2 threads/core, totalling 128 threads per node) and 2 GPU AMD Radeon Instinct MI50 with 32GB, hybrid executions present speedups from 1.08 up to 3.18 with respect to a nonhybrid GPU implementation, depending on the number of activated CUs.

An extension to Voro++ for multithreaded computation of Voronoi cells

Voro++ is a software library written in C++ for computing the Voronoi tessellation, a technique in computational geometry that is widely used for analyzing systems of particles. Voro++ was released in 2009 and is based on computing the Voronoi cell for each particle individually. Here, we take advantage of modern computer hardware, and extend the original serial version to allow for multithreaded computation of Voronoi cells via the OpenMP application programming interface. We test the performance of the code, and demonstrate that it can achieve parallel efficiencies greater than 95% in many cases. The multithreaded extension follows standard OpenMP programming paradigms, allowing it to be incorporated into other programs. We provide an example of this using the VoroTop software library, performing a multithreaded Voronoi cell topology analysis of up to 102.4 million particles. Program summaryProgram title:Voro++CPC Library link to program files:https://doi.org/10.17632/tddc4w4zkk.1Developer's repository link:https://github.com/chr1shr/voroLicensing provisions: BSD 3-clause (with LBNL modification)Programming language: C++External routines/libraries: OpenMPNature of problem: Multithreaded computation of the Voronoi tessellation in two and three dimensionsSolution method: The Voro++ library is built around several C++ classes that can be incorporated into other programs. The two largest components are the container... classes that spatially sort input particles into a grid-based data structure, allowing for efficient searches of nearby particles, and the voronoicell... classes that represent a single Voronoi cell as an arbitrary convex polygon or polyhedron. The Voronoi cell for each particle is built by considering a sequence of plane cuts based on neighboring particles, after which many different statistics (e.g. volume, centroid, number of vertices) can be computed. Since each Voronoi cell is calculated individually, the Voronoi cells can be computed using multithreading via OpenMP.

Parallelization of numerical solutions of shallow water equations by the finite volume method for implementation on multiprocessor systems and graphics processors

An overview of approaches to parallelization of grid-based numerical methods for solving shallow water equations for multiprocessor systems and graphics processors is presented. A multithreaded approach for shared-memory computing systems implemented on the basis of the OpenMP programming interface and a geometric decomposition approach with message-passing using the MPI library for distributed-memory computers are described. Multithreading for programming GPUs based on the OpenACC software interface is considered. For the COASTOX-UN system of two-dimensional modeling of hydrodynamics, sediment and radionuclide transport in river systems and coastal areas of the seas, the parallelization of its hydrodynamic model COASTOX-HD was carried out. In the developed numerical model, the shallow water equations are solved by finite-volume numerical methods on unstructured computational grids with triangular cells of variable size. The parallelization is implemented using a hybrid MPI+OpenACC approach targeting multiprocessor systems and GPUs. For multiprocessor computers, geometric decomposition and MPI-based messaging are used, and for GPUs, multithreading is implemented using OpenACC directives. The performance of the developed parallel hydrodynamic model was evaluated during the calculation of typical problems of hydrodynamics of shallow water bodies, river flood, and tsunami wave run-up on the coast on a Dell Precision Workstation 7920 multi-core workstation with two 20-core Intel Xeon Gold 6230 processors and NVIDIA Quadro RTX 5000 and NVIDIA GeForce RTX 3080 graphics cards. It is shown that the developed model has significantly accelerated the simulation on the considered multiprocessor system and the considered GPUs. The acceleration on GPUs depends on the size of the computational grid, increasing to saturation with an increase in the number of grid cells. It is established that for the developed parallel model, whose numerical schemes are related to algorithms with low computational intensity, the memory bandwidth of the NVIDIA architecture GPUs is a more important limiting factor of acceleration than their performance.

A parallel numerical algorithm by combining MPI and OpenMP programming models with applications in gravity field recovery

Satellite gravimetry missions have enabled the calculation of high-accuracy and high-resolution Earth gravity field models from satellite-to-satellite tracking data and gravitational gradients. However, calculating high maximum degree/order (e.g., 240 or even higher) gravity field models using the least squares method is time-consuming due to the vast amount of gravimetry observations. To improve calculation efficiency, a parallel algorithm has been developed by combining Message Passing Interface (MPI) and Open Multi-Processing (OpenMP) programming models to calculate and invert normal equations for the Earth gravity field recovery. The symmetrical feature of normal equations has been implemented to speed up the calculation progress and reduce computation time. For example, the computation time to generate the normal equation of an IGGT_R1 test version of degree/order 240 was reduced from 88 h to 27 h by considering the symmetrical feature. Here, the calculation was based on the high-performance computing cluster with 108 cores in the School of Geodesy and Geomatics, at Wuhan University. Additionally, the MPI parallel Gaussian-Jordan elimination method was modified to invert normal equation matrices and scaled up to 100 processor cores in this study while the traditional method was limited in a certain number of processors. Furthermore, the Cholesky decomposition from the ScaLAPACK library was used to compare with the parallel Gauss-Jordan elimination method. The numerical algorithm has effectively reduced the amount of calculation and sped up the calculation progress, and has been successfully implemented in applications such as building the gravity field models IGGT_R1 and IGGT_R1C.

Optimizing Iterative Data-flow Scientific Applications using Directed Cyclic Graphs

Data-flow programming models have become a popular choice for writing parallel applications as an alternative to traditional work-sharing parallelism. They are better suited to write applications with irregular parallelism that can present load imbalance. However, these programming models suffer from overheads related to task creation, scheduling and dependency management, limiting performance and scalability when tasks become too small. At the same time, many HPC applications implement iterative methods or multi-step simulations that create the same directed acyclic graphs of tasks on each iteration. By giving application programmers a way to express that a specific loop is creating the same task pattern on each iteration, we can create a single task DAG once and transform it into a cyclic graph. This cyclic graph is then reused for successive iterations, minimizing task creation and dependency management overhead. This paper presents the <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">taskiter</i> , a new construct we propose for the OmpSs-2 and OpenMP programming models, allowing the use of directed cyclic task graphs (DCTG) to minimize runtime overheads. Moreover, we present a simple <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">immediate successor</i> locality-aware heuristic that minimizes task scheduling overhead by bypassing the runtime task scheduler. We evaluate the implementation of the <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">taskiter</i> and the <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">immediate successor</i> heuristic in 8 iterative benchmarks. Using small task granularities, we obtain a geometric mean speedup of 2.56x over the reference OmpSs-2 implementation, and a 3.77x and 5.2x speedup over the LLVM and GCC OpenMP runtimes, respectively.

Critical Section Overhead Reduction for OpenMP Program by Nesting a Serial Loop to Increase Task Granularity of Parallel Loop

This paper presents a simple method to reduce performance loss due to a parallel program's massive critical sections of parallel numerical integration. The method is to transform a fine grain parallel loop into a coarse grain parallel loop which is nesting a sequential loop. The coarse grain parallel loop is by nesting a loop block to make task granularities coarser than that naive one. In addition to the overhead reduction, the method makes the parallel work fraction significantly larger than the serial fraction. As a result, nesting a serial loop within a parallel loop improves the parallel program’s performance. Compared to the naïve method, which does not scale performance of parallel program of numerical integration, the nesting serial loop method scales a parallel program up to 3.26 times fold relative to its sequential program on quad-core processor. This result shows that the proposed method makes parallel program much faster compared to the naïve method.&#x0D;

OmpTG: From OpenMP Programs to Task Graphs

Real-time systems are shifting them from single-core to multi-core processors. Software must be parallelized to fully utilize the computation power of multi-core architectures. OpenMP is a promising framework to develop parallel real-time software on multi-cores. OpenMP programs keep certain similarity to real-time task graph models, and this motivates much recent work done on real-time scheduling of OpenMP tasks. However, these studies conduct evaluations with randomly generated task graphs, which cannot well capture the structure features of realistic OpenMP programs. To fill the gap between theoretical real-time scheduling research and the OpenMP software reality, we develop an ompTG tool for transforming OpenMP programs into parallel task graphs. ompTG prepares a way to exhibit OpenMP such that the researchers in real-time community can easily understand: An OpenMP system consists of a set of tasks. There are interdependencies among tasks, and each task has an intra structure of the control-flow graph. Besides the topology of OpenMP tasks, we also provide a safe WCET for each vertex of OpenMP task graphs by using static WCET analysis techniques. Moreover, we derive the flow facts, e.g, infeasible path and loop bounds for the task graph, which is necessary information for real-time scheduling and analysis. As a case study, we collect 12 OpenMP programs from the BOTS benchmark, and transform them into task graphs, demonstrating the usage of ompTG.

Study on solid block landslide generated tsunami using a modified δ-les-SPH model

This article established a weakly compressible smoothed particle hydrodynamics (WCSPH) model to simulate the single- and multi-block landslide-tsunamis. This WCSPH model was developed based on the δ-LES-SPH method according to the characteristics of landslide-tsunamis. In this model, a modified coupled dynamic boundary method was proposed to mimic (in a uniform manner) the solid wall boundary of the fluid and the dynamic friction boundary of the slide, which expands the range of application of the δ-LES-SPH method. A particle shifting technique (PST) was introduced to improve the accuracy and robustness of the δ-LES-SPH method. The Fortran 90 language was used to implement the current SPH code, and OpenMP programming was adopted to improve the computing efficiency. Afterwards, the current procedure was utilized to model subaerial and submarine single-block rigid landslides and submarine multi-block landslides. Firstly, the SPH results of the slide movements, wave profiles and energy were compared with the experimental data and previous numerical results. Then, the energy equation is used to study the energy components of different types of landslide-tsunamis. Energy analysis reveals that the more complex the movement of the slide, the lower the efficiency of energy transfer. The consistency between the numerical results and the experimental data confirms the effectiveness and stability of the proposed method. The current model provides a useful tool to study the problem of tsunamis generated by rigid landslides.

Accelerated GPU simulation of the gaseous detonation cell structure

The aim of the present paper is to report on our recent results for GPU accelerated simulations of the gaseous detonation structure. Reactive Euler equations with a one-step Arrhenius chemistry model have been used for numerical simulation. And the NND space discretization scheme combined with Steger-Warming split method has been used. For time discretization we have applied the explicit third order Runge-Kutta method. We have obtained a speedup of 8 times ( in comparison to 30 threads openmp program) for the gaseous detonation simulation on a structure grid of 320 million points.

Load balancing method for heterogeneous CFD algorithms

Abstract. The problem of load balancing for unstructured heterogeneous numerical algorithms for simulation of physical processes is considered. A computational distribution method for hybrid supercomputers with multicore CPUs and massively parallel accelerators is described. The load balancing procedure includes determination of dual graph vertices and edges weights, devices’ performance test and two-level decomposition of the computational mesh based on domain decomposition method. First level decomposition involves the graph partitioning between supercomputer nodes. On the second level node subdomains are partitioned between the MPI- processes running on the nodes. The details of the proposed approach are considered on the example of an unstructured finite-volume algorithm for modeling the Navier-Stokes equations with polynomial reconstruction of variables and explicit time integration scheme. The parallel version of the algorithm is developed using the MPI, OpenMP and CUDA programming models. The parameters of performance, parallel efficiency and scalability of the heterogeneous program are given. The results mentioned are obtained during the simulation of a supersonic flow around a sphere on a mixed mesh consisting of tetrahedrons, triangular prisms, quadrangular pyramids and hexagons.

PPT-Multicore: performance prediction of OpenMP applications using reuse profiles and analytical modeling

We present PPT-Multicore, an analytical model embedded in the Performance Prediction Toolkit (PPT) to predict parallel application performance running on a multicore processor. PPT-Multicore builds upon our previous work towards a multicore cache model. We extract LLVM basic block labeled memory trace using an architecture-independent LLVM-based instrumentation tool only once in an application's lifetime. The model uses the memory trace and other parameters from an instrumented sequentially executed binary. We use a probabilistic and computationally efficient reuse profile to predict the cache hit rates and runtimes of OpenMP programs' parallel sections. We model Intel's Broadwell, Haswell, and AMD's Zen2 architectures and validate our framework using different applications from PolyBench and PARSEC benchmark suites. The results show that PPT-Multicore can predict cache hit rates with an overall average error rate of 1.23% while predicting the runtime with an error rate of 9.08%.

Numerical simulation of landslide generated impulse waves using a δ+-LES-SPH model

In this article, a δ+-LES-SPH model is presented to study the generation and propagation of impulse waves. In this model, a density diffusion term is added to the continuity equation to eliminate the numerical high-frequency oscillation of the pressure field. A particle shifting technique is introduced and modified in the motion equation to improve the accuracy and robustness of the smoothed particle hydrodynamics (SPH) model. The physical viscosity term and the turbulence viscosity term based on the large eddy simulation (LES) are used in the momentum equation to calculate the viscous force. To predict the rigid slide motion more effectively, we propose a rigid slide motion model that considers friction in the δ+-LES-SPH frame. The δ+-LES-SPH code was implemented in Fortran 90, and OpenMP programming technology was used to improve the calculation efficiency. Subsequently, this parallel SPH code was used to simulate subaerial and submarine rigid landslides in 2-D and more realistic 3-D applications. The SPH results were compared with the experimental data and other numerical results. The results show that the proposed model has a good convergence and less numerical dissipation. Satisfactory results were obtained for the rigid slide motion, wave generation, and long-distance propagation.

Design and Implementation of Multi-core Parallel Compiler Based on OpenMP

OpenMP supports parallel incremental development, and has become a mainstream parallel programming standard for shared memory systems. A parallel compiler running on multi-core DSP is designed for OpenMP programs. The main achievement includes translator and runtime. The translator converts OpenMP instructions in source files into function calls in runtime where the specific implementation is provided. The core problem of parallel compiler is how to design parallel strategy to allocate computing tasks to each core, which corresponds to the concept of parallel domain in OpenMP standard. The compiler realizes the of master-slave core by transforming parallel instructions and supporting runtime, which has guiding significance for the design of parallel compiler.

Correct program parallelisations

A commonly used approach to develop deterministic parallel programs is to augment a sequential program with compiler directives that indicate which program blocks may potentially be executed in parallel. This paper develops a verification technique to reason about such compiler directives, in particular to show that they do not change the behaviour of the program. Moreover, the verification technique is tool-supported and can be combined with proving functional correctness of the program. To develop our verification technique, we propose a simple intermediate representation (syntax and semantics) that captures the main forms of deterministic parallel programs. This language distinguishes three kinds of basic blocks: parallel, vectorised and sequential blocks, which can be composed using three different composition operators: sequential, parallel and fusion composition. We show how a widely used subset of OpenMP can be encoded into this intermediate representation. Our verification technique builds on the notion of iteration contract to specify the behaviour of basic blocks; we show that if iteration contracts are manually specified for single blocks, then that is sufficient to automatically reason about data race freedom of the composed program. Moreover, we also show that it is sufficient to establish functional correctness on a linearised version of the original program to conclude functional correctness of the parallel program. Finally, we exemplify our approach on an example OpenMP program, and we discuss how tool support is provided.

Performance analysis of CUDA, OpenACC and OpenMP programming models on TESLA V100 GPU

Graphics processors are widely utilized in modern supercomputers as accelerators. Ability to perform efficient parallelization and low-level allow scientists to greatly boost performance of their codes. Modern Nvidia GPUs feature low-level approaches, such as CUDA, along with high-level approaches: OpenACC and OpenMP. While the low-level approach aims to explore all possible abilities of SIMT GPU architecture by writing low-level C/C++ code, it takes significant effort from programmer. OpenACC and OpenMP programming models are opposite to CUDA. Using these models the programmer only have to identify the blocks of code to be parallelized using pragmas. We compare the performance of CUDA, OpenMP and OpenACC on state-of-the-art Nvidia Tesla V100 GPU in various typical scenarios that arise in scientific programming, such as matrix multiplication, regular memory access patterns and evaluate performance of physical simulation codes implemented using these programming models. Moreover, we study the performance matrix multiplication implemented in vendor-optimized BLAS libraries for Nvidia Tesla V100 GPU and modern Intel Xeon processor.

BSF-skeleton: A template for parallelization of iterative numerical algorithms on cluster computing systems

This article describes a method for creating applications for cluster computing systems using the parallel BSF-skeleton based on the original BSF (Bulk Synchronous Farm) model of parallel computations developed by the author earlier. This model uses the master/slave paradigm. The main advantage of the BSF model is that it allows to estimate the scalability of a parallel algorithm before its implementation. Another important feature of the BSF model is the representation of problem data in the form of lists that greatly simplifies the logic of building applications. The BSF-skeleton is designed for creating parallel programs in C++ using the MPI library. The scope of the BSF-skeleton is iterative numerical algorithms of high computational complexity. The BSF-skeleton has the following distinctive features.• The BSF-skeleton completely encapsulates all aspects that are associated with parallelizing a program.• The BSF-skeleton allows error-free compilation at all stages of application development.• The BSF-skeleton supports OpenMP programming model and workflows.

LLOV

In the era of Exascale computing, writing efficient parallel programs is indispensable, and, at the same time, writing sound parallel programs is very difficult. Specifying parallelism with frameworks such as OpenMP is relatively easy, but data races in these programs are an important source of bugs. In this article, we propose LLOV, a fast, lightweight, language agnostic, and static data race checker for OpenMP programs based on the LLVM compiler framework. We compare LLOV with other state-of-the-art data race checkers on a variety of well-established benchmarks. We show that the precision, accuracy, and the F1 score of LLOV is comparable to other checkers while being orders of magnitude faster. To the best of our knowledge, LLOV is the only tool among the state-of-the-art data race checkers that can verify a C/C++ or FORTRAN program to be data race free.