OpenMP Extensions Research Articles

Compute units in OpenMP: Extensions for heterogeneous parallel programming

SummaryThis article evaluates the current support for heterogeneous OpenMP 5.2 applications regarding the simultaneous activation of host and device computing units (e.g., CPUs, GPUs, or FPGAs). The article identifies limitations in the current OpenMP specification and describes the design and implementation of novel OpenMP extensions and runtime support for heterogeneous parallel programming. The Compute Unit (CUs) abstraction is introduced in the OpenMP programming model. The Compute Unit abstraction is defined in terms of an aggregation of computing elements (e.g., CPUs, GPUs, FPGAs). On top of CUs, the article describes dynamic work sharing constructs and schedulers that address the inherent differences in compute power of host and device CUs. New constructs and the corresponding runtime support are described for the new abstractions. The article evaluates the case of a hybrid multilevel parallelization of the NPB‐MZ benchmark suite. The implementation exploits both coarse‐grain and fine‐grain parallelism, mapped to CUs of different nature (GPUs and CPUs). All CUs are activated using the new extensions and runtime support. We compare hybrid and nonhybrid executions under two state‐of‐the‐art work‐distribution schemes (Static and Dynamic Task schedulers). On a computing node composed of one AMD EPYC 7742 @ 2.250GHz (64 cores and 2 threads/core, totalling 128 threads per node) and 2 GPU AMD Radeon Instinct MI50 with 32GB, hybrid executions present speedups from 1.08 up to 3.18 with respect to a nonhybrid GPU implementation, depending on the number of activated CUs.

Source-to-Source Automatic Differentiation of OpenMP Parallel Loops

This article presents our work toward correct and efficient automatic differentiation of OpenMP parallel worksharing loops in forward and reverse mode. Automatic differentiation is a method to obtain gradients of numerical programs, which are crucial in optimization, uncertainty quantification, and machine learning. The computational cost to compute gradients is a common bottleneck in practice. For applications that are parallelized for multicore CPUs or GPUs using OpenMP, one also wishes to compute the gradients in parallel. We propose a framework to reason about the correctness of the generated derivative code, from which we justify our OpenMP extension to the differentiation model. We implement this model in the automatic differentiation tool Tapenade and present test cases that are differentiated following our extended differentiation procedure. Performance of the generated derivative programs in forward and reverse mode is better than sequential, although our reverse mode often scales worse than the input programs.

SLASs: A fully automatic auto-tuned linear algebra library based on OpenMP extensions implemented in OmpSs (LASs Library)

In this work we have implemented a novel Linear Algebra Library on top of the task-based runtime OmpSs-2. We have used some of the most advanced OmpSs-2 features; weak dependencies and regions, together with the final clause for the implementation of auto-tunable code for the BLAS-3 trsm routine and the LAPACK routines npgetrf and npgesv. All these implementations are part of the first prototype of sLASs library, a novel library for auto-tunable codes for linear algebra operations based on LASs library. In all these cases, the use of the OmpSs-2 features presents an improvement in terms of execution time against other reference libraries such as, the original LASs library, PLASMA, ATLAS and Intel MKL. These codes are able to reduce the execution time in about 18% on big matrices, by increasing the IPC on gemm and reducing the time of task instantiation. For a few medium matrices, benefits are also seen. For small matrices and a subset of medium matrices, specific optimizations that allow to increase the degree of parallelism in both, gemm and trsm tasks, are applied. This strategy achieves an increment in performance of up to 40%.

Using CPU-GPU cloud clusters as an OpenMP accelerator

Rendering an image from a 3D scene requires a large amount of computation and designers need to use high-performance computer systems to keep the rendering time acceptable. They usually rent computing power directly from cloud service providers (e.g. AWS and Azure). However, integrating them within the standard workflow of modeling softwares can become a complex task. This project uses an extension of OpenMP 4.X 1-2 to eliminate any major interactions with the end-user, minimizing the complexity of cloud integration and optimizing the workflow. It applies such approach to a ray-tracing application, a simplified version of the engines used by professional 3D modeling software (e.g. Blender 3). It automatically offloads the rendering process from the user computer to computer cluster within the Microsoft Azure cloud, brings the resulting images back after the computation ends and displays them directly on the screen of the user computer, thus providing a transparent programming model and good speed-ups over local execution.

Tinker 8: Software Tools for Molecular Design.

The Tinker software, currently released as version 8, is a modular molecular mechanics and dynamics package written primarily in a standard, easily portable dialect of Fortran 95 with OpenMP extensions. It supports a wide variety of force fields, including polarizable models such as the Atomic Multipole Optimized Energetics for Biomolecular Applications (AMOEBA) force field. The package runs on Linux, macOS, and Windows systems. In addition to canonical Tinker, there are branches, Tinker-HP and Tinker-OpenMM, designed for use on message passing interface (MPI) parallel distributed memory supercomputers and state-of-the-art graphical processing units (GPUs), respectively. The Tinker suite also includes a tightly integrated Java-based graphical user interface called Force Field Explorer (FFE), which provides molecular visualization capabilities as well as the ability to launch and control Tinker calculations.

Numerical modeling of exciton–polariton Bose–Einstein condensate in a microcavity

A novel, optimized numerical method of modeling of an exciton–polariton superfluid in a semiconductor microcavity was proposed. Exciton–polaritons are spin-carrying quasiparticles formed from photons strongly coupled to excitons. They possess unique properties, interesting from the point of view of fundamental research as well as numerous potential applications. However, their numerical modeling is challenging due to the structure of nonlinear differential equations describing their evolution. In this paper, we propose to solve the equations with a modified Runge–Kutta method of 4th order, further optimized for efficient computations. The algorithms were implemented in form of C++ programs fitted for parallel environments and utilizing vector instructions. The programs form the EPCGP suite which has been used for theoretical investigation of exciton–polaritons. Program summaryProgram title: EPCGPCatalogue identifier: AFBQ_v1_0Program summary URL:http://cpc.cs.qub.ac.uk/summaries/AFBQ_v1_0.htmlProgram obtainable from: CPC Program Library, Queen’s University, Belfast, N. IrelandLicensing provisions: BSD-3No. of lines in distributed program, including test data, etc.: 2157No. of bytes in distributed program, including test data, etc.: 498994Distribution format: tar.gzProgramming language: C++ with OpenMP extensions (main numerical program), Python (helper scripts).Computer: Modern PC (tested on AMD and Intel processors), HP BL2x220.Operating system: Unix/Linux and Windows.Has the code been vectorized or parallelized?: Yes (OpenMP)RAM: 200 MB for single runClassification: 7, 7.7.Nature of problem: An exciton–polariton superfluid is a novel, interesting physical system allowing investigation of high temperature Bose–Einstein condensation of exciton–polaritons-quasiparticles carrying spin. They have brought a lot of attention due to their unique properties and potential applications in polariton-based optoelectronic integrated circuits. This is an out-of-equilibrium quantum system confined within a semiconductor microcavity. It is described by a set of nonlinear differential equations similar in spirit to the Gross–Pitaevskii (GP) equation, but their unique properties do not allow standard GP solving frameworks to be utilized. Finding an accurate and efficient numerical algorithm as well as development of optimized numerical software is necessary for effective theoretical investigation of exciton–polaritons.Solution method: A Runge–Kutta method of 4th order was employed to solve the set of differential equations describing exciton–polariton superfluids. The method was fitted for the exciton–polariton equations and further optimized. The C++ programs utilize OpenMP extensions and vector operations in order to fully utilize the computer hardware.Running time: 6h for 100 ps evolution, depending on the values of parameters

An OpenMP Extension that Supports Thread-Level Speculation

OpenMP directives are the de-facto standard for shared-memory parallel programming. However, OpenMP does not guarantee the correctness of the parallel execution of a given loop if runtime data dependences arise. Consequently, many highly-parallel regions cannot be safely parallelized with OpenMP due to the possibility of a dependence violation. In this paper, we propose to augment OpenMP capabilities, by adding thread-level speculation (TLS) support. Our contribution is threefold. First, we have defined a new speculative clause for variables inside parallel loops. This clause ensures that all accesses to these variables will be carried out according to sequential semantics. Second, we have created a new, software-based TLS runtime library to ensure correctness in the parallel execution of OpenMP loops that include speculative variables. Third, we have developed a new GCC plugin, which seamlessly translates our OpenMP speculative clause into calls to our TLS runtime engine. The result is the ATLaS C Compiler framework, which takes advantage of TLS techniques to expand OpenMP functionalities, and guarantees the sequential semantics of any parallelized loop.

Fast reconstruction of 3D volumes from 2D CT projection data with GPUs.

BackgroundBiomedical image reconstruction applications require producing high fidelity images in or close to real-time. We have implemented reconstruction of three dimensional conebeam computed tomography(CBCT) with two dimensional projections. The algorithm takes slices of the target, weights and filters them to backproject the data, then creates the final 3D volume. We have implemented the algorithm using several hardware and software approaches and taken advantage of different types of parallelism in modern processors. The two hardware platforms used are a Central Processing Unit (CPU) and a heterogeneous system with a combination of CPU and GPU. On the CPU we implement serial MATLAB, parallel MATLAB, C and parallel C with OpenMP extensions. These codes are compared against the heterogeneous versions written in CUDA-C and OpenCL.FindingsOur results show that GPUs are particularly well suited to accelerating CBCT. Relative performance was evaluated on a mathematical phantom as well as on mouse data. Speedups of up to 200x are observed by using an AMD GPU compared to a parallel version in C with OpenMP constructs.ConclusionsIn this paper, we have implemented the Feldkamp-Davis-Kress algorithm, compatible with Fessler’s image reconstruction toolbox and tested it on different hardware platforms including CPU and a combination of CPU and GPU. Both NVIDIA and AMD GPUs have been used for performance evaluation. GPUs provide significant speedup over the parallel CPU version.

OpenMDSP: Extending OpenMP to Program Multi-Core DSPs

Multi-core digital signal processors (DSPs) are widely used in wireless telecommunication, core network transcoding, industrial control, and audio/video processing technologies, among others. In comparison with general-purpose multi-processors, multi-core DSPs normally have a more complex memory hierarchy, such as on-chip core-local memory and non-cache-coherent shared memory. As a result, efficient multi-core DSP applications are very difficult to write. The current approach used to program multi-core DSPs is based on proprietary vendor software development kits (SDKs), which only provide low-level, non-portable primitives. While it is acceptable to write coarse-grained task-level parallel code with these SDKs, writing fine-grained data parallel code with SDKs is a very tedious and error-prone approach. We believe that it is desirable to possess a high-level and portable parallel programming model for multi-core DSPs. In this paper, we propose OpenMDSP, an extension of OpenMP designed for multi-core DSPs. The goal of OpenMDSP is to fill the gap between the OpenMP memory model and the memory hierarchy of multi-core DSPs. We propose three classes of directives in OpenMDSP, including 1) data placement directives that allow programmers to control the placement of global variables conveniently, 2) distributed array directives that divide a whole array into sections and promote the sections into core-local memory to improve performance, and 3) stream access directives that promote big arrays into core-local memory section by section during parallel loop processing while hiding the latency of data movement by the direct memory access (DMA) of a DSP. We implement the compiler and runtime system for OpenMDSP on FreeScale MSC8156. The benchmarking results show that seven of nine benchmarks achieve a speedup of more than a factor of 5 when using six threads.

WillItFit: a framework for fitting of constrained models to small-angle scattering data

A software framework for analysis of small-angle scattering data is presented. On the basis of molecular constraints and prior knowledge of the chemical composition of the sample, the software is capable of simultaneously fitting small-angle X-ray and neutron scattering data to analytical or semi-analytical models of biomacromolecules. The software features various fitting routines along with the possibility of incorporating instrumental resolution effects on the fit. Finally, trust region estimation, based on the profile likelihood strategy, is implemented. The algorithms and models are written in C, whereas the user interface is written in Python. Parallelization is implemented using the OpenMP extensions to C. The source code is available for free upon request orviathe associated code repository. The software runs on Linux, Windows and OSX and is available as an open-source initiative published under the General Publishing License.

Optimization of the Code of the Numerical Magnetosheath-Magnetosphere Model

Abstract The proposed three dimensional model contains two earlier developed 3D regional numerical models: a grid-characteristic model of the magnetosheath and a finite element model of the magnetosphere. The model output is the distribution of gas-dynamic parameters in the magnetosheath and of magnetic field inside the magnetosphere. The efforts are focused on the modernization of the existing software, written in Fortran, using several techniques for parallel programming such as OpenMP extensions. After analyzing the numerical performance of the model a possible scenario for the code optimization is shown. First results with the improved variant of the model are presented.

OpenStream

We present OpenStream, a data-flow extension of OpenMP to express dynamic dependent tasks. The language supports nested task creation, modular composition, variable and unbounded sets of producers/consumers, and first-class streams. These features, enabled by our original compilation flow, allow translating high-level parallel programming patterns, like dependences arising from StarSs' array regions, or universal low-level primitives like futures. In particular, these dynamic features can be embedded efficiently and naturally into an unmanaged imperative language, avoiding the complexity and overhead of a concurrent garbage collector. We demonstrate the performance advantages of a data-flow execution model compared to more restricted task and barrier models. We also demonstrate the efficiency of our compilation and runtime algorithms for the support of complex dependence patterns arising from StarSs benchmarks.

Numerical model for macroscopic quantum superpositions based on phase-covariant quantum cloning

Macroscopically populated quantum superpositions pose a question to what extent the macroscopic world obeys quantum mechanical laws. Recently, such superpositions for light, generated by an optimal quantum cloner, have been demonstrated. They are of fundamental and technological interest. We present numerical methods useful for modeling of these states. Their properties are governed by a Gaussian hypergeometric function, which cannot be reduced to either elementary or easily tractable functions. We discuss the method of efficient computation of this function for half-integer parameters and a moderate value of its argument. We show how to dynamically estimate a cutoff for infinite sums involving this function performed over its parameters. Our algorithm exceeds double precision and is parallelizable. Depending on the experimental parameters it chooses one of the several ways of summation to achieve the best efficiency. The methods presented here can be adjusted for analysis of similar experimental schemes.Program summaryProgram title: MQSVISCatalogue identifier: AEMR_ v1_ 0Program summary URL: http://cpc.cs.qub.ac.uk/summaries/AEMR_v1_0.htmlProgram obtainable from: CPC Program Library, Queen’s University, Belfast, N. IrelandLicensing provisions: Standard CPC licence, http://cpc.cs.qub.ac.uk/licence/licence.htmlNo. of lines in distributed program, including test data, etc.: 1643No. of bytes in distributed program, including test data, etc.: 13212Distribution format: tar.gzProgramming language: C with OpenMP extensions (main numerical program), Python (helper scripts).Computer: Modern PC (tested on AMD and Intel processors), HP BL2x220.Operating system: Unix/Linux.Has the code been vectorized or parallelized?: Yes (OpenMP).RAM: 200 MB for single run for 1000×1000 tileClassification: 4.15, 18.External routines: OpenMPNature of problem:Recently, macroscopically populated quantum superpositions for light, generated by an optimal quantum cloner, have been demonstrated. They are of fundamental and technological interest. Their properties are governed by Gaussian hypergeometric function 2F1 of half-integer parameters, which cannot be reduced to either elementary or easily tractable functions. Computation of the photon number distribution, visibility, and mean number of photons, necessary for characterization of these states, requires evaluation of infinite sums involving this function performed over its parameters.Solution method:The MQSVIS program suite computes various quantum indicators, such as photon number distribution, visibility, mean number of photons, and variance, for macroscopic quantum superpositions of light. It takes losses (modeled with a beam splitter) and imperfect photodetection (modeled with a Weierstrass transform applied to the photon number distribution) into account. Cutoffs of the infinite hypergeometric sums are estimated dynamically, and precision is enhanced with the computation of expressions in logarithmic form. Depending on the experimental parameters, the program chooses one of several ways of summation to achieve the best efficiency. The program is parallelized using OpenMP standard, which ensures the best utilization of multicore processors, and splits the work into tiles computed with different nodes of a computer cluster. This allows computation of the required indicators for realistic values of the parameters.Running time:1–2 h for a 1000×1000 tile, depending on the values of the parameters.

Exploiting coarse-grain speculative parallelism

Speculative execution at coarse granularities (e.g., code-blocks, methods, algorithms) offers a promising programming model for exploiting parallelism on modern architectures. In this paper we present Anumita, a framework that includes programming constructs and a supporting runtime system to enable the use of coarse-grain speculation to improve program performance, without burdening the programmer with the complexity of creating, managing and retiring speculations. Speculations may be composed by specifying surrogate code blocks at any arbitrary granularity, which are then executed concurrently, with a single winner ultimately modifying program state. Anumita provides expressive semantics for winner selection that go beyond time to solution to include user-defined notions of quality of solution. Anumita can be used to improve the performance of hard to parallelize algorithms whose performance is highly dependent on input data. Anumita is implemented as a user-level runtime with programming interfaces to C, C++, Fortran and as an OpenMP extension. Performance results from several applications show the efficacy of using coarse-grain speculation to achieve (a) robustness when surrogates fail and (b) significant speedup over static algorithm choices.

Exploiting memory affinity in OpenMP through schedule reuse

In this paper we explore the idea of reusing loop schedules to improve the scalability of numerical codes in shared-memory architectures with non-uniform memory access. The main objective is to implicitly construct affinity links between threads and data accesses and reuse them as much as possible along the execution of the program. These links are created through the definition and reuse of iteration schedules which are either defined statically by the user or created dynamically at run time. The paper does not include a formal proposal of OpenMP extensions but includes some experiments showing the usefulness of constructing affinity links in both regular and irregular codes.

Flexible control structures for parallelism in OpenMP

OpenMP cannot handle some very common programming idioms like recursive control and list or tree data structures. We present the workqueuing model and show it as a natural, flexible, and easy to use extension of OpenMP that is available as a commercial product. A detailed description of workqueuing is presented, together with performance results and pointers to the source code used. Copyright © 2000 John Wiley & Sons, Ltd.

NanosCompiler: supporting flexible multilevel parallelism exploitation in OpenMP

This paper describes the support provided by the NanosCompiler to nested parallelism in OpenMP. The NanosCompiler is a source-to-source parallelizing compiler implemented around a hierarchical internal program representation that captures the parallelism expressed by the user (through OpenMP directives and extensions) and the parallelism automatically discovered by the compiler through a detailed analysis of data and control dependences. The compiler is finally responsible for encapsulating work into threads, establishing their execution precedences and selecting the mechanisms to execute them in parallel. The NanosCompiler enables the experimentation with different work allocation strategies for nested parallel constructs. Some OpenMP extensions are proposed to allow the specification of thread groups and precedence relations among them. Copyright © 2000 John Wiley & Sons, Ltd.