OpenMP Parallel Research Articles

Auto-Tuning of Dynamic Scheduling Applied to 3D Reverse Time Migration on Multicore Systems

Reverse time migration (RTM) is an algorithm widely used in the oil and gas industry to process seismic data. It is a computationally intensive task that suits well in parallel computers. Methods such as RTM can be parallelized in shared memory systems through scheduling iterations of parallel loops to threads. However, several aspects, such as memory size and hierarchy, number of cores, and input size, make optimal scheduling very challenging. In this paper, we introduce a run-time strategy to automatically tune the dynamic scheduling of parallel loops iterations in iterative applications, such as the RTM, in multicore systems. The proposed method aims to reduce the execution time of such applications. To find the optimal granularity, we propose a coupled simulated annealing (CSA) based auto-tuning strategy that adjusts the chunk size of work that OpenMP parallel loops assign dynamically to worker threads during the initialization of a 3D RTM application. Experiments performed with different computational systems and input sizes show that the proposed method is consistently better than the default OpenMP schedulers, static, auto, and guided, causing the application to be up to 33% faster. We show that the possible reason for this performance is the reduction of cache misses, mainly level L3, and low overhead, inferior to 2%. Having shown to be robust and scalable for the 3D RTM, the proposed method could also improve the performance of similar wave-based algorithms, such as full-waveform inversion (FWI) and other iterative applications.

Real-time simulation of hydrodynamic and scalar transport in large river-lake systems

Real-time coupling of a hydro-environmental model and an optimization model is difficult because hydrodynamic model (HDM) and scalar transport model (STM) are both time-consuming, especially when the computational domain is large (e.g., river-lake system). This study presents a new 1D hydro-environmental model for free-surface flows and scalar transport in large river-lake systems. Using a prediction–correction method, hydrodynamic is simulated by solving local linear systems based on domain decomposition. A flux-form Eulerian-Lagrangian method (ELM) is constructed to solve advective transport of scalars in 1D grid systems, and a nested technique is proposed to reduce its startup cost. The resulting STM and HDM both allow large time steps for which the Courant–Friedrichs–Lewy number (CFL) is greater than 1, and they are parallelized using the open multiprocessing technique (OpenMP). Moreover, the STM is good at solving multiscalar transport and has low startup cost. The new model is tested using the Jing-Dongting (JDT) system which is covered by a grid of 2382 cells (with 113,600 sub-grids). Stable and accurate simulations are achieved at large time steps for which the CFL can be larger than 5. A sequential run of the new model runs tens of times faster than that of a conventional 1D model such as the Mike11. Moreover, the efficiency of the new model can be further improved by the OpenMP parallel technique. In the test of scale property of the HDM + STM model (using 32 kinds of scalars and 16 cores), a parallel run is 11.8 times faster than a sequential run, and it only takes the new model 33.3 s to complete a simulation of a 1-year process of unsteady flow and scalar transport in the JDT system.

Efficient optical parameter mapping based on time-domain radiative transfer equation combined with parallel programming.

A two-dimensional optical parameter mapping based on the time-domain radiative transfer equation (TD-RTE) is studied in this work. The finite element method with structured and unstructured grids is employed to solve TD-RTE and OpenMP parallel technology is employed to improve the computing efficiency. The sequential quadratic programming algorithm is used as a powerful optimization method to reconstruct absorption and scattering parameter fields and the maximum a posteriori estimation is employed by introducing the regularization term into the objective function to improve the ill-posed inverse problem. In addition, the effects of measurement errors on reconstruction accuracy are investigated thoroughly. All the simulation results demonstrate that the reconstructed scheme we developed is accurate and efficient in optical parameter mapping based on TD-RTE.

Source-to-source compilation targeting OpenMP-based automatic parallelization of C applications

Directive-driven programming models, such as OpenMP, are one solution for exploring the potential parallelism when targeting multicore architectures. Although these approaches significantly help developers, code parallelization is still a non-trivial and time-consuming process, requiring parallel programming skills. Thus, many efforts have been made toward automatic parallelization of the existing sequential code. This article presents AutoPar-Clava, an OpenMP-based automatic parallelization compiler which: (1) statically detects parallelizable loops in C applications; (2) classifies variables used inside the target loop based on their access pattern; (3) supports reduction clauses on scalar and array variables whenever it is applicable; and (4) generates a C OpenMP parallel code from the input sequential version. The effectiveness of AutoPar-Clava is evaluated by using the NAS and Polyhedral Benchmark suites and targeting a x86-based computing platform. The achieved results are very promising and compare favorably with closely related auto-parallelization compilers, such as Intel C/C++ Compiler (icc), ROSE, TRACO and CETUS.

Application of efficient TEP FE computation on accurate fabrication of cylindrical leg structure of jack-up rig

Cylindrical leg structure is an essential component of jack-up rig, and its fabrication accuracy will significantly influence the operation performance and service lift. Due to the welding distortion generated during rack and cylinder joining and its effect on dimensional profile of cylindrical leg structure, the experiment to fabricate the cylindrical leg structure and dimensional measurement were conducted in advance, and highly efficient Thermal Elastic Plastic (TEP) FE computation with Iterative Substructure Method (ISM) and OpenMP parallel computation was proposed and employed for welding distortion prediction. Simplified cylinder FE model was considered to clarify the mechanism of dimensional variation in advance, and efficiency of proposed TEP FE computation was also demonstrated. Quarter FE model of rack and cylinder welded structure was then employed due to its symmetrical feature while computed results have a good agreement comparing to the measurement. Meanwhile, in order to improve the fabrication accuracy, bead-on-plate welding on inner surface of cylindrical leg structure was numerically examined, while inverse deformation can be generated and has significant influence to mitigate welding distortion during rack and cylinder joining.

VeloxChem: A Python‐driven density‐functional theory program for spectroscopy simulations in high‐performance computing environments

AbstractAn open‐source program named VeloxChem has been developed for the calculation of electronic real and complex linear response functions at the levels of Hartree–Fock and Kohn–Sham density functional theories. With an object‐oriented program structure written in a Python/C++ layered fashion, VeloxChem enables time‐efficient prototyping of novel scientific approaches without sacrificing computational efficiency, so that molecular systems involving up to and beyond 500 second‐row atoms (or some 10,000 contracted and in part diffuse Gaussian basis functions) can be routinely addressed. In addition, VeloxChem is equipped with a polarizable embedding scheme for the treatment of the classical electrostatic interactions with an environment that in turn is modeled by atomic site charges and polarizabilities. The underlying hybrid message passing interface (MPI)/open multiprocessing (OpenMP) parallelization scheme makes VeloxChem suitable for execution in high‐performance computing cluster environments, showing even slightly beyond linear scaling for the Fock matrix construction with use of up to 16,384 central processing unit (CPU) cores. An efficient—with respect to convergence rate and overall computational cost—multifrequency/gradient complex linear response equation solver enables calculations not only of conventional spectra, such as visible/ultraviolet/X‐ray electronic absorption and circular dichroism spectra, but also time‐resolved linear response signals as due to ultra‐short weak laser pulses. VeloxChem distributed under the GNU Lesser General Public License version 2.1 (LGPLv2.1) license and made available for download from the homepage https://veloxchem.org.This article is categorized under: Software > Quantum Chemistry Electronic Structure Theory > Density Functional Theory Theoretical and Physical Chemistry > Spectroscopy

Hybrid Distributed/Shared Memory Model for the RI-MP2 Method in the Fragment Molecular Orbital Framework.

The general distributed data interface (GDDI) that was developed for the fragment molecular orbital (FMO) method is combined with the shared memory OpenMP parallel middleware to support a threading multilevel parallelism. First, GDDI partitions [logical] compute nodes into groups, which are statically or dynamically assigned to different fragments. A small number of processes are created on each compute node. Each process subsequently spawns multiple threads for the actual computation. The performance of the hybrid GDDI/OpenMP approach relative to the pure GDDI model was examined in terms of the FMO/RI-MP2 method; that is, the second-order Moller-Plesset (MP2) correlation energy was evaluated using the resolution-of-the-identity (RI) and the FMO approximations. The GDDI and OpenMP workload balances are handled by an arithmetic progression and a loop fusion, respectively. Other OpenMP properties, such as threadprivate or shared memory, are combined with the low memory demand of the RI two-electron integrals to enhance the performance. Benchmark calculations demonstrate that because the hybrid parallel model can make use of multiprocessor resources more efficiently than the regular distributed memory-based GDDI model, calculations for small to large water clusters containing 139-2165 molecules and an ionic liquid cluster exhibit node linear scaling and speedups of a factor of 10×.

The Chronus Quantum software package

AbstractThe Chronus Quantum (ChronusQ) software package is an open source (under the GNU General Public License v2) software infrastructure which targets the solution of challenging problems that arise in ab initio electronic structure theory. Special emphasis is placed on the consistent treatment of time dependence and spin in the electronic wave function, as well as the inclusion of relativistic effects in said treatments. In addition, ChronusQ provides support for the inclusion of uniform finite magnetic fields as external perturbations through the use of gauge‐including atomic orbitals. ChronusQ is a parallel electronic structure code written in modern C++ which utilizes both message passing implementation and shared memory (OpenMP) parallelism. In addition to the examination of the current state of code base itself, a discussion regarding ongoing developments and developer contributions will also be provided.This article is categorized under: Software > Quantum Chemistry Electronic Structure Theory > Ab Initio Electronic Structure Methods Electronic Structure Theory > Density Functional Theory

Source-to-Source Parallelization Compilers for Scientific Shared-Memory Multi-core and Accelerated Multiprocessing: Analysis, Pitfalls, Enhancement and Potential

Parallelization schemes are essential in order to exploit the full benefits of multi-core architectures, which have become widespread in recent years, especially for scientific applications. In shared memory architectures, the most common parallelization API is OpenMP. However, the introduction of correct and optimal OpenMP parallelization to applications is not always a simple task, due to common parallel shared memory management pitfalls and architecture heterogeneity. To ease this process, many automatic parallelization compilers were created. In this paper we focus on three source-to-source compilers—AutoPar, Par4All and Cetus—which were found to be most suitable for the task, point out their strengths and weaknesses, analyze their performances, inspect their capabilities and suggest new paths for enhancement. We analyze and compare the compilers’ performances over several different exemplary test cases, with each test case pointing out different pitfalls, and suggest several new ways to overcome these pitfalls, while yielding excellent results in practice. Moreover, we note that all of those source-to-source parallelization compilers function in the limits of OpenMP 2.5—an outdated version of the API which is no longer in optimal accordance with nowadays complicated heterogeneous architectures. Therefore we suggest a path to exploit the new features of OpenMP 4.5, as it provides new directives to fully utilize heterogeneous architectures, specifically ones that have a strong collaboration between CPUs and GPGPUs, thus it outperforms previous results by an order of magnitude.

An Out-of-Core Eigen-Solver with OpenMP Parallel Scheme for Large Spare Damped System

An out-of-core block Lanczos method with the OpenMP parallel scheme was developed to solve large spare damped eigenproblems. The symmetric generalized eigenproblem is first solved using the block Lanczos method with the preconditioned conjugate gradient (PCG) method, and the condensed damped eigenproblem is then solved to obtain the complex eigenvalues. Since the PCG solvers and out-of-core schemes are used, a large-scale eigenproblem can be solved using minimal computer memory. The out-of-core arrays only need to be read once in each Lanczos iteration, so the proposed method requires little extra CPU time. In addition, the second-level OpenMP parallel computation in the PCG solver is suggested to avoid using a large block size that often increases the number of iterations needed to achieve convergence.

GenASiSBasics: Object-oriented utilitarian functionality for large-scale physics simulations (Version 3)

GenASiS Basics provides Fortran 2003 classes furnishing extensible object-oriented utilitarian functionality for large-scale physics simulations on distributed memory supercomputers. This functionality includes physical units and constants; display to the screen or standard output device; message passing; I/O to disk; and runtime parameter management and usage statistics. This revision---Version 3 of Basics---includes a significant name change, some minor additions to functionality, and a major addition to functionality: infrastructure facilitating the offloading of computational kernels to devices such as GPUs.

Acceleration of object tracking in high‐speed videogrammetry using a parallel OpenMP and SIMD strategy

AbstractHigh‐speed cameras are able to effectively and efficiently obtain the location of moving objects over time in many practical applications. In order to meet the requirement for fast computing and processing in the object‐tracking process, this paper adopts parallel computing to process multiple video‐image sequences by using open multi‐processing (OpenMP) and single‐instruction multiple data (SIMD) simultaneously, combined with a coarse‐to‐fine matching method. Experimental results showed that the proposed method: (1) has a high accuracy compared with other methods; (2) demonstrates a higher computational efficiency than other methods; and (3) has an efficiency running on a laptop equivalent to other methods running on a computer workstation.

LibreGrowth: A tumor growth code based on reaction–diffusion equations using shared memory

In recent years, in-silico experimentation within the field of oncological medicine has been intensively investigated with the aim of better understanding tumor dynamics and dose–response relationships in cancer treatments. In a series of previous works, Luján et al. (2018, 2017, 2016)we described the micro-environmental influence on micro-tumor infiltration patterns through in-silico/in-vitro experimentation. Here we present the latest version of the software utilized for, but not limited to, those studies: LibreGrowth, a libre tumor growth code able to simulate the core growth and peripheral tumor cell infiltration, considering a benign and a malignant stages. We implemented a reaction–diffusion based model, with spatially variable diffusion coefficient, into a three-dimensional domain, using C++ and OpenMP over a GNU/Linux system. LibreGrowth aims to provide a flexible implementation for depicting heterogeneous tissues and infiltration processes, and to shed light in current therapy optimization strategies. Program summaryProgram Title: LibreGrowthProgram Files doi:http://dx.doi.org/10.17632/zp2my52xpv.1Licensing provisions: GPLv3Programming language: C++, OpenMPSupplementary material:Nature of problem: In the field of computational oncology, mathematical models based on reaction–diffusion equations describing tumor proliferation and invasion into peripheral host tissue have proved to be of clinical relevance. In-silico/in-vitro experimentation could help in the design of new strategies able to predict as much as possible the invasive behavior of a tumor, based on its particular properties and the bio-physicochemical characteristics of its microenvironment.Solution method: We introduced LibreGrowth, the latest version of the codes used in our previous studies to model the growth and infiltration of a tumor. It implements mentioned reaction–diffusion model through the standard finite difference method, using C++ and OpenMP parallelization technology.

Foreword to the special issue of the workshop on high performance computing systems (XVIII Simpósio em Sistemas Computacionais de Alto Desempenho, WSCAD 2017)

Foreword to the special issue of the workshop on high performance computing systems (XVIII Simpósio em Sistemas Computacionais de Alto Desempenho, WSCAD 2017)

Evaluating the effect of grid size and type in integrated 1D/2D coupled urban inundation modelling on the interacting discharge between the surface and sewerage system

AbstractTo obtain accurate flood analysis results, research using high‐resolution Digital Elevation Models is becoming important and studies to increase the calculation speed and accuracy are being performed. Numerous researchers have studied parallelisation techniques to improve the accuracy of urban flood analysis by grid resolution and by 1D/2D coupled models. However, there have been few studies on the effects of interacting discharge between the surface and sewerage system on urban flooding analysis using parallelisation techniques. Therefore, in this study, the efficiency of the OpenMP parallelisation technique was investigated according to grid size and type to analyse the influence of the distribution of storm drain boxes on the results of fully integrated 1D/2D urban inundation modelling focused on the interacting discharge between the surface and storm drain boxes.

Just type polyana and press Enter: a post-processing application designed with simplicity of use in mind

ABSTRACT We present a new version of POLYANA, an application that computes radial distribution functions for centres of mass of whole molecules or subunits thereof, based on DL_POLY trajectories. User friendliness and ease of future extensions were fundamental design principles of POLYANA development. Thus, the new version adds, among others, a flexible syntax to define groups of atoms; calculation of radial distribution functions beyond half the simulation cell size; OpenMP parallelism and other features on top of existing functionality. The impatient user can compile and execute it immediately to obtain molecular radial distribution functions; on the other hand, easy to learn directives offer more advanced capabilities. POLYANA is a self-contained Fortran code, built in a modular fashion that makes it easy to add new functions. It is free and open-source and it is distributed under the MIT license.

SLAC – a semi-Lagrangian artificial compressibility solver for steady-state incompressible flows

PurposeThe purpose of this paper is the development of a new density-based (DB) semi-Lagrangian method to speed up the conventional pressure-based (PB) semi-Lagrangian methods.Design/methodology/approachThe semi-Lagrangian-based solvers are typically PB, i.e. semi-Lagrangian pressure-based (SLPB) solvers, where a Poisson equation is solved for obtaining the pressure field and ensuring a divergence-free flow field. As an elliptic-type equation, the Poisson equation often relies on an iterative solution, so it can create a challenge of parallel computing and a bottleneck of computing speed. This study proposes a new DB semi-Lagrangian method, i.e. the semi-Lagrangian artificial compressibility (SLAC), which replaces the Poisson equation by a hyperbolic continuity equation with an added artificial compressibility (AC) term, so a time-marching solution is possible. Without the Poisson equation, the proposed SLAC solver is faster, particularly for the cases with more computational cells, and better suited for parallel computing.FindingsThe study compares the accuracy and the computing speeds of both SLPB and SLAC solvers for the lid-driven cavity flow and the step-flow problems. It shows that the proposed SLAC solver is able to achieve the same results as the SLPB, whereas with a 3.03 times speed up before using the OpenMP parallelization and a 3.35 times speed up for the large grid number case (512 × 512) after the parallelization. The speed up can be improved further for larger cases because of increasing the condition number of the coefficient matrixes of the Poisson equation.Originality/valueThis paper proposes a method of avoiding solving the Poisson equation, a typical computing bottleneck for semi-Lagrangian-based fluid solvers by converting the conventional PB solver (SLPB) to the DB solver (SLAC) through the addition of the AC term. The method simplifies and facilitates the parallelization process of semi-Lagrangian-based fluid solvers for modern HPC infrastructures, such as OpenMP and GPU computing.

FisherMP: fully parallel algorithm for detecting combinatorial motifs from large ChIP-seq datasets.

Detecting binding motifs of combinatorial transcription factors (TFs) from chromatin immunoprecipitation sequencing (ChIP-seq) experiments is an important and challenging computational problem for understanding gene regulations. Although a number of motif-finding algorithms have been presented, most are either time consuming or have sub-optimal accuracy for processing large-scale datasets. In this article, we present a fully parallelized algorithm for detecting combinatorial motifs from ChIP-seq datasets by using Fisher combined method and OpenMP parallel design. Large scale validations on both synthetic data and 350 ChIP-seq datasets from the ENCODE database showed that FisherMP has not only super speeds on large datasets, but also has high accuracy when compared with multiple popular methods. By using FisherMP, we successfully detected combinatorial motifs of CTCF, YY1, MAZ, STAT3 and USF2 in chromosome X, suggesting that they are functional co-players in gene regulation and chromosomal organization. Integrative and statistical analysis of these TF-binding peaks clearly demonstrate that they are not only highly coordinated with each other, but that they are also correlated with histone modifications. FisherMP can be applied for integrative analysis of binding motifs and for predicting cis-regulatory modules from a large number of ChIP-seq datasets.

Parallel adaptive Monte Carlo integration with the event generator WHIZARD

We describe a new parallel approach to the evaluation of phase space for Monte-Carlo event generation, implemented within the framework of the Whizard package. The program realizes a twofold self-adaptive multi-channel parameterization of phase space and makes use of the standard OpenMP and MPI protocols for parallelization. The modern MPI3 feature of asynchronous communication is an essential ingredient of the computing model. Parallel numerical evaluation applies both to phase-space integration and to event generation, thus covering computing-intensive parts of physics simulation for a realistic collider environment.

HAMS: A Frequency-Domain Preprocessor for Wave-Structure Interactions—Theory, Development, and Application

This paper presents the theoretical background, the numerical implementation, and the applications of a new software that has been developed in recent years for the analysis of wave-structure interactions. The software is developed in the frequency domain, as a preprocessor of computing the wave excitation force, the added mass, and the wave radiation damping, for the input to a time-domain solver via the Fourier cosine and sine transforms. In addition, it can also predict the motion responses of a marine structure with sufficient accuracy, with or without the presence of a mooring system. Unlike other frequency-domain software, such as WAMIT® and Hydrostar®, the present software currently employs the least squares method in association with a partially extended boundary integral equation method to remove the so-called “irregular frequencies”. Calculation of the free-surface Green’s function employs a combination of fast-convergent series expansions in different parametric sub-regions. The solution of the resultant linear algebraic system employs the lower-upper (LU) decomposition method. Symmetry properties can be exploited, and the open multi-processing (OpenMP) parallelization technique can be applied to reduce the computation burden. The accuracy and the efficiency of the developed software are finally confirmed by numerical validations on three benchmark cases of a floating ellipsoid, a truncated circular cylinder and the OC4 DeepCwind semisubmersible floating wind turbine. A free executable version of the software is available to the research communities with a hope of facilitating the advancements in the researches that are relevant to ocean engineering and marine renewable energies.