Parallel Programming Interface Research Articles

Benchmarking parallel programming for single-board computers

Within the computing continuum, SBCs (single-board computers) are essential in the Edge and Fog, with many featuring multiple processing cores and GPU accelerators. In this way, parallel computing plays a crucial role in enabling the full computational potential of SBCs. However, selecting the best-suited solution in this context is inherently complex due to the intricate interplay between PPI (parallel programming interface) strategies, SBC architectural characteristics, and application characteristics and constraints. To our knowledge, no solution presents a combined discussion of these three aspects. To tackle this problem, this article aims to provide a benchmark of the best-suited parallelism PPIs given a set of hardware and application characteristics and requirements. Compared to existing benchmarks, we introduce new metrics, additional applications, various parallelism interfaces, and extra hardware devices. Therefore, our contributions are the methodology to benchmark parallelism on SBCs and the characterization of the best-performing parallelism PPIs and strategies for given situations. We are confident that parallel computing will be mainstream to process edge and fog computing; thus, our solution provides the first insights regarding what kind of application and parallel programming interface is the most suited for a particular SBC hardware.

Programming in OpenCL and its advantages in a GPU Framework

Abstract: OpenCL is a framework used for building applications that mostly run on heterogenous platforms containing CPUs, GPUs, and DSPs. OpenCL provides an interface for parallel programming that can be used to take advantage of the GPUs high parallel computing power. Programmers who need complete control over the parallelization process and who are required to write portable heterogeneous code mostly use OpenCL. OpenCL views a processing unit as a collection of compute units which in turn are made up work items. According to OpenCL’s workflow and memory hierarchy, each work item is a thread as far in terms of control and memory model. A collection of work items is called a work group which is mapped to a compute unit. The language used to write “compute kernels” is called kernel language. OpenCL uses C/C++ to carry over the kernel computations done on the device. The host code specifies the kernel specifications that is needed for the computation of the device which includes creating buffers, calling kernels, mapping the memory back to CPU from device, etc. OpenCL also has specific optimization techniques that helps improve parallelization while computing on a GPU which results in better performance numbers

Robust color image watermarking using multi-core Raspberry pi cluster

Image authentication approaches have gotten a lot of interest recently as a way to safeguard transmitted images. Watermarking is one of the many ways used to protect transmitted images. Watermarking systems are pc-based that have limited portability that is difficult to use in harsh environments as military use. We employ embedded devices like Raspberry Pi to get around the PC’s mobility limitations. Digital image watermarking technology is used to secure and ensure digital images’ copyright by embedding hidden information that proves its copyright. In this article, the color images Parallel Robust watermarking algorithm using Quaternion Legendre-Fourier Moment (QLFM) in polar coordinates is implemented on Raspberry Pi (RPi) platform with parallel computing and C++ programming language. In the host image, a binary Arnold scrambled image is embedded. Watermarking algorithm is implemented and tested on Raspberry Pi model 4B. We can combine many Raspberry Pi’s into a ‘cluster’ (many computers working together as one) for high-performance computation. Message Passing Interface (MPI) and OpenMP for parallel programming to accelerate the execution time for the color image watermarking algorithm implemented on the Raspberry Pi cluster.

A Mathematical Model of Parallel Programs and an Approach Based on it to Verification of MPI Programs

The paper presents a new mathematical model of parallel programs, on the basis of which it is possible, in particular, to verify parallel programs presented on a certain subset of the parallel programming interface MPI. This model is based on the concepts of a sequential and distributed process. A parallel program is modeled as a distributed process in which sequential processes communicate by asynchronously sending and receiving messages over channels. The main advantage of the described model is the ability to simulate and verify parallel programs that generate an indefinite number of sequential processes. The proposed model is illustrated by the application of verification of the matrix multiplication MPI program.

GPU-accelerated smoothed particle hydrodynamics modeling of jet formation and penetration capability of shaped charges

The prediction of the penetration of three-dimensional (3D) shaped charge into steel plates is a challenging task. In this paper, the smoothed particle hydrodynamics (SPH) method is applied to simulate the jet formation generated by the shaped charge detonation and its damage to steel plates. The Jones–Wilkins–Lee (JWL) equation of state (EOS), Tillotson EOS, and elastic–perfectly plastic constitutive model were incorporated into SPH for the modeling of explosive detonation and dynamic behavior of metal material. The compute unified device architecture (CUDA) parallel programming interface has been employed in SPH to improve the computational efficiency of SPH. Firstly, the constitutive models and EOSs are validated by 3D TNT slab detonation and aluminum–aluminum (Al–Al) high velocity impact. Then the jet formation of the shaped charge detonation and its penetration into the steel plates are investigated using the graphics processing unit (GPU)-accelerated SPH methodology. The numerical results of these test cases are compared against the published experimental data or analytical result, which shows that the GPU-accelerated SPH methodology is capable of tackling the 3D shaped charge detonation and penetration involving millions of particles with high computational efficiency.

Requirements-preserving design automation for multiprocessor embedded system applications

The number of processors is increasing, but the usefulness of parallel computation is not better leveraged due to the inflexibility of design and implementation for multiprocessor embedded system applications. A higher level abstraction (i.e., a parallel programming framework) can ease the programmers to define parallelism for tasks in an application but designers still face the complexity of mapping high-level requirements to the syntax and semantics of a parallel programming interface. Nevertheless, embedded system applications execute various periodic tasks that are carried out repeatedly within a certain time interval and these tasks may be able to run in parallel to utilize the system efficiently. Therefore, in this paper, we propose a parallel loop-based task construct approach to automate the design process of embedded system applications from AADL models for a parallel programming framework. To illustrate the applicability of our mechanism, we use a well-known parallel programming interface called OpenMP to demonstrate the automation process in mapping tasks over multiple processor cores. To ensure meeting high-level requirements of embedded system applications, we analyze the existing OpenMP scheduling mechanisms and propose a layer of adaptation. We show that our proposed adaptation layer facilitates a tighter execution time bound for time-sensitive tasks or a better throughput for tasks that require higher quality of service. Thus, the proposed design automation framework is applicable for a variety of applications with different quality of service (QoS) requirements preserved at the lower level.

PAMPAR: A new parallel benchmark for performance and energy consumption evaluation

SummaryThis paper presents PAMPAR, a new benchmark to evaluate the performance and energy consumption of different Parallel Programming Interfaces (PPIs). The benchmark is composed of 11 algorithms implemented in PThreads, OpenMP, MPI‐1, and MPI‐2 (spawn) PPIs. Previous studies have used some of these pseudo‐applications to perform this type of evaluation in different architectures since there is no benchmark that offers this variety of PPIs and communication models. In this work, we measure the energy and performance of each pseudo‐application in a single architecture, varying the number of threads/processes. We also organize the pseudo‐applications according to their memory accesses, floating‐point operations, and branches. The goal is to show that this set of pseudo‐applications has enough features to build a parallel benchmark. The results show that there is no single best case that provides both better performance and low energy consumption in the presented scenarios. Moreover, the pseudo‐applications usage of the system resources are different enough to represent different scenarios and be efficient as a benchmark.

FUNCTIONAL TESTS FOR MULTIPROCESSOR COMPUTER SYSTEMS

The article covers the technique of construction of a subsystem functional tests designed to check the configuration of the instrument and measure the performance of a multiprocessor computing system (platform). Algorithms for determining the configuration of the system and measure the performance of processors in heterogeneous computing system. For interaction between the processors was chosen as the most common parallel programming interface standard for data exchange MPI. Performance measurement was carried out using a known test sets Intel - MPI Benchmark and HPL LINPACK, which allow to fully assess the bandwidth of communication network and the performance of the modules of Central processor computing platform. Received graphs of throughput and performance. The methods of measuring the speed exchange with memory and storage on magnetic disks. To measure the performance of graphics processors proposed an original methodology based on the parallelization of computing across multiple processors Given the overall estimation of maximum performance heterogeneous computing system. The obtained results confirm the effectiveness of the proposed methods to verify the claimed functionality of multiprocessor computing systems.

Investigating different general-purpose and embedded multicores to achieve optimal trade-offs between performance and energy

Thread-level parallelism (TLP) is being widely exploited in embedded and general-purpose multicore processors (GPPs) to increase performance. However, parallelizing an application involves extra executed instructions and accesses to the shared memory, to communicate and synchronize. The overhead of accessing the shared memory, which is very costly in terms of delay and energy because it is at the bottom of the hierarchy, varies depending on the communication model and level of data exchange/synchronization of the application. On top of that, multicore processors are implemented using different architectures, organizations and memory subsystems. In this complex scenario, we evaluate 14 parallel benchmarks implemented with 4 different parallel programming interfaces (PPIs), with distinct communication rates and TLP, running on five representative multicore processors targeted to general-purpose and embedded systems. We show that while the former presents the best performance and the latter will be the most energy efficient, there is no single option that offers the best result for both. We also demonstrate that in applications with low levels of communication, what matters is the communication model, not a specific PPI. On the other hand, applications with high communication demands have a huge search space that can be explored. For those, Pthreads is the most efficient PPI for Intel Processors, while OpenMP is the best for ARM ones. MPI is the worst choice in almost any scenario, and gets very inefficient as the TLP increases. We also evaluate energy delayxproduct (EDxP), weighting performance towards energy by varying the value of x. In a representative case where energy is the most important, three different processors can be the best alternative for different values of x. Finally, we explore how static power influences total energy consumption, showing that its increase brings benefits to ARM multiprocessors, with the opposite effect for Intel ones.

Different Approaches to Parallelization of Sparse Matrix Assembly Operation

The aim of this paper is to evaluate efficiency of different approaches to parallelization ofstiffness matrix assembly operations, that can be found in any finite element software. OpenMP [1, 2]and POSIX Threads (Pthreads) programming models are two considered in this paper. The OpenMPmodel consist of an Application Program Interface (API) for multi-platform shared-memory parallelprogramming in C/C++. POSIX is an acronym for Portable Operating System Interface and Pthreadsstandards [3] defined as a set of C/C++ language [4] programming types and procedure calls forshared-memory parallel programming. The paper shows that parallelization can efficiently exploit thepower of modern available hardware, significantly reducing the needed computation time. Differentapproaches of each programming model are used for parallelization of stiffness matrix assemblycompared and their efficiency is evaluated in this paper.The different parallelization strategies were implemented in OOFEM [5] which is a free finiteelement code with object oriented architecture for solving mechanical, transport and fluid mechanicsproblems that operates on various platforms. The finite element method leads to set of algebraic equa-tions which components are assembled from contributions of individual elements. In this paper wefocus on assembly of sparse matrix contributions, such as stiffness as mass matrices. Domain decom-position paradigm, where the whole domain is decomposed into sub domain, which contributions areevaluated and assembled by individual threads is hard.The paper discuss the differences between approaches based on OpenMP and Pthreads, datascoping specification for correct parallel programming and memory allocation synchronization andscheduling.

Burnup analysis of the VVER-1000 reactor using thorium-based fuel

Abstract This paper aims to investigate 232Th/233U fuel cycles in a VVER-1000 reactor through calculation by computer. The 3D core geometry of VVER-1000 system was designed using the Serpent Monte Carlo 1.1.19 Code. The Serpent Code using parallel programming interface (Message Passing Interface-MPI), was run on a workstation with 12-core and 48 GB RAM. 232Th/235U/238U oxide mixture was considered as fuel in the core, when the mass fraction of 232Th was increased as 0.05–0.1–0.2–0.3–0.4 respectively, the mass fraction of 238U equally was decreased. In the system, the calculations were made for 3 000 MW thermal power. For the burnup analyses, the core is assumed to deplete from initial fresh core up to a burnup of 16 MWd/kgU without refuelling considerations. In the burnup calculations, a burnup interval of 360 effective full power days (EFPDs) was defined. According to burnup, the mass changes of the 232Th, 233U, 238U, 237Np, 239Pu, 241Am and 244Cm were evaluated, and also flux and criticality of the system were calculated in dependence of the burnup rate.

Taming the parallel effect zoo

A fundamental challenge of parallel programming is to ensure that the observable outcome of a program remains deterministic in spite of parallel execution. Language-level enforcement of determinism is possible, but existing deterministic-by-construction parallel programming models tend to lack features that would make them applicable to a broad range of problems. Moreover, they lack extensibility : it is difficult to add or change language features without breaking the determinism guarantee. The recently proposed LVars programming model, and the accompanying LVish Haskell library, took a step toward broadly-applicable guaranteed-deterministic parallel programming. The LVars model allows communication through shared monotonic data structures to which information can only be added, never removed, and for which the order in which information is added is not observable. LVish provides a Par monad for parallel computation that encapsulates determinism-preserving effects while allowing a more flexible form of communication between parallel tasks than previous guaranteed-deterministic models provided. While applying LVar-based programming to real problems using LVish, we have identified and implemented three capabilities that extend its reach: inflationary updates other than least-upper-bound writes; transitive task cancellation; and parallel mutation of non-overlapping memory locations. The unifying abstraction we use to add these capabilities to LVish---without suffering added complexity or cost in the core LVish implementation, or compromising determinism---is a form of monad transformer , extended to handle the Par monad. With our extensions, LVish provides the most broadly applicable guaranteed-deterministic parallel programming interface available to date. We demonstrate the viability of our approach both with traditional parallel benchmarks and with results from a real-world case study: a bioinformatics application that we parallelized using our extended version of LVish.

A Parallel Cooperative Hybridization Approach to thep-Median Problem

The p-median problem is difficult to solve and many heuristic methods have been developed. This paper discusses a new hybridization approach in which existing heuristics are used as ‘experts’ in a problem-solving team where they cooperate by sharing their solutions with each other. This approach is demonstrated using two heuristics called the ‘vertex substitution method’ and ‘simulated annealing’. Five cooperation strategies are designed and implemented using an application program interface for parallel programming. Computational experiments show that the strategy named ‘parallel consensus’ can significantly improve the quality of solutions found when compared with the original heuristics. The results also show significant speedups of the program in a multicore parallel computing environment.

Fast Implementation of FDTD-Compatible Green's Function on Multicore Processor

In this letter, numerically efficient implementation of the finite-difference time domain (FDTD)-compatible Green's function on a multicore processor is presented. Recently, closed-form expression of this discrete Green's function (DGF) was derived, which simplifies its application in the FDTD simulations of radiation and scattering problems. Unfortunately, the new DGF expression involves binomial coefficients, whose computations may cause long runtimes and numerical problems. The proposed fast implementation of the DGF is based on the multiple precision arithmetic and employs a common programming language extended with the OpenMP parallel programming interface. As a result, the speedup factor of three orders of magnitude compared to the previous implementation was obtained, thus applicability of the DGF in FDTD simulations was significantly improved.

Improved version of parallel programming interface for distributed data with multiple helper servers

We present an improved version of the Parallel Programming Interface for Distributed Data with Multiple Helper Servers (PPIDDv2) library, which provides a common application programming interface that is based on the most frequently used functionality of both MPI-2 and GA. Compared with the previous version, the PPIDDv2 library introduces multiple helper servers to facilitate global data structures, and allows programmers to make heavy use of large global data structures efficiently. Program summaryProgram title: PPIDDv2Catalogue identifier: AEEF_v2_0Program summary URL:http://cpc.cs.qub.ac.uk/summaries/AEEF_v2_0.htmlProgram obtainable from: CPC Program Library, Queenʼs University, Belfast, N. IrelandLicensing provisions: Standard CPC licence, http://cpc.cs.qub.ac.uk/licence/licence.htmlNo. of lines in distributed program, including test data, etc.: 22 997No. of bytes in distributed program, including test data, etc.: 184 477Distribution format: tar.gzProgramming language: Fortran, CComputer: Many parallel systemsOperating system: VariousHas the code been vectorised or parallelised?: Yes. 2–1024 processors usedRAM: 50 MbytesClassification: 6.5External routines: Global Arrays or MPI-2Catalogue identifier of previous version: AEEF_v1_0Journal reference of previous version: Comput. Phys. Comm. 180 (2009) 2673Does the new version supersede the previous version?: YesNature of problem: Many scientific applications require management and communication of data that is global, and the standard MPI-2 protocol provides only low-level methods for the required one-sided remote memory access.Solution method: The Parallel Programming Interface for Distributed Data (PPIDD) library provides an interface, suitable for use in parallel scientific applications, that delivers communications and global data management. The library can be built either using the Global Arrays (GA) toolkit, or a standard MPI-2 library. This abstraction allows the programmer to write portable parallel codes that can utilise the best, or only, communications library that is available on a particular computing platform.Reasons for new version: In the previous version, functionality in global data structure was mainly implemented by MPI-2 passive one-sided operations. In real applications which make heavy use of global data structures, very poor performance was observed.Summary of revisions: Multiple helper servers are introduced to facilitate the manipulation and management of global data structure. Mutual exclusion is also implemented by the help of a data server, and becomes much more robust and efficient. In addition, flexible options are provided to choose different settings for helper servers. Significant improvement has been seen in performance tests.Running time: Problem-dependent. The test provided with the distribution takes only a few seconds to run.

Reduced Function Set Abstraction (RFSA) for MPI-IO

Programmers productivity has always been overlooked as compared to the performance optimizations in high performance computing (HPC) community. In many parallel programming languages like MPI/MPI-IO, the performance optimizations are provided as various function options to the programmers. In order to write an efficient code, they are required to know the exact usage of the optimization functions, hence programmer productivity is limited. In this paper, we present RFSA, a Reduced Function Set Abstraction based on an existing parallel programming interface (MPI-IO) for I/O. The purpose of RFSA is to hide the performance optimization functions from the application developer, and relieve the application developer from deciding on a specific function. The proposed set of functions rely on a selection algorithm to decide among the most common optimizations provided by MPI-IO. We implement a selection algorithm for I/O functions like read, write, etc., and also merge a set of functions for data types and file views. By running different parallel I/O benchmarks on both medium-scale clusters and NERSC supercomputers, we show an improved programmer productivity (35.7% on average). This approach incurs an overhead of 2–5% for one particular optimization, and shows performance improvement of 17% when a combination of different optimizations is required by an application.

Parallel programming interface for distributed data

The Parallel Programming Interface for Distributed Data (PPIDD) library provides an interface, suitable for use in parallel scientific applications, that delivers communications and global data management. The library can be built either using the Global Arrays (GA) toolkit, or a standard MPI-2 library. This abstraction allows the programmer to write portable parallel codes that can utilise the best, or only, communications library that is available on a particular computing platform. Program summary Program title: PPIDD Catalogue identifier: AEEF_v1_0 Program summary URL: http://cpc.cs.qub.ac.uk/summaries/AEEF_1_0.html Program obtainable from: CPC Program Library, Queen's University, Belfast, N. Ireland Licensing provisions: Standard CPC licence, http://cpc.cs.qub.ac.uk/licence/licence.html No. of lines in distributed program, including test data, etc.: 17 698 No. of bytes in distributed program, including test data, etc.: 166 173 Distribution format: tar.gz Programming language: Fortran, C Computer: Many parallel systems Operating system: Various Has the code been vectorised or parallelized?: Yes. 2–256 processors used RAM: 50 Mbytes Classification: 6.5 External routines: Global Arrays or MPI-2 Nature of problem: Many scientific applications require management and communication of data that is global, and the standard MPI-2 protocol provides only low-level methods for the required one-sided remote memory access. Solution method: The Parallel Programming Interface for Distributed Data (PPIDD) library provides an interface, suitable for use in parallel scientific applications, that delivers communications and global data management. The library can be built either using the Global Arrays (GA) toolkit, or a standard MPI-2 library. This abstraction allows the programmer to write portable parallel codes that can utilise the best, or only, communications library that is available on a particular computing platform. Running time: Problem dependent. The test provided with the distribution takes only a few seconds to run.

Parallel programming over ChinaGrid

Grid computing is becoming more and more attractive for providing a convenient uniform platform for coordinating highly distributed and heterogeneous resources and services. In this paper, a practical task-level distributed Parallel Programming Interface (PPI) for grid computing is introduced GridPPI. It is an MPI-like interface with high-level parallel tasking over the grid. GridPPI is a prototype of Grid-API, which supports all operations that are necessary for such task-level distributed parallel computing over grids, including service discovering and selecting, task submitting and reporting, etc. In this paper, the Web Service Resource Framework (WSRF) service-oriented implementation and evaluation of the GridPPI on the ChinaGrid Support Platform (CGSP) are discussed in detail. A performance analysis is also made that shows that our efforts could provide a flexible and effective programming library in the grid environment.

Language Support for Multi-Paradigm and Multi-Grain Parallelism on Smp-Cluster

The characteristics of large-scale parallel applications are multi-paradigm and multi-grain parallel in essence. The key factor in improving the performance of parallel application systems is to determine suitable parallel paradigms and grains according to the nature of the practical problem. Therefore, it is necessary to provide multi-paradigm and multi-grain parallel programming interface for development of large-scale parallel application systems. This paper proposes a multi-paradigm and multi-grain parallel execution model integrated coarse-grain parallelism (paralleled by macro tasks), mid-grain parallelism (paralleled by basic program blocks), and fine-grain parallelism (paralleled in repetition blocks). This model also supports the task parallel, data parallel, and sequential executing. In this paper we also discuss the programming mechanism of this model by extended OpenMP specification. The extensions include computing resource partition, defining different grain task groups, mapping from task groups to the respective processor groups, out-of-core computing, asynchronous parallel I/O , and definition of sequential relationship of tasks. We compare the performance of different implementations of benchmark, using the same numerical algorithm but employing different programming approaches, including MPI, MPI+OpenMP, and our extended OpenMP. We also discuss a case based on SMP-Cluster and network storage architecture.

A parallel programming interface for out-of-core cluster applications

Clusters of workstations are a practical approach to parallel computing that provide high performance at a low cost for many scientific and engineering applications. In order to handle problems with increasing data sets, methods supporting parallel out-of-core computations must be investigated. Since writing an out-of-core version of a program is a difficult task and virtual memory systems do not perform well in some cases, we have developed a parallel programming interface and the support library to provide efficient and convenient access to the out-of-core data. This paper focuses on how these components extend the range of problem sizes that can be solved on the cluster of workstations. Execution time of Jacobi iteration when using our interface, virtual memory and PVFS are compared to characterize the performance for various problem sizes, and it is concluded that our new interface significantly increases the sizes of problems that can be efficiently solved.