Shared-memory Parallel Programs Research Articles

New efficient fractal models for MapReduce in OpenMP parallel environment

Parallel data processing is one of the specific infrastructure applications categorized as a service provided by cloud computing. In cloud computing environments, data-intensive applications increasingly use the parallel processing paradigm known as MapReduce. MapReduce is based on a strategy called "divide and conquer," which uses ordinary computers, also called "nodes," to do processing in parallel. This paper looks at how open multiprocessing (OpenMP), the best shared-memory parallel programming model for high-performance computing, can be used in the MapReduce application using proposed fractal network models. Two fractal network models are offered, and their work is compared with a well-known network model, the hypercube. The first fractal network model achieved an average speedup of 3.239 times while an efficiency ranged from 73-95%. In the second model of the network, the speedup got to 3.236 times while keeping an efficiency of 70-92%. Furthermore, the path-finding algorithm employed in the recommended fractal network models remarkably identified all paths and calculated the shortest and longest routes.

New efficient fractal models for MapReduce in OpenMP parallel environment

Parallel data processing is one of the specific infrastructure applications categorized as a service provided by cloud computing. In cloud computing environments, data-intensive applications increasingly use the parallel processing paradigm known as MapReduce. MapReduce is based on a strategy called "divide and conquer," which uses ordinary computers, also called "nodes," to do processing in parallel. This paper looks at how open multiprocessing (OpenMP), the best shared-memory parallel programming model for high-performance computing, can be used in the MapReduce application using proposed fractal network models. Two fractal network models are offered, and their work is compared with a well-known network model, the hypercube. The first fractal network model achieved an average speedup of 3.239 times while an efficiency ranged from 73-95%. In the second model of the network, the speedup got to 3.236 times while keeping an efficiency of 70-92%. Furthermore, the path-finding algorithm employed in the recommended fractal network models remarkably identified all paths and calculated the shortest and longest routes.

Acceleration of a Production-Level Unstructured Grid Finite Volume CFD Code on GPU

Due to the complex topological relationship, poor data locality, and data racing problems in unstructured CFD computing, how to parallelize the finite volume method algorithms in shared memory to efficiently explore the hardware capabilities of many-core GPUs has become a significant challenge. Based on a production-level unstructured CFD software, three shared memory parallel programming strategies, atomic operation, colouring, and reduction were designed and implemented by deeply analysing its computing behaviour and memory access mode. Several data locality optimization methods—grid reordering, loop fusion, and multi-level memory access—were proposed. Aimed at the sequential attribute of LU-SGS solution, two methods based on cell colouring and hyperplane were implemented. All the parallel methods and optimization techniques implemented were comprehensively analysed and evaluated by the three-dimensional grid of the M6 wing and CHN-T1 aeroplane. The results show that using the Cuthill–McKee grid renumbering and loop fusion optimization techniques can improve memory access performance by 10%. The proposed reduction strategy, combined with multi-level memory access optimization, has a significant acceleration effect, speeding up the hot spot subroutine with data races three times. Compared with the serial CPU version, the overall speed-up of the GPU codes can reach 127. Compared with the parallel CPU version, the overall speed-up of the GPU codes can achieve more than thirty times the result in the same Message Passing Interface (MPI) ranks.

An innovativefractal architecture model for implementing MapReduce in an open multiprocessing parallel environment

One of the infrastructure applications that cloud computing offers as a service is parallel data processing. MapReduce is a type of parallel processing used more and more by data-intensive applications in cloud computing environments. MapReduce is based on a strategy called "divide and conquer," which uses regular computers, also called "nodes," to do processing in parallel. This paper looks at how open multiprocessing (OpenMP), the best shared-memory parallel programming model for high-performance computing, can be used with the proposed fractal network model in the MapReduce application. A well-known model, the cube, is used to compare the fractal network model and its work. Where experiments demonstrated that the fractal model is preferable to the cube model. The fractal model achieved an average speedup of 2.7 and an efficiency rate of 67.7%. In contrast, the cube model could only reach an average speedup of 2.5 and an efficiency rate of 60.4%.

An efficient multiple scanning order algorithm for accumulative least-cost surface calculation

The least-cost surface (LCS) calculation is a compute-intensive problem conventionally solved by the queue-based Dijkstra’s algorithm. Alternative raster-based scanning algorithms have also been proposed which use a moving window to scan the whole study area iteratively. Here we propose improvements to the raster-based algorithms. The main improvement is to implement multiple scanning orders (MSO) to replace the conventional single scanning order (SSO, typically from upper-left corner to lower-right corner, row by row). We compared the performance of different algorithms over different cost surfaces and with different numbers of source points. The comparison shows that a raster-based algorithm adopting MSO has a substantially better performance than a conventional raster-based algorithm using SSO. An MSO raster-based algorithm is generally comparable to the queue-based Dijkstra’s algorithm, and surpasses the latter over a relatively simple cost surface (e.g. in which the cost is resampled) and/or when the number of source points is relatively large. Our empirical experiments suggest that MSO reduces the time complexity from to to Additionally, we found that the MSO raster-based algorithm can be easily parallelized using shared-memory parallel programming.

A survey on optimizations towards best-effort hardware transactional memory

Transactional memory has been attracting increasing attention in recent years, and it provides optimistic concurrency control schemes for shared-memory parallel programs. The rapid development and wide adoption of transactional memory make this programming paradigm promising for achieving breakthroughs in massively parallel computing. There has been a large number of discussions towards transactional memory systems, which aimed at providing relatively simple and intuitive synchronization construction for shared-memory parallel programs without sacrificing performance. Hardware transactional memory (HTM) has become commercially available in mainstream processors, however, due to several inherent architectural limitations that will abort hardware transactions, such as cache overflows, context switches, hardware as well as software exceptions, etc., nowadays HTM systems come in a best-effort way, which necessitates the adoption of a software fallback path to ensure forward progress. In this paper, we survey state-of-the-art software-side optimizations for best-effort hardware transaction system, as well as several novel performance tuning techniques. Research efforts about joint usage of HTM and non-volatile memory (NVM) are also discussed.

Retrofitting parallelism onto OCaml

OCaml is an industrial-strength, multi-paradigm programming language, widely used in industry and academia. OCaml is also one of the few modern managed system programming languages to lack support for shared memory parallel programming. This paper describes the design, a full-fledged implementation and evaluation of a mostly-concurrent garbage collector (GC) for the multicore extension of the OCaml programming language. Given that we propose to add parallelism to a widely used programming language with millions of lines of existing code, we face the challenge of maintaining backwards compatibility--not just in terms of the language features but also the performance of single-threaded code running with the new GC. To this end, the paper presents a series of novel techniques and demonstrates that the new GC strikes a balance between performance and feature backwards compatibility for sequential programs and scales admirably on modern multicore processors.

A comparison of the shared-memory parallel programming models OpenMP, OpenACC and Kokkos in the context of implicit solvers for high-order FEM

We consider the application of three performance-portable programming models in the context of a high-order spectral element, implicit time-stepping solver for the Navier–Stokes equations. We aim to evaluate whether the use of these models allows code developers to deliver high-performance solvers for computational fluid dynamics simulations that are capable of effectively utilising both many-core CPU and GPU architectures. Using the core elliptic solver for the Navier–Stokes equations as a benchmarking guide, we evaluate the performance of these models on a range of unstructured meshes and give guidelines for the translation of existing codebases and their data structures to these models.

A Generalized and Parallelized SSIM-Based Multilevel Thresholding Algorithm

ABSTRACTMultilevel thresholding is a widely used technique to perform image segmentation. It consists of dividing an input image into several distinct regions by finding the optimal thresholds according to a certain objective function. In this work, we generalize the use of the SSIM quality measure as an objective function to solve the multilevel thresholding problem using empirically tuned swarm intelligence algorithms. The experimental study we have conducted shows that our approach, producing near-exact solutions, is more effective compared to the state-of-the-art methods. Moreover, we show that the computation complexity has been significantly reduced by adopting a shared-memory parallel programming paradigm for all the algorithms we have implemented.

FastJT: An R package for robust and efficient feature selection for machine learning and genome-wide association studies

BackgroundParametric feature selection methods for machine learning and association studies based on genetic data are not robust with respect to outliers or influential observations. While rank-based, distribution-free statistics offer a robust alternative to parametric methods, their practical utility can be limited, as they demand significant computational resources when analyzing high-dimensional data. For genetic studies that seek to identify variants, the hypothesis is constrained, since it is typically assumed that the effect of the genotype on the phenotype is monotone (e.g., an additive genetic effect). Similarly, predictors for machine learning applications may have natural ordering constraints. Cross-validation for feature selection in these high-dimensional contexts necessitates highly efficient computational algorithms for the robust evaluation of many features.ResultsWe have developed an R extension package, fastJT, for conducting genome-wide association studies and feature selection for machine learning using the Jonckheere-Terpstra statistic for constrained hypotheses. The kernel of the package features an efficient algorithm for calculating the statistics, replacing the pairwise comparison and counting processes with a data sorting and searching procedure, reducing computational complexity from O(n2) to O(n log(n)). The computational efficiency is demonstrated through extensive benchmarking, and example applications to real data are presented.ConclusionsfastJT is an open-source R extension package, applying the Jonckheere-Terpstra statistic for robust feature selection for machine learning and association studies. The package implements an efficient algorithm which leverages internal information among the samples to avoid unnecessary computations, and incorporates shared-memory parallel programming to further boost performance on multi-core machines.

Performance evaluation of OpenMP's target construct on GPUs - exploring compiler optimisations

OpenMP is a directive-based shared memory parallel programming model and has been widely used for many years. From OpenMP 4.0 onwards, GPU platforms are supported by extending OpenMP's high-level p...

Bounding data races in space and time

We propose a new semantics for shared-memory parallel programs that gives strong guarantees even in the presence of data races. Our local data race freedom property guarantees that all data-race-free portions of programs exhibit sequential semantics. We provide a straightforward operational semantics and an equivalent axiomatic model, and evaluate an implementation for the OCaml programming language. Our evaluation demonstrates that it is possible to balance a comprehensible memory model with a reasonable (no overhead on x86, ~0.6% on ARM) sequential performance trade-off in a mainstream programming language.

Featherlight on-the-fly false-sharing detection

Shared-memory parallel programs routinely suffer from false sharing---a performance degradation caused by different threads accessing different variables that reside on the same CPU cacheline and at least one variable is modified. State-of-the-art tools detect false sharing via a heavyweight process of logging memory accesses and feeding the ensuing access traces to an offline cache simulator. We have developed F eather , a lightweight, on-the-fly false-sharing detection tool. F eather achieves low overhead by exploiting two hardware features ubiquitous in commodity CPUs: the performance monitoring units (PMU) and debug registers. Additionally, F eather is a first-of-its-kind tool to detect false sharing in multi-process applications that use shared memory. F eather allowed us to scale false-sharing detection to myriad codes. F eather detected several false-sharing cases in important multi-core and multi-process codes including previous PPoPP artifacts. Eliminating false sharing resulted in dramatic (up to 16x) speedups.

GVF snake algorithm-a parallel approach

Multicore architecture is an emerging computer technology where multiple processing element will be acting as independent processing cores by sharing a common memory. Digital image segmentation is a widely used medical imaging application to extract regions of interest. GVF Active Contour is a region based segmentation technique which extracts curved and irregular shaped regions by diffusing gradient vectors and by the influence of internal and external forces. This requires prior knowledge on the geometric position and anatomical structures to locate the specific region defined within an image domain. This process requires complex mathematical calculations which in turn results in the immense consumption of CPU processing time. This may adversely affect the overall performance efficiency of the process. With the advancements in multicore technology, this processing time delay can be reduced by adapting parallelization in the computation of GVF field to the specific region of interest which is to be segmented. OpenMP is a shared memory parallel programming construct, which could implement multicore parallelism with extensive and powerful APIs thereby supporting the functionalities required to attain parallelism. This article provides a high level overview of OpenMP, its effectiveness and ease of implementation in adapting parallelism to existing traditional sequential methods using instruction, data and loop level parallelism. Performance comparison could be done with sequential versions of the program written in Matlab, Java and C languages with the proposed parallelized version of OpenMP. The result is also comparable with different operating systems like Windows and Linux.

Parallel simulation of dam-break flow by OpenMP-based SPH method

Smoothed particle hydrodynamics (SPH), a Lagrangian mesh-free particle numerical method, is suitable for simulating strong impact and large deformation problems. In the method, quantities of particles can ensure high precision. However, with the increase of the particle numbers, the calculation efficiency becomes a challenge for applying the method to the engineering practice. OpenMP, a Portable Shared Memory Parallel Programming, is a great solution to improve the efficiency of SPH algorithm. In the paper, dam-break flow is simulated by SPH method. Vortex centre is discovered, the role of numerical technique is compared. At the same time, two parallel schemes for SPH algorithm is introduced, the speedup ratio with respect to number of particles or threads are revealed.

A Fast Implementation of Minimum Spanning Tree Method and Applying it to Kruskal’s and Prim’s Algorithms

In last decade, application developers attained improved performances by merely employing the machines based on higher-clocked processors. However, in 2003 multi-core processors emerged and eradicated the old processor manufacturing technology based on increasing processor’s clock frequencies. After emergence of new parallel processor architectures, serial applications must be re-engineered into parallel versions to exploit the computing power of the existing hardware. In this paper, we present an efficient parallel implementation of minimum spanning tree algorithm to take advantage of the computing power of multi-core machines. Computer network routing, civil infrastructure planning and cluster analysis are typically use-cases of spanning tree problem. The experimental results show that the proposed algorithm is scalable for different machine and graph sizes. The methodology is simple and can easily be implemented using different shared-memory parallel programming models.

A general and efficient divide-and-conquer algorithm framework for multi-core clusters

Divide-and-conquer is one of the most important patterns of parallelism, being applicable to a large variety of problems. In addition, the most powerful parallel systems available nowadays are computer clusters composed of distributed-memory nodes that contain an increasing number of cores that share a common memory. The optimal exploitation of these systems often requires resorting to a hybrid model that mimics the underlying hardware by combining a distributed and a shared memory parallel programming model. This results in longer development times and increased maintenance costs. In this paper we present a very general skeleton library that allows to parallelize any divide-and-conquer problem in hybrid distributed-shared memory systems with little effort while providing much flexibility and good performance. Our proposal combines a message-passing paradigm at the process level and a threaded model inside each process, hiding the related complexity from the user. The evaluation shows that this skeleton provides performance comparable, and often better than that of manually optimized codes while requiring considerably less effort when parallelizing applications on multi-core clusters.

OpenMP-Based Approach for High Level C Loops Synthesis

The complexity of embedded systems design is continuously augmented, due to the increasing quantity of components and distinct functionalities incorporated into a single system. To deal with this situation, abstraction level of projects is incessantly raised. In addition, techniques to accelerate the code production process have appeared. In this context, the automatic code generation is an interesting technique for the embedded systems project. This work presents an automatic VHDL code generation method based on the OpenMP parallel programming specification. In order to synthesize C code for loops into hardware, the authors applied the directives of OpenMP, which specifies portable implementations of shared memory parallel programs. A case study focused on the use of embedded systems for the DCT algorithm is presented in this paper to demonstrate the feasibility of the proposed approach.

GEMs: shared-memory parallel programming for Node.js

JavaScript is the most popular programming language for client-side Web applications, and Node.js has popularized the language for server-side computing, too. In this domain, the minimal support for parallel programming remains however a major limitation. In this paper we introduce a novel parallel programming abstraction called Generic Messages (GEMs). GEMs allow one to combine message passing and shared-memory parallelism, extending the classes of parallel applications that can be built with Node.js. GEMs have customizable semantics and enable several forms of thread safety, isolation, and concurrency control. GEMs are designed as convenient JavaScript abstractions that expose high-level and safe parallelism models to the developer. Experiments show that GEMs outperform equivalent Node.js applications thanks to their usage of shared memory.

Power System Dynamic Simulations Using a Parallel Two-Level Schur-Complement Decomposition

As the need for faster power system dynamic simulations increases, it is essential to develop new algorithms that exploit parallel computing to accelerate those simulations. This paper proposes a parallel algorithm based on a two-level, Schur-complement-based, domain decomposition method. The two-level partitioning provides high parallelization potential (coarse- and fine-grained). In addition, due to the Schur-complement approach used to update the sub-domain interface variables, the algorithm exhibits high global convergence rate. Finally, it provides significant numerical and computational acceleration. The algorithm is implemented using the shared-memory parallel programming model, targeting inexpensive multi-core machines. Its performance is reported on a real system as well as on a large test system combining transmission and distribution networks.