Shared Memory Parallel Architectures Research Articles

Parallel near-optimal pathfinding based on landmarks

We present a new approach for path finding in weighted graphs using pre-computed minimal distance fields. By selecting the most promising minimal distance field at any given node and switching between them, our algorithm aims to find the shortest path possible. As we show, this approach scales excellently for various topologies, graph sizes and hardware specifications while maintaining a mean length error below 1% and reasonable memory consumption. By utilizing a simplified structure and keeping backtracking to a minimum, we are able to leverage the same approach on the massively parallel GPUs or any other shared memory parallel architecture, reducing the run time even further.

Benchmarking of a distributed-memory, high-order discontinuous finite element flow solver on a shared-memory parallel architecture

High-order numerical schemes implemented on high-performance parallel computers are of special interest for contemporary numerical simulations, especially in computational fluid dynamics. In this study, first, a high-order parallel flow solver is presented for some test cases of aerodynamic simulations. The flow solver is based on a discontinuous Galerkin finite element method on arbitrary grids with different orders of polynomial approximation for solving the compressible flow model. Second, the distributed-memory parallel implementation of the flow solver is benchmarked on a shared-memory multicore system. A distributed-memory parallel application can be executed on shared-memory architectures by assuming that each of the parallel processes assumes separate memory address space, although all are present in a common memory bank. This approach can offer an effective measure to address several issues related to limited resources, especially for uninterrupted electric supply. The scalability of the parallel application is analyzed by varying the problem workload per process for the test cases. For some test cases in the present study, over 90% parallel efficiency per process is also observed. The performance of the distributed-memory program on the shared-memory architecture establishes suitability and robustness of the approach for small to medium scale problems, at least.

GPGPU-based parallel computing applied in the FEM using the conjugate gradient algorithm: a review

Parallelization of the finite-element method (FEM) has been contemplated by the scientific and high-performance computing community for over a decade. Most of the computations in the FEM are related to linear algebra that includes matrix and vector computations. These operations have the single-instruction multiple-data (SIMD) computation pattern, which is beneficial for shared-memory parallel architectures. General-purpose graphics processing units (GPGPUs) have been effectively utilized for the parallelization of FEM computations ever since 2007. The solver step of the FEM is often carried out using conjugate gradient (CG)-type iterative methods because of their larger convergence rates and greater opportunities for parallelization. Although the SIMD computation patterns in the FEM are intrinsic for GPU computing, there are some pitfalls, such as the underutilization of threads, uncoalesced memory access, lower arithmetic intensity, limited faster memories on GPUs and synchronizations. Nevertheless, FEM applications have been successfully deployed on GPUs over the last 10 years to achieve a significant performance improvement. This paper presents a comprehensive review of the parallel optimization strategies applied in each step of the FEM. The pitfalls and trade-offs linked to each step in the FEM are also discussed in this paper. Furthermore, some extraordinary methods that exploit the tremendous amount of computing power of a GPU are also discussed. The proposed review is not limited to a single field of engineering. Rather, it is applicable to all fields of engineering and science in which FEM-based simulations are necessary.

Performance upgrades to the MCNP6 burnup capability for large scale depletion calculations

The first MCNP based inline Monte Carlo depletion capability was officially released from the Radiation Safety Information and Computational Center as MCNPX 2.6.0. With the merger of MCNPX and MCNP5, MCNP6 combined the capability of both simulation tools, as well as providing new advanced technology, in a single radiation transport code. The new MCNP6 depletion capability was first showcased at the International Congress for Advancements in Nuclear Power Plants (ICAPP) meeting in 2012. At that conference the new capabilities addressed included the combined distributive and shared memory parallel architecture for the burnup capability, improved memory management, physics enhancements, and new predictability as compared to the H.B Robinson Benchmark. At Los Alamos National Laboratory, a special purpose cluster named “tebow,” was constructed such to maximize available RAM per CPU, as well as leveraging swap space with solid state hard drives, to allow larger scale depletion calculations (allowing for significantly more burnable regions than previously examined). As the MCNP6 burnup capability was scaled to larger numbers of burnable regions, a noticeable slowdown was realized. This paper details two specific computational performance strategies for improving calculation speedup: (1) retrieving cross sections during transport; and (2) tallying mechanisms specific to burnup in MCNP. To combat this slowdown new performance upgrades were developed and integrated into MCNP6 1.2.

High-Level Strategies for Parallel Shared-Memory Sparse Matrix-Vector Multiplication

The sparse matrix-vector multiplication is an important computational kernel, but is hard to efficiently execute even in the sequential case. The problems--namely low arithmetic intensity, inefficient cache use, and limited memory bandwidth--are magnified as the core count on shared-memory parallel architectures increases. Existing techniques are discussed in detail, and categorized chiefly based on their distribution types. Based on this, new parallelization techniques are proposed. The theoretical scalability and memory usage of the various strategies are analyzed, and experiments on multiple NUMA architectures confirm the validity of the results. One of the newly proposed methods attains the best average result in experiments on a large set of matrices. In one of the experiments it obtains a parallel efficiency of 90 percent, while on average it performs close to 60 percent.

Exploiting Parallelism in Query Processing for Web Document Search Using Shared-Memory and Cluster-Based Architectures

Achieving interactive response times when searching for documents on the web has become a challenge especially with the tremendous increase in the size of information available nowadays. Incorporating parallelism in search engines is one of the approaches towards achieving this aim. In this paper, we present a model for parallel query processing. Then, this model is extended particularly for usage on shared-memory and cluster parallel architectures. A special simulator, reflecting the proposed model, was developed allowing parameters concerning the data set, queries and architectures to be varied. A total of 32 experiments were conducted and the output was studied for the effect of varying different parameters. A number of performance measures such as average response time, speedup and efficiency are computed to study the effect of varying the parameters. Results show that in terms of average response time, speedup and efficiency, the proposed model for parallel query processing on shared-memory architecture outperforms that on cluster-based architecture.

A Comparative Study and Evaluation of Parallel Programming Models for Shared-Memory Parallel Architectures

Nowadays, shared-memory parallel architectures have evolved and new programming frameworks have appeared that exploit these architectures: OpenMP, TBB, Cilk Plus, ArBB and OpenCL. This article focuses on the most extended of these frameworks in commercial and scientific areas. This paper shows a comparative study of these frameworks and an evaluation. The study covers several capacities, such as task deployment, scheduling techniques, or programming language abstractions. The evaluation measures three dimensions: code development complexity, performance and efficiency, measure as speedup per watt. For this evaluation, several parallel benchmarks have been implemented with each framework. These benchmarks are created to cover certain scenarios, like regular memory access or irregular computation. The conclusions show some highlights, like the fact that some frameworks (OpenMP, Cilk Plus) are better for transforming quickly a sequential code, others (TBB) have a small footprint which is ideal for small problems, and others (OpenCL) are suited for heterogeneous architectures but they require a very complex development process. The conclusions also show that the vectorization support is more critical than multitasking to achieve efficiency for those problems where this approach fits.

Flexible Scheduling and Thread Allocation for Synchronous Parallel Tasks

We describe a task model and dynamic scheduling and resource allocation mechanism for synchronous parallel tasks to be executed on SPMD-programmed synchronous shared-memory MIMD parallel architectures with uniform, unit-time memory access and strict memory consistency, also known in the literature as PRAMs (Parallel Random Access Machines). Our task model provides a two-tier programming model for PRAMs that flexibly combines SPMD and fork-join parallelism within the same application. It offers flexibility by dynamic scheduling and late resource binding while preserving the PRAM execution properties within each task, the only limitation being that the maximum number of threads that can be assigned to a task is limited to what the underlying architecture provides. In particular, our approach opens for automatic performance tuning at run-time by controlling the thread allocation for tasks based on run-time predictions. By a prototype implementation of a synchronous parallel task API in the SPMD-based PRAM language Fork and experimental evaluation with example programs on the SBPRAM simulator, we show that a realization of the task model on a SPMD-programmable PRAM machine is feasible with moderate runtime overhead per task.

A Low-Overhead Asynchronous Interconnection Network for GALS Chip Multiprocessors

A new asynchronous interconnection network is introduced for globally-asynchronous locally-synchronous (GALS) chip multiprocessors. The network eliminates the need for global clock distribution, and can interface multiple synchronous timing domains operating at unrelated clock rates. In particular, two new highly-concurrent asynchronous components are introduced which provide simple routing and arbitration/merge functions. Post-layout simulations in identical commercial 90 nm technology indicate that comparable recent synchronous router nodes have 5.6-10.7 more energy per packet and 2.8-6.4 greater area than the new asynchronous nodes. Under random traffic, the network provides significantly lower latency and identical throughput over the entire operating range of the 800 MHz network and through mid-range traffic rates for the 1.36 GHz network, but with degradation at higher traffic rates. Preliminary evaluations are also presented for a mixed-timing (GALS) network in a shared-memory parallel architecture, running both random traffic and parallel benchmark kernels, as well as directions for further improvement.

Using Situs for the integration of multi-resolution structures

Situs is a modular and widely used software package for the integration of biophysical data across the spatial resolution scales. It has been developed over the last decade with a focus on bridging the resolution gap between atomic structures, coarse-grained models, and volumetric data from low-resolution biophysical origins, such as electron microscopy, tomography, or small-angle scattering. Structural models can be created and refined with various flexible and rigid body docking strategies. The software consists of multiple, stand-alone programs for the format conversion, analysis, visualization, manipulation, and assembly of 3D data sets. The programs have been ported to numerous platforms in both serial and shared memory parallel architectures and can be combined in various ways for specific modeling applications. The modular design facilitates the updating of individual programs and the development of novel application workflows. This review provides an overview of the Situs package as it exists today with an emphasis on functionality and workflows supported by version 2.5.Electronic supplementary materialThe online version of this article (doi:10.1007/s12551-009-0026-3) contains supplementary material, which is available to authorized users.

The H ii Region of a Primordial Star

The concordance model of cosmology and structure formation predicts the formation of isolated very massive stars at high redshifts in dark matter dominated halos of 10{sup 5} to 10{sup 6} Msun. These stars photo-ionize their host primordial molecular clouds, expelling all the baryons from their halos. When the stars die, a relic H II region is formed within which large amounts of molecular hydrogen form which will allow the gas to cool efficiently when gravity assembles it into larger dark matter halos. The filaments surrounding the first star hosting halo are largely shielded and provide the pathway for gas to stream into the halo when the star has died. We present the first fully three dimensional cosmological radiation hydrodynamical simulations that follow all these effects. A novel adaptive ray casting technique incorporates the time dependent radiative transfer around point sources. This approach is fast enough so that radiation transport, kinetic rate equations, and hydrodynamics are solved self-consistently. It retains the time derivative of the transfer equation and is explicitly photon conserving. This method is integrated with the cosmological adaptive mesh refinement code enzo, and runs on distributed and shared memory parallel architectures. Where applicable the three dimensional calculation not onlymore » confirm expectations from earlier one dimensional results but also illustrate the multi-fold hydrodynamic complexities of H II regions. In the absence of stellar winds the circumstellar environments of the first supernovae and putative early gamma-ray bursts will be of low density {approx}1 cm{sup -3}. Albeit marginally resolved, ionization front instabilities lead to cometary and elephant trunk like small scale structures reminiscent of nearby star forming regions.« less

Implementation of a primal–dual method for SDP on a shared memory parallel architecture

Primal–dual interior point methods and the HKM method in particular have been implemented in a number of software packages for semidefinite programming. These methods have performed well in practice on small to medium sized SDPs. However, primal–dual codes have had some trouble in solving larger problems because of the storage requirements and required computational effort. In this paper we describe a parallel implementation of the primal–dual method on a shared memory system. Computational results are presented, including the solution of some large scale problems with over 50,000 constraints.

Novel multi-block strategy for CAD tools for microfluidics type applications

A novel strategy to implement multi-block schemes in Computer aided design (CAD) tools for a variety of applications specifically dealing with multi-physics transport phenomena is presented here. Multi-block schemes are domain decomposition schemes in which the domain is split into several smaller sub-domains, and independent, synchronous calculations are performed in the sub-domains individually, retaining the global identity of the problem. These schemes are very appropriate for problems with complex geometries and physics, and problems that are computationally intensive. They also allow parallelization on distributed and shared memory parallel architectures. Therefore, for complex multi-physics applications like microfluidics, which is spotlighted here, multi-block strategies can offer many advantages over traditional finite element or finite volume schemes. However, from a CAD tool implementation point of view multi-block schemes are still in infancy. One important reason for this is the different kind of geometry and mesh builders required for multi-block schemes owing to the inherently different philosophy entailed. To circumvent this, in this work we present some novel strategies to utilize the commonly used geometry builders and meshing tool of finite element technique for multi-block solvers, and develop a versatile CAD tool based on domain decomposition technique. In particular a lucid methodology is presented using which rudimentary and widely available geometry builder and meshing tool for finite element calculations can be utilized to implement a powerful multi-block scheme using structured grids and finite-volume discretization, both of which are amenable to ease of implementation and numerical efficiency for the class of problems considered. The proposed multi-block methodology can also be used with finite difference or finite element calculations in individual blocks. It will be shown that this formulation will have the potential to be the bulwark of next generation of micro- and nano-scale fluidics type problems that have applications in areas like proteomics, lab-on-a-chip design, synthetic biomaterials, etc.

A Methodological Approach to Parallel Simulated Annealing on an SMP System

Simulated annealing (SA) is a stochastic optimization technique which guarantees under certain conditions to converge to a global minimum. The major disadvantage of this technique is its very slow convergence: this makes it not suitable for many complex optimization problems. Different parallel versions of the algorithm have been proposed, but none of them addresses recent 2-way symmetric multiprocessor (SMP) machines. In this paper, we present a novel approach to the parallel implementation of SA on an SMP system. In addition, we offer an adaptive method to dynamically change the program execution flow at run time, as to obtain the maximum benefit from these shared memory parallel architectures. Since we only exploit time measures for this purpose, we obtain a problem independent and a general purpose implementation. The effectiveness of the method is demonstrated by extensively analyzing the traveling salesman problem (TSP) as a target case study, on a system under different workload conditions.

A Methodological Approach to Parallel Simulated Annealing on an SMP System

Simulated annealing (SA) is a stochastic optimization technique which guarantees under certain conditions to converge to a global minimum. The major disadvantage of this technique is its very slow convergence: this makes it not suitable for many complex optimization problems. Different parallel versions of the algorithm have been proposed, but none of them addresses recent 2-way symmetric multiprocessor (SMP) machines. In this paper, we present a novel approach to the parallel implementation of SA on an SMP system. In addition, we offer an adaptive method to dynamically change the program execution flow at run time, as to obtain the maximum benefit from these shared memory parallel architectures. Since we only exploit time measures for this purpose, we obtain a problem independent and a general purpose implementation. The effectiveness of the method is demonstrated by extensively analyzing the traveling salesman problem (TSP) as a target case study, on a system under different workload conditions.

Parallel algorithms for indefinite linear systems

Saddle-point problems give rise to indefinite linear systems that are challenging to solve via iterative methods. This paper surveys two recent techniques for solving such problems arising in computational fluid dynamics. The systems are indefinite due to linear constraints imposed on the fluid velocity. The first approach, known as the multilevel algorithm, employs a hierarchical technique to compute the constrained linear space for the unknowns, followed by the iterative solution of a positive definite reduced problem. The second approach exploits the banded structure of sparse matrices to obtain a different reduced system which is determined by the unknowns common to adjacent block rows. Although the reduced system in this approach may still be indefinite, the algorithm converges to the solution at an accelerated rate. These methods have two desirable characteristics, namely, robust numerical convergence and efficient parallelizability. The paper presents the performance of these methods for incompressible particulate flow problems on a shared-memory parallel architecture.

A hybrid approach for the modelling and simulation of a virtually shared memory parallel computer architecture

Modelling and simulation have been assigned crucial roles in the design, development, analysis and evaluation of computer architectures. The design of parallel architectures in particular is a complex and difficult endeavour that makes modelling and simulation essential tools. In this case, high simulation performance is a prerequisite since, large workloads need to be simulated for a realistic analysis of the parallel system. This paper presents the approach that has been followed for the modelling and simulation of PiSMA, a virtual shared memory parallel architecture, scaleable up to 100 s of processors.

Relaxed balance for search trees with local rebalancing

Search trees with relaxed balance were introduced with the aim of facilitating fast updating on shared-memory asynchronous parallel architectures. To obtain this, rebalancing has been uncoupled from the updating, so extensive locking in connection with updates is avoided. Rebalancing is taken care of by background processes, which do only a constant amount of work at a time before they release locks. Thus, the rebalancing and the associated locks are very localized in time as well as in space. In particular, there is no exclusive locking of whole paths. This means that the amount of parallelism possible is not limited by the height of the tree.

Array Operation Synthesis to Optimize HPF Programs on Distributed Memory Machines

An increasing number of programming languages, such as Fortran 90, HPF, and APL, provide a rich set of intrinsic array functions and array expressions. These constructs, which constitute an important part of data parallel languages, provide excellent opportunities for compiler optimizations. The synthesis of consecutive array operations or array expressions into a composite access function of the source arrays at compile time has been shown (A. T. Budd, ACM Trans. Programm. Lang. Syst.6 (July 1984), 297–313; G. H. Hwang et al., in “Proc. of ACM SIGPLAN Conference on Principles and Practice of Parallel Programming, 1995,” pp. 112–122) to be an effective scheme for optimizing programs on flat shared memory parallel architectures. It remains, however, to be studied how the synthesis scheme can be incorporated into optimizing HPF-like programs on distributed memory machines by taking into account communication costs. In this paper, we propose solutions to address this issue. We first show how to perform array operation synthesis on HPF programs, and we demonstrate its performance benefits on distributed memory machines with real applications. In addition, to prevent a situation we call “synthesis anomaly,” we present an optimal solution to guide the array synthesis process on distributed memory machines. Due to the optimal problem being NP-hard, we further develop a practical strategy that compilers can use on distributed memory machines with HPF programs. Our synthesis engine is implemented as a Web-based tool, called Syntool, and experimental results show significant performance improvement over the base codes for HPF code fragments from real appli- cations on parallel machines. Our experiments were performed on three distributed memory machines: an 8-node DEC Alpha Farm, a 16-node IBM SP-2, and a 16-node nCUBE/2.

Massively parallel software rendering for visualizing large-scale data sets

We describe two highly scalable, parallel software volume-rendering algorithms - one renders unstructured grid volume data and the other renders isosurfaces. We designed one algorithm for distributed-memory parallel architectures to render unstructured grid volume data. We designed the other for shared-memory parallel architectures to directly render isosurfaces. Through the discussion of these two algorithms, we address the most relevant issues when using massively parallel computers to render large-scale volumetric data. The focus of our discussion is direct rendering of volumetric data.