Shared Memory Machines Research Articles

Novel modifications of parallel Jacobi algorithms

We describe two main classes of one-sided trigonometric and hyperbolic Jacobi-type algorithms for computing eigenvalues and eigenvectors of Hermitian matrices. These types of algorithms exhibit significant advantages over many other eigenvalue algorithms. If the matrices permit, both types of algorithms compute the eigenvalues and eigenvectors with high relative accuracy. We present novel parallelization techniques for both trigonometric and hyperbolic classes of algorithms, as well as some new ideas on how pivoting in each cycle of the algorithm can improve the speed of the parallel one-sided algorithms. These parallelization approaches are applicable to both distributed-memory and shared-memory machines. The numerical testing performed indicates that the hyperbolic algorithms may be superior to the trigonometric ones, although, in theory, the latter seem more natural.

Rebound

As we move to large manycores, the hardware-based global check-pointing schemes that have been proposed for small shared-memory machines do not scale. Scalability barriers include global operations, work lost to global rollback, and inefficiencies in imbalanced or I/O-intensive loads. Scalable checkpointing requires tracking inter-thread dependences and building the checkpoint and rollback operations around dynamic groups of communicating processors. To address this problem, this paper introduces Rebound , the first hardware-based scheme for coordinated local checkpointing in multiprocessors with directory based cache coherence. Rebound leverages the transactions of a directory protocol to track inter-thread dependences. In addition, it boosts checkpointing efficiency by: (i) delaying the writeback of data to safe memory at checkpoints, (ii) supporting operation with multiple checkpoints, and (iii) optimizing checkpointing at barrier synchronization. Finally, Rebound introduces distributed algorithms for checkpointing and rollback sets of processors. Simulations of parallel programs with up to 64 threads show that Rebound is scalable and has very low overhead. For 64 processors, its average performance overhead is only 2%, compared to 15% for global checkpointing.

Vascular System Modeling in Parallel Environment - Distributed and Shared Memory Approaches

This paper presents two approaches in parallel modeling of vascular system development in internal organs. In the first approach, new parts of tissue are distributed among processors and each processor is responsible for perfusing its assigned parts of tissue to all vascular trees. Communication between processors is accomplished by passing messages, and therefore, this algorithm is perfectly suited for distributed memory architectures. The second approach is designed for shared memory machines. It parallelizes the perfusion process during which individual processing units perform calculations concerning different vascular trees. The experimental results, performed on a computing cluster and multicore machines, show that both algorithms provide a significant speedup.

Mapping iterative medical imaging algorithm on cell accelerator.

Algebraic reconstruction techniques require about half the number of projections as that of Fourier backprojection methods, which makes these methods safer in terms of required radiation dose. Algebraic reconstruction technique (ART) and its variant OS-SART (ordered subset simultaneous ART) are techniques that provide faster convergence with comparatively good image quality. However, the prohibitively long processing time of these techniques prevents their adoption in commercial CT machines. Parallel computing is one solution to this problem. With the advent of heterogeneous multicore architectures that exploit data parallel applications, medical imaging algorithms such as OS-SART can be studied to produce increased performance. In this paper, we map OS-SART on cell broadband engine (Cell BE). We effectively use the architectural features of Cell BE to provide an efficient mapping. The Cell BE consists of one powerPC processor element (PPE) and eight SIMD coprocessors known as synergetic processor elements (SPEs). The limited memory storage on each of the SPEs makes the mapping challenging. Therefore, we present optimization techniques to efficiently map the algorithm on the Cell BE for improved performance over CPU version. We compare the performance of our proposed algorithm on Cell BE to that of Sun Fire ×4600, a shared memory machine. The Cell BE is five times faster than AMD Opteron dual-core processor. The speedup of the algorithm on Cell BE increases with the increase in the number of SPEs. We also experiment with various parameters, such as number of subsets, number of processing elements, and number of DMA transfers between main memory and local memory, that impact the performance of the algorithm.

Parallel Computation in Medical Imaging Applications

Parallel Computation in Medical Imaging Applications

Workshare Process of Thread Programming and MPI Model on Multicore Architecture

Comparison between OpenMP for thread programming model and MPI for message passing programming model will be conducted on multicore shared memory machine architectures in order to find which has a better performance in terms of speed and throughput. Application used to assess the scalability of the evaluated parallel programming solutions is matrix multiplication with customizable matrix dimension. Many research done on a large scale parallel computing which using high scale benchmark such as NSA Parallel Benchmark (NPB) for their testing standardization [2]. This research will be conducted on a small scale parallel computing that emphasize more on the performance evaluation between MPI and OpenMP parallel programming model using self created benchmark. It also describes how workshare processes done on different parallel programming model. It gives comparative result between message passing and shared memory programming model in runtime and amount of throughput. Testing methodology also simple and has high usability on the available resources.

Some useful strategies for unstructured edge‐based solvers on shared memory machines

AbstractThree strategies for shared memory parallel edge‐based solvers are proposed which guarantee that nodes belonging to one thread are not accessed by other threads for vertex‐centered discretizations (replace nodes by cells in case of cell‐centered discretizations). The algorithms reorder the edges in groups in order for the parallelization to take place at the edge level, possibly through multiple passes, which constitutes the bulk of the work in an edge‐based solver. These strategies are presented in an increasing order of programming effort and their performances are also compared. Various renumbering algorithms are considered. Results and timings are given for a classical Computational Fluid Dynamics compressible edge‐based solver and a Numerical Weather Prediction compressible dynamic solver for dry air, as well as computational details to illustrate the efficiency of the proposed approach. The influence of the point renumbering on the final edge grouping and efficiency is also studied through numerical results. Copyright © 2010 John Wiley & Sons, Ltd.

Multicore Parallelization of Min-Cost Flow for CAD Applications

Computational complexity has been the primary challenge of many very large scale integration computer-aided design (CAD) applications. The emerging multicore and many-core microprocessors have the potential to offer scalable performance improvements. How to explore the multicore resources to speed up CAD applications is thus a natural question but also a huge challenge for CAD researchers. This paper proposes a methodology to explore concurrency via nondeterministic transactional models, and to program them on multicore processors for CAD applications. Various run-time scheduling implementations on multicore shared-memory machines are discussed and the most efficient one is identified. The proposed methodology is applied to the min-cost flow problem which has been identified as the key problem in many design optimizations, from wire-length optimization in detailed placement to timing-constrained voltage assignment. A concurrent algorithm for min-cost flow has been developed based on the methodology. Experiments on voltage island generation in floorplanning have demonstrated its efficiency and scalable speedup over different numbers of cores.

Comparison of MPI Benchmark Programs on Shared Memory and Distributed Memory Machines (Point-to-Point Communication)

There are several benchmark programs available to measure the performance of MPI on parallel computers. The most common use of MPI benchmarks software are SKaMPI, Pallas MPI Benchmark, MPBench, Mpptest and MPIBench. It is interesting to analyze the differences between different benchmark. Presently, there have been few comparisons done between the different benchmarks. Thus, in this paper we discuss a comparison of the techniques used and the functionality of each benchmark, and also a comparison of the results on a distributed memory machine and shared memory machine for point-to-point communication. All of the MPI benchmarks listed above will be compared in this analysis. It is expected that the results from different benchmarks should be similar, however this analysis found substantial differences in the results for certain MPI communications, particularly for shared memory machines.

KRASH

In this article we present KRASH, a tool for reproducible generation of system-level CPU load. This tool is intended for use in shared memory machines equipped with multiple CPU cores which are usually exploited concurrently by several users. The objective of KRASH is to enable parallel application developers to validate their resources use strategies on a partially loaded machine by replaying an observed load in concurrence with their application. To reach this objective, we present a method for CPU load generation which behaves as realistically as possible: the resulting load is similar to the load that would be produced by concurrent processes run by other users. Nevertheless, contrary to a simple run of a CPU-intensive application, KRASH is not sensitive to system scheduling decisions. The main benefit brought by KRASH is this reproducibility: no matter how many processes are present in the system the load generated by our tool strictly respects a given load profile. To our knowledge, KRASH is the only tool that implements the generation of a dynamic load profile (a load varying with time). When used to generate a constant load, KRASH result is among the most realistic ones. Furthermore, KRASH provides more flexibility than other tools.

A Parallel Harmonic-Balance Approach to Steady-State and Envelope-Following Simulation of Driven and Autonomous Circuits

In this paper, we present a parallel harmonic-balance approach, applicable to the steady-state and envelope-following analyses of both driven and autonomous circuits. Our approach is centered on a naturally parallelizable preconditioning technique that speeds up the core computation in harmonic-balance-based analysis. As a coarse-grained parallel approach by algorithm construction, the proposed method facilitates parallel computing via the use of domain knowledge and simplifies parallel programming compared with fine-grained strategies. The proposed parallel preconditioning technique can be combined with more conventional parallel approaches such as parallel device model evaluation, parallel fast Fourier transform operation, and parallel matrix-vector product to further improve runtime efficiency. In our message-passing-interface-based implementation over a cluster of workstations and multithreading-based implementation on a shared-memory machine, favorable runtime speedups with respect to the conventional serial approaches and the serial implementations of the same parallel algorithms are achieved.

ALTDSE: An Arnoldi–Lanczos program to solve the time-dependent Schrödinger equation

We describe a general ab initio and non-perturbative method to solve the time-dependent Schrödinger equation (TDSE) for the interaction of a strong attosecond laser pulse with a general atom. While the field-free Hamiltonian and the dipole matrices may be generated using an arbitrary primitive basis, they are assumed to have been transformed to the eigenbasis of the problem before the solution of the TDSE is propagated in time using the Arnoldi–Lanczos method. Probabilities for survival of the ground state, excitation, and single ionization can be extracted from the propagated wavefunction. Program summary Program title: ALTDSE Catalogue identifier: AEDM_v1_0 Program summary URL: http://cpc.cs.qub.ac.uk/summaries/AEDM_v1_0.html Program obtainable from: CPC Program Library, Queen's University, Belfast, N. Ireland Licensing provisions: Standard CPC licence, http://cpc.cs.qub.ac.uk/licence/licence.html No. of lines in distributed program, including test data, etc.: 2154 No. of bytes in distributed program, including test data, etc.: 30 827 Distribution format: tar.gz Programming language: Fortran 95. [A Fortran 2003 call to “flush” is used to simplify monitoring the output file during execution. If this function is not available, these statements should be commented out.]. Computer: Shared-memory machines Operating system: Linux, OpenMP Has the code been vectorized or parallelized?: Yes RAM: Several Gb, depending on matrix size and number of processors Supplementary material: To facilitate the execution of the program, Hamiltonian field-free and dipole matrix files are provided. Classification: 2.5 External routines: LAPACK, BLAS Nature of problem: We describe a computer program for a general ab initio and non-perturbative method to solve the time-dependent Schrödinger equation (TDSE) for the interaction of a strong attosecond laser pulse with a general atom [1,2]. The probabilities for survival of the initial state, excitation of discrete states, and single ionization due to multi-photon processes can be obtained. Solution method: The solution of the TDSE is propagated in time using the Arnoldi–Lanczos method. The field-free Hamiltonian and the dipole matrices, originally generated in an arbitrary basis (e.g., the flexible B-spline R-matrix (BSR) method with non-orthogonal orbitals [3]), must be provided in the eigenbasis of the problem as input. Restrictions: The present program is restricted to a 1S e initial state and linearly polarized light. This is the most common situation experimentally, but a generalization is straightforward. Running time: Several hours, depending on the number of threads used.

Simulating macroscale brain circuits with microscale resolution

Event Abstract Back to Event Simulating macroscale brain circuits with microscale resolution Susanne Kunkel1*, Tobias C. Potjans2, 3, Abigail Morrison4 and Markus Diesmann1, 3, 4 1 Bernstein Center for Computational Neuroscience, Germany 2 Research Center Jülich, Germany 3 RIKEN Computational Science Research Program, Japan 4 RIKEN Brain Science Institute, Japan Thanks to distributed computing techniques, today it is possible to routinely simulate local cortical networks of around 105 neurons with up to 109 synapses on clusters and multiple processor shared memory machines. Simulations of this type carried out with NEST [1] scale well up to at least 1000 processors. However, simulations of microscale networks corresponding to approximately 1mm3 of the cortex are limited in their explanatory power. To understand the functions of the brain, we need to simulate macroscale circuits involving multiple interacting areas [2]. One approach is to develop brain-scale networks (see Figure) in which the individual nodes are realized by microcircuits at the resolution of point neurons and synapses such as the layered local cortical network model we recently developed [3]. These networks will be one or two orders of magnitude larger than the previously studied models not only in terms of numbers of neurons and synapse but also in terms of computational load.This presents a number of challenges to current simulation technology. Firstly, as the number of processors increases, the memory overhead due to serial data structures eventually dominates the total memory usage and so limits the parallelization. An example of this is the usage of proxies representing remote neurons on each machine and providing an interface to the local neurons. Although such representations of remote neurons require much less memory than local neurons, their proportion of the total neuronal memory usage approaches one as the number of machines increases. Secondly, the size of the maximal synaptic delay determines the size of the spike buffers each neuron uses to queue incoming spikes [4]. For local networks these buffers are small, as the synaptic delays are in the order of milliseconds. If interacting brain areas are to be considered, the synaptic delays increase by an order of magnitude, which entails a corresponding increase of memory usage for the spike buffers. It is also important that a distinction can be made between the axonal and the dendritic components of synaptic delays, as this affects the behavior of plasticity models dependent on relative spike timing [5,6]. We quantify the effects of these memory limitations in our application up to the order of 10k processors and present strategies for addressing these problems.Next-Generation Supercomputer Project of MEXT, EU Grant 15879 (FACETS), BMBF Grant 01GQ0420, Helmholtz Alliance on Systems Biology INCF-09-46

Parallel implementations of streamline simulators

We discuss various strategies for parallelizing streamline simulators and present a single-phase shared memory implementation. The choice of a shared memory programming model is motivated by its suitability for streamline simulation, as well as the rapid advance of multicore processors, which are readily available at low-cost. We show that streamline-based methods are easily parallelizable on shared memory architectures through their decomposition of the multidimensional transport equations into a large set of independent 1D transport solves. We tested both a specialized explicit load balancing algorithm that optimizes the streamline load distribution across threads to minimize the time that any of the threads are idle, and the dynamic load balancing algorithms provided by OpenMP on the shared memory machines. Our results clearly indicate that built-in schedulers are competitive with specialized load balancing strategies as long as the number of streamlines per thread is sufficiently high, which is the case in field applications. The average workload per thread is nominally insensitive to workload variations between individual streamlines, and any load balancing advantage offered by explicit strategies is not sufficient to overcome associated computational and parallel overhead. In terms of the allocation of streamlines or streamline segments to threads, we investigated both the distributed approach, in which threads are assigned streamline segments, and the owner approach, in which threads own complete streamlines. We found that the owner approach is most suitable. The slight advantage that the distributed approach has in terms of load balancing is not enough to compensate for the additional overheads. Moreover, the owner approach allows straightforward re-use of existing sequential codes, which is not the case for the distributed approach in case of implicit or adaptive implicit solution strategies. The tracing and mapping stages in streamline simulation have low parallel efficiency. However, in real-field models, the computational burden of the streamline solves is significantly heavier than that of the tracing and mapping stages, and therefore, the impact of these stages is limited. We tested the parallelization on three shared memory systems: a 24 dual-core processor Sun SPARC server; an eight-way Sun Opteron server, representative of the state-of-the-art shared memory systems in use in the industry; and the very recently released Sun Niagara II multicore machine that has eight floating point compute units on the chip. We test a single-phase flow problem on three heterogeneous reservoirs with varying well placements (this system gives the worst case scenario as the tracing and mapping costs are not negligible compared to the transport costs). For the SPARC and Opteron system, we find parallel efficiencies ranging between 60 and 75 for the tracer flow problems. The sublinear speedup is mostly due to communication overheads in the tracing and mapping stages. In applications with more complex physics, the relative contributions of these stages will decrease significantly, and we predict the parallel performance to be nearly linear. On the Niagara II, we obtain almost perfect linear scalability even for the single-phase flow problem thanks to the lowered communication costs on these architectures that have a shared cache. This result is all the more satisfactory considering that future server designs will be akin to this system.

A non-hybrid method for the PDF equations of turbulent flows on unstructured grids

In probability density function (PDF) methods of turbulent flows, the joint PDF of several flow variables is computed by numerically integrating a system of stochastic differential equations for Lagrangian particles. A set of parallel algorithms is proposed to provide an efficient solution of the PDF transport equation modeling the joint PDF of turbulent velocity, frequency and concentration of a passive scalar in geometrically complex configurations. In the vicinity of walls the flow is resolved by an elliptic relaxation technique down to the viscous sublayer, explicitly modeling the high anisotropy and inhomogeneity of the low-Reynolds-number wall region without damping or wall-functions. An unstructured Eulerian grid is employed to extract Eulerian statistics, to solve for quantities represented at fixed locations of the domain (i.e., the mean pressure and the elliptic relaxation tensor) and to track particles. All three aspects regarding the grid make use of the finite element method employing the simplest linear shapefunctions. To model the small-scale mixing of the transported scalar, the interaction by exchange with the conditional mean (IECM) model is adopted. An adaptive algorithm to compute the velocity-conditioned scalar mean is proposed that homogenizes the statistical error over the sample space with no assumption on the shape of the underlying velocity PDF. Compared to other hybrid particle-in-cell approaches for the PDF equations, the current methodology is consistent without the need for consistency conditions. The algorithm is tested by computing the dispersion of passive scalars released from concentrated sources in two different turbulent flows: the fully developed turbulent channel flow and a street canyon (or cavity) flow. Algorithmic details on estimating conditional and unconditional statistics, particle tracking and particle-number control are presented in detail. Relevant aspects of performance and parallelism on cache-based shared memory machines are discussed.

Parallel unsymmetric-pattern multifrontal sparse LU with column preordering

We present a new parallel sparse LU factorization algorithm and code. The algorithm uses a column-preordering partial-pivoting unsymmetric-pattern multifrontal approach. Our baseline sequential algorithm is based on UMFPACK 4, but is somewhat simpler and is often somewhat faster than UMFPACK version 4.0. Our parallel algorithm is designed for shared-memory machines with a small or moderate number of processors (we tested it on up to 32 processors). We experimentally compare our algorithm with SuperLU_MT, an existing shared-memory sparse LU factorization with partial pivoting. SuperLU_MT scales better than our new algorithm, but our algorithm is more reliable and is usually faster. More specifically, on matrices that are costly to factor, our algorithm is usually faster on up to 4 processors, and is usually faster on 8 and 16. We were not able to run SuperLU_MT on 32. The main contribution of this article is showing that the column-preordering partial-pivoting unsymmetric-pattern multifrontal approach, developed as a sequential algorithm by Davis in several recent versions of UMFPACK, can be effectively parallelized.

PARALLELIZATION OF THE Α‐STABLE MODELLING ALGORITHMS

Stable distributions have a wide sphere of application: probability theory, physics, electronics, economics, sociology. Particularly important role they play in financial mathematics, since the classical models of financial market, which are based on the hypothesis of the normality, often become inadequate. However, the practical implementation of stable models is a nontrivial task, because the probability density functions of α‐stable distributions have no analytical representations (with a few exceptions). In this work we exploit the parallel computing technologies for acceleration of numerical solution of stable modelling problems. Specifically, we are solving the stable law parameters estimation problem by the maximum likelihood method. If we need to deal with a big number of long financial series, only the means of parallel technologies can allow us to get results in a adequate time. We have distinguished and defined several hierarchical levels of parallelism. We show that coarse‐grained Multi‐Sets parallelization is very efficient on computer clusters. Fine‐grained Maximum Likelihood level is very efficient on shared memory machines with Symmetric multiprocessing and Hyper‐threading technologies. Hybrid application, which is utilizing both of those levels, has shown superior performance compared to single level (MS) parallel application on cluster of Pentium 4 HT nodes.

JaMP: an implementation of OpenMP for a Java DSM

Although OpenMP is a widely agreed-upon standard for the C/C++ and Fortran programming languages for the semi-automatic parallelization of programs for shared memory machines, not much has been don...

Parallel object monitors

AbstractCoordination of parallel activities on a shared memory machine is a crucial issue for modern software, even more with the advent of multi‐core processors. Unfortunately, traditional concurrency abstractions force programmers to tangle the application logic with the synchronization concern, thereby compromising understandability and reuse, and fall short when fine‐grained and expressive strategies are needed. This paper presents a new concurrency abstraction called POM, parallel object monitor, supporting expressive means for coordination of parallel activities over one or more objects, while allowing a clean separation of the coordination concern from application code. Expressive and reusable strategies for concurrency control can be designed, thanks to a full access to the queue of pending requests, parallel execution of dispatched requests together with after‐actions, and complete control over reentrancy. A small domain‐specific aspect language is provided to adequately configure pre‐packaged, off‐the‐shelf synchronizations. Copyright © 2007 John Wiley & Sons, Ltd.

INTRA- AND INTER-ITERATION 3D OSEM PET IMAGE RECONSTRUCTION

Most of the recent medical imaging modalities use noninvasive means to obtain the activity information inside human organs, so doctors may detect the initial symptoms of a disease as early as possible and give appropriate treatment. PET is to use radio-isotopes which can emit positrons in its clinical and research uses. By injecting the nuclear medicine drug formed by labeling a radioelement to molecules of deoxidized glucose to a patient and after the cells in the patient body absorbing it through metabolic functions, the detectors will receive annihilation coincidence events formed from the number of the positrons generated from the response of the labeled deoxidized glucose molecules. The most critical module of the modality therefore is the procedure regarding how to reconstruct good quality images using the collected projection information. However, in the reconstruction process, MLEM involves massive data in considerable number of iterations in order to yield accurate images and takes quite a long time in computation. Ordered Subsets Expectation Maximization (OSEM) was proposed to accelerate the reconstruction process by expediting the convergence while maintaining the same image quality as those produced by MLEM. Since then, OSEM iterative algorithm has become the de facto reconstruction method adopted by most PET installations. To further improve the image quality, both clinical and research data have been acquired in 3D mode on the majority of the current systems. The accompanied computational load of iterative reconstruction increases considerably resulting from the 3D OSEM method. Attributed to the fact of the recent technological advancement, many high-performance parallel methods have been proposed to speed up the reconstruction process. These methods in general are to partition data into several sets before applying any parallel acceleration. They do not take on the nature of OSEM method to identify the intrinsic data dependencies. This project intends to analyze the iterative natures of the 3D OSEM method, particularly the intra- and inter-iteration aspects of the reconstruction method along with the latest shared-memory parallel machine architecture. Experiments will be conducted to demonstrate its superior performance over the existing methods.