Shared Memory Architecture Research Articles

Automated dynamic detection of busy–wait synchronizations

AbstractWith the advent of multicores, multithreaded programming has acquired increased importance. In order to obtain good performance, the synchronization constructs in multithreaded programs need to be carefully implemented. These implementations can be broadly classified into two categories: busy–wait and schedule‐based. For shared memory architectures, busy–wait synchronizations are preferred over schedule‐based synchronizations because they can achieve lower wakeup latency, especially when the expected wait time is much shorter than the scheduling time. While busy–wait synchronizations can improve the performance of multithreaded programs running on multicore machines, they create a challenge in program debugging, especially in detecting and identifying the causes of data races. Although significant research has been done on data race detection, prior works rely on one important assumption—the debuggers are aware of all the synchronization operations performed during a program run. This assumption is a significant limitation as multithreaded programs, including the popular SPLASH‐2 benchmark have busy–wait synchronizations such as barriers and flag synchronizations implemented in the user code. We show that the lack of knowledge of these synchronization operations leads to unnecessary reporting of numerous races. To tackle this problem, we propose a dynamic technique for identifying user‐defined synchronizations that are performed during a program run. Both software and hardware implementations are presented. Furthermore, our technique can be easily exploited by a record/replay system to significantly speedup the replay. It can also be leveraged by a transactional memory system to effectively resolve a livelock situation. Our evaluation confirms that our synchronization detector is highly accurate with no false negatives and very few false positives. We further observe that the knowledge of synchronization operations results in 23% reduction in replay time. Finally, we show that using synchronization knowledge livelocks can be efficiently avoided during runtime monitoring of programs. Copyright © 2009 John Wiley & Sons, Ltd.

A numerical differentiation library exploiting parallel architectures

We present a software library for numerically estimating first and second order partial derivatives of a function by finite differencing. Various truncation schemes are offered resulting in corresponding formulas that are accurate to order O ( h ) , O ( h 2 ) , and O ( h 4 ) , h being the differencing step. The derivatives are calculated via forward, backward and central differences. Care has been taken that only feasible points are used in the case where bound constraints are imposed on the variables. The Hessian may be approximated either from function or from gradient values. There are three versions of the software: a sequential version, an OpenMP version for shared memory architectures and an MPI version for distributed systems (clusters). The parallel versions exploit the multiprocessing capability offered by computer clusters, as well as modern multi-core systems and due to the independent character of the derivative computation, the speedup scales almost linearly with the number of available processors/cores. Program summary Program title: NDL (Numerical Differentiation Library) Catalogue identifier: AEDG_v1_0 Program summary URL: http://cpc.cs.qub.ac.uk/summaries/AEDG_v1_0.html Program obtainable from: CPC Program Library, Queen's University, Belfast, N. Ireland Licensing provisions: Standard CPC licence, http://cpc.cs.qub.ac.uk/licence/licence.html No. of lines in distributed program, including test data, etc.: 73 030 No. of bytes in distributed program, including test data, etc.: 630 876 Distribution format: tar.gz Programming language: ANSI FORTRAN-77, ANSI C, MPI, OPENMP Computer: Distributed systems (clusters), shared memory systems Operating system: Linux, Solaris Has the code been vectorised or parallelized?: Yes RAM: The library uses O ( N ) internal storage, N being the dimension of the problem Classification: 4.9, 4.14, 6.5 Nature of problem: The numerical estimation of derivatives at several accuracy levels is a common requirement in many computational tasks, such as optimization, solution of nonlinear systems, etc. The parallel implementation that exploits systems with multiple CPUs is very important for large scale and computationally expensive problems. Solution method: Finite differencing is used with carefully chosen step that minimizes the sum of the truncation and round-off errors. The parallel versions employ both OpenMP and MPI libraries. Restrictions: The library uses only double precision arithmetic. Unusual features: The software takes into account bound constraints, in the sense that only feasible points are used to evaluate the derivatives, and given the level of the desired accuracy, the proper formula is automatically employed. Running time: Running time depends on the function's complexity. The test run took 15 ms for the serial distribution, 0.6 s for the OpenMP and 4.2 s for the MPI parallel distribution on 2 processors.

A parallel line segment intersection strategy based on uniform grids

The line segment intersection problem is one of the basic problems in computational geometry and has been widely used in spatial analysis in Geographic Information Systems (GIS). Lots of traditional algorithms study the problem in a serial environment. However, in GIS, a spatial object is much more complicated and is considered to be always composed of multiple line segments, and one line segment connects another line segment at its endpoint. On the other hand, along with the advances made in computer hardware, more and more personal computers have multiple cores or CPUs equipped. Thus, to make full use of the increasing computing resources, parallel technique is applied as one of the most available methods. Apparently, the traditional algorithms should be improved to take advantage of the technologies. Under these circumstances, based on the modified uniform grid algorithm, which is adapted to dealing with spatial objects in GIS, this paper proposes a parallel strategy in a shared memory architecture. Also, experimental results are given in the final part of this paper to demonstrate the efficiency this strategy brings.

Architecture and Physical Implementation of a Third Generation 65 nm, 16 Core, 32 Thread Chip-Multithreading SPARC Processor

This third-generation Chip-Multithreading (CMT) SPARC processor consists of 16 cores with shared memory architecture and supports a total of 32 main threads plus 32 scout threads. It is targeted for high-performance servers, and is optimized for both single- and multi-threaded applications. The 396 mm2 chip is fabricated in an 11 metal layer 65-nm CMOS process and operates at a nominal frequency of 2.3 GHz, consuming a maximum power of 250 W at 1.2 V. This paper provides an overview of the architectural highlights and describes the physical implementation challenges and solutions including circuit innovations in memory arrays, register files, and floating-point hardware that boost the performance and circuit robustness with low area overhead.

Low-cost reconfigurable VLSI architecture for fast fourier transform

In this paper, a low-cost reconfigurable FFT processor employing novel dual-path pipelined sharedmemory architecture is presented. Based on this architecture, an elaborate memory configuration scheme is designed to make single-port SRAM available. Moreover, a mixed-radix butterfly unit is also designed, which makes the processor capable of multimode operation. Compared with previous ones, the proposed architecture can greatly reduce area. In addition, an optimized data scaling approach is proposed and the Signal-to-Quantization Noise Ratio (SQNR) of an 8Kpoint fixed-point FFT can achieve 52.7dB with the wordlength of 13bit. A test chip for DVB-T/H is implemented with the proposed architecture and fabricated in 0.18-μm single-poly six-metal CMOS process. The core area of this chip is 2.83mm2 with the power dissipation of 25.8mW at 20MHz.

Parallel implementations of streamline simulators

We discuss various strategies for parallelizing streamline simulators and present a single-phase shared memory implementation. The choice of a shared memory programming model is motivated by its suitability for streamline simulation, as well as the rapid advance of multicore processors, which are readily available at low-cost. We show that streamline-based methods are easily parallelizable on shared memory architectures through their decomposition of the multidimensional transport equations into a large set of independent 1D transport solves. We tested both a specialized explicit load balancing algorithm that optimizes the streamline load distribution across threads to minimize the time that any of the threads are idle, and the dynamic load balancing algorithms provided by OpenMP on the shared memory machines. Our results clearly indicate that built-in schedulers are competitive with specialized load balancing strategies as long as the number of streamlines per thread is sufficiently high, which is the case in field applications. The average workload per thread is nominally insensitive to workload variations between individual streamlines, and any load balancing advantage offered by explicit strategies is not sufficient to overcome associated computational and parallel overhead. In terms of the allocation of streamlines or streamline segments to threads, we investigated both the distributed approach, in which threads are assigned streamline segments, and the owner approach, in which threads own complete streamlines. We found that the owner approach is most suitable. The slight advantage that the distributed approach has in terms of load balancing is not enough to compensate for the additional overheads. Moreover, the owner approach allows straightforward re-use of existing sequential codes, which is not the case for the distributed approach in case of implicit or adaptive implicit solution strategies. The tracing and mapping stages in streamline simulation have low parallel efficiency. However, in real-field models, the computational burden of the streamline solves is significantly heavier than that of the tracing and mapping stages, and therefore, the impact of these stages is limited. We tested the parallelization on three shared memory systems: a 24 dual-core processor Sun SPARC server; an eight-way Sun Opteron server, representative of the state-of-the-art shared memory systems in use in the industry; and the very recently released Sun Niagara II multicore machine that has eight floating point compute units on the chip. We test a single-phase flow problem on three heterogeneous reservoirs with varying well placements (this system gives the worst case scenario as the tracing and mapping costs are not negligible compared to the transport costs). For the SPARC and Opteron system, we find parallel efficiencies ranging between 60 and 75 for the tracer flow problems. The sublinear speedup is mostly due to communication overheads in the tracing and mapping stages. In applications with more complex physics, the relative contributions of these stages will decrease significantly, and we predict the parallel performance to be nearly linear. On the Niagara II, we obtain almost perfect linear scalability even for the single-phase flow problem thanks to the lowered communication costs on these architectures that have a shared cache. This result is all the more satisfactory considering that future server designs will be akin to this system.

Branch-and-Bound interval global optimization on shared memory multiprocessors

The focus of this paper is on the analysis and evaluation of a type of parallel strategies applied to the algorithm Advanced Multidimensional Interval analysis Global Optimization (AMIGO). We investigate two parallel versions of AMIGO, called Parallel AMIGO (PAMIGO) algorithm, Global-PAMIGO and Local-PAMIGO. The idea behind our study is that in order to exploit the potential parallelism of algorithms, researchers need to adapt them to the target computer architectures. Our PAMIGO algorithms have been designed for shared memory architectures and are based on a threaded programming model, which is suitable to be run on current personal computers with multicore processors. Our first experimental results show a promising speed-up up to four process units. We analyse the loss of efficiency when the number of process units is greater than four by obtaining a profile of the algorithm executions. Secondly we experiment with the use of a local memory allocator per thread. This increases the efficiency by reducing the number of lock conflicts given by the standard system memory allocator. Our experimental results for both PAMIGO versions, using up to 15 process units, obtain a good performance for hard to solve problems on unicore and multicore processors. It is noteworthy that both versions of PAMIGO obtain a similar performance. Our experiments may be useful for researchers who use parallel B&B algorithms.

A comparative evaluation of hybrid distributed shared-memory systems

Distributed Shared-Memory (DSM) systems are shared-memory multiprocessor architectures in which each processor node contains a partition of the shared memory. In hybrid DSM systems coherence among caches is maintained by a software-implemented coherence protocol relying on some hardware support. Hardware support is provided to satisfy every node hit (the common case) and software is invoked only for accesses to remote nodes. In this paper we compare the design and performance of four hybrid distributed shared memory (DSM) organizations by detailed simulation of the same hardware platform. We have implemented the software protocol handlers for the four architectures. The handlers are written in C and assembly code. Coherence transactions are executed in trap and interrupt handlers. Together with the application, the handlers are executed in full detail in execution-driven simulations of six complete benchmarks with coarse-grain and fine-grain sharing. We relate our experience implementing and simulating the software protocols for the four architectures. Because the overhead of remote accesses is very high in hybrid systems, the system of choice is different than for purely hardware systems.

Efficiency estimate for distributed computation of Gröbner bases and involutive bases

Several years ago, we presented a program complex for parallel computation of Grobner bases that works on computers with shared-memory architecture. Unfortunately, the number of the processors that we can use is small (from 2 to 16) because of hardware constraints. This paper presents a program for distributed computation of bases that relies on the same principles but works in a network consisting of heterogeneous machines. The effectiveness of such an approach is estimated from the standpoint of the processor capacity usage and the required network bandwidth, and methods optimizing usage of these resources are specified.

Code and Data Structure Partitioning for Parallel and Flexible MPSoC Specification Using Designer-Controlled Recoding

MultiProcessor systems-on-chip (MPSoCs) are increasingly being used to build efficient and cost-effective embedded systems that meet the necessary real-time requirements. However, programming heterogeneous MPSoCs is highly challenging. The existing automatic parallelizing techniques, although effective on homogeneous shared-memory architectures, are insufficient for MPSoCs, which are typically characterized by heterogeneous processing elements and memory architectures. The lack of effective automatic techniques for recoding and parallelization requires designers to manually partition the code and the data structures in the reference application to generate a parallel and flexible specification model. Such manual algorithm partitioning by the designer is time consuming and error prone. In this paper, we motivate the need for automation in system specification and present a novel designer-controlled approach to recode applications written in a C-based System-Level Description Language. We present six automated source code transformations that, under the control of the designer, automatically partition and reorganize code and data structures to create a parallel and flexible abstract specification model that can be mapped onto a heterogeneous MPSoC using a top-down system-level design flow. Our experimental results show significant productivity gains and quality improvements in the end design.

A parallel implementation of the Wang–Landau algorithm

The Wang–Landau algorithm is a flat-histogram Monte Carlo method that performs random walks in the configuration space of a system to obtain a close estimation of the density of states iteratively. It has been applied successfully to many research fields. In this paper, we propose a parallel implementation of the Wang–Landau algorithm on computers of shared memory architectures by utilizing the OpenMP API for distributed computing. This implementation is applied to Ising model systems with promising speedups. We also examine the effects on the running speed when using different strategies in accessing the shared memory space during the updating procedure. The allowance of data race is recommended in consideration of the simulation efficiency. Such treatment does not affect the accuracy of the final density of states obtained.

Measuring and Evaluating Parallel State-Space Exploration Algorithms

We argue in this paper that benchmarking should be complemented by direct measurement of parallelisation overheads when evaluating parallel state-space exploration algorithms. This poses several challenges that so far have not been addressed in the literature: what exactly are those overheads, how can and cannot they be measured, and how should system models be selected in order to expose the causes of parallelisation (in)efficiencies? We discuss and answer these questions based on our experience with parallelising Saturation – a symbolic algorithm for generating state-spaces of asynchronous system models – on a shared-memory architecture. Doing so will hopefully spare newcomers to the growing PDMC community from having to learn these lessons the hard way, as we did over a painful period of almost three years.

Latency Impact on Spin-Lock Algorithms for Modern Shared Memory Multiprocessors

In 2006, John Mellor-Crummey and Michael Scott received the Dijkstra Prize in Distributed Computing. This prize was for their 1991 paper on algorithms for scalable synchronization on shared memory multiprocessors, which included a novel spin-lock algorithm (a.k.a. MCS spin-lock). Their spin-lock algorithm distributes spin locations in memory to lessen the impact of bandwidth limitations. Their empirical work and architectural suggestions have since had a major impact on how the field has viewed spin-locks. Motivated by emerging architectures with an increasing number of cores, we present an empirical study on recent shared memory architectures, including IBM P5+ and SGI ccNUMA systems. Our results show that latency will have a much greater impact on performance than bandwidth on these and future architectures. Several testcases and a tabular overview of our results are included.

INTRA- AND INTER-ITERATION 3D OSEM PET IMAGE RECONSTRUCTION

Most of the recent medical imaging modalities use noninvasive means to obtain the activity information inside human organs, so doctors may detect the initial symptoms of a disease as early as possible and give appropriate treatment. PET is to use radio-isotopes which can emit positrons in its clinical and research uses. By injecting the nuclear medicine drug formed by labeling a radioelement to molecules of deoxidized glucose to a patient and after the cells in the patient body absorbing it through metabolic functions, the detectors will receive annihilation coincidence events formed from the number of the positrons generated from the response of the labeled deoxidized glucose molecules. The most critical module of the modality therefore is the procedure regarding how to reconstruct good quality images using the collected projection information. However, in the reconstruction process, MLEM involves massive data in considerable number of iterations in order to yield accurate images and takes quite a long time in computation. Ordered Subsets Expectation Maximization (OSEM) was proposed to accelerate the reconstruction process by expediting the convergence while maintaining the same image quality as those produced by MLEM. Since then, OSEM iterative algorithm has become the de facto reconstruction method adopted by most PET installations. To further improve the image quality, both clinical and research data have been acquired in 3D mode on the majority of the current systems. The accompanied computational load of iterative reconstruction increases considerably resulting from the 3D OSEM method. Attributed to the fact of the recent technological advancement, many high-performance parallel methods have been proposed to speed up the reconstruction process. These methods in general are to partition data into several sets before applying any parallel acceleration. They do not take on the nature of OSEM method to identify the intrinsic data dependencies. This project intends to analyze the iterative natures of the 3D OSEM method, particularly the intra- and inter-iteration aspects of the reconstruction method along with the latest shared-memory parallel machine architecture. Experiments will be conducted to demonstrate its superior performance over the existing methods.

Parallel algorithms for large applied problems: Issues and technologies

Topical issues of parallelizing computational methods and technologies for implementing the main mathematical modeling stages in solving a wide range of large applied problems are considered. Computation-information process models, classification of the main types of algorithms, and principles of their efficient mapping onto the distributed-and shared-memory computer architecture are described.

Parallel protein secondary structure prediction schemes using Pthread and OpenMP over hyper-threading technology

Protein secondary structure prediction has a fundamental influence on today's bioinformatics research. In this work, tertiary classifiers for the protein secondary structure prediction are implemented on Denoeux Belief Neural Network (DBNN) architecture. Hydrophobicity matrix, orthogonal matrix, BLOSUM62 matrix and PSSM matrix are experimented separately as the encoding schemes for DBNN. Hydrophobicity matrix, BLOSUM62 matrix and PSSM matrix are applied to DBNN architecture for the first time. The experimental results contribute to the design of new encoding schemes. Our accuracy of the tertiary classifier with PSSM encoding scheme reaches 72.01%, which is almost 10% better than the previous results obtained in 2003. Due to the time consuming task of training the neural networks, Pthread and OpenMP are employed to parallelize DBNN in the Hyper-Threading enabled Intel architecture. Speedup for 16 Pthreads is 4.9 and speedup for 16 OpenMP threads is 4 in the 4 processors shared memory architecture. Both speedup performance of OpenMP and Pthread is superior to that of other research. With the new parallel training algorithm, thousands of amino acids can be processed in reasonable amount of time. Our research also shows that Hyper-Threading technology for Intel architecture is efficient for parallel biological algorithms.

Exploration of distributed shared memory architectures for NoC-based multiprocessors

Multiprocessor system-on-chip (MP-SoC) platforms represent an emerging trend for embedded multimedia applications. To enable MP-SoC platforms, scalable communication-centric interconnect fabrics, such as networks-on-chip (NoCs), have been recently proposed. The shared memory represents one of the key elements in designing MP-SoCs to provide data exchange and synchronization support. This paper focuses on the energy/delay exploration of a distributed shared memory architecture, suitable for low-power on-chip multiprocessors based on NoC. A mechanism is proposed for the data allocation on the distributed shared memory space, dynamically managed by an on-chip hardware memory management unit (HwMMU). Moreover, the exploitation of the HwMMU primitives for the migration, replication, and compaction of shared data is discussed. Experimental results show the impact of different distributed shared memory configurations for a selected set of parallel benchmark applications from the power/-performance perspective. Furthermore, a case study for a graph exploration algorithm is discussed, accounting for the effects of the core mapping and the network topology on energy and performance at the system level.

MovieMaker: a parallel movie-making software for large-scale simulations

We have developed a parallel rendering software for scientific visualization of large-scale, three-dimensional, time development simulations. The goal of this software, MovieMaker, is to generate a movie, or a series of visualization images from totally one TB-scale data within one night (or less than 12 hours). The isocontouring, volume rendering, and streamlines are implemented. MovieMaker is a parallel program for the shared memory architecture with dynamic load balancing and overlapped disk I/O.

ScalaBLAST: A Scalable Implementation of BLAST for High-Performance Data-Intensive Bioinformatics Analysis

Genes in an organism's DNA (genome) have embedded in them information about proteins, which are the molecules that do most of a cell's work. A typical bacterial genome contains on the order of 5,000 genes. Mammalian genomes can contain tens of thousands of genes. For each genome sequenced, the challenge is to identify protein components (proteome) being actively used for a given set of conditions. Fundamentally, sequence alignment is a sequence matching problem focused on unlocking protein information embedded in the genetic code, making it possible to assemble a tree of life by comparing new sequences against all sequences from known organisms. But, the memory footprint of sequence data is growing more rapidly than per-node core memory. Despite years of research and development, high-performance sequence alignment applications either do not scale well, cannot accommodate very large databases in core, or require special hardware. We have developed a high-performance sequence alignment application, ScalaBLAST, which accommodates very large databases and which scales linearly to as many as thousands of processors on both distributed memory and shared memory architectures, representing a substantial improvement over the current state-of-the-art in high-performance sequence alignment with scaling and portability. ScalaBLAST relies on a collection of techniques - distributing the target database over available memory, multilevel parallelism to exploit concurrency, parallel I/O, and latency hiding through data prefetching - to achieve high-performance and scalability. This demonstrated approach of database sharing combined with effective task scheduling should have broad ranging applications to other informatics-driven sciences

SAC—A Functional Array Language for Efficient Multi-threaded Execution

We give an in-depth introduction to the design of our functional array programming language SAC, the main aspects of its compilation into host machine code, and its parallelisation based on multi-threading. The language design of SAC aims at combining high-level, compositional array programming with fully automatic resource management for highly productive code development and maintenance. We outline the compilation process that maps SAC programs to computing machinery. Here, our focus is on optimisation techniques that aim at restructuring entire applications from nested compositions of general fine-grained operations into specialised coarse-grained operations. We present our implicit parallelisation technology for shared memory architectures based on multi-threading and discuss further optimisation opportunities on this level of code generation. Both optimisation and parallelisation rigorously exploit the absence of side-effects and the explicit data flow characteristic of a functional setting.