Shared Memory Implementation Research Articles

Application of Pfortran and Co-Array Fortran in the Parallelization of the GROMOS96 Molecular Dynamics Module

After at least a decade of parallel tool development, parallelization of scientific applications remains a significant undertaking. Typically parallelization is a specialized activity supported only partially by the programming tool set, with the programmer involved with parallel issues in addition to sequential ones. The details of concern range from algorithm design down to low-level data movement details. The aim of parallel programming tools is to automate the latter without sacrificing performance and portability, allowing the programmer to focus on algorithm specification and development. We present our use of two similar parallelization tools, Pfortran and Cray's Co-Array Fortran, in the parallelization of the GROMOS96 molecular dynamics module. Our parallelization started from the GROMOS96 distribution's shared-memory implementation of the replicated algorithm, but used little of that existing parallel structure. Consequently, our parallelization was close to starting with the sequential version. We found the intuitive extensions to Pfortran and Co-Array Fortran helpful in the rapid parallelization of the project. We present performance figures for both the Pfortran and Co-Array Fortran parallelizations showing linear speedup within the range expected by these parallelization methods.

Parallel algorithms to solve two-stage stochastic linear programs with robustness constraints

In this paper we present a parallel method for solving two-stage stochastic linear programs with restricted recourse. The mathematical model considered here can be used to represent several real-world applications, including financial and production planning problems, for which significant changes in the recourse solutions should be avoided because of their difficulty to be implemented. Our parallel method is based on a primal-dual path-following interior point algorithm, and exploits fruitfully the dual block-angular structure of the constraint matrix and the special block structure of the matrices involved in the restricted recourse model. We describe and discuss both message-passing and shared-memory implementations and we present the numerical results collected on the Origin2000.

Parallelization of a dynamic unstructured algorithm using three leading programming paradigms

The success of parallel computing in solving real-life computationally intensive problems relies on their efficient mapping and execution on large-scale multiprocessor architectures. Many important applications are both unstructured and dynamic in nature, making their efficient parallel implementation a daunting task. This paper presents the parallelization of a dynamic unstructured mesh adaptation algorithm using three popular programming paradigms on three leading supercomputers. We examine an MPI message-passing implementation on the Cray T3E and the SGI Origin2000, a shared-memory implementation using the cache coherent nonuniform memory access (CC-NUMA) feature of the Origin2000, and a multithreaded version on the newly released Tera Multithreaded Architecture (MTA). We compare several critical factors of this parallel code development, including runtime, scalability, programmability, portability, and memory overhead. Our overall results demonstrate that multithreaded systems offer tremendous potential for quickly and efficiently solving some of the most challenging real-life problems on parallel computers.

Constructive, Deterministic Implementation of Shared Memory on Meshes

This paper describes a scheme to implement a shared address space of size m on an n-node mesh, with m polynomial in n, where each mesh node hosts a processor and a memory module. At the core of the simulation is a hierarchical memory organization scheme (HMOS), which governs the distribution of the shared variables, each replicated into multiple copies, among the memory modules, through a cascade of bipartite graphs. Based on the expansion properties of such graphs, we devise a protocol that accesses any n-tuple of shared variables in worst-case time $O(n^{1/2+\eta})$, for any constant $\eta > 0$, using $O(1/\eta^{1.59})$ copies per variable, or in worst-case time O(n1/2 log n), using O(log1.59n ) copies per variable. In both cases the access time is close to the natural $O(\sqrt{n})$ lower bound imposed by the network diameter. A key feature of the scheme is that it can be made fully constructive when m is not too large, thus providing in this case the first efficient, constructive, deterministic scheme in the literature for bounded-degree processor networks. For larger memory sizes, the scheme relies solely on a nonconstructive graph of weak expansion. Finally, the scheme can be efficiently ported to other architectures, as long as they exhibit certain structural properties. In the paper we discuss the porting to multidimensional meshes and to the pruned butterfly, an area-universal network which is a variant of the fat-tree.

Linearizable read/write objects

We study the cost of using message passing to implement linearizable read/write objects for shared-memory multiprocessors under various assumptions on the available timing information. We take as cost measures the worst-case response times for performing read and write operations in distributed implementations of virtual shared memory consisting of such objects, and the sum of these response times. It is assumed that processes have clocks that run at the same rate as real time and are within δ of each other, for some known precision constant δ ⩾ 0. All messages incur a delay in the range [ d− u, d] for some known constants u and d, 0 ⩽ u ⩽ d. For the perfect clocks model, where clocks are perfectly synchronized, i.e., δ = 0, and every message incurs a delay of exactly d, we present a linearizable implementation which achieves worst-case response times for read and write operations of βd and (1 − β) d, respectively; β is a trade-off parameter, 0 ⩽ β ⩽ 1, which may be tuned to account for the relative frequencies of read and write operations. This implementation is optimal with respect to the sum of the worst-case response times for read and write operations. We next turn to the approximately synchronized clocks model, where clocks are only approximately synchronized, i.e., β > 0, and message delays can vary, i.e., u > 0. Our first major result is the first known linearizable implementation for this model which achieves worst-case response times of less than βd + 3 u + min { δ, u} + ε, and ( l − β) d + 3 u for read and write operations, respectively, under a mild restriction on the trade-off parameter β, 0 ⩽ β < 1− u d ; ε is any arbitrary constant such that 0 ⩽ ε ⩽ min {2 u, d − u}. This implementation employs a novel use of approximately synchronized clocks in order to utilize the lower bound on message delay time and achieve bounds on worst-case response times that depend on the message delay uncertainty u. For a wide range of values of u, these bounds improve upon previously known ones for implementations that supports consistency conditions even weaker than linearizability. Our next major result is a lower bound of d + min {δ, u} 2 on the sum of the worst-case response times for read and write operations, for the approximately synchronized clocks model. This bound applies to linearizable implementations possessing some natural symmetry properties; the bound is shown using the technique of “shifting” executions. Corresponding lower bounds, but with no symmetry assumptions, are shown on the individual worst-case response times for read and write operations. Our bounds for the approximately synchronized clocks model extend naturally to the imperfect clocks model, where clocks may be arbitrarily far from each other, i.e., δ = ∞.

Combining compile-time and run-time support for efficient software distributed shared memory

We describe an integrated compile time and run time system for efficient shared memory parallel computing on distributed memory machines. The combined system presents the user with a shared memory programming model. The run time system implements a consistent shared memory abstraction using memory access detection and automatic data caching. The compiler improves the efficiency of the shared memory implementation by directing the run time system to exploit the message passing capabilities of the underlying hardware. To do so, the compiler analyzes shared memory accesses and transforms the code to insert calls to the run time system that provide it with the access information computed by the compiler. The run time system is augmented with the appropriate entry points to use this information to implement bulk data transfer and to reduce the overhead of run time consistency maintenance. In those cases where the compiler analysis succeeds for the entire program, we demonstrate that the combined system achieves performance comparable to that produced by compilers that directly target message passing. If the compiler analysis is successful only for parts of the program, for instance, because of irregular accesses to some of the arrays, the resulting optimizations can be applied to those parts for which the analysis succeeds. If the compiler analysis fails entirely, we rely on the run time maintenance of shared memory and thereby avoid the complexity and the limitations of compilers that directly target message passing. The result is a single system that combines efficient support for both regular and irregular memory access patterns.

SCALABILITY OF SPARSE CHOLESKY FACTORIZATION

A variety of algorithms have been proposed for sparse Cholesky factorization, including left-looking, right-looking, and supernodal algorithms. This article investigates shared-memory implementations of several variants of these algorithms in a task-oriented execution model with dynamic scheduling. In particular, we consider the degree of parallelism, the scalability, and the scheduling overhead of the different algorithms. Our emphasis lies in the parallel implementation for relatively large numbers of processors. As execution platform, we use the SB-PRAM, a shared-memory machine with up to 2048 processors. This article can be considered as a case study in which we try to answer the question of which performance we can hope to get for a typical irregular application on an ideal machine on which the locality of memory accesses can be ignored but for which the overhead for the management of data structures still takes effect. The investigation shows that certain algorithms are the best choice for a small number of processors, while other algorithms are better for many processors.

Execution behavior analysis and performance prediction for a shared-memory implementation of an irregular particle simulation method

Many computational-intensive problems from science and engineering are irregular in nature. This makes it difficult to develop an efficient parallel implementation, even for shared-memory machines. As a typical example, we investigate a parallel implementation of an irregular particle simulation algorithm. We concentrate on the issue which programming and system support is needed to yield an efficient implementation for a large number of processors. As an execution platform we use the SB-PRAM, a shared memory machine with up to 2048 processors. The processors of the SB-PRAM can access the global memory in unit time which is the basis for an exact performance prediction. Common approaches for parallel implementations like lock protection for concurrent accesses and sequential or distributed task queues are replaced by more efficient access mechanisms and data structures which can be realized by the powerful multiprefix operations of the SB-PRAM. Their use simplifies the implementation and yields large speedup values.

PQE2000: HPC tools for industrial applications

It is time for high-quality products to emerge for HPC systems, but achieving this goal requires a new way of transferring results from the research environment to the application marketplace. As the product of three Italian research agencies and one industrial company: the National Research Council (CNR), National Agency for New Technologies, Energy, and Environment (ENEA), National Institute for Nuclear Physics (NFN), and Finmeccanica's Quadrics Supercomputers World Ltd. (QSW), the PQE2000 project adopts this viewpoint with respect to innovative software technologies and composite MPP architectures for petaflops computing. The author discusses the PQE2000 software technology, shared memory implementations and the PQE2000 architecture for MPP computations.

Shared memory implementation of a parallel switch-level circuit simulator

article Free Access Share on Shared memory implementation of a parallel switch-level circuit simulator Authors: Yu-an Chen Computer Science Department, University of California at Los Angeles, Los Angeles, CA Computer Science Department, University of California at Los Angeles, Los Angeles, CAView Profile , Rajive Bagrodia Computer Science Department, University of California at Los Angeles, Los Angeles, CA Computer Science Department, University of California at Los Angeles, Los Angeles, CAView Profile Authors Info & Claims ACM SIGSIM Simulation DigestVolume 28Issue 1July 1998 pp 134–141https://doi.org/10.1145/278009.278025Online:01 July 1998Publication History 6citation92DownloadsMetricsTotal Citations6Total Downloads92Last 12 Months5Last 6 weeks1 Get Citation AlertsNew Citation Alert added!This alert has been successfully added and will be sent to:You will be notified whenever a record that you have chosen has been cited.To manage your alert preferences, click on the button below.Manage my Alerts New Citation Alert!Please log in to your account Save to BinderSave to BinderCreate a New BinderNameCancelCreateExport CitationPublisher SiteeReaderPDF

A shared-memory implementation of the hierarchical radiosity method

The radiosity method is a simulation method from computer graphics to visualize the global illumination in scenes containing diffuse objects within an enclosure. A variety of realizations (including parallel approaches) were proposed to achieve a high efficiency while guaranteeing the same accuracy of the graphical representation. The hierarchical radiosity method reduces the computational costs considerably but results in a highly irregular algorithm which makes a parallel implementation more difficult. We investigate a task-oriented shared memory implementation and present optimizations with different behavior concerning locality and granularity. To be able to concentrate on load balancing and scalability issues, we use a shared-memory machine with uniform memory access time, the SB-PRAM.

Dynamic resolution: A runtime technique for the parallelization of modifications to directed acyclic graphs

Static program analysis limits the performance improvements possible from compile-time parallelization.Dynamic parallelization shifts a portion of the analysis from compile-time to runtime, thereby enabling optimizations whose static detection is overly expensive or impossible.Dynamic resolution is a dynamic-parallelization technique for finding loop and nonloop parallelism in imperative, sequential programs that destructively manipulate dynamic directed acyclic graphs (DAGs). Dynamic resolution uses runtime reference counts on heap data, a runtime linearization of threads, and a simple static analysis to dynamically detect potential heap aliases and to correctly coordinate parallel access to shared structures. We describe dynamic resolution in the context of two imperative procedures: DAG rewrite and destructive quicksort. The description is couched in the pointer-safe language ML; with some programmer assertions and custom macros for pointer and memory manipulation, dynamic resolution is applicable to pointer-unsafe languages (C extended with threads) as well. Furthermore, with programmer identification of cyclic structure, dynamic resolution can be used to find parallelism in programs that manipulate cyclic structures. Shared-memory implementations of dynamic resolution for ML and C have attained parallel speedup for nontrivial sequential procedures such as destructive quicksort; empirical speedup results obtained on fast contemporary machines are given.

Relaxed consistency and coherence granularity in DSM systems

During the past few years, two main approaches have been taken to improve the performance of software shared memory implementations: relaxing consistency models and providing fine-grained access control. Their performance tradeoffs, however, we not well understood. This paper studies these tradeoffs on a platform that provides access control in hardware but runs coherence protocols in software, We compare the performance of three protocols across four coherence granularities, using 12 applications on a 16-node cluster of workstations. Our results show that no single combination of protocol and granularity performs best for all the applications. The combination of a sequentially consistent (SC) protocol and fine granularity works well with 7 of the 12 applications. The combination of a multiple-writer, home-based lazy release consistency (HLRC) protocol and page granularity works well with 8 out of the 12 applications. For applications that suffer performance losses in moving to coarser granularity under sequential consistency, the performance can usually be regained quite effectively using relaxed protocols, particularly HLRC. We also find that the HLRC protocol performs substantially better than a single-writer lazy release consistent (SW-LRC) protocol at coase granularity for many irregular applications. For our applications and platform, when we use the original versions of the applications ported directly from hardware-coherent shared memory, we find that the SC protocol with 256-byte granularity performs best on average. However, when the best versions of the applications are compared, the balance shifts in favor of HLRC at page granularity.

Parallel computation for microwave circuit simulation

In this paper, the results of implementing a harmonic balance simulator, AGILE, on a variety of massively parallel computers (MPCs) is given. Descriptions of the computer hardware, which includes both shared-memory and message-passing implementations, and algorithms used to parallelize the computations are presented. The computers used include the CM-5, KSR-1, and a networked set of nonheterogeneous workstations using UNIX RPC. A key aspect is the description of the variety of algorithms used for each computer and the results obtained.

The IFS model: A parallel production weather code

The integrated Forecasting System (IFS) of the European Centre for Medium-range Weather Forecasts (ECMWF) is a spectral weather forecasting model, which daily produces weather forecasts on up to 16 processors of a CRAY C90. This paper describes the shared-memory implementation of the code and the subsequent development that has been carried out in order to generate a parallel version, suitable for a scalable distributed-memory architecture with many processors. Performance results presented for several vector and parallel systems indicate that the parallelization effort has been successful in achieving good performance and high efficiency.

Parallel sparse QR factorization on shared memory architectures

We discuss a parallel shared memory implementation of multifrontal QR factorization. To achieve high performance for general large and sparse matrices, a combination of tree and node level parallelism is used. Acceptable load balancing is obtained by the use of a pool-of-tasks approach. For the storage of frontal and update matrices, we use a buddy system based on Fibonacci blocks. It turns out to be more efficient than blocks of size 2 i , as proposed by other authors. Also the order in which memory space for update and frontal matrices are allocated is shown to be of importance. An implementation of the proposed algorithm on the CRAY X-MP/416 (four processors), gives speedups of about three with about 20% of extra real memory space required.

Optical parallel-access shared memory system: analysis and experimental demonstration

An optical implementation of a parallel-access shared memory uses a single photorefractive crystal and can realize the set of memory modules in a digital shared memory computer. This implementation uses two arrays of sources that are individually coherent but mutually incoherent from region to region across each array, and it permits incoherent/coherent double angular multiplexing of data in the crystal. A complete instruction set for its memory access consists of four operations, READ, WRITE, SELECTIVE ERASE, and REFRESH, which can be applied to any memory module independent of (and parallel with) instructions to the other memory modules. In addition, a memory module can execute a sequence of READ operations simultaneously with the execution of a WRITE operation to accommodate differences efficiently in optical recording and readout times common to optical volume storage media. An experimental shared memory system demonstrates two memory modules, each consisting of up to two 5-bit data blocks, implemented in a single Fe:LiNbO(3) crystal. The projected performance of the optical parallel-access shared memory system is analyzed and compared with conventional page-addressed volume holographic memories.

Sequential consistency versus linearizability

The power of two well-known consistency conditions for shared-memory multiprocessors,sequential consistencyandlinearizability, is compared. The cost measure studied is the worst-case response time in distributed implementations of virtual shared memory supporting one of the two conditions. Three types of shared-memory objects are considered: read/write objects, FIFO queues, and stacks. If clocks are only approximately synchronized (or do not exist), then for all three object types it is shown that linearizability is more expensive than sequential consistency. We show that, for all three data types, the worst-case response time is very sensitive to the assumptions that are made about the timing information available to the system. Under the strong assumption that processes have perfectly synchronized clocks, it is shown that sequential consistency and linearizability are equally costly. We present upper bounds for linearizability and matching lower bounds for sequential consistency. The upper bounds are shown by presenting algorithms that use atomic broadcast in a modular fashion. The lower-bound proofs for the approximate case use the technique of “shifting,” first introduced for studying the clock synchronization problem.

Software versus hardware shared-memory implementation

We compare the performance of software-supported shared memory on a general-purpose network to hardware-supported shared memory on a dedicated interconnect.Up to eight processors, our results are based on the execution of a set of application programs on a SGI 4D/480 multiprocessor and on TreadMarks, a distributed shared memory system that runs on a Fore ATM LAN of DECstation-5000/240s. Since the DECstation and the 4D/480 use the same processor, primary cache, and compiler, the shared-memory implementation is the principal difference between the systems. Our results show that TreadMarks performs comparably to the 4D/480 for applications with moderate amounts of synchronization, but the difference in performance grows as the synchronization frequency increases. For applications that require a large amount of memory bandwidth, TreadMarks can perform better than the SGI 4D/480.Beyond eight processors, our results are based on execution-driven simulation. Specifically, we compare a software implementation on a general-purpose network of uniprocessor nodes, a hardware implementation using a directory-based protocol on a dedicated interconnect, and a combined implementation using software to provide shared memory between multiprocessor nodes with hardware implementing shared memory within a node. For the modest size of the problems that we can simulate, the hardware implementation scales well and the software implementation scales poorly. The combined approach delivers performance close to that of the hardware implementation for applications with small to moderate synchronization rates and good locality. Reductions in communication overhead improve the performance of the software and the combined approach, but synchronization remains a bottleneck.

A comparative study of PVM workstation cluster implementations of a two-phase subsurface flow model

Four versions of a message-passing, distributed-memory, distributed-disk multicomputer application for simulating two-phase fluid flow in porous media are studied. The FORTRAN 77 finite element code, solved in parallel using an iterative domain-decomposition method, was ported from a shared-memory multiprocessor implementation to a network of heterogeneous computers using the PVM (Parallel Virtual Machine) software system. Issues concerning the calculation of speedups in cluster computing are discussed, and the performance of the four versions is compared. Parallel efficiencies of up to nearly 60% were achieved using 17 homogeneous networked workstations; speedups were comparable to the shared-memory implementation. We conclude that network multicomputing using PVM is an attractive alternative to hardware multiprocessors for large-grained parallel implementations of computationally intensive problems in hydrology.