Virtual Shared Memory Research Articles

Porting industrial codes and developing sparse linear solvers on parallel computers

We address the main issues when porting existing codes from serial to parallel computers and when developing portable parallel software on MIMD multiprocessors (shared memory, virtual shared memory, and distributed memory multiprocessors, and networks of computers). We discuss the use of numerical libraries as a way of developing portable and efficient parallel code. We illustrate this by using examples from our experience in porting industrial codes and in designing parallel numerical libraries. We report in some detail on the parallelization of scientific applications coming from Centre National d'Etudes Spatiales and from Aérospatiale, and we illustrate how it is possible to develop portable and efficient numerical software by considering the parallel solution of sparse linear systems of equations.

Virtual shared memory programming on workstation clusters

Workstation clusters have recently attracted high interest as a technology providing supercomputer class performance at much lower price levels. Today the message passing programming model dominates the application development, despite the overhead and the complexity introduced by the explicitly coded data transfers. We give an introduction to the virtual shared memory programming model and report on the experiences with the MNFS system. We show that shared memory with weak coherency can perform competitive to message passing and provide an excellent tool for parallelizing programs using complex data structures with dynamic load balancing.

BLOCK PRECONDITIONED CONJUGATE GRADIENT METHODS ON A DISTRIBUTED VIRTUAL SHARED MEMORY MULTIPROCESSOR

We study both shared and distributed approaches for the parallel implementation of the SSOR and Jacobi block preconditioned Krylov methods on a distributed virtual shared memory computer: a BBN TC2000. We consider the solution of block tridiagonal systems arising from the discretization of 3D partial differential equations, which diagonal blocks correspond to the discretization of 2D partial differential equations. The solution of the diagonal subproblems required for the preconditionings are performed using a domain decomposition method with overlapped subdomains: a variant of the Schwarz alternating method.

Solving the Helmholtz Equation At High-Wave Numbers On a Parallel Computer With a Shared Virtual Memory

In this paper, we address the implementation of finite element codes on a parallel computer providing a shared virtual memory. We solve the Helmholtz equa tion at high-wave numbers by a method combining the least-squares formulation and finite-element dis cretization. Numerical experiments concerning the scattering of planar waves by nonconvex obstacles show the efficiency of the new algorithm. Parallelism relies heavily on shared variables and parallel loops; performances are enhanced by considering local ver sus global nodes and by renumbering the mesh. Ex periments are implemented on a KSR computer with up to 20 processors.

Comparing distributed memory and virtual shared memory parallel programming models

The virtues of the shared memory and distributed memory parallel programming models have been much debated. Conventionally the debate could be reduced to programming convenience on the one hand, and high scalability factors on the other. More recently the debate has become somewhat blurred with the provision of virtual shared memory models built on machines with physically distributed memory. The intension of such models/machines is to provide scalable shared memory, i.e. to provide both programmer convenience and high scalability. In this paper, the different models are considered from experiences gained with a number of systems ranging from applications in both commerce and science to languages and operating systems. Case studies are introduced as appropriate.

LINEAR ALGEBRA CALCULATIONS ON A VIRTUAL SHARED MEMORY COMPUTER

We evaluate the impact of the memory hierarchy of virtual shared memory computers on the design of algorithms for linear algebra. On classical shared memory multiprocessor computers, block algorithms are used for efficiency. We study here the potential and the limitations of such approaches on globally addressable distributed memory computers. The BBN TC2000 belongs to this class of computers and will be used to illustrate our discussion.We describe the implementation of Level 3 BLAS and examine the performance of some of the LAPACK routines. The impact of the number of processors with respect to the choice of the variants of classical matrix factorizations (for example, KJI, JKI, JIK for the LU factorization) is discussed. We also study the factorization of sparse matrices based on a multifrontal approach. The ideas introduced for the parallelization of full linear algebra codes are applied to the sparse case. We discuss and illustrate the limitations of this approach in sparse multifrontal factorization.We show that the speed-ups obtained on the BBN TC2000 for the class of methods presented here are comparable to those obtained on more classical shared memory computers, such as the Alliant FX/80, the CRAY-2 and the IBM 3090/VF.

Unstructured CFD computations on the KSR-1: Preliminary results

We report on our preliminary results for unstructured CFD computations on the KSR-1 parallel processor. While this machine features a virtual shared memory environment, it has been our experience that high performance results for unstructured computations are attainable only when using a message passing paradigm.

VIRTUAL SHARED MEMORY MACHINES— AN APPLICATION OF PVM∗

We show how a shared memory architecture can be simulated using computers of different architectures. A description of a parallel Prolog (ROLOG) implementation for virtual shared memory machines is included. This paper reveals some general techniques and guidelines for (re)writing software for these machines.

Overview of the KOAN programming environment for the iPSC/2 and performance evaluation of the BECAUSE test program 2.51

In this paper, we describe a Fortran programming environment using the KOAN Shared Virtual Memory on a distributed memory parallel architecture. We then discuss its use for parallelizing a Because benchmark application and present performance results. Initial findings show that it is possible to obtain good performance without the need for low level parallel programming.

Parallel supercomputer architectures and their programming models

The paper presents a taxonomy of the existing forms of parallel computer architectures, based on the characteristics of the hardware architecture and the abstract machine layered upon it. The abstract machine reflects the programming models provided. The main classes of hardware architectures are: physically shared memory systems and distributed memory systems. Distributed memory systems may be remote memory access architectures or message passing architectures. The major forms of abstract machine architecture are: message passing systems and logically shared memory architectures. Three solutions for logically shared memory architectures are known (1) distributed shared memory architectures, (2) multi-threaded architectures, and (3) virtual shared memory architectures, All three types are discussed in detail under the aspects of performance, programmability, and scalability, and their corresponding programming paradigms are characterized. The implications of the three concepts on node architecture and the requirements of latency minimization or latency hiding are discussed and illustrated by examples taken from pioneering realizations of the three kinds of architecture such as DASH, ∗T, and MANNA.

Adaptive memory coherence algorithms in DSVM

Based on the characteristics of distrubuted system and the behavior of parallel programs, this paper presents the fixed and randomized competitive memory coherence algorithms for distributed shared virtual memory. These algorithms exploit parallel programs' locality of reference and exhibit good competitive property. Our simulation shows that the fixed and randomized algorithms achieve better performance and higher stability than other strategies such as write-invalidate and write-update.

Virtual Shared Memory: Algorithms and Complexity

We consider the Block PRAM model of Aggarwal et al. ( in "Proceedings, First Annual ACM Symposium on Parallel Algorithms and Architectures, 1989," pp. 11-21). For a Block PRAM model with n /log n processors and communication latency l = O (log n ), we show that prefix sums can be performed in time O ( l log n /log l ), but list ranking requires time Ω( l log n ); these bounds are tight. These results justify an intuitive observation of Gazit et al. ( in "Proceedings, 1987 Princeton Workshop on Algorithm, Architecture and Technology Issues for Models of Concurrent Computation," pp. 139-156) that algorithm designers should, when possible, replace the list ranking procedure with the prefix sums procedure. We demonstrate the value of this technique in choosing between two optimal PRAM algorithms for finding the connected components of dense graphs. We also give theoretical improvements for integer sorting and many other algorithms based on prefix sums, and suggest a relationship between the issue of graph density for the connected components problem and alternative approaches to integer sorting.

Sequential consistency versus linearizability

The power of two well-known consistency conditions for shared-memory multiprocessors,sequential consistencyandlinearizability, is compared. The cost measure studied is the worst-case response time in distributed implementations of virtual shared memory supporting one of the two conditions. Three types of shared-memory objects are considered: read/write objects, FIFO queues, and stacks. If clocks are only approximately synchronized (or do not exist), then for all three object types it is shown that linearizability is more expensive than sequential consistency. We show that, for all three data types, the worst-case response time is very sensitive to the assumptions that are made about the timing information available to the system. Under the strong assumption that processes have perfectly synchronized clocks, it is shown that sequential consistency and linearizability are equally costly. We present upper bounds for linearizability and matching lower bounds for sequential consistency. The upper bounds are shown by presenting algorithms that use atomic broadcast in a modular fashion. The lower-bound proofs for the approximate case use the technique of “shifting,” first introduced for studying the clock synchronization problem.

The Linda alternative to message-passing systems

The use of distributed data structures in a logically-shared memory is a natural, readily-understood approach to parallel programming. The principal argument against such an approach for portable software has always been that efficient implementations could not scale to massively-parallel, distributed memory machines. Now, however, there is growing evidence that it is possible to develop efficient and portable implementations of virtual shared memory models on scalable architectures. In this paper we discuss one particular example: Linda. After presenting an introduction to the Linda model, we focus on the expressiveness of the model, on techniques required to build efficient implementations, and on observed performance both on workstation networks and distributed-memory parallel machines. Finally, we conclude by briefly discussing the range of applications developed with Linda and Linda's suitability for the sorts of heterogeneous, dynamically-changing computational environments that are of growing significance.

The Efficiency of Linda for General Purpose Scientific Programming

Linda (Linda is a registered trademark of Scientific Computing Associates, Inc.) is a programming language for coordinating the execution and interaction of processes. When combined with a language for computation (such as C or Fortran), the resulting hybrid language can be used to write portable programs for parallel and distributed multiple instruction multiple data (MIMD) computers. The Linda programming model is based on operations that read, write, and erase a virtual shared memory. It is easy to use, and lets the programmer code in a very expressive, uncoupled programming style. These benefits, however, are of little value unless Linda programs execute efficiently. The goal of this article is to demonstrate that Linda programs are efficient making Linda an effective general purpose tool for programming MIMD parallel computers. Two arguments for Linda′s efficiency are given; the first is based on Linda′s implementation and the second on a range of case studies spanning a complete set of parallel algorithm classes.

Applications of Telos

EuLisp has an integrated object system with reflective capabilities. We discuss some example applications which use these facilities to experiment with some advanced and powerful concepts, namely, finalization, virtual shared memory and persistence. A secondary goal is to attempt to illustrate the additional possibilities of metaobject programming over non-metalevel techniques.

Using shared virtual memory for parallel join processing

In this paper, we show that shared virtual memory, in a shared-nothing multiprocessor, facilitates the design and implementation of parallel join processing algorithms that perform significantly better in the presence of skew than previously proposed parallel join processing algorithms. We propose two variants of an algorithm for parallel join processing using shared virtual memory, and perform a detailed simulation to investigate their performance. The algorithm is unique in that it employs both the shared virtual memory paradigm and the message-passing paradigm used by current shared-nothing parallel database systems. The implementation of the algorithm requires few modifications to existing shared-nothing parallel database systems.

The detection and elimination of useless misses in multiprocessors

In this paper we introduce a new classification of misses in shared-memory multiprocessors based on interprocessor communication. We identify the set of essential misses, i.e., the smallest set of misses necessary for correct execution. Essential misses include cold misses and true sharing misses. All other misses are useless misses and can be ignored without affecting the correctness of program execution. Based on the new classification we compare the effectiveness of five different protocols which delay and combine invalidations leading to useless misses. In cache-based systems the protocols are very effective and have miss rates close to the essential miss rate. In virtual shared memory systems the techniques are also effective but leave room for improvements.

Parallelising a financial system

This paper reports on the parallelising of a financial system that calculates actuarial liabilities. The purpose of such a system is to evaluate the liabilities of a pension scheme by calculating the expected cost of each member to the system in the future. An analysis of the system has been carried out and two levels of parallelism have been detected. From the more coarse-grained of these levels a parallel model has been proposed. A parallel simulation of this model has been developed both on a Meiko Computing Surface distributed store machine populated with Inmos transputers and Intel i860 microprocessors, and on a network of workstations to enable detailed evaluation of the potential speedup. Figures are given and various metrics are used to analyse the speedup and efficiency factors. The potential for making use of virtual shared memory and lightweight thread packages at this coarse-grained level of parallelism is considered. The more fine-grained level of parallelism identified is discussed with a view to massively parallel architectures.

Library for architecture-independent development of structured grid applications

P++ is an innovative user environment that simplifies the development of efficient, parallel programs for large scale scientific applications. It permits portability across the widest variety of computer architectures. The target machines are distributed memory computers with different types of node architectures (scalar, vector, or superscalar), but the requirements of shared memory computers are also addressed. Unfortunately, robust, architecture-independent and efficient software environments have been notable to date primarily by their absence. Such a simplifying environment for the development of software is sorely needed to take advantage of the current and future developments in advanced computational hardware. The P++ environment does this using a standard language, C++, with absolutely no modification of the compiler. For parallel communication it employs existing widely portable communications libraries. Such an environment allows existing C++ language compilers to be used to develop software in the preferred serial environment, and such software to be efficiently run, unchanged, in all target environments.The explicit goal of P++ is the support of advanced computational methods for the solution of large-scale computational problems. For efficiency reasons and also for simplification, the use of P++ is restricted to the large class of structured grid methods for partial differential equations. Applications using P++ can selectively exploit the added degree of freedom presented by parallel processing by use of an optimization module within the array language interface. A collection of defaults ensures deterministic behavior. In this way, complicated optimization issues of parallel execution may be easily introduced by the setting of switches within the user interface, which are then interpreted at runtime. Through the use of Virtual Shared Memory, restricted to array variables (Virtual Shared Grids), issues such as partitioning become matters of optimization, and not criteria for correct execution. Due to the restriction and optimization for structured grids, we expect the same parallel performance as for codes based on the traditionally used explicit Message Passing model. To speed the development of the P++ environment, we use an efficient, commercially available array language library, M++, developed in C++; the M++ array interface is also used as the P++ array interface.Recognizing that the acceptance of advanced parallel computers depends on their use by scientists, and not parallel researchers, it is hoped that P++ will simplify access to these parallel machines. While significantly simplifying our own research on parallel self adaptive local refinement methods, it is hoped that P++ will more generally simplify the introduction of structured grid methods for large scale scientific computing onto advanced parallel machines.