Sequent Symmetry Research Articles

Machine independent AND and OR parallel execution of logic programs. II. Compiled execution

For pt.I. see ibid., p. 170-80. In pt.I, we presented a binding environment for the AND and OR parallel execution of logic programs. This environment was instrumental in rendering a compiler for the AND and OR parallel execution of logic programs machine independent. In this paper, we describe a compiler based on the Reduce-OR process model (ROPM) for the parallel execution of Prolog programs, and provide performance of the compiler on five parallel machines: the Encore Multimax, the Sequent Symmetry, the NCUBE 2, the Intel i860 hypercube and a network of Sun workstations. The compiler is part of a machine independent parallel Prolog development system built on top of a run time environment for parallel programming called the Chare kernel, and runs unchanged on these multiprocessors. In keeping with the objectives behind the ROPM, the compiler supports both on and independent AND parallelism in Prolog programs and is suitable for execution on both shared and nonshared memory machines. We discuss the performance of the Prolog compiler in some detail and describe how grain size can be used to deliver performance that is within 10% of the underlying sequential Prolog compiler on one processor, and scale linearly with increasing number of processors on problems exhibiting sufficient parallelism. The loose coupling between parallel and sequential components makes it possible to use the best available sequential compiler as the sequential component of our compiler.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>

Spin-lock synchronization on the Butterfly and KSR1

The drawbacks of the simple spin-lock limit its effective use to small critical sections. Applications with large critical sections and a large number of processors require more efficient algorithms to minimize processor and network overheads. Variations on the spin-lock have been tested on the Sequent Symmetry, a bus-based shared-memory multiprocessor. Algorithms for scalable synchronization have also been tested on the BBN Butterfly I, a large-scale shared-memory multiprocessor with a multistage interconnection network(MIN). We have extended the investigation to the BBN GP1000 and TC2000, both MIN-based multiprocessors with network contention heavier than that on the Butterfly I. We have also implemented algorithms on Kendall Square Research's KSR1, a hierarchical-ring multiprocessor system, to study the effects of cache coherence. The execution behavior of spin-lock algorithms is significantly different between MIN-based and HR-based architectures. Our tests suggest that HR-based architectures handle network and memory contention more efficiently than MIN-based architectures. However, our results also suggest how spin-locks can be made cost-effective on both. >

Locking Effects in Multiprocessor Implementations of Protocols

We investigate how to exploit shared memory multiprocessors for parallel protocol processing. We present a multiprocessor implementation of the x-kernel protocol environment from the University of Arizona. A “processor-per-message” paradigm is used to partition the work over processors. Locks are used to protect shared protocol state and data. Mutual exclusion by locking can be costly if the parallel protocol code frequently accesses shared state and data. This paper addresses the effect of locking on performance.The x-kernel is implemented in user address space on a Sequent Symmetry. Performance results of UDP/IP and TCP/IP are given, and we compare them to performance predictions from simulations. The measured speedups for the parallel implementations relative to the sequential ones are more than 12 times for UDP and 3 times for TCP. These limits are set by locking and machine contention.

Toward parallel mathematical software for elliptic partial differential equations

Three approaches to parallelizing important components of the mathematical software package ELLPACK are considered: an explicit approach using compiler directives available only on the target machine, an automatic approach using an optimizing and parallelizing precompiler, and a two-level approach based on extensive use of a set of low level computational kernels. The focus is on shared memory architectures. Each approach to parallelization is described in detail, along with a discussion of the effort involved. Performance on a test problem, using up to sixteen processors of a Sequent Symmetry S81, is reported and discussed. Implications for the parallelization of a broad class of mathematical software are drawn.

Locking effects in multiprocessor implementations of protocols

We investigate how to exploit shared memory multiprocessors for parallel protocol processing. We present a multiprocessor implementation of the x-kernel protocol environment from the University of Arizona. A "processor-per-message" paradigm is used to partition the work over processors. Locks are used to protect shared protocol state and data. Mutual exclusion by locking can be costly if the parallel protocol code frequently accesses shared state and data. This paper addresses the effect of locking on performance.The x-kernel is implemented in user address space on a Sequent Symmetry. Performance results of UDP/IP and TCP/IP are given, and we compare them to performance predictions from simulations. The measured speedups for the parallel implementations relative to the sequential ones are more than 12 times for UDP and 3 times for TCP. These limits are set by locking and machine contention.

Message-oriented parallel implementation of Moded Flat GHC

We proposed in Ref. 5) a new,message-oriented implementation technique for Moded Flat GHC that compiled unification for data transfer into message passing. The technique was based on constraint-based program analysis, and significantly improved the performance of programs that used goals and streams to implement reconfigurable data structures. In this paper we discuss how the technique can be parallelized. We focus on a method for shared-memory multiprocessors, called theshared-goal method, though a different method could be used for distributed-memory multiprocessors. Unlike other parallel implementations of concurrent logic languages which we callprocess-oriented, the unit of parallel execution is not an individual goal but a chain of message sends caused successively by an initial message send. Parallelism comes from the existence of different chains of message sends that can be executed independently or in a pipelined manner. Mutual exclusion based on busy waiting and on message buffering controls access to individual, shared goals. Typical goals allowlast-send optimization, the message-oriented counterpart of last-call optimization. We have built an experimental implementation on Sequent Symmetry. In spite of the simple scheduling currently adopted, preliminary evaluation shows good parallel speedup and good absolute performance for concurrent operations on binary process trees.

Parallel ELLPACK for shared memory multiprocessors

This paper describes a parallel version of ELLPACK for shared memory multiprocessors. ELLPACK is a system for numerically solving elliptic PDEs. It consists of a very high level language for defining PDE problems and selecting methods of solution, and a library of approximately 50 problem solving modules. Each of the modules performs one of the basic steps in solving an elliptic PDE: discretization, reordering of equations and unknowns, linear system solution, etc. ELLPACK may be used to solve linear elliptic PDEs posed on general two-dimensional domains or in three-dimensional boxes, nonlinear problems, time-dependent problems and systems of elliptic equations. Earlier work considered three discretization modules ( five point star, hodie and hermite collocation), two linear system solution modules ( linpack spd band and jacobi cg) and a triple module ( hodie fft), which includes both discretization and solution, all for rectangular domains and simple boundary conditions. Here we describe parallel versions of six additional modules ( hermite collocation, hodie helmholtz, five point star, band ge, sor, symmetric sor cg) for general boundary conditions and domains, and discuss modifications to the ELLPACK preprocessor, the tool that translates an ELLPACK “program” into FORTRAN. The parallelization strategy is based on kernels, with the machine-dependent and performance critical code located in a relatively small number of kernels. We report performance of these parallel ELLPACK modules on a Sequent Symmetry S81 shared memory multiprocessor with 26 processors for two simple test problems, and a problem involving a composite laminate with spatially varying fiber orientations.

Utilizing Global Simulation Information in Conservative Parallel Simulation on Shared Memory Multiprocessors

Global simulation information is accessible on shared memory multiprocessors and can improve the efficiency of parallel simulation. However, most existing concurrent simulators do not aggressively exploit this information. In this paper, we propose a Directly Accessing Information (DAI) scheme, a conservative simulation scheme that collects useful global simulation information in shared memory systems to reduce non-essential blocking and resolve local deadlocks. A parallel queueing network simulator was constructed on a Sequent Symmetry multiprocessor. Special implementation techniques were used to eliminate locking mechanisms for accessing global simulation information. Also, search pruning techniques were developed to reduce the amount of global simulation information used to unblock Logical Processes (LPs). Experimental results demonstrate that the DAI scheme achieves good speedups and substantially outperforms conventional conservative schemes in a reasonably large problem domain where message densities are larger than approximately 0.75.

On the Parallel Elliptic Single/Multigrid Solutions about Aligned and Nonaligned Bodies Using the Virtual Machine for Multiprocessors

Parallel elliptic single/multigrid solutions around an aligned and nonaligned body are presented and implemented on two multi‐user and single‐user shared memory multiprocessors (Sequent Symmetry and MOS) and on a distributed memory multiprocessor (a Transputer network). Our parallel implementation uses the Virtual Machine for Muli‐Processors (VMMP), a software package that provides a coherent set of services for explicitly parallel application programs running on diverse multiple instruction multiple data (MIMD) multiprocessors, both shared memory and message passing. VMMP is intended to simplify parallel program writing and to promote portable and efficient programming. Furthermore, it ensures high portability of application programs by implementing the same services on all target multiprocessors. The performance of our algorithm is investigated in detail. It is seen to fit well the above architectures when the number of processors is less than the maximal number of grid points along the axes. In general, the efficiency in the nonaligned case is higher than in the aligned case. Alignment overhead is observed to be up to 200% in the shared‐memory case and up to 65% in the message‐passing case. We have demonstrated that when using VMMP, the portability of the algorithms is straightforward and efficient.

A dynamic processor allocation policy for multiprogrammed shared-memory multiprocessors

We propose and evaluate empirically the performance of a dynamic processor-scheduling policy for multiprogrammed shared-memory multiprocessors. The policy is dynamic in that it reallocates processors from one parallel job to another based on the currently realized parallelism of those jobs. The policy is suitable for implementation in production systems in that: We have evaluated our scheduler and compared it to alternatives using a set of prototype implementations running on a Sequent Symmetry multiprocessor. Using a number of parallel applications with distinct qualitative behaviors, we have both evaluated the policies according to the major criterion of overall performance and examined a number of more general policy issues, including the advantage of “space sharing” over “time sharing” the processors of a multiprocessor, and the importance of cooperation between the kernel and the application in reallocating processors between jobs. We have also compared the policies according to other criteia important in real implementations, in particular, fairness and respone time to short, sequential requests. We conclude that a combination of performance and implementation considerations makes a compelling case for our dynamic scheduling policy.

Portable, parallelizing Pascal compiler

The structure and implementation of P/sup 3/C, a compiler that uses relatively simple parallelization techniques to produce highly efficient code for both shared-memory and distributed-memory MIMD multiprocessors, are discussed. P/sup 3/C is a fully automatic, portable parallelizing Pascal compiler for scientific code, which is characterized by loops operating on regular data structures. P/sup 3/C compiles the same source code to all target machines without modification. P/sup 3/C gives an accurate estimate about whether the parallel code will actually reduce execution time over serial code, taking into account the associated overhead. It derives the estimate by statically analyzing the program at compile time, referring to a table of the target machine's parameters. Speedups of up to 24 have been achieved on a Sequent Symmetry with 25 active processors. The estimated speedup was within 8% of the actual figure.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>

Dynamic parallel evaluation of the cross-product set using time-stamps

We consider the problem of dynamically generating the cross-product (cartesian product) of two or more sets, in parallel, on shared memory machines. Dynamic generation means that the cross-product set is computed on the fly as various elements of its constituent sets are generated asynchronously. Dynamic evaluation of the cross-product set finds application in a number of problems such as parallel evaluation of data-base joins. Generation of the cross-product set using the straight-forward nested-loop algorithms is inherently sequential because when elements of two constituent sets are generated simultaneously, computing the tuples of the cross-product set corresponding to these elements has to be done sequentially. We present a simple modification of the nested loop algorithm using time-stamps which results in a significant amount of parallelism being extracted. We present results of experiments done on the Sequent Symmetry multiprocessor which show that the overhead introduced for time-stamping is minimal while speed-ups obtained are linear.

Programming in PACLIB

This paper gives a short overview on PACLIB, a new system for p arallel a lgebraic c omputation on shared memory computers. PACLIB has been developed as a professional tool for the simple design and efficient implementation of parallel algorithms in computer algebra and related areas. It provides concurrency, shared memory communication, non-determinism, speculative parallelism, streams and pipelining and a parallelized garbage collection. PACLIB has been implemented on a Sequent Symmetry multiprocessor and is portable to other shared memory machines and workstations.

Parallel computations of turbulent diffusion in convective boundary layers on shared-memory machines

Currently, time-splitting finite difference or finite element schemes are commonly used in air-pollution modeling. In light of the recent advances in parallel processing, it is crucial to evaluate the performance of these time-splitting schemes on different multiprocessors. In this paper, we use a K-theory diffusion model and a second-order closure model to simulate turbulent diffusion in convective boundary layers. The objective is to assess the “portability” of time-splitting finite element schemes for shared-memory multiprocessors, such as IBM 3090-600J, the Alliant FX/8, the Sequent Symmetry S81 and the Encore Multimax. It was found that significant speed-ups were obtained as a result of parallelization with small programming efforts. Furthermore, our two- and three-dimensional results also suggest that second-order closure models are now computationally affordable and feasible for realistic air pollution modeling.

Generalized Networks: Parallel Algorithms and an Empirical Analysis

The objective of this research was to develop and empirically test new simplex-based parallel algorithms for the generalized network optimization problem. One of these algorithms is essentially a “data parallel” method in which each processor executes identical code on a portion of the data. (However, since the data sets are not necessarily disjoint, “locks” are used to ensure exclusive access.) A second algorithm exhibits “control parallelism,” using different processors to simultaneously execute the different subtasks of the simplex method. “Locks” are not needed in this second approach, but, instead, at the beginning of each pivot, an “audit” of the proposed entering arc is performed in order to ensure correctness of the method. These parallel algorithms were implemented on the Sequent Symmetry multiprocessor, empirically tested on a variety of problems produced by two random problem generators, and compared with two leading state-of-the-art serial codes. Good speedups were obtained relative to the serial codes, and the relative performance of the two parallel methods was found to be dependent on connectedness properties of the optimal solutions of the test problems. The largest test problem, a generalized transportation problem having 30,000 nodes and 1.2 million arcs was optimized in approximately 11 minutes by our parallel code, displaying a speedup of 13 on 15 processors. INFORMS Journal on Computing, ISSN 1091-9856, was published as ORSA Journal on Computing from 1989 to 1995 under ISSN 0899-1499.

On the parallelization of an adaptive multigrid algorithm for a class of flow problems

We describe the parallelization of an adaptive multigrid code that solves the steady, laminar, incompressible Navier-Stokes equations. Parallelism was achieved as loop concurrency on a shared-memory multiprocessor. Analysis of the code for parallelism, in particular the location of data dependencies that inhibit parallelism, was aided by the use of the software tool VecPar_77®. We present speedup results obtained on a Sequent Symmetry®. We describe the scalability of the algorithm by presenting results from an execution simulator. analyzed using a model of loop-based, shared-memory parallel computation.

Fast parallel implementation of lazy languages—the EQUALS experience

This paper describes EQUALS, a fast parallel implementation of a lazy functional language on a commercially available shared-memory parallel machine, the Sequent Symmetry. In contrast to previous implementations, we detect parallelism automatically by propagating exhaustive (normal form) demand. Another important difference between EQUALS and previous implementations is the use of reference counting for memory management instead of garbage collection. Our implementation shows that reference counting leads to very good scalability, low memory requirements and improved locality. We compare our results with sequntial SML/NJ as well as parallel ( v, G -machine and GAML implementations.

On the performance of the CREL system

This paper presents the performance results of a comprehensive approach to the parallel execution of rule systems. It describes the semantics of a Concurrent Rule Execution Language, CREL, and the architecture of the system that compiles and executes CREL programs. The system has been designed to avoid run-time overhead by performing extensive compile-time analysis and by parallelizing compilation. Static dependency analysis, based on serializability, coupled with a set of optimizing transforms, partitions the program into subsets, called clusters. Clusters execute independently of each other and communicate though asynchronous message passing. At run time two additional sources of parallelism are exploited—run-time consistency checking allowing multiple rules to fire, and match-level parallelism. The CREL system is implemented on a Sequent Symmetry shared-memory computer. This paper presents the results of a factorial experiment that isolates and evaluates each source of parallelism in the CREL system and each possible combination of those methods. The results suggest that multiple-rule firing is the single most important source of parallelism in CREL programs and that the use of static dependency analysis based on serializability is instrumental to effectively exploit parallelism.

An Empirical Analysis of the Dense Assignment Problem: Sequential and Parallel Implementations

The best algorithms for the dense assignment problem are acknowledged to be the auction algorithm and the shortest augmenting path algorithm. In this investigation we present an empirical analysis of two of the current best software implementations of these two methods on three different serial machines. These software implementations were developed by D. P. Bertsekas of the Massachusetts Institute of Technology and by R. Jonker and T. Volgenant of the University of Amsterdam. This report is an independent evaluation of the software implementation of these two algorithms. For the sample of problems examined and the sample of hardware used (IBM 3081D, Sequent Symmetry S81, and VAX 750), we found that the shortest augmenting path algorithm was the fastest. We also report our empirical results with a parallel version of the shortest augmenting path algorithm. On 1200×1200 dense assignment problems, speedups of approximately four were achieved using ten processors. Million arc problems were solved in less than twelve seconds on a Sequent Symmetry S81 with the parallel shortest augmenting path algorithm. INFORMS Journal on Computing, ISSN 1091-9856, was published as ORSA Journal on Computing from 1989 to 1995 under ISSN 0899-1499.

Simulation of Compositional Reservoir Phenomena on a Distributed-Memory Parallel Computer

Summary Distributed-memory parallel computer architectures parallel computer architectures appear to offer high performance at moderate cost for reservoir simulation applications. In particular, the simulation of compositional reservoir phenomena shows great promise for phenomena shows great promise for parallel applications because of the parallel applications because of the large parallel content of the compositional formulations. This paper focuses on the application of a distributed-memory parallel computer, the iPSC/860, to the solution of two compositional simulations; one based on the Third SPE Comparative Solution problem and another based on a real production compositional model. An improved linear equation solution technique based on multigrid and domain decomposition methods is compared with other techniques in serial and parallel environments. For a hypothetical heterogeneous example, the new technique showed a high degree of parallel efficiency. Finally, results parallel efficiency. Finally, results show that performance of the compositional simulations with a fully parallel simulator is comparable to that of parallel simulator is comparable to that of current mainframe supercomputers. Introduction Over the past decade, computer performance has advanced significantly. Current performance has advanced significantly. Current single-processor performance is limited by speed-of-light considerations. These limitations indicate that, to achieve speed ups greater than an order of magnitude above current technology, significant architectural changes must be made. The emerging parallel computer architectures appear to be parallel computer architectures appear to be the most likely avenue for this advancement. Parallel computers have existed for Parallel computers have existed for several decades, but only recently have the architectures and languages allowed these machines to be used for significant work. The parallel machines have three varieties: shared memory, distributed memory, and very long instruction word. Some shared memory machines are the Cray Y-MP 8/32, the IBM 3090 600S/VF, the Sequent Symmetry, and the ETA 10. The CPU's and memories are connected so that all memory is accessible by all CPU's. This is illustrated in Fig. 1, which shows a cross-bar-like connection among the memory banks and CPU's. For the high-performance supercomputers like the Cray X-MP and IBM 3090, this global memory connection can be costly because of the requirement of rapid access to memory. To overcome this difficulty, several distributed-memory architectures have evolved. These are characterized by grids of less-powerful CPU's connected in some fashion for communication of data and synchronization (Fig. 2). One of the most popular of these connection schemes is the hypercube topology. The N-Cube and Intel iPSC/860 use this architecture. Each may have more than 100 processors. Both the N-Cube and Intel are processors. Both the N-Cube and Intel are multiple-instruction, multiple-data computers. Each CPU may execute different sets of instructions on different data. Another emerging distributed-memory, massively parallel computer is the connection machine, parallel computer is the connection machine, which is a single-instruction, multiple-data architecture with up to thousands of processors. All CPU's execute each instruction in processors. All CPU's execute each instruction in lock step with one another. The very-long-instruction-word computer, such as the Multiflow and FPS 264, exploit local parallelism within a user's program. By combining many parallel program. By combining many parallel operations of the functional units of the machine into each instruction, the compiler can achieve significant speedups. Several authors have dealt with the application of parallel computing to petroleum reservoir simulation in shared-memory parallel environments. Scott et al. parallel environments. Scott et al. investigated the parallelization of the coefficient routines and linear equation solvers for a black-oil model on a Denelcor HEP. Chien et al. investigated compositional modeling in parallel on a Cray X-MP 4/16. Barua and Horne applied parallel computing using a nonlinear equation solver for the black-oil case on the Encore Multimax. Killough and Wheeler looked at parallel linear equation solvers on both the Cray X-MP and IBM 3090. Each application involved a shared memory parallel computer. The question still remained as to whether a distributed-memory architecture could be used efficiently for simulation of petroleum reservoirs. More recently, parallelization of reservoir simulators has been accomplished on distributed-memory parallel computers. van Daalen et al. showed a speedup of a factor of 40 on 60 processors on the Transputer-based Meiko computer. Wheeler and Smith showed that black-oil modeling could be performed efficiently on a hypercube. Application of compositional reservoir modeling to the distributed-memory, message-passing Intel iPSC/2 Hypercube was investigated by Killough and Bhogeswara. Simulations using a commercial compositional reservoir simulator (VIP-COMP) showed that high parallel efficiencies could be obtained with a model problem based on the Third SPE problem based on the Third SPE Comparative Solution Project. The key issues in this parallelization involved the data structure of the program, message passing of data among processors, and development of parallel linear equation solvers for the parallel linear equation solvers for the model.