8K Processors Research Articles

Reducing latency cost in 2D sparse matrix partitioning models

Abstract Sparse matrix partitioning is a common technique used for improving performance of parallel linear iterative solvers. Compared to solvers used for symmetric linear systems, solvers for nonsymmetric systems offer more potential for addressing different multiple communication metrics due to the flexibility of adopting different partitions on the input and output vectors of sparse matrix-vector multiplication operations. In this regard, there exist works based on one-dimensional (1D) and two-dimensional (2D) fine-grain partitioning models that effectively address both bandwidth and latency costs in nonsymmetric solvers. In this work, we propose two new models based on 2D checkerboard and jagged partitioning. These models aim at minimizing total message count while maintaining a balance on communication volume loads of processors; hence, they address both bandwidth and latency costs. We evaluate all partitioning models on two nonsymmetric system solvers implemented using the widely adopted PETSc toolkit and conduct extensive experiments using these solvers on a modern system (a BlueGene/Q machine) successfully scaling them up to 8K processors. Along with the proposed models, we put practical aspects of eight evaluated models (two 1D- and six 2D-based) under thorough analysis. To the best of our knowledge, this is the first work that analyzes practical performance of 2D models on this scale. Among evaluated models, the models that rely on 2D jagged partitioning obtain the most promising results by striking a balance between minimizing bandwidth and latency costs.

Three-dimensional dynamic soil analysis of a live load in Ullevi Stadium

An explicit finite difference code, ELWA, has been developed for solving a three-dimensional (3D) formulation of the damped elastic wave equation for a soil medium in the time domain. The code was implemented on a 8K processors massively data-parallel computer. In this paper, a description is given of the mathematical model forming the basis for the code as well as of its application towards explaining the occurrence of extreme ground and structural vibrations during a rock concert in Gothenburg, Sweden. During two rock concerts held at a large outdoor stadium in Gothenburg, in the summer of 1985, the part of the audience standing on the pitch in front of the stage started to jump in time to the music. This rhythmic jumping caused the soil deposit below to vibrate heavily, leading to severe vibration in the flexible superstructure of the stadium. The vibrations became so severe in the upper parts of the stands that the audience left their places. Residents in the neighbourhood of the stadium complained of heavy vibrations in their respective residences. A model of the soil medium under the stadium arena is described along with the excitation forces used to model audience behaviour when listening to rhythmic and excitable music. Using ELWA, an analysis of soil vibrations for these conditions shows that the reflection of vibrations from the solid rock underlying the clay deposit upon, which the stadium is based, causes standing waves to arise at the associated resonant frequencies. These waves will be transmitted to the stadium superstructure through soil–foundation interaction. The most important parameters controlling the wave amplitude were found to be the thickness of the clay deposit and the frequency of loading.

Explicitly restarted Lanczos algorithms in an MPP environment

The Lanczos algorithm is one of the principal methods for the computation of a small part of the eigenspectrum of large, sparse, real symmetric matrices. A single-vector, explicitly restarted variant of the Lanczos method is proposed in this paper. The algorithm finds only one eigenpair at a time using a deflation technique in which the Lanczos factorization for the current eigenpair is generated in the null space of all previously computed eigenvectors. This approach yields a fixed k-step restarting scheme which permits short Lanczos factorizations and almost completely eliminates the reorthogonalization among the Lanczos vectors. The orthogonalization strategy developed falls naturally into the class of selective orthogonalization strategies as classified by Simon. `Reverse communication' software for the implementation of the proposed variant on a Connection Machine CM-200 with 8K processors and on a Cray T3D with 32 processors is discussed. Test results on the CM-200 using examples from the Harwell–Boeing collection of sparse matrices show the method to be very effective when compared with Sorensen's state-of-the-art routine taken from the ARPACK library. The method has fixed, small storage requirements, copes easily with genuinely multiple eigenvalues and is guaranteed to converge to the desired eigenvalues.

Time and Space Optimal Data Parallel Volume Rendering Using Permutation Warping

In this paper we present a data parallel volume rendering algorithm that possesses numerous advantages over prior published solutions. Volume rendering is a three-dimensional graphics rendering algorithm that computes views of sampled medical and simulation data, but has been much slower than other graphics algorithms because of the data set sizes and the computational complexity. Our algorithm usespermutation warpingto achieve linear speedup (run time is O(S/P) forPprocessors whenP\\= O(S/logS) forS\\=n3samples), linear storage (O(S)) for large data sets, arbitrary view directions, and high-quality filters. We derived a new processor permutation assignment of five passes (our prior known solution was eight passes), and a new parallel compositing technique that is essential for scaling linearly on machines that have more processors than view rays to process (P>n2). We show a speedup of 15.7 for a 16k processor over a 1k processor MasPar MP-1 (16 is linear) and two frames/second with a 1283volume and trilinear view reconstruction. In addition, we demonstrate volume sizes of 2563, constant run time over angles 5 to 75°, filter quality comparisons, and communication congestion of just 19 to 29\\%.

Distributed boundary representation and Boolean operations on a massively parallel computer

Boolean operation is usually the most expensive task in solid modelling. Considerable amount of time is required in a union, difference, or intersection operation especially when an object grows in complexity after a series of operations. Parallel processing seems to be a promising technique for improving the speed in the manipulation of complex solid objects. This paper presents a technique for performing Boolean operations in a distributed environment. In this method, a distributed boundary representation scheme is adopted for representing objects in an array of processors of a SIMD computer. Solid objects with non-manifold boundary is assumed and is modelled with a cell-complex based boundary representation. The concept of half-wedge is employed to allow easy retrieval of neighbourhood or topological information in a distributed processing environment. Parallel algorithms for obtaining the complement of solids are presented. Algorithms for evaluating the intersecting regions and hence the result of Boolean operations between objects are also discussed. The proposed method is implemented on a DECmpp 1200/Sx machine with 8K processors. Test results on the experimental system shows that the method is efficient for a test piece with 1300 or more entities. The actual performance of the system may also depends on the relative sizes of the entities involved.

Efficient Sparse Cholesky Factorization on a Massively Parallel SIMD Computer

We investigate the effect of load balancing when performing Cholesky factorization on a massively parallel SIMD computer. In particular we describe a supernodal algorithm for performing sparse Cholesky factorization. The way the matrix is mapped onto the processors has significant effect on its efficiency. We show that this assignment problem can be modeled as a graph coloring problem in a weighted graph. By a simple greedy algorithm, we obtain substantial speedup compared with previously suggested data mapping schemes. Experimental runs have been made on a 16K processor MasPar MP-2 parallel computer using symmetric test matrices with irregular sparsity structure. On these problems our implementation achieves performance rates of well above 200 Mflops in double precision arithmetic.

Optimization and Performance Analysis of a Massively Parallel Dynamic Programming Algorithm for Rna Secondary Structure Prediction

An optimized and parallelized form of a dynamic pro gramming algorithm capable of generating optimal and suboptimal RNA secondary structures is pre sented. Implementation of this algorithm on a MasPar MP-2 with 16K processors is shown to perform ex tremely well for very large nucleic acid sequences such as HIV (AIDS) and Rhinovirus (common cold). These sequences are, respectively, 9,128 and 7,208 nu cleotides in length. By taking advantage of the parallel nature of MasPar and also optimizing the communica tion requirements the implementation essentially re duced the algorithm from order O(n3) to 0(n2). This reduction in complexity enables us to fold large RNA sequences in reasonable amounts of time. This capa bility has proven to be a valuable tool in studying the molecular structure and biological function of mole cules this large.

2D and 3D Optimal Parallel Image Warping

Spatial image warping is useful for image processing and graphics. In this paper, we present concurrent-read-exclusive-write (CREW) and exclusive-read-exclusive-write (EREW) parallel-random-access-machine (PRAM) algorithms that achieve O(1) asymptotic run time. The significant result is the creative processor assignment that results in an EREW PRAM algorithm. The forward algorithm calculates any nonscaling affine transform including arbitrary skewings, translations, and rotations. The EREW algorithm is the most efficient in practice, and the MasPar MP-1 with 16K processors rotates a 4-million-element image in under a second and a 2-million-element volume in one-half of a second. This high performance allows interactive viewing of volumes from arbitrary viewpoints and illustrates linear speedup. This practical efficiency is analyzed and illustrated by using a bridging model of computation. We develop the mixed cost communication machine (MCCM) to quantify the communication costs and correlate these costs to the MasPar MP-1. The forward algorithm has provable N = 1 congestion on the MCCM, while the backward algorithm has congestion N > 1 which varies with the transform. There are also important quality advantages using our direct warping techniques; empirical measurements are given to provide comparisons to multipass warps.

An implementation of a tree code on a SIMD, parallel computer

We describe a fast tree algorithm for gravitational N-body simulation on SIMD parallel computers. The tree construction uses fast, parallel sorts. The sorted lists are recursively divided along their x, y and z coordinates. This data structure is a completely balanced tree (i.e., each particle is paired with exactly one other particle) and maintains good spatial locality. An implementation of this tree-building algorithm on a 16k processor Maspar MP-1 performs well and constitutes only a small fraction (approximately 15%) of the entire cycle of finding the accelerations. Each node in the tree is treated as a monopole. The tree search and the summation of accelerations also perform well. During the tree search, node data that is needed from another processor is simply fetched. Roughly 55% of the tree search time is spent in communications between processors. We apply the code to two problems of astrophysical interest. The first is a simulation of the close passage of two gravitationally, interacting, disk galaxies using 65,636 particles. We also simulate the formation of structure in an expanding, model universe using 1,048,576 particles. Our code attains speeds comparable to one head of a Cray Y-MP, so single instruction, multiple data (SIMD) type computers can be used for these simulations. The cost/performance ratio for SIMD machines like the Maspar MP-1 make them an extremely attractive alternative to either vector processors or large multiple instruction, multiple data (MIMD) type parallel computers. With further optimizations (e.g., more careful load balancing), speeds in excess of today's vector processing computers should be possible.

Efficient massively parallel simulation of dynamic channel assignment schemes for wireless cellular communications

Fast, efficient parallel algorithms are presented for discrete event simulations of dynamic channel assignment schemes for wireless cellular communication networks. The driving events are called arrivals and departures, in continuous time, to cells geographically distributed across the service area. A dynamic channel assignment scheme decides which call arrivals to accept, and which channels to allocate to the accepted calls, attempting to minimize call blocking while ensuring co-channel interference is tolerably low. Specifically, the scheme ensures that the same channel is used concurrently at different cells only if the pairwise distances between those cells are sufficiently large. Much of the complexity of the system comes from ensuring this separation. The network is modeled as a system of interacting continuous time automata, each corresponding to a cell. To simulate the model, we use conservative methods; i.e., methods in which no errors occur in the course of the simulation and so no rollback or relaxation is needed. Implemented on a 16K processor MasPar MP-1, an elegant and simple technique provides speedups of about 15x over an optimized serial simulation running on a high speed workstation. A drawback of this technique, typical of conservative methods, is that processor utilization is rather low. To overcome this, we developed new methods that exploit slackness in event dependencies over short intervals of time, thereby raising the utilization to above 50% and the speedup over the optimized serial code to about 120x with respect to the workstation simulation.

Pyramid mappings onto hypercubes for computer vision: Connection machine comparative study

AbstractThe paper presents a comparative analysis for algorithms that map pyramids onto hypercubes. The analysis is based on some important performance measures from graph theory and actual results from a Connection Machine system CM‐2 containing 16K processors. Connection Machine results are presented for pyramid algorithms that compute the perimeter of objects, apply 2‐dimensional convolution, and segment images.

Parallel Discrete Event Simulation on SIMD Computers

A simulator based on a synchronous parallel simulation scheme is developed on an SIMD computer, a connection machine (CM-2) with 8K processors. Multistage interconnection networks of different sizes (2 to 16 stages) are simulated. Two categories of experiments are performed: symmetric and asymmetric workloads. Four metrics are used to evaluate performance of the simulator: degree of detected parallelism, degree of potential parallelism, speedup, and efficiency. The obtained results demonstrate good speedups (over 2000) in some cases.

Compiling machine-independent parallel programs

Initial evidence is presented that explicitly parallel, machine-independent programs can automatically be translated into parallel machine code that is competitive in performance with hand-written code.The programming language used is Modula-2*, an extension of Modula-2, which incorporates both data and control parallelism in a portable fashion. An optimizing compiler targeting MIMD, SIMD, and SISD machines translates Modula-2* into machine-dependent C code.The performance of the resulting code is compared to that of equivalent, carefully hand-coded and tuned prograins. On a MasPar MP-1 (SIMD machine with up to 16k processors) the Modula-2* programs typically achieve 80% of the performance of the hand-coded parallel versions. When targeting sequential processors, the Modula-2* programs reach 90% of the performance of hand-coded sequential C. (There are no MIMD results yet.)The effects of two major optimization techniques, synchronization point elimination and data/process alignment are also quantified.

A SIMD approach to parallel heuristic search

Serial search algorithms often exhibit exponential run times and may require an exponential amount of storage as well. Thus, the design of parallel search algorithms with limited memory is of obvious interest. This paper presents an efficient SIMD parallel algorithm, called IDPS (for iterative-deepening parallel search). At a broad level IDPS is a parallel version of IDA∗. While generically we have called our algorithm an IDPS, performance of four variants of it has been studied through experiments conducted on the well-known test-bed problem for search algorithms, namely the Fifteen Puzzle. During the experiments, data were gathered under two different static load balancing schemes. Under the first scheme, an unnormalized average efficiency of approximately 3 4 was obtained for 4K, 8K, and 16K processors. Under the second scheme, unnormalized average efficiencies of 0.92 and 0.76, and normalized average efficiencies of 0.70 and 0.63 were obtained for 8K and 16K processors, respectively. We show (as shown previously only for MIMD machines) that for admissible search, high average speedup can be obtained for problems of significant size. We believe that this research will enhance AI problem solving using parallel heuristic search algorithms.

Depth-first heuristic search on a SIMD machine

We present a parallel implementation of Iterative-Deepening-A∗, a depth-first heuristic search, on the single-instruction, multiple-data (SIMD) Connection Machine ★ ★ Connection Machine is a trademark of Thinking Machines Corporation. . Heuristic search of an irregular tree represents a new application of SIMD machines. The main technical challenge is load balancing, and we explore three different techniques in combination. We also use a simple method for dynamically determining when to stop searching and start load balancing. We achieve an efficiency of 69%, for a speedup of 5685 on 8K processors, an efficiency of 64%, for a speedup of 10,435 on 16K processors, and an efficiency of 53%, for a speedup of 17,300 on 32K processors on the Fifteen Puzzle. On hard problem instances, we achieved efficiencies as high as 80%, for a speedup of 26,215 on 32K processors. Our analysis indicates that work only needs to increase as P log P to maintain constant efficiency, where P is the number of processors. This high degree of scalability was confirmed empirically for the range of 16 to 32,768 (32K) processors.

Experiments with Krylov subspace methods on a massively parallel computer

In this note, we compare some Krylov subspace iterative methods on the MASPAR, a massively parallel computer with 16K processors. In particular, we apply these methods to solve large sparse nonsymmetric linear systems arising from elliptic partial differential equations. The methods under consideration include conjugate gradient type methods, semiiterative methods, and a hybrid variant. Our numerical results show that, on the MASPAR, one should compare iterative methods rather on the basis of total computing time than on the basis of number of iterations required to achieve a given accuracy. Our limited numerical experiments here suggest that, in terms of total computing time, semiiterative and hybrid methods are very attractive for such MASPAR implementations.

Implementation of the FCHC lattice gas model on the Connection Machine

The 4-dimensional FCHC lattice gas model has been implemented on a Connection Machine CM-2 with 16K processors. Symmetries are used to reduce the collision table to a size that fits into local memory. This method avoids the degradation of the Reynolds coefficientR*, but at the price of increased computing time. Bit shuffling between parallel lattices is introduced to reduce the discrepancy between measured viscosities and those predicted from the Boltzmann approximation. Thereby a model with a negative shear viscosity is obtained: a fluid having a uniform initial velocity is unstable and organized nonuniform motions develop. Because of the buildup of very strong correlations between the parallel lattices, the discrepancy with the Boltzmann values decreases only very slowly with the number of parallel lattices.

Efficient massively parallel Euler solver for two-dimensional unstructured grids

We describe a data parallel mesh-vertex upwind finite-volume scheme for solving the Euler equations on triangular unstructured meshes. A novel vertex-based partitioning of the problem is introduced that minimizes the computation and communication costs associated with distributing the computation to the processors of a massively parallel computer. Finally, we compare the performance of this unstructured computation on 8K processors of the Connection Machine CM-2 with one processor of a Cray-YMP

Direct particle simulation on the Connection Machine CM-2

Particle simulation is a useful technique for analyzing low density flows. The Connection Machine CM-2 is a useful test bed for studying the fine-grain data objects decomposition and the coarse-grain domain decomposition single instruction multiple datastream (SIMD) approaches to particle simulation. Both approaches are investigated for the model problem of uniform flow through a channel and the algorithms required for the SIMD domain decomposition approach are presented. For this model problem, the domain decomposition approach is superior to the data objects decomposition approach with an average particle update time of 2.9 μs compared with 5.1 μs on an 8K processor CM-2. An unresolved issue with the domain decomposition approach is the effect of a poor partitioning on flows with real geometries. Initial results with the channel flow problem indicate that a poor partitioning has only a small (15%) detrimental effect on the overall performance.

Surface Growth with Power-Law Noise in 2+1 Dimensions

Large scale simulations of stochastic growth of 2+1 dimensional surfaces were carried out on a 16K processor Connection Machine. We introduce a family of models for which we could reproduce the known scaling behavior of kinetic roughening in the presence of bounded noise. For noise amplitudes η distributed to P(η)sim η^{-μ-1}, the growth exponents depend on μ and they are well described by the recently formula based on the scaling of rare events.