IBM SP2 Research Articles

Prefix Computations on Symmetric Multiprocessors

We introduce a new prefix computation algorithm on linked lists which builds upon the sparse ruling set approach of Reid-Miller and Blelloch. Besides being somewhat simpler and requiring nearly half the number of memory accesses, we can bound our complexity with high probability instead of merely on average. Moreover, whereas Reid-Miller and Blelloch targeted their algorithm for implementation on a vector multiprocessor architecture, we develop our algorithm for implementation on the symmetric multiprocessor architecture (SMP). These symmetric multiprocessors dominate the high-end server market and are currently the primary candidate for constructing large scale multiprocessor systems. Our prefix computation algorithm was implemented in C using POSIX threads and run on four symmetric multiprocessors: the HP-Convex Exemplar (S-Class), the IBM SP-2 (High Node), the SGI Power Challenge, and the DEC AlphaServer. We ran our code using a variety of benchmarks which we identified to examine the dependence of our algorithm on memory access patterns. For some problems, our algorithm actually matched or exceeded the performance of the standard sequential solution using only a single thread. Moreover, in spite of the fact that the processors must compete for access to main memory, our algorithm still achieved scalable performance with up to 16 processors, which was the largest platform available to us.

Parallel PIC plasma simulation through particle decomposition techniques

Parallelization of a particle-in-cell (PIC) code has been accomplished through a “particle decomposition” technique instead of the more usual “domain decomposition” one. The adopted technique requires a moderate effort in porting the code in parallel form and results in intrinsic load balancing and modest inter-processor communication. The resulting data parallel implementation has been carried out within the High Performance Fortran (HPF) framework, and tested on the IBM SP parallel system. The performance tests obtained confirm the hypothesis of high effectiveness of the strategy, if targeted towards moderately parallel architectures. Optimal use of resources is also discussed with reference to a specific physics problem.

Parallel Compositional Reservoir Simulation on Clusters of PCs

The authors have ported a fully implicit equation-of-state (EOS) compositional parallel reservoir simulator to run on clusters of PCs. They report on the performance of the code on two clusters, both in terms of scalability and in absolute performance relative to an IBM SP. The simulator scales well through 16 processors on the clusters and is comparable in execution time with the SP.

Skew-insensitive Parallel Algorithms for Relational Join

Join is the most important and expensive operation in relational databases. The parallel join operation is very sensitive to the presence of the data skew. In this paper, we present two new parallel join algorithms for coarse-grained machines, which work optimally in presence of arbitrary amount of data skew. The first algorithm is sort-based and the second is hash-based. Both of these algorithms employ a preprocessing phase (prior to the redistribution phase) to equally partition the work among the processors. These algorithms are shown to be theoretically as well as practically scalable. Experimental results are provided on the IBM SP-2.

Search engine case study: searching the web using genetic programming and MPI

The generation of a Web page follows distinct sources for the incorporation of information. The earliest format of these sources was an organized display of known information determined by the page designers' interest and/or design parameters. The sources may have been published in books or other printed literature, or disseminated as general information about the page designer. Due to a growth in Web pages, several new search engines have been developed in addition to the refinement of the already existing ones. The use of the refined search engines, however, still produces an array of diverse information when the same set of keywords are used in a Web search. Some degree of consistency in the search results can be achieved over a period of time when the same search engine is used, yet, most initial Web searches on a given topic are treated as final after some form of refinement/adjustment of the keywords used in the search process. To determine the applicability of a genetic programming (GP) model for the diverse set of Web documents, search strategies behind the current search engines for the World Wide Web were studied. The development of a GP model resulted in a parallel implementation of a pseudo-search engine indexer simulator. The training sets used in this study provided a small snapshot of the computational effort required to index Web documents accurately and efficiently. Future results will be used to develop and implement Web crawler mechanisms that are capable of assessing the scope of this research effort. The GP model results were generated on a network of SUN workstations and an IBM SP2.

Parallel Implementation of a Central Decomposition Method for Solving Large-Scale Planning Problems

We use a decomposition approach to solve three types of realistic problems: block-angular linear programs arising in energy planning, Markov decision problems arising in production planning and multicommodity network problems arising in capacity planning for survivable telecommunication networks. Decomposition is an algorithmic device that breaks down computations into several independent subproblems. It is thus ideally suited to parallel implementation. To achieve robustness and greater reliability in the performance of the decomposition algorithm, we use the Analytic Center Cutting Plane Method (ACCPM) to handle the master program. We run the algorithm on two different parallel computing platforms: a network of PC's running under Linux and a genuine parallel machine, the IBM SP2. The approach is well adapted for this coarse grain parallelism and the results display good speed-up's for the classes of problems we have treated.

Parallel Implementation of a Class of Adaptive Signal Processing Applications

Recently, High Performance Computing (HPC) platforms have been employed to realize many computationally demanding applications in signal and image processing. These applications require real-time performance constraints to be met. These constraints include latency as well as throughput. In order to meet these performance requirements, efficient parallel algorithms are needed. These algorithms must be engineered to exploit the computational characteristics of such applications. In this paper we present a methodology for mapping a class of adaptive signal processing applications onto HPC platforms such that the throughput performance is optimized. We first define a new task model using the salient computational characteristics of a class of adaptive signal processing applications. Based on this task model, we propose a new execution model. In the earlier linear pipelined execution model, the task mapping choices were restricted. The new model permits flexible task mapping choices, leading to improved throughput performance compared with the previous model. Using the new model, a three-step task mapping methodology is developed. It consists of (1) a data remapping step, (2) a coarse resource allocation step, and (3) a fine performance tuning step. The methodology is demonstrated by designing parallel algorithms for modern radar and sonar signal processing applications. These are implemented on IBM SP2 and Cray T3E, state-of-the-art HPC platforms, to show the effectiveness of our approach. Experimental results show significant performance improvement over those obtained by previous approaches. Our code is written using C and the Message Passing Interface (MPI). Thus, it is portable across various HPC platforms.

Spatial and temporal data parallelization of the H.261 video coding algorithm

The parallelization of the H.261 video coding algorithm on the IBM SP2(R) multiprocessor system is described. The effect of parallelizing computations and communications in the spatial, temporal, and both spatial-temporal domains are considered through the study of frame rate, speedup, and implementation efficiency, which are modeled and measured with respect to the number of nodes (n) and parallel methods used. Four parallel algorithms were developed, of which the first two exploited the spatial parallelism in each frame, and the last two exploited both the temporal and spatial parallelism over a sequence of frames. The two spatial algorithms differ in that one utilizes a single communication master, while the other attempts to distribute communications across three masters. On the other hand, the spatial-temporal algorithms use a pipeline structure for exploiting the temporal parallelism together with either a single master or multiple masters. The best median speedup (frame rate) achieved was close to 15 [15 frames per second (fps)] for 352/spl times/240 video on 24 nodes, and 13 (37 fps) for QCIF video, by the spatial algorithm with distributed communications. For n<10, the single-master spatial algorithm performs better with efficiency up to 90%, while the multiple-master spatial algorithm is superior for n>10, with efficiency up to 70%. The spatial-temporal algorithms achieved average speedup performance, but are most scalable for large n.

High-performance file I/O in Java: Existing approaches and bulk I/O extensions

There is a growing interest in using Java as the language for developing high-performance computing applications. To be successful in the high-performance computing domain, however, Java must not only be able to provide high computational performance, but also high-performance I/O. In this paper, we first examine several approaches that attempt to provide high-performance I/O in Java—many of which are not obvious at first glance—and evaluate their performance on two parallel machines, the IBM SP and the SGI Origin2000. We then propose extensions to the Java I/O library that address the deficiencies in the Java I/O API and improve performance dramatically. The extensions add bulk (array) I/O operations to Java, thereby removing much of the overhead currently associated with array I/O in Java. We have implemented the extensions in two ways: in a standard JVM using the Java Native Interface (JNI) and in a high-performance parallel dialect of Java called Titanium. We describe the two implementations and present performance results that demonstrate the benefits of the proposed extensions. Copyright © 2001 John Wiley & Sons, Ltd.

Three parallel programming paradigms: comparisons on an archetypal PDE computation

Three paradigms for distributed-memory parallel computation that free application programmer from details of message passing are compared for an archetypal structured scientific computation -- a nonlinear, structured-grid partial differential equation boundary value problem -- using same algorithm on same hardware. All of paradigms -- parallel languages represented by Portland Group's HPF, (semi-)automated serial-to-parallel source-to-source translation represented by CAP-Tools from University of Greenwich, and parallel libraries represented by Argonne's PETSc -- are found to be easy to use for this problem class, and all are reasonably effective in exploiting concurrency after a short learning curve. The level of involvement required by application programmer under any paradigm includes specification of data partitioning, corresponding to a geometrically simple decomposition of domain of PDE. Programming in SPMD style for PETSc library requires writing only routines that discretize PDE and its Jacobian, managing subdomain-to-processor mappings (affine global-to-local index mappings), and interfacing to library solver routines. Programming for HPF requires a complete sequential implementation of same algorithm as a starting point, introduction of concurrency through subdomain blocking (a task similar to index mapping), and modest experimentation with rewriting loops to elucidate to compiler latent concurrency. Programming with CAPTools involves feeding same sequential implementation to CAPTools interactive parallelization system, and guiding source-to-source code transformation by responding to various queries about quantities knowable only at runtime. Results representative of the state of practice for a scaled sequence of structured grid problems are given on three of most important contemporary high-performance platforms: IBM SP, SGI Origin 2000, and CRAYY T3E.

Overlapping Communication and Computation with OpenMP and MPI

Machines comprised of a distributed collection of shared memory or SMP nodes are becoming common for parallel computing. OpenMP can be combined with MPI on many such machines. Motivations for combing OpenMP and MPI are discussed. While OpenMP is typically used for exploiting loop‐level parallelism it can also be used to enable coarse grain parallelism, potentially leading to less overhead. We show how coarse grain OpenMP parallelism can also be used to facilitate overlapping MPI communication and computation for stencil‐based grid programs such as a program performing Gauss‐Seidel iteration with red‐black ordering. Spatial subdivision or domain decomposition is used to assign a portion of the grid to each thread. One thread is assigned a null calculation region so it was free to perform communication. Example calculations were run on an IBM SP using both the Kuck &amp; Associates and IBM compilers.

A Fully Asynchronous Multifrontal Solver Using Distributed Dynamic Scheduling

In this paper, we analyze the main features and discuss the tuning of the algorithms for the direct solution of sparse linear systems on distributed memory computers developed in the context of a long term European research project. The algorithms use a multifrontal approach and are especially designed to cover a large class of problems. The problems can be symmetric positive definite, general symmetric, or unsymmetric matrices, both possibly rank deficient, and they can be provided by the user in several formats. The algorithms achieve high performance by exploiting parallelism coming from the sparsity in the problem and that available for dense matrices. The algorithms use a dynamic distributed task scheduling technique to accommodate numerical pivoting and to allow the migration of computational tasks to lightly loaded processors. Large computational tasks are divided into subtasks to enhance parallelism. Asynchronous communication is used throughout the solution process to efficiently overlap communication with computation. We illustrate our design choices by experimental results obtained on an SGI Origin 2000 and an IBM SP2 for test matrices provided by industrial partners in the PARASOL project.

Parallel Sparse Supports for Array Intrinsic Functions of Fortran 90

Fortran 90 provides a rich set of array intrinsic functions. Each of these array intrinsic functions operates on the elements of multi-dimensional array objects concurrently. They provide a rich source of parallelism and play an increasingly important role in automatic support of data parallel programming. However, there is no such support if these intrinsic functions are applied to sparse data sets. In this paper, we address this open gap by presenting an efficient library for parallel sparse computations with Fortran 90 array intrinsic operations. Our method provides both compression schemes and distribution schemes on distributed memory environments applicable to higher-dimensional sparse arrays. This way, programmers need not worry about low-level system details when developing sparse applications. Sparse programs can be expressed concisely using array expressions, and parallelized with the help of our library. Our sparse libraries are built for array intrinsics of Fortran 90, and they include an extensive set of array operations such as CSHIFT, EOSHIFT, MATMUL, MERGE, PACK, SUM, RESHAPE, SPREAD, TRANSPOSE, UNPACK, and section moves. Our work, to our best knowledge, is the first work to give sparse and parallel sparse supports for array intrinsics of Fortran 90. In addition, we provide a complete complexity analysis for our sparse implementation. The complexity of our algorithms is in proportion to the number of nonzero elements in the arrays, and that is consistent with the conventional design criteria for sparse algorithms and data structures. Our current testbed is an IBM SP2 workstation cluster. Preliminary experimental results with numerical routines, numerical applications, and data-intensive applications related to OLAP (on-line analytical processing) show that our approach is promising in speeding up sparse matrix computations on both sequential and distributed memory environments if the programs are expressed with Fortran 90 array expressions.

Characterizing land surface anisotropy from AVHRR data at a global scale using high performance computing

We used the multi-temporal ten-day composite data from the Advanced Very High Resolution Radiometer (AVHRR) for the years 1983 to 1986 to retrieve the Bidirectional Reflectance Distribution Function (BRDF) using high performance computing techniques. Three different models are used: a simple linear model, a semi-empirical iterative model and a temporal model. The objectives of this study were to compare the performance of different BRDF models at a global scale, assess the computational requirements and optimize the algorithm implementation using high performance computational techniques, and to determine if there is any coherent spatial structure in the coefficients of different BRDF models corresponding to different land cover types. The standard error between model computed reflectances and the input data was used to quantify the performance of the models. Even though the iterative model is computationally more expensive (158 minutes) than either the simple linear model (15 minutes) or the temporal model (16 minutes), the results from all the three models were very similar when the BRDF was estimated at discrete time periods. If the BRDF models were applied without dividing the input data into discrete time intervals, then the temporal model gave better results than the other two. All the models were run on an IBM SP2 parallel machine with 16 CPUs. Most of the mountainous and snow covered areas in high latitudes had null values since the cloud screening algorithm used in the Pathfinder processing performed poorly in distinguishing between snow and clouds. The BRDF coefficients of the iterative model and the Fourier coefficients of the temporal model showed a strong spatial structure corresponding to known variations in land cover.

Parallel processing of adaptive meshes with load balancing

Many scientific applications involve grids that lack a uniform underlying structure. These applications are often also dynamic in nature in that the grid structure significantly changes between successive phases of execution. In parallel computing environments, mesh adaptation of unstructured grids through selective refinement/coarsening has proven to be an effective approach. However, achieving load balance while minimizing interprocessor communication and redistribution costs is a difficult problem. Traditional dynamic load balancers are mostly inadequate because they lack a global view of system loads across processors. In this paper, we propose a novel and general-purpose load balancer that utilizes symmetric broadcast networks (SBN) as the underlying communication topology and compare its performance with a successful global load balancing environment, called PLUM, specifically created to handle adaptive unstructured applications. Our experimental results on an IBM SP2 demonstrate that the SBN-based load balancer achieves lower redistribution costs than that under PLUM by overlapping processing and data migration.

Scheduling loops with partial loop-carried dependencies

This paper deals with task scheduling, where each task is one particular iteration of a DO loop with partial loop-carried dependencies. Independent iterations of such loops can be scheduled in an order different from the one of classical serial execution, so as to increase program performance. The approach that we present is based both on the use of a directive added to the High Performance Fortran (HPF2) language, which specifies the dependencies between iterations, and on inspector/executor support, implemented in the C oLUMBO library, which builds the task graph and schedules tasks associated with iterations. We validate our approach by showing results achieved on an IBM SP2 for a sparse Cholesky factorization algorithm applied to real problems.

Design of parallel algorithms for the single resource allocation problem

Three new optimal parallel algorithms are presented for the single resource allocation problem. They run in the simplest networks: pipelines and rings. All of them have been implemented using PVM and MPI. Four representative platforms of the current state of parallel computing hardware were used for the experiences presented in this paper: the IBM SP2, the Cray T3E, the Silicon Origin 2000 and the Digital Alpha Server 8400. Computational results prove the good scalability of these three algorithms for practical cases.

Multi-scale meshfree parallel computations for viscous, compressible flows

In this paper, a Petrov–Galerkin formulation for a viscous, compressible fluid using the meshless reproducing kernel particle method (RKPM) is first presented. The location of shocks is determined from the high-scale part of the solution using multiresolution analysis. A parallel computer implementation for distributed memory architectures is then proposed, including a discussion of handling boundary conditions in parallel. Preliminary results for 1–16 processors of an IBM SP computer are presented, and future directions for a massively parallel implementation are outlined.

Compiler Algorithms for Optimizing Locality and Parallelism on Shared and Distributed-Memory Machines

Distributed-memory message-passing machines deliver scalable performance but are difficult to program. Shared-memory machines, on the other hand, are easier to program but obtaining scalable performance with large number of processors is difficult. Recently, scalable machines based on logically shared physically distributed memory have been designed and implemented. While some of the performance issues like parallelism and locality are common to different parallel architectures, issues such as data distribution are unique to specific architectures. One of the most important challenges compiler writers face is the design of compilation techniques that can work well on a variety of architectures. In this paper, we propose an algorithm that can be employed by optimizing compilers for different types of parallel architectures. Our optimization algorithm does the following: (1) transforms loop nests such that, where possible, the iterations of the outermost loops can be run in parallel across processors; (2) optimizes memory locality by carefully distributing each array across processors; (3) optimizes interprocessor communication using message vectorization whenever possible; and (4) optimizes cache locality by assigning appropriate storage layout for each array and by transforming the iteration space. Depending on the machine architecture, some or all of these steps can be applied in a unified framework. We report empirical results on an SGI Origin 2000 distributed-shared-memory multiprocessor and an IBM SP-2 distributed-memory message-passing machine to validate the effectiveness of our approach.

Solving algebraic Riccati equations on parallel computers using Newton's method with exact line search

We investigate the numerical solution of continuous-time algebraic Riccati equations via Newton's method on serial and parallel computers with distributed memory. We apply and extend the available theory for Newton's method endowed with exact line search to accelerate convergence. We also discuss a new stopping criterion based on recent observations regarding condition and error estimates. In each iteration step of Newton's method a stable Lyapunov equation has to be solved. We propose to solve these Lyapunov equations using iterative schemes for computing the matrix sign function. This approach can be efficiently implemented on parallel computers using ScaLAPACK. Numerical experiments on an ibm sp2 multicomputer report on the accuracy, scalability, and speed-up of the implemented algorithms.