Good Parallel Performance Research Articles

On the parallelization of multi-grid methods using a non-overlapping domain decomposition data structure

In the paper, the parallelization of multi-grid methods for solving second-order elliptic boundary value problems in two-dimensional domains is discussed. The parallelization strategy is based on a non-overlapping domain decomposition data structure such that the algorithm is well-suited for an implementation on a parallel machine with MIMD architecture. For getting an algorithm with a good parallel performance it is necessary to have as few communication as possible. In our implementation, communication is only needed within the smoothing procedures and the coarse-grid solver. The interpolation and restriction procedures can be performed without any communication. New variants of smoothers of Gauss-Seidel type having the same communication cost as Jacobi smoothers are presented. For solving the coarse-grid systems iterative methods are proposed that are applied to the corresponding Schur complement system. A numerical example, namely a plane linear elasticity problem, demonstrates the efficiency of the parallel multi-grid algorithm.

List Ranking and List Scan on the CRAYC90

Although parallel algorithms using linked lists, trees, and graphs have been studied extensively by the research community, implementations have met with limited success, even for the simplest algorithms. In this paper we present our results of a careful implementation study of parallel list ranking (and the related list-scan operation) and show that it can have substantial speed up over fast workstations. Obtaining good parallel performance for list ranking is a challenge for two reasons. First, although there are many asymptotic work-efficient algorithms, it is hard to keep the constants comparable to those of the sequential algorithm. In this paper we introduce a new parallel algorithm that both is work efficient and has small constant but sacrifices logarithmic time; it achieves only anO(log2n) time. We contend, however, that work efficiency and small constants are more important, given that multiprocessor machines are used for problems that are much larger than the number of processors and, therefore, the optimalO(logn) time is never achieved in practice. Second, list ranking is highly communication intensive and its memory access patterns are dynamic. We show, however, that by using high memory bandwidth multiprocessors, such as CrayC90 computers, programmed with virtual processors to hide latency we can ameliorate these problems. To the best of our knowledge, our implementation of list ranking and list scan on the CrayC90 is the fastest to date and is the first implementation that substantially outperforms fast workstations. The success of our algorithm is due to its moderate grain size and simplicity; the success of the implementation is due to pipelining reads and writes through vectorization to hide latency and optimizing performance by analyzing the expected execution time of the algorithm.

Understanding application performance on shared virtual memory systems

Many researchers have proposed interesting protocols for shared virtual memory (SVM) systems, and demonstrated performance improvements on parallel programs. However, there is still no clear understanding of the performance potential of SVM systems for different classes of applications. This paper begins to fill this gap, by studying the performance of a range of applications in detail and understanding it in light of application characteristics.We first develop a brief classification of the inherent data sharing patterns in the applications, and how they interact with system granularities to yield the communication patterns relevant to SVM systems. We then use detailed simulation to compare the performance of two SVM approaches---Lazy Released Consistency (LRC) and Automatic Update Release Consistency (AURC)---with each other and with an all-hardware CC-NUMA approach. We examine how performance is affected by problem size, machine size, key system parameters, and the use of less optimized program implementations. We find that SVM can indeed perform quite well for systems of at leant up to 32 processors for several nontrivial applications. However, performance is much more variable across applications than on CC-NUMA systems, and the problem sizes needed to obtain good parallel performance are substantially larger. The hardware-assisted AURC system tends to perform significantly better than the all-software LRC under our system assumptions, particularly when realistic cache hierarchies are used.

What Is Good Parallel Performance? And How Do We Get It?

What Is Good Parallel Performance? And How Do We Get It?

Massively Parallel Self-Consistent-Field Calculations

The advent of supercomputers with many computational nodes each with its own independent memory makes possible very fast computations. Our work is focused on the development of electronic structure techniques for the solution of Grand Challenge-size molecules containing hundreds of atoms. Our efforts have resulted in a scalable direct-SCF (self-consistent-field) Fock matrix construction kernel that is portable and efficient. Good parallel performance is obtained by allowing asynchronous communications and using a distributed data model. These requirements are easily handled by using the Global Array (GA) software library developed for this project. This algorithm has been incorporated into the new program NWChem.

Distributed Sparse Gaussian Elimination and Orthogonal Factorization

A unified framework is presented for a fully parallel solution of large, sparse nonsymmetric linear systems on distributed memory multiprocessors. Unlike earlier work, both symbolic and numeric steps are parallelized. Parallel Cartesian nested dissection is used to compute a fill-reducing ordering of A using a compact representation of the column intersection graph, and the resulting separator tree is used to estimate the structure of the factor and to distribute data and perform multifrontal numeric computations. When the matrix is nonsymmetric but square, the numeric computations involve Gaussian elimination with partial pivoting; when the matrix is overdetermined, row-oriented Householder transforms are applied to compute the triangular factor of an orthogonal factorization. Extensive empirical results are provided to demonstrate that the approach is effective both in preserving sparsity and achieving good parallel performance on an Intel iPSC/860.

Load Balancing and Data Locality in Adaptive Hierarchical N-Body Methods: Barnes-Hut, Fast Multipole, and Radiosity

Hierarchical N-body methods, which are based on a fundamental insight into the nature of many physical processes, are increasingly being used to solve large-scale problems in a variety of scientific/engineering domains. Applications that use these methods are challenging to parallelize effectively, however, owing to their nonuniform, dynamically changing characteristics and their need for long-range communication. In this paper, we study the partitioning and scheduling techniques required to obtain effective parallel performance on applications that use a range of hierarchical N-body methods. To obtain representative coverage, we first examine applications that use the two best methods known for classical N-body problems: the Barnes-Hut method and the fast multipole method. Then, we examine a recent hierarchical method for radiosity calculations in computer graphics, which applies the hierarchical N-body approach to a problem with very different characteristics. We find that straightforward decomposition techniques which an automatic scheduler might implement do not scale well, because they are unable to simultaneously provide load balancing and data locality. However, all the applications yield very good parallel performance if appropriate partitioning and scheduling techniques are implemented by the programmer. For the applications that use the Barnes-Hut and fast multipole methods, simple yet effective partitioning techniques can be developed by exploiting some key insights into both the methods and the classical problems that they solve. Using a novel partitioning technique, even relatively small problems achieve 45-fold speedups on a 48-processor Stanford DASH machine (a cache-coherent, shared address space multiprocessor) and 118-fold speedups on a 128-processor simulated architecture. The very different characteristics of the radiosity application require a different partitioning/scheduling approach to be used for it; however, it too yields very good parallel performance.

Implementation of the lanczos method for structural vibration analysis on a parallel computer

The eigenvalue problem associated with structural vibration analysis is a major, computationally-intensive activity in large-scale finite element calculations. Advances in parallel computers together with appropriate solution methods have the potential for providing high-speed computational power to aid eigenvalue solutions for these large problems. The key to exploiting this potential is the development of appropriate methods tailored for such parallel computers. This paper reports on experiences from a study involving the implementation of the Lanczos method on a parallel computer. The results of this study show that introducing shifts, assigning each processor a different region in the eigenvalue spectrum, and implementing the Lanczos calculation steps in parallel is an effective strategy for speeding up calculations. This approach provides good parallel performance and easy balance of processor workload. Two example vibration problems were solved to assess the behavior of the Lanczos implementation. The test-problem results include examples of the Lanczos phenomenon where lack of orthogonality in the vectors can result in spurious eigenvalues. Tests were incorporated in the parallel calculations which detected these spurious eigenvalues. The parallel eigenvalue algorithm demonstrates that significant speedups in calculation time can be realized over traditional sequential methods.