Design Of Parallel Algorithms Research Articles

Improved QMRCGSTAB method in distributed parallel environments

An improved version of the QMRCGSTAB method (IQMRCGSTAB method) for solving large sparse linear systems with non-symmetric coefficient matrices is proposed. The proposed method combines elements of numerical stability and parallel algorithm design without increasing computational costs. Performance analysis shows that the IQMRCGSTAB method has better parallelism and scalability than the original method. Numerical experiments show the efficiency of the new method.

Heuristics for work distribution of a homogeneous parallel dynamic programming scheme on heterogeneous systems

In this paper the possibility of including automatic optimization techniques in the design of parallel dynamic programming algorithms in heterogeneous systems is analyzed. The main idea is to automatically approach the optimum values of a number of algorithmic parameters (number of processes, number of processors, processes per processor), and thus obtain low execution times. Hence, users could be provided with routines which execute efficiently, and independently of the experience of the user in heterogeneous computing and dynamic programming, and which can adapt automatically to a new network of processors or a new network configuration.

A parallel version of QMRCGSTAB method for large linear systems in distributed parallel environments

In this paper, a parallel QMRCGSTAB method (PQMRCGSTAB method) for solving large sparse linear systems with unsymmetrical coefficient matrices is proposed for distributed parallel environments. The method reduces four global synchronization points to one by reconstructing QMRCGSTAB method. It combines the elements of numerical stability with the characters of design of parallel algorithms. The cost is only a little increased computation. Performance analysis shows that PQMRCGSTAB method has better parallelism and scalability than QMRCGSTAB method. Numerical experiments show the effectiveness of our method.

PLX: An Instruction Set Architecture and Testbed for Multimedia Information Processing

PLX is a concise instruction set architecture (ISA) that combines the most useful features from previous generations of multimedia instruction sets with newer ISA features for high-performance, low-cost multimedia information processing. Unlike previous multimedia instruction sets, PLX is not added onto a base processor ISA, but designed from the beginning as a standalone processor architecture optimized for media processing. Its design goals are high performance multimedia processing, general-purpose programmability to support an ever-growing range of applications, simplicity for constrained environments where low power and low cost are paramount, and scalability for higher performance in less constrained multimedia systems. Another design goal of PLX is to facilitate exploration and evaluation of novel techniques in instruction set architecture, microarchitecture, arithmetic, VLSI implementations, compiler optimizations, and parallel algorithm design for new computing paradigms. Key characteristics of PLX are a fully subword-parallel architecture with novel features like wordsize scalability from 32-bit to 128-bit words, a new definition of predication, and an innovative set of subword permutation instructions. We demonstrate the use and high performance of PLX on some frequently-used code kernels selected from image, video, and graphics processing applications: discrete cosine transform, pixel padding, clip test, and median filter. Our results show that a 64-bit PLX processor achieves significant speedups over a basic 64-bit RISC processor and over IA-32 processors with MMX and SSE multimedia extensions. Using PLX's wordsize scalability feature, PLX-128 often provides an additional 2× speedup over PLX-64 in a cost-effective way. Superscalar or VLIW (Very Long Instruction Word) PLX implementations can also add additional performance through inter-instruction, rather than intra-instruction parallelism. We also describe the PLX testbed and its software tools for architecture and related research.

SCHEDULING ON LARGE SCALE DISTRIBUTED PLATFORMS: FROM MODELS TO IMPLEMENTATIONS

Today, large scale parallel systems are available at low cost, Many powerful such systems have been installed all over the world and the number of users is always increasing. The difficulty of using them efficiently is growing with the complexity of the interactions between more and more architectural constraints and the diversity of the applications. The design of efficient parallel algorithms has to be reconsidered under the influence of new parameters of such platforms (namely, cluster, grid and global computing) which are characterized by a larger number of heterogeneous processors, often organized in several hierarchical sub-systems. At each step of the evolution of the parallel processing field, researchers designed adequate computational models whose objective was to abstract the real world in order to be able to analyze the behavior of algorithms. In this paper, we will investigate two complementary computational models that have been proposed recently: Parallel Task (PT) and Divisible Load (DL). The Parallel Task (i.e. tasks that require more than one processor for their execution) model is a promising alternative for scheduling parallel applications, especially in the case of slow communication media. The basic idea is to consider the application at a coarse level of granularity. Another way of looking at the problem (which is somehow a dual view) is the Divisible Load model where an application is considered as a collection of a large number of elementary – sequential – computing units that will be distributed among the available resources. Unlike the PT model, the DL model corresponds to a fine level of granularity. We will focus on the PT model, and discuss how to mix it with simple Divisible Load scheduling. As the main difficulty for distributing the load among the processors (usually known as the scheduling problem) in actual systems comes from handling efficiently the communications, these two models of the problem allow us to consider them implicitly or to mask them, thus leading to more tractable problems. We will show that in spite of the enormous complexity of the general scheduling problem on new platforms, it is still useful to study theoretical models. We will focus on the links between models and actual implementations on a regional grid with more than 500 processors.

Efficient parallel algorithms and software for compressed octrees with applications to hierarchical methods

We describe the design and implementation of efficient parallel algorithms, and a software library for the parallel implementation of compressed octree data structures. Octrees are widely used in supporting hierarchical methods for scientific applications such as the N-body problem, molecular dynamics and smoothed particle hydrodynamics. The primary goal of our work is to identify and abstract the commonalities present in various hierarchical methods using octrees, design efficient parallel algorithms for them, and encapsulate them in a software library. We designed provably efficient parallel algorithms and implementation strategies that perform well irrespective of the spatial distribution of data in the computational domain. The library will enable rapid development of applications, allowing application developers to use efficient parallel algorithms developed for this purpose, without the necessity of having detailed knowledge of the algorithms or of implementing them. The software is developed in C using the Message Passing Interface (MPI). We report experimental results on an IBM xSeries parallel computer.

New Consideration on the Evaluation Model of Cluster Area Network

Traditional Cluster Area Network(cLAN)'s evaluation model takes only latency, bandwidth, routing, congestion, network topology and some related aspects into consideration. Are these factors ENOUGH to describe the real applications' communication behavior or predict its performance on cLAN? In the large quantity of NAS Parallel Benchmarks' tests(version 2.4) on a modern supercomputer——DeepComp 1800, which is of LINUX Cluster architecture, it is found that the real performance of cLAN could be greatly affected by a special communication pattern(LU pattern). Further investigation reveals that the cLAN's capacity of dealing with LU mode is independent of the known performance factors such as latency, bandwidth and so on. So it is necessary to take some new considerations on cLAN's evaluation model and add one new factor to reflect the abnormal phenomenon. The new model also provides some challenges in parallel algorithm design and application performance improvement on the LINUX Cluster.

Parallel microgenetic algorithm design for photonic crystal and waveguide structures.

We have developed a powerful parallel genetic algorithm design tool for photonic crystal and waveguide structures. The tool employs a small-population-size genetic algorithm (microgenetic algorithm) for global optimization and a two-dimensional finite-difference time-domain method to rigorously design and optimize the performance of photonic devices. We discuss the implementation and performance of this design tool. We demonstrate its application to two photonic devices, a defect taper coupler to connect conventional waveguides and photonic crystal waveguides, and a sharp 90 degrees waveguide bend for low index contrast waveguides.

Emulations between QSM, BSP and LogP: a framework for general-purpose parallel algorithm design

We present work-preserving emulations with small slowdown between LogP and two other parallel models: BSP and QSM. In conjunction with earlier work-preserving emulations between QSM and BSP, these results establish a close correspondence between these three general-purpose parallel models. Our results also correct and improve on results reported earlier on emulations between BSP and LogP. In particular we shed new light on the relative power of stalling and non-stalling LogP models. The QSM is a shared-memory model with only two parameters— p, the number of processors, and g, a bandwidth parameter. The simplicity of the QSM parameters makes QSM a convenient model for parallel algorithm design, and simple work-preserving emulations of QSM on BSP and QSM on LogP show that algorithms designed for the QSM will also map quite well to these other models. The simplicity and generality of QSM present a strong case for the use of QSM as the model of choice for parallel algorithm design. We present QSM algorithms for three basic problems—prefix sums, sample sort and list ranking. We show that these algorithms are optimal in terms of both the total work performed and the number of ‘phases’ for input sizes of practical interest. For prefix sums, we present a matching lower bound that shows our algorithm to be optimal over the complete range of these parameters. We then examine the predicted and simulated performance of these algorithms. These results suggest that QSM analysis will predict algorithm performance quite accurately for problem sizes that arise in practice.

BSPGRID: VARIABLE RESOURCES PARALLEL COMPUTATION AND MULTIPROGRAMMED PARALLELISM

This paper introduces a new framework for the design of parallel algorithms that may be executed on multiprogrammed architectures with variable resources. These features, in combination with an implied ability to handle fault tolerance, facilitates environments such as the GRID. A new model, BSPGRID is presented, which exploits the bulk synchronous paradigm to allow existing algorithms to be easily adapted and used. It models computation, communication, external memory accesses (I/O) and synchronization. By combining the communication and I/O operations BSPGRID allows the easy design of portable algorithms while permitting them to execute on non-dedicated hardware and/or changing resources, which is typical for machines in a GRID. However, even with this degree of dynamicity, the model still offers a simple and tractable cost model. Each program runs in its own virtual BSPGRID machine. Its emulation on a real computer is demonstrated to show the practicality of the framework. A dense matrix multiplication algorithm and its emulation in a multiprogrammed environment is given as an example.

Parallel Complexity of Matrix Multiplication

Effective design of parallel matrix multiplication algorithms relies on the consideration of many interdependent issues based on the underlying parallel machine or network upon which such algorithms will be implemented, as well as, the type of methodology utilized by an algorithm. In this paper, we determine the parallel complexity of multiplying two (not necessarily square) matrices on parallel distributed-memory machines and/or networks. In other words, we provided an achievable parallel run-time that can not be beaten by any algorithm (known or unknown) for solving this problem. In addition, any algorithm that claims to be optimal must attain this run-time. In order to obtain results that are general and useful throughout a span of machines, we base our results on the well-known LogP model. Furthermore, three important criteria must be considered in order to determine the running time of a parallel algorithm; namely, (i) local computational tasks, (ii) the initial data layout, and (iii) the communication schedule. We provide optimality results by first proving general lower bounds on parallel run-time. These lower bounds lead to significant insights on (i)–(iii) above. In particular, we present what types of data layouts and communication schedules are needed in order to obtain optimal run-times. We prove that no one data layout can achieve optimal running times for all cases. Instead, optimal layouts depend on the dimensions of each matrix, and on the number of processors. Lastly, optimal algorithms are provided.

Parallel Algorithms for Dynamic Shortest Path Problems

The development of intelligent transportation systems (ITS) and the resulting need for the solution of a variety of dynamic traffic network models and management problems require faster‐than‐real‐time computation of shortest path problems in dynamic networks. Recently, a sequential algorithm was developed to compute shortest paths in discrete time dynamic networks from all nodes and all departure times to one destination node. The algorithm is known as algorithm DOT and has an optimal worst‐case running‐time complexity. This implies that no algorithm with a better worst‐case computational complexity can be discovered. Consequently, in order to derive algorithms to solve all‐to‐one shortest path problems in dynamic networks, one would need to explore avenues other than the design of sequential solution algorithms only. The use of commercially‐available high‐performance computing platforms to develop parallel implementations of sequential algorithms is an example of such avenue. This paper reports on the design, implementation, and computational testing of parallel dynamic shortest path algorithms. We develop two shared‐memory and two message‐passing dynamic shortest path algorithm implementations, which are derived from algorithm DOT using the following parallelization strategies: decomposition by destination and decomposition by transportation network topology. The algorithms are coded using two types of parallel computing environments: a message‐passing environment based on the parallel virtual machine (PVM) library and a multi‐threading environment based on the SUN Microsystems Multi‐Threads (MT) library. We also develop a time‐based parallel version of algorithm DOT for the case of minimum time paths in FIFO networks, and a theoretical parallelization of algorithm DOT on an ‘ideal’ theoretical parallel machine. Performances of the implementations are analyzed and evaluated using large transportation networks, and two types of parallel computing platforms: a distributed network of Unix workstations and a SUN shared‐memory machine containing eight processors. Satisfactory speed‐ups in the running time of sequential algorithms are achieved, in particular for shared‐memory machines. Numerical results indicate that shared‐memory computers constitute the most appropriate type of parallel computing platforms for the computation of dynamic shortest paths for real‐time ITS applications.

A parallel algorithm for the eight-puzzle problem using analogical reasoning

The emphasis in this article will be on people's analogical use of fairly explicit pieces of knowledge in problem solving situations. Analogical reasoning (AR) has long been recognized as an important component of problem solving. In general, AR involves using the (possibly modified) solution of one problem to solve another. The preconditions that need to be satisfied in order to apply AR are to locate the common features that are shared among the two problems. The case study considered here is the eight-puzzle problem. The underlying learning is a three-stage process. First, is the building stage, where heuristic search techniques are used to find solutions to a set of initial puzzles. The next stage, the reminding stage, identifies a group of solutions with similar properties to the input puzzle (supposedly, this puzzle is solvable). During the final stage, the analogical transformation stage, many movement operators are applied to similar solutions already extracted from the reminding stage, in order to find the solution to the original (input) puzzle. One of the major drawbacks is that AR requires a lot of computational power for global searching before it can be converged. Therefore, in order to outperform heuristic algorithms, a parallel version of AR is introduced. Parallel computation is focused on the last two stages of AR. A design and analysis of parallel algorithms of the reminding stage and the analogical transformation stage are presented. © 2002 Wiley Periodicals, Inc.

Architecture independent parallel algorithm design: theory vs practice

We propose architecture independent parallel algorithm design as a framework for writing parallel code that is scalable, portable and reusable. Towards this end we study the performance of some dense matrix computations such as matrix multiplication, LU decomposition and matrix inversion. Although optimized algorithms for these problems have been extensively examined before, a systematic study of an architecture independent design and analysis of parallel algorithms and their performance (including matrix computations) has not been undertaken. Even though more refined algorithms and implementations (sequential or parallel) for the stated problems exist, the complexity and performance of the introduced algorithms is sufficient to raise the issues that are important in architecture independent parallel algorithm design. Two established distributions of an input matrix among the processors of a parallel machine are examined and the particular theoretical and practical merits of each one are also discussed. The algorithms we propose have been implemented and tested on a variety of parallel systems that include the SGI Power Challenge, the IBM SP2 and the Cray T3D. Our experimental results support our claims of efficiency, portability and reusability of the presented algorithms.

Scalable Atomistic Simulation Algorithms for Materials Research

A suite of scalable atomistic simulation programs has been developed for materials research based on space‐time multiresolution algorithms. Design and analysis of parallel algorithms are presented for molecular dynamics (MD) simulations and quantum‐mechanical (QM) calculations based on the density functional theory. Performance tests have been carried out on 1,088‐processor Cray T3E and 1,280‐processor IBM SP3 computers. The linear‐scaling algorithms have enabled 6.44‐billion‐atom MD and 111,000‐atom QM calculations on 1,024 SP3 processors with parallel efficiency well over 90%. production‐quality programs also feature wavelet‐based computational‐space decomposition for adaptive load balancing, spacefilling‐curve‐based adaptive data compression with user‐defined error bound for scalable I/O, and octree‐based fast visibility culling for immersive and interactive visualization of massive simulation data.

Quantitative performance analysis of the improved quasi-minimal residual method on massively distributed memory computers

For the solutions of linear systems of equations with unsymmetric coefficient matrices, we have proposed an improved version of the quasi-minimal residual (IQMR) method [Proceedings of The International Conference on High Performance Computing and Networking (HPCN-97) (1997); IEICE Trans Inform Syst E80-D (9) (1997) 919] by using the Lanczos process as a major component combining elements of numerical stability and parallel algorithm design. For the Lanczos process, stability is obtained by a coupled two-term procedure that generates Lanczos vectors scaled to unit length. The algorithm is derived so that all inner products and matrix–vector multiplications of a single iteration step are independent and the communication time required for inner product can be overlapped efficiently with computation time. In this paper, a theoretical model of computation and communications phases is presented to allow us to give a quantitative analysis of the parallel performance with a two-dimensional grid topology. The efficiency, speed-up, and runtime are expressed as functions of the number of processors scaled by the number of processors that gives the minimal runtime for the given problem size. The model not only evaluates effectively the improvements in performance due to communication reduction by overlapping, but also provides useful insight into the scalability of the IQMR method. The theoretical results on the performance are demonstrated by experimental timing results carried out on a massively parallel distributed memory Parsytec system.

Linear-scaling density-functional-theory calculations of electronic structure based on real-space grids: design, analysis, and scalability test of parallel algorithms

We have implemented parallel algorithms for density-functional-theory (DFT) based electronic-structure calculations. These include a plane-wave based algorithm, a real-space-grid algorithm based on a high-order finite difference method, and a linear-scaling real-space algorithm using localized orbitals. Parallelization schemes are described for these algorithms, and the computational complexity and the communications involved in the resulting parallel algorithms are analyzed. Scalability tests of these algorithms on massively parallel computers show that the linear-scaling DFT algorithm is highly scalable. For a 110,592-atom gallium arsenide system on 1024 IBM SP3 processors, the parallel efficiency is as high as 93%.

Identity-plus-row matrix decomposition and its application in design of parallel projection algorithms

Many image processing operations can be abstracted into matrix operations. With the help of matrix analysis, we can understand the inherent properties of the operations and thus design better algorithms. In this paper, we propose a matrix decomposition method referred to as identity-plus-row decomposition. The decomposition is particularly useful in design of parallel projection algorithms on mesh-connected computers. Projection is a frequently used process in image processing and visualization. In volume graphics, projection is used to render the essential content of a three-dimensional volume onto a two-dimensional image plane. For Radon transform, projection is used to transform the image space into a parameter space. By applying the identity-plus-row matrix decomposition method, we solve the data redistribution problem due to the irregular data access patterns present in those applications on single instruction stream, multiple data stream (SIMD) meshconnected computers, developing fast algorithms for volume rendering and Radon transform on SIMD mesh-connected computers.

Molecular models in computer simulation of liquid crystals

A brief historical survey of progress in computer simulations of liquid crystal phases is provided. We stress the main ideas and developments in the field, concentrating on three different levels of molecular modelling: lattice models, soft nonspherical particles and atomistic models. Emphasis is placed on current state-of-the-art developments including a new parallel molecular dynamics algorithm design for the study of flexible liquid crystals.

Merging on the BSP model

The design of complex parallel algorithms relies heavily on a set of primitive operations. In this work, we examine the problem of merging two sorted sequences in an architecture independent setting. We derive parallel algorithms that can be used on a variety of parallel machines and whose performance can be reliably predicted if these algorithms are analyzed under the bulk-synchronous parallel (BSP) model. While our algorithms are fairly simple themselves, description of their performance in terms of the BSP parameters is somewhat complicated. The main reward for quantifying these complications, is that it enables parallel software to be written once and for all that can be migrated efficiently among a variety of parallel platforms. The optimal merging algorithm presented in this work achieves asymptotically optimal parallel efficiency compared to any optimal sequential merging algorithm.