Optimal Speedup Research Articles

Faster output-sensitive parallel algorithms for 3D convex hulls and vector maxima

In this paper we focus on the problem of designing very fast parallel algorithms for the convex hull and the vector maxima problems in three dimensions that are output-size sensitive. Our algorithms achieve O( log log 2 n log h) parallel time and optimal O(n log h) work with high probability in the CRCW PRAM where n and h are the input and output size, respectively. These bounds are independent of the input distribution and are faster than the previously known algorithms. We also present an optimal speed-up (with respect to the input size only) sublogarithmic time algorithm that uses superlinear number of processors for vector maxima in three dimensions.

SCALABLE AND OPTIMAL SPEED-UP PARALLEL ALGORITHMS FOR TEMPLATE MATCHING ON ARRAYS WITH RECONFIGURABLE OPTICAL BUSES

The computational model on which the algorithms are developed is the array with reconfigurable optical buses (AROB). It integrates the advantages of both optical transmission and electronic computation. The main contributions of this paper are in designing several optimal and/or optimal speed-up template matching algorithms with varying degrees of parallelism on the AROB model. For an N × N digitized image and an M × M template, when the domains of the image and the template are O( log N)-bit integers, we first design several basic operations for window broadcasting and rotation. Then based on these basic operations, three efficient and scalable algorithms for template matching are derived using various numbers of processors on a two-dimensional (2-D) or 3-D AROB. For 1 ≤ r ≤ N, 1 ≤ p ≤ M ≤ q ≤ N, one runs in [Formula: see text] time using r × r processors, another runs in [Formula: see text], (resp. [Formula: see text]) time using pN × pN/ log M (resp. pN × pN × log N) processors, and the other runs in [Formula: see text] (resp. [Formula: see text]) time using pq × pq/ log M (or pq × pqN × log N) processors, respectively. The latter two algorithms can be tuned to run in O(1) time on a 2-D AROB. To the best of our knowledge, there are no algorithms which can reach this time complexity for this problem on a 2-D array architecture.

Parallelizing the Data Cube

This paper presents a general methodology for the efficient parallelization of existing data cube construction algorithms. We describe two different partitioning strategies, one for top-down and one for bottom-up cube algorithms. Both partitioning strategies assign subcubes to individual processors in such a way that the loads assigned to the processors are balanced. Our methods reduce inter processor communication overhead by partitioning the load in advance instead of computing each individual group-by in parallel. Our partitioning strategies create a small number of coarse tasks. This allows for sharing of prefixes and sort orders between different group-by computations. Our methods enable code reuse by permitting the use of existing sequential (external memory) data cube algorithms for the subcube computations on each processor. This supports the transfer of optimized sequential data cube code to a parallel setting. The bottom-up partitioning strategy balances the number of single attribute external memory sorts made by each processor. The top-down strategy partitions a weighted tree in which weights reflect algorithm specific cost measures like estimated group-by sizes. Both partitioning approaches can be implemented on any shared disk type parallel machine composed of p processors connected via an interconnection fabric and with access to a shared parallel disk array. We have implemented our parallel top-down data cube construction method in C++ with the MPI message passing library for communication and the LEDA library for the required graph algorithms. We tested our code on an eight processor cluster, using a variety of different data sets with a range of sizes, dimensions, density, and skew. Comparison tests were performed on a SunFire 6800. The tests show that our partitioning strategies generate a close to optimal load balance between processors. The actual run times observed show an optimal speedup of p.

Efficient primitive binary morphological algorithms on a massively parallel processor

One of the most important features in image analysis and understanding is shape. Mathematical morphology is the image pro- cessing branch that deals with shape analysis. The definition of all morphological transformations is based on two primitive operations, i.e., dilation and erosion. Since many applications require the solu- tion of morphological problems in real time, researching time effi- cient algorithms for these two operations is crucial. In this paper, efficient algorithms for the binary dilation and erosion are presented and evaluated for an advanced associative processor. Simulation results show that the proposed algorithms for this advanced archi- tecture reach a near optimal speedup compared to the serial algo- rithm. Additionally, it is proven that the implementation of this image processor is economically feasible. © 2001 SPIE and IS&T.

Branch and bound on the network model

Karp and Zhang developed a general randomized parallel algorithm for solving branch and bound problems. They showed that with high probability their algorithm attained optimal speedup within a constant factor (for p⩽ n/( logn) c , where p is the number of processors, n is the “size” of the problem, and c is a constant). Ranade later simplified the analysis and obtained a better processor bound. Karp and Zhang's algorithm works on models of computation where communication cost is constant. The present paper considers the Branch and Bound problem on networks where the communication cost is high. Suppose sending a message in a p processor network takes G = O( logp) time and node expansion (defined below) takes unit time (other operations being free). Then a simple randomized algorithm is presented which is, asymptotically, nearly optimal for p = O(2 log cn ) , where c is any constant <1/3 and n is the number of nodes in the input tree with cost no greater than the cost of the optimal leaf in the tree.

A new computation of shape moments via quadtree decomposition

The main contribution of this paper is in designing an optimal and/or optimal speed-up algorithm for computing shape moments. We introduce a new technique for computing shape moments. The new technique is based on the quadtree representation of images. We decompose the image into a number of non-overlapped squares, since the moment computation of squares is easier than that of the whole image. The proposed sequential algorithm reduces the computational complexity significantly. By integrating the advantages of both optical transmission and electronic computation, the proposed parallel algorithm can be run in O(1) time using N× N 1+1/ c processors when the input image is complicated. If the input image is simple (i.e., the image can be represented by a few quadtree nodes), the proposed parallel algorithm can be run in O(1) time using N× N processors. In the sense of the product of time and the number of processors used, the proposed parallel algorithm is time and cost optimal and achieves optimal speed-up.

Implementation and evaluation of MPI‐based parallel MD program

AbstractThe message‐passing interface (MPI)‐based object‐oriented particle–particle interactions (PPI) library is implemented and evaluated. The library can be used in the n‐particle simulation algorithm designed for a ring of p interconnected processors. The parallel simulation is scalable with the number of processors, and has the time requirement proportional to n2/p if n/p is large enough, which guarantees optimal speedup. In a certain range of problem sizes, the speedup becomes superlinear because enough cache memory is available in the system. The library is used in a simple way by any potential user, even with no deep programming knowledge. Different simulations using particles can be implemented on a wide spectrum of different computer platforms. The main purpose of this article is to test the PPI library on well‐known methods, e.g., the parallel molecular dynamics (MD) simulation of the monoatomic system by the second‐order leapfrog Verlet algorithm. The performances of the parallel simulation program implemented with the proposed library are competitive with a custom‐designed simulation code. Also, the implementation of the split integration symplectic method, based on the analytical calculation of the harmonic part of the particle interactions, is shown, and its expected performances are predicted. © 2001 John Wiley &amp; Sons, Inc. Int J Quant Chem 84: 23–31, 2001

EMPLOYING K-ARY n-CUBES FOR PARALLEL LAGRANGE INTERPOLATION

This paper proposes a parallel algorithm for computing anN( = Kn) point Lagrange interpolation on fc-ary n-cube networks. The algorithm consists of three phases: initialisation, main and final. There is no computation in the initialisation phase. The main phase is composed of N/2 steps, each consisting of four multiplications and four subtractions, and an additional step including one division and one multiplication. Communication in the main phase is based on an all-to-all broadcast algorithm on a Hamiltonian ring embedded in a k-ary n-cube. The final phase is carried out in n x ⌊k/l⌋ steps, each requiring one addition. A performance evaluation of the proposed algorithm reveals a near to optimum speedup for a typical range of sy:;tem parameters used in current state-of-the-art implementations. Our study also reveals that when implementation cost is taken into account low-dimensional K-ary n-cubes achieve better speedup than their higher-dimensional counterparts.

Simple Optimal Parallel Multiple Pattern Matching

In this paper, we present a simple algorithm for solving the multipattern matching problem, with optimal speedup. The best-known deterministic parallel algorithm for this problem also provides optimal speedup, but relies crucially on a sophisticated construction of an automaton. Since then, Rabin has introduced a simple and elegant parallel algorithm (M. Rabin, in “Sequences '91: Methods in Communication, Security, and Computer Science,” Springer-Verlag, New York/Berlin, 1993). This is a Monte-carlo algorithm based on finger-print functions and it, too, has optimal speedup. Our algorithm simultaneously achieves the goals of optimal speedup of the deterministic algorithm, as well as the simplicity of the randomized Monte-carlo design cited above. Our algorithm presented here can also be extended to solve the multidimensional pattern-matching problem, also with optimal speedup. Interestingly, the sequential version of the algorithm derived by slowing down our parallel design yields a new and simple (linear-time) algorithm for string matching. It is distinguished by its lack of dependence on failure-functions and its related automata-theoretic variants, periodicities, or special data structures, but essentially uses a carefully constructed divide-and-conquer approach. This approach is systematized by us into the shrink-and-spawn technique, with a range of applications in parallel string and pattern matching in a paper that appears in S. Muthukrishnan and K. Palem (in “Proceedings 5th ACM Symp. on Parallel Algorithms and Architectures, 1993,” pp. 69–78).

Iterative application of boundary conditions in the parallel implementation of the FDFD method

The authors present an implementation of the finite difference frequency domain (FDFD) algorithm, based on extending the problem domain and iterative application of boundary conditions, which allows efficient parallel solution of the electromagnetic problems defined over irregular computational domains. The proposed approach, applied jointly with implicit representation of the operator matrix and spectral transformations, has been used to develop a parallel solver for the open resonator problem, characterized by a nearly optimal speedup of computations.

Towards a characterisation of the behaviour of stochastic local search algorithms for SAT

Stochastic local search (SLS) algorithms have been successfully applied to hard combinatorial problems from different domains. Due to their inherent randomness, the run-time behaviour of these algorithms is characterised by a random variable. The detailed knowledge of the run-time distribution provides important information about the behaviour of SLS algorithms. In this paper we investigate the empirical run-time distributions for WalkSAT, one of the most powerful SLS algorithms for the Propositional Satisfiability Problem (SAT). Using statistical analysis techniques, we show that on hard Random-3-SAT problems, WalkSAT's run-time behaviour can be characterised by exponential distributions. This characterisation can be generalised to various SLS algorithms for SAT and to encoded problems from other domains. This result also has a number of consequences which are of theoretical as well as practical interest. One of these is the fact that these algorithms can be easily parallelised such that optimal speedup is achieved for hard problem instances.

Parallel Algorithms with Optimal Speedup for Bounded Treewidth

We describe the first parallel algorithm with optimal speedup for constructing minimum-width tree decompositions of graphs of bounded treewidth. On n-vertex input graphs, the algorithm works in O((log n)2) time using O(n) operations on the EREW PRAM. We also give faster parallel algorithms with optimal speedup for the problem of deciding whether the treewidth of an input graph is bounded by a given constant and for a variety of problems on graphs of bounded treewidth, including all decision problems expressible in monadic second-order logic. On n-vertex input graphs, the algorithms use O(n) operations together with O(log n log*n) time on the EREW PRAM, or O(log n) time on the CRCW PRAM.

Optimal Speed-Up Parallel Image Template Matching Algorithms on Processor Arrays with a Reconfigurable Bus System,

The image template matching problem is one of the fundamental problems of and has many practical applications in image processing, pattern recognition, and computer vision. It is a useful operation for filtering, edge detection, image registration, and object detection [13]. In this paper, we first design twoO[(M2/p2)log logM] andO[(M2/p2)+(M/p)log logp] time parallel image template matching algorithms on a 3-D processor array with a reconfigurable bus system usingp2N2processors with each processor containingO(1) andO(M/p) restricted memory for 1 ≤p≤M≤N, respectively, for anN×Ndigital image and anM×Mtemplate. By increasing the number of processors, these two proposed algorithms can be run inO(M2/p2) time for speeding up the time complexity usingp2M1/cN2andp2+1/cN2processors, respectively, wherecis a constant andc≥1. Furthermore, anO(1) time can be also obtained from these two proposed algorithms by usingM2+1/cN2processors. These results improve the best known bounds and achieve both optimal and optimal speed-up in their time and processor complexities.

More general parallel tree contraction: Register allocation and broadcasting in a tree

We extend the classic parallel tree-contraction technique of Miller and Reif to handle the evaluation of a class of expression trees that does not fit their original framework. We discuss applications to the following problems: (1) Register allocation, i.e., computing the number of registers needed to evaluate a given expression if all intermediate results are kept in registers; and (2) broadcasting in a tree, i.e., computing the number of steps needed to transmit a message from the root to all other nodes in a given rooted tree if each node is a processor that can communicate with a single neighbor in each step. We show that on inputs of size n, both problems can be solved with optimal speedup in O(( log n) 2) time on an EREW PRAM, in O( log n log log n) time on a CREW PRAM, and in O( log n) time on a CRCW PRAM.

A BUU Code for Parallel Computers

We have developed a distributed algorithm for the simulation of systems that evolve continuously, but which are also subject to discrete events that affect their evolution. As an example we consider the description through the BUU equations of the time evolution of a highly excited nuclear system, and show that this algorithm attains almost optimal speedups.

Automatic domain decomposition on unstructured grids (DOUG)

This paper describes a parallel iterative solver for finite element discretisations of elliptic partial differential equations on 2D and 3D domains using unstructured grids. The discretisation of the PDE is assumed to be given in the form of element stiffness matrices and the solver is automatic in the sense that it requires minimal additional information about the PDE and the geometry of the domain. The solver parallelises matrix–vector operations required by iterative methods and provides parallel additive Schwarz preconditioners. Parallelisation is implemented through MPI. The paper contains numerical experiments showing almost optimal speedup on unstructured mesh problems on a range of four platforms and in addition gives illustrations of the use of the package to investigate several questions of current interest in the analysis of Schwarz methods. The package is available in public domain from the home page http://www.maths.bath.ac.uk/∼mjh/.

Knapsack on VLSI: From algorithm to optimal circuit

We present a parallel solution to the unbounded knapsack problem on a linear systolic array. It achieves optimal speedup for this well-known, NP-hard problem on a model of computation that is weaker than the PRAM. Our array is correct by construction, as it is formally derived by transforming a recurrence equation specifying the algorithm. This recurrence has dynamic dependencies, a property that puts it beyond the scope of previous methods for automatic systolic synthesis. Our derivation thus serves as a case study. We generalize the technique and propose a systematic method for deriving systolic arrays by nonlinear transformations of recurrences. We give sufficient conditions that the transformations must satisfy, thus extending systolic synthesis methods. We address a number of pragmatic considerations: implementing the array on only a fixed number of PEs, simplifying the control to just two counters and a few latches, and loading the coefficients so that successive problems can be pipelined without any loss of throughput. Using a register level model of VLSI, we formulate a nonlinear optimization problem to minimize the expected running time of the array. The analytical solution of this problem allows us to choose the memory size of each PE in an optimal manner.

An $o(n^3 )$-Time Maximum-Flow Algorithm

We show that a maximum flow in a network with n vertices can be computed deterministically in $O({{n^3 } / {\log n}})$ time on a uniform-cost RAM. For dense graphs, this improves the previous best bound of $O(n^3 )$. The bottleneck in our algorithm is a combinatorial problem on (unweighted) graphs. The number of operations executed on flow variables is $O(n^{{8 / 3}} (\log n)^{{4 / 3}} )$, in contrast with $\Omega (nm)$ flow operations for all previous algorithms, where m denotes the number of edges in the network. A randomized version of our algorithm executes $O(n^{{3 / 2}} m^{{1 / 2}} \log n + {{n^2 (\log n)^2 } / {\log }}(2 + {{n(\log n)^2 } / m})$ flow operations with high probability. For the special case in which all capacities are integers bounded by U, we show that a maximum flow can be computed deterministically using $O(n^{{3 / 2}} m^{{1 / 2}} + n^2 (\log U)^{{1 / 2}} + \log U$ flow operations and $O(\min \{ {{nm,n^3 } / {\log n\} + n^2 }}(\log U)^{{1 / 2}} + \log U)$ time. We finally argue that several of our results yield parallel algorithms with optimal speedup.

Fast and Scalable Parallel Algorithms for Knapsack-like Problems

We present two new algorithms for searching in sortedX + Y + R + S, one based on heaps and the other on sampling. Each of the algorithms runs in timeO(n2logn) (nbeing the size of the sorted arraysX,Y,R, andS). Hence in each case, by constructing arrays of sizen=O(2s/4), we obtain a new algorithm for solving certain NP-complete problems such as knapsack onsdata items in time equal (up to a constant factor) to the best algorithm currently known. Each of the algorithms is capable of being efficiently implemented in parallel and so solving large instances of these NP-complete problems fast on coarse-grained distributed memory parallel computers. The parallel version of the heap based algorithm is communication-efficient and exhibits optimal speedup for a number of processors less thannusingO(n) space in each one; the sampling based algorithm exhibits optimal speedup for any number of processors up tonusingO(n) space in total provided that the architecture is capable of logarithmic time sorting.

Parallel Suffix–Prefix-Matching Algorithm and Applications

Our main result in this paper is a parallel algorithm for suffix--prefix- (s--p-) matching that has optimal speedup on a concurrent-read/concurrent-write parallel random-access machine (CRCW PRAM). Given a string of length $m$, the algorithm runs in time $O(\log m)$ using $m/ \log m$ processors. This algorithm is important because we utilize s--p matching as a fundamental building block to solve several pattern- and string-matching problems, such as the following: {1. string matching; 2. multitext/multipattern string matching; 3. multidimensional pattern matching; 4. pattern-occurrence detection; 5. on-line string matching.} In particular, our techniques and algorithms are the first to preserve optimal speedup in the context of pattern matching in higher dimensions and are the only known ones to do so for dimensions $d > 2$.