General Purpose Parallel Machines Research Articles

PIPELINED ARCHITECTURES FOR TRANSFORM DOMAIN LMS ADAPTIVE FILTERING

In this paper, efficient pipelined architectures for the implementation of the Transform Domain LMS (TD-LMS) adaptive filter are considered. Pipelining of the TD-LMS algorithm is achieved by introducing an amount of time delay into the original adaptive scheme. The resulting algorithm, called thereafter the delayed TD-LMS, performs adaptive filtering with delayed coefficients adaptation. A statistical performance analysis of the proposed delayed TD-LMS adaptive algorithm is presented, both for the mean error and the mean squared error of the filter coefficients. A closed form expression is derived for the estimation of the steady state excess mean squared error, in terms of the adaptation delay and the input signal characteristics. The adaptation delay introduced to the delayed TD-LMS algorithm is subsequently utilized for the development of pipelined architectures. By retiming the delays existing in the error signal feedback loop, efficient pipelined implementations of the delayed TD-LMS algorithm are developed. The proposed architectures are suitable for parallel implementation on a general-purpose parallel machine, or on dedicated hardware, integrated on ASIC or ASIP VLSI processors.

A parallel Viterbi decoding algorithm

AbstractIn this paper we express the Viterbi algorithm as a matrix–vector reduction in which multiplication is replaced by addition and addition by minimization. The resulting algorithm is then readily parallelized in a form suitable for implementation on a systolic processor array. We describe the algorithm for Bose–Chaudhuri–Hocquenghem (BCH) codes which have a task graph with its valence restricted to four inputs and four outputs. The method is also applicable to convolution codes, but the complexity of the task graph increases with the number of input bits for these codes. Results for BCH codes are given for two general purpose parallel machines, an IBM SP2 and a Meiko CS2. Copyright © 2001 John Wiley & Sons, Ltd.

Evaluating optical-flow algorithms on a parallel machine

Algorithmic development of optical-flow routines is hampered by slow turnaround times (to iterate over testing, evaluation, and adjustment of the algorithm). To ease the problem, parallel implementation on a convenient general-purpose parallel machine is possible. A generic parallel pipeline structure, suitable for distributed-memory machines, has enabled parallelisation to be quickly achieved. Gradient, correlation, and phase-based methods of optical-flow detection have been constructed to demonstrate the approach. The prototypes enabled comparisons to be made between the speed when parallelised and (already known) accuracy of the three methods when parallelised, on balance favouring the correlation method.

Trade-offs between Communication Throughput and Parallel Time

We study the effect of limited communication throughput on parallel computation in a setting where the number of processors is much smaller than the length of the input. Our model haspprocessors that communicate through a shared memory of sizem. The input has sizenand can be read directly by all the processors. We will be primarily interested in studying cases wheren⪢p⪢m. As a test case we study the list reversal problem. For this problem we prove a time lower bound ofΩ(n/mp). (A similar lower bound holds also for the problems of sorting, finding all unique elements, convolution, and universal hashing.) This result demonstrates that limiting the communication (i.e., smallm) could have significant effect on parallel computation. We show an almost matching upper bound ofO((n/mp)logO(1)n). The upper bound requires the development of a few interesting techniques which can alleviate the limited communication in some general settings. Specifically, we show how to emulate a large shared memory on a limited shared memory efficiently. The lower bound applies even to randomized machines, and the upper bound is a randomized algorithm. We also argue that some standard methodology for designing parallel algorithms appears to require a relatively high level of communication throughput. Our results suggest that new alternative methodologies that need a lower such level must be invented for parallel machines that enable a low level of communication throughput, since otherwise those machines will be severly handicapped as general purpose parallel machines. Although we do not rule that out, we cannot offer any encouraging evidence to suggest that such new methodologies are likely to be found.

A Randomized Parallel Sorting Algorithm with an Experimental Study

Previous schemes for sorting on general-purpose parallel machines have had to choose between poor load balancing and irregular communication or multiple rounds of all-to-all personalized communication. In this paper, we introduce a novel variation on sample sort which uses only two rounds of regular all-to-all personalized communication in a scheme that yields very good load balancing with virtually no overhead. Moreover, unlike previous variations, our algorithm efficiently handles the presence of duplicate values without the overhead of tagging each element with a unique identifier. This algorithm was implemented in Split-C and run on a variety of platforms, including the Thinking Machines CM-5, the IBM SP-2, and the Cray Research T3D. We ran our code using widely different benchmarks to examine the dependence of our algorithm on the input distribution. Our experimental results illustrate the efficiency and scalability of our algorithm across different platforms. In fact, it seems to outperform all similar algorithms known to the authors on these platforms, and its performance is invariant over the set of input distributions unlike previous efficient algorithms. Our results also compare favorably with those reported for the simpler ranking problem posed by the NAS Integer Sorting (IS) Benchmark.

Sloping-and-shaking

Most traditional merging and merging-based sorting algorithms are based on 2 sorters or 2 comparators. A new merging technique is developed, namely sloping-and-shaking multiway merging, and a corresponding multiway sorting method based only onk-sorters is proposed. The sloping-and-shaking merging algorithm mergesk sorted lists into one, wherek can be any prime number. The merging process is not a series of recursive applications of 2-way merging. It sorts the keys on them×k plane in vertical and horizontal directions, then along sloping lines with various slope rates step by step. Onlyk-sorters are needed in the merging or sorting process. The time needed to mergek sorted lists, withm of each, is (k+┌log2(m/k)┐)tk, and the time for sortingN keys is (1+(p−1)k+1/2(p−1)(p−2)┌log2k┐)tk, wherep=logkN, andtk is the time to sortk keys. The proposed algorithms can be implemented either by hardwared sorting networks, or on general purpose parallel and vector machines. The traditional odd-even merging can be viewed as a special case of the multiway merging proposed (whenk is 2). While theoretically the proposed algorithms provide a new understanding of parallel merging and sorting processes, they may be used in practice to construct sorting circuits faster than 2-sorter based sorting methods.

Simulating artificial neural networks on parallel architectures

Parallelization is necessary to cope with the high computational and communication demands of neuroapplications, but general purpose parallel machines soon reach performance limitations. The article explores two approaches: parallel simulation on general purpose computers, and simulation/emulation on neurohardware. Different parallelization methods are discussed, and the most popular techniques are explained. While the software approach looks for an optimal programming model for neural processing, the hardware approach tries to imitate the neuroparadigm using the best of silicon technology.

Practical parallel algorithms for personalized communication and integer sorting

A fundamental challenge for parallel computing is to obtain high-level, architecture independent, algorithms which efficiently execute on general-purpose parallel machines. With the emergence of message passing standards such as MPI, it has become easier to design efficient and portable parallel algorithms by making use of these communication primitives. While existing primitives allow an assortment of collective communication routines, they do not handle an important communication event when most or all processors have non-uniformly sized personalized messages to exchange with each other. We focus in this paper on the h-relation personalized communication whose efficient implementation will allow high performance implementations of a large class of algorithms. While most previous h-relation algorithms use randomization, this paper presents a new deterministic approach for h-relation personalized communication with asymptotically optimal complexity for h>p 2 . As an application, we present an efficient algorithm for stable integer sorting. The algorithms presented in this paper have been coded in Split-C and run on a variety of platforms, including the Thinking Machines CM-5, IBM SP-1 and SP-2, Cray Research T3D, Meiko Scientific CS-2, and the Intel Paragon. Our experimental results are consistent with the theoretical analysis and illustrate the scalability and efficiency of our algorithms across different platforms. In fact, they seem to outperform all similar algorithms known to the authors on these platforms.

Generalised matrix inversion and rank computation by successive matrix powering

Computation of the generalised inverse A + and rank of an arbitrary (including singular and rectangular) matrix A has many applications. This paper derives an iterative scheme to approximate the generalised inverse which can be expressed in the form of successive squaring of a composite matrix T . Given an m by n matrix A with m≈n , we show that the generalised inverse of A can be computed in parallel time ranging from O (log n ) to O (log 2 n ), similar to previous methods. The rank of matrix A is obtained along with the generalised inverse. The successive matrix squaring algorithm is generalised to higher-order schemes, where the composite matrix is repeatedly raised to an integer power l >2. This form of expression leads to a simplified notation compared with that of earlier methods, and helps to clarify the relationship between l , the order of the iterative scheme and K , the number of iterations. In particular, the accuracy achieved in approximating A + is a function only of the magnitude of l K and does not depend on the particular values chosen for l and K ; we argue that there is no obvious advantage in choosing l other than 2. Our derived error bound for the approximation to A + is tighter than that previously established. The same bound applies to the rank. Numerical experiments with different test matrices (square, rectangular, complex, singular, etc.) illustrate the method. They further demonstrate that our tighter error bound provides a useful guide to the number of iterations required. In the examples given, the specified accuracy was achieved after the calculated number of iterations, but no earlier. Implementation studies on a general-purpose parallel machine (CM-5) demonstrated a smaller than expected penalty for direct squaring of matrix T in comparison with multiplication of its component block matrices. For special-purpose VLSI architectures, the simple structure of the matrix squaring algorithm leads to a straightforward parallel implementation with low communication overheads.

Generalised matrix inversion and rank computation by successive matrix powering

Computation of the generalised inverse A + and rank of an arbitrary (including singular and rectangular) matrix A has many applications. This paper derives an iterative scheme to approximate the generalised inverse which can be expressed in the form of successive squaring of a composite matrix T. Given an m by n matrix A with m≈n, we show that the generalised inverse of A can be computed in parallel time ranging from O(log n) to O(log 2 n), similar to previous methods. The rank of matrix A is obtained along with the generalised inverse. The successive matrix squaring algorithm is generalised to higher-order schemes, where the composite matrix is repeatedly raised to an integer power l>2. This form of expression leads to a simplified notation compared with that of earlier methods, and helps to clarify the relationship between l, the order of the iterative scheme and K, the number of iterations. In particular, the accuracy achieved in approximating A + is a function only of the magnitude of l K and does not depend on the particular values chosen for l and K; we argue that there is no obvious advantage in choosing l other than 2. Our derived error bound for the approximation to A + is tighter than that previously established. The same bound applies to the rank. Numerical experiments with different test matrices (square, rectangular, complex, singular, etc.) illustrate the method. They further demonstrate that our tighter error bound provides a useful guide to the number of iterations required. In the examples given, the specified accuracy was achieved after the calculated number of iterations, but no earlier. Implementation studies on a general-purpose parallel machine (CM-5) demonstrated a smaller than expected penalty for direct squaring of matrix T in comparison with multiplication of its component block matrices. For special-purpose VLSI architectures, the simple structure of the matrix squaring algorithm leads to a straightforward parallel implementation with low communication overheads.

PMLS — A parallel multi-level VLSI simulator

Simulation is a key element in modern and future digital circuit design. However, simulation becomes a bottleneck with increasing design complexity. There are mainly two ways to get out of this situation: reduction of the simulation load through multi-level simulation and acceleration of the simulation through exploitation of parallelism. This paper reports of a new Parallel Multi-level VLSI Simulator (PMLS) for a general purpose parallel machine which combines multi-leveling and exploitation of parallelism at the circuit level. The VLSI simulator is implemented in the object-oriented language POOL and runs on the parallel DOOM machine. The paper surveys briefly the principles of digital-circuit simulation, the possibilities of exploiting parallelism, the DOOM machine, its programming language POOL, and describes the design and implementation of the simulator. Preliminary performance figures are also given.

Roundtable: point accelerators vs. general purpose machines for ECAD

Parallel CAD algorithms and machines span the range from the academically interesting to the commercially interesting, with a broad middle ground. The architectures include point accelerators, general-purpose CAD accelerators, and general-purpose parallel machines. A researcher in a university will evaluate these tools employing a different set of criteria than a user or supplier. In academia, one has to get funds and do interesting research; in a company, building, selling, and making a profit are the objectives.

How to share memory in a distributed system

The power of shared-memory in models of parallel computation is studied, and a novel distributed data structure that eliminates the need for shared memory without significantly increasing the run time of the parallel computation is described. More specifically, it is shown how a complete network of processors can deterministically simulate one PRAM step in O (log n /(log log n ) 2 ) time when both models use n processors and the size of the PRAM's shared memory is polynomial in n . (The best previously known upper bound was the trivial O ( n )). It is established that this upper bound is nearly optimal, and it is proved that an on-line simulation of T PRAM steps by a complete network of processors requires Ω( T (log n/ log log n )) time. A simple consequence of the upper bound is that an Ultracomputer (the currently feasible general-purpose parallel machine) can simulate one step of a PRAM (the most convenient parallel model to program) in O ((log n ) 2 log log n ) steps.

Functionally reconfigurable general purpose parallel machines and some image processing and pattern recognition applications

Functionally reconfigurable general purpose parallel machines (FRPM) could be reconfigured during the operation from SIMD to MIMD mode or vice versa (first aspect) and from one interconnection network to another according to the data storing order (second aspect). General purpose machines are considered in order to obtain an arbitrary data exchange between the processing elements they are built of. A model for describing such interconnection networks is presented. A full-information exchange network in introduced which is reconfigurable in a programming way to tree-, matrix-, cube-, linear-neighbourhood and FFT-network. Some schemes for constructing SIMD/MIMD reconfigurable machines are given. The usefullness of using FRMP for image processing and pattern recognition is discussed.