MIMD Machines Research Articles

A parallel algorithm for state assignment of finite state machines

Optimization of large sequential circuits has become unmanageable in CAD of VLSI due to time and memory requirements. We report a parallel algorithm for the state assignment problem for finite state machines. Our algorithm has three significant contributions: it is an asynchronous parallel algorithm portable across different MIMD machines; time and memory requirements reduce linearly with the number of processors, enabling the parallel implementation to handle large problem sizes; and the quality of the results for multiprocessor runs remains comparable to the serial algorithm on which it is based due to an implicit backtrack correction mechanism built into the parallel implementation.

An Evaluation of Parallel Multigrid as a Solver and a Preconditioner for Singularly Perturbed Problems

In this paper we try to achieve h-independent convergence with preconditioned GMRES ([Y. Saad and M. H. Schultz, SIAM J. Sci. Comput., 7 (1986), pp. 856--869]) and BiCGSTAB ([H. A. Van der Vorst, SIAM J. Sci. Comput., 13 (1992), pp. 63--644]) for two-dimensional (2D) singularly perturbed equations. Three recently developed multigrid methods are adopted as a preconditioner. They are also used as solution methods in order to compare the performance of the methods as solvers and as preconditioners. Two of the multigrid methods differ only in the transfer operators. One uses standard matrix-dependent prolongation operators from [J. E. Dendy Jr., J. Comput. Phys., 48 (1982), pp. 366--386], [W. Hackbusch, Multi-grid Methods and Applications, Springer, Berlin, 1985]. The second uses "upwind" prolongation operators, developed in [P. M. de Zeeuw, J. Comput.\ Appl.\ Math., 33 (1990), pp. 1--27]. Both employ the Galerkin coarse grid approximation and an alternating zebra line Gauss--Seidel smoother. The third method is based on the block LU decomposition of a matrix and on an approximate Schur complement. This multigrid variant is presented in [A. Reusken, A Multigrid Method Based on Incomplete Gaussian Elimination, University of Eindhoven, the Netherlands, 1995]. All three multigrid algorithms are algebraic methods. The eigenvalue spectra of the three multigrid iteration matrices are analyzed for the equations solved in order to understand the convergence of the three algorithms. Furthermore, the construction of the search directions for the multigrid preconditioned GMRES solvers is investigated by the calculation and solution of the minimal residual polynomials. For Poisson and convection-diffusion problems all solution methods are investigated and evaluated for finite volume discretizations on fine grids. The methods have been parallelized with a grid partitioning technique and are compared on an MIMD machine.

Declustering and load-balancing methods for parallelizing geographic information systems

Declustering and load balancing are important issues in designing a high performance geographic information system (HPGIS), which is a central component of many interactive applications such as real time terrain visualization. The current literature provides efficient methods for declustering spatial point data. However, there has been little work toward developing efficient declustering methods for collections of extended objects, like chains of line segments and polygons. We focus on the data partitioning approach to parallelizing GIS operations. We provide a framework for declustering collections of extended spatial objects by identifying the following key issues: (1) work load metric; (2) spatial extent of the work load; (3) distribution of the work load over the spatial extent; and (4) declustering method. We identify and experimentally evaluate alternatives for each of these issues. In addition, we also provide a framework for dynamically balancing the load between different processors. We experimentally evaluate the proposed declustering and load balancing methods on a distributed memory MIMD machine (Cray T3D). Experimental results show that the spatial extent and the work load metric are important issues in developing a declustering method. Experiments also show that the replication of data is usually needed to facilitate dynamic load balancing, since the cost of local processing is often less than the cost of data transfer for extended spatial objects. In addition, we also show that the effectiveness of dynamic load balancing techniques can be improved by using declustering methods to determine the subsets of spatial objects to be transferred during runtime.

Parallelizing image feature extraction on coarse-grain machines

In this paper, we present a fast parallel algorithm for feature extraction on coarse-grain MIMD machines. By maintaining algorithmic threads at each node, our algorithm enhances processor utilization and obtains large speed-ups. Our implementations show that, given a 1,024/spl times/1,024 image, speed-ups of 27.6 and 56.0 on a 32-node SP2 and a 64-node T3D can be achieved.

Parallelization and Aggregation of Nested Benders Decomposition

Dynamic multistage stochastic linear programming has many practical applications for problems whose current decisions have to be made under future uncertainty. There are a variety of methods for solving these problems, including nested Benders decomposition. In this method, recently shown to be superior to the alternatives for large problems, the problem is decomposed into a set of smaller linear programming problems. These problems can be visualised as being attached to the nodes of a tree which is formed from the realizations of the random data vectors determining the uncertainty in the problem. The tree is traversed forwards and backwards, with information from the solutions to each nodal linear programming problem being passed to its immediate descendants by the formation of their right hand sides and to its immediate ancestor in the form of cuts. Problems in the same time period can be solved independently and it is this inherent parallelism that is exploited in our parallel nested Benders algorithm. A parallel version of the MSLiP nested Benders code has been developed and tested on various types of MIMD machines. The differing structures of the test problems cause differing levels of speed-up. Results show that problems with few variables and constraints per node do not gain from this parallelization. Stage aggregation has been successfully explored for such problems to improve their parallel solution efficiency by increasing the size of the nodes and therefore the time spent calculating relative to the time spent communicating between processors.

Simulated Annealing Based Parallel State Assignment of Finite State Machines

Simulated annealing is an effective tool in many optimization problems in VLSI CAD but its time requirements are prohibitive. In this paper, we report parallel algorithms for a well established simulated annealing based algorithm for the state assignment problem for finite state machines. Our parallel annealing strategy uses parallel moves by multiple processes, each performing local moves within its assigned subspace of the state encoding space. The novelty is in the dynamic repartitioning of the state space among processors, so that each processor gets to perform moves on the entire space over time. This is important to keep the quality of the parallel algorithm comparable to the serial algorithm. Our algorithm gives quality results within 0.05% of the serial algorithm on 64 processors. Our algorithms, ProperJEDI and PartJEDI, are portable across a wide range of MIMD machines. PartJEDI is memory scalable and is incrementally developed from ProperJEDI which is data replicated. For a large circuit, ProperJEDI reduces the runtime from 11 h to 10 min on a 64-processor machine. For the same circuit, PartJEDI reduces the runtime from 11 h to 20 min and memory requirement from 114 to 2 MB.

Design of efficient parallel algorithms on shared memory multiprocessors

The design of parallel algorithms is studied in this paper. These algorithms are applicable to shared memory MIMD machines. In this paper, the emphasis is put on the methods for design of the efficient parallel algorithms. The design of efficient parallel algorithms should be based on the following considerations: algorithm parallelism and the hardware parallelism; granularity of the parallel algorithm; algorithm optimization according to the underling parallel machine. In this paper, these principles are applied to solve a model problem of the PDE. The speedup of the new method is high. The results were tested and evaluated on a shared memory MIMD machine. The practical results were agree with the predicted performance.

Parallel Ray-Tracing on Mimd Machine Using Dynamic Load-Balancing Mechanisms

This research investigates dynamic load-balancing mechanisms on a distributed-memory MIMD machine, through the implementation of an application chosen from the computational graphics field (ray-tracing). The application-system performance is compared using user-dependent farmer load-balancing approach and user-transparent, effective and scalable, dynamic load-balancing algorithms. Tasks are generate, at execution time, using a multiple-spawning mechanism. The selected implementation environment is composed of T800 transputers. The implementation of a real application have evidenced the significance of such load-balancing tools regarded to the practicability of the system-level resource management facilities and performance improvement.

A fast parallel SAT-solver — efficient workload balancing

We present a fast parallel SAT-solver on a message based MIMD machine. The input formula is dynamically divided into disjoint subformulas. Small subformulas are solved by a fast sequential SAT-solver running on every processor, which is based on the Davis-Putnam procedure with a special heuristic for variable selection. The algorithm uses optimized data structures to modify Boolean formulas. Additionally efficient workload balancing algorithms are used, to achieve a uniform distribution of workload among the processors. We consider the communication network topologiesd-dimensional processor grid and linear processor array. Tests with up to 256 processors have shown very good efficiency-values (>0.95).

A parallel algorithm for evaluating general linear recurrence equations

A block parallel partitioning method for evaluating general linearmth order recurrence equations is presented and applied to solve the eigenvalues of symmetric tridiagonal matrices. The algorithm is based on partitioning, in a way that ensures load balance during computation. This method is applicable to both shared memory and distributed memory MIMD systems. The algorithm achieves a speedup ofO(p) on a parallel computer withp-fold parallelism, which is linear and is greater than the existing results, and the data communication between processors is less than that required for other methods. For solving symmetric tridiagonal eigenvalue problems, our method is faster than the best previous results. The results were tested and evaluated on an MIMD machine, and were within 79% to 96% of the predicted performance for the second order linear recurrence problem.

A practical and portable model of programming for iterative solvers on distributed memory machines

A practical and portable model of parallel programming inspired from the systolic handling of array shift on SIMD machines is presented. All the communications are hidden to the user who can code parallel iterative solvers in a more natural way on MIMD machines. Parallel efficiency and speed performance of this model are then discussed on the iPSC/860, IBM SP2 and Cray T3D in four examples of preconditioned conjugate gradient to solve a diffusion problem.

ASYNCHRONOUS PARALLEL ALGORITHMS BASED ON DDM ON CM-5 (I) FOR SOLVING LINEAR ELLIPTIC PDE'S∗

This is the first in a series of papers about asynchronous parallel algorithms based on domain decomposition method (DDM) for solving partial differential equations on MIMD machines, especially on CM-5. These algorithms are all called Schwarz-Chaotic Over Relaxation (S-COR), the generalization of the Schwarz Alternating Method. In this paper, the linear elliptic partial differential equations is discussed. Numerical results are presented to demonstrate the effectiveness of these decomposition algorithms on the Thinking Machines CM-5 parallel supercomputer.

MENDELS ZONE: A parallel program development system based on formal specifications

MENDELS ZONE is a formal specification-based practical parallel program development system on an MIMD machine with 64 processors. It has been applied to the several practical applications: one such application is described in formal specifications totalling 3700 lines, and the corresponding parallel programs generated from these formal specifications totals 6200 lines. MENDELS ZONE has several key aspects. First, an algebraic specification is used for the functional part of parallel programs and a temporal logic specification is used for the synchronization part of the parallel programs. Second, Petri nets and data-flow diagrams are adopted for visualization. Third, MENDELS ZONE has the following unique features from a parallel programming viewpoint. (1) The confluence property of the complete term rewriting system ensures the quality of non-deterministic execution in parallel programs; (2) Serialization in parallel programming is specified using propositional temporal logic; (3) Parallel implementation of theorem provers contributes to practical use.

Robust process simulation using interval methods

Ideally, for the needs of robust process simulation, one would like a nonlinear equation solving technique that can find any and all roots to a problem, and do so with mathematical certainty. In general, currently used techniques do not provide such rigorous guarantees. One approach to providing such assurances can be found in the use of interval analysis, in particular the use of interval Newton methods combined with generalized bisection. However, these methods have generally been regarded as extremely inefficient. Motivated by recent progress in interval analysis, as well as continuing advances in computer speed and the availability of parallel computing, we consider here the feasibility of using an interval Newton/generalized bisection algorithm on process simulation problems. An algorithm designed for parallel computing on an MIMD machine is described, and results of tests on several problems are reported. Experiments indicate that the interval Newton/generalized bisection method works quite well on relatively small problems, providing a powerful method for finding all solutions to a problem. For larger problems, the method performs inconsistently with regard to efficiency, at least when reasonable initial bounds are not provided.

Steady state simulation using MIMD machines in the speedup simulator

A parallel nonlinear solver [9] is implemented in the flowsheeting package SPEEDUP. The implementation is done by integrating the solver into the package being then transparent to the user. It is tested by using two steady state simulation problems. The first is a small flowsheet involving only 1633 variables. The second example consists of a flowsheet with two columns, resulting in a problem with 5668 equations. Two MIMD parallel machines are considered. The first is a cluster of workstations running with PVM. The other is a proper parallel machine, a FUJITSU AP1000. The larger problem is tested in both machines to compare the parallel performance.

SIMD processors-expanding its horizons by emulating others

Parallel processing has established itself as the key enabling technology for the future of high performance computing. A parallel machine can be abstracted as a collection of processing elements (PEs) connected using some form of communication network. Parallel machines have been built along two major architectural formats: multiple-instruction streams multiple-data streams (MIMD) machines and single-instruction stream multiple-data streams (SIMD) machines. The hardware differences and general programming models for SIMD and MIMD processors have led many to believe each can solve only limited types of problems. SIMD machines, for instance, are often used in applications such as image processing. In that application, the same set of operations are performed on many different pieces of data. MIMD machines are commonly used when the control flow needs vary in correspondence with differences in data, such as parallel simulation.

Development of Parallel Algorithms for Computer Vision

Computer vision is an important research area where computationally-intensive and time critical problems need to be solved routinely. This paper described some parallel algorithms for image processing and computational geometry applicable to the field of 'robot vision' which was developed using PACE parallel computer, a closely coupled message passing MIMD machine, designed and developed at the Advanced Numerical Reserach & Analysis Group, Hyderabad.

Inside parallel computers: trends in interconnection networks

Computational scientists who depend on parallel computing to let them run larger models in less time will be disappointed unless the processors can pass information back and forth quickly. The interconnection networks through which processors communicate in tightly-coupled parallel machines thus remain a vital research topic for computer architects. Over the last two decades, we have seen an evolution in the demands placed on these networks. Early SIMD machines required the simultaneous transfer of data from each network input to each output for a relatively small set of communication configurations or permutations; whereas the SIMD and MIMD machines of today need to support varied patterns of synchronous and asynchronous traffic, respectively. Interconnection networks can be categorized according to a number of criteria such as topology, routing strategy and switching technique. They vary widely in their cost, fault tolerance, simplicity, amenability to partitioning, and their aggregate bandwidth for local and nonlocal traffic patterns.

Parallel LOD methods for second order time dependent PDEs

Time dependent problems in Partial Differential Equations (PDEs) are often solved by the Method Of Lines (MOL). In many important instances, the exact solution of the resulting system of Ordinary Differential Equations (ODEs) satisfies recurrence relations involving the matrix exponential function. This function is approximated by a new type of rational function possessing real and distinct poles which, consequently, readily admits a partial fraction expansion thereby allowing the distribution of the work in solving the corresponding linear algebraic systems in essentially Backward Euler-like solves on concurrent processors. Locally One Dimensional (LOD) splitting methods with enhanced parallelization are developed for multispace problems utilizing Strang-like splitting techniques. The use of rational approximants with distinct real poles in the temporal direction, in collusion with splitting techniques in the spatial directions, creates the potential for efficient coarse grain time-stepping parallel algorithms on MIMD machines. The resulting parallel algorithms possess appropriate stability properties, and are implemented on various parabolic and hyperbolic PDEs from the literature in higher space dimensional problems.

Parallel structural optimization applied to bone remodeling on distributed memory machines

This paper demonstrates parallel structural optimization methods on distributed memory MIMD machines. We have restricted ourselves to the simpler case of minimizing a multivariate non-linear function subject to bounds on the independent variables, when the objective function is expensive to evaluate as compared to the linear algebra portion of the optimization. This is the case in structural applications, when a large three-dimensional finite element mesh is used to model the structure. This paper demonstrates how parallelism can be exploited during the function and gradient computation as well as the optimization iterations. For the finite element analysis, a ''toruswrapped'' skyline solver is used. The reflective Newton method which attempts to reduce the number of iterations at the expense of more linear algebra per iteration is compared with the more conventional active set method. All code is developed for an Intel iPSC/860, but it can be ported to other distributed memory machines. The methods developed are applied to problems in bone remodeling. In the area of biomechanics, optimization models can be used to predict changes in the distribution of material properties in bone due to the presence of an artificial implant. The model we have used minimizes a linear combination of the mass and strain energy in the entire domain subject to bounds on the densities in each finite element. Early results show that the early reflective Newton method can outperform active set methods when a significant number of variables are not active at the minimum. Keywords: structural optimization, parallel finite element analysis