MIMD Machines Research Articles

Scheduling Simulations: An Experimental Approach to Time-Sharing Multiprocessor Scheduling Schemes

Real time systems that are logically programmed for scientific applications involve frequent job arrivals, thus requires a parallel architecture, so that maximum applications can be executed simultaneously resulting in less waiting time and maximum resource utilization. This must be achieved by workload partitioning & characterization, directs towards the development of Multiprocessor machines, a way to achieve parallel effects. Today, multiprocessor systems cover H/W replications that may replicates complete central processing units asynchronously or multiple executional units synchronously controlled by a different/common clock respectively. This research deals with the multiprocessor scheduling implemented via simulated time sharing environment containing logically programmed virtual processors and batch lists, each batch having its associated arrival time along with number of jobs where each job contains parameters such as Batch_id, Job_id and CPU Burst_time(defined as no. of cycles required) etc. The idea behind this theory is to distribute a number of simultaneously occurring jobs to virtual processor list corresponding to a scheduling algorithm. Synchronous architectures involve SIMD based model with data parallel aspects of computations, whereas Control parallel asynchronous MIMD machines are the future trends leading towards Instruction level parallel processors involving VLIW (very large instruction word) and superscalar machines. General Terms Multiprocessor Scheduling Policies & Mechanisms, Time Sharing & Space Sharing policies, Static vs Dynamic scheduling Decisions.

Parallel processing for image and video processing: Issues and challenges

Some meaningful hints about parallelization problems in image processing and analysis are discussed. The issues of the operation of various architectures used to solve vision problems, from the pipeline of dedicated operators to general purpose MIMD machines, passing through specialized SIMD machines and processors with extended instruction sets, and parallelization tools, from parallel library to parallel programming languages, are reviewed. In this context, a discussion of open issues and directions for future research is provided.

Parallel Preconditioned Conjugate Gradient Square Method Based on Normalized Approximate Inverses

A new class of normalized explicit approximate inverse matrix techniques, based on normalized approximate factorization procedures, for solving sparse linear systems resulting from the finite difference discretization of partial differential equations in three space variables are introduced. A new parallel normalized explicit preconditioned conjugate gradient square method in conjunction with normalized approximate inverse matrix techniques for solving efficiently sparse linear systems on distributed memory systems, using Message Passing Interface (MPI) communication library, is also presented along with theoretical estimates on speedups and efficiency. The implementation and performance on a distributed memory MIMD machine, using Message Passing Interface (MPI) is also investigated. Applications on characteristic initial/boundary value problems in three dimensions are discussed and numerical results are given.

A Distributed Normalized Explicit Preconditioned Conjugate Gradient Method

A new parallel normalized explicit preconditioned conjugate gradient method in conjunction with normalized approximate inverse matrix techniques is presented for solving efficiently sparse linear systems on multi-computer systems. Application of the proposed method on a three dimensional boundary value problem is discussed and numerical results are given. The implementation and performance on a distributed, memory MIMD machine, using message passing interface (MPI) is also investigated. E-mail: nmis@di.uoa.gr

Performance of the Alex AVX-2 MIMD architecture in learning the NetTalk database.

The process of training neural networks on parallel architectures has been used to assess the performance of so many parallel machines. In this paper, we are investigating the implementation of backpropagation (BP) on the Alex AVX-2 coarse-grained MIMD machine. A host-worker parallel implementation is carried out in order to train different networks to learn the NetTalk dictionary. First, a computational model is constructed using a single processor to complete the learning process. Also, a communication model for the host-worker topology is developed in order to compute the communication overhead in the broadcasting/gathering process. Both models are then used to predict the machine performance when p processors are used and a comparison with the actual measured performance of the parallel architecture implementation is carried out. Simulation results show that both models can be used effectively to predict the machine performance for the NetTalk problem. Finally, a comparison between the AVX-2 NetTalk implementation and the performance of other parallel platforms is presented.

Parallel computing with generalized cellular automata

Cellular automata (CA) are fundamental computational models of spatial phenomena, in which space is represented by a discrete lattice of cells. Each cell concurrently interacts with its neighborhood which, in traditional CA, is limited to the cell's nearest neighbors. In this paper we discuss generalized cellular automata (GCA), an important but unexplored class of CA, in which the cell's interaction domain extends beyond the nearest neighbors. The computational power necessary to run large scale CA (and GCA) models has only recently been available thanks to parallel processing. This paper focuses on implementation and performance of GCA in biological modeling. In particular, we present results of simulating the spread of epidemics and the creation of spatial infection patterns that are important for disease control. The simulation system is implemented on three different platforms: the MasPar MP-1 SIMD computer, the IBM SP-2 MIMD machine and a network of workstations (NOW) that consists of Sun SPARCstation 5 and UltraSPARC 2's connected via Ethernet. The system presented in this paper has been specialized for simulating a four species spatially explicit model, however, the implementation may be readily modified to represent other models. Simulation results are presented for simple epidemics and vector-borne diseases spread by parasites.

A scalable off-line MPEG-2 video encoding scheme using a multiprocessor system

Video compression plays a central role in a vast number of multimedia applications but its computational requirements overwhelm the capabilities of any present single processor system. In this paper, we explore the use of parallel machines like the Intel Paragon to compress MPEG-2 video sequences. The motivation is to build a production-based compression facility by exploiting the potential power of the available machine. Given a video sequence or a set of sequences, the aim of the parallel encoder is to achieve the maximum possible encoding rate. A collective scheduling scheme for the processors, I/O nodes, and disks is proposed that provides fast I/O, minimizes the idle times of processors, and enables the system to work in a highly balanced fashion. An efficient data layout scheme for storing video frames is also proposed in order for the I/O to sustain the desired data transfer rates. Using a small percentage of processors as the I/O nodes results in an efficient utilization of the system resources. As shown by experimental and analytical results, the encoding scheme is scalable and higher performance can be achieved with larger machines. The performance of the proposed scheme can be many times the real-time encoding rates with Standard Interface Format (SIF) and CCIR-601 video sequences. The experimental results indicate about two-fold gain in performance compared to the previous studies. Such a system is useful for the conversion of analog videos to compressed digital form in large studios, digital libraries, and other multimedia database environments. The proposed scheme partitions the system into groups of compute nodes, and I/O nodes, and can be easily extended to other MIMD machines or a set of networked workstations.

Continuous genetic networks

We show how random Boolean networks, a well-known model of the interaction among genes within a living cell, can be generalized by relaxing the Boolean approximation, therefore defining a class of continuous genetic networks, and by explicitly describing gene–chemical interactions. It is also shown that population growth can be modelled by describing in detail the interactions among a limited number of genes, while the effects of the remaining genes are represented by a collective variable. The model can describe specific metabolic systems, as shown by the case of a microorganism able to degrade aromatic compounds, and it can also be used to study the generic properties of genetic networks. For this purpose, theoretical and simulation results are given in the case where all the genes are arranged on a 2-D regular square topology, with connections among neighbouring sites, which represents a cellular-automata like model. Parallel simulation on a MIMD machine shows a good speed-up, thus confirming that these models are amenable to efficient parallel implementation.

Will SIMD make a comeback

Will SIMD make a comeback

Back-propagation learning algorithm and parallel computers: The CLEPSYDRA mapping scheme

This paper deals with the parallel implementation of the back-propagation of errors learning algorithm. To obtain the partitioning of the neural network on the processor network the author describes a new mapping scheme that uses a mixture of synapse parallelism, neuron parallelism and training examples parallelism (if any). The proposed mapping scheme allows to describe the back-propagation algorithm as a collection of SIMD processes, so that both SIMD and MIMD machines can be used. The main feature of the obtained parallel algorithm is the absence of point-to-point communication; in fact, for each training pattern, an all-to-one broadcasting with an associative operator (combination) and an one-to-all broadcasting (that can be both realized in log P time) are needed. A performance model is proposed and tested on a ring-connected MIMD parallel computer. Simulation results on MIMD and SIMD parallel machines are also shown and commented.

CP-PACS: A massively parallel processor at the University of Tsukuba

Computational Physics by Parallel Array Computer System (CP-PACS) is a massively parallel processor developed and in full operation at the Center for Computational Physics at the University of Tsukuba. It is an MIMD machine with a distributed memory, equipped with 2048 processing units and 128 GB of main memory. The theoretical peak performance of CP-PACS is 614.4 Gflops. CP-PACS achieved 368.2 Gflops with the Linpack benchmark in 1996, which at that time was the fastest Gflops rating in the world. CP-PACS has two remarkable features. Pseudo Vector Processing feature (PVP-SW) on each node processor, which can perform high speed vector processing on a single chip superscalar microprocessor; and a 3-dimensional Hyper-Crossbar (3-D HXB) Interconnection network, which provides high speed and flexible communication among node processors. In this article, we present the overview of CP-PACS, the architectural topics, some details of hardware and support software, and several performance results.

Parallel computing and quantum chromodynamics

The study of Quantum Chromodynamics (QCD) remains one of the most challenging topics in elementary particle physics. The lattice formulation of QCD, in which space–time is treated as a four-dimensional hypercubic grid of points, provides the means for a numerical solution from first principles but makes extreme demands upon computational performance. High Performance Computing (HPC) offers us the tantalising prospect of a verification of QCD through the precise reproduction of the known masses of the strongly interacting particles. It is also leading to the development of a phenomenological tool capable of disentangling strong interaction effects from weak interaction effects in the decays of one kind of quark into another, crucial for determining parameters of the Standard Model of particle physics. The 1980s saw the first attempts to apply parallel architecture computers to lattice QCD. The SIMD and MIMD machines used in these pioneering efforts were the ICL DAP and the Cosmic Cube, respectively. These were followed by the Connection Machine, the Meiko i860 Computing Surface and the Intel Hypercube. The end of the decade witnessed a rise in the development of special purpose dedicated parallel systems, notably the APE machines in Rome, the Columbia machines, the GF-11 system at IBM Research and the QCDPAX project in Tsukuba. The state-of-the-art is represented by the CP–PACS machine at Tsukuba, and QCDSP, the latest Columbia machine. We give a brief pedagogic review of lattice QCD, outline the computational methodology used and discuss the sources of systematic error that arise in numerical calculations. We outline some of the early calculations and discuss parallel architectures and their application to QCD, giving examples of both commercial and special purpose machines. After a short section on recent developments, we describe state-of-the-art machines and conclude with the prospects for the future.

A LOAD-BALANCED ALGORITHM FOR PARALLEL DIGITAL IMAGE WARPING

This paper introduces and compares three parallel algorithms to compute general geometric image transformations on MIMD machines. We propose three variants of a parallel general scheme. We focus on the load balancing and the data redistributions. Experimental results are reported and compared. The implementation has been done using PPCM library allowing us to run the program over different parallel machines. We compare logical communication schemes for message-passing machines. Since our parallel algorithm needs global communications such as multiscatters, we study the efficiency of two different logical topologies usable with PPCM. These studies allow us to find the best combination of algorithm and virtual topology to use on a given parallel machine.

MCGS: A Modified Conjugate Gradient Squared Algorithm for Nonsymmetric Linear Systems

The conjugate gradient squared (CGS) algorithm is a Krylov subspace algorithm that can be used to obtain fast solutions for linear systems (Axeb) with complex nonsymmetric, very large, and very sparse coefficient matrices (A). By considering electromagnetic scattering problems as examples, a study of the performance and scalability of this algorithm on two MIMD machines is presented. A modified CGS (MCGS) algorithm, where the synchronization overhead is effectively reduced by a factor of two, is proposed in this paper. This is achieved by changing the computation sequence in the CGS algorithm. Both experimental and theoretical analyses are performed to investigate the impact of this modification on the overall execution time. From the theoretical and experimental analysis it is found that CGS is faster than MCGS for smaller number of processors and MCGS outperforms CGS as the number of processors increases. Based on this observation, a set of algorithms approach is proposed, where either CGS or MGS is selected depending on the values of the dimension of the A matrix (N) and number of processors (P). The set approach provides an algorithm that is more scalable than either the CGS or MCGS algorithms. The experiments performed on a 128-processor mesh Intel Paragon and on a 16-processor IBM SP2 with multistage network indicate that MCGS is approximately 20% faster than CGS.

A frequency-domain parallel method for the numerical approximation of parabolic problems

A naturally parallelizable numerical method is introduced and analyzed for parabolic partial differential equations. Instead of solving the problem in the space-time formulation, we propose to solve it in the space-frequency formulation. Existence and uniqueness are given. Error estimates are given for each single frequency. Also given is a full estimate for errors coming from truncation, numerical quadrature rule in Fourier inversion and finite element discretization in the space-frequency domain. A numerical experiment on a parallel MIMD machine is also given.

Efficient parallelization of a parabolized flow solver

This article describes application of our theory of parallelization of implicit ADI schemes to parabolized flows. A parallel multi-domain version of a turbulent developing flow in a straight duct (case A) and viscous flow in a curved duct (case B) are presented. Semi-implicit and explicit methods for the determination of boundary values for the auxiliary ADI functions on the interfaces between the sub-domains are utilized. Numerical runs show that the proposed algorithm is valid in the regions with rapidly varying fields of governing variables (near-entrance region for the case A, region 30°< θ<60° for the case B) as well as in the regions with slow axial modification of the flowfield. The algorithm is suitable for small transverse velocity (case A) and for transverse velocity of order of streamwise velocity (case B). A simplified version of our theoretical model of parallel efficiency is developed and utilized for optimal multidomain partitioning. Computer runs of the multi-domain code are done on a Meiko CS and on a DEC Alpha farm with PVM communication software. The predictions of parallel efficiency obtained by the model compare well with those of actual computer runs. The parallelization parameters obtained are quite different for two considered MIMD machines. This fact confirms the importance of a priori estimation of parallelization efficiency of an algorithm and correct choice of a parallel computer.

Flexible Multiple Semicoarsening for Three-Dimensional Singularly Perturbed Problems

We present robust parallel multigrid-based solvers for 3D scalar partial differential equations. The robustness is obtained by combining multiple semicoarsening strategies, matrix-dependent transfer operators, and a Krylov subspace acceleration. The basis for the 3D preconditioner is a 2D method with multiple semicoarsened grids based on the MG-S method from [C. W. Oosterlee, Appl. Numer. Math., 19(1995), pp. 115--128] and [T. Washio and C. W. Oosterlee, GMD Arbeitspapier 949, GMD, St. Augustin, Germany, 1995]. The 2D method is generalized to three dimensions with a line smoother in the third dimension. The method based on semicoarsening has been parallelized with the grid partitioning technique [J. Linden, B. Steckel, and K. Stüben, Parallel Comput., 7(1988), pp. 461--475], [O. A. McBryan et al., Impact Comput. Sci. Engrg., 3(1991), pp. 1--75] and is evaluated as a solver and as a preconditioner on a MIMD machine. The robustness of the 3D method is shown for finite volume and finite difference discretizations of 3D anisotropic diffusion equations and convection-dominated convection-diffusion problems.

Steps towards steady-state process simulation on MIMD machines: Implementation in the SPEEDUP simulator

A parallel nonlinear solver ( Paloschi 1994) is implemented in the flowsheeting package SPEEDUP(Aspen Technology Inc.). The solver is a Quasi-Newton (QN) method approach where the parallelization is achieved by partitioning the domain and range of the system of nonlinear equations assigning one partition to each available processor. All significant tasks within the QN algorithm are performed in parallel. Linear systems are solved using a parallel preconditioned iterative linear solver (GMRES) and the preconditioner is based on a block-diagonal approximation. It is tested by using two steady-state simulation problems. The first is a plug-flow reactor flowsheet involving 1633 variables in the largest nonlinear block. The second example consists of a flowsheet with two columns resulting in a problem with 5668 equations in the largest nonlinear block. Two MIMD parallel machines are considered. The first is a cluster of workstations running with the message passing protocol PVM. The other is a proper parallel machine a FUJITSU AP1000. The larger problem is tested in both machines to compare the parallel performance. For the first example the execution time is reduced by a factor of 2.4 using 8 processors and 1.5 using 4 on the Fujitsu AP1000 machine. For the second example the reduction achieves a factor of 2.7 using 8 processors on a cluster of workstations and a factor of 7.1 using 16 processors on the Fujitsu AP1000.

User-Level Parallel File I/O

&#x0D; &#x0D; &#x0D; &#x0D; Parallel disk I/O subsystems are becoming more important in today’s large-scale parallel machines. Parallel disk systems provide a significant boost in I/O performance reducing the gap between processor and disk speeds. We describe a Unix-like file I/O user interface, implemented in a parallel file I/O subsystem on an MIMD machine, the nCUBE 2. Based on message passing, we develop parallel disk read/write algorithms to achieve higher parallelism when more disk drives are used. We use the closed queuing network model to analyze the effect of some tunable system parameters for our parallel file system. We then performed simulation experiments in order to obtain more realistic performance data, by comparing the original vendor-supplied file system with ours. The results indicate that the speedup in I/O performance is almost equal to the number of disk drives we use. Thus, our user-level parallel file I/O approach will provide scalable I/O performance.&#x0D; &#x0D; &#x0D; &#x0D;

Mapping Conjugate Gradient Algorithms for Neutron Diffusion Applications onto SIMD, MIMD, and Mixed-Mode Machines

The performance of conjugate gradient (CG) algorithms for the solution of the system of linear equations that results from the finite-differencing of the neutron diffusion equation was analyzed on SIMD, MIMD, and mixed-mode parallel machines. A block preconditioner based on the incomplete Cholesky factorization was used to accelerate the conjugate gradient search. The issues involved in mapping both the unpreconditioned and preconditioned conjugate gradient algorithms onto the mixed-mode PASM prototype, the SIMD MasPar MP-1, and the MIMD Intel Paragon XP/S are discussed. On PASM , the mixed-mode implementation outperformed either SIMD or MIMD alone. Theoretical performance predictions were analyzed and compared with the experimental results on the MasPar MP-1 and the Paragon XP/S. Other issues addressed include the impact on execution time of the number of processors used, the effect of the interprocessor communication network on performance, and the relationship of the number of processors to the quality of the preconditioning. Applications studies such as this are necessary in the development of software tools for mapping algorithms onto either a single parallel machine or a heterogeneous suite of parallel machines.