Connection Machine CM-2 Research Articles

Cellular neural networks as a general massively parallel computational paradigm

In this paper is presented the use of the discrete-time cellular neural network (DTCNN) paradigm to develop algorithms devised for general-purpose massively parallel processing (MPP) systems. This paradigm is defined in discrete N-dimensional spaces (lattices) and is characterized by the locality of the direct information transmission between the space points (cells) and by continuous values of data and parameters; the DTCNN paradigm is thus able to express most of the typical MPP applications. A general version of a DTCNN has been implemented and optimized for three MPP architectures, namely the Connection Machines CM-2 and CM-5 and the Cray T3D. The comparison between the three machine performances with those achieved by a standard SPARC-20 workstation shows that, particularly with large lattices, the speed-up allowed in the computational times is significant and the range of solvable problem sizes is widely extended.

Cellular neural networks as a general massively parallel computational paradigm

In this paper is presented the use of the discrete-time cellular neural network (DTCNN) paradigm to develop algorithms devised for general-purpose massively parallel processing (MPP) systems. This paradigm is defined in discrete N-dimensional spaces (lattices) and is characterized by the locality of the direct information transmission between the space points (cells) and by continuous values of data and parameters; the DTCNN paradigm is thus able to express most of the typical MPP applications. A general version of a DTCNN has been implemented and optimized for three MPP architectures, namely the Connection Machines CM-2 and CM-5 and the Cray T3D. The comparison between the three machine performances with those achieved by a standard SPARC-20 workstation shows that, particularly with large lattices, the speed-up allowed in the computational times is significant and the range of solvable problem sizes is widely extended.

Comparing SIMD and MIMD Programming Modes

The Connection Machine CM-5 supports both SIMD and MIMD programming modes. The SIMD mode is simulated over the MIMD mode. This simulation is likely to lead to a loss of performance in SIMD programs. This paper describes a comparison of the two programming modes with CM Fortran and message-passing Fortran. Two kinds of benchmarks are discussed. The first kind consists of synthetic benchmarks in which we measure time for basic arithmetic operations and communication time. The second kind consists of application benchmarks. The experimental results conclusively show that message-passing Fortran performs considerably better than CM Fortran. While the CM-5 is obsolete, these issues show up in the T3D and other current machines.

A parallel processing approach to acoustic tomography

Acoustic tomography is a widely utilized technique to reconstruct material properties from global wave propagation data. One of the main drawbacks to using tomography is the computationally intensive nature of the reconstruction algorithm. In this work, a new computational approach to acoustic tomography is presented using a computer with a massively parallel architecture (Connection Machine CM200). The objective was to develop a tomographic reconstruction algorithm, including ray tracing, which fully utilized the simultaneous computation capability of the Connection Machine. The modifications necessary to implement tomographic reconstructions in parallel are described. Results are presented for several synthetic reconstructions. The CPU time required for the parallel program remains relatively constant as the problem size increases, whereas the CPU time for a serial program increases rapidly. This illustrates the utility of this approach for complex problems.

A Data-Parallel Implementation of Hierarchical N-Body Methods

The O( N) hierarchical N-body algorithms and mas sively parallel processors allow particle systems of 100 million particles or more to be simulated in acceptable time. We describe a data-parallel implementation of Anderson's method and demonstrate both efficiency and scalability of the implementation on the Connec tion Machine CM-5/5E systems. The communication time for large particle systems amounts to about 10%- 25%, and the overall efficiency is about 35%, corre sponding to a performance of about 60 Mflop/s per CM-5E node, independent of the number of nodes.

The Network Architecture of the Connection Machine CM-5

The Connection Machine Model CM-5 Supercomputer is a massively parallel computer system designed to offer performance in the range of 1 teraflops (1012floating-point operations per second). The CM-5 obtains its high performance while offering ease of programming, flexibility, and reliability. The machine contains three communication networks: a data network, a control network, and a diagnostic network. This paper describes the organization of these three networks and how they contribute to the design goals of the CM-5.

Perspectives on supercomputing: three decades of change

I am fortunate to have had access to supercomputers for the last 28 years. Over this time I have used them to simulate time-dependent fluid flows in the compressible regime. Strong shocks and unstable multifluid boundaries, along with the phenomenon of fluid turbulence, have provided the simulation complexity that demands supercomputer power. The supercomputers I have used-the CDC 6600, 7600, and Star-100, the Cray-1, Cray-XMP, Cray-2, and Cray C-90, the Connection Machines CM-2 and CM-5, the Cray T3D, and the Silicon Graphics Challenge Array and Power Challenge Array-span three revolutions in supercomputer design: the introduction of vector supercomputing, parallel supercomputing on multiple CPUs, and supercomputing on hierarchically organized clusters of microprocessors with cache memories. The last revolution is still in progress, so its outcome is somewhat uncertain. I view these design revolutions through the prism of my specialty and through applications of the supercomputers I have used. Also, because these supercomputer design changes have driven equally important changes in numerical algorithms and the programs that implement them, I describe the three revolutions from this perspective.

The Physics of Strong Quark Binding from Lattice Simulations

Understanding the confinement mechanism in quantum chromodynamics has been a great challenge ever since a dual Meissner effect in a type II superconducting vacuum has been proposed to confine color charge by 't Hooft and Mandelstam.1l Also, various effective string and flux tube models have been conceived,ZJ,3J partly based on the dual superconductivity scenario, to arrive at a quantitative understanding of infrared aspects of QCD. Lattice gauge theory (LGT), in principle, offers the laboratory to test such ideas, as it allows for ab initio studies from the QCD Lagrangian. Early LGT attempts to compute color field distributions were necessarily limited by the available compute power and lattice methods of the period. They rendered qualitative rather than quantitative results, with lattice resolutions, a, and quarkantiquark separations, r, restrained to a >0.15 fm and r< 1 fm, respectively.4),s) Present quenched calculations have reached the accuracy necessary for quantitative studies of action and energy density distributions. These are not easily accessible since (a) the energy density carries dimension a-4 and therefore imposes a lower limit onto the lattice spacing a (due to statistical noise) and (b) one is forced to work on rather large lattice volumes to attain source separations sufficient for applicability of the string picture. On top of this one is faced with the ubiquitous problem of filtering ground state signals out of an excited state background. We have carried out a comprehensive investigation of flux tube profiles between static QQ pairs. Exploitation of state-of-the-art lattice techniques for statistical noise reduction and ground state enhancement as well as the power of parallel computers, namely small Connection Machines CM-2 and CM-5, enable us to

Spectral properties of inviscid solutions of the burgers equation with random initial conditions

The Burgers equation with random self-similar initial conditions is investigated numerically in the inviscid limit by a parallel fast Legendre transform algorithm, using Connection Machine CM-200. The use of this equation for solving the problem of nervous impulse propagation through axons is discussed. An attempt is made to simulate recent experiments where the form of the density of propagated nerve impulses, which initially had a power spectrum close to a white noise distribution, appeared similar to the triangular pulses that arise in the inviscid Burgers equation and where the 1/f power law was observed on scales larger than the typical time interval between pulses. It is shown that in the inviscid Burgers equation model the power spectra for different types of initial conditions in the developed “Burgers turbulence” regime (i.e., at a sufficiently large time) consists of two parts with a rather sharp transition between them: The spectrum virtually coincides with the initial spectra for low wavenumbers, and the 1/f2 law holds for high wavenumbers. There is no interval with an intermediate power law dependence such as 1/f. It is inferred that the true 1/f spectrum of nerve impulses propagating through axons cannot be explained in terms of the Burgers equation model and that other mechanisms must be taken into account.

Scaling up inductive learning with massive parallelism

Machine learning programs need to scale up to very large data sets for several reasons, including increasing accuracy and discovering infrequent special cases. Current inductive learners perform well with hundreds or thousands of training examples, but in some cases, up to a million or more examples may be necessary to learn important special cases with confidence. These tasks are infeasible for current learning programs running on sequential machines. We discuss the need for very large data sets and prior efforts to scale up machine learning methods. This discussion motivates a strategy that exploits the inherent parallelism present in many learning algorithms. We describe a parallel implementation of one inductive learning program on the CM-2 Connection Machine, show that it scales up to millions of examples, and show that it uncovers special-case rules that sequential learning programs, running on smaller datasets, would miss. The parallel version of the learning program is preferable to the sequential version for example sets larger than about 10K examples. When learning from a public-health database consisting of 3.5 million examples, the parallel rule-learning system uncovered a surprising relationship that has led to considerable follow-up research.

Data-Parallel Implementations of Dense Simplex Methods on the Connection Machine CM-2

We describe three data-parallel implementations of the simplex method for dense linear programming problems. The first implementation uses a full tableau and the most-negative reduced cost pivot rule, the second uses a tableau and the steepest-edge pivot rule, and the third is a revised method with explicit inverse. All are implemented on a Connection Machine CM-2 massively parallel computer system, using a variant of Fortran 90. Using special data structures called stripe arrays, we produce efficient implementations. We compare the implementations to one another and to MINOS 5.4 on a Sun workstation. Test problems are from NETLIB, supplemented by a few additional, genuinely dense models from real applications. An appendix also gives recent results on the Connection Machine CM-5. INFORMS Journal on Computing, ISSN 1091-9856, was published as ORSA Journal on Computing from 1989 to 1995 under ISSN 0899-1499.

Direct numerical simulation of turbulence on a Connection Machine CM-5

In this paper we report on our first experiences with direct numerical simulation of turbulent flow on a 16-node Connection Machine CM-5. The CM-5 has been programmed at a global level using data parallel Fortran. A two-dimensional direct simulation, where the pressure is solved using a Conjugate Gradient method without preconditioning, runs at 23% of the peak. Due to higher communication costs, 3D simulations run at 13% of the peak. A diagonalwise re-ordered Incomplete Choleski Conjugate Gradient method cannot compete with a standard CG-method on the CM-5.

UC: A Set-Based Language for Data-Parallel Programming

UC is a data-parallel extension of C designed for scientific computations on synchronous and asynchronous parallel architectures. The primary constructs of the language include sets, reductions, and parallel and asynchronous composition. Its communication model is that of a globally addressable memory, with no syntactic distinction between local and remote data references, Unlike most existing data-parallel languages, UC programs may be synchronized at multiple levels of granularity, from a strict expression-level synchronization to a coarser statement or function-level synchronization. This paper describes the language and its implementation on the Connection Machine CM-2. Experimental measurements that compare the performance of the UC compiler with that of programs written in commercial parallel languages such as CM Fortran, C*, and *Lisp are also presented.

The two-dimensional Navier-Stokes-Kuramoto-Sivashinsky equation on the Connection Machine

The two-dimensional Navier-Stokes equations with a large scale instability of the Kuramoto-Sivashinsky type, describing marginally negative eddy viscosity situations, is simulated on a Connection Machine CM-2. Up to millions of time steps at the resolution 256 2 and tens of thousands at the resolution 1024 2 are performed. A linear growth phase, a disorganized inverse cascade phase and a structured vortical phase are successively observed. In the vortical phase monopolar and multipolar structures are proliferating and display strongly depleted nonlinearities.

Cilk

Cilk (pronounced “silk”) is a C-based runtime system for multi-threaded parallel programming. In this paper, we document the efficiency of the Cilk work-stealing scheduler, both empirically and analytically. We show that on real and synthetic applications, the “work” and “critical path” of a Cilk computation can be used to accurately model performance. Consequently, a Cilk programmer can focus on reducing the work and critical path of his computation, insulated from load balancing and other runtime scheduling issues. We also prove that for the class of “fully strict” (well-structured) programs, the Cilk scheduler achieves space, time and communication bounds all within a constant factor of optimal. The Cilk runtime system currently runs on the Connection Machine CM5 MPP, the Intel Paragon MPP, the Silicon Graphics Power Challenge SMP, and the MIT Phish network of workstations. Applications written in Cilk include protein folding, graphic rendering, backtrack search, and the *Socrates chess program, which won third prize in the 1994 ACM International Computer Chess Championship.

Hypercube Algorithms for Parallel Processing of Pointer-Based Quadtrees

This paper studies the parallel construction and manipulation of pointer-based quadtrees on fine grained hypercube multiprocessors. Previous papers considered the parallel processing of linear quadtrees. Here we show that parallel pointer-based quadtrees are a viable alternative. We first solve the problem of efficiently constructing a pointer-based (or linear) quadtree from an image represented either by a binary matrix or a boundary code. Then we present efficient parallel manipulation algorithms for pointer-based quadtrees, such as finding the neighbors of all leaves in a quadtree or computing the union/intersection of two quadtrees. These algorithms improve on existing time complexities and can be implemented in line grained hypercube systems (e.g., the Connection Machine CM2). In the expected case, the space complexity is the same as for previous methods. In the worst case (of a degenerated quadtree), the space complexity increases by a factor which, for the hypercube, is smaller than the time complexity improvement. As a byproduct of our hypercube algorithms, we also obtain some PRAM algorithms for quadtrees that improve on known results.

Area and Perimeter Computation of the Union of a Set of Iso-rectangles in Parallel

Finding the area and perimeter of the union/intersection of a set of iso-rectangles is a very important part of circuit extraction in VLSI design. We combine two techniques, the uniform grid and the vertex neighborhoods, to develop a new parallel algorithm for the area and perimeter problems which has an average linear time performance but is not worst-case optimal. The uniform grid technique has been used to generate the candidate vertices of the union or intersection of the rectangles. An efficient point-in-rectangles inclusion test filters the candidate set to obtain the relevant vertices of the union or intersection. Finally, the vertex neighborhood technique is used to compute the mass properties from these vertices. This algorithm has an average time complexity of O((( n + k)/ p) + log p) where n is the number of input rectangle edges with k intersections on p processors assuming a PRAM model of computation. The analysis of the algorithm on a SIMD architecture is also presented. This algorithm requires very simple data structures which makes the implementation easy. We have implemented the algorithm on a Sun 4/280 workstation and a Connection Machine. The sequential implementation performs better than the optimal algorithm for large datasets. The parallel implementation on a Connection Machine CM-2 with 32K processors also shows good results.

Parallel simulation of DEDS via event synchronization

In this paper we use the event synchronization scheme to develop a new method for parallel simulation of many discrete event dynamic systems simultaneously. Though a few parallel simulation methods have been developed during the last several years, such as the well-known Standard Clock method, most of them are largely limited to Markovian systems. The main advantage of our method is its applicability to non-Markovian systems. For Markovian systems a comparison study on efficiency between our method and the Standard Clock method is done on Connection Machine CM-5. CM-5 is a parallel machine with both SIMD (Single Instruction, Multiple Data) and MIMD (Multiple Instruction Multiple Data) architectures. The simulation results show that if event rates of Markovian systems do not differ by much then both methods are compatible but the Standard Clock method performs better in most cases. For Markovian systems with very different event rates, our method often yields better results. Most importantly, our simulation results also show that our method works as efficiently for non-Markovian systems as for Markovian systems.

Massively-parallel handwritten character recognition based on the distance transform

A new statistical classifier for handwritten character recognition is presented. After a standard preprocessing phase for image binarization and normalization, a distance transform is applied to the normalized image, converting a black and white (B/W) into a gray scale picture. The latter is used as feature space for a k -Nearest-Neighbor classifier, based on a dissimilarity measure which generalizes the use of the distance transform itself. The classifier has been implemented on a massively-parallel processor, Connection Machine CM-2. Classification results of digits extracted from the U.S. Post Office ZIP code database and the upper-case letters of the NIST Test Data 1 are provided. The system has an accuracy of 96.73% on the digits and 94.51% on the upper-case letters when no rejection is allowed and an accuracy of 98.96%, on the digits and 98.72% on the upper-case letters at 1% error rate.

Performance Modeling for High-Order Finite Difference Methods On the Connection Machine Cm-2

This paper is concerned with modeling the perfor mance of high-order finite-difference schemes for hy perbolic problems on the Connection Machine CM-2. Specifically, we would like to determine whether the higher communication cost of higher-order methods makes them less favorable in a parallel setting than in a sequential setting. Since most difference methods are implemented using the cshift operator, we first de rive a timing model for it in CM-Fortran under the new slicewise compiler model. This model is then used to predict the performance of the difference methods with different orders applied to the 2D Bürgers' equa tions. In addition, we study the effect of varying differ ent machine performance parameters, such as the communication time and floating-point operation time, as well as problem parameters such as mesh size. Our analysis and numerical results indicate that among high-order finite difference methods, the fourth-order one is the most efficient method in that it achieves a moderate error tolerance (a few percent) with least running time.