Connection Machine Research Articles

Scaling up inductive learning with massive parallelism

Machine learning programs need to scale up to very large data sets for several reasons, including increasing accuracy and discovering infrequent special cases. Current inductive learners perform well with hundreds or thousands of training examples, but in some cases, up to a million or more examples may be necessary to learn important special cases with confidence. These tasks are infeasible for current learning programs running on sequential machines. We discuss the need for very large data sets and prior efforts to scale up machine learning methods. This discussion motivates a strategy that exploits the inherent parallelism present in many learning algorithms. We describe a parallel implementation of one inductive learning program on the CM-2 Connection Machine, show that it scales up to millions of examples, and show that it uncovers special-case rules that sequential learning programs, running on smaller datasets, would miss. The parallel version of the learning program is preferable to the sequential version for example sets larger than about 10K examples. When learning from a public-health database consisting of 3.5 million examples, the parallel rule-learning system uncovered a surprising relationship that has led to considerable follow-up research.

Parallel computation of radiation view factors between two arbitrarily oriented surfaces

Two implementations for parallel computation of radiation heat transfer view factors are formulated and tested for a model problem. Using a sufficiently large number of processors and a suitable communications paradigm, the solution time for the problem considered here scales linearly with the number of surface elements in the parallel implementation rather than the quadratic scaling obtained using a serial approach. For larger problems, significantly shorter solution times are obtained using the massively parallel Connection Machine 5 than those obtained on a single processor of the Cray C-90, a traditional vector supercomputer. Parallel performance was degraded for implementations using too few processors or data layouts leading to inefficient processor communication.

PARAMICS?Parallel microscopic simulation of road traffic

This paper describes work done on the original PARAMICS project, which was developed for the Edinburgh Parallel Computing Centre to examine parallel microscopic road traffic simulation. The simulator, constructed originally for a Thinking Machines Connection Machine (CM-200), uses a data-parallel approach to simulate approximately 200,000 vehicles on 20,000 miles of roadway. More recent work has focused on the use of a message-passing paradigm, with a 256-node CRAY T3D as the target machine. The message-passing version of PARAMICS, PARAMICS-MP, is inherently scalable and can model many smaller networks on a broad range of platforms.

Parallel computation of radiation view factors between two arbitrarily oriented surfaces

Two implementations for parallel computation of radiation heat transfer view factors are formulated and tested for a model problem. Using a sufficiently large number of processors and a suitable communications paradigm, the solution time for the problem considered here scales linearly with the number of surface elements in the parallel implementation rather than the quadratic scaling obtained using a serial approach. For larger problems, significantly shorter solution times are obtained using the massively parallel Connection Machine 5 than those obtained on a single processor of the Cray C-90, a traditional vector supercomputer. Parallel performance was degraded for implementations using too few processors or data layouts leading to inefficient processor communication.

Spectral properties of inviscid solutions of the burgers equation with random initial conditions

The Burgers equation with random self-similar initial conditions is investigated numerically in the inviscid limit by a parallel fast Legendre transform algorithm, using Connection Machine CM-200. The use of this equation for solving the problem of nervous impulse propagation through axons is discussed. An attempt is made to simulate recent experiments where the form of the density of propagated nerve impulses, which initially had a power spectrum close to a white noise distribution, appeared similar to the triangular pulses that arise in the inviscid Burgers equation and where the 1/f power law was observed on scales larger than the typical time interval between pulses. It is shown that in the inviscid Burgers equation model the power spectra for different types of initial conditions in the developed “Burgers turbulence” regime (i.e., at a sufficiently large time) consists of two parts with a rather sharp transition between them: The spectrum virtually coincides with the initial spectra for low wavenumbers, and the 1/f2 law holds for high wavenumbers. There is no interval with an intermediate power law dependence such as 1/f. It is inferred that the true 1/f spectrum of nerve impulses propagating through axons cannot be explained in terms of the Burgers equation model and that other mechanisms must be taken into account.

A parallel descent algorithm for convex programming

In this paper, we propose a parallel decomposition algorithm for solving a class of convex optimization problems, which is broad enough to contain ordinary convex programming problems with a strongly convex objective function. The algorithm is a variant of the trust region method applied to the Fenchel dual of the given problem. We prove global convergence of the algorithm and report some computational experience with the proposed algorithm on the Connection Machine Model CM-5.

A BLOCK-PARALLEL CONJUGATE GRADIENT METHOD FOR SEPARABLE QUADRATIC PROGRAMMING PROBLEMS^1

For a large-scale quadratic programming problem with separable objective function, a variant of the conjugate gradient method can effectively be applied to the dual problem. In this paper, we consider a block-parallel modification of the conjugate gradient method, which is suitable for implementation on a parallel computer. More precisely, the method proceeds in a block Jacobi manner and executes the conjugate gradient iteration to solve quadratic programming subproblems associated with respective blocks. We implement the method on a Connection Machine Model CM-5 in the Single-Program Multiple-Data model of computation. We report some numerical results, which show that the proposed method is effective particularly for problems with some block structure.

The Physics of Strong Quark Binding from Lattice Simulations

Understanding the confinement mechanism in quantum chromodynamics has been a great challenge ever since a dual Meissner effect in a type II superconducting vacuum has been proposed to confine color charge by 't Hooft and Mandelstam.1l Also, various effective string and flux tube models have been conceived,ZJ,3J partly based on the dual superconductivity scenario, to arrive at a quantitative understanding of infrared aspects of QCD. Lattice gauge theory (LGT), in principle, offers the laboratory to test such ideas, as it allows for ab initio studies from the QCD Lagrangian. Early LGT attempts to compute color field distributions were necessarily limited by the available compute power and lattice methods of the period. They rendered qualitative rather than quantitative results, with lattice resolutions, a, and quarkantiquark separations, r, restrained to a >0.15 fm and r< 1 fm, respectively.4),s) Present quenched calculations have reached the accuracy necessary for quantitative studies of action and energy density distributions. These are not easily accessible since (a) the energy density carries dimension a-4 and therefore imposes a lower limit onto the lattice spacing a (due to statistical noise) and (b) one is forced to work on rather large lattice volumes to attain source separations sufficient for applicability of the string picture. On top of this one is faced with the ubiquitous problem of filtering ground state signals out of an excited state background. We have carried out a comprehensive investigation of flux tube profiles between static QQ pairs. Exploitation of state-of-the-art lattice techniques for statistical noise reduction and ground state enhancement as well as the power of parallel computers, namely small Connection Machines CM-2 and CM-5, enable us to

Massively parallel finite element analysis of coupled, incompressible flows: A benchmark computation of baroclinic annulus waves

AbstractCoupled, three‐dimensional, time‐dependent, incompressible flows in a differentially heated, rotating annulus are simulated using a parallel implementation of the Galerkin finite element method on the Connection Machine 5 (CM‐5) supercomputer. The development of baroclinic annulus waves is computed and found to be consistent with previous experimental reseults. The implementation of a repeated spectral bisection element‐partitioning technique significantly increases the computation speed over a strategy which randomly maps elements to processors, yielding sustained calculation rates of 8.1 GFLOPS on 512 processors of the CM‐5.

A case study in parallel computation: Viscous flow around an ONERA M6 wing

AbstractWe examine the solution of a practical engineering problem on a parallel computer. The problem involves the steady laminar viscous flow about an ONERA M6 wing and the computer is a 64‐processing‐node Connection Machine CM‐5E. We show that efficient domain decomposition procedures lead to a balanced load on the processors and low communication times. The net result is that solutions can be attained in roughly 20 min elapsed time for a 48,011‐node, 266,566‐element unstructured mesh. We conclude that this is sufficiently fast to support the design process.

Three‐step explicit finite element computation of shallow water flows on a massively parallel computer

AbstractMassively parallel finite element strategies for large‐scale computations of shallow water flows and contaminant transport are presented. The finite element discretizations, carried out on unstructured grids, are based on a three‐step explicit formulation both for the shallow water equations and for the advection‐diffusion equation governing the contaminant transport. Parallel implementations of these unstructured‐grid‐based formulations are carried out on the Army High Performance Computing Research Center Connection Machine CM‐5. It is demonstrated with numerical examples that the strategies presented are applicable to large‐scale computations of various shallow water flow problems.

Data-Parallel Implementations of Dense Simplex Methods on the Connection Machine CM-2

We describe three data-parallel implementations of the simplex method for dense linear programming problems. The first implementation uses a full tableau and the most-negative reduced cost pivot rule, the second uses a tableau and the steepest-edge pivot rule, and the third is a revised method with explicit inverse. All are implemented on a Connection Machine CM-2 massively parallel computer system, using a variant of Fortran 90. Using special data structures called stripe arrays, we produce efficient implementations. We compare the implementations to one another and to MINOS 5.4 on a Sun workstation. Test problems are from NETLIB, supplemented by a few additional, genuinely dense models from real applications. An appendix also gives recent results on the Connection Machine CM-5. INFORMS Journal on Computing, ISSN 1091-9856, was published as ORSA Journal on Computing from 1989 to 1995 under ISSN 0899-1499.

Parallelizing the fast wavelet transform

Two new algorithms for the Fast Wavelet Transform on parallel computers are presented. They are compared to a previously published algorithm [3,4] and are found to outperform this algorithm for a Connection Machine (CM) Fortran implementation of a two-dimensional wavelet transform on a CM-200 and a CM-5.

Direct numerical simulation of turbulence on a Connection Machine CM-5

In this paper we report on our first experiences with direct numerical simulation of turbulent flow on a 16-node Connection Machine CM-5. The CM-5 has been programmed at a global level using data parallel Fortran. A two-dimensional direct simulation, where the pressure is solved using a Conjugate Gradient method without preconditioning, runs at 23% of the peak. Due to higher communication costs, 3D simulations run at 13% of the peak. A diagonalwise re-ordered Incomplete Choleski Conjugate Gradient method cannot compete with a standard CG-method on the CM-5.

An effective data parallel self‐starting explicit methodology for computational structural dynamics on the connection machine CM‐5

AbstractThis paper discusses the implementation aspects and our experiences towards a data parallel explicit self‐starting finite element transient methodology with emphasis on the Connection Machine (CM‐5) for linear and non‐linear computational structural dynamic applications involving structured and unstructured grids. The parallel implementation criteria that influence the efficiency of an algorithm include the amount of communication, communication routing, and load balancing. To provide simplicity, high level of accuracy, and to retain the generality of the finite element implementation for both linear and non‐linear transient explicit problems on a data parallel computer which permit optimum amount of communications, we implemented the present self‐starting dynamic formulations (in comparison to the traditional approaches) based on nodal displacements, nodal velocities, and elemental stresses on the CM‐5. Data parallel language CMFortran is employed with virtual processor constructs and with:SERIAL and:PARALLEL layout directives for arrays. The communications via the present approach involve only one gather operation (extraction of element nodal displacements or velocities from global displacement vector) and one scatter operation (dispersion of element forces onto global force vector) for each time step. These gather and scatter operations are implemented using the Connection Machine Scientific Software Library communication primitives for both structured and unstructured finite element meshes. The implementation aspects of the present self‐starting formulations for linear and elastoplastic applications on serial and data parallel machines are discussed. Numerical test models for linear and non‐linear one‐dimensional applications and a two‐dimensional unstructured finite element mesh are then illustrated and their performance studies are discussed.

Weather forecasting on parallel architectures

Global forecasting at the National Meteorological Center is performed using a spherical harmonics spectral model. The model is used in global data assimilation, and in the aviation and medium range forecast production suites. In this presentation, the major numerical and physical aspects of the model are highlighted, and parallelization considerations are analyzed and applied to three parallel computer architectures. The first parallelization is on a parallel vector processor shared memory platform such as Cray YMP or C90. The other two are on distributed memory platforms: a data parallel implementation on Connection Machines and a single program multiple data (SPMD) adaptation to a Cray T3D machine.

The numerical simulation of shock bifurcation near the end wall of a shock tube

The reflection of a normal shock wave from the end wall of a two-dimensional channel has been numerically simulated to investigate the unsteady, viscous interaction aspects of shock bifurcation. The numerical simulation implements a data-parallel version of the Flux-Corrected Transport algorithm that has been coupled to the viscous transport terms of the Navier–Stokes equations. All numerical simulations were performed on the Connection Machine, the CM-5. The results indicate that the shear layer in the bifurcation zone is unstable, and the large and small scale vortices lead to complex flow patterns. In addition, the high-speed, essentially inviscid flow, which is adjacent to the shear layer, is deflected over this region. As a result, weak shock and expansions waves are generated and a reattachment shock is formed at the trailing edge of the interaction region. The impact of heat transfer, Reynolds number, and incident shock strength on the viscous interaction is also investigated. Heat transfer to the walls weakens the interaction between the boundary layer and the reflected shock. However, the decreased Reynolds number and increased shock strength enhances the interaction.

Completely parallelizable preconditioning methods

AbstractA class of parallelizable preconditioned iterative methods for the solution of certain finite difference or finite element linear systems of equations is presented. The methods are based on calculation of approximate inverses of the SSOR factorization. The speed of the methods may be increased by making the approximation of the inverse more accurate. The construction of the preconditioning as well as the solution of the preconditioning systems (realized by matrix–vector multiplication) can be made in parallel over the total amount of meshpoints. The methods are suitable for implementation on massively parallel computers such as connection machines. Problems with constant as well as strongly varying orthotropy are examined and the methods are compared to other parallel techniques with respect to rate of convergence, computational complexity and consumed CM200 computing time. We report a small but significant decrease in computing time compared to the (until now) most frequently used completely parallel preconditioning, the Jacobi method.

The two-dimensional Navier-Stokes-Kuramoto-Sivashinsky equation on the Connection Machine

The two-dimensional Navier-Stokes equations with a large scale instability of the Kuramoto-Sivashinsky type, describing marginally negative eddy viscosity situations, is simulated on a Connection Machine CM-2. Up to millions of time steps at the resolution 256 2 and tens of thousands at the resolution 1024 2 are performed. A linear growth phase, a disorganized inverse cascade phase and a structured vortical phase are successively observed. In the vortical phase monopolar and multipolar structures are proliferating and display strongly depleted nonlinearities.

A dynamic route assignment model for guided and unguided vehicles with a massively parallel computing architecture

This article presents a dynamic assignment model which has been developed primarily for ATMS/ATIS real-time applications. In order to satisfy the real-time computational requirements, the proposed model has been developed on the Connection Machine, CM-2, a massively parallel computer. Its implementation on the Connection Machine is aimed to exploit the parallel nature of the problem and to take advantage of the underlying computing architecture. The model follows an integrated assignment-simulation framework which assigns both guided and unguided vehicles to the network dynamically in both spatial and temporal dimensions. It uses a learning process in which the new route assignment uses information gained from previous iteration in computation of new paths. During each iteration, guided vehicles follow routes according to time-dependent shortest paths while unguided vehicles follow static shortest paths. Several numerical examples have been carried out to illustrate the potential applications in analyzing the network performance under various ATMS scenarios.