Individual Processors Research Articles

CONNECT - an architecture for a highly parallel system based on building blocks

Parallel processing and parallel architectures have become a promising solution to the ever-increasing demands for computational power. This paper describes an approach which uses ‘self-sufficient’ building blocks with limited communication power to build efficient large-scale parallel systems. The architecture, called CONNECT, provides a medium granularity parallel processing environment with the following properties: the parallelism is fairly high, the interprocessor communication pattern is random with both small messages and large data sets being sent between processors, and a substantial amount of computation is done by individual processors.

Multichannel continuous harmonic analysis in real-time

A flexible, modular multichannel continuous real-time harmonic analyzer with the capability of precision time stamping via Global Positioning System (GPS) satellite signals of the acquired data is described. The key design features which provide this performance are discussed. These include remote distributed data conversion modules coupled via digital fiber-optic links to parallel individual digital signal processor Multibus II based modules which were all controlled by a highly integrated 486 based module. The resultant Multibus II parallel processor system was run under the iRMXIII real-time operating system and was interfaced via Ethernet to control and display workstations using a custom-designed Windows 3 environment. The ready extension of the harmonic monitoring system to transient measurement is also described.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>

An Unconventional Domain Decomposition Method for an Efficient Parallel Solution of Large-Scale Finite Element Systems

A domain decomposition algorithm based on a hybrid variational principle is developed for the parallel finite element solution of selfadjoint elliptic partial differential equations. The spatial domain is partitioned into a set of totally disconnected subdomains, each assigned to an individual processor. Lagrange multipliers are introduced to enforce compatibility at the interface points. Within each subdomain, the singularity due to the disconnection is resolved in a two-step procedure. First, the null space component of each local operator is eliminated from the local problem. Next, its contribution to the local solution is related to the Lagrange multipliers through an orthogonality condition. Finally, a conjugate projected gradient algorithm is developed for the solution of the coupled system of local null space components and Lagrange multipliers. When implemented on local memory multiprocessors, the proposed hybrid method requires fewer interprocessor communications than conventional Schur methods. It is also suitable for parallel/vector computers with shared memory. Moreover, unlike parallel direct solvers, it exhibits a degree of parallelism that is not limited by the bandwidth of the finite element system of equations. In this paper, it is applied to the solution of large-scale structural and solid mechanics problems.

A W-matrix methodology for solving sparse network equations on multiprocessor computers

The authors describe a methodology for solving efficiently sparse network equations on multiprocessor computers. The methodology is based on the matrix inverse factors (W-matrix) approach to the direct solution phase of Ax=b systems. A partitioning scheme of the W-matrix, based on the leaf-nodes of the factorization path tree, is proposed. The methodology allows the performance of all the updating operations on vector b in parallel, within each partition, using a row-oriented processing. The approach takes advantage of the processing power of the individual processors. Performance results are presented and discussed. >

Intermittent fault diagnosis in multiprocessor systems

The authors present and analyze a probabilistic model for the self-diagnosis capabilities of a multiprocessor system. In this model an individual processor fails with probability p and a nonfaulty processor testing a faulty processor detects a fault with probability q. This models the situation where processors can be intermittently faulty or the situation where tests are not capable of detecting all possible faults within a processor. An efficient algorithm that can achieve correct diagnosis with high probability in systems of O(n log n) connections, where n is the number of processors, is presented. It is the first algorithm to be able to diagnose a large number of intermittently faulty processors in a class of systems that includes hypercubes. It is shown that, under this model, no algorithm can achieve correct diagnosis with high probability in regular systems which conduct a number of tests dominated by n log n. Examples of systems which perform a modest number of tests are given in which the probability of correct diagnosis for the algorithm is very nearly one.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>

A method of finite element tearing and interconnecting and its parallel solution algorithm

AbstractA novel domain decomposition approach for the parallel finite element solution of equilibrium equations is presented. The spatial domain is partitioned into a set of totally disconnected subdomains, each assigned to an individual processor. Lagrange multipliers are introduced to enforce compatibility at the interface nodes. In the static case, each floating subdomain induces a local singularity that is resolved in two phases. First, the rigid body modes are eliminated in parallel from each local problem and a direct scheme is applied concurrently to all subdomains in order to recover each partial local solution. Next, the contributions of these modes are related to the Lagrange multipliers through an orthogonality condition. A parallel conjugate projected gradient algorithm is developed for the solution of the coupled system of local rigid modes components and Lagrange multipliers, which completes the solution of the problem. When implemented on local memory multiprocessors, this proposed method of tearing and interconnecting requires less interprocessor communications than the classical method of substructuring. It is also suitable for parallel/vector computers with shared memory. Moreover, unlike parallel direct solvers, it exhibits a degree of parallelism that is not limited by the bandwidth of the finite element system of equations.

The data alignment phase in compiling programs for distributed-memory machines

Programming distributed-memory machines requires that the data associated with a given computation be partitioned and distributed to the local storage of each individual processor. How this distribution is done affects the amount of data movement required and, therefore, the program performance. The alignment technique presented here focuses on minimizing the data movement between processors due to cross-references between multiple distributed arrays. It also simplifies, both conceptually and in practice, the task of data partition and communication generation in the context of a parallelizing compiler for distributed-memory machines. The problem of index domain alignment is formulated as finding a set of suitable alignment functions that embed the index domains of the arrays into a common index domain so as to minimize the cost of data movement. The cost function and the machine model used are abstractions of the current generation of SIMD and MIMD distributed-memory machines. The problem as formulated is shown to be NP-complete. A heuristic algorithm is devised and shown to both be efficient and provide excellent results.

Neural networks and knowledge engineering

A brief overview is given of the nature of present day artificial neural net computing and the authors emphasize the theme that in this mode of computing, knowledge is not represented symbolically, but in the form of distributed processing and localized decision rules. The authors propose that in neural-net computing, the processing is the representation. In other words, the very nature of the processing encodes the knowledge. There is no place and no need for a separate body of global rules to be used by the network for inferencing. If rules exist at all, they are in the nature of local processing steps carried out at individual processors in response to stimuli from other neurons. The authors develop this theme together with a theme which is closely related to it. The second notion is that neural networks may also be thought of and implemented in terms of heterogeneous networks, rather than always or only in terms of massive arrays of identical elemental processors.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>

Experience with mean value analysis model for evaluating shared bus, throughput-oriented multiprocessors

We report on our experience with the accuracy of mean value analysis analytical models for evaluating shared bus multiprocessors operating in a throughput-oriented environment. Having developed separate models for multiprocessors with circuit switched and split transaction, pipelined (packet switched) buses, wc compare the results of the models with those of an actual trace-driven simulation for 5,376 multiprocessor configurations.We find that the analytical models are accurate in predicting the individual processor throughputs and partial bus utilizations. For processor throughput, the difference between the results of the models and simulation are within 1% for 75% of the cases and within 3% in 94% of all cases. For partial bus utilization the model results are with 1% of simulation results in 70% of all cases and within 3% in 92% of all cases. The models are less accurate in predicting cache miss latency.

Parallel simulation of broadband ambiguity surfaces

A connection machine was used to implement parallel algorithms for the solution of the normal mode equation and for matched‐field processing over a large frequency band. The all‐acoustic layered model consists of piecewise linear sound‐speed and density profiles with a free surface and an acoustic half‐space below the bottom. Individual processors are dedicated to one particular mode with the set of processors spanning the frequency band for numerous sound‐speed profiles. Ambiguity surfaces as a function of range, depth, and frequency were generated in parallel using a linear estimator. These images show considerable variation in the sidelobes as a function of frequency while the main peak remains at the source location. [Work supported by the Office of Naval Research.]

VLSI-suited orthogonal solution of systems of linear equations

In this paper one- and two-dimensional processor arrays for the orthogonal solution of systems of linear equations are presented. Each processor cell of these processor arrays is able to carry out a Givens rotation. By using a modified Givens rotation algorithm only multiplications and additions must be executed in these processor cells. Compared to previous approaches for an orthogonal solution of systems of linear equations, the presented processor arrays have the following advantages: • -the hardware in an individual processor cell consists only of multipliers and adders. • -almost full utilization (asymptotically 100%) of these hardware components in an active processor cell can be achieved. • -the latency time for one GR, which determines the data pulse frequency of the processor arrays, can be reduced from O( b) to O(log 2 b), where b is the length of a dataword.

Adaptive performance optimization of loosely coupled processors

Routing stationary Poisson arrivals at a common scheduler probabilistically to N heterogeneous processors with exponential service distributions is considered. The objective is to minimize the overall average response time. Based on the solution to the known parameter problem, an adaptive estimator is proposed for the scheduler to learn the optimal routing probabilities when the system parameters are not known a priori. The demand is assumed to be less than the overall capacity to ensure the existence of a stable solution. The adaptive estimator utilizes the data that the scheduler can gather on its own, without any communication overhead. It is designed to function well even when a subset of processors are useless and should not receive jobs. The adaptive estimator converges to the optimum with probability 1 and in the mean square sense. Simulation experiments are presented to demonstrate its performance in a variety of practical situations. An extension involving additional unknown arrivals at individual processors is worked out.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>

Parallel optimization of tree structures for natural language parsing

Abstract A recurring problem in cognitive science involves fitting a labelled tree structure to raw data. My APRIL parser analyses natural-language grammar by optimizing a labelled tree over an input string using the stochastic optimization technique of simulated annealing. Stochastic optimization is inherently processing-intensive, so it is desirable to speed the process up by exploiting the power of parallel computing; but the task of optimizing a tree structure does not at first sight lend itself to being shared between multiple processors. I define an algorithm which allows this to be done at the cost of requiring individual processors to operate with imperfect knowledge of the current solution; and I describe results of an implementation of this algorithm which suggest that the cost may be worth paying.

Optimal design of fault-tolerant distributed systems based on a recursive algorithm

The authors address the issue of optimal design (in terms of the number of processors) of a distributed system which is based on a recursive algorithm for fault tolerance (RAFT). The reliability and performance of the system using RAFT are determined as a function of reliability of individual processors and the number of fault modes in a processor. Also discussed are how to determine the design policies when the objective is to minimize the average system failure. Several numerical examples illustrate the results. >

A note on the load balancing problem for coarse grained hypercube dictionary machines

The main problem for the design of dictionary machines on coarse grained hypercube multiprocessors, in comparison to the widely studied dictionary problem for fine grained hypercube multiprocessors, is that due to unequal distribution of the inserted and deleted records, the sizes of the sets stored at the individual processors may vary considerably. This problem, which is usually referred to as the load balancing problem, may lead to considerable degradation of the dictionary machine's performance. In this note we show that the load balancing problem for coarse grained hypercube dictionary machines can be solved with provable bounds on the sizes of the data sets, and with only little computational overhead.

Convergence and stability of distributed stochastic iterative processes

A mathematical framework is proposed for convergence analysis of stochastic iterative processes arising in applications of pseudogradient optimization algorithms. The framework is based upon the concepts of stochastic difference inequalities and vector Lyapunov functions, which are ideally suited for reducing the dimensionality problem arising in testing convergence of distributed parallel schemes. By applying the M-matrix conditions to a test matrix having a dimension equal to the number of the processor in the scheme, one can use the framework to select suitable scaling factors for each individual processor, producing a satisfactory convergence rate of the overall iterative process. The results provide new convergence tests for distributed iterative processes arising in decentralized extremal regulation, adaptation, and parameter estimation schemes. >

Designing massively parallel optical computers: a case study

This paper presents a case study in the design and analysis of a massively parallel optical computer, SPARO, a novel scalable computer intended for symbolic and numeric computing. SPARO was designed for fine-grained parallel processing of combinator graph reduction, a special case of the graph reduction computational model, found most appropriate for parallel optical processing in earlier studies. The architecture consists of a planar array of optical processors that communicate through simple messages (data packets) over an optical interconnection network. A technique called instruction passing is used to realize distributed control of the architecture. Instruction passing can also be used to implement complex structures such as recursion and iteration. Each individual processor in SPARO is a finite state machine that is implemented using symbolic substitution techniques, while gateable interconnects are used to realize data movements between the processors and network. Performance analysis of SPARO reveals that while discrete computing structures can be implemented using optical techniques, massively parallel optical architectures for traditional computational models are currently unable to compete with electronic ones due to the lack of large scale addressable optical memory devices and large scale integratable optical computing elements. However, optical interconnections appear very promising for providing the network throughput necessary for these parallel architectures.

The use of feedback in multiprocessors and its application to tree saturation control

Using feedback control schemes in multiprocessor systems is proposed. In a multiprocessor, individual processors do not have complete control over, nor information about, the overall state of the system. The potential exists, then, for the processors to unknowingly interact in such a way as to degrade the performance of the system. An example of this is the problem of tree saturation caused by hot-spot accesses in multiprocessors using multistage interconnection networks. Tree saturation degrades the performance of all processors in the system, including those not participating in the hot spot activity. Feedback schemes can be used to control tree saturation, reducing degradation to memory request that are not to the hot spot, thereby increasing overall system performance. As a companion to feedback schemes, damping schemes are also considered. Simulation studies show that feedback schemes can improve overall system performance significantly and with relatively little hardware cost in many cases. Damping schemes in conjunction with feedback are shown to further improve. >

Optimal fault-tolerant signal detection

G.G.L. Meyer and H.L. Weinert (see ibid., vol.ASSP-34, p.973-8, Aug. 1986) use the majority rule for fault-tolerant detection. The authors extend this and examine a redundant system with a general voting rule. They present an optimal design for each individual processor and the optimal selection of the k-out-of-M voting rule. They give an analytic solution to the optimal voting policy for the special case of low false alarm rate, with arbitrary failure probabilities and receiver operating characteristics (ROCs). Performance is improved significantly by using the optimal voting rule instead of the majority voting rule, especially in the tail regions of the ROC. >

Analysis of prioritized crossbar multiprocessor systems

This paper analyzes the performance of crossbar multiprocessor systems under the assumption that processors are organized in a priority hierarchy. A basic model is derived for estimating the probability of acceptance of a memory request for each individual processor with a different priority. Improvements are made by using an adjusted memory request rate. Analytical results are validated by comparison with simulation data for configurations up to 64 X 64 with request rate ⩽ 1. Finally, it is shown that the effective memory bandwidth can be calculated accurately from acceptance probabilities of individual processors.