Regular Iterative Algorithms Research Articles

A multilayer neural network with piecewise-linear structure and back-propagation learning

A multilayer neural network which is given a two-layer piecewise-linear structure for every cascaded section is proposed. The neural networks have nonlinear elements that are neither sigmoidal nor of a signum type. Each nonlinear element is an absolute value operator. It is almost everywhere differentiable, which makes back-propagation feasible in a digital setting. Both the feedforward signal propagation and the backward coefficient update rules belong to the class of regular iterative algorithms. This form of neural network specializes in functional approximation and is anticipated to have applications in control, communications, and pattern recognition.

Multi-dimensional parallel computing structures for regular iterative algorithms

In this paper the mapping of algorithms into multidimentional systolic structures by setting bounds on the computation time complexity, considering the dimension of these parallel computing structures, is discussed. A decomposition method based on the properties of associativity and commutativity of linear operators used, thus mapping to a higher dimensional structure than that by a conventional approach, without increasing the number of processing elements is proposed. This makes 3D integration reasonably popular. An improvement of latency which is crucial for real time applications is achieved.

Subspace scheduling and parallel implementation of non-systolic regular iterative algorithms

The study of Regular Iterative Algorithms (RIAs) was introduced in a seminal paper by Karp, Miller, and Winograd in 1967. In more recent years, the study of systolic architectures has led to a renewed interest in this class of algorithms, and the class of algorithms implementable on systolic arrays (as commonly understood) has been identified as a precise subclass of RIAs. In this paper, we shall study the dependence structure of RIAs that are not systolic; examples of such RIAs include matrix pivoting algorithms and certain forms of numerically stable two-dimensional filtering algorithms. It has been shown that the so-called hyperplanar scheduling for systolic algorithms can no longer be used to schedule and implement non-systolic RIAs. Based on the analysis of a so-called computability tree we generalize the concept of hyperplanar scheduling and determine linear subspaces in the index space of a given RIA such that all variables lying on the same subspace can be scheduled at the same time. This subspace scheduling technique is shown to be asymptotically optimal, and formal procedures are developed for designing processor arrays that will be compatible with our scheduling schemes. Explicit formulas for the schedule of a given variable are determined whenever possible; subspace scheduling is also applied to obtain lower dimensional processor arrays for systolic algorithms.

A family of new efficient arrays for matrix multiplication

The authors present a regular iterative algorithm for matrix multiplication and show that several well-known matrix multiplication arrays are directly obtained from it, differing only in the choice of iteration vector. They then present a regular iterative algorithm for matrix multiplication using the S. Winograd method (1968) and show in detail how to derive one array from this algorithmic description. Other arrays in the same family can similarly be obtained for different choices of the iteration space. The new arrays compute the product of two matrices faster than available conventional arrays and use a smaller number of processor cells. >

Regular iterative algorithms and their implementation on processor arrays

Some recent results are summarized concerning a class of algorithms known as regular iterative algorithms, particularly with respect to their implementations on processor arrays. Regular iterative algorithms contain all algorithms executed by systolic arrays as a proper subclass and are therefore of considerable importance in real-time signal processing applications. Some general concepts concerning the design of parallel architectures are introduced, and the importance of devising special techniques that utilize any available structure in the algorithm is highlighted. A generic description of the existing methodologies for the systematic design of systolic arrays is given. Using some simple examples, the limitations of these methods are shown. A formal methodology is proposed that overcomes the difficulties in the existing procedures.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>

Array architectures for iterative algorithms

Regular mesh-connected arrays are shown to be isomorphic to a class of so-called regular iterative algorithms. For a wide variety of problems it is shown how to obtain appropriate iterative algorithms and then how to translate these algorithms into arrays in a systematic fashion. Several "systolic" arrays presented in the literature are shown to be specific cases of the variety of architectures that can be derived by the techniques presented here. These include arrays for Fourier Transform, Matrix Multiplication, and Sorting.