Individual Processors Research Articles

FINDING OPTIMUM WAVEFRONT OF PARALLEL COMPUTATION∗

Data parallelism, in which the same operation is performed on many elements of an n-dimensional array, is one of the most powerful methods of extracting parallelism in scientific computation. One form of data parallelism involves defining a sequence of parallel wavefronts of a computation. Each wavefront consists of an (n - l)-dimcnsional subarray of the evaluated array and all wavefront elements are evaluated simultaneously. Different wavefronts result in different performance, so the question arises how to determine the wavefronts that result in the minimum computation time. Wavefront determination should define also allocation of wavefront elements to processors. In this paper we present efficient algorithms for determining the optimum wavefront and for partitioning it into sections assigned to individual processors. Presented algorithms are applicable to computations that are defined over two or higher dimensional arrays and are executed on distributed memory machines interconnected into a one or two-dimensional processor array.

The GF11 parallel computer

GF11 is a parallel computer operational at IBM's T.J. Watson Research Center. It is based on the SIMD (Single Instruction Multiple Data) model of parallel computing. GF11 attains its peak execution rate of 11.3 GigaFlops by using 566 identical processing elements, each capable of delivering 20 MegaFlops. Each processor has its own 64 Kb static RAM that can access a 32-bit word on each floating point operation, a 2 Mb dynamic RAM that operates at one fourth of the SRAM speed, and a 1 Kb register file that provides four accesses per floating point operation. The processors communicate through a 576×576 Benes network, organized as three stages of 24×24 crossbar switches. The network provides 11.3 Gb/sec of communication bandwidth to the processors and allows the processors to dynamically reconfigure themselves into arrays f various dimensions and sizes or other interesting interconnection patterns such as a tree, hypercube, etc. This configuration can take place on every word transfer without sacrificing the bandwidth. GF11 has several architectural enhancements to circumvent the limitations of the standard SIMD model such as the ability to perform multiple operations in every instruction and the ability to modify the operations occurring within individual processors based on processor specific data. Preliminary benchmarking efforts on some applications indicate that near peak performance can be sustained on most applications, including some that were previously believed to be ill suited SIMD machines. Minimal restructuring of programs and algorithms is required for achieving this performance. The architecture of GF11 is summarized in this paper and the implementations of Finite Element analysis, LU decomposition, Gaussian Elimination, and Fast Fourier Transform are discussed to illustrate GF11's ability to deliver good performance with minor program restructing.

Tools for real-time signal-processing research

The use of digital signal processing (DSP) devices for real-time communication applications is discussed. The authors comment on distinguishing aspects of DSP architecture, describing not so much individual processors as those features common to DSPs and distinct from modern general-purpose processors. They describe three DSP32xx-based machines that support DSP algorithm implementation: SURF-board, HoBo, and DSP3. They also described rtpi, a source-code debugger for workstations and for the AT&T DSP32C signal-processor integrated circuit, and dspx, a collection of subroutines and host programs that provides an execution environment for DSPs akin to the UNIX environment. These tools facilitate the transfer of algorithms from mainframes or workstations to DSP hardware. Included are case studies of two real-time implementations: the low-delay CELP (LD-CELP) speech coder and the decoder side of the perceptual audio coder (PAC), an algorithm that compresses CD-quality audio into a 128-kb/s stream without perceptible distortion. >

Parallel hypothesis verification

The verification of identity and pose hypotheses in model-based three-dimensional (3D) object recognition systems can involve a time-consuming image rendering operation followed by pixel-level comparison of the input and rendered images. In situations where many such hypotheses need to be verified, exploitation of inherent data-parallelism between hypotheses and their corresponding object models offers the potential for increased efficiency of the overall object recognition system. A prototype system is described for distribution of hypotheses and the accompanying rendering tasks to individual processors in a loosely-coupled computing environment, and excellent performance improvements are demonstrated over a single-processor implementation.

Design of a VLIW architecture constructed from standard RISC chips: A case study of hardware/software codesign

This paper describes the design of a VLIW software/hardware architecture based on off-the-shelf RISC-processors as a tightly coupled project of compiler and hardware construction. It is shown how problems arising from using predefined complex components (Motorola MC88K) have been solved by jointly designing special hardware components and developing advanced code generation techniques. Topics discussed in detail include how to guarantee strict synchronization between the individual processors, register-to-register communication and handling of interrupts.

Application of massive parallel computer to computational wind engineering

The airflow around a 2D square rib placed within a boundary layer is computed by means of a massive parallel computer. The Navier-Stokes equations are solved by a finite difference method with the well-known simulation method of ABMAC. The present paper deals with the implementation and optimization of this numerical method in connection with the massive parallel computer system, which is a MIMD (Multi-Instructions and Multi-Data Stream) computing system consisting of up to 1024 processors, each with 16 Mbytes of local memory. Parallelization of the algorithm is achieved by regional partitioning, i.e. grid partitioning; a grid system of 242 × 242 cells is divided into subdomains, and data belonging to each subdomain is distributed to each processor. Computations are carried out for different numbers of processors and for different algorithms of data exchange between the processors; the speedup ratio (i.e. increasing ratio of computing speed) and the efficiency of parallelization were measured. The results show that the communication method between individual processors and the size of the divided subdomains are rather crucial in their effects on granularity (i.e. the ratio of calculating time to communicating time for each processor).

Application of massive parallel computer to computational wind engineering

The airflow around a 2D square rib placed within a boundary layer is computed by means of a massive parallel computer. The Navier-Stokes equations are solved by a finite difference method with the well-known simulation method of ABMAC. The present paper deals with the implementation and optimization of this numerical method in connection with the massive parallel computer system, which is a MIMD (Multi-Instructions and Multi-Data Stream) computing system consisting of up to 1024 processors, each with 16 Mbytes of local memory. Parallelization of the algorithm is achieved by regional partitioning, i.e. grid partitioning; a grid system of 242 × 242 cells is divided into subdomains, and data belonging to each subdomain is distributed to each processor. Computations are carried out for different numbers of processors and for different algorithms of data exchange between the processors; the speedup ratio (i.e. increasing ratio of computing speed) and the efficiency of parallelization were measured. The results show that the communication method between individual processors and the size of the divided subdomains are rather crucial in their effects on granularity (i.e. the ratio of calculating time to communicating time for each processor).

Segmented array reconstructions for tomographic imaging of large objects: Theory and an application to neutron computed tomography

many nondestructive testing applications, high resolution images are desired of large engineering superstructures, such as aircraft. Traditionally, this is done by sequentially imaging small segments of the objects using small area detectors. Recently, we have developed a position sensitive neutron detector system using a LiF-ZnS scintillator screen and a cooled charge coupled device (CCD) camera. Large arrays of these individual detectors can be assembled to image large areas simultaneously at high resolution. For tomography, radiographs are obtained from multiple viewing angles with the large area detector. Tomographic reconstruction of large arrays is prohibitively time consuming and has several associated practical problems. By using the inherent segmentation of the detector, the reconstruction problem can be reduced to one of several small area reconstructions performed in parallel. A parallel tomographic reconstruction scheme based on segmented arrays is presented. It is shown that the parallel algorithm produces images that are identical to images produced without segmentation, however, using smaller individual arrays. The intrinsically parallel segmented array reconstruction provides a means of rapidly reconstructing images of any dimension using small, tractable arrays. An N x N two-dimensional tomographic reconstruction can be performed with m individual detector segments and m2 processors each reconstructing 2N/m x 2 x N/m image segments. This approach is demonstrated for neutron tomography; however, it is applicable to any tomographic imaging problem.

The FAVOR algorithm for 3D PET data and its implementation using a network of transputers

The recent appearance of septa-retractable positron emission tomography (PET) scanners has created a demand for fast three-dimensional (3D) reconstruction. While algorithms based on filtered backprojection have been developed to reconstruct PET data acquired in 3D (volume mode), they are computationally intensive owing to the large numbers of coincidence lines and image voxels that must be processed. Such linear algorithm are well suited to parallel implementation on a network of individual processors. The authors present an algorithm for fast volume reconstruction (FAVOR) of PET data, and describe its parallel implementation on a network of 44 T800 transputers. A reconstruction time of 1200 s has been achieved with this configuration of 44 T800 transputers. A reconstruction time of 1200 s has been achieved with this configuration of 44 T800 transputers comparable to that required by a single i860 processor. The prospects exist for 3D reconstruction times of around 120 s with the new generation of T9000 transputers.

Work-Optimal Asynchronous Algorithms for Shared Memory Parallel Computers

This paper develops shared memory algorithms for asynchronous processor systems that require the same expected work as the best PRAM algorithms. These algorithms operate efficiently under general asynchronous processor behavior (where individual processor speeds are allowed to vary widely over time). This paper achieves these results by employing a methodology that uses randomization to schedule subtasks of a parallel program. The resulting algorithms allow processors to (i) have arbitrary asynchronous behavior, (ii) have fail-stop errors, (iii) join a computation at any time, and (iv) have no unique identifiers. This paper develops a performance metric for asynchronous parallel computations, called work, which is the total number of instructions (including busy-waiting instructions) performed by a collection of parallel processors during a computation. The main result is to compute any associative function of n variables with $O(n)$ expected work, using up to $n/\log n\log^* n$ asynchronous processors, and with $O(n\log n)$ expected work using up to n processors. These results provide a synchronization primitive that can be used to transform any PRAM program into an asynchronous PRAM program.

An overview of High Performance Fortran

Since its introduction over three decades ago, Fortran has been the language of choice for scientific programming for sequential computers. Exploiting the full capability of modern architectures, however, increasingly requires more information than ordinary Fortran 77 or Fortran 90 programs provide. This information applies to such areas as• Opportunities for parallel execution• Type of available parallelism - MIMD, SIMD, or some combination• Allocation of data among individual processor memories• Placement of data within a single processor

Dynamic adaptive surface rendering within a distributed memory multiprocessor

We describe a software-based approach to computing surface renderings from three-dimensional volume primitives using a general purpose distributed memory multiple-instruction, multiple-data multiprocessor architecture. The system reduces total elapsed image rendering time by exploiting distributed computing and using dynamic algorithm selection to reduce individual processor rendering time. We briefly describe the system pipeline architecture, the operation of the pipeline, and the computational bottleneck our research addresses. We then describe two adaptively terminating surface rendering algorithms. Our new approach to reducing surface rendering time uses the two algorithms and a procedure for predicting individual processor workload from the image-rendering parameters. The technique we describe locally and dynamically selects the optimum surface-rendering algorithm at each processor in the multiprocessor machine, thereby substantially reducing the total elapsed image-rendering time.

SMILI: Visualization of asynchronous massively parallel programs

A visualization model has been developed to analyze the performance of a massively parallel algorithm. Most visualization tools that have been developed so far for performance analysis are based generally on individual processor information and communication patterns (e.g., processor load, message traffic, etc.). However, these tools are inadequate for massively parallel computations. It is difficult to comprehend the visual information for many processors. The scientific visualization in multicomputing for interpretation of large amounts of information (SMILI) model addresses this problem by using abstract representations to attain a composite picture, which gives better insight to the behavior of the algorithm. Chernoff's Faces have been selected to represent the multidimensional data because of their ability to portray multidimensional data in a very perceptive manner. SMILI has been used on an asynchronous massively parallel partial differential equation solver based on the multigrid paradigm. The visualization tool helps in tuning the control parameters of the multigrid algorithm to get optimal results. In asynchronous algorithms, the non-deterministic way in which messages are exchanged may lead to some unforeseeable behavior. SMILI helps detect the anomalies and indicate the causes of irregularities that may arise during execution. Once the causes have been determined, the control parameters can be further tuned to eliminate the erroneous behavior in the consecutive executions.

A distributed simulator for power system analysis using a hypercube computer

The use of distributed systems is expected to realize characteristics of flexibility, high modularity, ease of development, high-speed processing and cost performance in several fields. Distributed systems on parallel or distributed computers are studied in order to obtain these characteristics. We have developed a distributed simulation method for the extension or maintenance of power systems in the simulator. In this paper, we describe the distribution methods of power system simulation. First, we propose a distributed algorithm for solving the network equations of power system simulation. Second, we describe a way of mapping tasks, which are to be separated for distributed simulation, to a hypercube computer, NCube/10. In the hypercube computer, each bus and control circuits of a generator in a power system are assigned to individual processors. Finally, we present the results of installing and evaluating the proposed algorithm and distributed simulation techniques on a hypercube computer.

Real-time data processes in a balloon-borne apparatus

In a balloon-borne experiment signals from detectors are processed in a balloon-borne apparatus to fit the data bandwidth of storage devices installed in the apparatus. The process begins at front-end processors receiving signals from detectors and proceeds under successive stages by dedicated processors. Event data are finally fed to a computer bank where the events are further selected before being written into the storage devices. The computer bank is made of transputer modules. The transputers are widely employed and some processors are implemented with the transputers. The integration of the processors is described and the function and implementation of the individual processors are outlined. The authors are planning a series of balloon-borne experiments searching for antiparticles of cosmic origins. Firstly the authors will measure the antiproton flux. Antiprotons need to be identified under the overwhelming background of protons. >

On the complexity of deadlock-free programs on a ring of processors

A combinatorial view of deadlock (as in Dijkstra's self-stabilizing systems) is presented which leads to precise lower bounds on the complexity of programs. Specifically, we consider a directed ring of k individual processors, each having n states, identical programs, and asynchronous activity. Our main theorem establishes the minimum size (i.e., complexity) of a program for which no global state is in deadlock.

A parallel computation revealing the role of the in vivo environment in shaping the catalytic structure of a mitochondrial RNA transcript

We study the search for folded structures performed by an RNA molecule as it is being assembled by sequential incorporation of nucleotides. The specific process we shall focus on is the transcription of intron 4 of the yeast apocytochrome b gene. We prove that the structure generated by sequential folding is endowed with catalytic potential. More specifically, the formation of all conserved interactions in the catalytic core and intramolecular splicing substrate may be achieved by sequential folding in vivo, concurrent with transcription itself. We base our analysis on a parallel Monte Carlo simulation, assigning an individual processor to each competing folding pathway. In this way, we show that the group I mitochondrial intron requires an in vivo environment to aid the folding into a structure presenting the catalytically-active helix P7 and the splicing substrate upon which the catalytic core exerts its function. It is inferred that a base pair disruption caused by interaction with a ribosome is required to bias the folding pathway so that helix P7 is formed. Furthermore, it is shown that a disruption in the early stages of transcription, followed by the perturbation described is essential to shape the 3' splicing site. Our study suggests that pre-mRNAs of group I might achieve catalytic potential in a fundamentally different way from ribosomal RNAs of the same group, where thermodynamics appear to control the formation of the catalytic structural motif. The role of the trans-acting factor, on the other hand, cannot be recovered in vitro by recombination with the transcript.

Exploiting parallelism on a fine-grained MIMD architecture based upon channel queues

We present techniques for exploiting fine-grained parallelism extracted from sequential programs on a fine-grained MIMD system. The system exploits fine-grained parallelism through parallel execution of instructions on multiple processors as well as pipelined nature of individual processors. The processors can communicate data values via globally shared registers as well as dedicated channel queues. Compilation techniques are presented to utilize these mechanisms. A scheduling algorithm has been developed to distribute operations among the processors in a manner that reduces communication among the processors. The compiler identifies data dependencies which require synchronization and enforces them using channel queues. Delays that may result by attempting write operations to a full channel queue are avoided by spilling values from channels to local registers. If an interprocessor data dependency does not require synchronization, then the data value is passed through a shared register or shared memory.

SPIRE

The SPIRE-Architecture is designed for high speed uniprocessor execution of sequential problems as well for parallel multiprocessor systems where individual processors perform operations on complex data structures. The idea of SPIRE-Architecture lies in a new instruction format expanded by a dedicated execution clause and compilative instruction execution. To mirror this concept in practice some new components has been added to the hardware.The simulation results show that SPIRE can processes data up to 30 times faster than classic von Neumann processor and profits directly from complexity of used data structures. Programm code is about twice as compact as that of RISC processors. Furthermore, a proposed hardware concept and consistent rules of instruction coding allows compilers to generate very efficient code.

On the assignment problem of arbitrary process systems to heterogeneous distributed computer systems

The authors propose and evaluate an efficient hierarchical clustering and allocation algorithm that drastically reduces the interprocess communications cost while observing lower and upper bounds of utilization for the individual processors. They compare the algorithm with branch-and-bound-type algorithms that can produce allocations with minimal communication cost, and show a very encouraging time complexity/suboptimality tradeoff in favor of the algorithm, at least for a class of process clusters and their random combinations which it is believed occur naturally in distributed applications. The heuristic allocation is well suited for a changing environment, where processors may fail or be added to the system and where the workload patterns may change unpredictably and/or periodically.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>