Multicomputer Architecture Research Articles

PROPERTIES OF SIMULATED ANNEALING AND GENETIC ALGORITHMS FOR MAPPING DATA TO MULTICOMPUTERS

We experimentally analyze some properties of simulated annealing algorithms (SA) and genetic algorithms (GA) for mapping data to multicomputers. These properties include sensitiviiy to user parameters, fault tolerance capability, and applicability to different multicomputer topologies. Some user parameters are included in the objective function and are architecture- or problem-dependent parameters. The others are used in the GA and SA algorithms. The fault tolerance capability is demonstrated by mapping data to a multicomputer with some faulty processors. We assume a hypercube multicomputer architecture in most experiments. However, mapping to mesh, array, ring, tree, and star graph topologies is also demonstrated. The experimental results show that the GA and SA are insensitive to user parameters in wide ranges, completely fault tolerant, and unbiased towards particular multicomputer topologies. These properties of flexibility and general applicability, which are lacking in other heuristic algorithms, make the GA and SA attractive for automatic parallelization systems.

A theory of wormhole routing in parallel computers

Virtually all theoretical work on message routing in parallel computers has dwelt on packet routing: messages are conveyed as packets, an entire packet can reside at a node of the network, and a packet is sent from the queue of one node to the queue of another node until its reaches its destination. A trend in multicomputer architecture, however, is to use wormhole routing. In wormhole routing a message is transmitted as a contiguous stream of bits, physically occupying a sequence of nodes/edges in the network. Thus, a message resembles a worm burrowing through the network. In this paper we give theoretical analyses of simple wormhole routing algorithms, showing them to be nearly optimal for butterfly and mesh connected networks. Our analysis requires initial random delays in injecting messages to the network. We report simulation results suggesting that the idea of random initial delays may have an impact beyond theoretical analysis.

Extending PVM to a massively parallel architecture

Heterogeneous computing is one of the key issues for reaching high performances with limited costs. In this area, PVM is a wide-spread programming environment supported by a collection of architectures, ranging from Unix workstations to various kinds of parallel machines. We have enriched this set of architectures with the Meiko Computing Surface, a multicomputer architecture based on the transputer technology. The paper describes how the porting has been implemented. The resulting environment allows a user to distribute any application partly on ‘traditional’ PVM nodes, and partly on transputer ones. The advantages deriving from the inclusion of the Meiko Computing Surface in the PVM environment are particularly evident with fine-grained parallel applications: as an example, we show the experimental results obtained for a computational vision application.

The specification of a generic multicomputer using Lotos

Formal Methods of specification can play an important role in exploring the behaviour of complex systems, as distributed systems or communication protocols. Lotos, based on algebra of processes, has been chosen as an international standard for specifying many systems. We show in this paper the model for a popular class of parallel machines: multicomputers. This paper establishes a guide to employ the Lotos technique in specifying problems on multicomputer architecture. We focus our attention in a new feature of these machines: the dynamic reconfiguration of the network.

A multicomputer architecture with a segmented shared bus

Single-bus multicomputer systems can support only a relatively small number of processing elements as the bus quickly becomes the communication bottleneck. Multiple-bus systems provide increased communication bandwidth and can support larger numbers of processing elements. The segmented-bus architecture is an important extension of these systems and offers even more communication bandwidth. The segmented bus contains switches between every pair of adjacent processing elements and can be dynamically segmented into smaller buses that can simultaneously connect more than one pair of processing elements. This paper examines the design of the processing elements, the mathematical modeling and the simulation of the architecture. It relates the performance of the system to the number of processing elements and the bus segmentation and allocation procedure, given a specific load.

An automatic parallelization framework for multicomputers

Several researchers have looked into various issues related to automatic parallelization of sequential programs for multicomputers. But there is a need for a coherent framework which encompasses all these issues. In this paper we present a such a framework which takes best advantage of the multicomputer architecture. We resort to tiling transformation for iteration space partitioning and propose a scheme of automatic data partitioning and dynamic data distribution. We have tried a simple implementation of our scheme on a transputer based multicomputer [1] and the results are encouraging.

Design and evaluation of hardware strategies for reconfiguring hypercubes and meshes under faults

This paper discusses the design of two reconfiguration strategies for distributed memory multicomputer architectures under failures. The specific architectures to which we apply the techniques are hypercubes and meshes. The first scheme uses spare processors attached to certain processors in the hypercube or mash using a novel embedding technique. The second approach places spare processors along specific links in the hypercube or mesh. Both schemes involve the mapping of logical links of a virtual machine onto a set of physical links in the final reconfigured machine and hence suffer some performance degradation. We characterize the performance degradation through trace-driven simulation of real applications running on the faulty and reconfigured system. We find that the schemes have high reliability, suffer little degradation in performance, and are very low in cost. >

SYNC_WAVE: a high accuracy and low overhead algorithm for clock, synchronization in transputer networks

This paper addresses the implementation of global time in transputer systems by software synchronization of local processor timers. After an introduction on the synchronization problem, SYNC_WAVE, a synchronization algorithm for multicomputer architectures, is briefly described. The rest of the paper gives details of an existing implementation of SYNC_WAVE for transputer-based systems. Most of the devised code is presented and commented on, showing the methods used to improve synchronization accuracy and to read the global clock from within a user process. The performance results obtained are shown, and the usefulness and portability of the proposed code are discussed.

A comparison of message passing and shared memory architectures for data parallel programs

Shared memory and message passing are two opposing communication models for parallel multicomputer architectures. Comparing such architectures has been difficult, because applications must be hand-crafted for each architecture, often resulting in radically different sources for comparison. While it is clear that shared memory machines are currently easier to program, in the future, programs will be written in high-level languages and compiled to the specific parallel target, thus eliminating this difference.In this paper, we evaluate several parallel architecture alternatives --- message passing, NUMA, and cachecoherent shared memory --- for a collection of scientific benchmarks written in C*, a data-parallel language. Using a single suite of C* source programs, we compile each benchmark and simulate the interconnect for the alternative models. Our objective is to examine underlying, technology-independent costs inherent in each alternative. Our results show the relative work required to execute these data parallel programs on the different architectures, and point out where some models have inherent advantages for particular data-parallel program styles.

Extending Unix for scalable computing

Because it retrieves all instructions and data from a single memory, the von Neumann computer architecture has a fundamental speed limit. The scalable multicomputer architecture, which uses many microprocessors together to solve a single problem and can run at teraflop speeds, may be a solution. While teraflop processor technology is known, the scalable operating and I/O system technology necessary for those speeds are not known. The authors describe how Unix can be extended to scalable computing, permitting teraflop speeds and offering parallel computing to users unfamiliar with parallel programming. They designed this technology into the system software of the Ncube-2, the predecessor to Ncube's announced teraflop parallel computer. The authors describe the system in detail and provide some performance results.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>

Architectural requirements of parallel scientific applications with explicit communication

This paper studies the behavior of scientific applications running on distributed memory parallel computers. Our goal is to quantify the floating point, memory, I/O and communication requirements of highly parallel scientific applications that perform explicit communication. In addition to quantifying these requirements for fixed problem sizes and numbers of processors, we develop analytical models for the effects of changing the problem size and the degree of parallelism for several of the applications. We use the results to evaluate the trade-offs in the design of multicomputer architectures.

Analysis of hierarchial bus-based multicomputer architectures

Hierarchical interconnection networks with cluster structure have been proposed for use in multicomputer systems for parallel and distributed processing. However, little attention has been paid to the study of the underline communication network at the message passing level. The main contribution of this paper is to present a performance model which relates the performance criteria with the architecture design and implementation, and to derive a method for optimizing the hierarchy in such systems. The proposed methodology can be easily augmented to include additional objectives and/or decision variables.

Wafer-scale optimization using computational availability

It is shown that, given the ability to restructure wafer-level designs, there are different ways to employ redundancy. Redundancy is evaluated by estimating system computational availability over a mission lifetime. This technique is illustrated using two wafer-scale integration (WSI) case studies. The first is a very-fine-grained programmable systolic data processor (PSDP) that contains 4- and 8-b paths, RAM, and control optimized for signal and data processing applications. The second, the Mosaic multicomputer architecture, is a less fine-grained homogeneous architecture in which each node contains a 16-b microprocessor and associated RAM and ROM. Potential benefits of implementing these parallel processing architectures in wafer scale are discussed. >

Theoretical Performance-Based Cost-Effectiveness of Multicomputers

The CPU cycles that are stolen to relay messages can significantly affect the performance of a multicomputer system. This degradation in performance in turn affects the overall cost-effectiveness of such a system. This paper compares the cost-effectiveness of four multicomputer architectures that have received a great deal of recent attention: the bidirectional straight line, the unidirectional ring, the square mesh, and the binary hypercube. Cost-effectiveness is measured by finding the ratio of the total dollar cost of a multicomputer, based upon the total cost of processors and communication links, to the total potential utilization of the processors making up the system, where the potential utilization is the fraction of time a processor has to do useful processing after subtracting time spent handling messages. The smaller the value of this ratio, the greater the cost-effectiveness of the system.

A parallel logic system on a multicomputer architecture

This paper describes the implementation of a logic programming language on a massively parallel architecture. This implementation is based on the AND/OR Process Model which allows the exploitation of both AND and OR parallelism in logic programs. A distributed memory model is used, and a decentralized control mechanism has been designed. The multicomputer, which the system has been implemented on, consists of a network of Inmos Transputers. The AND/OR processes are implemented as Occam processes mapped onto the Transputer nodes. After the presentation of the system architecture and a deep discussion of the distributed memory management, some preliminary performance results are discussed.

Paradigm: a highly scalable shared-memory multicomputer architecture

Paradigm (parallel distributed global memory), a shared-memory multicomputer architecture that is being developed to show that one can build a large-scale machine using high-performance microprocessors, is discussed. The Paradigm architecture allows a parallel application program to execute any of its tasks on any processor in the machine, with all the tasks in a single address space. The focus is on novel design techniques that support scalability. The key performance issues are identified, and some results to date from this work and experience with the VMP architecture design on which it is based are summarized.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>

Evaluation and design of an ultra-reliable distributed architecture for fault tolerance

The issues related to the experimental evaluation of an early conceptual prototype of the MAFT (multicomputer architecture for fault tolerance) architecture are discussed. A completely automated testing approach was designed to allow fault-injection experiments to be performed, including stuck-at and memory faults. Over 2000 injection tests were run and the system successfully tolerated all faults. Concurrent with the experimental evaluation, an analytic evaluation was carried out to determine if higher levels of reliability could be achieved. The lessons learned in the evaluation phase culminated in a new design of the MAFT architecture for applications needing ultrareliability. The design uses the concept of redundantly self-checking functions to address the rigid requirements proposed for a future generation of mission-critical avionics. The testing of three subsystems critical to the operation of the new MAFT is presented with close to 50-k test cycles performed over 51 different IC devices to verify the designs.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>

IWarp multicomputer with an embedded switching network

This brief describes the iWarp multicomputer architecture. The two components of each iWarp cell, a 20 MFLOPS and 20 MIPS computation agent and a 320 Mbyte s −1 communication agent for interfacing with other cells, can operate independently, but their location on a single chip allows for tight coupling. A variety of communication methods are supported by iWarp.

Knowledge-based environment for investigating multicomputer architectures

Multicomputers for massively parallel processing will eventually employ billions of processing elements, each of which will be capable of communicating with every other processing element. A knowledge-based modelling and simulation environment (KBMSE) for investigating such multicomputer architecture at a discrete-event system level is described. The KBMSE implements the discrete-event system specification (DEVS) formalism in an object-oriented programming system of Scheme (a lisp dialect), which supports building models in a hierarchical, modular manner, a systems-oriented approach not possible in conventional simulation languages. The paper presents a framework for knowledge-based modelling and simulation by exemplifying modelling a hypercube multicomputer architecture in the KBMSE. The KBMSE has been tested on a variety of domains characterized by complex, hierarchical structures such as advanced multicomputer architectures, local area computer networks, intelligent multi-robot organizations, and biologically based life-support systems.

Macropipelined multicomputer architecture for image analysis

We present a scheme for macropipelining an L-stage image-analysis algorithm into a message-passing Ns-processor multicomputer system in which Ns &gt; L. The resulting architectures achieve high speeds in processing multiple images. Most image-processing applications consist of a sequence of tasks, e.g., preprocessing, detection, segmentation, feature extraction, and classification. This sequence lends itself to an assignment of the tasks to a series-connected set of processors, or pipelining of the tasks. We refer to this form of pipelining as macropipelining. To minimize the effects of throughput-limiting tasks, or bottlenecks, in this pipeline, we introduce a performance model that accounts for both the computation aspects and the communication aspects of parallel processing. With the help of this model, we assign the appropriate number of processors to each task so as to balance the workloads. We then generate a problem graph describing the relationships among the tasks. We use an estimator of the frame-time of the image-processing system as an objective function for choosing a mapping of the problem processing graph into a system graph. This estimator takes account of computation times and communication intensities the tasks in the problem graph, and it accounts for link contentions. To find an efficient mapping, we use a among heuristic optimization in which possible bottlenecks are given high priority in the mapping procedure. We tested our macropipelining scheme on a target-recognition algorithm in a simulated hypercube computer system. The results support our belief that this scheme yields effective architectures for high-speed processing of long sequences of images.