Communication In Parallel Computers Research Articles

Modelling of Communication Complexity in Computers

We have been oft used to ignore the influence of communication in sequential computers or in better cases we have been supposed that this influence should be included in computation complexity representing performed I/O instructions. Multiple using of high performed CPU (central processor unit) within a single computer (SMP parallel computer) but also multiple using of high performed computers (workstations) as the building computing node of dominant parallel computers nowadays (network of workstations - NOW) or integrated network of NOW modules - Grid).The extended communication steps in parallel computers represent inter process communication (IPC) of decomposed parts of given parallel algorithm (PA) named as parallel processes. Then we can simply say that any PA consists of parallel processes with their sequential character and IPC among parallel processes. In relation to increasing role of communications in parallel computers this paper is devoted to modelling of communication complexity (number of communication steps) in the same way as we use to analyse computation complexity so in sequential algorithms. Based on this extended approach we are able better to optimise performance of all existed parallel computers. Another important problem is to analyse critical parts of communication influence to parallel computer performance and that at first in minimisation of communications at developing stages of any PA and secondly at its whole performance optimisation. Finally we have illustrated from the point of user the important role of communication in parallel computers on the chosen examples.

Adaptive and deadlock-free routing for irregular faulty patterns in mesh multicomputer

Message routing achieves the internode communication in parallel computers. A reliable routing is supposed to be deadlock-free and fault-tolerant. While many routing algorithms are able to tolerate a large number of faults enclosed by rectangular faulty blocks, there is no existing algorithm that is capable of handling irregular faulty patterns for wormhole networks. In this paper, a two-staged adaptive and deadlock-free routing algorithm called "Routing for Irregular Faulty Patterns" (RIFP) is proposed. It can tolerate irregular faulty patterns by transmitting messages from sources or to destinations within faulty blocks via multiple "intermediate nodes." A method employed by RIFP is first introduced to generate intermediate nodes using the local failure information. By its aid, two communicating nodes can always exchange their data or intermediate results if there is at least one path between them. RIFP needs two virtual channels per physical link in meshes.

Integrating performance monitoring and communication in parallel computers

A large and increasing gap exists between processor and memory speeds in scalable cache-coherent multiprocessors. To cope with this situation, programmers and compiler writers must increasingly be aware of the memory hierarchy as they implement software. Tools to support memory performance tuning have, however, been hobbled by the fact that it is difficult to observe the caching behavior of a running program. Little hardware support exists specifically for observing caching behavior; furthermore, what support does exist is often difficult to use for making fine-grained observations about program memory behavior.Our work observes that in a multiprocessor, the actions required for memory performance monitoring are similar to those required for enforcing cache coherence. In fact, we argue that on several machines, the coherence/communication system itself can be used as machine support for performance monitoring. We have demonstrated this idea by implementing the FlashPoint memory performance monitoring tool. FlashPoint is implemented as a special performance-monitoring coherence protocol for the Stanford FLASH Multiprocessor. By embedding performance monitoring into a cache-coherence scheme based on a programmable controller, we can gather detailed, per-data-structure, memory statistics with less than a 10% slowdown compared to unmonitored program executions. We present results on the accuracy of the data collected, and on how FlashPoint performance scales with the number of processors.

Optimizing memory system performance for communication in parallel computers

Communication in a parallel system frequently involves moving data from the memory of one node to the memory of another; this is the standard communication model employed in message passing systems. Depending on the application, we observe a variety of patterns as part of communication steps, e.g., regular (i.e. blocks of data), strided, or irregular (indexed) memory accesses. The effective speed of these communication steps is determined by the network bandwidth and the memory bandwidth, and measurements on current parallel supercomputers indicate that the performance is limited by the memory bandwidth rather than the network bandwidth. Current systems provide a wealth of options to perform communication, and a compiler or user is faced with the difficulty of finding the communication operations that best use the available memory and network bandwidth. This paper provides a framework to evaluate different solutions for inter-node communication and presents the copy-transfer model; this model captures the contributions of the memory system to inter-node communication. We demonstrate the usefulness of this simple model by applying it to two commercial parallel systems, the Cray T3D and the Intel Paragon.In particular we identify two methods to transfer data between nodes in these two machines. In buffer-packing transfers, a contiguous block of data is transferred across the network. If the data are not stored contiguously, they are copied to (gathering) or from (scattering) buffers in local memory before and after the transfer. Chained transfers perform gathering, transfer and scattering in one step, reading the data elements with some non-sequential pattern and immediately transferring them on to the destination.Our model and measurements indicate that chaining of the gather, transfer, and scatter operations results in better performance than buffer packing for many important access patterns. Most standard message passing libraries (like MPI, PVM or NX) force the parallelizing compiler (or the programmer) to employ the buffer-packing communication operations. However, the addition of hardware support dedicated to communication (e.g., DMAs, line-transfer units) now gives the compiler a wider range of options.

Free-space optical interconnection technology in parallel processing systems

A critical element in building parallel computers that use very fast processing elements is interprocessor communication. To utilize the processing elements efficiently, the interconnects in such parallel computers must provide high data rates and low latencies with extremely high reliability, compactness, and reasonable cost. We discuss the application of free-space optical links for interprocessor communication in parallel computers. Free-space transmitter and receiver modules have been fabricated and installed in a card cage enclosure. Repeatable signal levels have been achieved, after board removal and replacement, without the use of any micropositioners or adjustments. A time-division-multiplexed (TDM) ring network using high-speed (1 Gbit/s), point-to-point, free-space links has been designed to provide full connectivity among a set of processors connected to the ring nodes. Work is under way to demonstrate the 1DM ring network concept in a multiprocessor computer.