Distributed Shared Memory Architecture Research Articles

Communication Complexity of the Fast Multipole Method and its Algebraic Variants

A combination of hierarchical tree-like data structures and data access patterns from fast multipole methods and hierarchical low-rank approximation of linear operators from H-matrix methods appears to form an algorithmic path forward for efficient implementation of many linear algebraic operations of scientific computing at the exascale. The combination provides asymptotically optimal computational and communication complexity and applicability to large classes of operators that commonly arise in scientific computing applications. A convergence of the mathematical theories of the fast multipole and H-matrix methods has been underway for over a decade. We recap this mathematical unification and describe implementation aspects of a hybrid of these two compelling hierarchical algorithms on hierarchical distributed-shared memory architectures, which are likely to be the first to reach the exascale. We present a new communication complexity estimate for fast multipole methods on such architectures. We also show how the data structures and access patterns of H-matrices for low-rank operators map onto those of fast multipole, leading to an algebraically generalized form of fast multipole that compromises none of its architecturally ideal properties.

Predicting the performance measures of an optical distributed shared memory multiprocessor by using support vector regression

Recent advances in the development of optical technologies suggest the possible emergence of optical interconnects within distributed shared memory (DSM) multiprocessors. The performance of these DSM architectures must be evaluated under varying values of DSM parameters. In this paper, we develop a Support Vector Regression (SVR) model for predicting the performance measures (i.e. average network latency, average channel waiting time and average processor utilization) of a DSM multiprocessor architecture interconnected by the Simultaneous Optical Multiprocessor Exchange Bus (SOME-Bus), which is a high-bandwidth, fiber-optic interconnection network. The basic idea is to collect a small number of data points by using a statistical simulation and predict the performance measures of the system for a large set of input parameters based on these. OPNET Modeler is used to simulate the DSM-based SOME-Bus multiprocessor architecture and to create the training and testing datasets. The prediction error and correlation coefficient of the SVR model is compared to that of Multiple Linear Regression (MLR) and feedforward Artificial Neural Network (ANN) models. Results show that the SVR-RBF model has the lowest prediction error and is more robust. It is concluded that SVR model shortens the time quite a bit for obtaining the performance measures of a DSM multiprocessor and can be used as an effective tool for this purpose.

Truss: A Reliable, Scalable Server Architecture

Traditional techniques that mainframes use to increase reliability -special hardware or custom software - are incompatible with commodity server requirements. The Total Reliability Using Scalable Servers (TRUSS) architecture, developed at Carnegie Mellon, aims to bring reliability to commodity servers. TRUSS features a distributed shared-memory (DSM) multiprocessor that incorporates computation and memory storage redundancy to detect and recover from any single point of transient or permanent failure. Because its underlying DSM architecture presents the familiar shared-memory programming model, TRUSS requires no changes to existing applications and only minor modifications to the operating system to support error recovery.

Parallel Logic Programming Systems on Scalable Architectures

Parallel logic programming (PLP) systems are sophisticated examples of symbolic computing systems. PLP systems address problems such as allocating dynamic memory, scheduling irregular computations, and managing different types of implicit parallelism. Most PLP systems have been developed for bus-based architectures. However, the complexity of PLP systems and the large amount of data they process raise the question of whether logic programming systems can still achieve good performance on modern scalable architectures, such as distributed shared-memory (DSM) systems. In this work we use execution-driven simulation of a cache-coherent DSM architecture to investigate the performance of Andorra-I, a state-of-the-art PLP system, on a modern multiprocessor. The results of this simulation show that Andorra-I exhibits reasonable running time performance, but it does not scale well. Our detailed analysis of cache misses and their sources expose several opportunities for improvements in Andorra-I. Based on this analysis, we modify Andorra-I using a set of simple techniques that led to significantly better running time and scalability. These results suggest that Andorra-I can and should perform well on modern multiprocessors. Furthermore, as Andorra-I shares its main data structures with several PLP systems, we conclude that the methodology and techniques used in our work can greatly benefit these other PLP systems.

An efficient and scalable approach for implementing fault-tolerant DSM architectures

Distributed Shared Memory (DSM) architectures are attractive to execute high performance parallel applications. Made up of a large number of components, these architectures have however a high probability of failure. We propose a protocol to tolerate node failures in cache-based DSM architectures. The proposed solution is based on backward error recovery and consists of an extension to the existing coherence protocol to manage data used by processors for the computation and recovery data used for fault tolerance. This approach can be applied to both Cache Only Memory Architectures (COMA) and Shared Virtual Memory (SVM) systems. The implementation of the protocol in a COMA architecture has been evaluated by simulation. The protocol has also been implemented in an SVM system on a network of workstations. Both simulation results and measurements show that our solution is efficient and scalable.

The MIT Alewife Machine

A variety of models for parallel architectures, such as shared memory, message passing, and data flow, have converged in the recent past to a hybrid architecture form called distributed shared memory (DSM). Alewife, an early prototype of such DSM architectures, uses hybrid software and hardware mechanisms to support coherent shared memory, efficient user level messaging, fine grain synchronization, and latency tolerance. Alewife supports up to 512 processing nodes connected over a scalable and cost effective mesh network at a constant cost per node. Four mechanisms combine to achieve Alewife's goals of scalability and programmability: software extended coherent shared memory provides a global, linear address space; integrated message passing allows compiler and operating system designers to provide efficient communication and synchronization; support for fine grain computation allows many processors to cooperate on small problem sizes; and latency tolerance mechanisms-including block multithreading and prefetching-mask unavoidable delays due to communication. Extensive results from microbenchmarks, together with over a dozen complete applications running on a 32-node prototype, demonstrate that integrating message passing with shared memory enables a cost efficient solution to the cache coherence problem and provides a rich set of programming primitives. Our results further show that messaging and shared memory operations are both important because each helps the programmer to achieve the best performance for various machine configurations.

Prospects of distributed shared memory for reducing global traffic in shared-bus multiprocessors

As an effort, not a mutually exclusive but rather complementary to developing better backplane bus, this paper considers adapting distributed shared-memory (DSM) architectures to improve traditional shared-bus designs. We consider two well-known DSM architectures, namely Cache-coherent Non-Uniform Memory Architecture (NUMA) and Cache-Only Memory Architecture (COMA), in reducing bus traffic. Our study shows that COMA provides excellent opportunity of significantly reducing bandwidth requirement for the bus while cache-coherent NUMA provides a marginal improvement.