Cache-Only Memory Architecture Research Articles

Implementation of cache coherence protocol for COMA multiprocessor systems based on the scalable coherent interface

The Scalable Coherent Interface (SCI) provides high bandwidth, low latency communication in systems with point-to-point links. SCI can also support cache coherence in shared memory multiprocessors. Existing Cache-Only Memory Architecture (COMA) systems are all based on communication and cache coherence protocols other than SCI. Hierarchical COMA systems generally suffer from high message latencies. In this paper, we examine the implementation of COMA multiprocessors using SCI protocol. Implementing the SCI communication protocol in hierarchical COMA systems reduces latencies and its cache coherence protocol will reduce directory sizes. In this work, a hierarchical COMA system was modified so that it fits the SCI. In addition, a new replacement policy is proposed in which replaced lines swap storage locations with new ones fetched into the attraction memory. Similar to other COMA systems, this new system still gives high performance when part of the attraction memory is left unallocated. However, the performance of this system is slightly influenced by the initial distribution of data across the system.

Modeling and evaluating the time overhead induced by BER in COMA multiprocessors

Designing multiprocessors based on distributed shared memory (DSM) architecture considerably increases their scalability. But as the number of nodes in a multiprocessor increases, the probability of encountering failures in one or more nodes of the system raises as a serious problem. Thus, every large-scale multiprocessor should be equipped with mechanisms that tolerate node failures. Backward error recovery (BER) is one of the most feasible strategies to build fault tolerant multiprocessors and it can be shown that among various DSM-based architectures, cache only memory architecture (COMA) is the most suitable for implementing BER. The main reason is the existence of built-in mechanisms for data replication in COMA memory system. BER is applicable to COMA multiprocessors with minor hardware redundancy, but it will obviously cause some other kinds of overheads. The most important overhead induced by BER is the time required to produce and store recovery data. This paper introduces an analytical model for predicting the amount of this time overhead and then verifies the correctness of the model through comparing the results predicted from this model with the previously published simulation results. Both the analytical model and simulation results show that the overhead is nearly independent of the number of nodes. The immediate result is that BER is a cost-effective strategy for tolerating node failures in large-scale COMA multiprocessors with large numbers of nodes.

SDAARC: an extended cache-only memory architecture

An established data distribution paradigm - extended to include automatic distribution of instruction sequences - might help overcome the traditional difficulties associated with parallel programming. The self-distributing associative architecture (SDAARC) that we describe is based on the cache-only memory architecture concept, but extends the data migration mechanisms with migrating microthreads and a scheduling concept. We present the architecture in a top-down fashion, consider it first as a complete system consisting of compiler and runtime technology, then examine the compiler technology and the runtime system in greater detail.

The Test of Time

We invited Karl E. Grosspietsch from the Fraunhofer Institute of Autonomous Intelligent Systems to organize a special issue on unorthodox computer architectures. This issue presents articles on somewhat unfamiliar topics includingartificial neural networks for ultra-large-scale integration,a parallel neural processor for real-time processing,an intelligent memory architecture for artificial neural networks,a symbolic-learning method based on multiple-valued logic using recon-figurable hardware, anda parallel-processor architecture based on the extension of the cache-only memory architecture to the instruction sequence.

An efficient and scalable approach for implementing fault-tolerant DSM architectures

Distributed Shared Memory (DSM) architectures are attractive to execute high performance parallel applications. Made up of a large number of components, these architectures have however a high probability of failure. We propose a protocol to tolerate node failures in cache-based DSM architectures. The proposed solution is based on backward error recovery and consists of an extension to the existing coherence protocol to manage data used by processors for the computation and recovery data used for fault tolerance. This approach can be applied to both Cache Only Memory Architectures (COMA) and Shared Virtual Memory (SVM) systems. The implementation of the protocol in a COMA architecture has been evaluated by simulation. The protocol has also been implemented in an SVM system on a network of workstations. Both simulation results and measurements show that our solution is efficient and scalable.

Cache-only memory architectures

The shared memory concept makes it easier to write parallel programs, but tuning the application to reduce the impact of frequent long latency memory accesses still requires substantial programmer effort. Researchers have proposed using compilers, operating systems, or architectures to improve performance by allocating data close to the processors that use it. The Cache-Only Memory Architecture (COMA) increases the chances of data being available locally because the hardware transparently replicates the data and migrates it to the memory module of the node that is currently accessing it. Each memory module acts as a huge cache memory in which each block has a tag with the address and the state. The authors explain the functionality, architecture, performance, and complexity of COMA systems. They also outline different COMA designs, compare COMA to traditional nonuniform memory access (NUMA) systems, and describe proposed improvements in NUMA systems that target the same performance obstacles as COMA.

Coherence and Replacement Protocol of DICE—A Bus-Based COMA Multiprocessor

As microprocessors become faster and demand more bandwidth, the already limited scalability of a shared bus decreases even further. DICE, a shared-bus multiprocessor, utilizes cache only memory architecture (COMA) to effectively decrease the speed gap between modern high-performance microprocessors and the bus. DICE tries to optimize COMA for a shared-bus medium, in particular to reduce the detrimental effects of cache coherence and the “last memory block” problem on replacement. In this paper, we present the coherence and replacement protocol of the DICE multiprocessor and its design trade-offs. We describe a four-state write-invalidate coherence protocol in detail. Replacement, which poses a unique overhead problem of COMA, requires that a victim block with ownership be relocated to a remote node in order not to discard the last cached memory block. We show that the relocation process can be efficiently implemented by using a temporary storage called relocation buffer and a priority-based selection algorithm. We present performance results that show a drastic reduction in global bus traffic compared to a traditional shared-bus multiprocessor architecture.

Design of a bus-based shared-memory multiprocessor DICE

DICE is a shared-bus multiprocessor based on a distributed shared-memory architecture, known as cache-only memory architecture (COMA). Unlike previous COMA proposals for large-scale multiprocessing, DICE utilizes COMA to effectively decrease the speed gap between modem high-performance microprocessors and the bus. DICE tries to optimize COMA for a shared-bus medium, in particular to reduce detrimental effects of the cache coherence and the `last memory block' problem on replacement. In this paper, we present a global bus design of DICE based on the IEEE futurebus+ backplane bus and the Texas Instruments chip-set. Our design demonstrates that necessary bus transactions for DICE can be done efficiently with existing standard bus signals. Considering the benefits of COMA and the moderate design complexity it adds to the conventional shared-bus multiprocessor design, a bus-based COMA multiprocessor, such as DICE, can become a viable candidate for future shared-bus multiprocessor designs.

Prospects of distributed shared memory for reducing global traffic in shared-bus multiprocessors

As an effort, not a mutually exclusive but rather complementary to developing better backplane bus, this paper considers adapting distributed shared-memory (DSM) architectures to improve traditional shared-bus designs. We consider two well-known DSM architectures, namely Cache-coherent Non-Uniform Memory Architecture (NUMA) and Cache-Only Memory Architecture (COMA), in reducing bus traffic. Our study shows that COMA provides excellent opportunity of significantly reducing bandwidth requirement for the bus while cache-coherent NUMA provides a marginal improvement.

Options for dynamic address translation in COMAs

In modern processors, the dynamic translation of virtual addresses to support virtual memory is done before or in parallel with the first-level cache access. As processor technology improves at a rapid pace and the working sets of new applications grow insatiably the latency and bandwidth demands on the TLB (Translation Lookaside Buffer) are getting more and more difficult to meet. The situation is worse in multiprocessor systems, which run larger applications and are plagued by the TLB consistency problem.We evaluate and compare five options for virtual address translation in the context of COMAs (Cache Only Memory Architectures). The dynamic address translation mechanism can be located after the cache access provided the cache is virtual. In a particular design, which we call V-COMA for Virtual COMA, the physical address concept and the traditional TLB are eliminated. While still supporting virtual memory, V-COMA reduces the address translation overhead to a minimum.V-COMA scales well and works better in systems with large number of processors. As a machine running on virtual addresses, V-COMA provides a simple and consistent hardware model to the operating system and the compiler, in which further optimization opportunities are possible.

Reducing the Read-miss Penalty for Flat COMA Protocols

In flat cache-only memory architectures (COMA), an attraction-memory miss must first interrogate a directory before a copy of the requested data can be located, which often involves three network traversals. By keeping track of the identity of a potential holder of the copy-called a hint-one network traversal can be saved which reduces the read penalty. We have evaluated the reduction of the read-miss penalty provided by hints using detailed architectural simulations and four benchmark applications. The results show that a previously proposed protocol using hints can actually make the read-miss penalty larger because when the hint is not correct, an extra network traversal is needed. This has motivated us to study a new protocol, using hints, that simultaneously sends a request to the potential holder and to the directory. This protocol reduces the read-miss penalty for all applications but the protocol complexity does not seem to justify the performance improvement.

COMA

Due to the increasing number of their components, Scalable Shared Memory Multiprocessors (SSMMs) have a very high probability of experiencing failures. Tolerating node failures therefore becomes very important for these architectures particularly if they must be used for long-running computations. In this paper, we show that the class of Cache Only Memory Architectures (COMA) are good candidates for building fault-tolerant SSMMs. A backward error recovery strategy can be implemented without significant hardware modification to previously proposed COMA by exploiting their standard replication mechanisms and extending the coherence protocol to transparently manage recovery data. Evaluation of the proposed fault-tolerant COMA is based on execution driven simulations using some of the Splash applications. We show that, for the simulated architecture, the performance degradation caused by fault-tolerance mechanisms varies from 5% in the best case to 35% in the worst case. The standard memory behavior is only slightly perturbed. Moreover, results also show that the proposed scheme preserves the architecture scalability and that the memory overhead remains low for parallel applications using mostly shared data.

An argument for simple COMA

We present design details and some initial performance results of a novel scalable shared memory multiprocessor architecture. This architecture features the automatic data migration and replication capabilities of cache-only memory architecture (COMA) machines, without the accompanying hardware complexity. A software layer manages cache space allocation at a page-granularity — similarly to distributed virtual shared memory (DVSM) systems —leaving simpler hardware to maintain shared memory coherence at a cache line granularity. By reducing the hardware complexity, the machine cost and development time are reduced. We call the resulting hybrid hardware and software multiprocessor architecture Simple COMA. Preliminary results indicate that the performance of Simple COMA is comparable to that of more complex contemporary all-hardware designs.

Evaluating the memory overhead required for COMA architectures

Cache only memory architectures (COMA) have an inherent memory overhead due to the organization of main memory as a large cache called an attraction memory. This overhead consists of memory left unallocated for performance reasons as well as additional physical memory required due to the cache organization of memory. In this work, we examine the effect of data reshuffling and data replication on the memory overhead. Data reshuffling occurs when space needs to be allocated to store a remote memory line in the local memory. Data that is reshuffled is sent between memories via replacement messages. A simple mathematical model predicts the frequency of data reshuffling as a function of the attraction memory parameters. Simulation data shows that the frequency of data reshuffling is sensitive to the allocation policy and associativity of the memory but is relatively unaffected by the block size chosen. The simulation data also shows that data replication in the attraction memory is important for good performance, but most gains can be achieved through replication in the processor caches.

The KSR1

Kendall Square Research introduced the KSR1 system in 1991. The architecture is based on a ring of rings of 64-bit microprocessora. It is a distributed, shared memory system and is scalable. The memory structure is unique and is the key to understanding the system. Different levels of caching eliminates physical memory addressing and leads to the ALLCACHE™ scheme. Since requested data may be found in any of several caches, the initial access time is variable. Once pulled into the local (sub) cache, subsequent access times are fixed and minimal. Thus, the KSR1 is a Cache-Only Memory Architecture (COMA) system.This paper describes experimentation and an analytic model of the KSR1. The focus is on the poststore programmer option. With the poststore option, the programm er can elect to broadcast the updated value of a variable to all processors that might have a copy. This may save time for threads on other processors, but delays the broadcasting thread and places additional traffic on the ring. The specific issue addressed is to determine under what conditions poststore is beneficial. The analytic model and the experimental observations are in good agreement. They indicate that the decision to use poststore depends both on the application and the current system load.

DDM-a cache-only memory architecture

The Data Diffusion Machine (DDM), a cache-only memory architecture (COMA) that relies on a hierarchical network structure, is described. The key ideas behind DDM are introduced by describing a small machine, which could be a COMA on its own or a subsystem of a larger COMA, and its protocol. A large machine with hundreds of processors is also described. The DDM prototype project is discussed, and simulated performance results are presented.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>