Directory-based Cache Coherence Research Articles

Simulation Cache Coherence Protocols in Multicore Processors

The cache coherence problem is the challenge of keeping multiple cache synchronized when one of the processors update its local copy of data which is shared among multiple cache. This paper discusses several different varieties of cache coherence protocols including with their pros and cons, and using simulation technique it will address this problem and compare between two protocols that use to solve it: Directory-based protocol and Snooping protocol. Simulation results have shown that snooping based systems are appropriate for high bandwidth systems while directory-based cache coherence protocols are suitable for lower bandwidth systems.

Distributed Shared Mamory for NoC-based MPSoCs

As the number of cores and functionalities integrated in embedded devices increases, the amount of memory used on these devices also increases, justifying the development of memory architectures that present scalability, low energy consumption and low latency. Two factors impact the scalability of MPSoC systems: cache access latency and energy consumption. The distance of the processors to the cache banks and the cache coherence protocol influences both factors. This work proposes a physically distributed data L2 cache as the cache architecture for a NoC-based MPSoC, because it allows the concept of clustering, and the implementation of data migration algorithms to reducing cache access latency. Results show that the number of cycles required to execute a given application might reduce 23% with the appropriate number of L2 cache banks. Also, a directory-based cache coherence protocol was implemented, exploiting the NoC physical services to improve performance. Results show a reduction of 17% in the number of clock cycles and a reduction up to 86% (average reduction: 39%) in energy consumption for some cache transactions.

An Automatic Proving Approach to Parameterized Verification

Formal verification of parameterized protocols such as cache coherence protocols is a significant challenge. In this article, we propose an automatic proving approach and its prototype paraVerifier to handle this challenge within a unified framework as follows: (1) To prove the correctness of a parameterized protocol, our approach automatically discovers auxiliary invariants and the corresponding dependency relations among the discovered invariants and protocol rules from a small instance of the to-be-verified protocol, and (2) the discovered invariants and dependency graph are then automatically generalized into a parameterized form and sent to the theorem prover, Isabelle. As a side product, the final verification result of a protocol is provided by a formal and human-readable proof. Our approach has been successfully applied to a number of benchmarks, including snoopying-based and directory-based cache coherence protocols.

Efficient distributed memory management with RDMA and caching

Recent advancements in high-performance networking interconnect significantly narrow the performance gap between intra-node and inter-node communications, and open up opportunities for distributed memory platforms to enforce cache coherency among distributed nodes. To this end, we propose GAM, an efficient distributed in-memory platform that provides a directory-based cache coherence protocol over remote direct memory access (RDMA). GAM manages the free memory distributed among multiple nodes to provide a unified memory model, and supports a set of user-friendly APIs for memory operations. To remove writes from critical execution paths, GAM allows a write to be reordered with the following reads and writes, and hence enforces partial store order (PSO) memory consistency. A light-weight logging scheme is designed to provide fault tolerance in GAM. We further build a transaction engine and a distributed hash table (DHT) atop GAM to show the ease-of-use and applicability of the provided APIs. Finally, we conduct an extensive micro benchmark to evaluate the read/write/lock performance of GAM under various workloads, and a macro benchmark against the transaction engine and DHT. The results show the superior performance of GAM over existing distributed memory platforms.

ReDirect

As we enter the dark silicon era, architects should not envision designs in which every transistor remains turned on permanently but rather ones in which portions of the chip are judiciously turned on/off depending on the characteristics of a workload. At the same time, due to the increasing cost per transistor, architects should also consider new ways to re-purpose transistors to increase their architectural value. In this work, we consider the design of directory-based cache coherence in light of the dark silicon era and the need to re-purpose transistors. We point out that directories are not needed all of the time, and we argue that directories (and coherence) should be off unless it is actually needed for correctness. In our design, directories will be disabled and powered off for workloads with no sharing. Then only when parallel workloads need cache coherence will directories be enabled in proportion to the sharing that is present. At the same time, we exploit the structural similarities of directories and cache. If a directory is idle, then we reconfigure it to be used as extra capacity in the last-level cache. Since our novel approach can keep most directories off, we are free to select sparse overprovisioned directory designs that are reconfigurable to large amounts of cache that can significantly boost performance when the directory is idle. We call these combined features Reconfigured Dark Directories , since directories are usually dark (off) and can be reconfigured. Our results for Reconfigurable Dark Directories running SPEC 2006 applications show a performance benefit, on average, of 17% for an 8× overprovisioned fully mapped directory on a 64-tile system under low system concurrency (10% under heavy concurrency), or a 29% average speedup for a 2× overprovisioned directory on 256-tile system (10% under heavy concurrency) to systems with a conventional sparse directory design using the same overprovisioning factor.

Directory Based Cache Coherency, Organization, Operations and Challenges in Implementation - Study.

Today's systems are designed with Multi Core Architecture. The idea behind this is to achieve high system throuput. Once the Processor clock speed reached its saturation, designers opted for having multiple cores. Each Core or Processor equipped with their own private cache memory. But under Chip Multiprocessor, where all the processor have access to shared memory, having respective cache memory will result with Cache Coherency Problem. In Directory Protocol, for each block of data there is a directory entry that contains a number of pointers. The purpose of this number is to mention the locations of block copies. The important advantage of directory based protocols is that they scale much better than snoopy protocols. In addition to this it has the advantage of ability to exploit arbitrary point-to-point interconnects. But mean time it also has the overhead in terms of the storage and manipulation of directory state. This paper discus different Directory Based implementation, operations along with and its implementation difficulties.

To be silent or not: on the impact of evictions of clean data in cache-coherent multicores

Maintaining coherence across hundreds or even thousands of cores is not an easy task. Among all of the proposed solutions until now, directory-based cache coherence has been advocated as the most feasible way of beating the scalability hurdles that arise at such large scale. Thanks to the knowledge accumulated during the last four decades, there is general consensus on the impact of most of the design aspects of directory coherence on performance, energy consumption and cost. However, there is one subtle design point for which we have observed some divergences in contemporary research works on cache-coherent multicores. Specifically, while some recent works assume a silent replacement policy for evictions of clean data in the last-level private caches, others implement just the opposite that we call a noisy replacement policy, and even others do not mention how these evictions are managed. In this work, we put this important aspect into the spotlight, demonstrating that the way in which evictions of clean data are managed can have important influence on the performance and energy consumption of a directory-based cache coherence protocol. We show that the noisy replacement policy leads to a significant increase in the total traffic (around 20% in several cases, 9.6% on average) compared with the silent policy. Given the important fraction of the total power budget that the on-chip interconnection network of future manycores is expected to consume, assuming the silent replacement policy for clean data will lead to non-negligible energy savings. Moreover, and what is more important, we have observed that depending on the particular directory structure used, assuming silent replacements could affect performance or not. This means that the use of noisy replacements is not justified in all cases, since it would increase unnecessarily network traffic without leading to any performance advantages.

Improving multiprocessor performance with fine-grain coherence bypass

Efficient and scalable cache coherence protocol is crucial to high-performance servers with shared-memory. The directory-based cache coherence protocol is more desirable than the snooping-based protocol with respect to the scalability. However, even for the former protocol, scaling to a large number of cores is also challenging due to the additional area requirements of the directories. We observed that a significant percentage of the referenced memory blocks were only accessed by a single core (even in parallel applications) which could be considered as private memory blocks. An intuitive motivation from this observation is that memory blocks accessed by a single core do not require coherence maintenance. The issue is to identify the private block and track the change of its access pattern. We propose a novel hardware approach to (1) dynamically identify the shared memory blocks at the cache block level, and (2) bypass the coherence procedure for the private memory blocks. This approach increases the effectiveness of the directory-based approach and therefore improves the system performance. Experimental results showed that, our approach can on an average (1) avoid the coherence tracking of about 54% referenced memory blocks, (2) reduce the coherence overhead by 77%, (3) avoid 8% L2 cache misses, and (4) shorten the execution time of parallel applications by 13%.

A SystemC Cache Simulator for a Multiprocessor Shared Memory System

In this research we built a SystemC Level-1 data cache system in a distributed shared memory architectural environment, with each processor having its own local cache. Using a set of Fast-Fourier Transform and Random trace files we evaluated the cache performance, based on the number of cache hits/misses, of the caches using snooping and directory-based cache coherence protocols. A series of experiments were carried out, with the results of the experiments showing that the directory-based MOESI cache coherency protocol has a performance edge over the snooping Valid-Invalid cache coherency protocol.

Efficient Dir0B Cache Coherency for Many-core CMPs

Directory-based cache coherency is commonly accepted as the design of choice to provide high performance and scalability in coherency maintenance for many-core CMPs. However, the on-chip area overhead required to encode sharer sets may compromise their success as core count increases. In this work, we propose the Express COherence NOtification (ECONO) protocol, a simple and efficient Dir0B cache coherence protocol that does not require sharer sets encoding while approaching performance of a conventional directory-based protocol. To accomplish that, ECONO relies on express coherence notifications which are broadcast atomically over a dedicated lightweight on-chip network leveraging state-of-the-art technology. Detailed full-system simulation using a representative set of benchmarks corroborates our statement.

Priority-based squash reducing methods in thread level speculation

Thread level speculation (TLS) aggressively transforms long serial programme into multiple short parallel threads to significantly boost the performance of sequential programmes. But frequent squashing, which is caused by violation between multiple parallel threads, will greatly offset the benefits from parallelisation. Most existing works only focus on improving TLS scheme itself directly, such as preventing false-sharing, pre-computing or predicting values and so on. On the contrast, we realise that squashes are caused by messages whose arriving orders are violated. Thus, these squashes can be reduced by rearranging TLS messages. For reducing TLS squashes, in this paper, we first propose a priority-aware network-on-chip (NoC), which uses a priority-based packet arbitration policy to reorder messages at router. Further, we extend this priority scheme by employing prioritising policy into the directory for TLS system that uses directory-based cache coherence protocol. The extension results in a cost-less version. Experimental evaluation for five typical application kernels of SPEC2000 shows that our NoC approach reduces squashing rate by 22% in best case and 15% on average.

Rebound

As we move to large manycores, the hardware-based global check-pointing schemes that have been proposed for small shared-memory machines do not scale. Scalability barriers include global operations, work lost to global rollback, and inefficiencies in imbalanced or I/O-intensive loads. Scalable checkpointing requires tracking inter-thread dependences and building the checkpoint and rollback operations around dynamic groups of communicating processors. To address this problem, this paper introduces Rebound , the first hardware-based scheme for coordinated local checkpointing in multiprocessors with directory based cache coherence. Rebound leverages the transactions of a directory protocol to track inter-thread dependences. In addition, it boosts checkpointing efficiency by: (i) delaying the writeback of data to safe memory at checkpoints, (ii) supporting operation with multiple checkpoints, and (iii) optimizing checkpointing at barrier synchronization. Finally, Rebound introduces distributed algorithms for checkpointing and rollback sets of processors. Simulations of parallel programs with up to 64 threads show that Rebound is scalable and has very low overhead. For 64 processors, its average performance overhead is only 2%, compared to 15% for global checkpointing.

A Direct Coherence Protocol for Many-Core Chip Multiprocessors

Future many-core CMP designs that will integrate tens of processor cores on-chip will be constrained by area and power. Area constraints make impractical the use of a bus or a crossbar as the on-chip interconnection network, and tiled CMPs organized around a direct interconnection network will probably be the architecture of choice. Power constraints make impractical to rely on broadcasts (as, for example, Token-CMP does) or any other brute-force method for keeping cache coherence, and directory-based cache coherence protocols are currently being employed. Unfortunately, directory protocols introduce indirection to access directory information, which negatively impacts performance. In this work, we present DiCo-CMP, a novel cache coherence protocol especially suited to future many-core tiled CMP architectures. In DiCo-CMP, the task of storing up-to-date sharing information and ensuring ordered accesses for every memory block is assigned to the cache that must provide the block on a miss. Therefore, DiCo-CMP reduces the miss latency compared to a directory protocol by sending requests directly to the cache that provides the block in a cache miss. These latency reductions result in improvements in execution time of up to 6 percent, on average, over a directory protocol. In comparison with Token-CMP, our protocol only sends one request message for each cache miss, as such is able to reduce network traffic by 43 percent.

Dealing with Transient Faults in the Interconnection Network of CMPs at the Cache Coherence Level

The importance of transient faults is predicted to grow due to current technology trends of increased scale of integration. One of the components that will be significantly affected by transient faults is the interconnection network of chip multiprocessors (CMPs). To deal efficiently with these faults and differently from other authors, we propose to use fault-tolerant cache coherence protocols that ensure the correct execution of programs when not all messages are correctly delivered. We describe the extensions made to a directory-based cache coherence protocol to provide fault tolerance and provide a modified set of token counting rules which are useful to design fault-tolerant token-based cache coherence protocols. We compare the directory-based fault-tolerant protocol with a token-based fault-tolerant one. We also show how to adjust the fault tolerance parameters to achieve the desired level of fault tolerance and measure the overhead achieved to be able to support very high fault rates. Simulation results using a set of scientific, multimedia, and commercial applications show that the fault tolerance measures have virtually no impact on execution time with respect to a non-fault-tolerant protocol. Additionally, our protocols can support very high rates of transient faults at the cost of slightly increased network traffic.

A Use-Level Simulator for Tiled Chip Multiprocessor

随着片上晶体管资源的增多和互连线延迟的加大,分片式多核微处理器已成为多核处理器设计的新方向.为了对这种新型处理器进行体系结构的深入研究和设计空间的探索,设计并实现了针对分片式多核处理器的用户级多核性能模拟器.该多核模拟器在龙芯2号单处理器核的基础上,完整地模拟了基于目录的Cache一致性协议和存储转发式片上互联网络的结构模型,详细地刻画了由于系统乱序处理各种请求应答和请求之间的冲突而造成的时序特性,可以通过运行各种串行或并行的工作负载对多核处理器的各种重要性能指标加以评估,为多核处理器的结构设计提供了快速、灵活、高效的研究平台.

Lightweight Transactional Memory systems for NoCs based architectures: Design, implementation and comparison of two policies

Hardware Transactional Memory (HTM) is an attractive design concept which simplifies parallel programming by shifting the problem of correct synchronization between threads to the underlying hardware memory system. There has recently been much work dealing with Hardware Transactional Memory systems, but to the best of our knowledge, all assume a write-back cache coherence protocol. As no existing HTM system is based on a write-through coherence protocol, we propose the design and implementation of an original Transactional Memory system based on a Write-Through Invalidate directory based cache coherence protocol, and we perform the comparison of this system with a more common write-back MESI (Modified Exclusive Shared Invalid) based HTM system. The comparison is done on an architecture with two possible memory repartitions, either centralized or distributed, at the cycle-accurate level. Additionally, our work takes into account the difficulties related to on-chip communication using network like interconnects, in order to be able to target the embedded domain. We compare the execution performances of both HTM systems on two micro-kernels and on a subset of the SPLASH-2 benchmarks. Results show that the coherence protocol has an impact on the execution times, but that no solution outperforms the other. However, the write-back has overall slightly better results, especially when the memory is not distributed.

Performance evaluation of directory protocols on an optical broadcast-based distributed shared memory multiprocessor

Recent advances in the development of optical technologies suggest the possible emergence of broadcast-based optical interconnects within cache–coherent distributed shared memory (DSM) multiprocessor architectures. It is well known that the cache–coherence protocol is a critical issue in designing such architectures because it directly affects memory latencies. In this paper, we evaluate via simulation the performance of three directory-based cache–coherence protocols; strict request–response, intervention forwarding and reply forwarding on the Simultaneous Optical Multiprocessor Exchange Bus (SOME-Bus), which is a low-latency and high-bandwidth broadcast-based fiber-optic interconnection network supporting DSM. The simulated system contains 64 nodes, each of which has a processor, a cache controller, a directory controller and an output channel. Simulations have been conducted for each protocol to measure average processor utilization, average network latency and average number of packets transferred over the network for varying values of the important DSM parameters such as the ratio of the mean channel service time to mean thread run time ( T/R), probability of a cache block being in modified state { P( M)}, the fraction of write misses { P( W)} and home node contention rate. The results reveal that for all cases, except for low values of P( M), intervention forwarding gives the worst performance (lowest processor utilization and highest latency). The performance of strict request–response and reply forwarding is comparable for several values of the DSM parameters and contention rate. For a contention rate of 0%, the increase of P( M) makes reply forwarding perform better than strict request–response. The performance of all protocols decreases with the increase of P( W) and contention rate. However, the performance of strict request–response is the least affected among other protocols due to the negative impact of the increase of P( W) and contention rate. Therefore, for the full contention case (i.e. contention rate of 100%); for low values of P( M), or for mid values of P( M) and high values of P( W), strict request–response performs better than reply forwarding. These results are significant in the sense that they provide an insight to multiprocessor architecture designers for comparing the performance of different directory-based cache–coherence protocols on a broadcast-based interconnection network for different values of the DSM parameters and varying rates of contention.

Scalable directory architecture for distributed shared memory chip multiprocessors

Traditional Directory-based cache coherence protocol is far from optimal for large-scale cache coherent shared memory multiprocessors due to the increasing latency to access directories stored in DRAM memory. Instead of keeping directories in main memory, we consider distributing the directory together with L2 cache across all nodes on a Chip Multiprocessor. Each node contains a processing unit, a private L1 cache, a slice of L2 cache, memory controller and a router. Both L2 cache and memories are distributed shared and interleaved by a subset of memory address bits. All nodes are interconnected through a low latency two dimensional Mesh network. Directory, being a split component to L2 cache, only stores sharing information for blocks while L2 cache stores only data blocks exclusive with L1 cache. Shared L2 cache can increase total effective cache capacity on chip, but also increase the miss latency when data is on a remote node. Being different from Directory Cache structure, our proposal totally removes the directory from memory, which saves memory space and reduces access latency. Compared to L2 cache that combines directory information internally, our L2 cache structure saves up to 88% cache space and achieves similar performance.

Predicate abstraction with indexed predicates

Predicate abstraction provides a powerful tool for verifying properties of infinite-state systems using a combination of a decision procedure for a subset of first-order logic and symbolic methods originally developed for finite-state model checking. We consider models containing first-order state variables, where the system state includes mutable functions and predicates. Such a model can describe systems containing arbitrarily large memories, buffers, and arrays of identical processes. We describe a form of predicate abstraction that constructs a formula over a set of universally quantified variables to describe invariant properties of the first-order state variables. We provide a formal justification of the soundness of our approach and describe how it has been used to verify several hardware and software designs, including a directory-based cache coherence protocol.

The impact of wrong-path memory references in cache-coherent multiprocessor systems

The core of current-generation high-performance multiprocessor systems is out-of-order execution processors with aggressive branch prediction. Despite their relatively high branch prediction accuracy, these processors still execute many memory instructions down mispredicted paths. Previous work that focused on uniprocessors showed that these wrong-path (WP) memory references may pollute the caches and increase the amount of cache and memory traffic. On the positive side, however, they may prefetch data into the caches for memory references on the correct-path. While computer architects have thoroughly studied the impact of WP effects in uniprocessor systems, there is no comparable work for multiprocessor systems. In this paper, we explore the effects of WP memory references on the memory system behavior of shared-memory multiprocessor (SMP) systems for both broadcast and directory-based cache coherence. Our results show that these WP memory references can increase the amount of cache-to-cache transfers by 32%, invalidations by 8% and 20% for broadcast and directory-based SMPs, respectively, and the number of writebacks by up to 67% for both systems. In addition to the extra coherence traffic, WP memory references also increase the number of cache line state transitions by 21% and 32% for broadcast and directory-based SMPs, respectively. In order to reduce the performance impact of these WP memory references, we introduce two simple mechanisms—filtering WP blocks that are not likely-to-be-used and WP aware cache replacement—that yield speedups of up to 37%.