Cache Coherence Research Articles

Subtleties of Run-Time Virtual Address Stacks

The run-time virtual address (VA) stack has some unique properties, which have garnered the attention of researchers. The stack one-dimensionally grows and shrinks at its top, and contains data that is seemingly local/private to one thread, or process. Most prior related research has focused on these properties. However, this article aims to demonstrate how conventional wisdom pertaining to the run-time VA stack fails to capture some critical subtleties and complexities. We first explore two widely established assumptions surrounding the VA stack area: (1) Data accesses can be classified as falling either under VA-stack-area accesses, or non-stack-area accesses, with no aliasing; (2) The VA stack data is completely private and invisible to other threads/processes. Subsequently, we summarize a representative selection of related work that pursued the micro-architectural concept of using run-time VA stacks to extend the general-purpose register file. We then demonstrate why these assumptions are invalid, by using examples from prior work to highlight the potential hazards regarding data consistency, shared memory consistency, and cache coherence. Finally, we suggest safeguards against these hazards. Overall, we explore the function-critical issues that future operating systems and compilers should address to effectively reap all the benefits of using run-time VA stacks.

PIM-enabled instructions

Processing-in-memory (PIM) is rapidly rising as a viable solution for the memory wall crisis, rebounding from its unsuccessful attempts in 1990s due to practicality concerns, which are alleviated with recent advances in 3D stacking technologies. However, it is still challenging to integrate the PIM architectures with existing systems in a seamless manner due to two common characteristics: unconventional programming models for in-memory computation units and lack of ability to utilize large on-chip caches. In this paper, we propose a new PIM architecture that (1) does not change the existing sequential programming models and (2) automatically decides whether to execute PIM operations in memory or processors depending on the locality of data. The key idea is to implement simple in-memory computation using compute-capable memory commands and use specialized instructions, which we call PIM-enabled instructions, to invoke in-memory computation. This allows PIM operations to be interoperable with existing programming models, cache coherence protocols, and virtual memory mechanisms with no modification. In addition, we introduce a simple hardware structure that monitors the locality of data accessed by a PIM-enabled instruction at runtime to adaptively execute the instruction at the host processor (instead of in memory) when the instruction can benefit from large on-chip caches. Consequently, our architecture provides the illusion that PIM operations are executed as if they were host processor instructions. We provide a case study of how ten emerging data-intensive workloads can benefit from our new PIM abstraction and its hardware implementation. Evaluations show that our architecture significantly improves system performance and, more importantly, combines the best parts of conventional and PIM architectures by adapting to data locality of applications.

Callback

Cache coherence protocols based on self-invalidation allow a simpler design compared to traditional invalidation-based protocols, by relying on data-race-free (DRF) semantics and applying self-invalidation on racy synchronization points exposed to the hardware. Their simplicity lies in the absence of invalidation traffic, which eliminates the need to track readers in a directory, and reduces the number of transient protocol states. With the addition of self-downgrade these protocols can become effectively directory-free. While this works well for race-free data, unfortunately, lack of explicit invalidations compromises the effectiveness of any synchronization that relies on races. This includes any form of spin waiting, which is employed for signaling, locking, and barrier primitives. In this work we propose a new solution for spin-waiting in these protocols, the callback mechanism, that is simpler and more efficient than explicit invalidation. Callbacks are set by reads involved in spin waiting, and are satisfied by writes (that can even precede these reads). To implement callbacks we use a small (just a few entries) directory-cache structure that is intended to service only these "spin-waiting" races. This directory structure is self-contained and is not backed up in any way. Entries are created on demand and can be evicted without the need to preserve their information. Our evaluation shows a significant improvement both over explicit invalidation and over exponential back-off, the state-of-the-art mechanism for self-invalidation protocols to avoid spinning in the shared cache.

Fusion

Chip designers have shown increasing interest in integrating specialized fixed-function coprocessors into multicore designs to improve energy efficiency. Recent work in academia [11, 37] and industry [16] has sought to enable more fine-grain offloading at the granularity of functions and loops. The sequential program now needs to migrate across the chip utilizing the appropriate accelerator for each program region. As the execution migrates, it has become increasingly challenging to retain the temporal and spatial locality of the original program as well as manage the data sharing. We show that with the increasing energy cost of wires and caches relative to compute operations, it is imperative to optimize data movement to retain the energy benefits of accelerators. We develop FUSION, a lightweight coherent cache hierarchy for accelerators and study the tradeoffs compared to a scratchpad based architecture. We find that coherency, both between the accelerators and with the CPU, can help minimize data movement and save energy. FUSION leverages temporal coherence [32] to optimize data movement within the accelerator tile. The accelerator tile includes small per-accelerator L0 caches to minimize hit energy and a per-tile shared cache to improve localized-sharing between accelerators and minimize data exchanges with the host LLC. We find that overall FUSION improves performance by 4.3× compared to an oracle DMA that pushes data into the scratchpad. In workloads with inter-accelerator sharing we save up to 10x the dynamic energy of the cache hierarchy by minimizing the host-accelerator data ping-ponging.

Hardware Trends

This article is about three trends in computer hardware, and some of the challenges and opportunities that I think they provide for the distributed computing community. A common theme in all of these trends is that hardware is moving away from assumptions that have often been made about the relative performance of different operations (e.g., computation versus network communication), the reliability of operations (e.g., that memory accesses are reliable, but network communication is not), and even some of the basic properties of the system (e.g., that the contents of main memory are lost on power failure). Section 1 introduces "rack-scale" systems and the kinds of properties likely in their interconnect networks. Section 2 describes challenges in systems with shared physical memory but without hardware cache coherence. Section 3 discusses non-volatile byte-addressable memory. The article is based in part on my talk at the ACM PODC 2014 event in celebration of Maurice Herlihy's sixtieth birthday.

Modification of Cache Inclusion Property for Multicore Systems

The cache coherency protocol guarantees the validity of the cache block by keeping it with the latest updated contents. In multi-level cache memory hierarchy, the inclusion property enforces the lower layer cache to maintain the superset of its higher layer cache contents. The inclusion property has its benefits for cache coherence, but it may waste valuable cache blocks and bandwidth by invalidating the duplicated contents in the higher level cache. In this paper, a new cache contention management scheme is presented with modified inclusion property to avoid unnecessary cache block invalidation.

PVCoherence: Designing Flat Coherence Protocols for Scalable Verification

The goal of this work is to design cache coherence protocols with many cores such that they can be verified with existing verification methodologies. In particular, the authors focus on flat (nonhierarchical) coherence protocols using a mostly automated methodology based on parametric verification. They present design guidelines that, if followed by architects, enable parametric verification of protocols with arbitrary numbers of cores.

龙芯GS464E处理器核架构设计

Loongson GS464E is the most recent processor architecture introduced by Loongson Technology. In this paper, we describe the key aspects of the GS464E architecture. Compared to the previous GS464 architecture, GS464E focuses on improving the performance of memory access and branch prediction, using enlarged queues, caches, and TLBs in the processor. Support for the MIPS DSP instruction set and virtualization is also provided. Specifically, the memory subsystems have a 3-level cache hierarchy each with an LRU replacement policy, and also support multi-processor cache coherence. With the aforementioned optimization features, GS464E has become an innovative, high-performance processor architecture.

一种面向高性能计算的自主众核处理器结构

Owing to advances in semiconductor technology, many-core processors have been widely used for high-performance computing. Through the support of the national 863 program and the national major project of HeGaoJi, research and development in many-core processors has significantly progressed in China. In this paper, we present a homegrown, many-core processor architecture for high-performance computing systems. The processor is heterogeneous and supports different application features with a unified ISA architecture, a unified execution model, and a share-memory that supports cache coherence. This paper illustrates techniques for solving“memory wall”,“power well”, and“reliability wall”problems in the processor. The prototype chip achieved a peak performance of more than 1 TFlops with four MPEs and 256 CPEs during experimental tests.

NUMA-Aware Scalable and Efficient In-Memory Aggregation on Large Domains

Business Intelligence (BI) is recognized as one of the most important IT applications in the coming big data era. In recent years, non-uniform memory access (NUMA) has become the de-facto architecture of multiprocessors on the new generation of enterprise servers. Such new architecture brings new challenges to optimization techniques on traditional operators in BI. Aggregation, for example, is one of the basic building blocks of BI, while its processing performance with existing hash-based algorithms scales poorly in terms of the number of cores under NUMA architecture. In this paper, we provide new solutions to tackle the problem of parallel hash-based aggregation, especially targeting at domains of extremely large cardinality. We propose a NUMA-aware radix partitioning (NaRP) method which divides the original huge relation table into subsets, without invoking expensive remote memory access between nodes of the cores. We also present a new efficient aggregation algorithm (EAA), to aggregate the partitioned data in parallel with low cache coherence miss and locking costs. Theoretical analysis as well as empirical study on an IBM X5 server prove that our proposals are at least two times faster than existing methods.

ICCI: In-Cache Coherence Information

In this paper we introduce ICCI, a new cache organization that leverages shared cache resources and flat coherence protocols to provide inexpensive hardware cache coherence for large core counts (e.g., 512), achieving execution times close to a non-scalable sparse directory while noticeably reducing the energy consumption of the memory system. Very simple changes in the system with respect to traditional bit-vector directories are enough to implement ICCI. Moreover, ICCI does not introduce any storage overhead with respect to a broadcast-based protocol, yet it provides large storage space for coherence information. ICCI makes smarter use of cache resources by dynamically allowing last-level cache entries to store blocks or sharing codes. This way, just the minimum number of directory entries required at runtime are allocated. Besides, ICCI suffers a negligible amount of directory-induced invalidations. Results for a 512-core CMP show that ICCI reduces the energy consumption of the memory system by up to 48 percent compared to a tag-embedded directory, up to 15 percent compared to a sparse directory, and up to 8 percent compared to the state-of-the-art Scalable Coherence Directory which ICCI also outperforms in execution time. In addition, ICCI can be used in combination with elaborated sharing codes to apply it to extremely large core counts. We also show analytically that ICCI’s dynamic allocation of entries makes it a suitable candidate to store coherence information efficiently for very large core counts (e.g., over 200K cores), based on the observation that data sharing makes fewer directory entries necessary per core as core count increases.

M7: Oracle's Next-Generation Sparc Processor

The Oracle Sparc M7 processor more than triples the throughput of the Sparc M6 processor, while increasing per-thread performance, power efficiency, and I/O bandwidth. M7 contains 32 8-thread, dual-issue, out-of-order Sparc cores. To minimize L3 cache hit latency, M7 features a partitioned L3 cache, with a novel on-chip network for communication between the cache partitions, coherence control, and memory. Memory bandwidth is more than 3 times that of the Oracle Sparc M6 processor. On-chip accelerators providing query filtering and decompression can provide another order of magnitude performance increase for these tasks. M7 keeps power consumption at a minimum through multiple power-saving techniques. M7 scales to 32-processor shared memory multiprocessing systems (SMPs). Coherent shared memory is also supported between SMPs in a cluster. Such a cluster is robust against both communication and SMP failure. Application data integrity guards against pointer-related software vulnerabilities without slowing the processor. Fine-grained memory migration supports concurrent garbage collection.

Hybrid Shared-aware Cache Coherence Transition Strategy

Chip-multiprocessors have played a significant role in real parallel computer architecture design. For integrating tens of cores into a chip, designs tend towards with physically distributed last level caches. This naturally results in a Non-Uniform Cache Access design, where on-chip access latencies depend on the physical distances between requesting cores and home cores where the data is cached. Therefore, data movement and management impact access latency and power consumption. Remote misses limit the performance of multi-threaded applications, so using data locality is fundamental importance in Chip-multiprocessors. In this work we observed that a shared data writing behavior dramatically wastes precious on-chip cache resource and seriously affects the whole system performance. Therefore, we emphasis on improving the performance of applications that exhibit high data sharing, and propose a new prediction mechanism to predict accurately the impact of shared data and a scalable, efficient hybrid shared-aware cache coherence transition strategy which collaborate with directory-based MESI cache coherence protocol. In order to evaluate our proposal transition strategy, we experiment with the NAS Parallel Benchmarks and a modern Intel Harpertown multi-core machine. Results show the whole performance gains of up to 21% opposed to the traditional write-invalidate cache coherence transition strategy.

Adaptive Cache Coherence Mechanisms with Producer–Consumer Sharing Optimization for Chip Multiprocessors

In chip multiprocessors (CMPs), maintaining cache coherence can account for a major performance overhead. Write-invalidate protocols adapted by most CMPs generate high cache-to-cache misses under producer–consumer sharing patterns. Accordingly, this paper presents three cache coherence mechanisms optimized for CMPs. First, to reduce coherence misses observed in write-invalidate-based protocols, we propose a dynamic write-update mechanism augmented on top of a write-invalidate protocol. This mechanism is specifically triggered at the detection of a producer–consumer sharing pattern. Second, we extend this adaptive protocol with a bandwidth-adaptive mechanism to eliminate performance degradation from write-updates under limited bandwidth. Finally, proximity-aware mechanism is proposed to extend the base adaptive protocol with latency-based optimizations. Experimental analysis is conducted on a set of scientific applications from the SPLASH-2 and NAS parallel benchmark suites. The proposed mechanisms were shown to reduce coherence misses by up to 48% and in return speed up application performance up to 30%. Bandwidth-adaptive mechanism is proven to perform well under varying levels of available bandwidth. Results from our proposed proximity-aware extension demonstrated up to 6% performance gain over the base adaptive protocol for 64-core tiled CMP runs. In addition, the analytical model provided good estimates for performance gains from our adaptive protocols.

On-Chip Active Messages for Speed, Scalability, and Efficiency

This paper describes and quantifies the benefits of adding low-overhead active messages to many-core, cache-coherent chip-multiprocessors. The active messages we analyze are user defined and trigger the atomic execution of a custom software handler at the destination. Programmers can use these active messages to both move data with less overhead than cache coherency and, more importantly, explicitly send computation to data. Doing so greatly improves (11× speed, 4.8× energy) communication idioms such as shared object modification, reductions, data walks, point-to-point communication, and all-to-all communication. Active messages enhance program scalability: applications using them run 63 percent faster with 11 percent less energy on 256 cores. The relative benefits of active messages grow with larger numbers of cores.

Architectural Support for Data-Driven Execution

The exponential growth of sequential processors has come to an end, and thus, parallel processing is probably the only way to achieve performance growth. We propose the development of parallel architectures based on data-driven scheduling. Data-driven scheduling enforces only a partial ordering as dictated by the true data dependencies, which is the minimum synchronization possible. This is very beneficial for parallel processing because it enables it to exploit the maximum possible parallelism. We provide architectural support for data-driven execution for the Data-Driven Multithreading (DDM) model. In the past, DDM has been evaluated mostly in the form of virtual machines. The main contribution of this work is the development of a highly efficient hardware support for data-driven execution and its integration into a multicore system with eight cores on a Virtex-6 FPGA. The DDM semantics make barriers and cache coherence unnecessary, which reduces the synchronization latencies significantly and makes the cache simpler. The performance evaluation has shown that the support for data-driven execution is very efficient with negligible overheads. Our prototype can support very small problem sizes (matrix 16×16) and ultra-lightweight threads (block of 4x4) that achieve speedups close to linear. Such results cannot be achieved by software-based systems.

� ���������� ����������� ������ ����������� �������������� ������ � ����������� �������

Abstract. This article analyzes existing methods of verification of cache coherence protocols of scalable systems. Based on the research literature, the paper describes a method of formal parameterized verification of safety properties of cache coherence protocols. The paper proposes a design of a verification system for cache coherence protocols. The article analyzes the method in terms of development and examination of the corresponding Promela model of the German cache coherence protocol and discusses extension and automation of the method needed to adapt it to verification challenges of the Elbrus microprocessors. Keywords: formal verification; model checking; deductive verification; cache coherence protocol; Elbrus 1. Introduction Modern microprocessor systems are scalable – the number of cores per chip increases and chips are combined into clusters. Each processor of the system has access to the shared address space. However, memory is physically distributed among the processors in order to increase the bandwidth and reduce the latency to local memory. Thus, access to the local memory is faster than access to the remote memory. To decrease the memory bandwidth demands of a processor, processors are equipped with multilevel caches. Caching of shared data introduces the problem of cache coherence. To solve the problem, computer architects often use hardware mechanisms that implement cache coherence protocols. Concurrent work of many hardware devices (for example, cache and main memory controllers), which exchange information in accordance with a cache coherence protocol, results in a colossal size of the protocol’s state space. This, in turn, makes verification of cache coherence protocols an extremely hard task.

Gem5-gpu: A Heterogeneous CPU-GPU Simulator

gem5-gpu is a new simulator that models tightly integrated CPU-GPU systems. It builds on gem5, a modular full-system CPU simulator, and GPGPU-Sim, a detailed GPGPU simulator. gem5-gpu routes most memory accesses through Ruby, which is a highly configurable memory system in gem5. By doing this, it is able to simulate many system configurations, ranging from a system with coherent caches and a single virtual address space across the CPU and GPU to a system that maintains separate GPU and CPU physical address spaces. gem5-gpu can run most unmodified CUDA 3.2 source code. Applications can launch non-blocking kernels, allowing the CPU and GPU to execute simultaneously. We present gem5-gpu ’s software architecture and a brief performance validation. We also discuss possible extensions to the simulator. gem5-gpu is open source and available at gem5-gpu.cs.wisc.edu.

Cooperative Computing Techniques for a Deeply Fused and Heterogeneous Many-Core Processor Architecture

Due to advances in semiconductor techniques, many-core processors have been widely used in high performance computing. However, many applications still cannot be carried out efficiently due to the memory wall, which has become a bottleneck in many-core processors. In this paper, we present a novel heterogeneous many-core processor architecture named deeply fused many-core (DFMC) for high performance computing systems. DFMC integrates management processing elements (MPEs) and computing processing elements (CPEs), which are heterogeneous processor cores for different application features with a unified ISA (instruction set architecture), a unified execution model, and share-memory that supports cache coherence. The DFMC processor can alleviate the memory wall problem by combining a series of cooperative computing techniques of CPEs, such as multi-pattern data stream transfer, efficient register-level communication mechanism, and fast hardware synchronization technique. These techniques are able to improve on-chip data reuse and optimize memory access performance. This paper illustrates an implementation of a full system prototype based on FPGA with four MPEs and 256 CPEs. Our experimental results show that the effect of the cooperative computing techniques of CPEs is significant, with DGEMM (double-precision matrix multiplication) achieving an efficiency of 94%, FFT (fast Fourier transform) obtaining a performance of 207 GFLOPS and FDTD (finite-difference time-domain) obtaining a performance of 27 GFLOPS.

LockSim: An Event-Driven Simulator for Modeling Spin Lock Contention

Spin lock contention in operating systems can limit scalability on multicore systems so significantly that an increase in the number of cores actually leads to reduced speedup (i.e., scalability collapse). Modeling spin lock contention is an effective way to understand the scalability collapse phenomenon and explore collapse avoidance schemes. However, previous spin lock models have disadvantages in accuracy and efficiency. To overcome these drawbacks, this paper proposes LockSim, an event-driven simulator which models both the sequential execution in lock-protected codes (i.e., critical sections) and shared hardware resource contention caused by the cache coherence protocol. Our simulator is verified against real-world workloads with different degrees of spin lock contention. Experimental results suggest that LockSim can reproduce the scalability collapse phenomenon with better accuracy than previous work. Besides, several metrics are also used to characterize this phenomenon and collapse avoidance methods are investigated.