Shared Cache Research Articles

Performance Metrics and Models for Shared Cache

Performance metrics and models are prerequisites for scientific understanding and optimization. This paper introduces a new footprint-based theory and reviews the research in the past four decades leading to the new theory. The review groups the past work into metrics and their models in particular those of the reuse distance, metrics conversion, models of shared cache, performance and optimization, and other related techniques.

CLU: Co-Optimizing Locality and Utility in Thread-Aware Capacity Management for Shared Last Level Caches

Most chip-multiprocessors nowadays adopt a large shared last-level cache (SLLC). This paper is motivated by our analysis and evaluation of state-of-the-art cache management proposals which reveal a common weakness. That is, the existing alternative replacement policies and cache partitioning schemes, targeted at optimizing either locality or utility of co-scheduled threads, cannot deliver consistently the best performance under a variety of workloads. Therefore, we propose a novel adaptive scheme, called CLU, to interactively co-optimize the locality and utility of co-scheduled threads in thread-aware SLLC capacity management. CLU employs lightweight monitors to dynamically profile the LRU (least recently used) and BIP (bimodal insertion policy) hit curves of individual threads on runtime, enabling the scheme to co-optimize the locality and utility of concurrent threads and thus adapt to more diverse workloads than the existing approaches. We provide results from extensive execution-driven simulation experiments to demonstrate the feasibility and efficacy of CLU over the existing approaches (TADIP, NUCACHE, TA-DRRIP, UCP, and PIPP).

IMPROVING peLIFO CACHE REPLACEMENT POLICY: HARDWARE REDUCTION AND THREAD-AWARE EXTENSION

Studying blocks behavior during their lifetime in cache can provide useful information to reduce the miss rate and therefore improve processor performance. According to this rationale, the peLIFO replacement algorithm [M. Chaudhuri, Proc. Micro'09, New York, 12–16 December, 2009, pp. 401–412], which learns dynamically the number of cache ways required to satisfy short-term reuses preserving the remaining ways for long-term reuses, has been recently proposed. In this paper, we propose several changes to the original peLIFO policy in order to reduce the implementation complexity involved, and we extend the algorithm to a shared-cache environment considering dynamic information about threads behavior to improve cache efficiency. Experimental results confirm that our simplification techniques reduce the required hardware with a negligible performance penalty, while the best of our thread-aware extension proposals reduces average CPI by 8.7% and 15.2% on average compared to the original peLIFO and LRU respectively for a set of 43 multi-programmed workloads on an 8 MB 16-way set associative shared L2 cache.

A cache-aware motion estimation organization for a hardware-based H.264 encoder

The video resolution required for many types of video content has increased as technology has advanced. For the real-time encoding of the high resolutions such as full high definition (FHD), quad-FHD (QFHD) and beyond, various fast motion estimation (ME) algorithms have been researched. Caches are used for many fast MEs in a hardware-based encoder, in order to increase local memory utilization and thereby reduce external memory access. However, most previous works do not pay attention to the amount of cache access from multiple MEs. In a multi-core environment for high resolution videos, access conflicts directly affect the computation time. In this paper, various types of caches are compared in terms of the size, hit ratio, cache port conflicts and hardware overhead. To reduce the amount of cache access associated with the basic shared cache, zigzag snake scan and selective data-storage schemes are proposed for integer and fractional MEs, respectively. Additionally, the cache access arbitration hides the computation delay which arises due to a cache port conflict in a pipeline system. The proposed schemes are applicable for the existing cache design achieving a good scalability in a multi-core environment. Simulation results show that the ME computation time reduced by the proposed schemes is comparable to that of the dual-port shared cache which shows the least amount of port conflicts.

Thread Migration Prediction for Distributed Shared Caches

Chip-multiprocessors (CMPs) have become the mainstream parallel architecture in recent years; for scalability reasons, designs with high core counts tend towards tiled CMPs with physically distributed shared caches. This naturally leads to a Non-Uniform Cache Access (NUCA) design, where on-chip access latencies depend on the physical distances between requesting cores and home cores where the data is cached. Improving data locality is thus key to performance, and several studies have addressed this problem using data replication and data migration. In this paper, we consider another mechanism, hardware-level thread migration. This approach, we argue, can better exploit shared data locality for NUCA designs by effectively replacing multiple round-trip remote cache accesses with a smaller number of migrations. High migration costs, however, make it crucial to use thread migrations judiciously; we therefore propose a novel, on-line prediction scheme which decides whether to perform a remote access (as in traditional NUCA designs) or to perform a thread migration at the instruction level. For a set of parallel benchmarks, our thread migration predictor improves the performance by 24% on average over the shared-NUCA design that only uses remote accesses.

Reducing Shared Cache Misses via dynamic Grouping and Scheduling on Multicores

Multicore technology enables the system to perform more tasks with higher overall system performance.However, this performance can’t be exploited well due to the high miss rate in the second level shared cache among the cores which represents one of the multicore’s challenges. This paper addresses the dynamic co-scheduling of tasks in multicore real-time systems. The focus is on the basic idea of the megatask technique for grouping the tasks that may affect the shared cache miss rate ,and the Pfair scheduling that is then used for reducing the concurrency within the grouped tasks while ensuring the real time constrains. Consequently the shared cache miss rate is reduced.The dynamic co-scheduling is proposed through the combination of the symbiotic technique with the megatask technique for co-scheduling the tasks based on the collected information using two schemes. The first scheme is measuring the temporal working set size of each running task at run time, while the second scheme is collecting the shared cache miss rate of each running task at run time. Experiments show that the proposed dynamic coscheduling can decrease the shared cache miss rate compared to the static one by 52%.This indicates that the dynamic coscheduling is important to achieve high performance with shared cache memory for running high workloads like multimedia applications that require real-time response and continuousmedia data types.

A memory management approach based on page coloring for multi-core systems

Most multi-core platforms currently adopt shared cache among the processor cores. Due to the problem of cache contention,it is extremely difficult to predict the w orst-case execution time of the computation tasks running on different cores. A page-coloring technique w as proposed to avoid cache contention in memory access for multi-core platforms. The advantage is that the w orst case execution time of tasks running on individual core can be estimated separately using the existing WCET analysis methods for uni-processor systems. A memory management system w as designed based on the Linux to support page coloring mechanism,w hich is evaluated using the standard benchmarks. Experimental results show n that the execution time for different processor cores becomes deterministic w hen it is executed on the same platform running in the Linux supported w ith the management system.

Tree‐based scheme for reducing shared cache miss rate leveraging regional, statistical and temporal similarities

Cache miss can have a major impact on overall performance of many-core systems. A miss may result in extra traffic and delay because of coherency messages. This has been reduced in coarse-grain coherency protocols where only shared misses require a coherency message. Conventional off-chip methods manage the shared miss rate by relying on reuse histories. However the pertinent memory overhead that comes with reuse histories makes them impractical for on-chip multi-processor systems. In this study, a new scheme has been proposed to reduce shared cache miss rate in multi-processor system-on-chips that benefits from novel prefetching techniques to L2 caches from off-chip memories or other remote L2 caches located on-chip. In the proposed scheme, the previously proposed Virtual Tree Coherence (VTC) method has been extended to limit block forwarding messages to true sharers within each region. Instead of relying on exact reuse histories, shared regions are searched for regional, temporal and statistical similarities. These similarities are exploited for determining the sharers that should receive the forwarded blocks. The proposed method has been evaluated with Splash-2 workloads. Simulation results indicate that the proposed method has reduced shared miss count by up to 75%, and improved interconnect traffic by up to 47% compared with VTC.

Counter-Based Approaches for Efficient WCET Analysis of Multicore Processors with Shared Caches

To enable hard real-time systems to take advantage of multicore processors, it is crucial to obtain the worst-case execution time (WCET) for programs running on multicore processors. However, this is challenging and complicated due to the inter-thread interferences from the shared resources in a multicore processor. Recent research used the combined cache conflict graph (CCCG) to model and compute the worst-case inter-thread interferences on a shared L2 cache in a multicore processor, which is called the CCCG-based approach in this paper. Although it can compute the WCET safely and accurately, its computational complexity is exponential and prohibitive for a large number of cores. In this paper, we propose three counter-based approaches to significantly reduce the complexity of the multicore WCET analysis, while achieving absolute safety with tightness close to the CCCG-based approach. The basic counter-based approach simply counts the worst-case number of cache line blocks mapped to a cache set of a shared L2 cache from all the concurrent threads, and compares it with the associativity of the cache set to compute the worst-case cache behavior. The enhanced counter-based approach uses techniques to enhance the accuracy of calculating the counters. The hybrid counter-based approach combines the enhanced counter-based approach and the CCCG-based approach to further improve the tightness of analysis without significantly increasing the complexity. Our experiments on a 4-core processor indicate that the enhanced counter-based approach overestimates the WCET by 14% on average compared to the CCCG-based approach, while its averaged running time is less than 1/380 that of the CCCG-based approach. The hybrid approach reduces the overestimation to only 2.65%, while its running time is less than 1/150 that of the CCCG-based approach on average.

Cache isolation for virtualization of mixed general-purpose and real-time systems

Shared cache has been widely used to lessen the communication overhead between multi-core processors and off-chip main memories. In a virtualization execution environment, real-time tasks supported by real-time operating systems (RTOS) need to share the last level cache with other general purpose tasks (supported by general purpose operating systems (GPOS) or execute on bare metal). Since current hypervisors are not aware of the attributes of tasks in the virtual machines, real-time tasks cannot be adequately supported by the hypervisor. Therefore, real-time task will suffer from many cache misses, and the response of a virtual machine may be poor due to cache pollution, especially if real-time tasks compete for cache resources with heavy general-purpose tasks. To address this issue, we propose two cache optimization management policies. One is cache partitioning (CAP) - which isolates the cache statically for real-time tasks and general-purpose tasks using page coloring. Hence the predictability of cache usage is greatly increased. CAP reduces both cache pollution, and the amount of cache space that real-time tasks use, even if other cache is not used by general purpose tasks. We also study allocating different proportions of the cache for real-time tasks and find that, a proper cache ratio is optimal to guarantee response time. Due to this, another dynamic cache optimization policy is proposed, called page table prefetching (PTP). PTP employs a daemon-like thread to maintain the cache, so as to make the contents used by real-time tasks be in cache. PTP is implemented by utilizing a dedicated core assigned to general-purpose tasks to do prefectching of a page table into shared physical cache for a soft real-time task. As the page table is referenced frequently by tasks, PTP gives a considerable advantage because fewer TLB misses occur. The results from the Cyclitest and CPUspec2006 benchmarks show that both CAP and PTP can provide better response for real-time tasks. CAP has benefits due to its simplicity, but PTP has advantages in flexibility and high cache utilization due to its dynamic cached maintenance. In addition, the experimental results show that PTP improves real-time performance more but sacrifices some GP performance.

Studying multicore processor scaling via reuse distance analysis

The trend for multicore processors is towards increasing numbers of cores, with 100s of cores-- i.e . large-scale chip multiprocessors (LCMPs)--possible in the future. The key to realizing the potential of LCMPs is the cache hierarchy, so studying how memory performance will scale is crucial. Reuse distance (RD) analysis can help architects do this. In particular, recent work has developed concurrent reuse distance (CRD) and private reuse distance (PRD) profiles to enable analysis of shared and private caches. Also, techniques have been developed to predict profiles across problem size and core count, enabling the analysis of configurations that are too large to simulate. This paper applies RD analysis to study the scalability of multicore cache hierarchies. We present a framework based on CRD and PRD profiles for reasoning about the locality impact of core count and problem scaling. We find interference-based locality degradation is more significant than sharing-based locality degradation. For 256 cores running small problems, the former occurs at small cache sizes, allowing moderate capacity scaling of multicore caches to achieve the same cache performance (MPKI) as a single-core cache. At very large problems, interference-based locality degradation increases significantly in many of our benchmarks. For shared caches, this prevents most of our benchmarks from achieving constant-MPKI scaling within a 256 MB capacity budget; for private caches, all benchmarks cannot achieve constant-MPKI scaling within 256 MB.

The locality-aware adaptive cache coherence protocol

Next generation multicore applications will process massive amounts of data with significant sharing. Data movement and management impacts memory access latency and consumes power. Therefore, harnessing data locality is of fundamental importance in future processors. We propose a scalable, efficient shared memory cache coherence protocol that enables seamless adaptation between private and logically shared caching of on-chip data at the fine granularity of cache lines. Our data-centric approach relies on in-hardware yet low-overhead runtime profiling of the locality of each cache line and only allows private caching for data blocks with high spatio-temporal locality. This allows us to better exploit the private caches and enable low-latency, low-energy memory access, while retaining the convenience of shared memory. On a set of parallel benchmarks, our low-overhead locality-aware mechanisms reduce the overall energy by 25% and completion time by 15% in an NoC-based multicore with the Reactive-NUCA on-chip cache organization and the ACKwise limited directory-based coherence protocol.

CPN-Based Modeling and Analysis of Shared Multi-Channel Cache CMP Architecture

As a formal modeling and analysis method, colored petri nets (CPN) fits to construct formal model for software/hardware system with lots of communication and parallel/synchronous sharing behavior. Then behavior analysis of system function and performance based on the CPN model can be unfolded to verify the system feasibility. The CPN model of a kind of new CMP architecture shared multi-channel L2 Cache (AUMCC) is constructed in this article. Processing simulating of instruction set is deeply penetrated on the AUMCC CPN model, and analysis based on simulating results well verifies the feasibility of AUMCC architecture.

A Novel Cache Sharing Scheme and Implementation in Multi-Core Real-Time Linux System

Memory allocation and management of real-time task and sub real-time task have been becoming the hot problem that people concern all the time. Page coloring technology is a kind of practical method which can realize isolation of cache space. This article aimed at the problem of cache resource competition caused by shared memory on multi-core real-time system, to solve the collision of cache visit, put forward the scheme of cache space isolation based on page coloring technology and the realization method. The problem that data collide with the visit on cache can be finally avoided, by dividing the shared cache space.

WARP: Workload Nature Adaptive Replacement Policy

In the present universal scenario where the dependence on heterogeneous multi-core processors is tremendous, dealing with algorithms focusing on coherence between shared caches is imperative. WARP redesigns the replacement policy in the last level cache. In this policy, the shared (clean) lines and the private exclusive line are given the first two priorities dynamically followed by private modified and shared lines. By this method a set of victims are presented over which any replacement policy can be chosen to select a viable victim. Implementing the same, a significant performance improvement is observed with the last level shared cache. This performance improvement was solely from the policy implemented on the last level cache and not from any other parameters. General Terms Computer Architecture, High Performance Computing.

Multicore Real-Time Scheduling to Reduce Inter-Thread Cache Interferences

The worst-case execution time (WCET) of each real-time task in multicore processors with shared caches can be significantly affected by inter-thread cache interferences. The worst-case inter-thread cache interferences are dependent on how tasks are scheduled to run on different cores. Therefore, there is a circular dependence between real-time task scheduling, the worst-case inter-thread cache interferences, and WCET in multicore processors, which is not the case for single-core processors. To address this challenging problem, we present an offline real-time scheduling approach for multicore processors by considering the worst-case inter-thread interferences on shared L2 caches. Our scheduling approach uses a greedy heuristic to generate safe schedules while minimizing the worst-case inter-thread shared L2 cache interferences and WCET. The experimental results demonstrate that the proposed approach can reduce the utilization of the resulting schedule by about 12% on average compared to the cyclic multicore scheduling approaches in our theoretical model. Our evaluation indicates that the enhanced scheduling approach is more likely to generate feasible and safe schedules with stricter timing constraints in multicore real-time systems.

Critical Thread Guided Fine-grained Adaptive Capacity Management for Shared CMP Caches

Critical Thread Guided Fine-grained Adaptive Capacity Management for Shared CMP Caches

Efficient Reuse Distance Analysis of Multicore Scaling for Loop-Based Parallel Programs

Reuse Distance (RD) analysis is a powerful memory analysis tool that can potentially help architects study multicore processor scaling. One key obstacle, however, is that multicore RD analysis requires measuring Concurrent Reuse Distance (CRD) and Private-LRU-stack Reuse Distance (PRD) profiles across thread-interleaved memory reference streams. Sensitivity to memory interleaving makes CRD and PRD profiles architecture dependent, preventing them from analyzing different processor configurations. For loop-based parallel programs, CRD and PRD profiles shift coherently across RD values with core count scaling because interleaving threads are symmetric. Simple techniques can predict such shifting, making the analysis of numerous multicore configurations from a small set of CRD and PRD profiles feasible. Given the ubiquity of parallel loops, such techniques will be extremely valuable for studying future large multicore designs. This article investigates using RD analysis to efficiently analyze multicore cache performance for loop-based parallel programs, making several contributions. First, we provide an in-depth analysis on how CRD and PRD profiles change with core count scaling. Second, we develop techniques to predict CRD and PRD profile scaling, in particular employing reference groups [Zhong et al. 2003] to predict coherent shift, demonstrating 90% or greater prediction accuracy. Third, our CRD and PRD profile analyses define two application parameters with architectural implications: C core is the minimum shared cache capacity that “contains” locality degradation due to core count scaling, and C share is the capacity at which shared caches begin to provide a cache-miss reduction compared to private caches. And fourth, we apply CRD and PRD profiles to analyze multicore cache performance. When combined with existing problem scaling prediction, our techniques can predict shared LLC MPKI (private L2 cache MPKI) to within 10.7% (13.9%) of simulation across 1,728 (1,440) configurations using only 36 measured CRD (PRD) profiles.

IBM Blue Gene/Q memory subsystem with speculative execution and transactional memory

The memory subsystem of the IBM Blue Gene®/Q Compute chip features multi-versioning and access conflict detection. Its ordered and unordered transaction modes implement both speculative execution (SE) and transactional memory (TM). Blue Gene/Q's large shared second-level cache serves as storage for speculative versions, allowing up to 30 MB of speculative state for the 64 threads of a Blue Gene/Q node, which in the extreme can be associated with a single large transaction. Using the shared access to speculative data, the SE model implements forwarding, allowing data produced by one thread to be accessed by another thread while both are still speculative. This paper presents an overview of Blue Gene/Q's approach to TM and SE: the memory subsystem hardware and operating system extensions, IBM XL compiler support via OpenMP® extensions, and a cost estimation model for executing code speculatively. The model is validated using synthetic benchmarks.

Exploiting reuse locality on inclusive shared last-level caches

Optimization of the replacement policy used for Shared Last-Level Cache (SLLC) management in a Chip-MultiProcessor (CMP) is critical for avoiding off-chip accesses. Temporal locality, while being exploited by first levels of private cache memories, is only slightly exhibited by the stream of references arriving at the SLLC. Thus, traditional replacement algorithms based on recency are bad choices for governing SLLC replacement. Recent proposals involve SLLC replacement policies that attempt to exploit reuse either by segmenting the replacement list or improving the rereference interval prediction. On the other hand, inclusive SLLCs are commonplace in the CMP market, but the interaction between replacement policy and the enforcement of inclusion has barely been discussed. After analyzing that interaction, this article introduces two simple replacement policies exploiting reuse locality and targeting inclusive SLLCs: Least Recently Reused (LRR) and Not Recently Reused (NRR). NRR has the same implementation cost as NRU, and LRR only adds one bit per line to the LRU cost. After considering reuse locality and its interaction with the invalidations induced by inclusion, the proposals are evaluated by simulating multiprogrammed workloads in an 8-core system with two private cache levels and an SLLC. LRR outperforms LRU by 4.5% (performing better in 97 out of 100 mixes) and NRR outperforms NRU by 4.2% (performing better in 99 out of 100 mixes). We also show that our mechanisms outperform rereference interval prediction, a recently proposed SLLC replacement policy and that similar conclusions can be drawn by varying the associativity or the SLLC size.