LRU Replacement Policy Research Articles

BullsEye : Scalable and Accurate Approximation Framework for Cache Miss Calculation

For Affine Control Programs or Static Control Programs (SCoP), symbolic counting of reuse distances could induce polynomials for each reuse pair. These polynomials along with cache capacity constraints lead to non-affine (semi-algebraic) sets; and counting these sets is considered to be a hard problem. The state-of-the-art methods use various exact enumeration techniques relying on existing cardinality algorithms that can efficiently count affine sets. We propose BullsEye , a novel, scalable, accurate, and problem-size independent approximation framework. It is an analytical cache model for fully associative caches with LRU replacement policy focusing on sampling and linearization of non-affine stack distance polynomials. First, we propose a simple domain sampling method that can improve the scalability of exact enumeration. Second, we propose linearization techniques relying on Handelman’s theorem and Bernstein’s representation . To improve the scalability of the Handelman’s theorem linearization technique, we propose template (Interval or Octagon) sub-polyhedral approximations. Our methods obtain significant compile-time improvements with high-accuracy when compared to HayStack on important polyhedral compilation kernels such as nussinov , cholesky , and adi from PolyBench , and harris , gaussianblur from LLVM -TestSuite. Overall, on PolyBench kernels, our methods show up to 3.31× (geomean) speedup with errors below ≈ 0.08% (geomean) for the octagon sub-polyhedral approximation.

Fast modeling L2 cache reuse distance histograms using combined locality information from software traces

Abstract To mitigate the performance gap between CPU and the main memory, multi-level cache architectures are widely used in modern processors. Therefore, modeling the behaviors of the downstream caches becomes a critical part of the processor performance evaluation in the early stage of Design Space Exploration (DSE). In this paper, we propose a fast and accurate L2 cache reuse distance histogram model, which can be used to predict the behaviors of the multi-level cache architectures where the L1 cache uses the LRU replacement policy and the L2 cache uses LRU/Random replacement policies. We use the profiled L1 reuse distance histogram and two newly proposed metrics, namely the RST table and the Hit-RDH, that describing more detailed information of the software traces as the inputs. For a given L1 cache configuration, the profiling results can be reused for different configurations of the L2 cache. The output of our model is the L2 cache reuse distance histogram, based on which the L2 cache miss rates can be evaluated. We compare the L2 cache miss rates with the results from gem5 cycle-accurate simulations of 15 benchmarks chosen from SPEC CPU 2006 and 9 benchmarks from SPEC CPU 2017. The average absolute error is less than 5%, while the evaluation time for each L2 configuration can be sped up almost 30× for four L2 cache candidates.

Per-Operation Reusability Based Allocation and Migration Policy for Hybrid Cache

Recently, a hybrid cache consisting of SRAM and STT-RAM has attracted much attention as a future memory by complementing each other with different memory characteristics. Prior works focused on developing data allocation and migration techniques considering write-intensity to reduce write energy at STT-RAM. However, these works often neglect the impact of operation-specific reusability of a cache line. In this paper, we propose an energy-efficient per-operation reusability-based allocation and migration policy (ORAM) with a unified LRU replacement policy. First, to select an adequate memory type for allocation, we propose a cost function based on per-operation reusability – gain from an allocated cache line and loss from an evicted cache line for different memory types – which exploits the temporal locality. Besides, we present a migration policy, victim and target cache line selection scheme, to resolve memory type inconsistency between replacement policy and the allocation policy, with further energy reduction. Experiment results show an average energy reduction in the LLC and the main memory by 12.3 and 21.2 percent, and the improvement of latency and execution time by 21.2 and 8.8 percent, respectively, compared with a baseline hybrid cache management. In addition, the Energy-Delay Product (EDP) is improved by 36.9 percent over the baseline.

Writeback-Aware LLC Management for PCM-Based Main Memory Systems

With the increase in the number of data-intensive applications on today's workloads, DRAM-based main memories are struggling to satisfy the growing data demand capacity. Phase Change Memory (PCM) is a type of non-volatile memory technology that has been explored as a promising alternative for DRAM-based main memories due to its better scalability and lower leakage energy. Despite its many advantages, PCM also has shortcomings such as long write latency, high write energy consumption, and limited write endurance, which are all related to the write operations. In this article, we propose a novel writeback-aware Last Level Cache (LLC) management scheme named WALL to reduce the number of LLC writebacks and consequently improve performance, energy efficiency, and lifetime of a PCM-based main memory system. First, we investigate the writeback behavior of LLC sets and show that writebacks are not uniformly distributed among sets; some sets observe much higher writeback rates than others. We then propose a writeback-aware set-balancing mechanism, which employs the underutilized LLC sets with few writebacks as an auxiliary storage for the evicted dirty lines from sets with frequent writebacks. We also propose a simple and effective writeback-aware replacement policy to avoid the eviction of the dirty blocks that are highly reused after being evicted from the cache. Our experimental results show that WALL achieves an average of 30.9% reduction in the total number of LLC writebacks, compared to the baseline scheme, which uses the LRU replacement policy. As a result, WALL can reduce the memory energy consumption by 23.1% and enhance PCM lifetime by 1.29×, on average, on an 8-core system with a 4GB PCM main memory, running memory-intensive applications.

龙芯GS464E处理器核架构设计

Loongson GS464E is the most recent processor architecture introduced by Loongson Technology. In this paper, we describe the key aspects of the GS464E architecture. Compared to the previous GS464 architecture, GS464E focuses on improving the performance of memory access and branch prediction, using enlarged queues, caches, and TLBs in the processor. Support for the MIPS DSP instruction set and virtualization is also provided. Specifically, the memory subsystems have a 3-level cache hierarchy each with an LRU replacement policy, and also support multi-processor cache coherence. With the aforementioned optimization features, GS464E has become an innovative, high-performance processor architecture.

Learning your limit

The gap between processor and memory performance has become a focal point for microprocessor research and development over the past three decades. Modern architectures use two orthogonal approaches to help alleviate this issue: (1) Almost every microprocessor includes some form of on-chip storage, usually in the form of caches, to decrease memory latency and make more effective use of limited memory bandwidth. (2) Massively multithreaded architectures, such as graphics processing units (GPUs), attempt to hide the high latency to memory by rapidly switching between many threads directly in hardware. This paper explores the intersection of these two techniques. We study the effect of accelerating highly parallel workloads with significant locality on a massively multithreaded GPU. We observe that the memory access stream seen by on-chip caches is the direct result of decisions made by the hardware thread scheduler. Our work proposes a hardware scheduling technique that reacts to feedback from the memory system to create a more cache-friendly access stream. We evaluate our technique using simulations and show a significant performance improvement over previously proposed scheduling mechanisms. We demonstrate the effectiveness of scheduling as a cache management technique by comparing cache hit rate using our scheduler and an LRU replacement policy against other scheduling techniques using an optimal cache replacement policy.

HIT RATE MAXIMIZATION BY LOGICAL CACHE PARTITIONING IN A MULTI-CORE ENVIRONMENT

It is imperative for any level of cache memory in a multi-core architecture to have a well defined, dynamic replacement algorithm in place to ensure consistent superlative performance. The most prevalently used LRU replacement policy does not acquaint itself dynamically to the changes in the workload. As a result, it can lead to sub-optimal performance for certain applications whose workloads exhibit frequently fluctuating patterns. To overcome the limitation of this conventional LRU approach, our paper proposes a novel counter-based replacement technique which logically partitions the cache elements into four zones based on their ‘likeliness’ to be referenced by the processor in the near future. Categorizing the elements into different zones is achieved with the help of a 3-bit counter that is associated with every cache line. On a cache hit, the corresponding element is promoted from one zone to another zone. Replacement candidates are chosen from the zones in the ascending order of their ‘likeliness factor’ (i.e.,) the first search space for the victim would be the never likely to be referenced zone, followed by the subsequent zones till the most likely to be referenced zone is reached. Periodic zone demotion of elements also occurs to make sure that stale data does not pollute the cache. Experimental results obtained by using the PARSEC benchmarks have shown almost 7% improvement in the overall number of hits and 3% improvement in the average cache occupancy percentage when compared to LRU algorithm.

ARI

Decreasing the traffic from the CPU LLC to main memory is a very important issue in modern systems. Recent work focuses on cache misses, overlooking the impact of writebacks on the total memory traffic, energy consumption, IPC, and so forth. Policies that foster a balanced approach, between reducing write traffic to memory and improving miss rates, can increase overall performance and improve energy efficiency and memory system lifetime for NVM memory technology, such as phase-change memory (PCM). We propose Adaptive Replacement and Insertion (ARI), an adaptive approach to last-level CPU cache management, optimizing the two parameters (miss rate and writeback rate) simultaneously. Our specific focus is to reduce writebacks as much as possible while maintaining or improving the miss rate relative to conventional LRU replacement policy. ARI reduces LLC writebacks by 33%, on average, while also decreasing misses by 4.7%, on average. In a typical system, this boosts IPC by 4.9%, on average, while decreasing energy consumption by 8.9%. These results are achieved with minimal hardware overheads.

LvtPPP: Live-Time Protected Pseudopartitioning of Multicore Shared Caches

Partition enforcement policy is essential in the cache partition, and its main function is to protect the lines and retain the cache quota of each core. This paper focuses online protection based on its generation time rather than the CPU core ID that it belongs to or the position of the replacement stack, where it is located. The basic idea is that when a line is live, it must be protected and retained in the cache; when the line is “dead,” it needs to be evicted as early as possible. Therefore, the live-time protected counter (LvtP, four bits) is augmented to trace the lines' live time. Moreover, dead blocks are predicted according to the access event sequence. This paper presents a pseudopartition approach-LvtPPP and proposes a two-cascade victim selection mechanism to alleviate dead blocks based on the LRU replacement policy and the LvtP counter. LvtPPP also supports flexible handling of allocation deviation by introducing a parameter λ to adjust the generation time of the line. There is significant improvement of the performance and fairness in LvtPPP over PIPP and UCP according to the evaluation results based on Simics.

BitTorrent traffic from a caching perspective

Abstract BitTorrent currently contributes a significant amount of inter-ISP traffic. This has motivated research and development to explore caching and locality-aware neighbor selection mechanisms for costly traffic reduction. Recent researches have analyzed the possible effects of caching BitTorrent traffic and have provided preliminary results on its cacheability. However, little is known about the specifics of caching design that affect cache effectiveness and operation, such as replacement policy and cache size. This study addresses this gap with a comprehensive analysis of BitTorrent caching based on traces of user behavior in four popular BitTorrent sites. Our trace-driven simulation results show differences in BitTorrent traffic caching compared to that of the Web and other peer-to-peer applications. Differently from Web and other peer-to-peer caching, larger caches are necessary to achieve similar caching effectiveness in BitTorrent traffic. Furthermore, in BitTorrent caching, the LRU replacement policy that takes the temporal locality into account shows the best performance. We also use a locality-aware neighbor selection mechanism as a baseline to evaluate the LRU caching effectiveness. We find that LRU caching can provide greater traffic reduction than locality-aware neighbor selection in several scenarios of cache size and number of ISP clients.

고성능 프로세서를 위한 카운터 기반의 캐시 교체 알고리즘

캐시 메모리의 성능에 큰 영향을 미치는 요소 중 하나인 캐시 교체 알고리즘 중에서 가장 최적의 성능을 가지는 알고리즘은 LRU알고리즘이다. LRU알고리즘은 데이터의 temporal locality특성이 강한 프로그램에서 좋은 성능을 보여주지만, 그렇지 않은 프로그램에서는 많은 캐시 미스를 발생시킨다. 본 논문에서는 LRU알고리즘의 이러한 단점을 개선하기 위한 새로운 카운터 기반 교체 알고리즘인 DCR(Dynamic Counter based Replacement) 알고리즘을 제안한다. 본 논문에서는 캐시에 저장된 이후로 교체되기 전까지 다시 사용되지 않는 데이터인 zero reuse line의 발생 추이를 관찰함으로써 프로그램의 temporal locality 특성이 시간에 따라 동적으로 변화함을 보였다. 그리고 이에 착안하여 제안하는 DCR 알고리즘은 주기적으로 zero reuse line 의 수를 카운트하여 프로그램의 temporal locality 변화에 대응할 수 있도록 하였다. DCR 알고리즘은 기존의 LRU알고리즘과 비교하여 최대 2.7%, 평균 0.47%의 미스율 감소를 보였다.

Adaptive Capacity Sharing through Probabilistic Controlled Placement

As capacity demands vary among simultaneously executed threads in chip multiprocessors, dynamically managing cache resources according to the run-time demands is effective to improve L2 cache performance. Differed from existing dynamic cache management schemes based on LRU replacement policy, we propose an adaptive capacity sharing mechanism based on a global reuse replacement policy. This mechanism adopts decoupled tag and data arrays, and partitions the data arrays into private and shared regions. Capacity sharing is accomplished by determining whether to place the incoming data into the private data region or into the shared data region, which is controlled by probabilities. Our mechanism includes: (1) A VMON monitor to predict run-time capacity demands. (2) A PCS algorithm to determine the probabilities. (3) A probabilistic controlled placement scheme to enforce capacity sharing. We evaluated our mechanism with a full system simulation of an 8-core CMP and used parallel programs from PARSEC benchmark suite. We found that with the same total L2 cache capacity, our mechanism exceeds the conventional private cache managed by LRU policy, the private cache without sharing managed by reuse replacement policy, and an existing adaptive sharing scheme based on LRU policy.

A Buffer Cache Management for Locality with Prefetching

paper proposes buffer cache architecture. The proposed model consists of three units - main buffer cache unit, pre-fetch unit and LRU Evict Unit. An algorithm is proposed to retain the evicted entry from main buffer cache unit in the LRU Evict Unit thereby giving it a second chance of access. On subsequent access in the LRU Evict Unit, the entry is promoted to the pre- fetch unit to accommodate more entries in the main buffer cache unit. On access in the pre-fetch unit, an entry is fetched into the main buffer cache. The LRU replacement policy is used in all the units. The proposed model is compared with buffer cache architecture with LRU replacement algorithm. The performance is comparable for sequential input where as 3% improvement in performance is seen for random input.

A Novel Cache Replacement Policy via Dynamic Adaptive Insertion and Re-Reference Prediction

Previous research illustrates that LRU replacement policy is not efficient when applications exhibit a distant re-reference interval. Recently RRIP policy is proposed to improve the performance for such kind of workloads. However, the lack of access recency information in RRIP confuses the replacement policy to make the accurate prediction. To enhance the robustness of RRIP for recency-friendly workloads, we propose an Dynamic Adaptive Insertion and Re-reference Prediction (DAI-RRP) policy which evicts data based on both re-reference prediction value and the access recency information. DAI-RRP makes adaptive adjustment on insertion position and prediction value for different access patterns, which makes the policy robust across different workloads and different phases. Simulation results show that DAI-RRP outperforms LRU and RRIP. For a single-core processor with a 1MB 16-way set last-level cache (LLC), DAI-RRP reduces CPI over LRU and Dynamic RRIP by an average of 8.1% and 2.7% respectively. Evaluations on quad-core CMP with a 4MB shared LLC show that DAI-RRP outperforms LRU and Dynamic RRIP (DRRIP) on the weighted speedup metric by an average of 8.1% and 15.7% respectively. Furthermore, compared to LRU, DAI-RRP consumes the similar hardware for 16-way cache, or even less hardware for high-associativity cache. In summary, the proposed policy is practical and can be easily integrated into existing hardware approximations of LRU.

Two Cooperative Partitions for the Lase Level Cache

Currently the most widely used replacement policy in the last cache is the LRU algorithm. Popular as it is, LRU performs poorly when it has a memory-intensive workload whose working set is larger than the available cache capacity. The algorithm evicts a line without taking into account its longer history of usage. The result is that some live lines may be replaced by some newly-referenced blocks, though it is possible that the new blocks will never be used again in the future.Hence, improving the replacement policy is critical to the improvement of cache efficiency and the overall system performance. In this paper we present a technique to retain the live line in the cache while evicting the dead line early by dividing the last level cache into two cooperative partitions (TCP), both of which use the LRU replacement policy. When a live line is evicted from one partition, it will be placed into the other partition. This new technique named TCP, as observed in our experiment, achieved an L2 cache miss rate reduction of 9.8% on average with fifteen benchmarks from the SPEC CPU2000 suite, and resulted in an IPC improvement of 2.4% on average.

High performance cache replacement using re-reference interval prediction (RRIP)

Practical cache replacement policies attempt to emulate optimal replacement by predicting the re-reference interval of a cache block. The commonly used LRU replacement policy always predicts a near-immediate re-reference interval on cache hits and misses. Applications that exhibit a distant re-reference interval perform badly under LRU. Such applications usually have a working-set larger than the cache or have frequent bursts of references to non-temporal data (called scans). To improve the performance of such workloads, this paper proposes cache replacement using Re-reference Interval Prediction (RRIP). We propose Static RRIP (SRRIP) that is scan-resistant and Dynamic RRIP (DRRIP) that is both scan-resistant and thrash-resistant. Both RRIP policies require only 2-bits per cache block and easily integrate into existing LRU approximations found in modern processors. Our evaluations using PC games, multimedia, server and SPEC CPU2006 workloads on a single-core processor with a 2MB last-level cache (LLC) show that both SRRIP and DRRIP outperform LRU replacement on the throughput metric by an average of 4% and 10% respectively. Our evaluations with over 1000 multi-programmed workloads on a 4-core CMP with an 8MB shared LLC show that SRRIP and DRRIP outperform LRU replacement on the throughput metric by an average of 7% and 9% respectively. We also show that RRIP outperforms LFU, the state-of the art scan-resistant replacement algorithm to-date. For the cache configurations under study, RRIP requires 2X less hardware than LRU and 2.5X less hardware than LFU.

Design of a DI model-based Content Addressable Memory for Asynchronous Cache

This paper presents a novel approach in the design of a CAM for an asynchronous cache. The architecture of cache mainly consists of four units: control logics, content addressable memory, completion signal logic units and instruction memory. The pseudo-DCVSL is useful to make a completion signal which is a reference for handshake control. The proposed CAM is a very simple extension of the basic circuitry that makes a completion signal based on DI model. The cache has 2.75KB CAM for 8KB instruction memory. We designed and simulated the proposed asynchronous cache including CAM. The results show that the cache hit ratio is up to 95% based on pseudo ?LRU replacement policy.

Set-Dueling-Controlled Adaptive Insertion for High-Performance Caching

The commonly used LRU replacement policy causes thrashing for memory- intensive workloads. A simple mechanism that dynamically changes the insertion policy used by LRU replacement reduces cache misses by 21 percent and requires a total storage overhead of less than 2 bytes.

Adaptive insertion policies for high performance caching

The commonly used LRU replacement policy is susceptible to thrashing for memory-intensive workloads that have a working set greater than the available cache size. For such applications, the majority of lines traverse from the MRU position to the LRU position without receiving any cache hits, resulting in inefficient use of cache space. Cache performance can be improved if some fraction of the working set is retained in the cache so that at least that fraction of the working set can contribute to cache hits. We show that simple changes to the insertion policy can significantly reduce cache misses for memory-intensive workloads. We propose the LRU Insertion Policy (LIP) which places the incoming line in the LRU position instead of the MRU position. LIP protects the cache from thrashing and results in close to optimal hitrate for applications that have a cyclic reference pattern. We also propose the Bimodal Insertion Policy (BIP) as an enhancement of LIP that adapts to changes in the working set while maintaining the thrashing protection of LIP. We finally propose a Dynamic Insertion Policy (DIP) to choose between BIP and the traditional LRU policy depending on which policy incurs fewer misses. The proposed insertion policies do not require any change to the existing cache structure, are trivial to implement, and have a storage requirement of less than two bytes. We show that DIP reduces the average MPKI of the baseline 1MB 16-way L2 cache by 21%, bridging two-thirds of the gap between LRU and OPT.

Hierarchical Binary Set Partitioning in Cache Memories

In this paper, a new cache placement scheme is proposed to achieve higher hit ratios with respect to the two conventional schemes namely set-associative and direct mapping. Similar to set-associative, in this scheme, cache space is divided into sets of different sizes. Hence, the length of tag fields associated to each set is also variable and depends on the partition it is in. The proposed mapping function has been simulated with some standard trace files and statistics are gathered and analyzed for different cache configurations. The results reveal that the proposed scheme exhibits a higher hit ratio compared to the two well-known mapping schemes, namely set-associative and direct mapping, using LRU replacement policy.