Cache Management Policies Research Articles

Finding optimal non-datapath caching strategies via network flow

Flash and non-volatile memory (NVM) devices have only a limited number of write-erase cycles. Consequently, when employed as caches, cache management policies may choose not to cache certain requested items in order to extend device lifespan. In this work, we propose a simple single-parameter utility function to model the trade-off between maximizing hit-rate and minimizing write-erase cycles for such caches, and study the problem of developing an off-line strategy for deciding whether to write a new item to cache, and if so which item already in the cache to replace. Our main result is, OPTm, an efficient, network flow based algorithm which finds optimal cache management policy under this new setting.

CASHT: Contention Analysis in Shared Hierarchies with Thefts

Cache management policies should consider workloads’ contention behavior when managing a shared cache. Prior art makes estimates about shared cache behavior by adding extra logic or time to isolate per workload cache statistics. These approaches provide per-workload analysis but do not provide a holistic understanding of the utilization and effectiveness of caches under the ever-growing contention that comes standard with scaling cores. We present Contention Analysis in Shared Hierarchies using Thefts, or CASHT, 1 a framework for capturing cache contention information both offline and online. CASHT takes advantage of cache statistics made richer by observing a consequence of cache contention: inter-core evictions, or what we call THEFTS. We use thefts to complement more familiar cache statistics to train a learning model based on Gradient-boosting Trees (GBT) to predict the best ways to partition the last-level cache. GBT achieves 90+% accuracy with trained models as small as 100 B and at least 95% accuracy at 1 kB model size when predicting the best way to partition two workloads. CASHT employs a novel run-time framework for collecting thefts-based metrics despite partition intervention, and enables per-access sampling rather than set sampling that could add overhead but may not capture true workload behavior. Coupling CASHT and GBT for use as a dynamic policy results in a very lightweight and dynamic partitioning scheme that performs within a margin of error of Utility-based Cache Partitioning at a 1/8 the overhead.

SSD internal cache management policies: A survey

SSDs are now increasingly being adopted as primary nonvolatile storage in diverse domains. The internal caches in the SSDs boost SSD performance, lifetime, and robustness. There are three types of internal caches: data cache and mapping cache in the SSD front-end, and flash-based cache in the SSD back-end; they are designed to meet specific requirements like controlling traffic to the back-end, optimization of response time, improving SSD lifetime etc. The underlying memory technology also influences cache design decisions. This work aims to present a methodological survey of cache management policies for these three types of internal caches in SSD. In particular, our work (a) makes a taxonomical classification of cache management policies for each of the cache types and performs a comparative analysis of the research works in each micro-category, (b) derives a set of guidelines that can be helpful for a future cache designer, and (c) enumerates a number of future research directions for designing an optimal SSD internal cache management policy. Our work is distinct from other studies in that we analyze policies for all three types of SSD internal caches, whereas this has not been done in the prior surveys.

Performance Improvement of DAG-Aware Task Scheduling Algorithms with Efficient Cache Management in Spark

Directed acyclic graph (DAG)-aware task scheduling algorithms have been studied extensively in recent years, and these algorithms have achieved significant performance improvements in data-parallel analytic platforms. However, current DAG-aware task scheduling algorithms, among which HEFT and GRAPHENE are notable, pay little attention to the cache management policy, which plays a vital role in in-memory data-parallel systems such as Spark. Cache management policies that are designed for Spark exhibit poor performance in DAG-aware task-scheduling algorithms, which leads to cache misses and performance degradation. In this study, we propose a new cache management policy known as Long-Running Stage Set First (LSF), which makes full use of the task dependencies to optimize the cache management performance in DAG-aware scheduling algorithms. LSF calculates the caching and prefetching priorities of resilient distributed datasets according to their unprocessed workloads and significance in parallel scheduling, which are key factors in DAG-aware scheduling algorithms. Moreover, we present a cache-aware task scheduling algorithm based on LSF to reduce the resource fragmentation in computing. Experiments demonstrate that, compared to DAG-aware scheduling algorithms with LRU and MRD, the same algorithms with LSF improve the JCT by up to 42% and 30%, respectively. The proposed cache-aware scheduling algorithm also exhibits about 12% reduction in the average job completion time compared to GRAPHENE with LSF.

Efficient streaming subgraph isomorphism with graph neural networks

Queries to detect isomorphic subgraphs are important in graph-based data management. While the problem of subgraph isomorphism search has received considerable attention for the static setting of a single query, or a batch thereof, existing approaches do not scale to a dynamic setting of a continuous stream of queries. In this paper, we address the scalability challenges induced by a stream of subgraph isomorphism queries by caching and re-use of previous results. We first present a novel subgraph index based on graph embeddings that serves as the foundation for efficient stream processing. It enables not only effective caching and re-use of results, but also speeds-up traditional algorithms for subgraph isomorphism in case of cache misses. Moreover, we propose cache management policies that incorporate notions of reusability of query results. Experiments using real-world datasets demonstrate the effectiveness of our approach in handling isomorphic subgraph search for streams of queries.

Design of the IBM z15 microprocessor

The latest-generation IBM Z processor provides enhanced performance and compute capacity compared to its IBM z14 predecessor. This article describes some of the major improvements in both process and design including out-of-order load-and-store sequencing, single-instruction multiple-data and floating point enhancements, a new modulo arithmetic engine for accelerating elliptic curve cryptography, a hardware sort accelerator, and a workflow that modernized the development of these features. Outside of the central processing unit (CPU), the cache sizes have increased on all levels, and each processor chip now contains 12 CPUs. System topology changes have been introduced allowing up to five drawers to exist in a fully populated system. The processor cache subsystem includes numerous improvements in the area of fetch, store, and cache management policies aimed at speeding up both traditional data serving workloads and highly virtualized environments alike.

Addressing variability in reuse prediction for last-level caches

Last-Level Cache (LLC) represents the bulk of a modern CPU processor's transistor budget and is essential for application performance as LLC enables fast access to data in contrast to much slower main memory. However, applications with large working set size often exhibit streaming and/or thrashing access patterns at LLC. As a result, a large fraction of the LLC capacity is occupied by dead blocks that will not be referenced again, leading to inefficient utilization of the LLC capacity. To improve cache efficiency, the state-of-the-art cache management techniques employ prediction mechanisms that learn from the past access patterns with an aim to accurately identify as many dead blocks as possible. Once identified, dead blocks are evicted from LLC to make space for potentially high reuse cache blocks. In this thesis, we identify variability in the reuse behavior of cache blocks as the key limiting factor in maximizing cache efficiency for state-of-the-art predictive techniques. Variability in reuse prediction is inevitable due to numerous factors that are outside the control of LLC. The sources of variability include control-flow variation, speculative execution and contention from cores sharing the cache, among others. Variability in reuse prediction challenges existing techniques in reliably identifying the end of a block's useful lifetime, thus causing lower prediction accuracy, coverage, or both. To address this challenge, this thesis aims to design robust cache management mechanisms and policies for LLC in the face of variability in reuse prediction to minimize cache misses, while keeping the cost and complexity of the hardware implementation low. To that end, we propose two cache management techniques, one domain-agnostic and one domain-specialized, to improve cache efficiency by addressing variability in reuse prediction.

Selective bypassing and mapping for heterogeneous applications on GPGPUs

Modern GPGPU supports executing multiple tasks with different run time characteristics and resource utilization. Having an efficient execution and resource management policy has been shown to be a critical performance factor when handling the concurrent execution of tasks with different run time behavior. Previous policies either assign equal resources to disparate tasks or allocate resources based on static or standalone behavior profiling. Treating tasks equally cannot efficiently utilize the system resources, while the standalone profiling ignores the correlated impact when running tasks concurrently and could hint incorrect task behavior. This paper addresses the above drawbacks and proposes a heterogeneity aware Selective Bypassing and Mapping (SBM) to manage both computing and cache resources for multiple tasks in a fine-grain manner. The light-weight run time profiling of SBM properly characterizes the disparate behavior of the concurrently executed multiple tasks, and selectively applies suited cache management and workgroup mapping policies to each task. When compared with the previous coarse-grained policies, SBM can achieve an average of 138% and up to 895% performance enhancement. When compared with the state-of-art fine-grained policy, SBM can achieve an average of 58% and up to 378% performance enhancement.

Reuse Distance-based Victim Cache for Effective Utilisation of Hybrid Main Memory System

Hybrid main memories comprising DRAM and Non-volatile memories (NVM) are projected as potential replacements of the traditional DRAM-based memories. However, traditional cache management policies designed for improving the hit rate lack awareness of the comparative latency of read-write for NVM blocks where the write latency is more than the read latency. Therefore, developing cache management techniques that reduce costly write-backs of the NVM blocks, yet maintain a fair hit rate in the cache, is of paramount importance. We propose two techniques based on the use of a small victim cache associated with the last-level cache that helps in retaining on the chip critical DRAM and NVM blocks. Victim cache being a scarce resource, we intend to keep only performance-critical blocks in the victim cache by exploiting the idea of reuse distance. The first technique, Victim Cache Replacement Policy, works on the replacement policy of the victim cache by preferential eviction of DRAM blocks over NVM blocks. However, the second technique, Prioritized Partitioning of victim cache, logically partitions the victim cache, giving a smaller share to the DRAM blocks and a relatively larger share to the NVM blocks. Experimental evaluation on full-system simulator shows significant improvement in system performance and reduction in the number of write-backs to the NVM partition of the main memory compared to the baseline and existing technique. Additionally, NVM reads and DRAM miss rate are also improved, leading to further performance enhancement.

Integrated Cache Scheduling Replacement Algorithm to Reduce Cache Pollution

Cache memory management has recently become one of the most crucial research topics in the area of high performance computing. The cache memory has gained popularity due to its capability to offer faster access compared to main memory so processor can access data directly through cache very efficiently. As processor execution speed is faster than accessing memory so the goal is to keep programs rapidly accessible for processing. Cache memory helps not only to access data quickly but also improves the performance of the system. Various cache management policies have been developed to improve the performance of the system. In this paper we focus the issues related with the existing replacement policies and present a new algorithm to overcome these issues. In this paper, we proposed Integrated Cache Scheduling Replacement Algorithm (ICSRA) that will remove the object on the basis of integration of all the parameters like size, frequency and recency if the cache is full. This algorithm consider all size of object to replace with the condition of frequency and timestamp parameter to reduce the cache pollution [1]. Cache pollution means the cache unnecessary waste its space by holding such an object which is no longer be accessed. ICSRA helps to enhance the utilization of cache memory in an efficient manner as well as maximize the cache hit ratio to improve the performance of the system. A detailed classification has also been included based on different parameters which are depending on the analysis of the existing techniques.

ECR: Eviction‐cost‐aware cache management policy for page‐level flash‐based SSDs

SummaryCache management policy plays a key role in offering low latency access to flash‐based SSDs. Most existing solutions including LRU and its successors only focus on improving the cache hit ratio, but rarely consider to reduce the waiting time of the eviction operation in the page‐level mapping FTLs. As the workloads spreading across internal chips of modern flash‐based SSDs are often highly imbalanced when workloads are write‐intensive, the time cost of evicting a dirty page from cache varies in a wide range. In this paper, we propose a novel eviction‐cost‐aware cache management policy, called ECR, to minimize the eviction cost in write‐dominant applications. ECR gives a higher probability to evict a page, which causes the shortest waiting time in the corresponding chip queue. To achieve this, we introduce a monitor module to keep track of states of all chip queues, and a multi‐LRU list structure to accelerate the selection of a victim chip and a target page in cache to perform an eviction. Our experimental results show that ECR can significantly reduce the average response time by as much as 59.55% and 44.84% compared to LRU and GCaR‐CFLRU, respectively, where GCaR‐CFLRU is the combination of state‐of‐the‐art algorithm GCaR and CFLRU.

A Maximum Cache Value Policy in Hybrid Memory-Based Edge Computing for Mobile Devices

Edge computing is proposed to bridge mobile devices with cloud computing data centers in the era of mobile big data, as an intermediate level of computing power. One important issue in edge computing is how to improve performance for mobile devices. Current systems utilize cache in multicore systems to reduce memory access cost with an acceptable hardware cost. However, existing cache management policies are unable to maximize cache value in the newly developed hybrid memory platform that combines phase-change memory and dynamic random-access memory. In this paper, we propose maximizes cache value (MCV), an efficient cache management policy, which MCV to minimize memory access cost in a hybrid main memory platform for edge computing. Extensive simulation studies indicate that this strategy can improve performance in hybrid main memory-based edge computing for mobile devices.

Reducing Writebacks Through In-Cache Displacement

Non-Volatile Memory (NVM) technology is a promising solution to fulfill the ever-growing need for higher capacity in the main memory of modern systems. Despite having many great features, however, NVM’s poor write performance remains a severe obstacle, preventing it from being used as a DRAM alternative in the main memory. Most of the prior work targeted optimizing writes at the main memory side and neglected the decisive role of upper-level cache management policies on reducing the number of writes. In this article, we propose a novel cache management policy that attempts to maximize write-coalescing in the on-chip SRAM last-level cache (LLC) for the sake of reducing the number of costly writes to the off-chip NVM. We decouple a few physical ways of the LLC to have a dedicated and exclusive storage for the dirty blocks after being evicted from the cache and before being sent to the off-chip memory. By displacing dirty blocks in exclusive storage, they are kept in the cache based on their rewrite distance and are evicted when they are unlikely to be reused shortly. To maximize the effectiveness of exclusive storage, we manage it as a Cuckoo Cache to offer associativity based on the various applications’ demands. Through detailed evaluations targeting various single- and multi-threaded applications, we show that our proposal reduces the number of writebacks by 21%, on average, over the state-of-the-art method and enhances both performance and energy efficiency.

A Novel Adaptive Database Cache Optimization Algorithm Based on Predictive Working Sets in Cloud Environment

Many applications are hosted on cloud databases where several applications share the same database instance. Such database management system exhibits periodic behavior in terms of data references. For example, U.S. customers access data at a particular time while Japanese customers access data at some other time. The periodicity of data references is translated into periodic block references. This periodicity of block references can be used to improve cache performance by improving the block replacement policy. This paper proposes a novel cache replacement policy by considering periodicity of references of database blocks. It also proposes how to estimate the probability of a block reference in a specified time interval using “Parzen Windows,” and determines a working set based on this probability and defines the cache management policy using this working set. The experimentation shows considerable improvement of the hit ratio as the performance measure of the buffer cache management as compared with the other state-of-art algorithms.

ReD: A reuse detector for content selection in exclusive shared last-level caches

The reference stream reaching a chip multiprocessor Shared Last-Level Cache (SLLC) shows poor temporal locality, making conventional cache management policies inefficient. Few proposals address this problem for exclusive caches. In this paper, we propose the Reuse Detector (ReD), a new content selection mechanism for exclusive hierarchies that leverages reuse locality at the SLLC, a property that states that blocks referenced more than once are more likely to be accessed in the near future. Being placed between each L2 private cache and the SLLC, ReD prevents the insertion of blocks without reuse into the SLLC. It is designed to overcome problems affecting similar recent mechanisms (low accuracy, reduced visibility window and detector thrashing). ReD improves performance over other state-of-the-art proposals (CHAR, Reuse Cache and EAF cache). Compared with the baseline system with no content selection, it reduces the SLLC miss rate (MPI) by 10.1% and increases harmonic IPC by 9.5%.

A fault-tolerant last level cache for CMPs operating at ultra-low voltage

Voltage scaling to values near the threshold voltage is a promising technique to hold off the many-core power wall. However, as voltage decreases, some SRAM cells are unable to operate reliably and show a behavior consistent with a hard fault. Block disabling is a micro-architectural technique that allows low-voltage operation by deactivating faulty cache entries, at the expense of reducing the effective cache capacity. In the case of the last-level cache, this capacity reduction leads to an increase in off-chip memory accesses, diminishing the overall energy benefit of reducing the voltage supply. In this work, we exploit the reuse locality and the intrinsic redundancy of multi-level inclusive hierarchies to enhance the performance of block disabling with negligible cost. The proposed fault-aware last-level cache management policy maps critical blocks, those not present in private caches and with a higher probability of being reused, to active cache entries. Our evaluation shows that this fault-aware management results in up to 37.3% and 54.2% fewer misses per kilo instruction (MPKI) than block disabling for multiprogrammed and parallel workloads, respectively. This translates to performance enhancements of up to 13% and 34.6% for multiprogrammed and parallel workloads, respectively.

A Multilevel Cache Management Policy for Performance Improvement in Distributed System

A Multilevel Cache Management Policy for Performance Improvement in Distributed System

Shared Last-Level Cache Management and Memory Scheduling for GPGPUs with Hybrid Main Memory

Memory intensive workloads become increasingly popular on general purpose graphics processing units (GPGPUs), and impose great challenges on the GPGPU memory subsystem design. On the other hand, with the recent development of non-volatile memory (NVM) technologies, hybrid memory combining both DRAM and NVM achieves high performance, low power, and high density simultaneously, which provides a promising main memory design for GPGPUs. In this article, we explore the shared last-level cache management for GPGPUs with consideration of the underlying hybrid main memory. To improve the overall memory subsystem performance, we exploit the characteristics of both the asymmetric read/write latency of the hybrid main memory architecture, as well as the memory coalescing feature of GPGPUs. In particular, to reduce the average cost of L2 cache misses, we prioritize cache blocks from DRAM or NVM based on observations that operations to NVM part of main memory have a large impact on the system performance. Furthermore, the cache management scheme also integrates the GPU memory coalescing and cache bypassing techniques to improve the overall system performance. To minimize the impact of memory divergence behaviors among simultaneously executed groups of threads, we propose a hybrid main memory and warp aware memory scheduling mechanism for GPGPUs. Experimental results show that in the context of a hybrid main memory system, our proposed L2 cache management policy and memory scheduling mechanism improve performance by 15.69% on average for memory intensive benchmarks, whereas the maximum gain can be up to 29% and achieve an average memory subsystem energy reduction of 21.27%.

RT-CaCC: A Reliable Transport With Cache-Aware Congestion Control Protocol in Wireless Sensor Networks

Reliability of data transport in wireless sensor networks is critical in the presence of high packet loss level either due to link contentions or buffer congestions. Mechanisms such as intermediate caching and congestion control can enhance the reliability performance of transport protocols in the event of packet loss. However, these two mechanisms are designed independently for most cache-based transport protocols. In this paper, we developed a new reliable transport protocol with a cache-aware congestion control mechanism called RT-CaCC. RT-CaCC utilizes cache management policies such as cache insertion, cache elimination, and cache size allocation to mitigate packet losses in the network while maximizing cache utilization and bandwidth allocation. The protocol was evaluated in network topologies with different packet loss scenarios. Results showed that RT-CaCC obtained an average of 15%–38% improvement gain compared with baseline protocols. In addition, RT-CaCC consistently outperformed other cache-based transport protocols like distributed TCP caching and end-to-end reliable and congestion aware transport layer protocol in terms of cache utilization, end-to-end delay, and fairness metrics. Finally, the comparison between simulation and analytical results showed that there is an almost perfect match with the difference between curve points never exceeding 14% of the analytical cost.

On Caching and Routing in Information-Centric Networks

With the exponential growth of content in recent years, users are primarily interested in obtaining particular content and are not concerned with the host housing the content. By treating content as a first class citizen, information- centric networks (ICN) seek to transform the Internet from a host-to-host communication model to a content-centric model. A key component of ICN is to cache content at storage-enabled routers. By caching content at in-network routers, network performance can be improved by delivering content from routers closer to the user and not from the origin servers (content custodians). In this article, we provide an overview of the state-of-the-art cache management and routing policies in ICN. We present a small comparison study of some state-of-the-art algorithms on a real Internet topology to help the reader appreciate how the different strategies compare against one another. Our simulation results demonstrate that it is hard to pick a winner among existing policies. We conclude the article with a discussion of open research questions.