Cache Management Policies Research Articles

Bankshot

Emerging non-volatile storage (e.g., Phase Change Memory, STTRAM) allow access to persistent data at latencies an order of magnitude lower than SSDs. The density and price gap between NVMs and denser storage make NVM economically most suitable as a cache for larger, more conventional storage (i.e., NAND flashbased SSDs and disks). Existing storage caching architectures (even those that use fast flash-based SSDs) introduce significant software overhead that can obscure the performance benefits of faster memories. We propose Bankshot, a caching architecture that allows cache hits to bypass the OS (and the associated software overheads) entirely, while relying on the OS for heavy-weight operations like servicing misses and performing write backs. We evaluate several design decisions in Bankshot including different cache management policies and different levels of hardware, software support for tracking dirty data and maintaining meta-data. We find that with hardware support Bankshot can offer upto 5x speedup over conventional caching systems.

Evaluation of Background Push Content Download Services to Mobile Devices Over DVB Networks

This paper proposes a multicast content download service based on the use of residual network capacity to push multimedia content to available local storage in personal multimedia devices. The service under study is based on the FLUTE protocol. Specifically, FLUTE packets fill the spare capacity in the IP tunnels reserved for the primary streaming service (opportunistic insertion). The paper also evaluates the use of application layer-forward error correction (AL-FEC) parity to overcome transmission errors, object multiplexing to send the most popular multimedia contents more frequently and cache management policies that consider user preferences to keep in storage the most useful items. The service has been evaluated through simulations and measurements performed with an application prototype based on the DVB-H standards. The results show that AL-FEC enables the use of residual capacity for background content download services. In turn, AL-FEC, as well as object multiplexing, improves the relation between the number of content items and the overall access time. Moreover, results show that high percentages of requests can be served from the local cache of the service, provided that it is possible to estimate the popularity of content items and the user preferences.

Endurance-aware cache line management for non-volatile caches

Nonvolatile memories (NVMs) have the potential to replace low-level SRAM or eDRAM on-chip caches because NVMs save standby power and provide large cache capacity. However, limited write endurance is a common problem for NVM technologies, and today's cache management might result in unbalanced cache write traffic, causing heavily written cache blocks to fail much earlier than others. Although wear-leveling techniques for NVM-based main memories exist, we cannot simply apply them to NVM-based caches. This is because cache writes have intraset variations as well as interset variations, while writes to main memories only have interset variations. To solve this problem, we propose i 2 WAP, a new cache management policy that can reduce both inter- and intraset write variations. i 2 WAP has two features: Swap-Shift, an enhancement based on existing main memory wear leveling to reduce cache interset write variations, and Probabilistic Set Line Flush, a novel technique to reduce cache intraset write variations. Implementing i 2 WAP only needs two global counters and two global registers. In one of our studies, i 2 WAP can improve the NVM cache lifetime by 75% on average and up to 224%. We also validate that i 2 WAP is effective in systems with different cache configurations and workloads.

Low-energy volatile STT-RAM cache design using cache-coherence-enabled adaptive refresh

Spin-Torque Transfer RAM (STT-RAM) is a promising candidate for SRAM replacement because of its excellent features, such as fast read access, high density, low leakage power, and CMOS technology compatibility. However, wide adoption of STT-RAM as cache memories is impeded by its long write latency and high write power. Recent work proposed improving the write performance through relaxing the retention time of STT-RAM cells. The resultant volatile STT-RAM needs to be periodically refreshed to prevent data loss. When volatile STT-RAM is applied as the last-level cache (LLC) in chip multiprocessor (CMP) systems, frequent refresh operations could dissipate significant extra energy. In addition, refresh operations could severely conflict with normal read/write operations to degrade overall system performance. Therefore, minimizing the performance impact caused by refresh operations is crucial for the adoption of volatile STT-RAM. In this article, we propose Cache-Coherence-Enabled Adaptive Refresh (CCear) to minimize the number of refresh operations for volatile STT-RAM, adopted as the LLC for CMP systems. Specifically, CCear interacts with cache coherence protocol and cache management policy to minimize the number of refresh operations on volatile STT-RAM caches. Full-system simulation results show that CCear performs close to an ideal refresh policy with low overhead. Compared with state-of-the-art refresh policies, CCear simultaneously improves the system performance and reduces the energy consumption. Moreover, the performance of CCear could be further enhanced using small filter caches to accommodate the not-refreshed private STT-RAM blocks.

WADE

Emerging Non-Volatile Memory (NVM) technologies are explored as potential alternatives to traditional SRAM/DRAM-based memory architecture in future microprocessor design. One of the major disadvantages for NVM is the latency and energy overhead associated with write operations. Mitigation techniques to minimize the write overhead for NVM-based main memory architecture have been studied extensively. However, most prior work focuses on optimization techniques for NVM-based main memory itself, with little attention paid to cache management policies for the Last-Level Cache (LLC). In this article, we propose a Writeback-Aware Dynamic CachE (WADE) management technique to help mitigate the write overhead in NVM-based memory.<sup;>1</sup;> The proposal is based on the observation that, when dirty cache blocks are evicted from the LLC and written into NVM-based memory (with PCM as an example), the long latency and high energy associated with write operations to NVM-based memory can cause system performance/power degradation. Thus, reducing the number of writeback requests from the LLC is critical. The proposed WADE cache management technique tries to keep highly reused dirty cache blocks in the LLC. The technique predicts blocks that are frequently written back in the LLC. The LLC sets are dynamically partitioned into a frequent writeback list and a nonfrequent writeback list. It keeps a best size of each list in the LLC. Our evaluation shows that the technique can reduce the number of writeback requests by 16.5% for memory-intensive single-threaded benchmarks and 10.8% for multicore workloads. It yields a geometric mean speedup of 5.1% for single-thread applications and 7.6% for multicore workloads. Due to the reduced number of writeback requests to main memory, the technique reduces the energy consumption by 8.1% for single-thread applications and 7.6% for multicore workloads.

WADE

Emerging Non-Volatile Memory (NVM) technologies are explored as potential alternatives to traditional SRAM/DRAM-based memory architecture in future microprocessor design. One of the major disadvantages for NVM is the latency and energy overhead associated with write operations. Mitigation techniques to minimize the write overhead for NVM-based main memory architecture have been studied extensively. However, most prior work focuses on optimization techniques for NVM-based main memory itself, with little attention paid to cache management policies for the Last-Level Cache (LLC).

Process-Variation-Aware Adaptive Cache Architecture and Management

Fabricating circuits that employ ever-smaller transistors leads to dramatic variations in critical process parameters. This in turn results in large variations in execution/access latencies of different hardware components. This situation is even more severe for memory components due to minimum-sized transistors used in their design. Current design methodologies that are tuned for the worst case scenarios are becoming increasingly pessimistic from the performance angle, and thus, may not be a viable option at all for future designs. This paper makes two contributions targeting on-chip data caches. First, it presents an adaptive cache management policy based on nonuniform cache access. Second, it proposes a latency compensation approach that employs several circuit-level techniques to change the access latency of select cache lines based on the criticalities of the load instructions that access them. Our experiments reveal that both these techniques can recover significant amount of the lost performance due to worst case designs.

PIPP

Many multi-core processors employ a large last-level cache (LLC) shared among the multiple cores. Past research has demonstrated that sharing-oblivious cache management policies (e.g., LRU) can lead to poor performance and fairness when the multiple cores compete for the limited LLC capacity. Different memory access patterns can cause cache contention in different ways, and various techniques have been proposed to target some of these behaviors. In this work, we propose a new cache management approach that combines dynamic insertion and promotion policies to provide the benefits of cache partitioning, adaptive insertion, and capacity stealing all with a single mechanism. By handling multiple types of memory behaviors, our proposed technique outperforms techniques that target only either capacity partitioning or adaptive insertion.

Distributed Selfish Caching

Although cooperation generally increases the amount of resources available to a community of nodes, thus improving individual and collective performance, it also allows for the appearance of potential mistreatment problems through the exposition of one node's resources to others. We study such concerns by considering a group of independent, rational, self-aware nodes that cooperate using online caching algorithms, where the exposed resource is the storage at each node. Motivated by content networking applications - including Web caching, content delivery networks (CDNs), and peer-to-peer (P2P) - this paper extends our previous work on the offline version of the problem, which was conducted under a game-theoretic framework and limited to object replication. We identify and investigate two causes of mistreatment: 1) cache state interactions (due to the cooperative servicing of requests) and 2) the adoption of a common scheme for cache management policies. Using analytic models, numerical solutions of these models, and simulation experiments, we show that online cooperation schemes using caching are fairly robust to mistreatment caused by state interactions. To appear in a substantial manner, the interaction through the exchange of miss streams has to be very intense, making it feasible for the mistreated nodes to detect and react to exploitation. This robustness ceases to exist when nodes fetch and store objects in response to remote requests, that is, when they operate as level-2 caches (or proxies) for other nodes. Regarding mistreatment due to a common scheme, we show that this can easily take place when the "outlier" characteristics of some of the nodes get overlooked. This finding underscores the importance of allowing cooperative caching nodes the flexibility of choosing from a diverse set of schemes to fit the peculiarities of individual nodes. To that end, we outline an emulation-based framework for the development of mistreatment-resilient distributed selfish caching schemes.

An intermediate language level optimization framework for dynamic binary translation

This paper presents an intermediate language level optimization framework for dynamic binary translation. Performance is important to a dynamic binary translation system, so there has been a growing interest in exploring new optimization algorithms. The framework proposed in this paper includes efficient profiling, hot code recognition and smart code cache management policies. Profiling is responsible for collecting runtime information, which will be used by hot code recognition and code cache management algorithms. We only focus on recognizing the hottest code, and assign priorities to basic blocks according to their hotness to facilitate code cache management.

Data-Driven Multithreading Using Conventional Microprocessors

This paper describes the data-driven multithreading (DDM) model and how it may be implemented using off-the-shelf microprocessors. Data-driven multithreading is a nonblocking multithreading execution model that tolerates internode latency by scheduling threads for execution based on data availability. Scheduling based on data availability can be used to exploit cache management policies that reduce significantly cache misses. Such policies include firing a thread for execution only if its data is already placed in the cache. We call this cache management policy the CacheFlow policy. The core of the DDM implementation presented is a memory mapped hardware module that is attached directly to the processor's bus. This module is responsible for thread scheduling and is known as the thread synchronization unit (TSU). The evaluation of DDM was performed using simulation of the data-driven network of workstations (D <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> NOW). D <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> NOW is a DDM implementation built out of regular workstations augmented with the TSU. The simulation was performed for nine scientific applications, seven of which belong to the SPLASH-2 suite. The results show that DDM can tolerate well both the communication and synchronization latency. Overall, for 16 and 32-node D <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> NOW machines the speedup observed was 14.4 and 26.0, respectively

Frequency-based cache management policies for collaborative and non-collaborative topologies of segment based video caching proxies

This paper examines a novel cache management policy applied to non-collaborative and collaborative environments of more than one proxy server that serve homogeneous or even heterogeneous client requests for video streaming over the Internet. This cache management policy, which we call LRLFU because of a combination between an LRU and an LFU policy, is capable of capturing the changing popularities of the various videos by attaching a caching value to every video according to how recently and how frequently the video was requested, and decides to cache the most `valuable' videos. Our event-driven simulations have shown that LRLFU when applied to a simple non-collaborative topology of proxies and compared with previous work in this area (1) improve the byte-hit ratio (BHR), (2) significantly reduce the fraction of user requests with delayed starts and (3) require less CPU overhead. Furthermore, our simulation results have shown that the collaborative hierarchical tree topology of proxies that we examine achieves a much higher BHR when using the same overall cache capacity with the simple topology and in general provides better performance characteristics.

Managing bounded code caches in dynamic binary optimization systems

Dynamic binary optimizers store altered copies of original program instructions in software-managed code caches in order to maximize reuse of transformed code. Code caches store code blocks that may vary in size, reference other code blocks, and carry a high replacement overhead. These unique constraints reduce the effectiveness of conventional cache management policies. Our work directly addresses these unique constraints and presents several contributions to the code-cache management problem. First, we show that evicting more than the minimum number of code blocks from the code cache results in less run-time overhead than the existing alternatives. Such granular evictions reduce overall execution time, as the fixed costs of invoking the eviction mechanism are amortized across multiple cache insertions. Second, a study of the ideal lifetimes of dynamically generated code blocks illustrates the benefit of a replacement algorithm based on a generational heuristic. We describe and evaluate a generational approach to code cache management that makes it easy to identify long-lived code blocks and simultaneously avoid any fragmentation because of the eviction of short-lived blocks. Finally, we present results from an implementation of our generational approach in the DynamoRIO framework and illustrate that, as dynamic optimization systems become more prevalent, effective code cache-management policies will be essential for reliable, scalable performance of modern applications.

CacheFlow: Cache Optimizations for Data Driven Multithreading

Data-Driven Multithreading is a non-blocking multithreading model of execution that provides effective latency tolerance by allowing the computation processor do useful work, while a long latency event is in progress. With the Data-Driven Multithreading model, a thread is scheduled for execution only if all of its inputs have been produced and placed in the processor's local memory. Data-driven sequencing leads to irregular memory access patterns that could affect negatively cache performance. Nevertheless, it enables the implementation of short-term optimal cache management policies. This paper presents the implementation of CacheFlow, an optimized cache management policy which eliminates the side effects due to the loss of locality caused by the data-driven sequencing, and reduces further cache misses. CacheFlow employs thread-based prefetching to preload data blocks of threads deemed executable. Simulation results, for nine scientific applications, on a 32-node Data-Driven Multithreaded machine show an average speedup improvement from 19.8 to 22.6. Two techniques to further improve the performance of CacheFlow, conflict avoidance and thread reordering, are proposed and tested. Simulation experiments have shown a speedup improvement of 24% and 32%, respectively. The average speedup for all applications on a 32-node machine with both optimizations is 26.1.

Efficient index caching for data dissemination in mobile computing environments

Due to the limited power supply of mobile devices, much research has been done on reducing the power consumption of mobile devices in mobile computing environments. Since supporting indices on the broadcast data items can effectively reduce the power consumption of the mobile clients, most of the existing research on data broadcasting has focused on designing efficient indexing schemes. In this paper, we propose to cache indices on the mobile clients and to use cached indices to facilitate the access of the broadcast data items. To manage the cache usage of the mobile clients, we propose two cache management policies. The lower-level-index-first policy caches the index nodes that are at the lower level of the index tree, while the cut-plane-first policy caches the index nodes that belong to a cut-plane of the index tree. Through experiments, we compare the performance of the two proposed policies with some existing policies in terms of tuning time and access time. The experiments show that index caching significantly reduces the tuning time of a mobile client without increasing its access time. In terms of tuning time, the experiments show that, when the access pattern of a mobile client is not skew, the cut-plane-first policy outperforms the lower-level-index first policy, LRU and LRFU. On the contrary, when the mobile client has a small cache and its access pattern is skew, the lower-level-index-first policy outperforms the cut-plane-first policy, LRU and LRFU. In terms of access time, the lower-level-index first policy outperforms the cut-plane-first policy, LRU and LRFU.

Victim Replication

In this paper, we consider tiled chip multiprocessors (CMP) where each tile contains a slice of the total on-chip L2 cache storage and tiles are connected by an on-chip network. The L2 slices can be managed using two basic schemes: 1) each slice is treated as a private L2 cache for the tile 2) all slices are treated as a single large L2 cache shared by all tiles. Private L2 caches provide the lowest hit latency but reduce the total effective cache capacity, as each tile creates local copies of any line it touches. A shared L2 cache increases the effective cache capacity for shared data, but incurs long hit latencies when L2 data is on a remote tile. We present a new cache management policy, victim replication, which combines the advantages of private and shared schemes. Victim replication is a variant of the shared scheme which attempts to keep copies of local primary cache victims within the local L2 cache slice. Hits to these replicated copies reduce the effective latency of the shared L2 cache, while retaining the benefits of a higher effective capacity for shared data. We evaluate the various schemes using full-system simulation of both single-threaded and multi-threaded benchmarks running on an 8-processor tiled CMP. We show that victim replication reduces the average memory access latency of the shared L2 cache by an average of 16%for multi-threaded benchmarks and 24%for single-threaded benchmarks, providing better overall performance than either private or shared schemes.

Cache replacement policies for P2P file sharing protocols

AbstractPeer‐to‐peer (P2P) file‐sharing applications generate a large part of today's Internet traffic. The large volume of this traffic (thus high potential caching benefits) and the large cache sizes required (thus non‐trivial costs associated with caching) only underline that efficient cache replacement policies are important in this case. File popularity in P2P file‐sharing networks does not follow Zipf's law and several additional characteristics set the generated traffic apart from well‐studied Web traffic. All this lead us to conduct a focused study of efficient cache management policies for P2P file‐sharing traffic. This paper uses real‐world traces and trace driven simulations to compare traditional cache replacement policies and new policies that exploit characteristics of the P2P file‐sharing traffic generated by applications using FastTrack protocol. Copyright © 2004 AEI

Performance analysis of caching and prefetching strategies for palmtop-based navigational tools

Navigational tools that assist travelling people when they visit new areas are becoming increasingly common. Some of these tools are designed to be used with small portable devices (palmtops and also cellular phones) to give users more flexibility in their utilization (e.g. while walking in a town). The limited storage capacity of such devices makes it unrealistic to think that all the useful information can be stored in such devices, apart from the most basic one, like road maps and location of some points of interests. Other more detailed information should be loaded on the fly when needed, e.g., by using a wireless connection to some remote information service. However, this could cause unacceptably high latency to get the information. Another important factor that must be taken into account is the energy consumption caused by the access to the remote information service through a wireless link. Hence, the design of an information service that integrates the capabilities of palmtop-based navigational tools must be carefully designed around the goals of reducing both the perceived latency and the consumed energy. To this purpose, we investigate the use of caching and prefetching techniques. We present analytical and numerical results obtained by defining a probabilistic model of the typical utilization scenario of the considered service and corresponding cache management policies.

파일 타입을 이용한 웹 캐싱

본 논문에서는 웹상의 높은 가변성을 고려하여 웹 캐시 공간을 관리하는 새로운 접근 방법을 제안한다. 모든 문서를 저장하는데 하나의 캐시를 사용하는 것 대신에, 캐시를 분할하여 파일 타입에 따라 저장한다. 제안된 방법은 LFU, LRU와 SIZE 기반 알고리즘을 사용하여 현재의 캐시 관리 정책과 비교한다. 2가지의 서로 다른 실제 작업부하를 사용하며, 시뮬레이션을 통해 파일 타입별 캐싱이 적중률과 바이트적중률을 개선시킴을 보인다. This paper proposes a new access method which is to considered the high variability in World Wide Web and manage the web cache space. Instead of using a single cache, we divide a cache and store all documents according to their file types. Proposed method was compares with current cache management policies using LFU, LRU and SIZE base algorithm. Using two different workload, we show the improvement hitting ratio and byte hitting ratio through simulating on the file type caching.

On the sensitivity of cooperative caching performance to workload and network characteristics

A rich body of literature exists on several aspects of cooperative caching [1, 2, 3, 4, 5], including object placement and replacement algorithms [1], mechanisms for reducing the overhead of cooperation [2, 3], and the performance impact of cooperation [3, 4, 5]. However, while several studies have focused on quantifying the performance benefit of cooperative caching, their conclusions on the effectiveness of such cooperation vary significantly. The source of this apparent disagreement lies mainly in their different assumptions about workload and network characteristics, and about the degree of cooperation among caches.To more comprehensively evaluate the practical benefit of cooperative caching, we explore the sensitivity of the benefit of cooperation to workload characteristics such as object popularity distribution, temporal locality, one time referencing behavior, and to network characteristics such as latencies between clients, proxies, and servers. Furthermore, we identify a critical workload characteristic, which we call average access density, and show that it has a crucial impact on the effectiveness of cooperative caching.In this extended abstract, we report on a few important results selected from our extensive study reported in [6]. In particular, assuming an LFU-based cache management policy, we arrive at the following conclusions. First, cooperative caching is only effective when the average access density (defined as the ratio of the number of requests to the number of distinct objects in a time window) is relatively high. Second, the effectiveness of cooperative caching decreases as the skew in object popularity increases. Higher skew means that only a small number of objects are most frequently accessed reducing the benefit of larger caches, and therefore of cooperation.