Cache Replacement Decisions Research Articles

Intelligent Cooperative Caching at Mobile Edge based on Offline Deep Reinforcement Learning

Cooperative edge caching enables edge servers to jointly utilize their cache to store popular contents, thus drastically reducing the latency of content acquisition. One fundamental problem of cooperative caching is how to coordinate the cache replacement decisions at edge servers to meet users’ dynamic requirements and avoid caching redundant contents. Online deep reinforcement learning (DRL) is a promising way to solve this problem by learning a cooperative cache replacement policy using continuous interactions (trial and error) with the environment. However, the sampling process of the interactions is usually expensive and time-consuming, thus hindering the practical deployment of online DRL-based methods. To bridge this gap, we propose a novel Delay-awarE Cooperative cache replacement method based on Offline deep Reinforcement learning (DECOR), which can exploit the existing data at the mobile edge to train an effective policy while avoiding expensive data sampling in the environment. A specific convolutional neural network is also developed to improve the training efficiency and cache performance. Experimental results show that DECOR can learn a superior offline policy from a static dataset compared to an advanced online DRL-based method. Moreover, the learned offline policy outperforms the behavior policy used to collect the dataset by up to 35.9%.

Multi-neural network based tiled 360°video caching with Mobile Edge Computing

It is challenging to stream 360° videos over the mobile network for its stringent latency and high bandwidth requirements. Although edge-based viewport adaptive tiled 360° video streaming solutions alleviate the bandwidth demand, the backhaul congestion and low latency concern remain persistent when data is served from the Content Delivery Network over the Internet. Edge caching can help mitigate these issues by storing the content at the edge of the cellular networks on the base station. However, caching 360° videos is challenging because of the large file size, which is further convoluted by tile selection in caching decisions. In this work, we propose a Mobile Edge Computing (MEC) based tiled 360° caching solution that uses Long–Short-Term-Memory (LSTM) and Convolutional Neural Network (CNN) in conjunction to address the challenges associated with 360° video caching. Specifically, the LSTM model predicts the future popularity of the videos, assisting in cache replacement decisions. For the selected videos, the CNN model, which is trained using the saliency map of the video, identifies the most engaging tiles in the videos for caching using the video content itself. The caching of tiles instead of the whole 360° videos improves the caching efficiency of the resource-constrained MEC server. The LSTM model is optimized based on the loss value of different hyperparameters, and AUROC (Ares Under ROC Curve) is used to evaluate the accuracy of the CNN model. Both the models produce highly accurate results. The results from extensive simulations show that the proposed solution significantly outperforms the existing methods. It improves the cache hit rate by at least 10% and reduces the backhaul usage by at least 35% with significant improvement in end-to-end latency, which is crucial for the quality of experience in 360° video streaming.

Exploring a Hybrid Voting-based Eviction Policy for Caches and Sparse Directories on Manycore Architectures

In manycore systems, eviction decisions related to caches and memory coherence greatly impact system performance, thereby emphasizing their importance. Extensive research has produced numerous standalone eviction policies such as LRU, LFU, FIFO, etc. all aiming to attain the Bélády optimum solution. Standalone eviction policies optimize for a single attribute (recency, frequency, etc.), limiting their impact on applications exhibiting non-uniform memory access patterns. The Hybrid Voting-based Eviction Policy (HyVE) extends multiple standalone eviction policies with a ranking system and evaluates them using concepts from the voting theory domain. The goal of HyVE is to make better replacement decisions by creating a consensus among its constituent eviction policies. With its inherent voting properties, HyVE takes different replacement decisions compared to its standalone counterparts, making it a unique and new eviction policy. We deploy and evaluate HyVE as part of two case-studies: last-level cache replacement decisions in a generic manycore environment, and sparse directory eviction decisions on a tile-based distributed shared memory (DSM) architecture. We explore different variants of HyVE, and evaluate them using workloads from the PARSEC and SPLASH-2 benchmark suites in a simulation environment. We also compare HyVE to state-of-the-art set-dueling and learning-based eviction policies. For last-level cache replacement decisions, HyVE reduces cache misses by 7.4% compared to the LRU policy, whereas DRRIP and Hawkeye reduce cache misses by 5.5% and 9.2% respectively compared to the LRU policy. Though Hawkeye exhibits better performance on average, HyVE offers a unique advantage for certain workloads by using a voting-based approach to solve the replacement problem. For sparse directory eviction decisions, results show that HyVE reduces coherence traffic and execution time by up to 11% compared to the LRU policy. We have synthesized HyVE on an FPGA prototype. Hardware analysis results show that HyVE’s constituent policies contribute the most to its overheads, while HyVE’s ranking and voting extensions do not add significant overheads. Timing analysis results show that HyVE’s logic delay is comparable to that of standalone eviction policies. Lastly, we evaluate HyVE on the FPGA prototype using characteristic micro-benchmarks that further emphasize HyVE’s ability to remain agnostic to varying data access patterns.

Efficient Cache Management for Network-Coding-Assisted Data Broadcast

Network coding has been proven to be an effective technique in improving the performance of data broadcast systems. Previous studies have shown that the performance of network-coding-assisted data broadcast systems is highly related to client cache contents. In this paper, we investigate the cache management problem in network-coding-assisted data broadcast environments with the objective of minimizing the overall response time. We formulate the problem and prove that it is NP-hard. Furthermore, we derive a probabilistic model to analyze the decoding probability to give insight into the importance of the cache management scheme. On this basis, we propose a decoding-oriented cache management scheme, which is called decoding-oriented least recently used (DLRU), which incorporates decoding information in making cache replacement decisions. In addition, since individual clients may observe the global data access pattern from encoded packets broadcast by the server, we loosen the cache admission control policy and allow clients to cache decodable and potentially useful data items that are not currently being requested. Based on this idea, we further propose a variant scheme, which is called DLRU with cache partitioning (DLRU-CP), which couples the cache partitioning policy to DLRU. We build a simulation model and implement the most competing alternative solutions in the literature for performance evaluation. The simulation results conclusively demonstrate the superiority of the proposed schemes in terms of reducing the request response time in network-coding-assisted data broadcast systems.

Back to the future

Belady's algorithm is optimal but infeasible because it requires knowledge of the future. This paper explains how a cache replacement algorithm can nonetheless learn from Belady's algorithm by applying it to past cache accesses to inform future cache replacement decisions. We show that the implementation is surprisingly efficient, as we introduce a new method of efficiently simulating Belady's behavior, and we use known sampling techniques to compactly represent the long history information that is needed for high accuracy. For a 2MB LLC, our solution uses a 16KB hardware budget (excluding replacement state in the tag array). When applied to a memory-intensive subset of the SPEC 2006 CPU benchmarks, our solution improves performance over LRU by 8.4%, as opposed to 6.2% for the previous state-of-the-art. For a 4-core system with a shared 8MB LLC, our solution improves performance by 15.0%, compared to 12.0% for the previous state-of-the-art.

ARC-H: Adaptive replacement cache management for heterogeneous storage devices

Heterogeneous storage architectures combine the strengths of different storage devices in a synergistically useful fashion, and are increasingly being used in mobile storage systems. In this paper, we propose ARC-H, an adaptive cache replacement algorithm for heterogeneous storage systems consisting of a hard disk and a NAND flash memory. ARC-H employs a dynamically adaptive management policy based on ghost buffers and takes account of recency, I/O cost per device, and workload patterns in making cache replacement decisions. Realistic trace-driven simulations show that ARC-H reduces service time by up to 88% compared with existing caching algorithms with a 20Mb cache. ARC-H also reduces energy consumption by up to 81%.

Fragmental Proxy Caching for Streaming Multimedia Objects

In this paper, a fragmental proxy-caching scheme that efficiently manages the streaming multimedia data in proxy cache is proposed to improve the quality of streaming multimedia services. The novel data-fragmentation method in this scheme not only provides finer granularity caching units to allow more effective cache replacement, but also offers a unique and natural way of handling the interactive VCR functions in the proxy-caching environment. Furthermore, a cache-replacement scheme, based on user request arrival rates for different multimedia objects and the playback rates of these objects, is proposed to address the drawbacks in existing cache-replacement schemes, most of which consider only the user access frequencies in their cache-replacement decisions. In this cache-replacement scheme, a sliding history window is employed to monitor the dynamic user request arrivals, and a tunable-victimization procedure is used to provide an excellent method of managing the cached multimedia data in accordance with different quality-of-service requirements of the streaming multimedia applications. Performance studies demonstrate that the fragmental proxy-caching scheme significantly outperforms other caching schemes, in terms of byte-hit ratio and the number of delayed starts and can be tuned to either maximize the byte-hit ratio or minimize the number of delayed starts

Generating cache hints for improved program efficiency

One of the new extensions in EPIC architectures are cache hints. On each memory instruction, two kinds of hints can be attached: a source cache hint and a target cache hint. The source hint indicates the true latency of the instruction, which is used by the compiler to improve the instruction schedule. The target hint indicates at which cache levels it is profitable to retain data, allowing to improve cache replacement decisions at run time. A compile-time method is presented which calculates appropriate cache hints. Both kind of hints are based on the locality of the instruction, measured by the reuse distance metric.Two alternative methods are discussed. The first one profiles the reuse distance distribution, and selects a static hint for each instruction. The second method calculates the reuse distance analytically, which allows to generate dynamic hints, i.e. the best hint for each memory access is calculated at run-time.The implementation of the static hints scheme in the Open64-compiler for the Itanium processor shows a speedup of 10% on average on a set of pointer-intensive and regular loop-based programs. The analytical approach with dynamic hints was implemented in the FPT-compiler and shows up to 34% reduction in cache misses.

Resource-aware speculative prefetching in wireless networks

Mobile users connected to wireless networks expect performance comparable to those on wired networks for interactive multimedia applications. Satisfying Quality of Service (QoS) requirements for such applications in wireless networks is a challenging problem due to limitations of low bandwidth, high error rate and frequent disconnections of wireless channels. In addition, wireless networks suffer from varying bandwidth. In this paper we investigate object prefetching during times of connectedness and bandwidth availability to enhance user perceived connectedness. This paper presents an access model that is suitable for multimedia access in wireless networks. Access modelling for the purpose of predicting future accesses in the context of speculative prefetching has received much attention in the literature. The model recognizes that a web page, instead of just a single file, is typically a compound of several files. When it comes to making prefetch decisions, most previous studies in speculative prefetching resort to simple heuristics, such as prefetching an item with access probabilities larger than a manually tuned threshold. This paper takes a different approach. Specifically, it models the performance of the prefetcher, taking into account access predictions and resource parameters, and develops a prefetch policy based on a theoretical analysis of the model. Since the analysis considers cache as one of the resource parameters, the resulting policy integrates prefetch and cache replacement decisions. The paper investigates the effect of prefetching on network load. In order to make effective use of available resources and maximize access improvement, it is beneficial to prefetch all items with access probabilities exceeding certain threshold.

Performance evaluation of Web proxy cache replacement policies

The continued growth of the World-Wide Web and the emergence of new end-user technologies such as cable modems necessitate the use of proxy caches to reduce latency, network traffic and Web server loads. In this paper we analyze the importance of different Web proxy workload characteristics in making good cache replacement decisions. We evaluate workload characteristics such as object size, recency of reference, frequency of reference, and turnover in the active set of objects. Trace-driven simulation is used to evaluate the effectiveness of various replacement policies for Web proxy caches. The extended duration of the trace (117 million requests collected over 5 months) allows long term side effects of replacement policies to be identified and quantified. Our results indicate that higher cache hit rates are achieved using size-based replacement policies. These policies store a large number of small objects in the cache, thus increasing the probability of an object being in the cache when requested. To achieve higher byte hit rates a few larger files must be retained in the cache. We found frequency-based policies to work best for this metric, as they keep the most popular files, regardless of size, in the cache. With either approach it is important that inactive objects be removed from the cache to prevent performance degradation due to pollution.

Application-Controlled Paging for a Shared Cache

We propose a provably efficient application-controlled global strategy for organizing a cache of size k shared among P application processes. Each application has access to information about its own future page requests, and by using that local information along with randomization in the context of a global caching algorithm, we are able to break through the conventional $H_k \sim \ln k$ lower bound on the competitive ratio for the caching problem. If the P application processes always make good cache replacement decisions, our online application-controlled caching algorithm attains a competitive ratio of $2H_{P-1}+2 \sim 2 \ln P$. Typically, P is much smaller than k, perhaps by several orders of magnitude. Our competitive ratio improves upon the 2P+2 competitive ratio achieved by the deterministic application-controlled strategy of Cao, Felten, and Li. We show that no online application-controlled algorithm can have a competitive ratio better than min{HP-1 ,Hk}, even if each application process has perfect knowledge of its individual page request sequence. Our results are with respect to a worst-case interleaving of the individual page request sequences of the P application processes. We introduce a notion of fairness in the more realistic situation when application processes do not always make good cache replacement decisions. We show that our algorithm ensures that no application process needs to evict one of its cached pages to service some page fault caused by a mistake of some other application. Our algorithm not only is fair but remains efficient; the global paging performance can be bounded in terms of the number of mistakes that application processes make.

Prefetching over a network

We discuss CTIP, an implementation of a network filesystem extension of the successful TIP informed prefetching and cache management system. Using a modified version of TIP in NFS client machines (and unmodified NFS servers). CTIP takes advantage of application-supplied hints that disclose the application's future read accesses. CTIP uses these hints to aggressively prefetch file data from an NFS file server and to make better local cache replacement decisions. This prefetching hides disk latency and exposes storage parallelism. Preliminary measurements that show CTIP can reduce execution time by a ratio comparable to that obtained with local TIP over a suite of I/O-intensive hinting applications. (For four disks, the reductions in execution time range from 17% to 69%). If local TIP execution requires that data first be loaded from remote storage into a local scratch area, then CTIP execution is significantly faster than the aggregate time for loading the data and executing. Additionally, our measurements show that the benefit of CTIP for hinting applications improves in the face of competition from other clients for server resources. We conclude with an analysis of the remaining problems with using unmodified NFS servers.