Cache Eviction Research Articles

A hybrid Bi-level management framework for caching and communication in Edge-AI enabled IoT

The proliferation of IoT devices has led to a surge in network traffic, resulting in higher energy usage and response delays. In-network caching has emerged as a viable solution to address this issue. However, caching IoT data faces two key challenges: the transient nature of IoT content and the unknown spatiotemporal content popularity. Additionally, the use of a global view on dynamic IoT networks is problematic due to the high communication overhead involved. To tackle these challenges, this paper presents an adaptive management approach that jointly optimizes caching and communication in IoT networks using a novel bi-level control method called BC3. The approach employs two types of controllers: a global ILP-based optimal controller for long-term timeslots and local learning-based controllers for short-term timeslots. The long-term controller periodically establishes a global cache policy for the network and sends specific cache rules to each edge server. The local controller at each edge server solves the joint problem of bandwidth allocation and cache adaptation using deep reinforcement learning (DRL) technique. The main objective is to minimize energy consumption and system response time with utilizing the global and local observations. Experimental results demonstrate that the proposed approach increases cache hit rate by approximately 12% and uses 11% less energy compared to the other methods. Increasing the cache hit rate can lead to a reduction in about 17% in response time for user requests. Our bi-level control approach offers a promising solution for leveraging the network's global visibility while balancing communication overhead (as energy consumption) against system performance. Additionally, the proposed method has the lowest cache eviction, around 19% lower than the lowest eviction of the other comparison methods. The eviction metric is a metric to evaluate the effectiveness of adaptive caching approach designed for transient data.

Caiti: I/O transit caching for persistent memory-based block device

Byte-addressable non-volatile memory (NVM) sitting on the memory bus is employed to make persistent memory (pmem) in general-purpose computing systems and embedded systems for data storage. Researchers develop software drivers such as the block translation table (BTT) to build block devices on pmem, so programmers can keep using mature and reliable conventional storage stack while expecting high performance by exploiting fast pmem. However, our quantitative study shows that BTT underutilizes pmem and yields inferior performance, due to the absence of the imperative in-device cache. We add a conventional I/O staging cache made of DRAM space to BTT. As DRAM and pmem have comparable access latency, I/O staging cache is likely to be fully filled over time. Continual cache evictions and fsyncs thus cause on-demand flushes with severe stalls, such that the I/O staging cache is concretely unappealing for pmem-based block devices. We accordingly propose an algorithm named Caiti with novel I/O transit caching. Caiti eagerly evicts buffered data to pmem through CPU’s multi-cores. It also conditionally bypasses a full cache and directly writes data into pmem to further alleviate I/O stalls. Experiments confirm that Caiti significantly boosts the performance with BTT by up to 3.6×, without loss of block-level write atomicity.

Enhanced dynamic fine‐grained popularity‐based caching algorithm for ICN‐based edge computing networks

AbstractIn this letter, we propose an Enhanced Dynamic Fine‐Grained Popularity‐based Caching (ED‐FGPC) algorithm to efficiently cache static contents in ICN‐based edge computing networks. The content popularity changes with time. Thus, the ED‐FGPC algorithm considers the number of incoming interests, the practical file size, and the available cache size to adjust the content popularity threshold. This allows dynamic adjustment of the content popularity threshold. Additionally, ED‐FGPC incorporates a single bit called cached bit (CB) in the header field of the content. This eliminates caching redundancy in the on‐path routers. The cache eviction policy jointly considers the content popularity and the average content access timestamp for content replacement. Based on the content access pattern, this allows finer adjustment of the content eviction policy. We use analytical and mathematical modeling to derive the effectiveness and efficiency of our contribution. Results present a better hit ratio due to dynamic adjustment of the content popularity threshold and the elimination of caching redundancy from the on‐path routers.

Data Popularity for Cache Eviction Algorithms using Random Forests

In the HEP community the prediction of Data Popularity is a topic that has been approached for many years. Nonetheless, while facing increasing data storage challenges, especially in the upcoming HL-LHC era, there is still the need for better predictive models to answer the questions of whether particular data should be kept, replicated, or deleted. Caches have proven to be a convenient technique for partially automating storage management, potentially eliminating some of these questions. On the one hand, one can benefit even from simple cache eviction policies like LRU, on the other hand, we show that incorporation of knowledge about future access patterns has the potential to greatly improve cache performance. In this paper, we study data popularity on the file level, where the special relation between files belonging to the same dataset could be used in addition to the standard attributes. We turn to Machine Learning algorithms, such as Random Forest, which is well suited to work with Big Data: it can be parallelized, is more lightweight and easier to interpret than Deep Neural Networks. Finally, we compare the results with standard cache eviction algorithms and the theoretical optimum.

Towards a metrics suite for evaluating cache side-channel vulnerability: Case studies on an open-source RISC-V processor

Caches are key microarchitectural components of modern processors to improve memory access speed. However, attackers can readily implement timing side-channel attacks by exploiting the inherent time difference between cache hits and misses, which brings serious security threats to computer systems. In recent years, cache attacks have achieved fruitful consequences regarding attack techniques, targets, and platforms. At the same time, in order to measure the cache vulnerability, researchers have been looking at verifying the security policies and have proposed a variety of evaluation metrics. However, there are certain limitations in the current metrics, such as harsh assumptions, the distinguishing ability cannot be maintained all the time, the false positive and false negative noises can not be explicitly quantified, other tools are needed, and the verification environment is difficult to reproduce. To face these issues, we proposed a set of evaluation metrics, which includes the highest score (HS), minimum rounds to disclosure (MRD), and key score scissors differential (KSSD); we give the formal expressions in practice and theoretical, respectively. In order to facilitate the accurate reproduction of the experimental environment and results, we build a dual-core RISC-V bare-metal system based on the open-source framework Chipyard. On this basis, we conduct comprehensive evaluations to exploit vulnerabilities within the RISC-V cache architecture to verify the validity of the metrics suite. Furthermore, we compared the metrics with the commonly used success rate (SR) and guessing entropy (GE). The comparison results show that our proposed metrics suite can accurately depict the effects of attacks. Moreover, KSSD can maintain discrimination and converge quickly; MRD has more practical guidance; HS can specifically describe false negative noises. Finally, we perform a theoretical evaluation of AES algorithms with different T-table implementations using the proposed metrics, analyze the system's false positive and false negative noises, and suggest how the number of cache evictions can be determined under the random replacement policy.

Cache eviction for SSD-HDD hybrid storage based on sequential packing

Hybrid storage systems consist of NAND flash-based solid-state drives (SSDs) and conventional hard disk drives (HDDs). In which, the SSD device commonly acts as a cache for HDDs, to not only reduce overall power consumption, but also enhance responsiveness of a system. Cache eviction, which is responsible for moving the data out of SSD during garbage collection (GC) so as to maximize the use efficiency of cache, may greatly affect normal I/O processing. This paper proposes a technique of sequential packing-based cache eviction, which groups the flushed data pages in the GC block, with the (cold) sequential address data located in other SSD blocks. Then, the packed data pages can be flushed together onto the lower level storage of HDDs, to fully use their high sequential bandwidth. As a result, our approach can cut down negative effects of write amplification on the SSD cache, and contribute to better I/O performance of SSD-HDD hybrid storage.

CCALK: (When) CVA6 Cache Associativity Leaks the Key

In this work, we study an end-to-end implementation of a Prime + Probe covert channel on the CVA6 RISC-V processor implemented on a FPGA target and running a Linux OS. We develop the building blocks of the covert channel and provide a detailed view of its behavior and effectiveness. We propose a realistic scenario for extracting information of an AES-128 encryption algorithm implementation. Throughout this work, we discuss the challenges brought by the presence of a running OS while carrying out a micro architectural covert channel. This includes the effects of having other running processes, unwanted cache evictions and the OS’ timing behavior. We also propose an analysis of the relationship between the data cache’s characteristics and the developed covert channel’s capacity to extract information. According to the results of our experimentations, we present guidelines on how to build and configure a micro architectural covert channel resilient cache in a mono-core mono-thread scenario.

Context-based caching in mobile information-centric networks

Wireless networking is expected to sustain the direct interaction between personal users’ devices, and to provide connectivity on large-scale resource-constrained devices. However, conventional networking protocols fail in large scale mobile wireless environments, due to node mobility, dynamic topologies, and intermittent connectivity. Information-Centric Networking (ICN) has been considered the most promising candidate to overcome the drawbacks of host-centric architectures where Named Data Networking (NDN) is one of the well-known and studied architectures within the ICN paradigm.The main objective of this work is to improve both content availability and network performance in mobile environments regarding the ICN paradigm. This is provided through a context-based approach for the caching admission policy providing in-network caching and content replication, facilitating the efficient and timely delivery of information. Content popularity, freshness, proximity, source mobility type and network density are some of the metrics considered in the caching decision. We conducted a comparative study between our proposal and the NDN caching strategy by using two different datasets with real mobility and connectivity traces, addressing intermittent communication. According to our results, we observed that using a multi-criteria context-based cache admission policy improves cache hits, cache evictions, and request satisfaction ratios in mobile environments, thus improving content delivery and network efficiency.

A Failsoft Scheme for Mobile Live Streaming by Scalable Video Coding

In this study, we propose a mechanism called adaptive failsoft control to address peak traffic in mobile live streaming, using a chasing playback function. Although a cache system is avaliable to support the chasing playback function for live streaming in a base station and device-to-device communication, the request concentration by highlight scenes influences the traffic load owing to data unavailability. To avoid data unavailability, we adapted two live streaming features: (1) streaming data while switching the video quality, and (2) time variability of the number of requests. The second feature enables a fallback mechanism for the cache system by prioritizing cache eviction and terminating the transfer of cache-missed requests. This paper discusses the simulation results of the proposed mechanism, which adopts a request model appropriate for (a) avoiding peak traffic and (b) maintaining continuity of service.

The Role of Hysteresis in Caching Systems

Caching is a fundamental element of networking systems since the early days of the Internet. By filtering requests toward custodians, caches reduce the bandwidth required by the latter and the delay experienced by clients. The requests that are not served by a cache, in turn, comprise its miss stream. We refer to the dependence of the cache state and miss stream on its history as hysteresis. Although hysteresis is at the core of caching systems, a dimension that has not been systematically studied in previous works relates to its impact on caching systems between misses, evictions, and insertions. In this article, we propose novel mechanisms and models to leverage hysteresis on cache evictions and insertions. The proposed solutions extend TTL-like mechanisms and rely on two knobs to tune the time between insertions and evictions given a target hit rate. We show the general benefits of hysteresis and the particular improvement of the two thresholds strategy in reducing download times, making the system more predictable and accounting for different costs associated with object retrieval.

Contextual dimensions for cache replacement schemes in information-centric networks: a systematic review.

In recent years, information-centric networks (ICNs) have gained attention from the research and industry communities as an efficient and reliable content distribution network paradigm, especially to address content-centric and bandwidth-needed applications together with the heterogeneous requirements of emergent networks, such as the Internet of Things (IoT), Vehicular Ad-hoc NETwork (VANET) and Mobile Edge Computing (MEC). In-network caching is an essential part of ICN architecture design, and the performance of the overall network relies on caching policy efficiency. Therefore, a large number of cache replacement strategies have been proposed to suit the needs of different networks. The literature extensively presents studies on the performance of the replacement schemes in different contexts. The evaluations may present different variations of context characteristics leading to different impacts on the performance of the policies or different results of most suitable policies. Conversely, there is a lack of research efforts to understand how the context characteristics influence policy performance. In this direction, we conducted an extensive study of the ICN literature through a Systematic Literature Review (SLR) process to map reported evidence of different aspects of context regarding the cache replacement schemes. Our main findings contribute to the understanding of what is a context from the perspective of cache replacement policies and the context characteristics that influence cache behavior. We also provide a helpful classification of policies based on context dimensions used to determine the relevance of contents. Further, we contribute with a set of cache-enabled networks and their respective context characteristics that enhance the cache eviction process.

A New Upper Bound on Cache Hit Probability for Non-anticipative Caching Policies

Caching systems have long been crucial for improving the performance of a wide variety of network and web based online applications. In such systems, end-to-end application performance heavily depends on the fraction of objects transfered from the cache, also known as the cache hit probability. Many cache eviction policies have been proposed and implemented to improve the hit probability. In this work, we propose a new method to compute an upper bound on hit probability for all non-anticipative caching policies, i.e. for policies that have no knowledge of future requests. At each object request arrival, we use hazard rate (HR) function based ordering to classify the request as a hit or not. Under some statistical assumptions, we prove that our proposed HR based ordering model computes the maximum achievable hit probability and serves as an upper bound for all non-anticipative caching policies. We also provide simulation results to validate its correctness and to compare it to Belady's upper bound. We find it to almost always be tighter than Belady's bound.

Read-Ahead Efficiency on Mobile Devices: Observation, Characterization, and Optimization

Read-ahead schemes have been widely used in page cache to improve read performance of Linux systems. As the Android system inherits the Linux kernel, the traditional read-ahead scheme is directly transplanted to mobile devices. However, request sizes and page cache sizes on mobile devices are much smaller, which may degrade read-ahead efficiency and therefore hurt user experience. This article first observes that many pages pre-fetched by read-ahead are unused, which causes frequent page cache eviction. And these evict operations could induce extra access latency, especially when write-back is conducting. Then, this article proposes a new analysis model to characterize the factors that closely relate to the access latency. It is found that there exists a trade-off between read-ahead size and access latency. Finally, this article proposes two optimized read-ahead schemes to exploit this trade-off under different situations. Size-tuning scheme aims to find the proper maximum size of read-ahead according to the characteristics of mobile devices. While MobiRA scheme improves the read-ahead efficiency by dynamically tuning read-ahead size and stop-settings. Experimental results on real mobile devices show that the proposed schemes can increase the efficiency of read-ahead scheme and improve the overall performance of mobile devices.

Dynamic Coded Caching in Wireless Networks

We consider distributed and dynamic caching of coded content at small base stations (SBSs) in an area served by a macro base station (MBS). Specifically, content is encoded using a maximum distance separable code and cached according to a time-to-live (TTL) cache eviction policy, which allows coded packets to be removed from the caches at periodic times. Mobile users requesting a particular content download coded packets from SBSs within communication range. If additional packets are required to decode the file, these are downloaded from the MBS. We formulate an optimization problem that is efficiently solved numerically, providing TTL caching policies minimizing the overall network load. We demonstrate that distributed coded caching using TTL caching policies can offer significant reductions in terms of network load when request arrivals are bursty. We show how the distributed coded caching problem utilizing TTL caching policies can be analyzed as a specific single cache, convex optimization problem. Our problem encompasses static caching and the single cache as special cases. We prove that, interestingly, static caching is optimal under a Poisson request process, and that for a single cache the optimization problem has a surprisingly simple solution.

Prefetching and caching for minimizing service costs: Optimal and approximation strategies

Strategically prefetching data has been utilized in practice to improve caching performance. Apart from caching data items upon requests, they can be prefetched into the cache before requests actually occur. The caching and prefetching operations compete for the limited cache space, whose size is typically much smaller than the number of data items. A key question is how to design an optimal prefetching and caching policy, assuming that the future requests can be predicted to certain extend. This question is non-trivial even under an idealized assumption that the future requests are precisely known.To investigate this problem, we propose a cost-based service model. The objective is to find the optimal offline prefetching and caching policy that minimizes the accumulated cost for a given request sequence. By casting it as a min-cost flow problem, we are able to find the optimal policy for a data trace of length N in expected time O(N3∕2) via flow-based algorithms. However, this requires the entire trace for each request and cannot be applied in real time. To this end, we analytically characterize the optimal policy by obtaining an optimal cache eviction mechanism. We derive conditions under which proactive prefetching is a better choice than passive caching. Based on these insights, we propose a lightweight approximation policy that only exploits predictions in the near future. Moreover, the approximation policy can be applied in real time and processes the entire trace in O(N) expected time. We prove that the competitive ratio of the approximation policy is less than 2. Extensive simulations verify its near-optimal performance, for both heavy and light-tailed popularity distributions.

On formulation of online algorithm and framework of near-optimally tractable eviction for nonuniform caches

Distributed data sharing in Internet, social, and cloud computing paradigms incurs nonuniform costs such as consumed downstream bandwidth, downloading delays, and cloud-data-out monetary charges. To alleviate such costs, caches have been widely deployed and researched to find efficient online-cache-eviction algorithms. However, existing online algorithms have no offline foundation, thus they are far from a global optimum. This paper proposes a framework for developing an online cache eviction algorithm, which is grounded in a near-optimally and tractably offline algorithm namely Shortest Maximum Forward Distance (SMFD). On the formulation of the framework, the near-optimality and tractability properties of existing offline algorithms were evaluated through the formal evaluation of the optimality and computational tractability based upon variable object costs. Subsequently, the lowest-complexity suboptimal schemes were empirically investigated to seek a near-optimal offline one, which appears to be SMFD. Also, the results originally show that SMFD and its variant can practically achieve multiple conventionally upper bounds of limitless cache sizes in a stable and simultaneous manner. SMFD was then transformed into novel online SMFD by quantifying the offline property, maximum forward distance, of SMFD with cache-scope time-to-live approximation. To guarantee the effectiveness of online SMFD over evolving request streams, a safety bound guideline and a cache-scope time-to-live distribution model are also proposed. Finally, experience on the framework was gained via experiments based on deep-neural-network models.

Lifecycle-Aware Online Video Caching

The current explosion of video traffic compels service providers to deploy caches at edge networks. Nowadays, most caching systems store data with a high programming voltage corresponding to the largest possible ‘expiry date’, typically on the order of years, which maximizes the cache damage. However, popular videos rarely exhibit lifecycles longer than a couple of months. Consequently, the programming voltage can instead be adapted to fit the lifecycle and mitigate the cache damage accordingly. In this paper, we propose LiA-cache, a Lifecycle-Aware caching policy for online videos. LiA-cache finds both near-optimal caching retention times and cache eviction policies by optimizing traffic delivery cost and cache damage cost conjointly. We first investigate temporal patterns of video access from a real-world dataset covering 10 million online videos collected by one of the largest mobile network operators in the world. We next cluster the videos based on their access lifecycles and integrate the clustering into a general model of the caching system. Specifically, LiA-cache analyzes videos and caches them depending on their cluster label. Compared to other popular policies in real-world scenarios, LiA-cache can reduce cache damage up to 90 perce, while keeping a cache hit ratio close to a policy purely relying on video popularity.

Boosting read-ahead efficiency for improved user experience on mobile devices

Read-ahead schemes of page cache have been widely used to improve read performance of Linux systems. As Android system inherits the Linux kernel, traditional read-ahead schemes are directly applied in mobile devices. However, read request sizes and page cache sizes in mobile devices are much smaller than other platforms, which may decrease read-ahead efficiency and hurt user experience. The read-ahead efficiency is defined as hit pages / all pre-fetched pages in a sequential read. To study the efficiency of traditional read-ahead in mobile devices, this paper first observes that many pre-fetched pages are unused in page cache, which causes high page cache eviction ratio with high extra access latency. Then, this paper analyzes the factors that closely relate to the access latency. It is found that there exists a trade-off between read-ahead size and access latency. A size-tuning scheme is then proposed to explore this trade-off. Experimental results on real mobile devices have shown that the proposed scheme can reduce the number of pre-fetched pages and improve the efficiency of read-ahead without decreasing the page cache hit ratio.

Memway: in-memorywaylaying acceleration for practical rowhammer attacks against binaries

The Rowhammer bug is a novel micro-architectural security threat, enabling powerful privilege-escalation attacks on various mainstream platforms. It works by actively flipping bits in Dynamic Random Access Memory (DRAM) cells with unprivileged instructions. In order to set up Rowhammer against binaries in the Linux page cache, the Waylaying algorithm has previously been proposed. The Waylaying method stealthily relocates binaries onto exploitable physical addresses without exhausting system memory. However, the proof-of-concept Waylaying algorithm can be easily detected during page cache eviction because of its high disk I/O overhead and long running time. This paper proposes the more advanced Memway algorithm, which improves on Waylaying in terms of both I/O overhead and speed. Running time and disk I/O overhead are reduced by 90% by utilizing Linux tmpfs and inmemory swapping to manage eviction files. Furthermore, by combining Memway with the unprivileged posix fadvise API, the binary relocation step is made 100 times faster. Equipped with our Memway+fadvise relocation scheme, we demonstrate practical Rowhammer attacks that take only 15-200 minutes to covertly relocate a victim binary, and less than 3 seconds to flip the target instruction bit.

A Cache Fill and Migration Policy for STT-RAM-Based Multi-Level Hybrid Cache in 3D CMPs

Recently, in 3D Chip-Multiprocessors (CMPs), a hybrid cache architecture of SRAM and Non-Volatile Memory (NVM) is generally used to exploit high density and low leakage power of NVM and a low write overhead of SRAM. The conventional access policy does not consider the hybrid cache and cannot make good use of the characteristics of both NVM and SRAM technology. This paper proposes a Cache Fill and Migration policy (CFM) for multi-level hybrid cache. In CFM, data access was optimized in three aspects: Cache fill, cache eviction, and dirty data migration. The CFM reduces unnecessary cache fill, write operations to NVM, and optimizes the victim cache line selection in cache eviction. The results of experiments show that the CFM can improve performance by 24.1% and reduce power consumption by 18% when compared to conventional writeback access policy.