Cache Replacement Research Articles

A study on impacts of hardware changes on performance of different multithreading libraries based on Gem5

This paper presents a thorough investigation into the performance of various multithreading libraries on different hardware configurations utilizing gem5. The study compares the performance of commonly-used multithreading libraries, including OpenMP, Pthread, TBB, and Dlib, through several benchmarks representing distinct types of workloads. Key hardware factors that directly affect the performance of each library are identified and analyzed. The research findings provide valuable insights and recommendations for selecting an appropriate multithreading library for a given application and hardware platform. Results indicate that the performance of multithreading libraries is influenced by numerous hardware factors, such as L1 cache size, L1 cache associate, L1 cache replacement policy, number of threads, and memory type. The libraries respond differently to these factors, and the performance variability of each library depends on the nature of the workload. The study offers a detailed analysis and valuable insights into the effects of these hardware factors on each library. This study contributes by presenting a comprehensive comparison of several widely used multithreading libraries across numerous hardware configurations, identifying the key hardware factors that influence the performance of each library, and providing insights and recommendations for selecting an appropriate multithreading library for a given application and hardware platform. Ultimately, the study results can help software developers make informed decisions when selecting a multithreading library and optimizing performance for their applications.

PoSiF: A Transient Content Caching and Replacement Scheme for ICN-IoT

Efficient data management plays a critical role in facilitating smooth communication and optimizing resource utilization within the rapidly growing ecosystem of the internet of things (IoT), which creates vast amounts of content. The emergence of Information-Centric Networking (ICN) has positioned it as a promising paradigm for facilitating IoT applications, as it prioritizes content-centric communication. This inherent characteristic of ICN has the potential to greatly improve the performance of IoT systems through the reduction of latency, conservation of network capacity, and enhancement of scalability. Existing research in this area focuses primarily on the popularity and freshness of the contents. However, they overlook the limited cache size of constraint devices, which should be utilized efficiently. This article presents a popularity, size, and freshness-based (PoSiF) transient content caching, and cache replacement scheme for ICN-enabled IoT devices. For content placement and replacement, the PoSiF scheme aims to enhance cache utilization through these three parameters while addressing the tradeoff between content size and popularity. The PoSiF technique demonstrates a significant enhancement in cache hit rates, with an improvement of 21.6% compared to the existing CCS/CES scheme, while effectively reducing hop ratios by 42.5%. On the other hand, cache replacement observes improved cache hits of 18% over least recently used (LRU) and 7.2% over least frequently used (LFU) while decreasing the average distance by 4.1% and 7.2%, respectively. While the retrieved freshness is 8.12% and 3.2% less than the LRU and LFU schemes, PoSiF compensates by satisfying more queries from cache-based content storage than the alternative methods.

Enhancing Cache Robustness in Information-Centric Networks: Per-Face Popularity Approaches

Information-Centric Networking (ICN) is a new paradigm of network architecture that focuses on content rather than hosts as first-class citizens of the network. As part of these architectures, in-network storage devices are essential to provide end users with close copies of popular content, to reduce latency and improve the overall experience for the user but also to reduce network congestion and load on the content producers. To be effective, in-network storage devices, such as content storage routers, should maintain copies of the most popular content objects. Adversaries that wish to reduce this effectiveness can launch cache pollution attacks to eliminate the benefit of the in-network storage device caches. Therefore, it is crucial to protect these devices and ensure the highest hit rate possible. This paper demonstrates Per-Face Popularity approaches to reducing the effects of cache pollution and improving hit rates by normalizing assessed popularity across all faces of content storage routers. The mechanisms that were developed prevent consumers, whether legitimate or malicious, on any single face or small number of faces from overwhelmingly influencing the content objects that remain in the cache. The results demonstrate that per-face approaches generally have much better hit rates than currently used cache replacement techniques.

Design, threat analysis and countermeasures for cache replacement policy-affecting Hardware Trojans in the context of a many-core system

Hardware Trojans, which threaten system reliability and integrity, have emerged as a security threat to the IC industry. Hardware manufacturers acquire ICs from unreliable third parties to reduce time to market and cost. Integration of untrusted component(s) in the IC design cycle and malicious modification(s) of ICs in the design house or foundry is popularly known as the Hardware Trojan. Trojans are implanted within a system to cause system malfunction, destruction or leakage of confidential information. Here, we have designed stealthy Trojans that degrade system performance by intervening in the cache replacement policies of a system. One of the designed Trojans affects systems that use the least recently used (LRU) cache replacement policy and decreases the infected core’s HitRatio up to 7.09%. The other one affects systems that use least frequently used (LFU) cache replacement policy and decreases the HitRatio of the infected core up to 8.52%. The proposed mitigation technique can improve the infected core’s HitRatio up to 8.19%.

Intelligent Cooperative Caching at Mobile Edge based on Offline Deep Reinforcement Learning

Cooperative edge caching enables edge servers to jointly utilize their cache to store popular contents, thus drastically reducing the latency of content acquisition. One fundamental problem of cooperative caching is how to coordinate the cache replacement decisions at edge servers to meet users’ dynamic requirements and avoid caching redundant contents. Online deep reinforcement learning (DRL) is a promising way to solve this problem by learning a cooperative cache replacement policy using continuous interactions (trial and error) with the environment. However, the sampling process of the interactions is usually expensive and time-consuming, thus hindering the practical deployment of online DRL-based methods. To bridge this gap, we propose a novel Delay-awarE Cooperative cache replacement method based on Offline deep Reinforcement learning (DECOR), which can exploit the existing data at the mobile edge to train an effective policy while avoiding expensive data sampling in the environment. A specific convolutional neural network is also developed to improve the training efficiency and cache performance. Experimental results show that DECOR can learn a superior offline policy from a static dataset compared to an advanced online DRL-based method. Moreover, the learned offline policy outperforms the behavior policy used to collect the dataset by up to 35.9%.

OA-Cache: Oracle Approximation-Based Cache Replacement at the Network Edge

With the explosive increase in mobile data traffic generated by various application services like video-on-demand and stringent quality of experience requirements of users, mobile edge caching is a promising paradigm to reduce delivery latency and network congestions by serving content requests locally. However, how to conduct cache replacement when the cache is full is a challenging issue when faced with enormous content volume and limited cache capacity at the network edge while the future request pattern is unknown ahead. In this paper, we propose a cache replacement algorithm based on the oracle approximation named OA-Cache in an end-to-end manner to maximize the cache hit rate. Specifically, we construct a complex model that uses a temporal convolutional network to capture the long and short dependencies between content requests. Then, an attention mechanism is adopted to find out the correlations between requests in the sliding window and cached contents. Instead of training a policy to mimic Belady that evicts the content with the longest reuse distance, we cast the learning task into a classification model to distinguish unpopular contents from popular ones. Finally, we apply the knowledge distillation approach to assist in transferring knowledge from a large pre-trained complex network to a lightweight network to readily accommodate to the network edge scenario. To validate the effectiveness of OA-Cache, we conduct extensive experiments on real-world datasets. The evaluation results demonstrate that OA-Cache can achieve better performance compared to candidate algorithms.

Ebublio: Edge-Assisted Multiuser 360° Video Streaming

As one of the most important manifestations of virtual reality (VR), 3600 panoramic videos in recent years have experienced booming development due to the desire for immersive and interactive experiences. Compared to traditional videos, 3600 videos are featured with uncertain user field of view (FoV), more sensitive delay tolerance, and much higher bandwidth requirement, bringing unprecedented challenges to 3600 video streaming. Meanwhile, the development of 5G and mobile edge computing starts to pave the way for high-bandwidth low-latency video streaming. Some preliminary works focus on either individual FoV prediction or multi-user QoE oriented cache strategy design, while how to design a holistic solution toward optimizing the overall user QoE with considerations over fairness and long-term system cost remains a non-trivial problem. In this paper, we propose Ebublio, a novel intelligent edge caching framework to address the aforementioned challenges in 3600 video streaming. Ebublio consists of a collaborative FoV prediction (CFP) module and a long-term tile caching optimization (LTO) module to jointly optimize the long-term user QoE and system cost. The former module integrates the features of video content, user trajectory, and other users records for combined prediction. The latter one employs the Lyapunov framework and a subgradient optimization approach towards the optimal caching replacement policy. Our trace-driven evaluation demonstrates the superiority of our framework, with about 42% improvement in FoV prediction, and 36% improvement in QoE at similar traffic consumption.

CACHE DATA REPLACEMENT POLICY BASED ON RECENTLY USED ACCESS DATA AND EUCLIDEAN DISTANCE

Data access management in web-based applications that use relational databases must be well thought out because the data continues to grow every day. The Relational Database Management System (RDBMS) has a relatively slow access speed because the data is stored on disk. This causes problems with decreased database server performance and slow response times. One strategy to overcome this is to implement caching at the application level. This paper proposed SIMGD framework that models Application Level Caching (ALC) to speed up relational data access in web applications. The ALC strategy maps each controller and model that has access to the database into a node-data in the in-Memory Database (IMDB). Not all node-data can be included in IMDB due to limited capacity. Therefore, the SIMGD framework uses the Euclidean distance calculation method for each node-data with its top access data as a cache replacement policy. Node-data with Euclidean distance closer to their top access data have a high priority to be maintained in the caching server. Simulation results show at the 25KB cache configuration, the SIMGD framework excels in achieving hit ratios compared to the LRU algorithm of 6.46% and 6.01%, respectively.

In-Network Caching for ICN-Based IoT (ICN-IoT): A Comprehensive Survey

The Internet of Things (IoT) has already emerged as one of the most popular directions in today’s information and communication technology (ICT) domain. With its advancement over different application areas, such as smart home, smart healthcare, industry 4.0, etc., a huge amount of data has been generated by billions of IoT devices, which aggravates the shortcomings of IP-based networks, such as limited expressiveness of IP addressing, inefficient support for mobility and in-network caching. Building IoT on top of information-centric networking (ICN) is believed to be a promising solution to tackle the above challenge, especially the in-network caching of ICN can significantly benefit IoT in terms of reducing data and saving IoT devices’ energy. However, caching IoT data is more challenging than caching traditional Internet content, e.g., video, because IoT data are usually valid within a certain period of time, and IoT devices are typically constrained with battery. Hence, in this survey, we first review the current implementation proposals of ICN-IoT. Next, we present the conventional caching decision policies and replacement policies which could be adopted to mitigate the aforementioned challenges, e.g., reducing IoT traffic, saving energy, reducing data retrieval latency. Further, since leveraging machine learning (ML) techniques have the potential to further improve the caching efficiency by dealing with uncertainties, e.g., predicting unknown information, adaptively interacting with the environment, we also demonstrate the recently proposed ML-based caching schemes for ICN-IoT. In addition, we outline the open research issues and point out the future opportunities of caching in ICN-IoT.

Adaptive Management With Request Granularity for DRAM Cache Inside nand-Based SSDs

Most flash-based solid-state drives (SSDs) adopt an on-board Dynamic Random Access Memory (DRAM) to buffer hot write data. Then, the write or overwrite operations can be absorbed by the DRAM cache, given that there is sufficient locality in the applications’ I/O access pattern, to consequently avoid flushing the write data onto underlying SSD cells. After analyzing typical real-world workloads over SSDs, we observed that the buffered data of small size requests are more likely to be re-accessed than those of large write requests. To efficiently utilize the limited space of DRAM cache, this paper proposes an adaptive request granularity-based cache management scheme for SSDs. First, we introduce a request block corresponding to a write request, as the cache management granularity, and propose a dynamic manner for classifying small and large request blocks. Next, we design three-level linked lists for supporting different routines of upgradation for small and large request blocks, once their data have been hit in the cache. At last, we present a scheme of evicting the request blocks having the minimum cost in cache replacement, by taking both factors of access hotness and time discounting into account. Experimental results show that our proposal can yield improvements on cache hits and the overall I/O latency by 21.8% and 14.7% on average, compared to state-of-the-art cache management schemes inside SSDs.

CoOR: Collaborative Task Offloading and Service Caching Replacement for Vehicular Edge Computing Networks

In vehicular edge computing networks, edge service caching has emerged as a promising technology that supports delay sensitive applications. When the vehicles pass the coverage areas of roadside units (RSUs), the vehicles can offload part/all of their tasks to the RSUs that had cached the related service data in advance. However, it is difficult to utilize the service caching resources in RSUs efficiently, for the high mobility, variable task computing requests of vehicles and the limited storage and computation capacity of RSUs. In this paper, we propose a <underline xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">co</u> llaborative task <underline xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">o</u> ffloading and service caching <underline xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">r</u> eplacement scheme from a vehicle perspective, named CoOR, the task processing cooperation between the adjacent RSUs and the service caching replacement for vehicle are mainly considered. We formulate the system computing cost and delay minimization as a mixed integer programming problem, it faces challenges of task offloading and service caching coupling, the heterogeneous computation requests and dynamic data transmission conditions. Thus, we develop an iterative algorithm combined with the Gibbs sampling and deep reinforcement learning (DRL) to find the optimal decisions. Extensive simulation results show that the proposed schemes have good convergence and better performance than the traditional baselines.

Effective Mimicry of Bélády’s MIN Policy

This paper introduces the Mockingjay cache replacement policy, which advances the state-of-the-art by producing single-core performance that approaches that of Bélády’s MIN policy, and which does so with a surprisingly simple solution. The basic idea is to predict a line’s reuse distance so that when a line is inserted into the LLC, its ETA (Estimated Time of Arrival) can be computed. Lines are then evicted based on their ETA ordering. This paper describes the Mockingjay policy and explains the key ideas that make it successful.

NCache: A Machine-Learning Cache Management Scheme for Computational SSDs

Inside a solid-state disk (SSD), cache stores frequently accessed data to shorten user-I/O response time and reduce the number of read/write operations in flash memory, thereby improving SSD performance and lifetime. Most existing cache schemes anchor in the spatiotemporal locality of I/O requests in workloads. In the face of a long-time workload, high performance and hit rate often get lost in these caching schemes. Flash memory-aware caching schemes trade hit ratio to prolong SSD lifetime. In this paper, we advocate for a machine learNing-based caching scheme named NCache to optimize both hit ratio and SSD performance. In NCache, we construct a machine learning (i.e., ML) model to predict whether data are re-accessed before being evicted from the cache. The cache replacement scheme preferentially evicts data that would not be accessed in the cache. The cache space is conserved for valid data that are likely to be repeatedly accessed. A pipelined scheme is implemented to accelerate the ML model, alleviating the time-cost of NCache. A double-linked list boosts the data addressing and cache replacement process. NCache is orthogonal to the existing caching schemes within the flash translation layer. The results validate NCache under a handful of real-world enterprise traces. Taking prn_0 as an example, NCache reduces the response time of LRU, CFLRU, GCaR_LRU, GCaR_CFLRU, and LCR by up to 15% with an average of 6.4%. The erase count is slashed by 16% at the maximum. Importantly, NCache is adroit at optimizing write amplification by up to 15.9%.

Orca: Scalable Temporal Graph Neural Network Training with Theoretical Guarantees

Representation learning over dynamic graphs is critical for many real-world applications such as social network services and recommender systems. Temporal graph neural networks (T-GNNs) are powerful representation learning methods and have achieved remarkable effectiveness on continuous-time dynamic graphs. However, T-GNNs still suffer from high time complexity, which increases linearly with the number of timestamps and grows exponentially with the model depth, causing them not scalable to large dynamic graphs. To address the limitations, we propose Orca, a novel framework that accelerates T-GNN training by non-trivially caching and reusing intermediate embeddings. We design an optimal cache replacement algorithm, named MRU, under a practical cache limit. MRU not only improves the efficiency of training T-GNNs by maximizing the number of cache hits but also reduces the approximation errors by avoiding keeping and reusing extremely stale embeddings. Meanwhile, we develop profound theoretical analyses of the approximation error introduced by our reuse schemes and offer rigorous convergence guarantees. Extensive experiments have validated that Orca can obtain two orders of magnitude speedup over the state-of-the-art baselines while achieving higher precision on large dynamic graphs.

DAC: A dynamic active and collaborative cache management scheme for solid state disks

NAND Flash memory-based SSDs have been widely used in modern storage systems to improve the performance and energy consumption. Existing caching management schemes rarely consider the utilization of parallel resources in flash memory array inside an SSD while these parallel resources cannot be fully exploited. The I/Os-access conflicts deteriorate the SSD performance and enlarge the response time under write-intensive workloads. Most cache replacement strategies give priority to clean data to improve the I/O-access conflicts, thereby boosting the SSD performance. However, clean data entirely depends on missed read-I/Os requests. Ideally, data replacement ought to evict clean data in the cache thanks to low replacement cost. As such, flash memory-based caching schemes give the highest priority to clean data. However, the proportion of replaced clean data is low in cache data. To solve this issue, we propose a dynamic active and collaborative cache management named DAC, in which the cache is composed of cold cache, hot cache, ghost cold cache, and ghost hot cache. DAC determines hot-cold changes of the I/O requests in workloads based on ghost cache size, thereby adjusting the real cache size to well serve I/O requests. The dynamic write-back window (DWW) mechanism dynamically adjusts the write-back window size in the cold cache. The write-back thresholds automatically update according to the changes in I/O patterns in workloads. When a flash chip is in the idle state, DAC produces clean data by proactively writing normal cold cache data back into flash memory, and the clean data is migrated into the active cold cache. DAC prioritizes replacing data in the active cold cache, aiming to avoid evicting data that are just read from flash memory. This method is adept at boosting the replacement rate of clean data, thereby optimizing SSD performance. We undertake extensive experiments in the latest NVMe SSD simulator to validate the DAC’s edge over the seven existing caching schemes including LRU, CFLRU, GCaR-CFLRU, LCR, ARC, AD-LRU, and MQsim. The results unveil that our DAC shortens the response time by up to 60.8% with an average optimization rate of 24.4%. When it comes to response-time cliff, the maximum improvement offered by DAC reaches approximately 68.71%; meanwhile, DAC delivers an average optimization rate of 26.9%. Besides, the number of erase count reduces by up to 93.3% with an average improvement rate of 17.04%.

A delayed eviction caching replacement strategy with unified standard for edge servers

The current explosion of user traffic compels Internet service providers to cache contents at edge servers, so as to reduce the response time of user requests. To deal with such vast amounts of content in edge servers with limited storage, an efficient caching replacement strategy is necessary, consisting of admission and eviction processes. Facing such a large number of user requests and limited storage of edge servers, traditional caching replacement strategies have encountered two major problems. Firstly, they conduct eviction process immediately after cache miss, which ignores the fetching time, during which there may still be multiple access requests to the evicted contents. Secondly, most existing solutions treat admission process and eviction process separately, with different standards, some contents might be fetched and evicted back and forth, seriously affecting user experience. To solve these two problems and improve caching performance, we design two caching modules, Delayed-Eviction and Unified-Standard, and integrate them together into Adele framework. By conducting extensive simulations using real traces, we demonstrate that Adele can improve the hit rate by 31% compared with the state-of-the-art solutions.

DCEC: D2D-Enabled Cost-Aware Cooperative Caching in MEC Networks

Various kinds of powerful intelligent mobile devices (MDs) need to access multimedia content anytime and anywhere, which places enormous pressure on mobile wireless networks. Fetching content from remote sources may introduce overly long accessing delays, which will result in a poor quality of experience (QoE). In this article, we considered the advantages of combining mobile/multi-access edge computing (MEC) with device-to-device (D2D) technologies. We propose a D2D-enabled cooperative edge caching (DCEC) architecture to reduce the delay of accessing content. We designed the DCEC caching management scheme through the maximization of a monotone submodular function under matroid constraints. The DCEC scheme includes a proactive cache placement algorithm and a reactive cache replacement algorithm. Thus, we obtained an optimal content caching and content update, which minimized the average delay cost of fetching content files. Finally, simulations compared the DCEC network architecture with the MEC and D2D networks and the DCEC caching management scheme with the least-frequently used and least-recently used scheme. The numerical results verified that the proposed DCEC scheme was effective at improving the cache hit ratio and the average delay cost. Therefore, the users’ QoE was improved.

Efficient Continuous Subgraph Matching Scheme Based on Trie Indexing for Graph Stream Processing

With the expansion of the application range of big data and artificial intelligence technologies, graph data have been increasingly used to analyze the relationships among objects. With the advancement of network technology and the spread of social network services, there has been an increasing need for a continuous query processing algorithm that can manage large-volume graph streams generated in real time. In this paper, a sliding-window-based continuous subgraph matching algorithm that can efficiently control graph streams is proposed. The proposed scheme uses a query processing technique based on trie indexing. It establishes an index based on a materialized view of similar queries and conducts continuous query processing based on the materialized view to perform continuous query processing efficiently. It also provides wildcard operations on vertices and edges to consider various query types. Moreover, in this study, a two-level cache technique that can manage frequently used subgraphs and subgraphs that may be used in the future is developed, to handle intermediate query results in the form of a materialized view. Cache replacement techniques based on statistical data are also presented to improve the performance of the developed cache technique. The excellent performance of the proposed algorithm is verified by a conducting independent performance evaluation and comparative performance evaluation.

Meta-reinforcement learning for edge caching in vehicular networks

Content caching to local repositories closer to the user can significantly improve the utilization of the backbone network, reduce latency, and improve reliability. Nevertheless, proactive caching in fast varying environments, especially in vehicular networks has many challenges including the change of the popularity of data with time as well as the changing popularity due to the change of requesting vehicles associated with the roadside units (RSUs). Learning techniques, especially reinforcement learning (RL), play a significant role in caching. Nevertheless, faster adaptation to reach the optimal policy with the dynamic nature of vehicular caching is still highly needed. In this paper, we propose a meta-reinforcement learning (Meta-RL) algorithm for proactive caching and caching replacement techniques based on model agnostic meta-learning (MAML) that can learn and adapt to new tasks faster and can improve the overall hit rate. Simulation results show that the proposed meta-RL algorithm improves the efficiency of the content caching system and provides faster convergence. The proposed meta-RL exhibits a particularly superior performance in the case of changing popularity of cached data.

CacheSack: Theory and Experience of Google’s Admission Optimization for Datacenter Flash Caches

This article describes the algorithm, implementation, and deployment experience of CacheSack, the admission algorithm for Google datacenter flash caches. CacheSack minimizes the dominant costs of Google’s datacenter flash caches: disk IO and flash footprint. CacheSack partitions cache traffic into disjoint categories, analyzes the observed cache benefit of each subset, and formulates a knapsack problem to assign the optimal admission policy to each subset. Prior to this work, Google datacenter flash cache admission policies were optimized manually, with most caches using the Lazy Adaptive Replacement Cache algorithm. Production experiments showed that CacheSack significantly outperforms the prior static admission policies for a 7.7% improvement of the total cost of ownership, as well as significant improvements in disk reads (9.5% reduction) and flash wearout (17.8% reduction).