Caching Policy Research Articles

Cache allocation policy based on user preference using reinforcement learning in mobile edge computing

SummaryIn mobile edge computing (MEC), due to the limited computing resources and power of mobile augmented reality (MAR) devices, cache identification results which can reduce power consumption and executing time of mobile devices are the solution to process MAR tasks. In this paper, we study an allocation cache problem in MAR systems. The allocation cache problem is formulated as maximizing the cache utility of cache hit ratio and user preference factor. To solve this problem, a cache resource allocation and cache space adjustment policy for edge computing systems is proposed. We also propose an improved double deep Q‐network (DDQN) algorithm to learn this policy. Simulation results show that the policy greatly improves the cache hit ratio compared with the traditional caching policy.

ChainKV: A Semantics-Aware Key-Value Store for Ethereum System

The Log-Structure Merged tree (LSM-tree) based key-value (KV) store has been widely adopted as the storage engine for blockchain systems, such as Ethereum, in which blockchain data are uniformly transformed into randomly distributed KV items for persistence. However, blockchain semantics are ignored during this process, making the blockchain storage suffer from heavy read/write amplification problems. Moreover, as the Ethereum network scales up, tremendous data further exacerbates its storage burden. Until now, most studies have focused on sharding, data archiving, decentralized distributed storage, etc., to mitigate the burden of the storage layer. However, the incompatibility between Ethereum semantics and the characteristics of the storage engine is ignored. In this paper, we present ChainKV, a new semantics-aware storage paradigm to improve the storage management performance for the Ethereum system. Firstly, based on Ethereum blockchain semantics, ChainKV separately stores different types of data in multiple storage zones in the KV store to mitigate the read/write amplification problem. Secondly, following the mechanism of the verification process in the authenticated data structure (ADS), a new ADS data transformer is proposed to exploit the data locality when persisting ADS. Moreover, a new space gaming caching policy is adopted to coordinate the cache space management for two independent storage zones. Finally, we propose an optional lightweight node crash recovery mechanism to eliminate functional redundancy between the Ethereum protocol and the storage engine. The experimental results indicate that ChainKV outperforms the prior Ethereum systems by up to 1.99× and 4.20× for synchronization and query operations, respectively

Random Caching Design for Multi-User Multi-Antenna HetNets With Interference Nulling

The strong interference suffered by users can be a severe problem in cache-enabled networks (CENs) due to the content-centric user association mechanism. To tackle this issue, multi-antenna technology may be employed for interference management. In this paper, we consider a user-centric interference nulling (IN) scheme in two-tier multi-user multi-antenna CEN, with a hybrid most-popular and random caching policy at macro base stations (MBSs) and small base stations (SBSs) to provide file diversity. All the interfering SBSs within the IN range of a user are requested to suppress the interference at this user using zero-forcing beamforming. Using stochastic geometry analysis techniques, we derive a tractable expression for the area spectral efficiency (ASE). A lower bound on the ASE is also obtained, with which we then consider ASE maximization, by optimizing the caching policy and IN coefficient. To solve the resultant mixed integer programming problem, we design an alternating optimization algorithm to minimize the lower bound of the ASE. Our numerical results demonstrate that the proposed caching policy yields performance that is close to the optimum, and it outperforms several existing baselines.

An Adaptive Metadata Management Scheme Based on Deep Reinforcement Learning for Large-Scale Distributed File Systems

A major challenge confronting today’s distributed metadata management schemes is how to meet the dynamic requirements of various applications through effectively mapping and migrating metadata nodes to different metadata servers (MDS’s). Most of the existing works dynamically reallocate nodes to different servers adopting history-based coarse-grained methods, failing to make a timely and efficient update on the distribution of nodes. In this paper, we present the first deep reinforcement learning-leveraged distributed metadata management scheme, AdaM, to address the aforementioned dilemma. AdaM is an adaptive fine-grained metadata management scheme that trains an actor-critic network to migrate “hot” metadata nodes to different MDS’s based on its observations of the current “states” (i.e., access pattern, the structure of namespace tree and current distribution of nodes on MDS’s). Adaptive to varying access patterns, AdaM can automatically migrate hot metadata nodes among servers to keep load balancing while maintaining metadata locality. Besides, we propose a self-adaptive metadata cache policy, which dynamically combines the two strategies of managing caches on the server side and the client side to gain better query performance. Last but not least, we design a distributed metadata processing 2PC Protocol called MST-based 2PC to ensure data consistency. Experiments on a real-world dataset demonstrate the superiority of our proposed method over other schemes.

Simulation based QoS aware dynamic caching scheme for heterogeneous content requests in vehicular edge computing

The Internet of Vehicles (IoV) has witnessed a substantial surge in its expansion during recent years, with the integration of edge servers emerging as an increasingly prevalent practice to cater to content requests that demand a superior quality of service. Nonetheless, conventional cache policies often prove inadequate for IoV applications, primarily due to their incapacity to effectively handle a diverse array of content requests, the high-speed mobility of vehicles, and the inherent instability of network connections. Within the scope of this study, we commence by bifurcating various content requests into two distinct categories: latency-sensitive content request and bandwidth-sensitive content request. Subsequently, we establish a model to evaluate the service quality under the joint consideration of different types of content requests, the mobility of vehicles and storage capacity of edge nodes. Furthermore, we transform the quality-of-service (QoS) penalty function into a system reward function. This adaption enables us to propose an innovative edge cache scheme founded upon the Deep Deterministic Policy Gradient (DDPG) algorithm of Reinforcement Learning (RL) method, which empowers dynamic adjustments in response to the evolving IoV environment. To validate the effectiveness of our proposed approach, we harness the Simulation of Urban Mobility (SUMO) traffic simulation software and construct a traffic road scenario based on a specific part segment of Nanjing Beltway. A wide-ranging set of contrast experiments was performed to ensure the improved performance of the DDPG based deep reinforcement learning method. Simulation experimental results show that the proposed algorithm converges quickly and outperforms existing algorithms in terms of service quality-hit ratio for latency-sensitive content and transmission speed for bandwidth-sensitive content.

Optimal Load-Splitting and Distributed-Caching for Dynamic Content Over the Wireless Edge

In this work, we consider the problem of ‘fresh’ caching at distributed (front-end) local caches of content that is subject to ‘dynamic’ updates at the (back-end) database. We first provide new models and analyses of the average operational cost of a network of distributed edge-caches that utilizes wireless multicast to refresh aging content. We attack the problems of what to cache in each edge-cache and how to split the incoming demand amongst them (also called “load-splitting” in the rest of the paper) in order to minimize the operational cost. While the general form of the problem comes with an NP-hard Knapsack structure, we were able to completely solve the problem by judiciously choosing the number of edge-caches to be deployed over the network This reduces the complex problem to a solvable special case. Interestingly, our findings reveal that the optimal caching policy necessitates unequal load-splitting over the edge-caches even when all conditions are symmetric. Moreover, we find that edge-caches with higher load will generally cache fewer but relatively more popular content. We further investigate the tradeoffs between cost reduction and cache savings when employing equal and optimal load-splitting solutions for demand with Zipf( <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$z$</tex-math> </inline-formula> ) popularity distribution. Our analysis reveals that equal load-splitting to edge-caches achieves close-to-optimal for less predictable demand ( <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$z&lt;2$</tex-math> </inline-formula> ) while also saving in the cache size. On the other hand, for more predictable demand ( <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$z&gt;2$</tex-math> </inline-formula> ), optimal load-splitting results in substantial cost gains while decreasing the cache occupancy.

Federated Multi-Armed Bandit Learning for Caching in UAV-aided Content Dissemination

This paper proposes an Unmanned aerial vehicle (UAV) aided content management system in communication-challenged disaster scenarios. Without cellular infrastructure in such scenarios, communities of stranded users can be provided access to situation-critical content using a hybrid network of static and traveling UAVs. A set of relatively static anchor UAVs with vertical as well as lateral links can provide content access to its local users. A set of ferrying UAVs with only lateral links, but with wider mobility, can provision content to users while visiting their communities. The objective is to design a content dissemination system that learns caching policies on-the-fly for maximizing content availability to the users. This paper proposes a distributed Federated Multi-Armed Bandit (MAB) Learning technique for UAV-caching decisions in the presence of geo-temporal differences in content popularity and heterogeneity in content demands. The proposed mechanism is able to combine the expected reward maximization attribute of Multi-Armed Bandit, and distributed intelligence sharing nature of Federated Learning for caching decision at the UAVs. It is demonstrated that Federated aggregation of individual MAB models can improve system performance while making the learning fast and adaptive. This analysis is done for different user-specified tolerable access delay, heterogeneous popularity distributions, and inter-community geographical characteristics. The paper does functional verification and performance evaluation of the proposed caching framework under a wide range of network size, UAV distribution, content popularity, and ferrying UAV trajectories.

Leveraging artificial intelligence and mutual authentication to optimize content caching in edge data centers

Available online Edge data centers are designed to meet the stringent QoE requirements of delay-sensitive and computationally intensive services in Content Delivery Network (CDN) and 5G networks. The primary purpose of this paper was to formulate and solve the problem of optimizing many control variables jointly: (i) what contents to store by taking into consideration edge capacity, and (ii) what contents to recommend to each Internet of Everything (IoE) item, based on identity and access management (IAM). In reactive caching policy, we proposed a new Two-Factor Authentication (2FA) scheme founded upon the Elliptic Curve Cryptography (ECC) and one-way hash function for access control. More interestingly, we use Non-negative Matrix Factorization (NMF), Fuzzy C-Means (FCM), Random Forest (RF) and Pearson Correlation (PC) to improve the accuracy and latency of traditional data filtering models. The intelligent recommendation engine we propose is designed to be implemented by cloud for caching and prefetching contents at the edge. The experimental results validate the theoretical guarantees of the proposed solution and its ability to achieve significant performance gains compared to common baseline models.

EMS-i : An Efficient Memory System Design with Specialized Caching Mechanism for Recommendation Inference

Recommendation systems have been widely embedded into many Internet services. For example, Meta’s deep learning recommendation model (DLRM) shows high prefictive accuracy of click-through rate in processing large-scale embedding tables. The SparseLengthSum (SLS) kernel of the DLRM dominates the inference time of the DLRM due to intensive irregular memory accesses to the embedding vectors. Some prior works directly adopt near data processing (NDP) solutions to obtain higher memory bandwidth to accelerate SLS. However, their inferior memory hierarchy induces low performance-cost ratio and fails to fully exploit the data locality. Although some software-managed cache policies were proposed to improve the cache hit rate, the incurred cache miss penalty is unacceptable considering the high overheads of executing the corresponding programs and the communication between the host and the accelerator. To address the issues aforementioned, we propose EMS-i , an efficient memory system design that integrates Solide State Drive (SSD) into the memory hierarchy using Compute Express Link (CXL) for recommendation system inference. We specialize the caching mechanism according to the characteristics of various DLRM workloads and propose a novel prefetching mechanism to further improve the performance. In addition, we delicately design the inference kernel and develop a customized mapping scheme for SLS operation, considering the multi-level parallelism in SLS and the data locality within a batch of queries. Compared to the state-of-the-art NDP solutions, EMS-i achieves up to 10.9× speedup over RecSSD and the performance comparable to RecNMP with 72% energy savings. EMS-i also saves up to 8.7× and 6.6 × memory cost w.r.t. RecSSD and RecNMP, respectively.

No-regret Caching via Online Mirror Descent

We study an online caching problem in which requests can be served by a local cache to avoid retrieval costs from a remote server. The cache can update its state after a batch of requests and store an arbitrarily small fraction of each file. We study no-regret algorithms based on Online Mirror Descent (OMD) strategies. We show that bounds for the regret crucially depend on the diversity of the request process, provided by the diversity ratio R/h , where R is the size of the batch and h is the maximum multiplicity of a request in a given batch. We characterize the optimality of OMD caching policies w.r.t. regret under different diversity regimes. We also prove that, when the cache must store the entire file, rather than a fraction, OMD strategies can be coupled with a randomized rounding scheme that preserves regret guarantees, even when update costs cannot be neglected. We provide a formal characterization of the rounding problem through optimal transport theory, and moreover we propose a computationally efficient randomized rounding scheme.

Revenue Maximization: The Interplay Between Personalized Bundle Recommendation and Wireless Content Caching

In this paper, we explore the interplay between personalized bundle recommendation and cache decision on the performance of wireless edge caching networks. A revenue maximization perspective is provided. To this end, we first examine the quantitative impact of bundle recommendation on the content request probability of different users. We then specify the definition of system revenue, showing its dependence on bundle recommendation and caching policies. With that, a joint bundling, caching and recommendation decision problem is formulated to maximize the achievable system revenue, taking into account the constraints of user-distinguished recommendation quality, recommendation amount, and the cache capacity budget. To solve this non-tractable optimization problem, a divide-then-conquer methodology is adopted. Specifically, we first determine the bundle state per user, on which basis we perform the joint bundle recommendation and caching decision-making, wherein several bundling strategies with different time-complexity are devised. Last but not least, we provide detailed properties analysis for our proposed bundling and joint optimization algorithms. Comprehensive numerical simulations validate the performance enhancement of the designed solutions compared to extensive conventional single-item recommendation oriented benchmarks.

The Implementation of Interactive VR Application and Caching Strategy Design on Mobile Edge Computing (MEC)

Virtual reality (VR) and augmented reality (AR) have been proposed as revolutionary applications for the next generation, especially in education. Many VR applications have been designed to promote learning via virtual environments and 360° video. However, due to the strict requirements of end-to-end latency and network bandwidth, numerous VR applications using 360° video streaming may not achieve a high-quality experience. To address this issue, we propose relying on tile-based 360° video streaming and the caching capacity in Mobile Edge Computing (MEC) to predict the field of view (FoV) in the head-mounted device, then deliver the required tiles. Prefetching tiles in MEC can save the bandwidth of the backend link and support multiple users. Smart caching decisions may reduce the memory at the edge and compensate for the FoV prediction error. For instance, caching whole tiles at each small cell has a higher storage cost compared to caching one small cell that covers multiple users. In this paper, we define a tile selection, caching, and FoV coverage model as the Tile Selection and Caching Problem and propose a heuristic algorithm to solve it. Using a dataset of real users’ head movements, we compare our algorithm to the Least Recently Used (LRU) and Least Frequently Used (LFU) caching policies. The results show that our proposed approach improves FoV coverage by 30% and reduces caching costs by 25% compared to LFU and LRU.

Predictive edge caching through deep mining of sequential patterns in user content retrievals

Edge caching plays an increasingly important role in boosting user content retrieval performance while reducing redundant network traffic. The effectiveness of caching ultimately hinges on the accuracy of predicting content popularity in the near future. However, at the network edge, content popularity can be extremely dynamic due to diverse user content retrieval behaviors and the low-degree of user multiplexing. It is challenging for the traditional reactive caching systems to keep up with the dynamic content popularity patterns. In this paper, we propose a novel Predictive Edge Caching (PEC) system that predicts the future content popularity using fine-grained learning models that mine sequential patterns in user content retrieval behaviors, and opportunistically prefetches contents predicted to be popular in the near future using idle network bandwidth. Through extensive experiments driven by real content retrieval traces, we demonstrate that PEC can adapt to highly dynamic content popularity at network edge, and significantly improve cache hit ratio and reduce user content retrieval latency over the state-of-art caching policies. More broadly, our study demonstrates that edge caching performance can be boosted by deep mining of user content retrieval behaviors.

Ascent Similarity Caching With Approximate Indexes

Similarity search is a key operation in multimedia retrieval systems and recommender systems, and it will play an important role also for future machine learning and augmented reality applications. When these systems need to serve large objects with tight delay constraints, edge servers close to the end-user can operate as similarity caches to speed up the retrieval. In this paper we present, a new similarity caching policy which improves on the state of the art by using (i) an (approximate) index for the whole catalog to decide which objects to serve locally and which to retrieve from the remote server, and (ii) a mirror ascent algorithm to update the set of local objects with strong guarantees even when the request process does not exhibit any statistical regularity.

Computational models of episodic-like memory in food-caching birds

Birds of the crow family adapt food-caching strategies to anticipated needs at the time of cache recovery and rely on memory of the what, where and when of previous caching events to recover their hidden food. It is unclear if this behavior can be explained by simple associative learning or if it relies on higher cognitive processes like mental time-travel. We present a computational model and propose a neural implementation of food-caching behavior. The model has hunger variables for motivational control, reward-modulated update of retrieval and caching policies and an associative neural network for remembering caching events with a memory consolidation mechanism for flexible decoding of the age of a memory. Our methodology of formalizing experimental protocols is transferable to other domains and facilitates model evaluation and experiment design. Here, we show that memory-augmented, associative reinforcement learning without mental time-travel is sufficient to explain the results of 28 behavioral experiments with food-caching birds.

Joint Edge Computing and Caching Based on D3QN for the Internet of Vehicles

With the Internet of Vehicles (IOV), a lot of self-driving vehicles (SDVs) need to handle a variety of tasks but have very seriously limited computing and storage resources, meaning they cannot complete intensive tasks timely. In this paper, a joint edge computing and caching based on a Dueling Double Deep Q Network (D3QN) is proposed to solve the problem of the multi-task joint edge calculation and caching process. Firstly, the processes of offloading tasks and caching them to the base station are modeled as optimization problems to maximize system revenues, which are limited by system latency and energy consumption as well as cache space for computing task constraints. Moreover, we also take into account the negative impact of the number of unfinished tasks in relation to the optimization problem—the higher the number of unfinished tasks, the lower the system revenue. Secondly, we use the D3QN algorithm together with the cache models to solve the formulated NP-hard problem and select the optimal caching and offloading action by adopting an e-greedy strategy. Moreover, two cache models are proposed in this paper to cache tasks, namely the active cache, based on the popularity of the task, and passive cache, based on the D3QN algorithm. Additionally, tasks which deal with cache space are updated by computing the expulsion value based on type of popularity. Finally, simulation results show that the proposed algorithm has good performance in terms of the latency and energy consumption of the system and that it improves utilization of cache space and reduces the probability of unfinished tasks. Compared to the Deep Q Network with caching policy, with the Double Deep Q Network with caching policy and Dueling Deep Q Network with caching policy, the system revenue of the proposed algorithm is improved by 65%, 35% and 66%, respectively. The scenario of the IOV proposed in this article can be expanded to larger-scale IOV systems by increasing the number of SDVs and base stations, and the content caching and download functions of the Internet of Things can also be achieved through collaboration between multiple base stations. However, only the cache model is focused on in this article, and the design of the replacement model is not good enough, resulting in a low utilization of cache resources. In future work, we will analyze how to make joint decisions based on multi-agent collaboration for caching, offloading and replacement in IOV scenarios with multiple heterogeneous services to support different Vehicle-to-Everything services.

Adaptive Context Caching for IoT-Based Applications: A Reinforcement Learning Approach.

Making internet-of-things (IoT)-based applications context-aware demands large amounts of raw data to be collected, interpreted, stored, and reused or repurposed if needed from many domains and applications. Context is transient but interpreted data can be distinguished from IoT data in many aspects. Managing context in cache is a novel area of research that has been given very little attention. Performance metric-driven adaptive context caching (ACOCA) can have a profound impact on the performance and cost efficiency of context-management platforms (CMPs) when responding to context queries in realtime. Our paper proposes an ACOCA mechanism to maximize both the cost and performance efficiency of a CMP in near realtime. Our novel mechanism encompasses the entire context-management life cycle. This, in turn, distinctively addresses the problems of efficiently selecting context for caching and managing the additional costs of context management in the cache. We demonstrate that our mechanism results in long-term efficiencies for the CMP that have not been observed in any previous study. The mechanism employs a novel, scalable, and selective context-caching agent implemented using the twin delayed deep deterministic policy gradient method. It further incorporates an adaptive context-refresh switching policy, a time-aware eviction policy, and a latent caching decision management policy. We point out in our findings that the additional complexity of adaptation introduced to the CMP through ACOCA is significantly justified, considering the cost and performance gains achieved. Our algorithm is evaluated using a real-world inspired heterogeneous context-query load and a data set based on parking-related traffic in Melbourne, Australia. This paper presents and benchmarks the proposed scheme against traditional and context-aware caching policies. We demonstrate that ACOCA outperforms the benchmarks in both cost and performance efficiency, i.e., up to 68.6%, 84.7%, and 67% more cost efficient compared to traditional data caching policies to cache context, redirector mode, and context-aware adaptive data caching under real-world-like circumstances.

Cache-MAB: A reinforcement learning-based hybrid caching scheme in named data networks

The presence of content stores in routers enables in-network caching in Named Data Networks (NDN) to improve customer experience, especially when content distribution is considered. Typically, homogeneous caching is assumed in the network where each router operates its cache according to the installed policy. Numerous in-network caching policies that vary in admission logic and caching attributes have been urged. Hence, the policies vary in performance (efficiency) according to unbounded network factors such as caching capacity, users’ request pattern, content popularity distribution, and application type (amongst other contexts). A single scheme may not effectively realize all the network characteristics and vandalize the efficient use of precious cache spaces. Therefore, a new strategy capable of dynamically selecting the optimal caching policy under varying network contexts is inevitable. In this direction, we propose a reinforcement learning (RL)-based hybrid caching strategy, namely Cache-MAB, where the routers work in a distributed manner and learn to pick the most suitable policy for caching a content. The set of policies is formed by picking the policies that assume similar caching attributes for caching decisions. The proposed strategy decouples the caching policy selection from the admission logic used by the selected policy and models the suitable policy selection as a Multi-armed Bandit (MAB) problem. The hybrid strategy equips each router with a diverse set of caching policies and an RL agent that utilizes a reinforcement algorithm to solve the MAB problem and, hence, select the optimal policy in different cases. The optimal policy is the one that maximizes the local performance, for instance, the cache hit rate. Simulation results confirm that the hybrid strategy effectively achieves the optimal policy’s performance, having a performance gap of less than 1%, by adapting to different network scenarios.

A Genetic Algorithm-based Proactive Caching Policy in Edge Networks

The incredible growth in mobile traffic lays down a challenge to network design. Therefore, proactive caching technology has been proposed as an effective method to reduce network congestion and improve the quality of service. In this paper, we focus on the caching position problem in edge networks and use a mixed integer linear programming (MILP) model to provide the optimal selection. Due to the curse of dimensionality, a genetic algorithm-based caching policy is given to make a balance between performance and time complexity. Numerical investigations reveal that the proposed approach can reduce the total cost by around 20% compared with other widely-used caching methods.

Caching in Cellular Networks Based on Multipoint Multicast Transmissions

We consider cellular network caching with network-wide Orthogonal Multipoint Multicast (OMPMC) delivery. We apply a probabilistic model for content placement at the Base Stations (BSs). Content is delivered with multipoint multicast operating in file-specific orthogonal resources: all BSs caching a distinct file synchronously multicast it to requesting users in a dedicated resource. For a network modeled as a Poisson Point Process (PPP), an expression for the outage probability is derived. The outage-minimizing cache policy is found from a joint constrained optimization problem over cache placement and resource allocation. We devise principles by which the solution in one propagation environment can be generalized to another. To reduce computational complexity, we obtain a sub-optimal solution based on convex relaxation. We obtain an upper bound of the gap between the optimal and sub-optimal solutions. We compare the outage performance of OMPMC with delivery polices from the literature. Simulation results show that exploiting OMPMC with optimal cache placement and resource allocation outperforms single point cache delivery policies with a wide margin.