Cache Resources Research Articles

Proactive Content Caching Based on Actor–Critic Reinforcement Learning for Mobile Edge Networks

Mobile edge caching/computing (MEC) has emerged as a promising approach for addressing the drastic increasing mobile data traffic by bringing high caching and computing capabilities to the edge of networks. Under MEC architecture, content providers (CPs) are allowed to lease some virtual machines (VMs) at MEC servers to proactively cache popular contents for improving users’ quality of experience. The scalable cache resource model rises the challenge for determining the ideal number of leased VMs for CPs to obtain the minimum expected downloading delay of users at the lowest caching cost. To address these challenges, in this paper, we propose an actor-critic (AC) reinforcement learning based proactive caching policy for mobile edge networks without the prior knowledge of users’ content demand. Specifically, we formulate the proactive caching problem under dynamical users’ content demand as a Markov decision process and propose a AC based caching algorithm to minimize the caching cost and the expected downloading delay. Particularly, to reduce the computational complexity, a branching neural network is employed to approximate the policy function in the actor part. Numerical results show that the proposed caching algorithm can significantly reduce the total cost and the average downloading delay when compared with other popular algorithms.

Exploiting Data Skew for Improved Query Performance

Analytic queries enable sophisticated large-scale data analysis within many commercial, scientific and medical domains today. Data skew is a ubiquitous feature of these real-world domains. In a retail database, some products are typically much more popular than others. In a text database, word frequencies follow a Zipf distribution with a small number of very common words, and a long tail of infrequent words. In a geographic database, some regions have much higher populations (and therefore data measurements) than others. Current systems do not make the most of caches for exploiting skew. In particular, a whole cache line may remain cache resident even though only a small part of the cache line corresponds to a popular data item. In this article, we propose a novel index structure for repositioning data items to concentrate popular items into the same cache lines. The net result is better spatial locality, and better utilization of limited cache resources. We develop a theoretical model for analyzing the cache utilization, and implement database operators that are efficient in the presence of skew. Our experimental evaluation on real and synthetic data shows that exploiting skew can significantly improve in-memory query performance. In some cases, our techniques can speed up queries by over an order of magnitude.

Meta-Hierarchical Reinforcement Learning (MHRL)-Based Dynamic Resource Allocation for Dynamic Vehicular Networks

With the rapid development of vehicular networks, there is an increasing demand for extensive networking, computting, and caching resources. How to allocate multiple resources effectively and efficiently for dynamic vehicular networks is extremely important. Most existing works on resource management in vehicular networks assume static network conditions. In this paper, we propose a general framework that can enable fast-adaptive resource allocation for dynamic vehicular environments. Specifically, we model the dynamics of the vehicular environment as a series of related Markov Decision Processes (MDPs), and we combine hierarchical reinforcement learning with meta-learning, which makes our proposed framework quickly adapt to a new environment by only fine-tuning the top-level master network, and meanwhile the low-level sub-networks can make the right resource allocation policy. Extensive simulation results show the effectiveness of our proposed framework, which can quickly adapt to different scenarios, and significantly improve the performance of resource management in dynamic vehicular networks.

A Dynamic Cache Allocation Mechanism (DCAM) for Reliable Multicast in Information-Centric Networking

As a new network architecture, information-centric networking (ICN) decouples the identifiers and locators of network entities and makes full use of in-network cache technology to improve the content distribution efficiency. For reliable multicast, ICN in-network cache can help reduce the loss recovery delay. However, with the development of applications and services, a multicast tree node often serves multiple reliable multicast groups. How to reasonably allocate cache resources for each multicast group will greatly affect the performance of reliable multicast. In order to improve the overall loss recovery performance of reliable multicast, this paper designs a dynamic cache allocation mechanism (DCAM). DCAM considers the packet loss probability, the node depth of the multicast tree, and the multicast transmission rate of multicast group, and then allocates cache space for multicast group based on the normalized cache quota weight. We also explore the performance of three cache allocation mechanisms (DCAM, AARM, and Equal) combined with four cache strategies (LCE, CAPC, Prob, and ProbCache), respectively. Experimental results show that DCAM can adjust cache allocation results in time according to network changes, and its combinations with various cache strategies outperform other combinations. Moreover, the combination of DCAM and CAPC can achieve optimal performance in loss recovery delay, cache hit ratio, transmission completion time, and overhead.

Cybertwin-driven resource allocation using deep reinforcement learning in 6G-enabled edge environment

The recent emergence of sixth-generation (6G) enabled wireless communication technology has resulted in the rapid proliferation of a wide range of real-time applications. These applications are highly data-computation intensive and generate huge data traffic. Cybertwin-driven edge computing emerges as a promising solution to satisfy massive user demand, but it also introduces new challenges. One of the most difficult challenges in edge networks is efficiently offloading tasks while managing computation, communication, and cache resources. Traditional statistical optimization methods are incapable of addressing the offloading problem in a dynamic edge computing environment. In this work, we propose a joint resource allocation and computation offloading scheme by integrating deep reinforcement learning in Cybertwin enabled 6G wireless networks. The proposed system uses the potential of the MATD3 algorithm to provide QoS to end-users by minimizing the overall latency and energy consumption with better management of cache resources. As these edge resources are deployed in inaccessible locations, therefore, we employ secure authentication mechanism for Cybertwins. The proposed system is implemented in a simulated environment, and the results are calculated for different performance metrics with previous benchmark methodologies such as RRA, GRA, and MADDPG. The comparative analysis reveals that the proposed MATD3 reduces end-to-end latency and energy consumption by 13.8% and 12.5% respectively over MADDPG with a 4% increase in successful task completion.

An efficient resource utilization technique for scheduling scientific workload in cloud computing environment

<span lang="EN-US">Recently, number of data intensive workflow have been generated with growth of internet of things (IoT’s) technologies. Heterogeneous cloud framework has been emphasized by existing methodologies for executing these data-intensive workflows. Efficient resource scheduling plays a very important role provisioning workload execution on Heterogeneous cloud framework. Building tradeoff model in meeting energy constraint and workload task deadline requirement is challenging. Recently, number of multi-objective-based workload scheduling aimed at minimizing power budget and meeting task deadline constraint. However, these models induce significant overhead when demand and number of processing core increases. For addressing research problem here, the workload is modelled by considering each sub-task require dynamic memory, cache, accessible slots, execution time, and I/O access requirement. Thus, for utilizing resource more efficiently better cache resource management is needed. Here efficient resource utilization (ERU) model is presented. The ERU model is designed to utilize cache resource more efficiently and reduce last level cache failure and meeting workload task deadline prerequisite. The ERU model is very efficient when compared with standard resource management methodology in terms of reducing execution time, power consumption, and energy consumption for execution scientific workloads on heterogeneous cloud platform.</span>

Edge Intelligence for Mission-Critical 6G Services in Space-Air-Ground Integrated Networks

Next-generation wireless services will change our daily lives by supporting smart factories, intelligent transportation, augmented/virtual reality, and more. These sophisticated services are usually both data- and computation-intensive, and must meet stringent latency and reliability requirements that cannot be readily satisfied by cloud-based service processing. Therefore, the 6G cellular network is expected to jointly optimize communication, computing, caching, and control. A further aspiration of 6G is to conceive a seamless space-air-ground integrated network (SAGIN) for filling the vast coverage holes across the globe, bringing new opportunities for mission-critical services. In this article, we aim for conceiving ultrareliable and low-latency edge intelligence for supporting mission-critical services by harnessing the diversified communication, computing, and caching resources at the network edge of SAGIN. We critically appraise a number of key enabling techniques, including a number of new service-centric resource allocation techniques. Finally, a range of open challenges is discussed.

Distributed Reinforcement Learning for Privacy-Preserving Dynamic Edge Caching

Mobile edge computing (MEC) is a prominent computing paradigm which expands the application fields of wireless communication. Due to the limitation of the capacities of user equipments and MEC servers, edge caching (EC) optimization is crucial to the effective utilization of the caching resources in MEC-enabled wireless networks. However, the dynamics and complexities of content popularities over space and time as well as the privacy preservation of users pose significant challenges to EC optimization. In this paper, a privacy-preserving distributed deep deterministic policy gradient (P2D3PG) algorithm is proposed to maximize the cache hit rates of devices in the MEC networks. Specifically, we consider the fact that content popularities are dynamic, complicated and unobservable, and formulate the maximization of cache hit rates on devices as distributed problems under the constraints of privacy preservation. In particular, we convert the distributed optimizations into distributed model-free Markov decision process problems and then introduce a privacy-preserving federated learning method for popularity prediction. Subsequently, a P2D3PG algorithm is developed based on distributed reinforcement learning to solve the distributed problems. Simulation results demonstrate the superiority of the proposed approach in improving EC hit rate over the baseline methods while preserving user privacy.

QoS-Driven Resource Optimization for Intelligent Fog Radio Access Network: A Dynamic Power Allocation Perspective

The fog radio access network (Fog-RAN) has been considered a promising wireless access architecture to help shorten the communication delay and relieve the large data delivery burden over the backhaul links. However, limited by conventional inflexible communication design, Fog-RAN cannot be used in some complex communication scenarios. In this study, we focus on investigating a more intelligent Fog-RAN to assist the communication in a high-speed railway environment. Due to the train’s continuously moving, the communication should be designed intelligently to adapt to channel variation. Specifically, we dynamically optimize the power allocation in the remote radio heads (RRHs) to minimize the total network power cost considering multiple quality-of-service (QoS) requirements and channel variation. The impact of caching on the power allocation is considered. The dynamic power optimization is analyzed to obtain a closed-form solution in certain cases. The inherent tradeoff among the total network cost, delay and delivery content size is further discussed. To evaluate the performance of the proposed dynamic power allocation, we present an invariant power allocation counterpart as a performance comparison benchmark. The result of our simulation reveals that dynamic power allocation can significantly outperform the invariant power allocation scheme, especially with a random caching strategy or limited caching resources at the RRHs.

Deep Reinforcement Learning-Based Mobility-Aware UAV Content Caching and Placement in Mobile Edge Networks

With the proliferation of smart mobile devices, there is now an ever increasing craving for higher bandwidth for end user satisfaction. Increasing mobile traffic leads to congestion of backhaul networks. One promising solution to this problem is the mobile edge network and consequently mobile edge caching. There is an emerging paradigm shift toward the use of unmanned aerial vehicles (UAVs) to assist the traditional cellular networks and also to provide connectivity in places where there are no small base stations or faulty ones as a result of some natural disaster such as flooding. Hence, UAVs can be used to assist in content caching as well. This work proposes the use of human centric features, random waypoint user mobility model, and deep reinforcement learning to predict the location of the UAVs and the contents to cache at the UAVs. We formulated our problem as a Markov decision problem (MDP) and proposed a dueling reinforcement learning-based algorithm to solve the MDP problem. Our simulation results prove that our algorithm converges to an optimal solution and performs better than other baseline reinforcement learning algorithms in terms of quality of experience satisfaction, transmit power, and cache resource utilization

Understanding I/O Direct Cache Access Performance for End Host Networking

Direct Cache Access (DCA) enables a network interface card (NIC) to load and store data directly on the processor cache, as conventional Direct Memory Access (DMA) is no longer suitable as the bridge between NIC and CPU in the era of 100 Gigabit Ethernet. As numerous I/O devices and cores compete for scarce cache resources, making the most of DCA for networking applications with varied objectives and constraints is a challenge, especially given the increasing complexity of modern cache hardware and I/O stacks. In this paper, we reverse engineer details of one commercial implementation of DCA, Intel's Data Direct I/O (DDIO), to explicate the importance of hardware-level investigation into DCA. Based on the learned knowledge of DCA and network I/O stacks, we (1) develop an analytical framework to predict the effectiveness of DCA (i.e., its hit rate) under certain hardware specifications, system configurations, and application properties; (2) measure penalties of the ineffective use of DCA (i.e., its miss penalty) to characterize its benefits; and (3) show that our reverse engineering, measurement, and model contribute to a deeper understanding of DCA, which in turn helps diagnose, optimize, and design end-host networking.

Content Placement Considering the Temporal and Spatial Attributes of Content Popularity in Cache-Enabled UAV Networks

Unmanned aerial vehicles (UAVs) can provide low-cost connectivity to wireless devices in areas with no infrastructure coverage. In particular, the cache-enabled UAV network can meet the needs of Ground Terminals (GTs) by virtue of its content cache advantages, and reduces the use of backhaul links. However, since the temporal and spatial distribution of content popularity in the cache-enabled UAV network is dynamic, guaranteeing the utilization of UAV cache resources is challenging. Correspondingly, this paper presents a mathematical model for the temporal and spatial distribution characteristics of content popularity of the enabled UAV network, the remaining cache changes, and the user download experience effect, to propose an optimized content placement method in an enabled UAV network. Lastly, the acquired numerical results validate the effectiveness of our method in the cache resource allocation of UAV.

Collaborative Caching for Energy Optimization in Content-Centric Internet of Things

The development of the content-centric Internet of Things (C2IoT) enriches the services provided by the IoT devices, which diversifies the provided contents. However, system resources are terribly wasted by repeated delivering the same content. Efficiently utilizing caching resources can reduce the link load caused by delivering and forwarding contents, further improving the users’ quality of experience (QoE). However, the collaboration among independent decision nodes is insufficient, which consumes substantial energy redundantly. To solve the mentioned issue, we propose a genetic-algorithm-based collaborative caching scheme integrating on-path strategy and off-path strategy, named cost-oriented caching scheme (CCS), to minimize the energy consumption of requesting contents. First, based on the analysis of content popularity, users’ historical requests, and transmission delay, an optimization problem is formulated to minimize the communication delay of obtaining content. Then, we formulate a submodular function optimization problem to optimize energy consumption, and an improved genetic algorithm is proposed to solve the mentioned problem. Performance evaluations demonstrate that the proposed CCS is superior to other existing caching schemes.

A Profit-Aware Coalition Game for Cooperative Content Caching at the Network Edge

The user demands to delay-sensitive applications have put forward new requirements for the mobile networks. Edge caching as a promising way is proposed to enhance the Quality of Service (QoS) for end users at the network edge. Given the widely distributed edge nodes, the content providers (CPs) usually prefer to integrate them with cooperative caching services, by forming a cache coalition. Although such a coalition could be beneficial as a whole, it neglects the profits of individual members, which is one major concern in forming the coalition itself. Besides, due to the poor scalability, the conventional cooperation scheme, which only considers fixed edge nodes, cannot adapt to the spatial and temporal imbalance of user requests. In this article, we tackle the problems of coalition establishment and profit allocation among the coalition members. Particularly, by adopting both fixed edge nodes and mobile vehicles as caching nodes, we propose a hybrid service provisioning framework and cooperative service caching and workload scheduling methods. To maximize the profits in managing the caching resources in the established coalition, we devise an optimization model with a mixed-integer programming (MIP), in which the QoS requirements of end users and caching capacities of each coalition member are also considered as constraints. In addition, based on the Hedonic game theory, we propose a dynamic coalition algorithm to guide each member to join or leave the coalition at each time slot out of its own profits. The experimental results demonstrate that compared to the cases only considering fixed caching nodes, our hybrid caching scheme can improve: 1) the overall profit of the coalition by 53% and 2) the average profit of individual participants by 42%, respectively.

Content Caching-Enhanced Computation Offloading in Mobile Edge Service Networks

Cache enhanced computation offloading as a novel offloading paradigm in mobile edge computing (MEC) can reduce more task execution latency than traditional computation offloading by reusing of computation offloading data. However, existing works only focus on the enhancement between computation offloading and data caching but ignore the competition for cache resources between them. To this end, in this paper, we propose a caching enhanced computation offloading algorithm in mobile edge service networks (MESN), by considering the cache resources competition. We formulate a joint optimization problem of content caching and cache-enhanced computation offloading. Furthermore, we give the optimal caching strategy to achieve the equilibrium between the resources competition. By our offloading algorithm caching strategy, the average response time of computation and content request tasks can get further reduction. In addition, we design two low time complexity algorithms, i.e., mixed caching algorithm and enhanced offloading algorithm, to solve the sub-problems, i.e., smart base station (SBS) caching sub-problem and computation offloading sub-problem, transformed by the original optimization problem. The simulation results show that our algorithms can quickly converge and our scheme can reduce 20.52% average response time of all tasks at most compared with other schemes.

Collaborative Resource Sharing Game Based Cloud-Edge Offload Computing Orchestration Scheme

In the foreseeable future, the rapid growth of mobile communications and cloud systems has substantially promoted edge computing paradigm. Although edge computing technology has been attracting much interest, most current research is application-specific without considering a control perspective of cloud providers that provides general-purpose edge services. In this study, we present a new cloud-edge computing orchestration scheme that integrates the different resource sharing problems to maximize benefits for offloading computing services. Specifically, we focus on two cooperative game solutions - <i>Sim bargaining solution</i> and <i>interval Banzhaf Value</i> - to effectively share the computing, communication and cache resources in the cloud-edge system platform. In the proposed scheme, we formalize two different cooperative games, and they work together and act cooperatively to satisfy contradictory requirements. Under widely different service request situations, we aim at accelerating an efficient resource orchestration through dynamic workload balancing. The primary goal of our cooperative game approach is to find optimal control decisions toward the appropriate system operation. Finally, we evaluate the proposed scheme in terms of several performance criteria from the numerical simulations. Experimental results show that we can achieve a mutually desirable solution with a good system performance balance while enhancing the resource efficiency better than the existing cloud-edge control protocols.

COCP: Coupling Parameters Content Placement Strategy for In-Network Caching-Based Content-Centric Networking

On-path caching is the prominent module in Content-Centric Networking (CCN), equipped with the capability to handle the demands of future networks such as the Internet of Things (IoT) and vehicular networks. The main focus of the CCN caching module is data dissemination within the network. Most of the existing strategies of in-network caching in CCN store the content at the maximum number of routers along the downloading path. Consequently, content redundancy in the network increases significantly, whereas the cache hit ratio and network performance decrease due to the unnecessary utilization of limited cache storage. Moreover, content redundancy adversely affects the cache resources, hit ratio, latency, bandwidth utilization, and server load. We proposed an in-network caching placement strategy named Coupling Parameters to Optimize Content Placement (COCP) to address the content redundancy and associated problems. The novelty of the technique lies in its capability to minimize content redundancy by creating a balanced cache space along the routing path by considering request rate, distance, and available cache space. The proposed approach minimizes the content redundancy and content dissemination within the network by using appropriate locations while increasing the cache hit ratio and network performance. The COCP is implemented in the simulator (Icarus) to evaluate its performance in terms of the cache hit ratio, path stretch, latency, and link load. Extensive experiments have been conducted to evaluate the proposed COCP strategy. The proposed COCP technique has been compared with the existing state-of-the-art techniques, namely, Leave Copy Everywhere (LCE), Leave Copy Down (LCD), ProbCache, Cache Less for More (CL4M), and opt-Cache. The results obtained with different cache sizes and popularities show that our proposed caching strategy can achieve up to 91.46% more cache hits, 19.71% reduced latency, 35.43% improved path stretch and 38.14% decreased link load. These results confirm that the proposed COCP strategy has the potential capability to handle the demands of future networks such as the Internet of Things (IoT) and vehicular networks.

Socially Driven Joint Optimization of Communication, Caching, and Computing Resources in Vehicular Networks

To support multifarious vehicular applications, content sharing among vehicles or between vehicles and infrastructures can enable efficient service provisioning. In this work, we investigate joint communication, caching, and computing (3C) resource allocation to support efficient content sharing between content providers (CPs) and content requesters (CRs) in vehicular networks. To tackle the high complexity of joint 3C resource allocation, we decouple the problem into a long-term content caching strategy to allocate caching resources plus a method for short-term CP-CR pairing and corresponding communication-computing resource allocation. Specifically, a popularity and social similarity (P-SS) based caching strategy is proposed by incorporating both physical and social information. As selfish CRs may refuse to reveal their quality of service (QoS) requirements to the CPs and lead to the information asymmetry, we adopt contract theory to allocate communication and computing resources for each potential CR-CP pair. We then propose a stable-matching based algorithm to match CPs and CRs for efficient content sharing. Simulation results verify that the proposed scheme can effectively solve the problem with low complexity.

Edge Learning for Internet of Medical Things and Its COVID-19 Applications: A Distributed 3C Framework

With the global outbreak of COVID-19, the Internet of Medical Things (IoMT), as an extension of the Internet of Things (IoT), has received increasing attention due to its ability to remotely monitor the main symptoms. In addition, based on widely collected IoMT data, the spread and even the origin of a pandemic can be traced effectively. However, spectrum resources become scarce with the increasing number of patients. Therefore, the effective allocation of communication, computing, and cache (3C) resources becomes increasingly important. In this article, we first study the unique characters of IoMT and construct a distributed 3C resource framework. Interactions of gateways in IoMT are specified and formed as an infinitely repeating game, where each gateway performs strategy selection based on probability learning at the network edge to reach a Nash equilibrium. A case study is provided to demonstrate the high efficiency of the designed 3C resource allocation strategy. Finally, we discuss the challenges and open issues of edge learning with IoMT from three aspects: real-time edge learning, 3C resource management, and privacy.

Harmonizing Artificial Intelligence with Radio Access Networks: Advances, Case Study, and Open Issues

Driven by the demands of efficient network operation and high service availability, the convergence of artificial intelligence (AI) with radio access networks (RANs) has drawn considerable attention. However, current academic research mainly focuses on applying AI into optimizing RANs with a few discussions on architecture design. This article surveys the recent progress achieved by industry in integrating AI into RANs, and proposes an AI-driven fog RAN (F-RAN) paradigm. Specifically, being wrappers of Al-re-lated functionalities, AI capsules are presented as new network functions in the F-RAN domain. With AI capsules, computation and cache resources at various fog nodes can be utilized to facilitate real-time AI-based F-RAN optimization and alleviate the transmission burden incurred by network data collection. At the edge cloud, a centralized AI brain for F-RANs is deployed, which incorporates a wireless-oriented auto-AI platform and a digital colon of the network environment for offline AI model training and evaluation. By the interplay among AI capsules and the AI brain, universal and endogenous intelligence can be fully realized within F-RANs, which in turn enhances system performance. Furthermore, we demonstrate the effectiveness of a scalable deep-reinforcement-learning-based method in minimizing energy consumption for a computation offloading use case. At last, open issues are identified in terms of interface standardization, federated learning, and transfer learning.