Time-varying Content Popularity Research Articles

Deep Reinforcement Learning for Edge Caching with Mobility Prediction in Vehicular Networks.

As vehicles are connected to the Internet, various services can be provided to users. However, if the requests of vehicle users are concentrated on the remote server, the transmission delay increases, and there is a high possibility that the delay constraint cannot be satisfied. To solve this problem, caching can be performed at a closer proximity to the user which in turn would reduce the latency by distributing requests. The road side unit (RSU) and vehicle can serve as caching nodes by providing storage space closer to users through a mobile edge computing (MEC) server and an on-board unit (OBU), respectively. In this paper, we propose a caching strategy for both RSUs and vehicles with the goal of maximizing the caching node throughput. The vehicles move at a greater speed; thus, if positions of the vehicles are predictable in advance, this helps to determine the location and type of content that has to be cached. By using the temporal and spatial characteristics of vehicles, we adopted a long short-term memory (LSTM) to predict the locations of the vehicles. To respond to time-varying content popularity, a deep deterministic policy gradient (DDPG) was used to determine the size of each piece of content to be stored in the caching nodes. Experiments in various environments have proven that the proposed algorithm performs better when compared to other caching methods in terms of the throughput of caching nodes, delay constraint satisfaction, and update cost.

Intelligent Caching for Mobile Video Streaming in Vehicular Networks with Deep Reinforcement Learning

Caching-enabled multi-access edge computing (MEC) has attracted wide attention to support future intelligent vehicular networks, especially for delivering high-definition videos in the internet of vehicles with limited backhaul capacity. However, factors such as the constrained storage capacity of MEC servers and the mobility of vehicles pose challenges to caching reliability, particularly for supporting multiple bitrate video streaming caching while achieving considerable quality of experience (QoE). Motivated by the above challenges, in this paper, we propose an intelligent caching strategy that takes into account vehicle mobility, time-varying content popularity, and backhaul capability to improve the QoE of vehicle users effectively. First, based on the mobile video mean opinion score (MV-MOS), we designed an average download percentage (ADP) weighted QoE evaluation model. Then, the video content caching problem is formulated as a Markov decision process (MDP) to maximize the ADP weighted MV-MOS. Owing to the prior knowledge of video content popularity and channel state information that may not be available at the road side unit in practical scenarios, we propose a deep reinforcement learning (DRL)-based caching strategy to solve the problem while achieving a maximum ADP weighted MV-MOS. To accelerate its convergence speed, we further integrate the prioritized experience replay, dueling, and double deep Q-network technologies, which improve the performance of DRL algorithm. Numerical results demonstrate that the proposed DRL-based caching strategy significantly improves QoE, and achieves better video delivery reliability compared to existing non-learning approaches.

Two Time-Scale Caching Placement and User Association in Dynamic Cellular Networks

With the rapid growth of data traffic in cellular networks, edge caching has become an emerging technology for traffic offloading. We investigate the caching placement and content delivery in cache-enabling cellular networks. To cope with the time-varying content popularity and user location in practical scenarios, we formulate a long-term joint dynamic optimization problem of caching placement and user association for minimizing the content delivery delay which considers both content transmission delay and content update delay. To solve this challenging problem, we decompose the optimization problem into two sub-problems, the user association sub-problem in a short time scale and the caching placement in a long time scale. Specifically, we propose a low complexity user association algorithm for a given caching placement in the short time scale. Then we develop a deep deterministic policy gradient based caching placement algorithm which involves the short time-scale user association decisions in the long time scale. Finally, we propose a joint user association and caching placement algorithm to obtain a sub-optimal solution for the proposed problem. We illustrate the convergence and performance of the proposed algorithm by simulation results. Simulation results show that compared with the benchmark algorithms, the proposed algorithm reduces the long-term content delivery delay in dynamic networks effectively.

CoPace: Edge Computation Offloading and Caching for Self-Driving With Deep Reinforcement Learning

Currently, self-driving, emerging as a key automatic application, has brought a huge potential for the provision of in-vehicle services (e.g., automatic path planning) to mitigate urban traffic congestion and enhance travel safety. To provide high-quality vehicular services with stringent delay constraints, edge computing (EC) enables resource-hungry self-driving vehicles (SDVs) to offload computation-intensive tasks to the edge servers (ESs). In addition, caching highly reusable contents decreases the redundant transmission time and improves the quality of services (QoS) of SDVs, which is envisioned as a supplement to the computation offloading. However, the high mobility and time-varying requests of SDVs make it challenging to provide reliable offloading decisions while guaranteeing the resource utilization of content caching. To this end, in this paper we propose a <underline xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">co</u> llaborative com <underline xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">p</u> utation offlo <underline xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">a</u> ding and <underline xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">c</u> ont <underline xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">e</u> nt caching method, named CoPace, by leveraging deep reinforcement learning (DRL) in EC for self-driving system. Specifically, we first introduce OSTP to predict the future time-varying content popularity, taking into account the temporal-spatial attributes of requests. Moreover, a DRL-based algorithm is developed to jointly optimize the offloading and caching decisions, as well as the resource allocation (i.e., computing and communication resources) strategies. Extensive experiments with real-world datasets in Shanghai, China, are conducted to evaluate the performance, which demonstrates that CoPace is both effective and well-performed.

Stochastic Game Based Cooperative Alternating Q-Learning Caching in Dynamic D2D Networks

Edge caching has become an effective solution to cope with the challenges brought by the massive content delivery in cellular networks. In device-to-device (D2D) enabled caching cellular networks with time-varying content popularity distribution and user terminal (UT) location, we model these dynamic networks as a stochastic game to design a cooperative cache placement policy. The cache placement reward of each UT is defined as the caching incentive minus the transmission power cost for content caching and sharing. We consider the long-term cache placement reward of all UTs in this stochastic game. In an effort to solve the stochastic game problem, we propose a multi-agent cooperative alternating Q-learning (CAQL) based cache placement algorithm. The caching control unit is defined to execute the proposed CAQL, in which, the cache placement policy of each UT is alternatively updated according to the stable policy of other UTs during the learning process, until the stable cache placement policy of all the UTs in the cell is obtained. We discuss the convergence and complexity of CAQL, which obtains the stable cache placement policy with low space complexity. Simulation results show that the proposed algorithm can effectively reduce the backhaul load and the average content access delay in dynamic networks.

PA-Cache: Evolving Learning-Based Popularity- Aware Content Caching in Edge Networks

As ubiquitous and personalized services are growing boomingly, an increasingly large amount of traffic is generated over the network by massive mobile devices. As a result, content caching is gradually extending to network edges to provide low-latency services, improve quality of service, and reduce redundant data traffic. Compared to the conventional content delivery networks, caches in edge networks with smaller sizes usually have to accommodate more bursty requests. In this article, we propose an evolving learning-based content caching policy, named PA-Cache in edge networks. It adaptively learns time-varying content popularity and determines which contents should be replaced when the cache is full. Unlike conventional deep neural networks (DNNs), which learn a fine-tuned but possibly outdated or biased prediction model using the entire training dataset with high computational complexity, PA-Cache weighs a large set of content features and trains the multi-layer recurrent neural network from shallow to deeper when more requests arrive over time. We extensively evaluate the performance of our proposed PA-Cache on real-world traces from a large online video-on-demand service provider. The results show that PA-Cache outperforms existing popular caching algorithms and approximates the optimal algorithm with only a 3.8% performance gap when the cache percentage is 1.0%. PA-Cache also significantly reduces the computational cost compared to conventional DNN-based approaches.

Dynamic cache management in content delivery networks

The importance of content delivery networks (CDN) continues to rise with the exponential increase in generation and consumption of electronic media. In order to ensure high quality of experience, CDNs often deploy cache servers that capable of storing some of the popular files close to users. Such edge caching solutions not only increase the content availability, but also result in higher download rates and lower latency. Different from the classical eviction-based algorithms, the present work formulates the content placement problem from an optimization perspective and puts forth an online algorithm for the same. In contrast to the existing optimization-based solutions, the proposed algorithm is incremental and incurs very low computation cost, while yielding storage allocations that are provably near-optimal. The proposed algorithm can handle time-varying content popularity, thereby obviating the need for periodically estimating demand distribution. Using synthetic and real IPTV data, we show that the proposed policies outperform the traditional eviction based techniques in terms of various metrics.

Online Spatiotemporal Popularity Learning via Variational Bayes for Cooperative Caching

Herein, we focus on an end-to-end design of a proactive cooperative caching strategy for a multi-cell network. The design is challenging as it involves two interrelated problems: the ability to predict future content popularity and to meet network operation characteristics. To this end, we first formulate a cooperative content caching in order to optimize the aggregated network cost for delivering contents to users. An efficient proactive caching policy requires an accurate prediction of time-varying content popularity. Content popularity has temporal and spatial dependencies and therefore, we develop a probabilistic dynamical model for content popularity prediction by exploiting its spatiotemporal correlations. To achieve an accurate tracking and prediction of content popularity evolution, the proposed dynamical model is non-linear and incorporates non-Gaussian distributions. We use Variational Bayes (VB) approach for estimating the model parameters. The VB provides mathematical tractability. We then develop an online VB method that works with streaming data where content request arrives sequentially. Using extensive simulations study on a real-world dataset, we show that our online VB based dynamical model provides improved performance compared to conventional content caching policies.

The Design of Dynamic Probabilistic Caching with Time-Varying Content Popularity

In this paper, we design dynamic probabilistic caching for the scenario when the instantaneous content popularity may vary with time while it is possible to predict the average content popularity over a time window. Based on the average content popularity, optimal content caching probabilities can be found, e.g., from solving optimization problems, and existing results in the literature can implement the optimal caching probabilities via static content placement. The objective of this work is to design dynamic probabilistic caching that: i) converge (in distribution) to the optimal content caching probabilities under time-invariant content popularity, and ii) adapt to the time-varying instantaneous content popularity under time-varying content popularity. Achieving the above objective requires a novel design of dynamic content replacement because static caching cannot adapt to varying content popularity while classic dynamic replacement policies, such as LRU, cannot converge to target caching probabilities (as they do not exploit any content popularity information). We model the design of dynamic probabilistic replacement policy as the problem of finding the state transition probability matrix of a Markov chain and propose a method to generate and refine the transition probability matrix. Extensive numerical results are provided to validate the effectiveness of the proposed design.

The Study of Dynamic Caching via State Transition Field—the Case of Time-Invariant Popularity

In the second part of this two-part paper, we extend the study of dynamic caching via state transition field (STF) to the case of time-varying content popularity. The objective of this part is to investigate the impact of time-varying content popularity on the STF and how such impact accumulates to affect the performance of a replacement scheme. Unlike the case in the first part, the STF is no longer static over time, and we introduce instantaneous STF to model it. Moreover, we demonstrate that many metrics, such as instantaneous state caching probability and average cache hit probability over an arbitrary sequence of requests, can be found using the instantaneous STF. As a steady state may not exist under time-varying content popularity, we characterize the performance of replacement schemes based on how the instantaneous STF of a replacement scheme after a content request impacts on its cache hit probability at the next request. From this characterization, insights regarding the relations between the pattern of change in the content popularity, the knowledge of content popularity exploited by the replacement schemes, and the effectiveness of these schemes under time-varying popularity are revealed. In the simulations, different patterns of time-varying popularity, including the shot noise model, are experimented. The effectiveness of example replacement schemes under time-varying popularity is demonstrated, and the numerical results support the observations from the analytic results.

The Study of Dynamic Caching via State Transition Field—the Case of Time-Varying Popularity

In the second part of this two-part paper, we extend the study of dynamic caching via state transition field (STF) to the case of time-varying content popularity. The objective of this part is to investigate the impact of time-varying content popularity on the STF and how such impact accumulates to affect the performance of a replacement scheme. Unlike the case in the first part, the STF is no longer static over time, and we introduce instantaneous STF to model it. Moreover, we demonstrate that many metrics, such as instantaneous state caching probability and average cache hit probability over an arbitrary sequence of requests, can be found using the instantaneous STF. As a steady state may not exist under time-varying content popularity, we characterize the performance of replacement schemes based on how the instantaneous STF of a replacement scheme after a content request impacts on its cache hit probability at the next request. From this characterization, insights regarding the relations between the pattern of change in the content popularity, the knowledge of content popularity exploited by the replacement schemes, and the effectiveness of these schemes under time-varying popularity are revealed. In the simulations, different patterns of time-varying popularity, including the shot noise model, are experimented. The effectiveness of example replacement schemes under time-varying popularity is demonstrated, and the numerical results support the observations from the analytic results.

Modeling and Optimization of Mobility-Aware Dynamic Caching With Time-Varying Content Popularity

A mobility-aware dynamic caching optimization problem (MDCP) is modelled for optimal caching when the contents’ popularity may vary over time. The $\mathcal {NP}$ -hardness of MDCP is proved. For the problem's solution, first, by providing a reformulation, the optimum of MDCP can be computed in up to medium-scale system scenarios. In addition, a time-efficient algorithm is developed for large-scale scenarios. The results of performance evaluation are provided.

Online Edge Caching and Wireless Delivery in Fog-Aided Networks With Dynamic Content Popularity

Fog radio access network (F-RAN) architectures can leverage both cloud processing and edge caching for content delivery to the users. To this end, F-RAN utilizes caches at the edge nodes (ENs) and fronthaul links connecting a cloud processor to ENs. Assuming time-invariant content popularity, existing information-theoretic analyses of content delivery in F-RANs rely on offline caching with separate content placement and delivery phases. In contrast, this paper focuses on the scenario in which the set of popular content is time-varying, hence necessitating the online replenishment of the ENs’ caches along with the delivery of the requested files. The analysis is centered on the characterization of the long-term normalized delivery time (NDT), which captures the temporal dependence of the coding latencies accrued across multiple time slots in the high signal-to-noise ratio regime. Online edge caching and delivery schemes are investigated for both serial and pipelined transmission modes across fronthaul and edge segments. Analytical results demonstrate that, in the presence of a time-varying content popularity, the rate of fronthaul links sets a fundamental limit on the long-term NDT of F-RAN system. Analytical results are further verified by numerical simulation, yielding important design insights.

Low complexity content replication through clustering in Content-Delivery Networks

Contemporary Content Delivery Networks (CDN) handle a vast number of content items. At such a scale, the replication schemes require a significant amount of time to calculate and realize cache updates, and hence they are impractical in highly-dynamic environments. This paper introduces cluster-based replication, whereby content items are organized in clusters according to a set of features, given by the cache/network management entity. Each cluster is treated as a single item with certain attributes, e.g., size, popularity, etc. and it is then altogether replicated in network caches so as to minimize overall network traffic. Clustering items reduces replication complexity; hence it enables faster and more frequent caches updates, and it facilitates more accurate tracking of content popularity. However, clustering introduces some performance loss because replication of clusters is more coarse-grained compared to replication of individual items. This tradeoff can be addressed through proper selection of the number and composition of clusters. Due to the fact that the exact optimal number of clusters cannot be derived analytically, an efficient approximation method is proposed. Extensive numerical evaluations of time-varying content popularity scenarios allow to argue that the proposed approach reduces core network traffic, while being robust to errors in popularity estimation.

Performance analysis of Content-Centric and Content-Delivery networks with evolving object popularity

The Internet is currently mostly exploited as a means to perform massive digital content distribution. Such a usage profile was not specifically taken into account while initially designing the architecture of the network: as a matter of fact, the Internet was instead conceived around the concept of host-to-host communications between two remote machines. To solve this problem, Content-Delivery Networks (CDNs) are currently used as a well-established technology to serve content-driven demands through an infrastructure that is not tailored for that purpose. On the other hand, the novel paradigm of Content-Centric Networking (CCN) aims at filling the gap of this misalignment by changing the network-layer protocols, solving the content-distribution problem at its root. In this paper, we formulate novel optimization models to analyze the performance gains that CDN and CCN can achieve, by reducing the total amount of traffic exchanged through the network. We tackle this problem by adopting a time-varying content popularity evolution model that accurately represents the dynamic behavior of users. We discover that, in most of the cases, CDN reduces the overall traffic exchanged between network nodes, leading to better performance, whereas CCN should instead be preferred in those scenarios where CDN cannot quickly react to popularity evolution. On top of that, we show that very limited benefits can be obtained by changing the cache replacement algorithms. Finally, all our key findings are confirmed by simulation campaigns that further complement this work.