Content Delivery Problem Research Articles

Intelligent multimedia content delivery in 5G/6G networks: A reinforcement learning approach

AbstractMultimedia content in 5G/6G networks makes safe, confidential, and efficient content delivery difficult. Intelligent systems that adapt to the ever‐changing network environment are needed to distribute multimedia content in these networks. Reinforcement learning (RL) can optimize multimedia content distribution based on network congestion, capacity, and user preferences. This study proposes RL‐based intelligent multimedia content distribution. RL algorithms learn from the network environment and generate optimum judgments incorporating several aspects of the suggested framework. The framework delivers multimedia material securely and privately with great quality. This study provides an intelligent multimedia content delivery architecture that uses RL approaches to solve 5G/6G content delivery problems. This research presents an RL system optimized with the double DQN algorithm having a reward of 51604.93 in 7000 episodes for efficient video file sharing on intracity buses. The RL agent balances network congestion and bandwidth by leveraging multiple sources such as bus and intersection caches and base stations, improving secure multimedia content delivery in 5G/6G networks and enhancing the passenger experience. The study confirms the system's effectiveness using reward and loss metrics and identifies potential future research directions. Future work could explore additional RL algorithms, scalability for larger networks, complex delivery scenarios, and integration with blockchain and edge computing for improved security and efficiency in multimedia content delivery.

VeSoNet: Traffic-Aware Content Caching for Vehicular Social Networks Using Deep Reinforcement Learning

Vehicular social networking is an emerging application of the Internet of Vehicles (IoV) which aims to achieve seamless integration of vehicular networks and social networks. However, the unique characteristics of vehicular networks, such as high mobility and frequent communication interruptions, make content delivery to end-users under strict delay constraints extremely challenging. In this paper, we propose a social-aware vehicular edge computing architecture that solves the content delivery problem by using some vehicles in the network as edge servers that can store and stream popular content to close-by end-users. The proposed architecture includes three main components: 1) the proposed social-aware graph pruning search algorithm computes and assigns the vehicles to the shortest path with the most relevant vehicular content providers. 2) the proposed traffic-aware content recommendation scheme recommends relevant content according to its social context. This scheme uses graph embeddings in which the vehicles are represented by a set of low-dimension vectors (vehicle2vec) to store information about previously consumed content. Finally, we propose a deep reinforcement learning (DRL) method to optimise the content provider vehicle distribution across the network. The results obtained from a real-world traffic simulation show the effectiveness and robustness of the proposed system when compared to the state-of-the-art baselines.

Proactive Caching in D2D Assisted Multitier Cellular Network.

Cache-enabled networks suffer hugely from the challenge of content caching and content delivery. In this regard, cache-enabled device-to-device (D2D) assisted multitier cellular networks are expected to relieve the network data pressure and effectively solve the problem of content placement and content delivery. Consequently, the user can have a better opportunity to get their favored contents from nearby cache-enabled transmitters (CETs) through reliable and good-quality links; however, as expected, designing an effective caching policy is a challenging task due to the limited cache memory of CETs and uncertainty in user preferences. In this article, we introduce a joint content placement and content delivery technique for D2D assisted multitier cellular networks (D2DMCN). A support vector machine (SVM) is employed to predict the content popularity to determine which content is to be cached and where it is to be cached, thereby increasing the overall cache hit ratio (CHR). The content request is satisfied either by the neighboring node through the D2D link or by the cache-enabled base stations (BSs) of the multitier cellular networks (MCNs). Similarly, to solve the problem of optimal content delivery, the Hungarian algorithm is employed aiming to improve the quality of satisfaction. The simulation results indicate that the proposed content placement strategy effectively optimizes the overall cache hit ratio of the system. Similarly, an effective content delivery approach reduces the request content delivery delay and power consumption.

A Joint Reinforcement-Learning Enabled Caching and Cross-Layer Network Code in F-RAN With D2D Communications

In this paper, we leverage reinforcement learning (RL) and cross-layer network coding (CLNC) for efficiently pre-fetching requested contents to the local caches and delivering these contents to requesting users in a downlink fog-radio access network (F-RAN) with device-to-device (D2D) communications. In the considered system, fog access points (F-APs) and cache-enabled D2D (CE-D2D) users are equipped with local caches that alleviate traffic burden at the fronthaul and facilitate rapid delivery of the users’ contents. To this end, the CLNC scheme optimizes the coding decisions, transmission rates, and power levels of both F-APs and CE-D2D users, and RL scheme optimizes caching strategy. A joint content placement and delivery problem is formulated as an optimization problem with a goal to maximize system sum-rate. The problem is an NP-hard problem. To efficiently solve it, we first develop an innovative decentralized CLNC coalition formation (CLNC-CF) switch algorithm to obtain a stable solution for the content delivery problem, where F-APs and CE-D2D users utilize CLNC resource allocation. By considering statistics of channel and users’ content request into account, we then develop a multi-agent RL algorithm for optimizing the content placement at both F-APs and CE-D2D users. Simulation results show that the proposed joint CLNC-CF-RL framework can effectively improve the sum-rate by up to 30%, 60%, and 150%, respectively, compared to: 1) an optimal uncoded algorithm, 2) a standard rate-aware-NC algorithm, and 3) a benchmark classical NC with network-layer optimization.

Delay-Aware Content Delivery With Deep Reinforcement Learning in Internet of Vehicles

The rapid development of the Internet of Vehicles (IoV) enables various vehicular applications, such as image-aided navigation and traffic information management. It is important to provide efficient content delivery services for these vehicular applications. Caching popular content at roadside units (RSUs) is a promising way to improve content delivery efficiency. However, due to RSUs with limited cache space, it is very challenging to develop an effective content delivery policy that satisfies the high quality of service (QoS) requirements for vehicular applications. In this paper, we investigate the user-centric content delivery problem with service delay constraints in the IoV, where the objective is to minimize the vehicle’s cost under usage-based pricing. The problem of finding an optimal content delivery policy is modeled as a finite-horizon Markov decision process (MDP). Since the cache state of each RSU, and the wireless channel qualities between the vehicle and RSUs, are usually unknown to the vehicle a priori, the vehicle must learn the optimal delivery policy by interacting with the environment. To solve this problem and optimize the vehicle’s cost, we propose a double deep Q network (DDQN)-based algorithm, which implements dynamic content delivery decisions. Furthermore, the double deep Q network can overcome the large-scale state space and reduce Q value over-estimation. Numerical results show that our policy achieves a near-optimal performance when compared to the optimal policy that knows precisely cache state and wireless channel state. We also compare the effects of different caching strategies and vehicle mobility on the performance of the algorithm.

Topological Content Delivery With Feedback and Random Receiver Cache

We study the problem of content delivery in two-user interference channels with altering topology, random available cache at the receivers, and delayed channel knowledge at the transmitter. We establish a new set of outer-bounds on the achievable rates when each receiver has access to a random fraction of the message intended for the other receiver, and when each transmitter is aware of which part of its own message is known to the unintended receiver. The outer-bounds reveal the significant potential rate boost associated with even a small amount of side-information at each receiver. The key in deriving the bounds is to quantify the baseline entropy that will always become available to the unintended receiver given the altering topology and the already available side-information. We will also present matching achievable rates in certain scenarios and outline the challenges in more general settings.

Content Delivery Over Broadcast Erasure Channels With Distributed Random Cache

We study the content delivery problem between a transmitter and two receivers through erasure links, when each receiver has access to some random side-information about the files requested by the other user. The random side-information is cached at the receiver via the decentralized content placement. The distributed nature of receiving terminals may also make the erasure state of two links and indexes of the cached bits not perfectly known at the transmitter. We thus investigate the capacity gain due to various levels of availability of channel state and cache index information at the transmitter. More precisely, we cover a wide range of settings from global delayed channel state knowledge and a non-blind transmitter (i.e. one that knows the exact cache index information at each receiver) all the way to no channel state information and a blind transmitter (i.e. one that only statistically knows cache index information at the receivers). We derive new inner and outer bounds for the problem under various settings and provide the conditions under which the two match and the capacity region is characterized. Surprisingly, for some interesting cases the capacity regions are the same even with single-user channel state or single-user cache index information at the transmitter.

Joint Distributed Cache and Power Control in Haptic Communications: A Potential Game Approach

Since haptic communications have an extreme requirement for the latency performance, the overdue content delivery will decrease the application experience. In order to reduce the transmission delay, we try to construct a cache-enabled D2D-assisted content-sharing framework, where the neighboring helpers preset contents and adjust their transmission power to timely serve more requesters. In particular, we modify the existing definition of the “closest” friend and take the social relationship into account to measure the overall influence caused by available neighboring helpers. However, such cooperation consumes the storage space and energy of the helpers, and accordingly, we introduce blockchain technology to propose an effective incentive mechanism where the helpers act as consensus users of blockchain and obtain the reward, i.e., the allocated computing power by the base station (BS), by contributing their local resources. Guided by this, the benefit of each helper is defined as a tradeoff of the reward and overheads. Meanwhile, we formulate the multiuser content delivery problem with the goal of maximizing the global benefits as the multiuser content delivery game. Then, we prove that the proposed game is an exact potential game (EPG) with at least one pure-strategy Nash equilibrium (NE). Meanwhile, the context-aware content delivery-based concurrent better reply (CCDCBR) algorithm is proposed to achieve a desirable solution. Finally, simulation results verify the validity of the proposed game model as well as the proposed algorithm.

Deep convolutional neural networks for data delivery in vehicular networks

In vehicular networks, most content delivery schemes only utilize vehicle cooperation or powerful infrastructure to satisfy data requests. How to fully utilize vehicle-to-vehicle and vehicle-to-infrastructure communications to improve data acquisition still requires further analysis. In this paper, the content delivery problem is formulated as a maximum flow of a directed network, which implies the encounters and the requests. Despite of a high delivery ratio, the proposed Content delivery scheme using mAximum Flow (CAF) is infeasible in large-scale real-time applications due to high computational complexity. To solve this problem, we transform the GPS trajectory data into two-dimensional coverage grid maps which indicate the communication opportunities between vehicles and infrastructures in CAF. The map set, which consists of coverage grid maps in a storage cycle, and the number of satisfied requests obtained from CAF compose the training set that can be trained by the deep convolutional neural networks. This solution combining CAF with deep neural networks is called CAF-Net. In the experiments, we evaluate the performances of four popular architectures of deep convolutional neural networks when outputting the targets. The results show that ResNet 50 has the smallest error and the computation time of a delivery ratio is only 82.84 ms, which is a lot shorter than 4531.53 s using CAF. The results also demonstrate the feasibility of applying the deep learning framework to vehicular networks.

Content delivery enhancement in Vehicular Social Network with better routing and caching mechanism

Vehicular Social Network (VSN) emerges as one meaningful scenario of Mobile Social Network (MSN) and aims at addressing the traffic congestion and security problems in smart cities. However, the current IP based network connection cannot efficiently solve the high mobility and dynamically changing topology issues caused by VSN, which results in long routing latency and low cache hit ratio. In this regard, this work intends to optimize the routing latency and the cache hit ratio by solving the content delivery problem in VSN, during which the concepts of fog/edge computing and Information-Centric Networking (ICN) are introduced, since they can provide a new perspective of network connection in VSN by concentrating on contents instead of IP addresses. Firstly, the ICN routing table structure is expanded for the purpose of providing key routing basis for both interest and data packets. Secondly, the vehicle social information is fully explored from the points of relationship strength and interest degree to offer reliable data foundation for community detection and packet forwarding. Thirdly, the dynamic vehicle communities are constructed based on an improved community overlap propagation algorithm. By monitoring the content popularity and cache distribution in the dynamic communities, this paper proposes the vehicle-adaptive caching scheme which guarantees the cache diversity and improves the packet delivery rate. After that, a vehicle-adaptive interest packet routing scheme is proposed based on relationship strength and interest degree for both inter/intra communities. The experiment results indicate that the proposed mechanism can improve the network performance in terms of the packet delivery rate, the average hop, the average delay and the overhead.

UAV-Assisted Content Delivery in Intelligent Transportation Systems-Joint Trajectory Planning and Cache Management

Unmanned Aerial Vehicles (UAVs) are gaining growing interests due to the paramount roles they play, particularly these days, in enabling new services that help modernize our transportation, supply chain, search and rescue, among others. They are capable of positively influencing wireless systems through enabling and fostering emerging technologies such as autonomous driving, vertical industries, virtual reality and so many others. The Internet of Vehicles is a prime sector benefiting from the services offered by future cellular systems in general and UAVs in particular, and this paper considers the problem of content delivery to vehicles on road segments with either overloaded or no available communication infrastructure. Incoming vehicles demand service from a library of contents that is partially cached at the UAV; the content of the library is also assumed to change as new vehicles carrying more popular contents arrive. Each inbound vehicle makes a request and the UAV decides on its best trajectory to provide service while maximizing a certain operational utility. Given the energy limitation at the UAV, we seek an energy efficient solution. Hence, our problem consists of jointly finding caching decisions, UAV trajectory and radio resource allocation which is formulated mathematically as a Mixed Integer Non-Linear Problem (MINLP). However, owing to uncertainties in the environment (e.g., random arrival of vehicles, their requests for contents and their existing contents), it is often hard and impractical to solve using standard optimization techniques. To this end, we formulate our problem as a Markov Decision Process (MDP) and we resort to tools such as Proximal Policy Optimization (PPO), a very promising Reinforcement Learning method, along with a set of crafted algorithms to solve our problem. Finally, we conduct simulation-based experiments to analyze and demonstrate the superiority of our solution approach compared with four counterparts and baseline schemes.

Content-Centric Heterogeneous Fog Networks Relying on Energy Efficiency Optimization

Next-generation wireless communications are expected to provide reliable high-rate connectivity. The recent concept of fog-enabled architecture comes to rescue by invoking a substantial amount of data storage close to end-user devices for meeting these challenges. Thus, caching popular contents at heterogeneous devices, e.g., fog nodes (FNs) or fog access points (FAPs), constitutes a promising technique of reducing both the traffic and the energy consumption of the backhaul links. Therefore, in this paper, we propose an energy-efficient caching and node association algorithm for cache-aided fog networks. First, we solve the problem of energy-efficient content caching and delivery in the FNs/FAPs in conjunction with a fixed node association strategy, where the FNs communicate either with other FNs by node-to-node (N2N) communications or with the FAPs in their proximity. In both caching scenarios, we investigate the relationship between the caching probability of the file and the energy-efficient content delivery by formulating the associated energy efficiency (EE) optimization problem under caching memory constraints. Then, we derive a joint modulation mode allocation strategy and caching policy for each content caching node and conceive a joint node association and caching algorithm for maximizing the EE. Finally, we quantify both the overall EE and throughput for demonstrating that the proposed caching and transmission strategy achieves significant performance improvements.

A Reinforcement Learning-Based User-Assisted Caching Strategy for Dynamic Content Library in Small Cell Networks

This paper studies the problem of joint edge cache placement and content delivery in cache-enabled small cell networks in the presence of spatio-temporal content dynamics unknown a priori . The small base stations (SBSs) satisfy users’ content requests either directly from their local caches, or by retrieving from other SBSs’ caches or from the content server. In contrast to previous approaches that assume a static content library at the server, this paper considers a more realistic non-stationary content library, where new contents may emerge over time at different locations. To keep track of spatio-temporal content dynamics, we propose that the new contents cached at users can be exploited by the SBSs to timely update their flexible cache memories in addition to their routine off-peak main cache updates from the content server. To take into account the variations in traffic demands as well as the limited caching space at the SBSs, a user-assisted caching strategy is proposed based on reinforcement learning principles to progressively optimize the caching policy with the target of maximizing the weighted network utility in the long run. Simulation results verify the superior performance of the proposed caching strategy against various benchmark designs.

DACON: A Novel Traffic Prediction and Data-Highway-Assisted Content Delivery Protocol for Intelligent Vehicular Networks

Nowadays, to deal with driving safety-related issues and improve travel comfort, the V ehicul A r NET work (VANET) has gained tremendous attention from researchers in both academia and industry around the world. By taking advantage of vehicle-to-vehicle (V2V) and vehicle-to-infrastructure (V2I) communications, the VANET can significantly enhance road safety and travel comfort by improving drivers’ awareness of their surrounding road environment and providing entertainment-related data service for passengers, respectively. However, due to the highly dynamic nature of the network topology in VANET, how to achieve reliable data transmission and content delivery is a critical task for implementing VANETs. Accordingly, in this article, we provide a novel data-highway-assisted content delivery protocol for addressing the content delivery problem in VANETs, in which, we explore the advantages of the predicted vehicular traffic volume driven by a newly designed fast traffic flow prediction scheme. We evaluate the performance of the proposed traffic flow prediction scheme by using three different data sets with different vehicles traffic flow patterns are chosen from the England Highways data set. Moreover, extensive simulations have been implemented to evaluate the proposed content delivery protocol.

Deep Reinforcement Learning Approaches for Content Caching in Cache-Enabled D2D Networks

Internet of Things (IoT) technology suffers from the challenge that rare wireless network resources are difficult to meet the influx of a huge number of terminal devices. Cache-enabled device-to-device (D2D) communication technology is expected to relieve network pressure with the fact that the requesting contents can be easily obtained from nearby users. However, how to design an effective caching policy becomes very challenging due to the limited content storage capacity and the uncertainty of user mobility pattern. In this article, we study the jointly cache content placement and delivery policy for the cache-enabled D2D networks. Specifically, two potential recurrent neural network approaches [the echo state network (ESN) and the long short-term memory (LSTM) network] are employed to predict users’ mobility and content popularity, so as to determine which content to cache and where to cache. When the local cache of the user cannot satisfy its own request, the user may consider establishing a D2D link with the neighboring user to implement the content delivery. In order to decide which user will be selected to establish the D2D link, we propose the novel schemes based on deep reinforcement learning to implement the dynamic decision making and optimization of the content delivery problems, aiming at improving the quality of experience of overall caching system. The simulation results suggest that the cache hit ratio of the system can be well improved by the proposed content placement strategy, and the proposed content delivery approaches can effectively reduce the request content delivery delay and energy consumption.

User-In-The-Loop Content Delivery in Cellular Communication Networks with Heterogeneous User Behaviors

User-in-the-loop (UiL) content delivery is a recently proposed scheme for personalized content retrieval over mobile communication networks. It is a promising scheme that can better manage the overall user quality-of-experience (QoE) throughout the entire content retrieval process. The performance of the scheme, however, has only been investigated in a simplified system model that assumed synchronized user behavior, inflexible delay constraint, and identical quality-of-service (QoS) requirement among users. This paper studies the performance of UiL content delivery scheme in a generalized and realistic system model with asynchronous user behavior, flexible outage delay constraint, and customized user QoS. Heuristic algorithms and theoretical bounds are investigated for the UiL content delivery problem. The proposed system is shown to be effective in managing the user QoE in generalized practical scenarios.

Explicit Content Caching at Mobile Edge Networks with Cross-Layer Sensing.

The deployment density and computational power of small base stations (BSs) are expected to increase significantly in the next generation mobile communication networks. These BSs form the mobile edge network, which is a pervasive and distributed infrastructure that can empower a variety of edge/fog computing applications. This paper proposes a novel edge-computing application called explicit caching, which stores selective contents at BSs and exposes such contents to local users for interactive browsing and download. We formulate the explicit caching problem as a joint content recommendation, caching, and delivery problem, which aims to maximize the expected user quality-of-experience (QoE) with varying degrees of cross-layer sensing capability. Optimal and effective heuristic algorithms are presented to solve the problem. The theoretical performance bounds of the explicit caching system are derived in simplified scenarios. The impacts of cache storage space, BS backhaul capacity, cross-layer information, and user mobility on the system performance are simulated and discussed in realistic scenarios. Results suggest that, compared with conventional implicit caching schemes, explicit caching can better exploit the mobile edge network infrastructure for personalized content dissemination.

Scalable Content Delivery With Coded Caching in Multi-Antenna Fading Channels

We consider the content delivery problem in a fading multi-input single-output channel with cache-aided users. We are interested in the scalability of the equivalent content delivery rate when the number of users, $K$ , is large. Analytical results show that, using coded caching and wireless multicasting, without channel state information at the transmitter, linear scaling of the content delivery rate with respect to $K$ can be achieved in some different ways. First, if the multicast transmission spans over $L$ independent sub-channels, e.g., in quasi-static fading if $L = 1$ , and in block fading or multi-carrier systems if $L>1$ , linear scaling can be obtained, when the product of the number of transmit antennas and the number of sub-channels scales logarithmically with $K$ . Second, even with a fixed number of antennas, we can achieve the linear scaling with a threshold-based user selection requiring only one-bit feedbacks from the users. When CSIT is available, we propose a mixed strategy that combines spatial multiplexing and multicasting. Numerical results show that, by optimizing the power split between spatial multiplexing and multicasting, we can achieve a significant gain of the content delivery rate with moderate cache size.

QoS Provisionings for Device-to-Device Content Delivery in Cellular Networks

Device-to-device (D2D) technique provides a feasible way for content delivery among proximal users without deploying additional infrastructures. For the realization of D2D content delivery, it is important to guarantee quality-of-service (QoS) for supporting real-time content transmissions. In this paper, we formulate the content delivery problem based on a flow model and investigate mechanisms providing delay QoS guarantee. The delay performance is analyzed for the content dissemination with the aid of D2D communication. Based on the analyses of parameters affecting the performance of content delivery, an optimal resource allocation scheme is proposed to achieve the best energy efficiency of content delivery in cellular networks with guaranteed delay constraints for hybrid content requests. Then effects of content caching on D2D content delivery is studied and a caching management scheme is proposed to avoid cache overflow of users with a certain probability and satisfy QoS requirements of content delivery. Simulation results are presented that verify the analyzed influencing factors on D2D content delivery and the proposed QoS-provisioning schemes.

Social Network-Based Content Delivery in Device-to-Device Underlay Cellular Networks Using Matching Theory

With the popularity of social network-based services, the unprecedented growth of mobile date traffic has brought a heavy burden on the traditional cellular networks. Device-to-device (D2D) communication, as a promising solution to overcome wireless spectrum crisis, can enable fast content delivery based on user activities in social networks. In this paper, we address the content delivery problem related to optimization of peer discovery and resource allocation by combining both the social and physical layer information in D2D underlay networks. The social relationship, which is modeled as the probability of selecting similar contents and estimated by using the Bayesian nonparametric models, is used as a weight to characterize the impact of social features on D2D pair formation and content sharing. Next, we propose a 3-D iterative matching algorithm to maximize the sum rate of D2D pairs weighted by the intensity of social relationships while guaranteeing the quality of service requirements of both cellular and D2D links simultaneously. Moreover, we prove that the proposed algorithm converges to a stable matching and is weak Pareto optimal, and also provide the theoretical complexity. Simulation results show that the algorithm is able to achieve more than 90% of the optimum performance with a computation complexity 1000 times lower than the exhaustive matching algorithm. It is also demonstrated that the satisfaction performance of D2D receivers can be increased significantly by incorporating social relationships into the resource allocation design.