A Survey on Reinforcement Learning-Aided Caching in Heterogeneous Mobile Edge Networks

Nikolaos Nomikos,Spyros Zoupanos,Themistoklis Charalambous,Ioannis Krikidis

doi:10.1109/access.2022.3140719

Nikolaos Nomikos, Spyros Zoupanos + Show 2 more

Open Access

https://doi.org/10.1109/access.2022.3140719

Copy DOI

Abstract

Mobile networks experience a tremendous increase in data volume and user density due to the massive number of coexisting users and devices. An efficient technique to alleviate this issue is to bring the data closer to the users by exploiting cache-aided edge nodes, such as fixed and mobile access points, and even user devices. Meanwhile, the fusion of machine learning and wireless networks offers new opportunities for network optimization when traditional optimization approaches fail or incur high complexity. Among the various machine learning categories, reinforcement learning provides autonomous operation without relying on large sets of historical data for training. In this survey, reinforcement learning-aided mobile edge caching solutions are presented and classified, based on the networking architecture and optimization target. As sixth generation (6G) networks will be characterized by high heterogeneity, fixed cellular, fog, cooperative, vehicular, and aerial networks are studied. The discussion of these works reveals that there exist reinforcement learning-aided caching schemes with varying complexity that can surpass the performance of conventional policy-based approaches. Finally, several open issues are presented, stimulating further interest in this important research field.

Highlights

Today, the wide commercial roll-out of fifth generation (5G) networks has become a reality, better supporting enhanced mobile broadband services, ultra-reliable and ultralow latency (URLLC) critical applications, and massive machine type communications in the context of the Internet-of-Things (IoT) [1], [2]
A technique facilitating the evolution to 6G communications is mobile edge computing (MEC) and caching, where computation-intensive tasks take place near data collection and popular contents are in close proximity
Edge caching represents a major shift in network architecture design, since content is brought closer to the users in an intelligent and proactive manner

Summary

INTRODUCTION

The wide commercial roll-out of fifth generation (5G) networks has become a reality, better supporting enhanced mobile broadband (eMBB) services, ultra-reliable and ultralow latency (URLLC) critical applications, and massive machine type communications (mMTC) in the context of the Internet-of-Things (IoT) [1], [2]. Traditional non-learning-based techniques might fail, due to the dynamic nature of wireless environment involving a large number of parameters and constraints, exhibiting prohibitive complexity for online network optimization. In such cases, ML-aided MEC and caching can exploit the plethora of mobile data and answer the questions of where, when and what to cache, as well as which tasks should be computed at the edge [8]–[10]. The majority of the reviewed works are non-learning-based Another survey studies DL for edge caching, presenting the major DL categories and caching principles [18].

MACHINE LEARNING

FIXED CELLULAR NETWORKS

COOPERATIVE NETWORKS

Findings

CONCLUSION