HFDRL: An Intelligent Dynamic Cooperate Cashing Method Based on Hierarchical Federated Deep Reinforcement Learning in Edge-Enabled IoT

Abstract

The Internet of Things (IoT) has significantly increased the number of terminals and network traffic. It is necessary to exploit the full capacity of the network and optimize content transfer. Despite the powerful processing and storage capabilities of base stations in 5G technology, edge caching effectively reduces content access time and duplicate traffic, thus optimizing content transfer for more storage resources. The limited memory resources and the dynamic nature of the requested content have necessitated the use of smart caching methods. Sending the required data to central servers can cause additional network overload and learning disabilities due to private data. For this reason, a hierarchical federated deep reinforcement learning (HFDRL) is proposed in this article that uses the FDRL method to predict the user’s future requests and to determine the appropriate content replacement strategy. In addition, to perform learning and collaborate caching, the method by which partner devices are determined plays a key role in edge caching performance. HFDRL categorizes edge devices hierarchically, thus avoids the disadvantages of very small or large clusters, takes advantage of both. By minimizing the redundancy of content storage and latency, HFDRL improves the performance for each local base station network individually and the global network. Simulation results of the proposed method show that the hit rate and delay have improved, respectively, by an average of 55% and 67% compared to traditional methods, 40% and 56% compared to the collaborative method, and 14% and 15% compared to one-level FDRL without using hierarchical edge devices clustering.