Accelerated resource allocation based on experience retention for B5G networks

Abstract

The Beyond-Fifth-Generation (B5G) Wireless Communication Systems will require efficient resource allocation (RA) policies to fulfill future applications' increasing data rate and bandwidth demands. Reusing spectrum allows efficient use of the available spectrum, but at the same time, dynamic power allocation strategies are necessary for controlling interference among spectrum-sharing wireless devices. Recently, Deep Reinforcement Learning (DRL) approaches have been applied to find feasible resource allocation solutions demonstrating higher adaptability and robustness than iterative optimization algorithms. However, DRL-based models must constantly explore the network conditions to adapt to channel fluctuations during learning. Adjusting this model to the differences between the source and the new target model requires additional training time, and depending on its length, it may cause a system performance drop and hinder the DRL-based model's learning. One strategy to accelerate learning a new network environment is exploiting the generated knowledge from the already trained models. In this work, we focus on the Experience Replay (ER) retention mechanism to manage, during the learning, the knowledge that DRL models have about the wireless network environment. Our proposal transfers the parameters of a Deep Q-Network (DQN) model from the source to target environment to reduce the training time. We implement a Dual-Buffer (DB) retention mechanism to keep relevant experiences longer and increase the training samples' diversity to ease learning in new environments. This approach is evaluated in a B5G network to control the interference through optimal power transmission allocation. The experiment results show that our proposal stabilizes and accelerates the model learning compared to the standard FIFO experience replay (ER) retention mechanism reducing the training time and increasing the model's generalizability.