A Deep-Q Learning Scheme for Secure Spectrum Allocation and Resource Management in 6G Environment

Abstract

In this paper, we propose a dynamic spectrum allocation (DSA) scheme DeepBlocks at the backdrop of sixth-generation (6G) communication networks that address the challenges of fixed spectrum allocations (FSA). The scheme exploits the advantages of deep-Q-network (DQN) and minimizes the search state explosion through a reward-penalty framework. A dynamic allocation of unallocated resource blocks (RBs) to mobile units (MUs) is carried out and once the allocation of RBs is complete, we integrate blockchain (BC) to record the transactional ledgers. The resource usage of MUs is recorded through smart contracts (SCs). We model the proposed scheme as a convex optimization problem, and subproblems are decomposed into a Pareto-optimal solution via Techebyecheff decomposition. In the simulation, we compare our scheme against FSA, and fifth-generation (5G) based DSA schemes like reinforcement learning (RL), deep neural networks (DNN)-based, and duelling DQN based schemes. The comparative analysis of 6G-DQN is modeled in terms of reward formulation, scalability of 6G-DQN-assisted DSA, and profit scenarios of BC-based allocation through intelligent channel control. The scheme proposes significant findings, with the best fit learning rate of 0.0001, and takes 500 episodes to converge to 60 total resource blocks. The servicing latency of the scheme is 272.4 ms, compared to 2010 ms in the duelling DQN approach. In spectrum allocation, an improvement of 26.32% is observed against non-DQN approaches, and 13.57% in the fairness parameter for spectrum allocation due to BC inclusion. The findings present the scheme efficacy for DSA over the aforementioned conventional approaches.