A multi-channel and multi-user dynamic spectrum access algorithm based on deep reinforcement learning in Cognitive Vehicular Networks with sensing error

Abstract

In this paper, a spectrum access problem is proposed to improve the spectrum access rates of secondary vehicles in Cognitive Vehicular Networks, where the channel capacity mathematic model is established under the conditions of spectrum sensing errors rates and the dynamic occupancy spectrum rates. Meanwhile, an improved Q-learning method is proposed to conform the dynamic communication under the different conditions of the reward functions. In this function, a Deep Q Network method with a modified reward function (IDQN) is proposed to deal with the situation of multi-vehicle in multi-channel. In order to verify the effectiveness of the IDQN method, the Myopic method, the improved Q-learning method, and the traditional DQN method are compared on Python. The simulation results shown that the proposed IDQN method not only outperforms the compared methods in terms of channel utilization and channel capacity but also improves the ability that the vehicle adapts to the dynamic communication environment.