Joint spectrum allocation and power control in vehicular communications based on dueling double DQN

Abstract

Vehicular communications is one of the important applications of 5G communication systems. The allocation scheme of scarce wireless resources will directly affect the quality of communication and the utilization of resources. Traditional resource allocation methods in wireless networks have the disadvantages of difficulty in expressing optimization problems and high computational complexity. Therefore, machine learning (ML) is used to solve these problems. However, ML also suffers from curse of dimensionality, action redundancy, and overestimation. In this paper, a distributed Vehicle-to-Vehicle (V2V) spectrum allocation and power control scheme based on the Deep Reinforcement Learning (DRL) framework is proposed. First, the latency constraint of the V2V link is formulated as a reward function, and a positive reward is given when the constraint is not violated; then, each V2V link is regarded as an agent to realize distributed spectrum and transmission power allocation to ensure the minimum transmission overhead. We use Double Deep Q-learning (DDQN) with dueling architecture to find the mapping between the observed environment and the optimal resource allocation scheme. Compared with other algorithms, the proposed algorithm can learn to meet the strict latency constraints on the V2V link, and can effectively improve the network capacity of the Vehicle-to-Infrastructure (V2I) link in vehicular communications.