A Dynamic Resource Allocation Model Based on SMDP and DRL Algorithm for Truck Platoon in Vehicle Network

Abstract

The rapid development of self-driving cars and breakthroughs in key technologies have made the truck platoon possible. In addition to reducing truck fuel consumption and air pollution by reducing air resistance, effective platoon strategies can also maximize highway throughput while improving driving safety. However, the truck platoon strategy’s current resource allocation model is still in the preliminary research stage. Therefore, inspired by the successful experience of deep reinforcement learning (DRL) in solving resource allocation problems, this article proposes a dynamic resource allocation model for the truck platoon based on the semi-Markov decision process (SMDP) and DRL, which is used to maximize system revenue when considering the resource cost and income balance of the transportation system. Precisely, the proposed method first models the process of controlling the dynamic in and out of the truck platoon as SMDP. The action value in a specific state obtained by the planning algorithm is used as a DRL sample for model training. Finally, the SMDP is optimized through the trained model to obtain a truck platoon resource that approximates the optimal strategy distribution plan. The experimental results show that compared with the traditional greedy algorithm, value iteration, and <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">${Q}$ </tex-math></inline-formula> -learning scheme concerning solving the dynamic resource allocation model of the truck platoon, the Deep <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">${Q}$ </tex-math></inline-formula> -Network (DQN) used in this article can reduce the probability of request processing delay while causing the system to obtain higher rewards.