Meta-Reinforcement Learning Based Resource Allocation for Dynamic V2X Communications

Yi Yuan,Gan Zheng,Kai-Kit Wong,Khaled B Letaief

doi:10.1109/tvt.2021.3098854

Abstract

This paper studies the allocation of shared resources between vehicle-to-infrastructure (V2I) and vehicle-to-vehicle (V2V) links in vehicle-to-everything (V2X) communications. In existing algorithms, dynamic vehicular environments and quantization of continuous power become the bottlenecks for providing an effective and timely resource allocation policy. In this paper, we develop two algorithms to deal with these difficulties. First, we propose a deep reinforcement learning (DRL)-based resource allocation algorithm to improve the performance of both V2I and V2V links. Specifically, the algorithm uses deep Q-network (DQN) to solve the sub-band assignment and deep deterministic policy-gradient (DDPG) to solve the continuous power allocation problem. Second, we propose a meta-based DRL algorithm to enhance the fast adaptability of the resource allocation policy in the dynamic environment. Numerical results demonstrate that the proposed DRL-based algorithm can significantly improve the performance compared to the DQN-based algorithm that quantizes continuous power. In addition, the proposed meta-based DRL algorithm can achieve the required fast adaptation in the new environment with limited experiences.

Highlights

Vehicle-to-everything (V2X) communications have been recognized as a key technology to support the safe and efficient intelligent transportation services [1]
We propose a joint deep reinforcement learning (DRL)-based algorithm to achieve the optimal selection on sub-band and power for the V2X communication scenario, in which V2V communication links need to share the sub-band resources with V2I links, and propose a meta-based DRL algorithm to improve the adaptation ability in dynamic environments
Since each V2I link is preassigned a sub-band with fixed transmit power and the number of V2I links equals to the number of sub-bands, we focus on solving the sub-band assignment and the power allocation for V2V links

Summary

Introduction

Vehicle-to-everything (V2X) communications have been recognized as a key technology to support the safe and efficient intelligent transportation services [1]. In [4], Sun et al propose a radio resource management (RRM) algorithm to control the intracell interference and achieve the latency and reliability requirements of the D2D-based V2X system. Based on the similar V2X framework, a three-stage RRM algorithm is proposed to solve the similar optimization problem under the condition that the spectrum is shared between the V2I and V2V and among different V2V pairs [5]. A spectrum sharing and power allocation design is investigated under the condition that only the slowly varying largescale fading channel is considered to maximize the sum ergodic capacity of V2I links while satisfying reliability of V2V links in a D2D-enabled vehicular system [6]. A graph partition algorithm is exploited to control the interference caused by V2V links in order to satisfy the quality-of-service (QoS) requirements for V2I and V2V links, respectively [7]. The impacts of the queueing latency on the throughput and reliability are investigated in [8] and [9]

Objectives

Results

Conclusion