Double Deep Q-Network Based Dynamic Framing Offloading in Vehicular Edge Computing

Abstract

With the rapid development of Artificial Intelligence (AI) and the Internet of Vehicles (IoV), there is an increasing demand for deploying various intelligent applications on vehicles. Vehicular Edge Computing (VEC) is receiving extensive attention from both the industry and academia due to its benefits from the edge computing paradigm, which pushes computing tasks from the core of the network to the edge of the network. However, in the VEC environment considering vehicles to Road Side Units (RSUs), due to the mobility of vehicles, it is still a challenge to make dynamic and efficient offloading decisions for compute-intensive tasks, especially in the congestion situation. In order to minimize the total delay and waiting time of tasks from moving vehicles, we establish a dynamic offloading model for multiple moving vehicles whose tasks can be divided into sequential subtasks, so that the offloading decisions are more refined. Moreover, the proposed model is frame-based to avoid unnecessary waiting time, which makes offloading decisions when the subtasks of each vehicle are generated rather than offloading subtasks after gathering subtasks of vehicles for a time slot. Aiming to find the optimal offloading decision for sequential subtasks, we propose a Dynamic Framing Offloading algorithm based on Double Deep Q-Network (DFO-DDQN). Extensive experimental results demonstrate the effectiveness and superiority of the proposed DFO-DDQN when compared with other DRL-based methods and greedy-based methods.