DRL-Based Computation Offloading With Queue Stability for Vehicular-Cloud-Assisted Mobile Edge Computing Systems

Abstract

Computation offloading is considered as a promising method to improve computation performance of Intelligent Vehicles (IVs), where IVs can offload resource-hungry applications to Mobile Edge Computing (MEC) servers. However, due to the dynamic environment of Intelligent Transportation Systems, it's challenging to develop an effective offloading scheme that ensures stable operation of the system. Furthermore, IVs only offload tasks to MEC servers or parked vehicles, how to take full advantage of resources in a random traffic flow is a main challenge. In this paper, we propose a vehicular-cloud-assisted MEC network in urban cities to fully explore the idle resources and ensure the stability of task queue. Specifically, we develop a novel virtual platform based on deep neural network that predicts vehicles trajectory and creates vehicular cloud to integrate computation resources. We then design a vehicular-cloud-assisted offloading scheme that aims to maximize task throughput in continuous time slots subject to multi-constraint. To address the coupling among multiple variables and different time slots, we put forward a lightweight framework to apply the Lyapunov optimization in combination with deep reinforcement learning. In particular, we apply Lyapunov drift-plus-penalty approach to decompose the initial problem into per-slot subproblems and control the long-term task queue stability. To further reduce the computation complexity, we decouple the subproblem into three processes, and the deep Q-network is put forward to solve it. Extensive simulations demonstrate that under different task arrivals, our proposed scheme can achieve the task queue stability within 10 time slots with a low transmission power.