A Deep-Reinforcement-Learning-Based Computation Offloading With Mobile Vehicles in Vehicular Edge Computing

Abstract

Vehicular edge networks involves edge servers that are close to mobile devices to provide extra computation resource to complete the computation tasks of mobile devices with low latency and high reliability. Considerable efforts on computation offloading in vehicular edge networks have been developed to reduce the energy consumption and computation latency, in which roadside units (RSUs) are usually considered as the fixed edge servers. Nonetheless, the computation offloading with considering mobile vehicles as mobile edge servers in vehicular edge networks still needs to be further investigated. To this end, in this paper, we propose a deep reinforcement learning based computation offloading with mobile vehicles in vehicular edge computing, namely DRL-COMV, in which some vehicles (such as autonomous vehicle) are deployed and considered as the mobile edge servers that move in vehicular edge networks and cooperate with fixed edge servers to provide extra computation resource for mobile devices, in order to assist in completing the computation tasks of these mobile devices with great QoE (i.e.,low latency) for mobile devices. Particularly, the computation offloading model with considering both mobile and fixed edge servers is conducted to achieve the computation tasks offloading through vehicle-to-vehicle (V2V) communications, and a collaborative route planning is considered for these mobile edge servers to move in vehicular edge networks with objective of improving efficiency of computation offloading. Then, a deep reinforcement learning approach with designing rational reward function is proposed to determine the effective computation offloading strategies for multiple mobile devices and multiple edge servers with objective of maximizing both QoE (i.e., low latency) for mobile devices. Through performance evaluations, our results show that our proposed DRL-COMV scheme can achieve a great convergence and stability. Additionally, our results also demonstrate that our DRL-COMV scheme also can achieve better both QoE and task offloading requests hit ratio for mobile devices in comparison with existing approaches (i.e., DDPG, IMOPSOQ and GABDOS).