MIDP: An MDP-based intelligent big data processing scheme for vehicular edge computing

Abstract

The number of Vehicle Equipment (VE) connected to the Internet is increasing, and these VEs generate tasks that contain large amounts of data. Processing these tasks requires a lot of computing resources. Therefore, it is a promising issue that offloading compute-intensive tasks from resource-limited vehicles to Vehicular Edge Computing (VEC) servers, which involves big data transmission, processing and computation. In a network, multiple providers provide VEC servers. When a vehicle generates a task, our goal is to make an intelligent decision on whether and when to offload this task to VEC servers to minimize the task completion time and total big data processing time. When each vehicle passes VEC servers, the vehicle can decide to offload its task to the VEC server in the current communication range, or continue to drive until it reaches the next server's communication range. This issue can be considered as an asset selling problem. It is a challenging issue to make a smart decision for the vehicle with a location view because the vehicle is not sure when the next VEC server will be available and how much about the available computing capacity of the next VEC server. Firstly, this paper formulates the problem as a Markov Decision Process (MDP), defines and analyzes the state set, action set, reward model, and state transition probability distribution. Then it uses Asynchronous Advantage Actor-Critic (A3C) algorithm to solve this MDP problem, builds the various elements of the A3C algorithm, uses Actor (the strategy function) to generate two actions of the vehicle: offloading and moving without offloading. Thirdly, it uses Critic (the value function) to evaluate Actor's behavior, and guide Actor's actions in subsequent stages. The Actor starts from the initial state in the state space until it enters the termination state, forming a complete decision-making process. It minimizes the completion time of task offloading through learning thereby reducing the delay of big data processing. Compared to the Immediately Offload (IO) scheme and Expect Offload (EO) scheme, the MIDP scheme proposed in this paper reduces the average task offloading delay to 29.93% and 29.99%, close to the EO scheme in terms of task completion rate and up to 66.6% improvement compared to the IO scheme.