In recent years, a new technology called mobile edge computing (MEC) is proposed to alleviate mobile devices’ increasing demands on computing resources and service latency. However, users may stay away from the fixed MEC server or even leave the service scope of the MEC server during the mobile process, which will lead to service performance degradation or service interruption. Such an interruption is intolerable for vehicle in V2X. Therefore, it is necessary to consider the issue of service migration. In this paper, we model the service migration problem in V2X as a multidimensional Markov decision process problem and consider a variety of metrics including computing cost, migration cost and energy consumption. A novel user mobility model is also considered to accurately reflect the motion state of users in reality. Then, a novel service migration algorithm is proposed to solve this multidimensional MDP problem. Our proposed algorithm applies Actor-Critic and entropy to ensure both convergence and exportability. Experimental results show that our proposed algorithm outperforms the baselines and has relatively strong robustness.