Personalized Car-Following Control Based on a Hybrid of Reinforcement Learning and Supervised Learning

Abstract

With the development of intelligent vehicles, more research has focused on achieving human-like driving. As an important component of intelligent vehicle control, car-following control should ensure safety, tracking, comfort while considering the acceptance of human drivers. In this paper, we propose a car-following control strategy <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\bm{\pi}_{\bm{Hybrid}}$</tex-math> </inline-formula> based on a hybrid of reinforcement learning (RL) and supervised learning (SL). RL is used to achieve multi-objective collaborative optimization in car-following control, and SL is used to achieve human like car-following. Through the complementary advantages of the two learning methods, <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\bm{\pi}_{{Hybrid}}$</tex-math> </inline-formula> can achieve high performance car-following while matching the personalized car-following characteristics of human drivers. RL is used as the main framework of <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\bm{\pi}_{{Hybrid}}$</tex-math> </inline-formula> . In addition, the personalized car-following reference model (PCRM) of human drivers based on Gaussian mixture regression, and the motion uncertainty model of preceding vehicle (MUMPV) based on the sequence-to-sequence network are established and incorporated into the RL framework. PCRM can lead <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\bm{\pi }_{{Hybrid}}$</tex-math> </inline-formula> to learn the different characteristics of human drivers, and improve the anthropomorphism of <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\bm{\pi }_{{Hybrid}}$</tex-math> </inline-formula> ; MUMPV enables <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\bm{\pi }_{{Hybrid}}$</tex-math> </inline-formula> to consider the dynamic changes of the traffic environment and to become more robust. <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\bm{\pi }_{{Hybrid}}$</tex-math> </inline-formula> is trained and tested on High D dataset, and the generalizability verification is based on the self-built real vehicle data collection platform. The results show that <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\bm{\pi }_{{Hybrid}}$</tex-math> </inline-formula> can match human drivers’ personalized car-following characteristics and can outperform human drivers in safety, comfort, and tracking of the preceding vehicle.