Physics-Informed Particle-Based Reinforcement Learning for Autonomy in Signalized Intersections

Abstract

In this paper, we develop a framework to enhance the control of autonomous vehicles within signalized intersections by integrating system dynamics with imperfect sensor data. Although sensor data for autonomous vehicles is often partial, noisy, or missing, its alignment with underlying physics has the potential to significantly improve prediction accuracy. Our methodology involves a physics-based representation of system dynamics, accounting for stochastic elements originating from differences between real-world scenarios and model-based representations, using a Markov decision process (MDP) model that embraces system physics while accommodating uncertainties. Leveraging partial sensor data, we aim to diminish uncertainty associated with segments of the system model reflected in the data and better estimate the other physical variables that are not measurable through data. This framework develops particle-based reinforcement learning action policies, seamlessly integrating data and physics for controlling autonomous vehicles approaching signalized intersections. Numerical experiments showcase the effectiveness of these policies in ensuring safe control and decision-making in different scenarios of missing state variables and varying levels of data sparsity.