Toward personalized decision making for autonomous vehicles: A constrained multi-objective reinforcement learning technique

Abstract

Reinforcement learning promises to provide a state-of-the-art solution to the decision making problem of autonomous driving. Nonetheless, numerous real-world decision making problems involve balancing multiple conflicting or competing objectives. In addition, passengers may typically prefer to explore diversified driving modes through their specific preferences (i.e., relative importance of different objectives). Taking into account these demands, traditional reinforcement learning algorithms with applications in personalized self-driving vehicles remain challenging. Consequently, here we present a novel constrained multi-objective reinforcement learning technique for personalized decision making in autonomous driving, with the goal of learning a single model for Pareto optimal policies across the space of all possible user preferences. Specifically, a nonlinear constraint incorporating a user-specified preference and a vectorized action–value function is introduced to ensure both diversity in learned decision behaviors and efficient alignment between the user-specified preference and the corresponding optimal policy. Additionally, a constrained multi-objective actor–critic approach is advanced to approximate the Pareto optimal policies for any user-specified preferences while adhering to the nonlinear constraint. Finally, the proposed personalized decision making scheme for autonomous driving is assessed in a highway on-ramp merging scenario with dynamic traffic flows. The results demonstrate the effectiveness of our method by comparing it with classical and state-of-the-art baselines.