RRT-based maximum entropy inverse reinforcement learning for robust and efficient driving behavior prediction

Abstract

Advanced driver assistance systems have gained popularity as a safe technology that helps people avoid traffic accidents. To improve system reliability, a lot of research on driving behavior prediction has been extensively researched. Inverse reinforcement learning (IRL) is known as a prominent approach because it can directly learn complicated behaviors from expert demonstrations. Because driving data tend to have a couple of optimal behaviors from the drivers' preferences, i.e., sub-optimality issue, maximum entropy IRL has been getting attention with their capability of considering suboptimality. While accurate modeling and prediction can be expected, standard maximum entropy IRL needs to calculate the partition function, which requires large computational costs. Thus, it is not straightforward to apply this model to a high-dimensional space for detailed car modeling. In addition, existing research attempts to reduce these costs by approximating maximum entropy IRL; however, a combination of the efficient path planning and the proper parameter updating is required for an accurate approximation, and existing methods have not achieved them. In this study, we leverage a rapidly-exploring random tree (RRT) motion planner. With the RRT planner, we propose novel importance sampling for an accurate approximation from the generated trees. This ensures a stable and fast IRL model in a large high-dimensional space. Experimental results on artificial environments show that our approach improves stability and is faster than the existing IRL methods.