Learning Effectively from Intervention for Visual-based Autonomous Driving

Abstract

Imitation learning (IL) approaches like behavioral cloning have been used successfully to learn simple visual navigation policies by learning a large amount of data from expert driving behaviors. However, scaling up to the actual driving scenarios is still challenging for the IL approaches because they rely heavily on expert demonstrations requiring labeling every state the learner visits, which is not practical. Moreover, the expert demonstrations limit the performance upper bound. This work proposes a method to accelerate the learning efficiency inspired by human apprenticeship to promote end-to-end vision-based autonomous urban driving tasks. We employ a hierarchical structure for visual navigation, where the high-level agent is trained with the ground-truth data of the environment, and the trained policy is then executed to train a purely vision-based low-level agent. Moreover, in addition to the labeled demonstrations, the expert intervenes during the training of the low-level agent and brings efficient feedback information, interactively accelerating the training process. Such intervention provides critical knowledge that can be learned effectively for dealing with complex, challenging scenarios. We evaluate the method on the original CARLA benchmark and the more complicated NoCrash benchmark. Compared to the state-of-the-art methods, the proposed method achieves similar good results but requires fewer data and learns faster, effectively improving the sample efficiency.