Are Reactions to Ego Vehicles Predictable Without Data?: A Semi-Supervised Approach

Abstract

To make intelligent decisions in an autonomous vehicle, the system must predict the future reactions of surrounding vehicles for any given action plan of the ego vehicle. However, learning reactive trajectories is challenging due to scant action-reaction pair data. That is, building a dataset with multiple action-reaction pairs for an identical scene history is impossible in reality. Here, we propose a semi-supervised learning framework with auxiliary structures to handle this problem. The proposed training framework has two modules: <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <i>Action Reconstructor</i> </b> and <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <i>Identifier</i> </b> modules with corresponding loss functions referred to as the <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <i>Reconstruction Loss</i> </b> and <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <i>Association Loss</i> </b> . In addition to the conventional supervised approach pertaining to readily available data, the <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <i>Action Reconstructor</i> </b> module is employed to learn the dependencies on the ego vehicle in an unsupervised manner. Furthermore, reaction trajectory data corresponding to the augmented future trajectories of the ego vehicle are not available, meaning that the model must be trained in an unsupervised manner as well. The main idea of the proposed unsupervised learning method is to find the identity feature vector from both history and future trajectories and associate these features for each vehicle. This idea is realized by introducing the <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <i>Identifier</i> </b> network and the <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <i>Association Loss</i> </b> , which are used only during the training process. Interestingly, experimental results show that plausible reaction can be predicted for the augmented future trajectory of the ego vehicle, which indicates that the network can generalize the interactive behavior of vehicles from a partially labelled dataset.