End-to-end driving model based on deep learning and attention mechanism

Abstract

End-to-end deep learning has gained considerable interests in autonomous driving vehicles. End-to-end autonomous driving uses the deep convolutional neural network to establish input-to-output mapping. However, existing end-to-end driving models only predict steering angle with front-facing camera data and poorly extract spatial-temporal information. Based on deep learning and attention mechanism, we propose an end-to-end driving model which combines the multi-stream attention module with the multi-stream network. As a multimodal multitask model, the proposed end-to-end driving model not only fully extracts spatial-temporal information from multimodality, but also adopts the multitask learning method with hard parameter sharing to predict the steering angle and speed. Furthermore, the proposed multi-stream attention module predicts the attention weights of streams based on the multimodal feature fusion, which encourages the proposed end-to-end driving model to pay attention to streams that positively impact the prediction result. We demonstrate the efficiency of the proposed driving model on the public Udacity dataset compared to existing models. Experimental results show that the proposed driving model has better performances than other existing methods.