Deep embedded hybrid CNN–LSTM network for lane detection on NVIDIA Jetson Xavier NX

Abstract

In recent years, lane detection has become one of the most important factors in the progress of intelligent vehicles. To deal with the challenging problem of low detection precision and real-time performance of most traditional systems, we proposed a real-time deep lane detection system based on CNN Encoder–Decoder and Long Short-Term Memory (LSTM) networks for dynamic environments and complex road conditions. The CNN Encoder network is used to extract deep features from a dataset and to reduce their dimensionality. A corresponding decoder network is used to map the low resolution encoder feature maps to dense feature maps that correspond to road lane. The LSTM network processes historical data to improve the detection rate through the removal of the influence of false alarm patches on detection results. We propose three network architectures to predict the road lane: CNN Encoder–Decoder network, CNN Encoder–Decoder network with the application of Dropout layers and CNN Encoder–LSTM-Decoder network that are trained and tested on a public dataset comprising 12764 road images under different conditions. Experimental results show that the proposed hybrid CNN Encoder–LSTM-Decoder network that we have integrated into a Lane-Departure-Warning-System (LDWS) achieves high prediction performance namely an average accuracy of 96.36%, a Recall of 97.54%, and a F1-score of 97.42%. A NVIDIA Jetson Xavier NX supercomputer has been used, for its performance and efficiency, to realize an Embedded Deep LDWS.