Geometric and semantic analysis of road image sequences for traffic scene construction

Abstract

In this paper, a traffic scene construction framework is proposed based on geometric and semantic analysis of road image sequences using convolutional neural networks (CNNs). For geometric analysis branch of the framework, the image features and heatmaps are extracted to locate the keypoints of road boundaries by CNN. The recurrent module is employed to refine the heatmap prediction to generate more accurate keypoints. The scene layout is then specified according to the keypoints. For semantic segmentation branch of the framework, an encoder-decoder mechanism is utilized to divide the scene layout. A hall module is developed to connect the encoder and decoder parts, which improves the segmentation performance for tiny objects with lower computation cost. Furthermore, the 3D traffic scene models are constructed according to the geometric and semantic analysis results. The traffic simulation can be implemented based on the scene models. The extensive evaluations and comparisons demonstrate the effectiveness of the proposed framework.