Visualization pipeline of autonomous driving scenes based on FCCR-3D reconstruction

Abstract

Visual sensor-based human–computer interaction can provide environmental perception data of traffic road scenes for mechatronic intelligent vehicles. The existing three-dimensional (3D) reconstruction visualization schemes are mainly based on structured scene reconstruction with multiview geometry and are offline operations. It is difficult for large-scale autonomous driving scenes with wide baselines to ensure consistent reconstruction results in the wide field-of-view area and the center of the light-of-sight area. It is also difficult to achieve uniform reconstruction results for regions with drastic depth changes and inconsistent texture fineness. Therefore, we propose a 3D reconstruction-based visualization pipeline for autonomous driving scenes of wide baseline. First, the stereo geometric model is initialized. The parameters of distortion correction and polar calibration are calibrated. Second, the proposed feature chain code resampling (FCCR) is used to uniformly extract the features for regions with inconsistent texture fineness and depth variation. The filtered 3D data are obtained based on the stereo geometric model. Finally, the texture mapping of the mesh skeleton of 3D data is carried out by the bilinear combination of surrounding multilevel gradual textures. The visualization results of scene rendering and scene roaming are realized. The pipeline has been effectively verified on the simulated experimental platform, real vehicle test, and public dataset. It can meet the real-time 3D reconstruction visualization of wide-baseline scenes. The reconstruction texture is uniform, and the detail response is rich. We can present the intuitive, 3D, real-time, graphical physical quantities of scene data for drivers, intelligent vehicles, AR-HUD, and control systems. The method we present has research value and application significance for the practical engineering of electronic images.