Monocular vision-based multiinformation map construction framework for field terrain

Abstract

Three-dimensional (3D) spatial data, terrain type, and physical properties of field terrain are very important information for mobile robots in trajectory planning, force adjustment, motion control, dynamic parameter optimization, and so on. Obtaining the information in advance through vision can provide a prior perception to help make reasonable decisions and improve security and adaptability. We present the first monocular-vision-based multiinformation map construction framework for field terrain. The current state-of-the-art physical properties inference algorithm is promoted in illumination robustness, time consumption, and terrain richness. Our improvement makes it possible to apply physical properties inference to the real-time mapping of field terrain. An improved monocular vision and inertia fused localization scheme is used to introduce the gravity direction. Depending on the obtained terrain segmentation and camera pose, an unsupervised monocular scale-consistent terrain dense depth estimation method with comprehensive geometric constraints is designed. Finally, the components are systematically combined to form a complete framework, and the ambiguity of inference results between multiview frames is addressed. Comprehensive experiments are conducted to verify the effectiveness of each component and the performance of the overall framework. Our multiinformation map construction framework can output a gravity-aligned 3D RGB map, terrain map, friction property map, and stiffness property map at once with high accuracy.