Deep learning framework-based 3D shape reconstruction of tanks from a single RGB image

Abstract

In recent times, complicated three-dimensional shape reconstruction from a single RGB image has become a crucial technology in many industries such as Automotive, Healthcare, and Military. It is particularly challenging to reconstruct the complex shape of a military tank. Former methods infer 3D information from 2D images via shape deformation from ellipsoids, which has the problems of local adhesion, uneven surfaces, and distortion of the structure. This study introduces a new single-view 3D shape reconstruction (SVSR) framework with multi-scale feature extraction that splits shape reconstruction into three tasks—camera parameter prediction, initial shape construction, and deformation prediction. The shape-initialization module provides a variable initial shape by predicting the stretch and displacement parameters of each geometric component based on their topological relationships. The shape-deformation module predicts the directional deformation of each vertex. These two modules are dedicated separately to avoid shape adhesion and improve the local detail performance. Silhouette images of the tank’s overall shape and local geometric components are employed to eliminate the impact of component shielding and avoid structural distortion via both pixel loss and perceptual loss. Experiments on the main battle tank dataset demonstrate that our method can predict complicated 3D shapes with a low Chamfer distance value (0.0017). Our approach outperforms the other state-of-the-art methods in terms of Chamfer distance and F-score with at least a 10% improvement, with more realistic overall contours and part details. It has huge application prospects in dealing with other complex shape predictions by modifying the shape-initialization module.