IMMAT: Mesh Reconstruction from Single View Images by Medial Axis Transform Prediction

Abstract

The representation of a 3D shape is a key element for capturing the overall structure as well as the local details. In this paper, we propose to predict a mesh representation of the Medial Axis Transform (called medial mesh) as an intermediate representation with our IMMAT framework, to reconstruct the 3D shape from a single view image. Because the MAT contains the skeleton topology and local thickness information, it not only enhances the ability to reconstruct topologically complex shapes but also better preserves the local details with its thickness control. The framework consists of three modules. The Image2Sphere module predicts the medial spheres inside the shape surface and the Topology Prediction module predicts the topological relationship (skeleton) between the predicted spheres. Then the MAT Smoothing module smooths the predicted MAT and fine-tunes the coordinates and radii of the spheres by graph convolution. The final 3D surface can be reconstructed by converting the predicted MAT to an implicit surface through CSG operation and then extracting the boundary surface through Marching Cubes. Experimental results show that our method outperforms the state-of-the-art methods both quantitatively and qualitatively on the reconstruction task.