Abstract

The goal of image-based 3D reconstruction is to establish a high-quality 3D expression from images. In order to achieve a high-resolution and real-time 3D model, inspired by the open source COLMAP, we propose a novel framework (3DMAP) to reconstruct 3D scenes based on spherical cameras. Unlike traditional methods which focus on building a 3D plane by Poisson distribution function, our method illustrates the key processes of 3D reconstruction: locating the camera based on global feature, estimating the scene’s relative depth from monocular panoramic images, and obtaining a high-quality 3D surface reconstruction. In the camera locating part, we use a global descriptor augmentation model to build a labeled panorama dataset GDAP, in which the images are captured by our designed spherical cameras; In the depth estimation part, we propose a new network UMDE that can estimate the depth of both indoor and outdoor scenes; Finally, in the 3D surface reconstruction section, we turn the reconstruction problem to a graph optimization problem, called GraphFit, in which, we optimize the point clouds with s-t graph and smoothing method successively. We conduct experiments on our own dataset to demonstrate the effectiveness of our proposed 3DMAP framework. Experimental results show that our 3DMAP has achieved good evaluation scores and visual effects.

Highlights

  • Faculty of Intelligent Manufacturing, Wuyi University, Jiangmen 529000 China; Zhuhai 4Dage Network Technology, Zhuhai 519000 China; Deutsches Forschungszentrum für Künstliche Intelligenz

  • ⚫ We propose a panoramic image labeled dataset based on spherical camera (GDAP)

  • ⚫ We propose a 3D surface reconstruction algorithm (GraphFit) based on graph optimization

Read more

Summary

Introduction

Faculty of Intelligent Manufacturing, Wuyi University, Jiangmen 529000 China; Zhuhai 4Dage Network Technology, Zhuhai 519000 China; Deutsches Forschungszentrum für Künstliche Intelligenz. The popular surface reconstruction algorithms limited by the fitting function to balance the energy To solve these problems and utilize the rich information of wide FOV panoramic images, we propose a new framework according to COLMAP logic, called “3DMAP”. It mainly contains three parts, camera location, depth estimation and surface reconstruction. The traditional feature extracting algorithms, such as SIFT[1], SURF[2], AKAZE[3] are robust but slow due to the violent matching operation It will be slower when processing the panoramic images, what’s worse, it cannot deal with the distortion brought by the wide FOV

Objectives
Methods
Findings
Conclusion
Full Text
Paper version not known

Talk to us

Join us for a 30 min session where you can share your feedback and ask us any queries you have

Schedule a call

Disclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.