Abstract
Real-time 3D reconstruction is one of the current popular research directions of computer vision, and it has become the core technology in the fields of virtual reality, industrialized automatic systems, and mobile robot path planning. Currently, there are three main problems in the real-time 3D reconstruction field. Firstly, it is expensive. It requires more varied sensors, so it is less convenient. Secondly, the reconstruction speed is slow, and the 3D model cannot be established accurately in real time. Thirdly, the reconstruction error is large, which cannot meet the requirements of scenes with accuracy. For this reason, we propose a real-time 3D reconstruction method based on monocular vision in this paper. Firstly, a single RGB-D camera is used to collect visual information in real time, and the YOLACT++ network is used to identify and segment the visual information to extract part of the important visual information. Secondly, we combine the three stages of depth recovery, depth optimization, and deep fusion to propose a three-dimensional position estimation method based on deep learning for joint coding of visual information. It can reduce the depth error caused by the depth measurement process, and the accurate 3D point values of the segmented image can be obtained directly. Finally, we propose a method based on the limited outlier adjustment of the cluster center distance to optimize the three-dimensional point values obtained above. It improves the real-time reconstruction accuracy and obtains the three-dimensional model of the object in real time. Experimental results show that this method only needs a single RGB-D camera, which is not only low cost and convenient to use, but also significantly improves the speed and accuracy of 3D reconstruction.
Highlights
Real-time 3D reconstruction technology is a scientific problem that has been widely studied, and the core technology of many applications, including robotics, virtual reality, digital media, human–computer interaction, intangible cultural heritage protection, etc.Traditional real-time 3D reconstruction methods mainly rely on ordinary RGB cameras to collect images
We propose a visual information joint coding three-dimensional restoration method (VJTR) based on deep learning [33,34]
VViissuuaallIInnfoformrmataiotinoSneSgemgemnteanttioantiaonndaEnxdtrEaxcttiroanction RRGGBB--WWDDeeccuaaummsseeeetrtrhhaaeeiinnYYOrrOeeLaLaAllAttCiiCmmTTee++..++AAnnsseesstthwhwoooowwrrkknnttiioonn ssFFeeiiggggmmuurreeeenn22tt,tttthhhheeeevvRRiiGGssuuBBaalliimmiinnaaffggooeerrmmccooaallttlliieeooccnntteeccddoolliillnneeccrrteteeeaaddll ttbbiimmyy etethhiieses passed to the YOLACT++ network, which is used for information extraction, to obtain the segmented image RGB’
Summary
Real-time 3D reconstruction technology is a scientific problem that has been widely studied, and the core technology of many applications, including robotics, virtual reality, digital media, human–computer interaction, intangible cultural heritage protection, etc. Global alignment is performed to distribute errors in the Sensors 2021, 21, 5909 deformation space, which can solve the loop closure problem efficiently [25] On this basis, Maimone et al proposed an efficient system with six Kinects [26], which combines the individual three-dimensional grids from each Kinect only in the rendering stage to produce an intermediate stereoscopic view. In order to solve the above problems, in this research, we only use a single depth camera and propose a real-time 3D reconstruction method, which can quickly and accurately obtain a 3D model of the scene.
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Free) 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a call
