Field Of 3D Reconstruction Research Articles

Parametric indoor 3D reconstruction based on binary image feature point extraction

Abstract In recent years, with the rapid advancement of laser scanning technology, indoor 3D modeling technology has significantly developed, becoming a research hotspot in the field of 3D reconstruction. Addressing the issues of high complexity in decoding point cloud surfaces in traditional indoor reconstruction, this paper proposes an indoor 3D reconstruction method based on binary image feature point extraction. The method utilizes point cloud data for semantic segmentation of the indoor environment to accurately extract the ceiling area, which is then projected onto the XOY plane and converted into a binary image, thereby transforming the 3D problem into a 2D problem. Subsequently, key feature points are identified through feature point detection technology, and systematic model construction is carried out based on the sequence of these feature points and their associated parameters. This method effectively simplifies the complexity of traditional 3D modeling, achieving accurate and parametric indoor 3D reconstruction.

A Curb-Detection Network with a Tri-Plane BEV Encoder Module for Autonomous Delivery Vehicles

Curb detection tasks play a crucial role in the perception of the autonomous driving environment for logistics vehicles. With the popularity of multi-modal sensors under the BEV (Bird’s Eye View) paradigm, curb detection tasks are increasingly being integrated into multi-task perception networks, achieving robust detection results. This paper modifies and integrates the tri-plane spatial feature representation method of the EG3D network from the field of 3D reconstruction into a BEV-based multi-modal sensor detection network, including LiDAR, pinhole cameras, and fisheye cameras. The system collects a total of 24,350 frames of data under real road conditions for experimentation, proving the effectiveness of the proposed method.

Large-Scale 3D Reconstruction from Multi-View Imagery: A Comprehensive Review

Three-dimensional reconstruction is a key technology employed to represent virtual reality in the real world, which is valuable in computer vision. Large-scale 3D models have broad application prospects in the fields of smart cities, navigation, virtual tourism, disaster warning, and search-and-rescue missions. Unfortunately, most image-based studies currently prioritize the speed and accuracy of 3D reconstruction in indoor scenes. While there are some studies that address large-scale scenes, there has been a lack of systematic comprehensive efforts to bring together the advancements made in the field of 3D reconstruction in large-scale scenes. Hence, this paper presents a comprehensive overview of a 3D reconstruction technique that utilizes multi-view imagery from large-scale scenes. In this article, a comprehensive summary and analysis of vision-based 3D reconstruction technology for large-scale scenes are presented. The 3D reconstruction algorithms are extensively categorized into traditional and learning-based methods. Furthermore, these methods can be categorized based on whether the sensor actively illuminates objects with light sources, resulting in two categories: active and passive methods. Two active methods, namely, structured light and laser scanning, are briefly introduced. The focus then shifts to structure from motion (SfM), stereo matching, and multi-view stereo (MVS), encompassing both traditional and learning-based approaches. Additionally, a novel approach of neural-radiance-field-based 3D reconstruction is introduced. The workflow and improvements in large-scale scenes are elaborated upon. Subsequently, some well-known datasets and evaluation metrics for various 3D reconstruction tasks are introduced. Lastly, a summary of the challenges encountered in the application of 3D reconstruction technology in large-scale outdoor scenes is provided, along with predictions for future trends in development.

A review of 3D reconstruction methods based on deep learning

In computer vision, an important research area is three-dimensional reconstruction. Using computer technology to reconstruct three-dimensional models of objects has become an indispensable part of in-depth research in many fields. This thesis presents the development process of 3D reconstruction methods that use deep learning. Compared with traditional methods, the 3D reconstruction method based on deep learning has more flexible input and output and higher efficiency. This thesis classifies the methods by the type of 3D model representation and discusses different frameworks for 3D reconstruction based on deep learning. With the introduction of the method NeRF (Neural Radiance Field), the three-dimensional reconstruction work based on deep learning has got a great development. NeRF can achieve good results in a very short period of time in the face of various complex scenes. With the continuous improvement of NeRF by researchers, this method has achieved more amazing results. Finally, the existing problems in the field of 3D reconstruction, the causes of problems and possible solutions are analyzed. Finally, the future development trend and direction of this field are hypothesized and discussed.

Disparity Refinement for Stereo Matching of High-Resolution Remote Sensing Images Based on GIS Data

With the emergence of the Smart City concept, the rapid advancement of urban three-dimensional (3D) reconstruction becomes imperative. While current developments in the field of 3D reconstruction have enabled the generation of 3D products such as Digital Surface Models (DSM), challenges persist in accurately reconstructing shadows, handling occlusions, and addressing low-texture areas in very-high-resolution remote sensing images. These challenges often lead to difficulties in calculating satisfactory disparity maps using existing stereo matching methods, thereby reducing the accuracy of 3D reconstruction. This issue is particularly pronounced in urban scenes, which contain numerous super high-rise and densely distributed buildings, resulting in large disparity values and occluded regions in stereo image pairs, and further leading to a large number of mismatched points in the obtained disparity map. In response to these challenges, this paper proposes a method to refine the disparity in urban scenes based on open-source GIS data. First, we register the GIS data with the epipolar-rectified images since there always exists unignorable geolocation errors between them. Specifically, buildings with different heights present different offsets in GIS data registering; thus, we perform multi-modal matching for each building and merge them into the final building mask. Subsequently, a two-layer optimization process is applied to the initial disparity map based on the building mask, encompassing both global and local optimization. Finally, we perform a post-correction on the building facades to obtain the final refined disparity map that can be employed for high-precision 3D reconstruction. Experimental results on SuperView-1, GaoFen-7, and GeoEye satellite images show that the proposed method has the ability to correct the occluded and mismatched areas in the initial disparity map generated by both hand-crafted and deep-learning stereo matching methods. The DSM generated by the refined disparity reduces the average height error from 2.2 m to 1.6 m, which demonstrates superior performance compared with other disparity refinement methods. Furthermore, the proposed method is able to improve the integrity of the target structure and present steeper building facades and complete roofs, which are conducive to subsequent 3D model generation.

Automatic Real-Time Creation of Three-Dimensional (3D) Representations of Objects, Buildings, or Scenarios Using Drones and Artificial Intelligence Techniques

This work presents the development and evaluation of a real-time 3D reconstruction system using drones. The system leverages innovative artificial intelligence techniques in photogrammetry and computer vision (CDS-MVSNet and DROID-SLAM) to achieve the accurate and efficient reconstruction of 3D environments. By integrating vision, navigation, and 3D reconstruction subsystems, the proposed system addresses the limitations of existing applications and software in terms of speed and accuracy. The project encountered challenges related to scheduling, resource availability, and algorithmic complexity. The obtained results validate the applicability of the system in real-world scenarios and open avenues for further research in diverse areas. One of the tests consisted of a one-minute-and-three-second flight around a small figure, while the reconstruction was performed in real time. The reference Meshroom software completed the 3D reconstruction in 136 min and 12 s, while the proposed system finished the process in just 1 min and 13 s. This work contributes to the advancement in the field of 3D reconstruction using drones, benefiting from advancements in technology and machine learning algorithms.

Fast-Track Virtual Reality to Facilitate 3D Reconstruction in Congenital Heart Disease.

A limitation of the clinical use of 3D reconstruction and virtual reality (VR) systems is the relatively high cost and experience required to use hardware and software to effectively explore medical images. We have tried to simplify the process and validate a new tool developed for this purpose with a novel software. Five patients with right partial anomalous pulmonary venous return with adequate preoperative imaging acquired by magnetic resonance were enrolled. Five volunteers with no previous experience in the field of 3D reconstruction were instructed to use the software after a short video tutorial. Users were then asked to create a 3D model of each patient's heart with DIVA and their results were compared quantitatively and qualitatively with a benchmark reconstruction performed by an experienced user. All our participants recreated 3D models in a relatively short time, maintaining a good overall quality (average quality score ≥ 3 on a scale of 1-5). The overall trend of all the parameters analysed showed a statistical improvement between Case 1 and Case 5, as users become more and more experienced. DIVA is a very simple software that allows accurate 3D reconstruction in a relatively short time ("fast-track" VR). In this study, we demonstrated the potential use of DIVA by inexperienced users, with a significant improvement in quality and time after a few cases were performed. Further studies are needed to confirm the potential application of this technology on a larger scale.

3D face reconstruction from single image with generative adversarial networks

Traditional reconstruction techniques extract information from the object’s geometry or one or more 2D images. On the other hand, the limit of the existing methods is that they generate less precise objects. Thus the lack of robustness towards several face reconstruction problems, such as the position of the head, occlusion, noise, and lighting variation. Therefore, generative neural networks and graphical convolution networks have marked a significant evolution in the field of 3D reconstruction. This paper proposes a model for 3D face reconstruction from a single 2D image. Our model is composed of a generator and a discriminator based on convolutional graphic layers. Indeed, in order to generate a face mesh with expression, our idea is to use the landmarks associated with this image as input to the generator to reconstruct a face geometry with expression and improve the convergence rate. As a result, our model offers an accurate reconstruction of facial geometry with expression; thus, our model outperforms state-of-the-art methods through qualitative and quantitative comparison.

Oriented and Directional Chamfer Distance Losses for 3D Object Reconstruction From a Single Image

The application of deep learning in the field of 3D reconstruction has greatly improved the quality of 3D object reconstruction. For methods that take the point cloud as supervision information, previous research has mainly focused on the network architecture while setting Chamfer Distance (CD) loss as the default loss function. However, CD only contains distance information while ignoring directional information. In this paper, we introduce novel CD losses considering directions that can be used in a 3D reconstruction network. These CD losses consider both direction and distance information, and have two specific variants, Oriented Chamfer Distance (OCD) and Directional Chamfer Distance (DCD). Numerous experiments conducted on the deformable patch and point cloud reconstruction, show that some classic neural networks for 3D reconstruction with OCD or DCD loss can achieve better reconstruction results than those with CD loss.

Fast estimation method of volumes of landslide deposit by the 3D reconstruction of smartphone images

The fast estimation of volumes of landslide deposit has great significance for emergency rescue decision-making, such as deposit filling and excavation and road dredging. The existing methods for the volumes of landslide deposit estimation have the disadvantages of a long data acquisition cycle, poor timeliness and high cost. Therefore, this paper researches a fast estimation method of volumes of landslide deposit based on the 3D reconstruction of smartphone images. First, the improved algorithms of speeded up robust features (SURF) and oriented fast and rotated brief (ORB) algorithms are combined to realize the fast matching of landslide feature points. Second, the 3D scene sparse reconstruction of landslide deposit is realized through boundary constraints. Third, taking the steepest gradient into account, the interpolation of the bottom of the landslide deposit and the fast estimation of the volumes are carried out. Finally, a prototype system is developed, and a case study is performed. The experimental results show that the estimation results of the proposed method are between those of the commercial software ContextCapture, Metashape and Pix4Dmapper, which are commonly used in the field of 3D reconstruction, but the estimation speed is 1–2 times faster than that of the commercial software mentioned above. The proposed method can realize the low-cost and fast estimation of landslide deposit volumes and provide technical support for landslide emergency rescue.

Efficient Rock Mass Point Cloud Registration Based on Local Invariants

Point cloud registration is one of the basic research hotspots in the field of 3D reconstruction. Although many previous studies have made great progress, the registration of rock point clouds remains an ongoing challenge, due to the complex surface, arbitrary shape, and high resolution of rock masses. To overcome these challenges, a novel registration method for rock point clouds, based on local invariants, is proposed in this paper. First, to handle the massive point clouds, a point of interest filtering method based on a sum vector is adopted to reduce the number of points. Second, the remaining points of interest are divided into several cluster point sets and the centroid of each cluster is calculated. Then, we determine the correspondence between the original point cloud and the target point cloud by proving the inherent similarity (using the trace of the covariance matrix) of the remaining point sets. Finally, the rotation matrix and translation vector are calculated, according to the corresponding centroids, and a correction method is used to adjust the positions of the centroids. To illustrate the superiority of our method, in terms of accuracy and efficiency, we conducted experiments on multiple datasets. The experimental results show that the method has higher accuracy (about ten times) and efficiency than similar existing methods.

Fast variational multi-view segmentation through backprojection of spatial constraints

We present a variational segmentation method which exploits color, edge and spatial information between an arbitrary number of views. In contrast to purely image based information like color and gradient, spatial consistency is a new cue for segmentation, which originates from the field of 3D reconstruction. We show that this cue can be easily integrated in a variational formulation and allows pixel-accurate segmentation, even for objects which are hard to segment. The use of inherently parallel algorithms and the implementation on modern GPUs allows us to apply this method to semi-supervised and completely automatic settings. On publicly available datasets we show that our method is faster and more accurate than the state of the art. The successful applications within a catadioptric measurement system and multi-view background subtraction shows its practical relevance.