Application Of 3D Reconstruction Research Articles

An Overview of Methods and Applications of 3D Reconstruction

With the development of hardware equipment and theoretical knowledge, 3D reconstruction technology has played a key role in many fields such as industrial manufacturing, cultural relics protection and augmented reality, and has attracted great attention in the field of computer vision. 3D reconstruction methods can be divided into traditional methods and deep learning methods. Among them, the traditional method technology is mature and widely used at present, while the 3D reconstruction method based on deep learning is developing rapidly, which has the advantages of low dependence on equipment, strong method generalization and strong adaptability to the environment. In order to promote the development of subsequent research, this paper first gives a comprehensive classification and review of the existing 3D reconstruction methods, then analyzes their principles and performance, and finally further analyzes the existing problems and challenges, and explores possible future research directions, with the aim of providing new ideas for more in depth 3D reconstruction tasks.

Applications of 3D Reconstruction in Virtual Reality-Based Teleoperation: A Review in the Mining Industry

Although multiview platforms have enhanced work efficiency in mining teleoperation systems, they also induce “cognitive tunneling” and depth-detection issues for operators. These issues inadvertently focus their attention on a restricted central view. Fully immersive virtual reality (VR) has recently attracted the attention of specialists in the mining industry to address these issues. Nevertheless, developing VR teleoperation systems remains a formidable challenge, particularly in achieving a realistic 3D model of the environment. This study investigates the existing gap in fully immersive teleoperation systems within the mining industry, aiming to identify the most optimal methods for their development and ensure operator’s safety. To achieve this purpose, a literature search is employed to identify and extract information from the most relevant sources. The most advanced teleoperation systems are examined by focusing on their visualization types. Then, various 3D reconstruction techniques applicable to mining VR teleoperation are investigated, and their data acquisition methods, sensor technologies, and algorithms are analyzed. Ultimately, the study discusses challenges associated with 3D reconstruction techniques for mining teleoperation. The findings demonstrated that the real-time 3D reconstruction of underground mining environments primarily involves depth-based techniques. In contrast, point cloud generation techniques can mostly be employed for 3D reconstruction in open-pit mining operations.

Research on Three-dimensional Reconstruction

Nowadays, with the continuous development of the field of computer vision and computer graphics, 3D reconstruction technology has become an area that has attracted much of people’s attention. 3D reconstruction refers to creating a 3D model of an object in the real world by extracting information from multiple 2D images or sensor data. This technology has important usages in many fields, including virtual reality, augmented reality, cultural heritage preservation, medical image processing, industrial design, etc. This thesis examines the application of 3D reconstruction in various fields, discusses the definition of development history, its wide application in life, and its advantages and disadvantages. This article is divided into four parts and discusses the algorithm research of 3D reconstruction in four fields. Through an in-depth study of 3D reconstruction technology, we can better understand its role in different fields and provide strong support for developing and innovating related applications. As more accurate devices and algorithms continue to advance, 3D reconstruction technology is expected to show its significant potential in more and more fields and create more realistic and useful digital experiences for us.

Retracted: Clinical Application of Digital 3D Reconstruction and 3D Printing Technology in Endometrial Cancer (EC) Surgery.

[This retracts the article DOI: 10.1155/2022/9180216.].

A Robust Multi-Local to Global with Outlier Filtering for Point Cloud Registration

As a prerequisite for many 3D visualization tasks, point cloud registration has a wide range of applications in 3D scene reconstruction, pose estimation, navigation, and remote sensing. However, due to the limited overlap of point clouds, the presence of noise and the incompleteness of the data, existing feature-based matching methods tend to produce higher outlier matches, thus reducing the quality of the registration. Therefore, the generation of reliable feature descriptors and the filtering of outliers become the key to solving these problems. To this end, we propose a multi-local-to-global registration (MLGR) method. First, in order to obtain reliable correspondences, we design a simple but effective network module named the local geometric network (LG-Net), which can generate discriminative feature descriptors to reduce the outlier matches by learning the local latent geometric information of the point cloud. In addition, we propose a multi-local-to-global registration strategy to further filter outlier matches. We compute the hypothetical transformation matrix from local patch matches. The point match evaluated as an inlier under multiple hypothetical transformations will receive a higher score, and low-scoring point matches will be rejected. Finally, our method is quite robust under different numbers of samples, as it does not require sampling a large number of correspondences to boost the performance. The numerous experiments on well-known public datasets, including KITTI, 3DMatch, and ModelNet, have proven the effectiveness and robustness of our method. Compared with the state of the art, our method has the lowest relative rotation error and relative translation error on the KITTI, and consistently leads in feature matching recall, inlier ratio, and registration recall on 3DMatch under different numbers of point correspondences, which proves the robustness of our method. In particular, the inlier ratio is significantly improved by 3.62% and 4.36% on 3DMatch and 3DLoMatch, respectively. In general, the performance of our method is more superior and robust than the current state of the art.

Application of CT 3D reconstruction in the interlaminar approach of lumbar endoscopy.

Application of CT 3D reconstruction in the interlaminar approach of lumbar endoscopy.

Learning General Descriptors for Image Matching With Regression Feedback

Recent advances on feature descriptors for image matching put more emphasis on encoding invariances ( <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">e.g</i> . illumination invariance) to promote the descriptors’ discriminative power. However, according to the information entropy, more invariance implies greater certainty and less informativeness in a descriptor. Consequently, descriptors encoding too many invariances usually show poor generalization to unknown image changes, lacking enough informativeness to cover the large uncertainty in unseen scenes. This limits the application scenarios of learned descriptors. In this paper, we propose to alleviate this issue from the perspective of informativeness and we thus design hierarchical consistent constraint by introducing regression feedback in a self-supervised manner. Combined with the hardest-within-batch matching constraint, we form a novel dual supervision framework, to encourage the descriptor to learn an informative representation while maintaining a good discriminative power. Moreover, to fully mine the context information hidden in image and boost the informativeness in turn, we present AANet, a descriptor network that efficiently predicts dense description by the powerful Attentional Aggregation of multi-level features. Experiments across challenging feature matching on HPatches, RDNIM datasets, and visual localization tasks on Aachen Day-night dataset show that our method outperforms recent state-of-the-art descriptors while keeping encouraging efficiency. The application of visual 3D reconstruction on various scenarios also demonstrates the high generalization ability of our method.

Intervention planning in modern renal surgery

Three-dimensional (3D) reconstruction in renal surgery offers significant benefits in planning of the operation itself, in patient education and training fellows. The goal was to study the possible application and use of 3D reconstruction in modern renal surgery. A literature search in PubMed using keywords "3D" and "renal surgery" within the last 10years of publication was performed. The search revealed 655articles among them 53reviews. Relevant for this article were 120 articles which included 14reviews. The publication statistics showed astrong increase in the last 10years which may reflect the increasing application and demand for 3D technology in the field of renal surgery. Especially in robot-assisted minimally invasive surgery applications, the use of 3D technology was studied more often. The technical standards are not yet uniform and 3D printing in complex renal surgery remains cost- and time-intensive. Nevertheless, the results of the pioneering clinical studies showed positive results, and 3D technology can be used not only in planning the operation, but in patient education and also the training of fellows and staff. Alimitation to more complex cases seems to be sensible for this is still acostly tool. To identify complex cases in renal surgery, the established renometric scoring systems should be used more broadly in daily practice to identify patients who would profit most from this technique. In complex renal surgery, 3D reconstruction and modelling is rational; furthermore, it can be useful in patient education and in training of fellows.

Fast Point Cloud Registration Method with Incorporation of RGB Image Information

Point cloud registration has a wide range of applications in 3D reconstruction, pose estimation, intelligent driving, heritage conservation, and digital cities. The traditional iterative closest point (ICP) algorithm has strong dependence on the initial position, poor robustness, and low timeliness. To address the above issues, a fast point cloud registration method that incorporates RGB image information is proposed. The SIFT algorithm is used to detect feature points of point clouds corresponding to the RGB image, followed by feature point matching. The RANSAC algorithm is applied to remove erroneous point pairs in order to calculate the initial transformation matrix. After applying a pass-through filter for noise reduction and transiting down with a voxel grid, the point cloud is subjected to rotation and translation transformation for initial registration. On the basis of initial alignment, the FR-ICP algorithm is utilized for achieving precise registration. This method not only avoids the problem of ICP easily getting stuck in local optima, but also has higher registration accuracy and efficiency. Experimental studies were conducted based on point clouds of automotive parts collected in real scenes, and the results showed that the proposed method has a registration error of only 0.487 mm. Among the same group of experimental point clouds with comparable registration error, the proposed method showed a speed improvement of 69%/48% compared to ICP/FR-ICP with regard to registration speed.

Application of 3D-reconstruction and artificial intelligence for complete mesocolic excision and D3 lymphadenectomy in colon cancer

Mathematical algorithms 3D-reconstruction techniques and artificial intelligence are a current line of innovation in colorectal surgical oncology.The objective of this study is to show the initial experience of a 3D image processing and reconstruction system to perform complete mesocolic excision and D3-lymphadenectomy in colon cancer.It is applied to a splenic flexure neoplasm and in a right colon cancer with suspected retroperitoneal infiltration.3D image processing and reconstruction was employed to delimit 10 cm proximal and distal intestinal margins to the tumor and define its corresponding vascularization. In right colon cancer it showed position and exact dimensions of D3-lymphadenectomy area and possible retroperitoneal fascia infiltration.3D image processing and reconstruction allows to obtain valuable information from computerized tomography scan. It could be employed during surgical strategy planification to improve oncological results and reduce intraoperative complications.

被动式三维重建在建立桥梁缆索数字模型中的应用

被动式三维重建在建立桥梁缆索数字模型中的应用

Virtual anthropology: Forensic applications to cranial skeletal remains from the Spanish Civil War

Biological and forensic anthropologists face limitations while studying skeletal remains altered by taphonomic alterations and perimortem trauma, such as in remains from the Spanish Civil War. However, virtual anthropology techniques can optimize the information inferred from fragmented and deformed remains by generating and restoring three-dimensional bone models. We applied a low-cost 3D modelling methodology based on photogrammetry to develop novel forensic applications of virtual 3D skull reconstruction, assembly, restoration and ancestry estimation. Crania and mandible fragments from five Spanish Civil War victims were reconstructed with high accuracy, and only one cranium could not be assembled due to extensive bone loss. Virtual mirroring successfully restored reconstructed crania, producing 3D models with reduced deformation and perimortem trauma. High correlation between traditional and virtual craniofacial measurements confirmed that 3D models are suitable for forensic applications. Craniometric databases of world-wide and Spanish populations were used to assess the potential of discriminant analysis to estimate population ancestry. Our protocol correctly estimated the continental origin of 86.7 % of 15 crania of known origin, and despite low morphological differentiation within European populations, correctly identified 54.5 % as Spanish and 27.3 % of them with high posterior probabilities. Two restored crania from the Civil War were estimated as Spanish, and one as a non-Spanish European. Results were not conclusive for one cranium and did not confirm previous archeological hypotheses. Overall, our research shows the potential to assess the presence of foreign volunteers in the Spanish Civil War and highlights the added value of 3D-virtual techniques in forensic anthropology.

Clinical application of 3D reconstruction in pancreatic surgery: a narrative review

Progress in medicine requires not only innovation and development in the medical field but also the integration of the technology of other fields into the medical field. As an important technological advancement, three-dimensional (3D) reconstruction transforms traditional two-dimensional images into 3D images that are more consistent with the physiological habits of human eyes. It has been applied to the bones, heart, liver, and maxillofacial area, promoting the progress of medical technology and surgeons. This article introduces the progress of 3D reconstruction technology in the clinical application of pancreatic surgery, from the preoperative, intraoperative, and postoperative perspectives, as well as the education of young surgeons. It also puts forward new ideas for the further development of pancreatic surgery.

RADepthNet: Reflectance-aware monocular depth estimation

BackgroundMonocular depth estimation aims to predict a dense depth map from a single RGB image, and has important applications in 3D reconstruction, automatic driving, and augmented reality. However, existing methods directly feed the original RGB image into the model to extract depth features without avoiding the interference of depth-irrelevant information on depth-estimation accuracy, which leads to inferior performance. MethodsTo remove the influence of depth-irrelevant information and improve the depth-prediction accuracy, we propose RADepthNet, a novel reflectance-guided network that fuses boundary features. Specifically, our method predicts depth maps using the following three steps: (1) Intrinsic Image Decomposition. We propose a reflectance extraction module consisting of an encoder-decoder structure to extract the depth-related reflectance. Through an ablation study, we demonstrate that the module can reduce the influence of illumination on depth estimation. (2) Boundary Detection. A boundary extraction module, consisting of an encoder, refinement block, and upsample block, was proposed to better predict the depth at object boundaries utilizing gradient constraints. (3) Depth Prediction Module. We use an encoder different from (2) to obtain depth features from the reflectance map and fuse boundary features to predict depth. In addition, we proposed FIFADataset, a depth-estimation dataset applied in soccer scenarios. ResultsExtensive experiments on a public dataset and our proposed FIFADataset show that our method achieves state-of-the-art performance.

Historical-graphical analysis and digital preservation of cultural heritage: case study of the baptismal font of the church of Santiago Apóstol in Montilla (Córdoba, Spain)

Stone is a characteristic element in the construction of buildings and monuments in the Middle Ages in Spain. This is so mainly due to its durability and, in baptismal fonts, specifically, its symbolism: Jesus Christ is the cornerstone, he comes to save us and is the foundation of the life of the Christian and of the Universal Church. The application of virtual 3D reconstruction is essential in the conservation of this type of heritage, together with scientific techniques and methods, and the analysis of historical-graphic documentation. Focusing on these three aspects, the baptismal font of the church of Santiago Apóstol de Montilla will be taken as a reference. The baptismal font of the church of Santiago Apóstol in Montilla (Córdoba, Spain), whose exact origin is unknown, is one of the most important heritage assets. San Francisco Solano, known as the “Thaumaturge of the New World”, was baptised in it. For over four centuries, the temple has been the Main Church of the head of the Marquisate of Priego. This study aims to know, document and carry out a digital reconstruction of an important piece of the monumental heritage of the Catholic Church: the baptismal font of the church of Santiago Apóstol in Montilla. To this end, we searched a major literature review focused on the use of photogrammetry for reading old graphics and deciphering texts. After that we analysed the graphic documentation derived from the restoration process conducted by Cordovan restorer Vázquez Arjona in the late twentieth century. The application of 3D digitisation and documentation techniques has allowed not only to analyse this important heritage element in detail through a virtual reconstruction, but also to make a far-reaching discovery regarding one of its elements, the unknown inserted inscription in the baptismal font band.

Clinical Application of Digital 3D Reconstruction and 3D Printing Technology in Endometrial Cancer (EC) Surgery.

Aims We use CTA and magnetic resonance data to use digital three-dimensional reconstruction and 3D printing technology to reproduce the solid replication of the uterus and surrounding tissues in vitro, fully evaluate the adjacency of tumor tissues with surrounding important organs, blood vessels, and lymph nodes, and reduce the impact. The normal organ structure and function of the surgeon can shorten the operation time, reduce the bleeding during the operation, and reduce the perioperative complications of the patient to improve the prognosis of the patient. Materials and Methods Select 40 EC patients and divide them into group A (3D reconstruction data is transmitted to 3D printing equipment according to the results of CTA and MRI examination, and a 3D model is printed out according to the ratio of 1 : 1 for evaluation and judgment before surgery) and group B (according to MRI imaging examination, there were 20 cases each). Different surgical conditions, quality of life, adverse reactions, and clinical efficacy were evaluated in each group. Results The operation time, the time of the first anus exhaust, the hospitalization time after the operation, and the blood loss of the operation in group A were significantly lower than those in group B. Statistics showed that the difference was significant (P < 0.05). The quality of life scores of emotion, cognition, society, and overall health of group A were significantly higher than those of group B, while physical score, fatigue, nausea, vomiting, and pain were lower than those of group B, which were statistically significant (P < 0.05). Both groups of patients had complications after the operation, and they were asked to be followed up at the outpatient clinic 3 months after the operation. All patients recovered well. There were 19 and 18 patients in groups A and B, respectively, complaining of improvement in clinical symptoms, and the difference was not statistically significant (P < 0.05). Conclusion With the support of digital three-dimensional reconstruction and 3D printing technology, complex operations can be accurately performed, improving the efficacy and safety of patients after EC surgery, improving patient outcomes and quality of life, improving EC positioning accuracy, and reducing tumor residue.

EVALUATION OF STRUCTURAL FND FUNCTIONAL FEATURES OF REIMPLANTED AORTIC VALVE AFTER DAVID I PROCEDURE WITH THE USE OF HIGH-PRECISSION 3-DIMENSIONAL SEGMENTATION OF THE AORTIC ROOT

Aneurysm of the aortic root accompanied by aortic insufficiency is a common cardiovascular pathology. The optimal method of surgical treatment of this pathology is the reimplantation of the aortic valve into the aortic root synthetic prosthesis (David procedure). Currently, there is no comprehensive data on the influence of initial geometric parameters of the aortic root structures on the functioning of the reimplanted aortic valve after valve-preserving aortic root replacement. We have introduced a novel technique for creation and segmentation of a three-dimensional reconstruction of the aortic valve and root, which involves the sequential application of two computer-aided design environments: In Vesalius v 3.1.1 (Centro de Tecnologia da Informação Renato Archer (CTI)) and 3- Matic v. 13.0 (Materialise, Leuven, Belgium). The proposed method allows accurate measurement and evaluation of spatial parameters of the AV leaflets, fibrous ring, sinuses of Valsalva and sinotubular junction. The developed technique was used to evaluate the anatomical and functional features of the aortic valve in a 68-year-old patient who underwent David I procedure. Segmentation and measurement of the geometric parameters of the aortic root were performed before operation, as well as 18 months after surgical treatment. To assess the function of the reimplanted aortic valve during the long-term follow-up, a detailed echocardiographic assessment was used. Echocardiographic assessment included measuring volume and area of the regurgitation flow, the width of vena contracta, and the regurgitation fraction. The application of high-precision 3D reconstruction opens up additional opportunities in the field of surgical treatment of the aortic root and AV pathology: advantages in choosing an individual patient-oriented surgical tactics, parameters of the implanted surgical material and/or device, and in predicting long-term results of surgical treatment.

DeepProfile: Accurate Under-the-Clothes Body Profile Estimation

Accurate human body profiles have many potential applications. Image-based human body profile estimation can be regarded as a fine-grained semantic segmentation problem, which is typically used to locate objects and boundaries in images. However, existing image segmentation methods, such as human parsing, require significant amounts of annotation and their datasets consider clothes as part of the human body profile. Therefore, the results they generate are not accurate when the human subject is dressed in loose-fitting clothing. In this paper, we created and labeled an under-the-clothes human body contour keypoint dataset; we utilized a convolutional neural network (CNN) to extract the contour keypoints, then combined them with a body profile database to generate under-the-clothes profiles. In order to improve the precision of keypoint detection, we propose a short-skip multi-scale dense (SMSD) block in the CNN to keep the details of the image and increase the information flow among different layers. Extensive experiments were conducted to show the effectiveness of our method. We demonstrate that our method achieved better results—especially when the person was dressed in loose-fitting clothes—than and competitive quantitative performance compared to state-of-the-art methods, while requiring less annotation effort. We also extended our method to the applications of 3D human model reconstruction and body size measurement.

A Stereo Calibration Method of Multi-Camera Based on Circular Calibration Board

In the application of 3D reconstruction of multi-cameras, it is necessary to calibrate the camera used separately, and at the same time carry out multi-stereo calibration, and the calibration accuracy directly affects the effect of the 3D reconstruction of the system. Many researchers focus on the optimization of the calibration algorithm and the improvement of calibration accuracy after obtaining the calibration plate pattern coordinates, ignoring the impact of calibration on the accuracy of the calibration board pattern coordinate extraction. Therefore, this paper proposes a multi-camera stereo calibration method based on circular calibration plate focusing on the extraction of pattern features during the calibration process. This method preforms the acquisition of the subpixel edge acquisition based on Franklin matrix and circular feature extraction of the circular calibration plate pattern collected by the camera, and then combines the Zhang’s calibration method to calibrate the camera. Experimental results show that compared with the traditional calibration method, the method has better calibration effect and calibration accuracy, and the average reprojection error of the multi-camera is reduced by more than 0.006 pixels.

Application of Image-based 3D Reconstruction in Architectural Heritage Surveying and Mapping

Application of Image-based 3D Reconstruction in Architectural Heritage Surveying and Mapping