3D Reconstruction Research Articles

Reinventing 3D echocardiography: could AI-powered 3D reconstruction from 2D echocardiographic views serve as a viable alternative to 3D probes?

Reinventing 3D echocardiography: could AI-powered 3D reconstruction from 2D echocardiographic views serve as a viable alternative to 3D probes?

Visual reconstruction of milling workpiece surface topography based on phase shifting and complementary gray code

When the surface texture of a workpiece is extremely fine, cameras struggle to accurately capture depth information, making conventional machine vision methods insufficient for achieving micrometer-scale three-dimensional surface reconstructions. To overcome this limitation, the study focuses on high-precision 3D reconstruction of the surface morphology of milled workpieces. Given the smoothness of milled surfaces, their susceptibility to overexposure, and the difficulty in extracting depth information from two-dimensional image pixels, the paper proposes a novel method that combines phase-shifting with complementary Gray code to achieve micrometer-level surface reconstruction. The superiority of this method over traditional phase-shifting techniques is demonstrated through comparisons of overall morphology, two-dimensional power spectral density (2D PSD), and the average deviations of sampled surfaces. Results show that the proposed method reduces the average relative error in surface deviations by 25.89% compared to traditional techniques. Furthermore, cross-sectional analyses reveal that the reconstructed point cloud surface more closely mirrors the actual peak-to-valley characteristics of the real surface. Experimental results confirm that this method effectively captures the surface features of milled workpieces, indicating broad potential for applications in precision manufacturing.

Influence of video instruction on soccer kick velocity in young players

Soccer is a sport that necessitates proficiency in motor control and cognitive abilities, with kicking serving as a fundamental action. Improving kicking performance is vital for player advancement, leading to the inquiry: “How can video instruction promptly optimize kicking technique in young players?”. This research aimed to investigate the immediate effects of an instructional video on the kinematic variables of kicking to induce an acute alteration in the movement pattern. A cohort of 26 male youths, aged between 10 and 15 years, partook in the study and were segregated into an instruction group (IG) and a control group (CG). The IG received exposure to a video containing specific kicking instructions, while the CG viewed a video lacking such guidance. Both groups were recorded performing penalty kicks before and after viewing the video. Kinematic analysis was carried out utilizing the OpenPose artificial intelligence neural network for pose detection and subsequent 3D reconstruction. The primary discovery was that the length of the last stride (LS) notably increased in the IG following the instructional video. This outcome indicates that video instructions hold promise in promptly enhancing particular aspects of kicking technique in young soccer players.

3D micromorphological reconstruction and roughness characterization of wood surface based on sequence images

A novel non-contact optical inspection technique is proposed to precisely reconstruct the three-dimensional surface structure of wood and evaluate its roughness. The feasibility of using image fusion techniques to characterize the 3D roughness of wood surfaces is explored by combining high-resolution serial image acquisition techniques with advanced data processing methods. The work focuses on examining various cut surfaces of three distinct wood species: Larch, Catalpa, and Toona sinensis. The proposed 3D roughness parameters are verified using 3D reconstruction and precise measurements. The experimental results demonstrate that the proposed method exhibits strong adaptability in measuring wood surface roughness and achieves a high degree of consistency with the results obtained from traditional digital microscope systems. This study not only validates the possible use of multilayer sequence image fusion technology in measuring wood roughness, but also establishes a solid basis for the creation of 3D assessment criteria for wood surface roughness and associated inspection systems.

Reviewing Open Data Semantic 3D City Models to Develop Novel 3D Reconstruction Methods

Abstract. Structured semantic 3D city models are pivotal in creating urban 3D digital twins. The wide adoption of such models has been primarily enabled by robust, model-based, and automatic 3D reconstruction methods. However, these methods impose requirements on the reconstruction, mainly restricting the solution space to several model types and relying on accurate 2D footprints. Recent research shows that deep-learning-based methods promise highly generic solution space and are footprint-free. Yet, the current training and test datasets are limited, hindering the methods’ development. In this work, we analyze the ubiquity of already existing, open 3D city model datasets and their potential to serve as a large-scale training and test set for 3D reconstruction, where 27 potential dataset collections have been identified. Our review shows that more than 215 million building models are readily available. We firmly believe that this review will facilitate further research on robust automatic 3D city model reconstruction and serve as a reference for benchmarking 3D city models.

Perspective Transform-based Depth Estimation of Monocular Camera for Electrocution Threat Determination of Construction Machinery

Abstract. This paper discusses the application of perspective transform-based monocular camera depth estimation method in monitoring the safety distance between construction machinery and power lines. The traditional distance measurement relies on manual operation, which is inefficient and not precise enough, while the monocular depth estimation method based on deep learning can improve the accuracy, but faces high equipment cost and relies on a large amount of training data. In contrast, the depth estimation method based on perspective transformation proposed in this paper utilizes mathematical and physical principles to establish a mathematical model between the pixel distances of an image and the actual scene distances through perspective projection theory, which simplifies the hardware requirements and reduces the data dependence and improves the computational efficiency and applicability. Through experimental validation under different construction scenes and exposure conditions, the method demonstrates high accuracy and robustness, proving its practicality in construction safety management. This research provides a new technical direction and practical application possibility in the field of intelligent monitoring and 3D reconstruction.

High-precision 3D Modeling of Construction Waste Pile Bodies by Integrating Multi-source Data

Abstract. The instability of construction waste pile bodies, as an increasingly concerning disaster, poses significant risks to the safety of people's lives and property. Currently, there is limited research on high-precision 3D modeling techniques for construction waste pile bodies, which significantly hinders the accuracy and reliability of early detection and risk assessment of pile body instability. Therefore, constructing high-precision 3D models of construction waste pile bodies using multi-source data plays a crucial role in improving the accuracy and timeliness of early warning systems for pile body instability, offering significant theoretical research and practical application value. Building 3D models of construction waste pile bodies solely based on unmanned aerial vehicle (UAV) oblique photography data faces challenges, such as various degrees of data voids and insufficient model completeness, and few methods currently address the construction of 3D models of construction waste pile bodies through the fusion of multi-source data. This paper attempts to use the Iterative Closest Point (ICP) algorithm, combining SLAM laser point cloud data with UAV oblique photography data to fill data voids, and utilizes the fused point cloud to reconstruct the triangulation network, achieving the transformation from 3D point cloud models to 3D surface models. On the basis of texture mapping, a high-precision 3D model of the construction waste pile body is constructed. The research results show that the 3D model of the construction waste pile body, integrated with multi-source data, has a planar error of 0.0187m and an elevation error of 0.0368m, meeting the corresponding model accuracy requirements. It can more realistically restore the fine 3D features of the construction waste pile body, effectively compensating for the shortcomings of single-source data in 3D modeling of construction waste pile bodies, providing a new method for the 3D model reconstruction of construction waste pile bodies, and offering effective data support for construction waste research.

A Novel Platform and Workflow for Underwater Photogrammetry Surveys

Abstract. Underwater photogrammetry has gained popularity for creating three-dimensional (3D) maps and ortho-images of marine environments as compared to traditional echosounder surveys, they can be more cost-effective at creating high-resolution 3D models, and orthoimages are usually more informative than acoustic backscatter maps. This paper builds on previous work by the authors that developed an underwater image-capturing platform with several additional sensors, including GNSS, IMU, pressure depth sensor and single beam echosounder. This study aims to analyse the impact of calibration and sensor data integration into the photogrammetric processing workflow. The tests were performed using two underwater sites. The low-cost device's pressure sensor and tide station data outperform GNSS PPK-derived heights. Furthermore, it was observed that incorporating IMU motion sensor data did not improve the processing results. Additionally, utilising the echosounder point cloud proves valuable for enhancing the overall quality of the survey. Despite its lower density, it serves a dual purpose by validating the photogrammetry dataset and, more importantly, can be employed for correcting DSM height. This study further underlined the importance of reliable camera calibration for accurate 3D reconstruction.

Utilizing a Fully Digital Approach for Oral Squamous Cell Carcinoma Treatment and Zygomatic Implant-Based Rehabilitation for Maxillary Defects.

This clinical report details the functional and esthetic rehabilitation of a patient with a severe maxillary defect secondary to subtotal maxillectomy for oral squamous cell carcinoma using a maxillary prosthesis anchored by 4 zygomatic implants. The procedure involved meticulous subtotal maxillectomy and defect repair with zygomatic implant support, incorporating advanced digital surgical methods, including 3D reconstruction, computer-guided surgery, and photogrammetry (Icam4D). A 3D finite element analysis was conducted to assess the method's efficacy in analyzing stress distribution around the zygomatic implants. The patient expressed high satisfaction with the prosthesis's functionality, esthetics, speech, and swallowing capabilities, underscoring the value of zygomatic implant-supported maxillofacial prosthetics. This synergy of advanced planning, surgical precision, and biomechanical analysis marks a significant advancement in maxillofacial prosthetics.

Robust and accurate visual geo-localization using prior map constructed by handheld LiDAR SLAM with camera image and terrestrial LiDAR point cloud

ABSTRACT Visual geo-localization using prior map of known environments has extensive applications in the fields such as self-driving, augmented reality and navigation. Currently, such prior maps are usually constructed by visual SLAM or SFM. However, the recent advances in LiDAR SLAM technology with RGB camera demonstrate great potential in efficient and accurate prior map construction and visual geo-localization. In this research, we developed a novel prior map construction approach which seamlessly integrates a handheld LiDAR SLAM system, a global LiDAR localization algorithm and a sparse set of terrestrial LiDAR scans as accurate control within a factor graph optimization framework. The developed LiDAR SLAM system augmented by terrestrial LiDAR scans achieves an improved mapping accuracy of 8 cm under GNSS-denied conditions in the areas over 50,000 m2 while the state-of-art FAST-LIO2 alone fails and produces a mapping error over 30 m. The results show that a competitive accuracy of visual geo-localization on a mobile phone using the constructed prior map is achieved (i.e. 60 and 76 cm in two tests respectively). Finally, this work also demonstrates a novel LiDAR point cloud fusion method that produces optimized and consistent coarse-to-fine 3D reconstruction in large and complex scenes.

3D Building Modelling from ALS Point Clouds by Delaunay Triangulation and Graph Theories

Abstract. As an important task in 3D building reconstruction, 3D roof reconstruction attracts increasing attention. Existing methods have the issue of additional errors caused by multi-step processes, resulting in relatively high uncertainty and potentially affecting the reconstruction accuracy. Deep learning-based methods to achieve the direct extraction of roof vertices and edges for reconstructing 3D roof structures tackle the issue of additional errors while leading to another problem: the reliance on large labeled datasets for training. In this study, a fully rule-based method is proposed to achieve automatic 3D roof reconstruction. Roof vertices with their edges parallel to the x-y plane are first extracted from the point clouds, and subsequently, the left roof edges connecting vertices are inferred by using Delaunay triangulation. Finally, the 3D roof structure is reconstructed by using graph analysis based on the information of roof vertices and edges. This method simplifies the process of 3D roof reconstruction, and the experiment results demonstrate that the proposed method can effectively reconstruct 3D roof structures from point clouds.

A study on the digital restoration of an ancient city based on historic building information modeling of wooden architectural heritage: focusing on Suwon Hwaseong

This study describes the process of digitally reconstructing the ancient Korean city of Suwon Hwaseong in 3D utilizing Historic Building Information Modeling (HBIM) resources to accurately represent its wooden architectural heritage. Previous 3D reconstructions of cultural heritage have often prioritized appearance or remained partially disassembled. However, our reconstruction method offers a comprehensive representation of the appearance and the internal structure of wooden architectural heritage, which can be suitable for restoration maintenance. To ensure accuracy in digital restoration, we collected and utilized administrative records and historical materials, including the geography, fortress walls, folk houses, and Haenggung (the temporary palace of the Joseon Dynasty)—drawing from the archive of the Korean Cultural Heritage Service’s management records and the 1796 manuscript “Uigwe: Royal Protocols of the Hwaseong Fortress” which documents the construction of the ancient city of Suwon Hwaseong. Extensive architectural records were used to generate HBIM data, which digitized historical records, documents, and drawings to accurately represent the complex layout of the wooden architectural heritage. For the folk houses that lacked design records and the fortress walls that retained their original shape, we performed a digital restoration-based façade modeling. These elements of the ancient city of Suwon Hwaseong were assembled into a 3D model using Unreal Engine (version 5.1.1) to digitally reconstruct the city and enhance its visual representation. The digital restoration content, which utilizes visual effects and precise rendering from a game engine, can be used for the restoration, repair, and maintenance of both appearance and internal structures.

Super-large diameter slurry shield tunnel encountering boulder formation: Detection, treatment and data analysis

Boulders pose substantial challenges in coastal urban tunnel engineering. This study, for the first time, utilizes extensive closely spaced borehole data from 3486 drilling points obtained during actual tunnelling through boulder strata to evaluate the effectiveness of microtremor detection, analyze variations in operational parameters, and assess the efficiency of blasting treatment. Additionally, leveraging borehole data, a 3D reconstruction of the soil-boulder-bedrock stratum was developed, and the stratum content ratio at the excavation face was derived. A multivariate linear regression model was established to explore the relationship between boulder geology and construction parameters. The results demonstrated that microtremor surveys effectively identified boulder distributions by analyzing horizontal-to-vertical spectral ratio curves and differences in surface wave phase velocity. Field drilling confirmed the high accuracy of microtremor predictions, particularly in areas with dense or thick boulder layers. Post-blasting secondary drilling revealed boulder fragments smaller than 30 cm, meeting the operational requirements for shield cutter and crusher systems. The presence of boulders was found to increase thrust, torque, and overturning moments, as well as cause abnormal cutter wear. Blasting significantly reduced the standard deviation of operational data, demonstrating its effectiveness in stabilizing tunnelling performance. Regression analysis showed a strong correlation between geological conditions and tunnelling parameters, with results indicating that blasting operations effectively mitigate the adverse impacts of boulders on shield tunnelling performance. This study aims to provide valuable insights and references for future engineering projects encountering similar complex geological conditions.

Improving Underwater Photogrammetric 3D Reconstruction Processing of Shipwreck Sites

Abstract. Applying image processing algorithms to enhance the clarity of underwater images can significantly assist in the visual interpretation of subsea environments. Taking advantage of techniques to maximise the information extracted from the captured input data to produce the highest quality output ensures a higher utilisation of the valuable recorded content which may have been collected at the high cost of a laborious diver survey or an expensive ROV (Remotely Operated Vehicle) expedition. Adopting photogrammetric 3D reconstruction processing in underwater imaging has enabled the creation of high-fidelity digital 3D models which have been used to map subsea infrastructure for maintenance and repair, coral reefs for ecological monitoring, and in the primary context of this article, shipwrecks sites for maritime archaeology and accident investigation. Alongside exploring algorithms to reduce the characteristic haze of underwater photography, this article illustrates the advantages of optimising the processes utilised at distinct stages of the photogrammetric 3D reconstruction workflow to improve the photorealism of the digital 3D models.

The avian vocal system: 3D reconstruction reveals upper vocal tract elongation during head motion.

While birds' impressive singing abilities are made possible by the syrinx, the upper vocal system (i.e., trachea, larynx, and beak) could also play a role in sound filtration. Yet, we still lack a clear understanding of the range of elongation this system can undertake, especially along the trachea. Here, we used biplanar cineradiography and X-ray Reconstruction of Moving Morphology (XROMM) to record 15 species of cadaveric birds from 9 different orders while an operator moved the bird's cadaveric heads in different directions. In all studied species, we found elongation of the trachea to be correlated with neck extension, and significantly greater (ranging from 18% to 48% for the whole motion; and from 1.4% to 15.7% for the singing positions) than previously reported on a live singing bird (3%). This elongation or compression was not always homogeneous along its entire length. Some specimens showed increased lengthening in the rostral part and others in both the rostral and caudal parts of the vocal tract. The diversity of elongation patterns shows that trachea elongation is more complex than previously thought. Since tracheal lengthening affects sound frequencies, our results contribute to our understanding of the mechanisms involved in complex communication signals, one of the amazing traits we share with birds.

The Framework of Modeling Buddhist Head in 3D with Semantic Information

Abstract. Buddhist statues hold profound spiritual and cultural significance, yet their comprehensive analysis and identification remain largely reliant on manual annotation and expert scrutiny. Despite advancements in 3D modeling, the nuanced representation of these statues, essential for automated analysis, remains elusive. Integrating semantic information such as symbolic and cultural elements poses a further challenge. There currently exists no standardized method for modeling Buddhist heads in 3D with semantic information. This paper proposes a framework for the component-based analysis of Buddhist statues, particularly focusing on their heads, to bridge this gap. Drawing from diverse disciplines including archaeology, art history, and conservation, the framework leverages advancements in 3D reconstruction to provide a detailed depiction of intricate features. Building upon existing research outcomes, the framework facilitates the seamless integration of semantic information, capturing not only the physical attributes but also the cultural, historical, and spiritual dimensions of these sculptures. This approach aids in overcoming challenges in automated analysis and virtual restoration, providing a dynamic tool for exploration, preservation, and interpretation of cultural heritage. By enabling interdisciplinary synergy, the framework emerges as a dynamic tool for exploring, preserving, and interpreting the profound heritage of Buddhist art with unprecedented depth and clarity.

Preoperative Imaging-Guided Surgical Planning in Complex Pelvic Fractures - Innovations in Traumatology and Radiology for Enhanced Clinical Outcomes: A Systematic Review

This systematic review will explore role of preoperative imaging-guided surgical planning for the management of complex pelvic fractures. Pelvic fractures occurs in high-energy trauma and cause challenges for intricate anatomy and have a high association with vascular, visceral injuries. Traditional imaging methods frequently fail to capture injury complexity while increasing surgical risks. Recent advancements of imaging technology CT, MRI, 3D reconstructions along with and AI-based predictive models offer enhanced precision in diagnosing and planning surgical interventions with detailed insights into fracture patterns, bone density, and soft tissue damage which enable accurate surgical planning to reduce intraoperative complications and shorten recovery times. We aim to systematically investigate effectiveness of modern imaging techniques with potencies in clinical outcomes, with an emphasis on improved surgical accuracy and patient safety. Analyzing range of studies, we have established evidence-based recommendations for integrating advanced imaging technologies into routine trauma care showing critical role these innovations play in improving both short-term recovery and long-term clinical outcomes for patients having complex pelvic fractures.

AI-based 3D analysis of retinal vasculature associated with retinal diseases using OCT angiography.

Retinal vasculature is the only vascular system in the human body that can be observed in a non-invasive manner, with a phenotype associated with a wide range of ocular, cerebral, and cardiovascular diseases. OCT and OCT angiography (OCTA) provide powerful imaging methods to visualize the three-dimensional morphological and functional information of the retina. In this study, based on OCT and OCTA multimodal inputs, a multitask convolutional neural network model was built to realize 3D segmentation of retinal blood vessels and disease classification for different retinal diseases, overcoming the limitations of existing methods that can only perform 2D analysis of OCTA. Two hundred thirty sets of OCT and OCTA data from 109 patients, including 138,000 cross-sectional images in normal and diseased eyes (age-related macular degeneration, retinal vein occlusion, and central serous chorioretinopathy), were collected from four commercial OCT systems for model training, validation, and testing. Experimental results verified that the proposed method was able to achieve a DICE coefficient of 0.956 for 3D segmentation of blood vessels and an accuracy of 91.49% for disease classification, and further enabled us to evaluate the 3D reconstruction of retinal vessels, explore the interlayer connections of superficial and deep vasculatures, and reveal the 3D quantitative vessel characteristics in different retinal diseases.

3D reconstruction and depth estimation method for local anomalies of rail surface based on multi-view stereo matching

Abstract Detecting rail surface anomalies has become crucial for ensuring the safety of train operations. However, manual detection methods are time-consuming and labor-intensive. Although traditional detection models based on one-dimensional signals or two-dimensional images can effectively identify the existence of defects, they are difficult to use to obtain local depth characteristic information, leading to difficulties in accurately assessing the extent of damage in abnormal regions. Rail track maintenance strategies are typically developed based on the detected different depth ranges of defects. To address this issue, this paper proposes a local depth estimation method based on the point cloud of the target rail surface reconstructed by PatchmatchNet. Firstly, according to the acquisition method of multi-view images and the sampling environment at the site, a practical data collection protocol is established for generating a multi-view dataset of rail surface. Next, dense point cloud of the target rail surface were reconstructed by PatchmatchNet. Subsequently, a method for estimating the local anomaly depth is developed based on the point cloud. Experimental results demonstrate that the proposed method achieves a minimum error of approximately 5.5% within a maximum depth range of 0.35mm, when compared to depth measurements obtained using a structured light camera. Finally, this paper presents a more comprehensive three-dimensional representation of the local abnormal physical structure, surpassing traditional one-dimensional and two-dimensional forms. This enhanced representation offers richer information for the effective determination of whether the anomalies constitute defects and aids in distinguishing true defects from other surface irregularities. Such advancement significantly improves the precision of defect assessment and supports the development of highly precise repair strategies.

LUT-based phase error compensation method for large-step phase-shifting algorithm in DLP4500-based FPP system.

As a low-cost professional digital light projection device, the DLP4500 have been widely applied in fringe projection profilometry (FPP), for both laboratory and practical application. However, our recent experiments revealed a new hardware-induced projection instability when the projection pattern data exceeds its buffer capacity (48 bits). This phenomenon undermines the measurement accuracy advantage of the phase-shifting (PS) algorithms with large number of shifting steps, and eventually leads unwanted and complicated error to 3D reconstruction. In this paper, we experimentally investigate the new hardware-induced phase error and proposed a LUT-based phase error compensation method. In this method, a standard plate with a precision manufactured plate is used as the standard reference for the phase error evaluation, where an ideal plane fitting and the projector pixel reprojection process are introduced to generate the ideal reference phase. Comprehensive experiments are conduct to verify the stability of the proposed method in LUT creation. Comprehensive experiments are conduct, and the results show that (i) the method works creates LUTs stably at different plate positions, (ii) the plate with regular manufacturing accuracy (not exceeding 0.01 mm) can meet the application requirements of the proposed method. Both quantitative and qualitative experimental results successfully verify the effectiveness of proposed method in LUT creation and phase error reduction.