Volumetric Reconstruction Research Articles

Volumetric reconstruction of settling mud flocs: A new insight of equilibrium flocculation in saline water

Mud flocculation and settling play key role in understanding sediment transport cycle and affect water quality in estuaries and coastal seas. However, the morphological irregularity and structural instability of fragile mud flocs set huge obstacles for quantifying geometric property accurately and establishing reliable predicting tools in settling dynamics via previous observing strategies based on instant measured and 2-dimensional imagery floc parameterizations. Here we designed a multi-camera apparatus targeting capturing multiple angles of individual flocs, and developed a multi-view segmentation algorithm on floc images analysis. We finally accomplished batch of 3-dimensional reconstruction obtaining each settling floc's volumetric size in equilibrium flocculation. The results indicate a stable bimodal floc size distribution in equilibrium flocculation with a dominant peak of microflocs (<200 μm) and a secondary smaller peak of macroflocs (> 200 μm). The flocculi (<50 μm) shows more spherical outlines with dense structure while the larger-sized macroflocs (>200 μm) have high irregular morphologies with high porosity and visible biological debris attaching, and the microflocs (50–200 μm) tend to be irregular in shape and dense inside. The terminal settling velocity of mud flocs shows increasing with floc size in <200 μm but keeps stable around 1–2 mm s−1 after >200 μm due to the increase in size being compensated by the decrease of density according to the fractal theory on floc geometry. The higher organic matter content within larger porous flocs reduces the macroflocs effective density. These lead to high volumetric settling flux but low mass settling flux of macroflocs in natural water systems. This work provides new insight to reveal more accurate mud floc geometric parameterizations in volumetric aspect and reliable characterizations of equilibrium flocculation using a fast and sound batch of direct measuring approach. This may importantly improve the predictions of suspended mud dynamics in nature.

Three-Dimensional Single Random Phase Encryption.

In this paper, we propose a new optical encryption technique that uses the single random phase mask. In conventional optical encryptions such as double random phase encryption (DRPE), two different random phase masks are required to encrypt the primary data. For decryption, DRPE requires taking the absolute value of the decrypted data because it is complex-valued. In addition, when key information is revealed, the primary data may be reconstructed by attackers. To reduce the number of random phase masks and enhance the security level, in this paper, we propose single random phase encryption (SRPE) with additive white Gaussian noise (AWGN) and volumetric computational reconstruction (VCR) of integral imaging. In our method, even if key information is known, the primary data may not be reconstructed. To enhance the visual quality of the decrypted data by SRPE, multiple observation is utilized. To reconstruct the primary data, we use VCR of integral imaging because it can remove AWGN by average effect. Thus, since the reconstruction depth can be another key piece of information of SRPE, the security level can be enhanced. In addition, it does not require taking the absolute value of the decrypted data for decryption. To verify the validity of our method, we implement the simulation and calculate performance metrics such as peak sidelobe ratio (PSR) and structural similarity (SSIM). In increasing the number of observations, SSIM for the decrypted data can be improved dramatically. Moreover, even if the number of observations is not enough, three-dimensional (3D) data can be decrypted by SRPE at the correct reconstruction depth.

Multi-view 3D reconstruction based on deep learning: A survey and comparison of methods

An important objective in computer vision is to analyze multiple images and subsequently reconstruct the shape and structure in 3D. Traditional multi-view 3D reconstruction techniques extract and match key features from images with known camera parameters. However, this approach is inefficient and fails to fully exploit the advantages of multi-view information. Advancements in deep learning have revolutionized multi-view 3D reconstruction by enabling end-to-end 3D shape inferencing without the need for sequential feature matching typically found in conventional algorithms. Recent rapid progress in this field necessitates a thorough review of current algorithms and provide insight into method of improving 3D reconstruction performance. This review classifies reconstruction algorithms according to their resultant model, including depth map, voxel, point cloud, mesh, and implicit surface. Additionally, this review encompasses the inclusion of frequently employed network training loss functions for network training, assessment metrics, and the incorporation of 3D datasets. Experimental results are also presented to assess the performance of different algorithms. Finally, the paper concludes with a summary, discussion of challenges, and potential future directions.

Three Dimensional Reconstruction of Two-step Moving Objects based on Phase-shifting Profilometry

In recent years, 3D object reconstruction based on phase-shifting profilometry has gradually received attention and been widely applied. Domestic and foreign scholars have been continuously researching and exploring the accuracy and speed of three-dimensional measurement, and gradually developing towards dynamic measurement. Most dynamic measurements require projecting multiple stripe patterns to obtain sufficient object phase information, and the more stripes there are, the greater the phase error caused by motion. This article proposes the use of increasing the sampling fringe pattern during the projection period of a fringe pattern to achieve high frame rate dynamic 3D object reconstruction. By combining the intensity values of ambient light; Finally, the phase information of the object is extracted by tracking the motion information obtained from the moving object. This article demonstrates the feasibility of this method through simulation experiments and improves the frame rate of 3D reconstruction of moving objects.

Evaluating the Point Cloud of Individual Trees Generated from Images Based on Neural Radiance Fields (NeRF) Method

Three-dimensional (3D) reconstruction of trees has always been a key task in precision forestry management and research. Due to the complex branch morphological structure of trees themselves and the occlusions from tree stems, branches and foliage, it is difficult to recreate a complete three-dimensional tree model from a two-dimensional image by conventional photogrammetric methods. In this study, based on tree images collected by various cameras in different ways, the Neural Radiance Fields (NeRF) method was used for individual tree dense reconstruction and the exported point cloud models are compared with point clouds derived from photogrammetric reconstruction and laser scanning methods. The results show that the NeRF method performs well in individual tree 3D reconstruction, as it has a higher successful reconstruction rate, better reconstruction in the canopy area and requires less images as input. Compared with the photogrammetric dense reconstruction method, NeRF has significant advantages in reconstruction efficiency and is adaptable to complex scenes, but the generated point cloud tend to be noisy and of low resolution. The accuracy of tree structural parameters (tree height and diameter at breast height) extracted from the photogrammetric point cloud is still higher than those derived from the NeRF point cloud. The results of this study illustrate the great potential of the NeRF method for individual tree reconstruction, and it provides new ideas and research directions for 3D reconstruction and visualization of complex forest scenes.

Niimath and fslmaths: replication as a method to enhance popular neuroimaging tools.

Neuroimaging involves the acquisition of extensive 3D images and 4D time series data to gain insights into brain structure and function. The analysis of such data necessitates both spatial and temporal processing. In this context, "fslmaths" has established itself as a foundational software tool within our field, facilitating domain-specific image processing. Here, we introduce "niimath," a clone of fslmaths. While the term "clone" often carries negative connotations, we illustrate the merits of replicating widely-used tools, touching on aspects of licensing, performance optimization, and portability. For instance, our work enables the popular functions of fslmaths to be disseminated in various forms, such as a high-performance compiled R package known as "imbibe", a Windows executable, and a WebAssembly plugin compatible with JavaScript. This versatility is demonstrated through our NiiVue live demo web page. This application allows 'edge computing' where image processing can be done with a zero-footprint tool that runs on any web device without requiring private data to be shared to the cloud. Furthermore, our efforts have contributed back to FSL, which has integrated the optimizations that we've developed. This synergy has enhanced the overall transparency, utility and efficiency of tools widely relied upon in the neuroimaging community.

Deep Learning for 3D Reconstruction, Augmentation, and Registration: A Review Paper.

The research groups in computer vision, graphics, and machine learning have dedicated a substantial amount of attention to the areas of 3D object reconstruction, augmentation, and registration. Deep learning is the predominant method used in artificial intelligence for addressing computer vision challenges. However, deep learning on three-dimensional data presents distinct obstacles and is now in its nascent phase. There have been significant advancements in deep learning specifically for three-dimensional data, offering a range of ways to address these issues. This study offers a comprehensive examination of the latest advancements in deep learning methodologies. We examine many benchmark models for the tasks of 3D object registration, augmentation, and reconstruction. We thoroughly analyse their architectures, advantages, and constraints. In summary, this report provides a comprehensive overview of recent advancements in three-dimensional deep learning and highlights unresolved research areas that will need to be addressed in the future.

A Semantically Aware Multi-View 3D Reconstruction Method for Urban Applications

The task of 3D reconstruction of urban targets holds pivotal importance for various applications, including autonomous driving, digital twin technology, and urban planning and development. The intricate nature of urban landscapes presents substantial challenges in attaining 3D reconstructions with high precision. In this paper, we propose a semantically aware multi-view 3D reconstruction method for urban applications which incorporates semantic information into the technical 3D reconstruction. Our research primarily focuses on two major components: sparse reconstruction and dense reconstruction. For the sparse reconstruction process, we present a semantic consistency-based error filtering approach for feature matching. To address the challenge of errors introduced by the presence of numerous dynamic objects in an urban scene, which affects the Structure-from-Motion (SfM) process, we propose a computation strategy based on dynamic–static separation to effectively eliminate mismatches. For the dense reconstruction process, we present a semantic-based Semi-Global Matching (sSGM) method. This method leverages semantic consistency to assess depth continuity, thereby enhancing the cost function during depth estimation. The improved sSGM method not only significantly enhances the accuracy of reconstructing the edges of the targets but also yields a dense point cloud containing semantic information. Through validation using architectural datasets, the proposed method was found to increase the reconstruction accuracy by 32.79% compared to the original SGM, and by 63.06% compared to the PatchMatch method. Therefore, the proposed reconstruction method holds significant potential in urban applications.

Enhancing Single-Plane Fluoroscopy: A Self-Calibrating Bundle Adjustment for Distortion Modeling.

Single-plane fluoroscopy systems with image intensifiers remain commonly employed in a clinical setting. The imagery they capture is vulnerable to several types of geometric distortions introduced by the system's components and their assembly as well as interactions with the local and global magnetic fields. In this study, the application of a self-calibrating bundle adjustment is investigated as a method to correct geometric distortions in single-plane fluoroscopic imaging systems. The resulting calibrated imagery is then applied in the quantitative analysis of diaphragmatic motion and potential diagnostic applications to hemidiaphragm paralysis. The calibrated imagery is further explored and discussed in its potential impact on areas of surgical navigation. This work was accomplished through the application of a controlled experiment with three separate Philips Easy Diagnost R/F Systems. A highly redundant (~2500 to 3500 degrees-of-freedom) and geometrically strong network of 18 to 22 images of a low-cost target field was collected. The target field comprised 121 pre-surveyed tantalum beads embedded on a 25.4 mm × 25.4 mm acrylic base plate. The modeling process resulted in the estimation of five to eight distortion coefficients, depending on the system. The addition of these terms resulted in 83-85% improvement in terms of image point precision (model fit) and 85-95% improvement in 3D object reconstruction accuracy after calibration. This study demonstrates significant potential in enhancing the accuracy and reliability of fluoroscopic imaging, thereby improving the overall quality and effectiveness of medical diagnostics and treatments.

3D Ultrasound Mosaic of the Whole Shoulder: A Feasibility Study

A protocol is proposed to acquire a tomographic ultrasound (US) scan of the musculoskeletal (MSK) anatomy in the rotator cuff region. Current clinical US imaging techniques are hindered by occlusions and a narrow field of view and require expert acquisition and interpretation. There is limited literature on 3D US image registration of the shoulder or volumetric reconstruction of the full shoulder complex. We believe that a clinically accurate US volume reconstruction of the entire shoulder can aid in pre-operative surgical planning and reduce the complexity of US interpretation. The protocol was used in generating data for deep learning model training to automatically register US mosaics in real-time. An in vivo 3D US tomographic reconstruction of the entire rotator cuff region was produced by registering 53 sequential 3D US volumes acquired by an MSK sonographer. Anatomical surface thicknesses and distances in the US mosaic were compared to their corresponding MRI measurements as the ground truth. The humeral head surface was marginally thicker in the reconstructed US mosaic than its original thickness observed in a single US volume by 0.65 mm. The humeral head diameter and acromiohumeral distance (ACHD) matched with their measured MRI distances with a reconstruction error of 0 mm and 1.2 mm, respectively. Furthermore, the demonstration of 20 relevant MSK structures was independently graded between 1 and 5 by two sonographers, with higher grades indicating poorer demonstration. The average demonstration grade for each anatomy was as follows: bones = 2, muscles = 3, tendons = 3, ligaments = 4–5 and labrum = 4–5. There was a substantial agreement between sonographers (Cohen’s Weighted kappa of 0.71) on the demonstration of the structures, and they both independently deemed the mosaic clinically acceptable for the visualisation of the bony anatomy. Ligaments and the labrum were poorly observed due to anatomy size, location and inaccessibility in a static scan, and artefact build-up from the registration and compounding approaches.

Comparative connectomics of dauer reveals developmental plasticity

A fundamental question in neurodevelopmental biology is how flexibly the nervous system changes during development. To address this, we reconstructed the chemical connectome of dauer, an alternative developmental stage of nematodes with distinct behavioral characteristics, by volumetric reconstruction and automated synapse detection using deep learning. With the basic architecture of the nervous system preserved, structural changes in neurons, large or small, were closely associated with connectivity changes, which in turn evoked dauer-specific behaviors such as nictation. Graph theoretical analyses revealed significant dauer-specific rewiring of sensory neuron connectivity and increased clustering within motor neurons in the dauer connectome. We suggest that the nervous system in the nematode has evolved to respond to harsh environments by developing a quantitatively and qualitatively differentiated connectome.

Optimal image 2D/3D by Krawtchouk moments and ABC algorithm

In our research, we provide an enhanced and effective Krawtchouk moments parameter p optimization approach using the Artificial Bee Colony (ABC) algorithm optimization technique for the purpose of completing tasks including the reconstruction and classification of 3D and 2D images with high quality. The proposed method, which uses the ABC algorithm to generate a suitable parameter based on Krawtchouck moments, aims to accurately characterize the image as a vector known as the descriptor vector of moments, which is then used to reconstruct the image for each order. The simulation and analysis of the findings demonstrate the high level of quality and effectiveness of the suggested KM-ABC approach for image reconstruction, as well as its excellent accuracy in images with and without noise.

Reconstruction of 3D Information of Buildings from Single-View Images Based on Shadow Information

Accurate building geometry information is crucial for urban planning in constrained spaces, fueling the growing demand for large-scale, high-precision 3D city modeling. Traditional methods like oblique photogrammetry and LiDAR prove time consuming and expensive for low-cost 3D reconstruction of expansive urban scenes. Addressing this challenge, our study proposes a novel approach to leveraging single-view remote sensing images. By integrating shadow information with deep learning networks, our method measures building height and employs a semantic segmentation technique for single-image high-rise building reconstruction. In addition, we have designed complex shadow measurement algorithms and building contour correction algorithms to improve the accuracy of building models in conjunction with our previous research. We evaluate the method’s precision, time efficiency, and applicability across various data sources, scenarios, and scales. The results demonstrate the rapid and accurate acquisition of 3D building data with maintained geometric accuracy (mean error below 5 m). This approach offers an economical and effective solution for large-scale urban modeling, bridging the gap in cost-efficient 3D reconstruction techniques.

Full-automatic high-precision scene 3D reconstruction method with water-area intelligent complementation and mesh optimization for UAV images

ABSTRACT Fast and high-precision urban scene 3D modeling is the foundational data infrastructure for the digital earth and smart cities. However, due to challenges such as water-area matching difficulties and issues like data redundancy and insufficient observations, existing full-automatic 3D modeling methods often result in water-area missing and many small holes in the models and insufficient local-model accuracy. To overcome these challenges, full-automatic high-precision scene 3D reconstruction method with water-area intelligent complementation on depth maps and mesh optimization is proposed. Firstly, SfM was used to calculated image poses and PatchMatch was used to generated initial depth maps. Secondly, a simplified GAN extracted water-area masks and ray tracing was used achieve high-precision auto-completed water-area depth values. Thirdly, fully connected CRF optimized water-areas and arounds in depth maps. Fourthly, high-precision 3D point clouds were obtained using depth map fusion based on clustering culling and depth least squares. Then, mesh was generated and optimized using similarity measurement and vertex gradients to obtain refined mesh. Finally, high-precision scene 3D models without water-area missing or holes were generated. The results showed that: to compare with the-state-of-art ContextCapture, the proposed method enhances model completeness by 14.3%, raises average accuracy by 14.5% and improves processing efficiency by 63.6%.

Nondestructive geochemical characterization of fossil hominin taphonomy and burial history

To date, only three Homo habilis specimens have been discovered that have associated craniodental and postcranial elements, providing a limited fossil record of the ontogeny and morphology of early members of the genus Homo. Recently, a nearly complete dentition, likely attributable to H. habilis, was discovered and excavated from early Pleistocene-age fluvial-lacustrine sediments of the upper Burgi Member of the Koobi Fora Formation at site F25787 in Area 13, near Ileret, Kenya. On the surface less than 15 m away, at site F25966, postcranial elements were found, which, if from the same individual as the nearby dentition, would represent the fourth associated craniodental and postcranial assemblage of this species. We developed a geochemical taphonomic history of these ca. 2 Ma hominin fossils using nondestructive X-ray based microanalytical tools (synchrotron and benchtop X-ray fluorescence chemical imaging and micro- and nano-computed tomography volumetric reconstruction), bulk analyses of sediments and paleosols at the excavation sites, and sedimentologic and stratigraphic observations. We integrate the chemical and physical taphonomic histories to test whether teeth (excavated in situ) and postcranial bones (eroded onto the outcrop surface) derive from a single individual. Minor differences in taphonomic history are attributable to the different biomineral properties of the dental and osseous components and to differences in physical damage during early post-mortem scavenging, dispersal, and burial in adjacent depositional settings. Microscale geochemical mapping enabled the temporal ordination of chemical and physical events in the specimens’ chemical taphonomic histories. Specifically, authigenic Fe- and K-bearing clays and Y, U, and Sr uptake occurred in post-burial fractures in bones and were also incorporated pervasively throughout dentin in teeth. Barite mineralization occurred along the latest fractures in both materials, and as a coating on tooth roots. The stratigraphic, taphonomic, and geochemical evidence supports the interpretation that the hominin fossils represent a single individual. Successful application of these nondestructive sample characterization methods demonstrates capabilities for thorough interrogation of the taphonomic histories of other potential hominin fossil associations, enabling more robust and accurate palaeontologic constraints using those relationships.

COMPARISON OF STATE-OF-THE-ART MULTI-VIEW STEREO SOLUTIONS FOR CLOSE RANGE HERITAGE DOCUMENTATION

Abstract. In recent years novel 3D reconstruction methods have been developed to improve the conventional image-based point cloud generation techniques. These novel methods generally attempt to address various challenges encountered in conventional methods, namely, the reconstruction of reflective surfaces and the amount of processing time required, both of which are major bottlenecks in heritage documentation and especially those related to large and complex objects. In this paper, we identified three types of 3D image-based reconstruction techniques and tested their usage on heritage datasets, namely (1) conventional multi-view stereo (MVS), (2) learning-based MVS, and (3) neural radiance fields (NeRF). The aim of this study is to determine the capabilities of these methods in reconstruction of three different heritage-related datasets with different challenges. Our results show that conventional MVS is nowadays a reliable solution for 3D reconstruction, in many instances recording good results relative to the reference terrestrial laser scans (TLS) when properly deployed. When applied to a challenging highly reflective scene, conventional MVS fared well using the PatchMatch algorithm (reaching an object completeness rate of 99.05%), while NeRF’s best performance was 99.98%. However, NeRF suffered from noisy data, some of which may stem from its radiance field-to-point cloud conversion method. The results show that there is great potential in using specific methods for specific cases, and research in combining them may yield interesting results in the future.

Graded arc beam in light needle microscopy for axially resolved, rapid volumetric imaging without nonlinear processes.

High-speed three-dimensional (3D) imaging is essential for revealing the structure and functions of biological specimens. Confocal laser scanning microscopy has been widely employed for this purpose. However, it requires a time-consuming image-stacking procedure. As a solution, we previously developed light needle microscopy using a Bessel beam with a wavefront-engineered approach [Biomed. Opt. Express13, 1702 (2022)10.1364/BOE.449329]. However, this method applies only to multiphoton excitation microscopy because of the requirement to reduce the sidelobes of the Bessel beam. Here, we introduce a beam that produces a needle spot while eluding the intractable artifacts due to the sidelobes. This beam can be adopted even in one-photon excitation fluorescence 3D imaging. The proposed method can achieve real-time, rapid 3D observation of 200-nm particles in water at a rate of over 50 volumes per second. In addition, fine structures, such as the spines of neurons in fixed mouse brain tissue, can be visualized in 3D from a single raster scan of the needle spot. The proposed method can be applied to various modalities in biological imaging, enabling rapid 3D image acquisition.

Fast Multi-View 3D reconstruction of seedlings based on automatic viewpoint planning

Three-dimensional models of plants provide valuable phenotypic information. Phenotypic measurements of plant structures are essential for monitoring plant growth and understanding plant responses to environmental changes. Existing 3D reconstruction techniques have achieved accurate reconstruction models of some plants such as corn and soybeans. However, several drawbacks exist in current plant 3D reconstruction systems, including high cost, fixed capture perspectives and complex operation. To address these issues and considering the influence of camera capture angles on model reconstruction accuracy, we investigated a viewpoint planning reconstruction method. We designed a plant seedling reconstruction system that utilizes a consumer-grade L515 LiDAR sensor and a precision turntable. We establish a viewpoint rule based on the camera imaging model of the three-dimensional spatial structure of leaves, constructing minimum constraints on the leaf normal vector and the camera optical axis. This enables to determine the optimal capture viewpoint for each leaf to obtain the maximum leaf information. The angle β between the leaf normal vector and the camera optical axis serves as the basis for the turntable rotation, systematically planning the rotation of the turntable to enable the camera to capture the maximum useful leaf information. Unlike static camera-based turntable reconstruction systems that capture images at fixed angle intervals, we plan the rotation of the turntable based on the acquired information. This allows us to obtain more effective point cloud information of plant seedlings from the same or even fewer images, reducing the collection of redundant images and reconstructing more accurate plant seedling 3D models. Additionally, we measured commonly used leaf traits in plant phenotypic studies, such as leaf length, leaf width, and leaf area. The leaf area measured based on the reconstructed seedling model exhibited high accuracy (R2 > 0.99). The results of this study demonstrate that the consumer-grade L515 LiDAR sensor combined with the proposed viewpoint planning method can effectively reconstruct accurate seedling 3D models and measure phenotypic information. Due to the absence of error accumulation caused by adjacent view registration, this approach offers advantages such as shorter reconstruction time, higher efficiency and increased accuracy compared to some multi-view point cloud registration and reconstruction methods. Therefore, this system can be applied to three-dimensional reconstruction and phenotype analysis of plant seedlings due to its high efficiency and accuracy.

Focus Estimation Methods for Use in Industrial SFF Imaging Systems

A Shape-From-Focus (SFF) is a three-dimensional imaging method based on focus information. It is not yet widely used for in-line industrial inspection or measurement tasks. The main reasons are the time it takes to capture a 3D image of the inspected product and the presence of interference affecting image quality. This paper compares operators for estimating focus in source images in the scope of their use in constructing an industrial 3D scanner. Interferences were introduced by using additional illuminators and changing the acquisition parameters. The use of industrial-grade cameras, industrial-grade illuminators, and electrically controlled optics are discussed. A novel approach in the research is using an electrically tunable lens to move the position of the image plane during 3D image acquisition. The research was conducted on various surfaces found on typical industrial products. The research showed which focus estimation operators can be applied to SFF imaging within the range of interference considered. It was also confirmed that using the centre of gravity method for scene reconstruction allows for an increase in resolution compared to the maximum method.

Multi-View Jujube Tree Trunks Stereo Reconstruction Based on UAV Remote Sensing Imaging Acquisition System

High-quality agricultural multi-view stereo reconstruction technology is the key to precision and informatization in agriculture. Multi-view stereo reconstruction methods are an important part of 3D vision technology. In the multi-view stereo 3D reconstruction method based on deep learning, the effect of feature extraction directly affects the accuracy of reconstruction. Aiming at the actual problems in orchard fruit tree reconstruction, this paper designs an improved multi-view stereo structure based on the combination of remote sensing and artificial intelligence to realize the accurate reconstruction of jujube tree trunks. Firstly, an automatic key frame extraction method is proposed for the DSST target tracking algorithm to quickly recognize and extract high-quality data. Secondly, a composite U-Net feature extraction network is designed to enhance the reconstruction accuracy, while the DRE-Net feature extraction enhancement network improved by the parallel self-attention mechanism enhances the reconstruction completeness. Comparison tests show different levels of improvement on the Technical University of Denmark (DTU) dataset compared to other deep learning-based methods. Ablation test on the self-constructed dataset, the MVSNet + Co U-Net + DRE-Net_SA method proposed in this paper improves 20.4% in Accuracy, 12.8% in Completion, and 16.8% in Overall compared to the base model, which verifies the real effectiveness of the scheme.