3D Point Cloud Model Research Articles

Point cloud registration method for maize plants based on conical surface fitting\u2014ICP

Reconstructing three-dimensional (3D) point cloud model of maize plants can provide reliable data for its growth observation and agricultural machinery research. The existing data collection systems and registration methods have low collection efficiency and poor registration accuracy. A point cloud registration method for maize plants based on conical surface fitting—iterative closest point (ICP) with automatic point cloud collection platform was proposed in this paper. Firstly, a Kinect V2 was selected to cooperate with an automatic point cloud collection platform to collect multi-angle point clouds. Then, the conical surface fitting algorithm was employed to fit the point clouds of the flowerpot wall to acquire the fitted rotation axis for coarse registration. Finally, the interval ICP registration algorithm was used for precise registration, and the Delaunay triangle meshing algorithm was chosen to triangulate the point clouds of maize plants. The maize plant at the flowering and kernel stage was selected for reconstruction experiments, the results show that: the full-angle registration takes 57.32 s, and the registration mean distance error is 1.98 mm. The measured value’s relative errors between the reconstructed model and the material object of maize plant are controlled within 5%, the reconstructed model can replace maize plants for research.

Digitization of two urban archaeological areas in Reggio Calabria (Italy): Roman Thermae and Greek fortifications

The digitization of Heritage represents a great challenge that Italy is facing in order to improve the state of documentation, conservation, enhancement and dissemination of its huge cultural assets. This paper, part of the project Circuito delle Aree Archeologiche Urbane di Reggio Calabria (a collaboration between the Municipality of Reggio Calabria and the CNR-IPCF of Messina), shows the complete workflow (from field survey to web-sharing) carried out for the enhancement of two archaeological sites located in the urban center of Reggio Calabria. The study-cases selected are represented by the Roman Thermae and the Greek fortification, both located along the Falcomatà waterfront. The metric surveys have been based on the use of terrestrial laser scanning and close-range photogrammetry, well-established techniques for the production of graphic documentation and 3D digital replicas of artifacts ranging from architectural up to object scale. The final outcome of our action is the implementation of a web-based platform (built up with Faro WebShare Cloud) where expert and non-expert users (e.g. scholars, professionals, administrators, tourists) can navigate image-based and 3D point-cloud models and, in addition, search metadata of their specific interest (media, old pictures, results of analyses etc.). This product, absolutely unprecedented and singular for a southern Italy Municipality, more than an interactive museum, is configured as an integrated and partecipative tool capable of improving the state of knowledge, conservation, valorization and dissemination of these sites and their environments. Because of its singularity, this solution can be also considered as a case-study for the evaluation of long-term effects on the growth of tourism, inclusivity and accessibility of cultural places during particular situations (maintenance works, pandemic emergency etc.).

DAMR: A deep gaussian mixture point cloud registration method with the dual attention mechanism

Point cloud registration is important for the processing of 3D model reconstruction, but with the challenge of low registration accuracy. In order to overcome this obstacle, this paper proposed a dual attention mechanism registration (DAMR) method for the point cloud registration. Firstly, the dual attention mechanism is utilized to extract key point features with different dimensions via assigning weights to the input point clouds. Secondly, the matching parameters are obtained by estimating the position correspondence between feature points and gaussian mixture model. Finally, the parameter unit blocks of two models are applied to recover the optimal transformation by matching parameters. In order to test the performance of our method, four groups of datasets include public data models and cultural relic models are adopted, respectively. Compared with traditional methods, our method has shown better performance to effectively ensured the registration accuracy of 3D point cloud models.

An improved computation method for asphalt pavement texture depth based on multiocular vision 3D reconstruction technology

The depth of an asphalt pavement structure is an important index used to reflect the anti-skid performance of the pavement, which directly affects the driving safety of vehicles. In the current specifications (e.g., China, European and American standards), the mean texture depth (MTD) of an asphalt pavement is generally measured by the sand-patch method (SPM). However, SPM is easily affected by the operator’s subjective experience and experimental environment, resulting in low accuracy and high scatter in the data. In view of such shortcomings, a fast and accurate method to obtain the texture depth of asphalt pavement was proposed by designing a new test tool and a computer-aided calculation method. The multiocular vision theory was adopted to reconstruct a three-dimensional (3D) point cloud model of pavement texture. To reflect the concavity and convexity of pavement texture and eliminate the influence of pavement slope, the iterative closest point (ICP) algorithm was employed to obtain the datum reference plane (DRP). On this basis, the pentahedral volume calculation method suitable for evaluating the texture depth of an asphalt pavement was proposed. Laboratory experiment results using SPM were inverted by equal volume calculation to obtain the texture reference plane (TRP) and MTD'. Meanwhile, a 3D average maximum elevation (Avg.EL) algorithm was obtained by the evolution of two-dimensional mean profile depth (MPD). Furthermore, three parameters, namely MTD, MTD', and Avg.EL, were used to measure the texture depth of two field pavement sections in service. The experimental results showed that the correlation between MTD' and MTD was better than that between Avg.EL and MTD, and the degree of scatter was smaller. MTD' could be used directly as a reference value for MTD. In the present work, the 3D volume calculation method of asphalt pavement texture depth was proposed, which integrates the technical flow of image acquisition, model reconstruction, and algorithm analysis. The developed method provides a technical reference for the application of computer vision technology in the analysis of pavement texture depth.

IoT-based low-cost 3D mapping using 2D Lidar for different materials

Three-dimensional (3D) mapping is the technology of profiling objects in three dimensions to map them in real life. 3D mapping is an excellent tool for analysis, surveying, and other engineering domains. 3D light detection and ranging (Lidar) technology usually does 3D mapping, but the cost of 3D Lidar is very high, which constrains its applicability in the Industry. The need for a low-cost 3D mapping has increased widely. A cost-effective 3D mapping based on the Internet of Things (IoT) using 2D Lidar is proposed. 2D Lidar is lower in cost than 3D Lidar, so it completely replaces the 3D Lidar for applications that need moderate or low accuracy. This research sought to confirm the applicability of this low-cost IoT-based advanced sensing technology, i.e., Lidar, in the engineering domain. A two-dimensional Lidar mapping system prototype is constructed, and the same is used to generate three-dimensional (3D) point cloud models. Lidar measurements are greatly dependent on the materials. Every material has different absorbance and reflectivity, which alter the Lidar measurements. In this research, we have shown how different material affects Lidar accuracy.

Bidirectional interaction between BIM and construction processes using a multisource geospatial data enabled point cloud model

Building Information Modeling (BIM) has increasingly been adopted as an as-planned construction state to provide essential support for fine construction. However, some deviations are inevitably induced between the BIM model and actual construction states in practical construction processes, and these deviations obstruct the unidirectional interaction between the BIM model and actual construction states, thus seriously degrading the fine construction quality. This paper proposes a bidirectional interaction mechanism between BIM and construction processes using a multisource geospatial data enabled point cloud model. In the forward interaction, parametric BIM model are obtained to provide essential information for fine construction. Particularly, a multisource geospatial data enabled point cloud modeling strategy is the core of the proposed method, which overcomes complex buildings characterized by severe occlusions, specular surfaces and similar components to capture accurate and complete 3D point cloud model that reflects the actual construction state. In the backward interaction, the point cloud model provides feedback to adjust the BIM model for ensuring a fit between the BIM model and actual construction state. Moreover, the resulting up-to-date BIM model can further instruct the subsequent construction processes; thus, BIM and construction become a closed task-oriented loop. The proposed method was applied to the curtain wall construction of the main stadium for the Chengdu 2022 31st Summer World University Games and compared with four other state-of-the-art point cloud registration methods. The results demonstrated the superior accuracy of our method over other methods. The results also showed that the proposed method could effectively maintain the interaction between the BIM model and actual construction states throughout the construction lifecycle and ensure the quality of fine construction.

Point cloud and BIM model registration based on genetic algorithm and ICP algorithm

With the rapid development of the construction industry, BIM technology, and 3D laser scanning technology are being used more and more widely, and there are many applications of combining BIM technology with 3D laser scanning technology, such as quality inspection, progress inspection, or digital preservation of ancient buildings. Therefore, this paper proposes a 3D point cloud and BIM model registration scheme based on genetic algorithm and ICP algorithm, firstly, the point cloud data is pre-processed by statistical denoising method for denoising and downsampling, and the BIM model data is converted to format data; then the coarse registration is performed by genetic algorithm, and the accurate registration is performed by ICP algorithm based on KD-tree, and finally, we experimentally verify the feasibility of the algorithm in this paper, and compared with the ICP algorithm, the registration efficiency and accuracy of the algorithm in this paper are greatly improved.

Hole Repairing Algorithm for 3D Point Cloud Model of Symmetrical Objects Grasped by the Manipulator.

For the engineering application of manipulator grasping objects, mechanical arm occlusion and limited imaging angle produce various holes in the reconstructed 3D point clouds of objects. Acquiring a complete point cloud model of the grasped object plays a very important role in the subsequent task planning of the manipulator. This paper proposes a method with which to automatically detect and repair the holes in the 3D point cloud model of symmetrical objects grasped by the manipulator. With the established virtual camera coordinate system and boundary detection, repair and classification of holes, the closed boundaries for the nested holes were detected and classified into two kinds, which correspond to the mechanical claw holes caused by mechanical arm occlusion and the missing surface produced by limited imaging angle. These two kinds of holes were repaired based on surface reconstruction and object symmetry. Experiments on simulated and real point cloud models demonstrate that our approach outperforms the other state-of-the-art 3D point cloud hole repair algorithms.

A Point Cloud Fusion Method for Space Target 3D Laser Point Cloud and Visible Light Image Reconstruction Point Cloud

In this paper, a high-precision spatial target lidar for 3D reconstruction of point cloud and visible light image 3D reconstruction point cloud fusion method is proposed. This fusion method uses the solved 3D reconstruction to invert the camera pose and 3D point cloud model centroid position optimization initial value selection. The registration accuracy and registration efficiency of the ICP algorithm are improved. At the same time, according to the characteristics of the two sets of point clouds, the Euclidean distance threshold is used to delete the noise points of the 3D point cloud edge, and the high-precision 3D reconstruction point with the scale information fusion is obtained. cloud. The simulation experiment of the spatial target simulation model shows that the fusion method can effectively improve the point cloud density, fill the reconstruction vulnerability, and improve the point cloud accuracy of the spatial target 3D reconstruction.

Novel system for the automatization of photogrammetric data capture for metrological tasks: application to study of gears

This paper presents a novel methodology for the macro-photogrammetric reconstruction of gears for metrological and maintenance purposes. The process presented includes the automation of close-range photogrammetric data acquisition using a referenced platform supported by measures gathered using an Articulated Coordinate Measure Machine and the subsequent Structure for Motion processing that allows the generation of dense 3D point cloud models with metrical scale and photorealistic colour texture of the gears. On these models, it is possible to directly take measurements, extract different profiles that can be exported in 2D formats and apply advanced processes that allow the computation of roughness and curvature parameters to evaluate the wear and possible defects or pathologies.

Canopy Volume Extraction of Citrus reticulate Blanco cv. Shatangju Trees Using UAV Image-Based Point Cloud Deep Learning

Automatic acquisition of the canopy volume parameters of the Citrus reticulate Blanco cv. Shatangju tree is of great significance to precision management of the orchard. This research combined the point cloud deep learning algorithm with the volume calculation algorithm to segment the canopy of the Citrus reticulate Blanco cv. Shatangju trees. The 3D (Three-Dimensional) point cloud model of a Citrus reticulate Blanco cv. Shatangju orchard was generated using UAV tilt photogrammetry images. The segmentation effects of three deep learning models, PointNet++, MinkowskiNet and FPConv, on Shatangju trees and the ground were compared. The following three volume algorithms: convex hull by slices, voxel-based method and 3D convex hull were applied to calculate the volume of Shatangju trees. Model accuracy was evaluated using the coefficient of determination (R2) and Root Mean Square Error (RMSE). The results show that the overall accuracy of the MinkowskiNet model (94.57%) is higher than the other two models, which indicates the best segmentation effect. The 3D convex hull algorithm received the highest R2 (0.8215) and the lowest RMSE (0.3186 m3) for the canopy volume calculation, which best reflects the real volume of Citrus reticulate Blanco cv. Shatangju trees. The proposed method is capable of rapid and automatic acquisition for the canopy volume of Citrus reticulate Blanco cv. Shatangju trees.

Hierarchical CNN-based occlusal surface morphology analysis for classifying posterior tooth type using augmented images from 3D dental surface models

Objective3D Digitization of dental model is growing in popularity for dental application. Classification of tooth type from single 3D point cloud model without assist of relative position among teeth is still a challenging task. MethodsIn this paper, 8-class posterior tooth type classification (first premolar, second premolar, first molar, second molar in maxilla and mandible respectively) was investigated by convolutional neural network (CNN)-based occlusal surface morphology analysis. 3D occlusal surface was transformed to depth image for basic CNN-based classification. Considering the logical hierarchy of tooth categories, a hierarchical classification structure was proposed to decompose 8-class classification task into two-stage cascaded classification subtasks. Image augmentations including traditional geometrical transformation and deep convolutional generative adversarial networks (DCGANs) were applied for each subnetworks and cascaded network. ResultsResults indicate that combing traditional and DCGAN-based augmented images to train CNN models can improve classification performance. In the paper, we achieve overall accuracy 91.35%, macro precision 91.49%, macro-recall 91.29%, and macro-F1 0.9139 for the 8-class posterior tooth type classification, which outperform other deep learning models. Meanwhile, Grad-cam results demonstrate that CNN model trained by our augmented images will focus on smaller important region for better generality. And anatomic landmarks of cusp, fossa, and groove work as important regions for cascaded classification model. ConclusionThe reported work has proved that using basic CNN to construct two-stage hierarchical structure can achieve the best classification performance of posterior tooth type in 3D model without assistance of relative position information. The proposed method has advantages of easy training, great ability to learn discriminative features from small image region.

Research on Classification Method of 3D Point Cloud Model Based on KE-PointVNet

With the popularization and use of three-dimensional data acquisition equipment, the acquisition of three-dimensional data is more convenient. Three-dimensional data contains rich shape and scale information. How to effectively and accurately classify, segment and identify three-dimensional data is a research hotspot in the field of computer vision. Aiming at the particularity of 3D point cloud data and the neglect of local correlation between points, this paper proposes a new end-to-end depth network framework, namely KE-PointVNet. KE-PointVNet can directly deal with point cloud and construct deep convolutional network. KE-PointVNet extracts local geometric features of each point based on the EdgeConv module. By using the KNN proximity algorithm to construct local neighborhood of the point cloud, it can avoid its non-local diffusion, and finally obtain the high-level semantics of the point cloud through the local aggregation vector VLAD layer. The experimental results show that this method has a higher classification accuracy than most of the existing point cloud classification methods.

In-Situ Block Characterization of Jointed Rock Exposures Based on a 3D Point Cloud Model

The importance of in-situ rock block characterization has been realized for decades in rock mechanics and engineering, yet how to reliably measure and characterize the geometrical properties of blocks in varied forms of exposures and patterns of jointing is still a challenging task. Using a point cloud model (PCM) of rock exposures generated from remote sensing techniques, we developed a consistent and comprehensive method for rock block characterization that is composed of two different procedures and a block indicator system. A semi-automatic procedure towards the robust extraction of in-situ rock blocks created by the deterministic discontinuity network on rock exposures (PCM-DDN) was developed. A 3D stochastic discrete fracture network (DFN) simulation (PCM-SDS) procedure was built based on the statistically valid representation of the discontinuity network geometry. A multi-dimensional block indicator system, i.e., the block size, shape, orientation, and spatial distribution pattern for systematic and objective block characterization, was then established. The developed method was applied to a synthetic model of cardboard boxes and three different rock engineering scenarios, including a road cut slope from Spain and two open-pit mining slopes from China. Compared with existing empirical methods, the proposed procedures and the block indicator system are dependable and practically feasible, which can help enhance our understanding of block geometry characteristics in related applications.

An Alternative Method for Long-Term Monitoring of Thai Historic Pagodas Based on Terrestrial Laser Scanning Data: A Case Study of Wat Krachee in Ayutthaya

This article presents an alternative method for the long-term monitoring of heritage pagodas in Thailand. In this method, terrestrial laser scanning (TLS) is used in combination with permanent survey markers. The Wat (temple) Krachee in the Ayutthaya Province of Thailand was chosen as a case study. This temple has several fantastic elements, including an inverted bell-shaped pagoda, two intertwined trees growing within it, and a chamber inside the pagoda. The preservation team working on the pagoda encountered a challenging problem and faced a decision to trim or not to trim the tree since it has a long-term effect on the pagoda’s structural stability. A high-accuracy terrestrial laser scanner was used to collect three-dimensional point cloud data. Permanent survey markers were constructed in 2018 to be used in long-term monitoring. The 3D surveying of the temple and the monitoring of the pagoda were carried out in five sessions during a period ending in 2020. A point cloud data analysis was performed to obtain the current dimensions, a displacement analysis, and the pagoda leaning angle. The results revealed that the terrestrial laser scanner is a high-performance piece of equipment offering efficient evaluation and long-term monitoring. However, in this study, permanent survey markers were also required as a benchmark for constraining each monitoring session. The 3D point cloud models could be matched with the assumption model elements to evaluate the damaged shape and to determine the original form. The significant elements of an inverted bell-shaped pagoda were investigated. Trimming the tree was found to cause the leaning angle of the pagoda to decrease. An equation was developed for predicting the leaning angle of the Wat Krachee pagoda for preservation and restoration planning in the future. From the results of this study, it is recommended that periodic monitoring should continue in order to preserve Thai pagodas in their original forms.

3D Point Cloud on Semantic Information for Wheat Reconstruction

Phenotypic analysis has always played an important role in breeding research. At present, wheat phenotypic analysis research mostly relies on high-precision instruments, which make the cost higher. Thanks to the development of 3D reconstruction technology, the reconstructed wheat 3D model can also be used for phenotypic analysis. In this paper, a method is proposed to reconstruct wheat 3D model based on semantic information. The method can generate the corresponding 3D point cloud model of wheat according to the semantic description. First, an object detection algorithm is used to detect the characteristics of some wheat phenotypes during the growth process. Second, the growth environment information and some phenotypic features of wheat are combined into semantic information. Third, text-to-image algorithm is used to generate the 2D image of wheat. Finally, the wheat in the 2D image is transformed into an abstract 3D point cloud and obtained a higher precision point cloud model using a deep learning algorithm. Extensive experiments indicate that the method reconstructs 3D models and has a heuristic effect on phenotypic analysis and breeding research by deep learning.

Semantic Structure from Motion for Railroad Bridges Using Deep Learning

Current maintenance practices consume significant time, cost, and manpower. Thus, a new technique for maintenance is required. Construction information technologies, including building information modeling (BIM), have recently been applied to the field to carry out systematic and productive planning, design, construction, and maintenance. Although BIM is increasingly being applied to new structures, its application to existing structures has been limited. To apply BIM to an existing structure, a three-dimensional (3D) model of the structure that accurately represents the as-is status should be constructed and each structural component should be specified manually. This study proposes a method that constructs a 3D model and specifies the structural component automatically using photographic data with a camera installed on an unmanned aerial vehicle. This procedure is referred to as semantic structure from motion because it constructs a 3D point cloud model together with semantic information. A validation test was carried out on a railroad bridge to validate the performance of the proposed system. The average precision, intersection over union, and BF scores were 80.87%, 66.66%, and 56.33%, respectively. The proposed method could improve the current scan-to-BIM procedure by generating the as-is 3D point cloud model by specifying the structural component automatically.

Construction of Feature Tensor Descriptor and Self-Similarity Analysis for 3D Point Cloud Models

<p indent=0mm>Local self-similarity of 3D model is a fundamental problem in the shape analysis. The construction of a local shape descriptor is very important to the final result of self-similarity analysis. To solve this problem, a self-similarity analysis method based on the tensor fusion feature descriptor is proposed. Firstly, the shape diameter function (SDF) of a point cloud model is approximately calculated by using relevant facets and antipodal points. Then, spectral clustering is used to segment the model into sub-blocks, and the three-dimensional feature tensor is constructed from the SDF, shape index (SI) and Gauss curvature (GS) matrix of <italic>K</italic>NN neighborhood points. Finally, the shape descriptor is obtained by constructing the mapping with the tensor norm, and then the similarity measure is defined and the self-similarity between the sub-blocks of the model is analyzed. Several state-of-the-art methods (including partial matching and saliency detection) are tested. In terms of not only the visual effect, but also the similarity measure and the relative errors, the results show that this method can effectively describe the shape and improves the recognition accuracy of similar sub-blocks of a point cloud model.

LPMNet: Latent part modification and generation for 3D point clouds

In this paper, we focus on latent modification and generation of 3D point cloud object models with respect to their semantic parts. Different to the existing methods which use separate networks for part generation and assembly, we propose a single end-to-end Autoencoder model that can handle generation and modification of both semantic parts, and global shapes. The proposed method supports part exchange between 3D point cloud models and composition by different parts to form new models by directly editing latent representations. This holistic approach does not need part-based training to learn part representations and does not introduce any extra loss besides the standard reconstruction loss. The experiments demonstrate the robustness of the proposed method with different object categories and varying number of points. The method can generate new models by integration of generative models such as GANs and VAEs and can work with unannotated point clouds by integration of a segmentation module.

Visualization of 3D rock mass properties in underground tunnels using extended reality

The term extended reality (XR) refers to a family of technologies that cover Virtual Reality (VR), Augmented Reality (AR), and Mixed Reality (MR). The main benefit of XR is that it can offer a new viewpoint on the surrounding environment by augmenting it with digital data and visualizations. Recent developments of XR enable its deployment for rock engineering applications, including underground tunnels. In this paper, we demonstrate two cases of the use of XR in an underground tunnel to display spatial information on the tunnel surface. One of the tunnels in the Underground Research Laboratory of Aalto University (URLA) was digitized using Structure-from-motion (SfM) photogrammetry. As a result, a high-resolution 3D point cloud and textured model of the tunnel were created. Next, the rock joint planes were obtained semi-automatically from the digitized rock surfaces. The results are then represented in their actual positions in the tunnel geometry. In the first case, we used VR to display the rock joint planes on the textured model of the tunnel. In the second case, the data was displayed in real-time in tunnel conditions through a mobile device. The results demonstrate that XR technology can be successfully used in underground construction to digitize the workplace and provide a new perspective on the work environment, which can potentially lead to an increase in safety and productivity.