Point Cloud Acquisition Research Articles

A Multi-View Real-Time Approach for Rapid Point Cloud Acquisition and Reconstruction in Goats

The body size, shape, weight, and scoring of goats are crucial indicators for assessing their growth, health, and meat production. The application of computer vision technology to measure these parameters is becoming increasingly prevalent. However, in real farm environments, obstacles, such as fences, ground conditions, and dust, pose significant challenges for obtaining accurate goat point cloud data. These obstacles lead to difficulties in rapid data extraction and result in incomplete reconstructions, causing substantial measurement errors. To address these challenges, we developed a system for real-time, non-contact acquisition, extraction, and reconstruction of goat point clouds using three depth cameras. The system operates in a scenario where goats walk naturally through a designated channel, and bidirectional distributed triggering logic is employed to ensure real-time acquisition of the point cloud. We also designed a noise recognition and filtering method tailored to handle complex environmental interferences found on farms, enabling automatic extraction of the goat point cloud. Furthermore, a distributed point cloud completion algorithm was developed to reconstruct missing sections of the goat point cloud caused by unavoidable factors such as railings and dust. Measurements of body height, body slant length, and chest circumference were calculated separately with deviation of no more than 25 mm and an average error of 3.1%. The system processes each goat in an average time of 3–5 s. This method provides rapid and accurate extraction and complementary reconstruction of 3D point clouds of goats in motion on real farms, without human intervention. It offers a valuable technological solution for non-contact monitoring and evaluation of goat body size, weight, shape, and appearance.

Automatic multi-storey building partitioning from point cloud via bidirectional cloth simulation

With the advancements in scanning technology, the rapid acquisition of point cloud spanning multiple storeys of buildings has become feasible. Accurately dividing the building into individual storeys deepens the understanding of the building environment, accelerates the processing of the vast point cloud, and is crucial for numerous fields such as model reconstruction, indoor navigation, and building management. However, it remains a challenge to handle the building with rooms across multiple storeys. This study presents a novel method to partition complex multi-storey building from cluttered point cloud. First, the non-vertical structural elements are detected by taking the difference of height and normal vector as growth conditions. Next, we iteratively employ bidirectional cloth with height variation constraints to simulate the floor and ceiling trend surface. In this step, candidate ceiling patches are obtained through the optimization of a cost function. Finally, the irregular boundary between adjacent storeys that used for partitioning the point cloud is rasterized in terms of the ceilings at lower storey and the floors at upper storey. Each point is orderly classified into the correspond storey with reference to the elevation of the estimated boundary. Through comprehensive evaluation and analysis, our method has demonstrated its effectiveness and adaptability across various datasets. Our method yields superior results compared to the advanced methods, achieving an approximate value of 99 % on the weighted recall, weighted precision and weighted F1-score. Besides, the proposed method is not limited to Manhattan world buildings and requires only the spatial coordinate of points.

Development of a Scanning Protocol for Anthropological Remains: A Preliminary Study

Structured-light scanning is a fast and efficient technique for the acquisition of 3D point clouds. However, the extensive and daily application of this class of scanners can be challenging because of the technical know-how necessary to validate the low-cost instrumentation. This challenge is worth accepting because of the large amount of data that can be collected accurately with the aid of specific technical protocols. This work is a preliminary study of the development of an acquisition protocol for anthropological remains performing tests in two opposite and extreme contexts: one characterised by a dark environment and one located in an open area and characterised by a very bright environment. This second context showed the influence of sunlight in the acquisition process, resulting in a colourless point cloud. It is a first step towards the development of a technical protocol for the acquisition of anthropological remains, based on the research of limits and problems associated with an instrument.

ICDDPM: Image-conditioned denoising diffusion probabilistic model for real-world complex point cloud single view reconstruction

Point cloud single-view reconstruction (PC-SVR) generates high-quality point clouds from low-cost 2D images, offering a cost-effective solution to the expensive and inefficient acquisition of point cloud for real-world scene typically achieved through expensive LiDAR systems. However, current models that generate point clouds from real-world 2D images (e.g. maritime vessels) still have shortcomings in terms of quality and model generalization. In this paper, we proposed a image-conditioned denoising diffusion probabilistic model (ICDDPM) for real-world complex point cloud single view reconstruction to address these issues. We re-designed the structure of classic diffusion model, use latent shape vectors to seamlessly integrate 2D image encoder, point cloud encoder, and conditional diffusion model, to cater PC-SVR task. By guiding the diffusion process with the 2D images, which serve as crucial conditional information, ICDDPM achieves end-to-end point cloud generation with superior quality. 2D images are also employed as input in the reverse diffusion process to further achieve point cloud generation. We conducted qualitative and quantitative experiments on synthetic dataset ShapeNet and real-world dataset PASCAL3D+ (focused on experiments of vessel point cloud data specifically). The results indicate that the ICDDPM model demonstrates superior performance compared to state-of-the-art models. It is capable of generating point clouds with a greater level of global and local details from various 2D image data. Additionally, the model exhibits strong generalization abilities and requires fewer computational resources.

Research on Point Cloud Acquisition and Calibration of Deep Hole Inner Surfaces Based on Collimated Ring Laser Beams.

In this study, a ring light point cloud calibration technique based on collimated laser beams is developed, aiming to reduce errors caused by the position and attitude changes of traditional ring light measurement devices. This article details the generation mechanism of the ring beam and the principle of deep hole measurement. It introduces the collimated beam as a reference, building on traditional ring light measurement devices, to achieve the synchronous acquisition of the ring beam and collimated spot images by an industrial camera. The Steger algorithm is employed to accurately extract the coordinates of the point cloud contours of both the ring beam and the collimated spot. By analyzing the shape and position changes of the collimated spot contour, the spatial position and attitude of the measuring device are precisely determined. This technique is applied to the 3D reconstruction of the inner surface of deep holes, ensuring the accurate restoration of the spatial positional attitude of the ring beam by incorporating the spatial positional attitude parameters of the measuring device to precisely calibrate the cross-sectional point cloud coordinates. Experimental results with ring gauges and deep hole workpieces demonstrate that this technique effectively reduces the percentage of point cloud data outside the tolerance range, and improves the accuracy of the 3D reconstruction model by 6.287%, thereby verifying the accuracy and practicality of this technique.

A comparative study of four combined methods for planning access to the anterior acetabular column for nail placement

To investigate global optimisation of anterior acetabular column pinning channels can be achieved based on large density point cloud data. Data were collected on the CT scans of the normal pelvis in 40 adults from January 2022 to January 2023, including 22 males and 18 females, aged 20 to 54 years old. Medical imaging data from three of the samples were selected for experimental study. In planning access for anterior acetabular column pinning, to address the issue of whether the current CAD planning methods were advanced or not, four combinations of the same point cloud acquisition channels, different directional line creation software, and the same 3D design and virtual experiment software were proposed: Mimics+Imageware+UG, Mimics+3DReshaper+UG, Mimics+ZEISS Quality Suite+UG and Mimics+Design X+UG, and directional lines created based on the centroid point set and solid point cloud of the secondary pruning model, respectively, and it applied to the planning of the left anterior column pinning channel of the three acetabular samples. The maximum internally connected cylinder without acetabular socket and pubic bone penetration was used as a safe passage for nailing of the anterior acetabular column to evaluate the advancement of each method. The fitting effect of the directional line was better than that of the unnoised solid point cloud when the central point set with obvious relevant features was selected as the sample points;and the combination of Mimics+Imageware+UG and Mimics+3DReshaper+UG could efficiently and stably obtain the desirable planning results when planning with the central point set, respectively, in the three acetabular samples 1, 2, 3. The maximum internal joint circle diameters obtained in samples 1, 2, and 3 were 10.35 mm, 9.62 mm, and 9.24 mm;and when the directional lines were based on the solid point cloud the combined methods of Mimics+ZEISS Quality Suite+UG and Mimics+Design X+UG were not applicable;whereas the Mimics+3DReshaper+UG the solid point cloud denoising planning method could stably obtain the maximum value of the safe channel for nail placement, and the maximum internal joint circle diameters obtained in acetabular samples 1, 2, and 3 are 10.66 mm, 10.96 mm, and 9.48 mm, respectively. It is recommended that the nail placement channel planners use robust Mimics+Imageware+UG or Mimics+3DReshaper+UG centre point set planning method, and if there is enough time, it is recommended to use the solid point cloud denoising planning method of Mimics+3DReshaper+UG in order to obtain the maximum value of safe channels for nail placement.

Machine learning-based 3D scan coverage prediction for smart-control applications

Automatic control of a workpiece being manufactured is a requirement to ensure in-line correction and thus move towards a more intelligent manufacturing system. There is therefore a need to develop control strategies which are capable of taking precise account of real working conditions and enabling first-time-right control. As part of such a smart-control strategy, this paper introduces a machine learning-based approach capable of accurately predicting a priori the 3D coverage of a part according to a scan configuration given as input, i.e. predicting before scanning it which areas of the part will be acquired for real. This corresponds to a paradigm shift, where coverage estimation no longer relies on theoretical visibility criteria, but on rules learned from a large amount of data acquired in real-life conditions. The proposed 3D Scan Coverage Prediction Network (3DSCP-Net) is based on a 3D feature encoding and decoding module, which is capable of taking into account the specifics of the scan configuration whose impact on the 3D coverage is to be predicted. To take account of real working conditions, features are extracted at various levels, including geometric ones, but also features characterising the way structured-light projection behaves. The method is thus able to incorporate inter-reflection and overexposure issues into the prediction process. The database used for the training was built using an ad-hoc platform specially designed to enable the automatic acquisition and labelling of numerous point clouds from a wide variety of scan configurations. Experiments on several parts show that the method can efficiently predict the scan coverage, and that it outperforms conventional approaches based on purely theoretical visibility criteria.

Self-supervised multi-echo point cloud denoising in snowfall

Snowfall can cause noise to light detection and ranging (LiDAR) data. This is a problem since it is used in many outdoor applications, e.g., autonomous driving. We propose the task of multi-echo denoising, where the goal is to pick the echo that represents the objects of interest and discard other echoes. Thus, the idea is to pick points from alternative echoes unavailable in standard strongest echo point clouds. Intuitively, we are trying to see through the snowfall. We propose a novel self-supervised deep learning method and the characteristics similarity regularization to achieve this goal. The characteristics similarity regularization utilizes noise characteristics to increase performance. The experiments with a real-world multi-echo snowfall dataset prove the efficacy of multi-echo denoising and superior performance to the baseline. Moreover, based on extensive experiments on a semi-synthetic dataset, our method achieves superior performance compared to the state-of-the-art in self-supervised snowfall denoising. Our work enables more reliable point cloud acquisition in snowfall. The code is available at https://github.com/alvariseppanen/SMEDen.

Structural Health Monitoring of Bridges with Personal Laser Scanning: Segment-based Analysis of systematic Point Cloud Deformations

Abstract. Bridge structures can be surveyed using a number of different methods. Established are image-based methods using structure from motion by an unmanned aerial vehicle (UAV), terrestrial laser scanning (TLS), or a combination of both methods. Beyond static terrestrial laser scanning, buildings can also be efficiently surveyed using personal laser scanner (PLS) systems. The advantage here is the greater flexibility and increased speed compared to the static method. On the other hand, the accuracy may be more critical, and the resulting point cloud will be more sensitive to systematic global or local deformations under unfavorable measurement conditions. For example, temporary influences can lead to local mapping errors. These include influences such as uneven measurement system motion or non-static, feature-sparse environments. This study investigates the acquisition of 3D point clouds representing the outer shell of a concrete bridge using a PLS system. We demonstrate a method for detecting possible deformations in PLS point clouds using the example of a bridge structure. For this purpose, the reference (TLS) and the PLS point clouds are segmented into individual clusters and a segment-based ICP fine registration is performed. Different RMSE values for the upper road section (0.061 m) and for the pillar segments (0.021 m) as well as different transformation parameters indicate slight displacements in the PLS point cloud. The analysis of the cloud-to-cloud distances showed that there were slight deformations in the Z direction in the area of the road surface. In the lateral direction, no significant residual deviations were found in the area of the bridge pillars.

Remote 3D Imaging and Classification of Pelagic Microorganisms with A Short‐Range Multispectral Confocal LiDAR

AbstractPlankton is essential to maintain healthy aquatic ecosystems since it influences the biological carbon pump globally. However, climate change‐induced alterations to oceanic properties threaten planktonic communities. It is therefore crucial to monitor their abundance to assess the health status of marine ecosystems. In situ optical tools unlock high‐resolution measurements of sub‐millimeter specimens, but state‐of‐the‐art underwater imaging techniques are limited to fixed and small close‐range volumes, requiring the instruments to be vertically dived. Here, a novel scanning multispectral confocal light detection and ranging (LiDAR) system for short‐range volumetric sensing in aquatic media is introduced. The system expands the inelastic confocal principle to multiple wavelength channels, allowing the acquisition of 4D point clouds combining near‐diffraction limited morphological and spectroscopic data that is used to train artificial intelligence (AI) models. Volumetric mapping and classification of microplastics is demonstrated to sort them by color and size. Furthermore, in vivo autofluorescence is resolved from a community of free‐swimming zooplankton and microalgae, and accurate spectral identification of different genera is accomplished. The deployment of this photonic platform alongside AI models overcomes the complex and subjective task of manual plankton identification and enables non‐intrusive sensing from fixed vantage points, thus constituting a unique tool for underwater environmental monitoring.

Plant-Denoising-Net (PDN): A plant point cloud denoising network based on density gradient field learning

Effective point cloud denoising is critical in 3D plant phenotyping applications, which reduces interference in subsequent algorithms and improves the accuracy of plant phenotypes measurement. Deep learning-based point cloud denoising algorithms have shown excellent denoising performance on simple CAD models. However, these algorithms suffer from issues including over-smoothing or shrinkage and low efficiency when applied on density uneven, incomplete, various types of noise and complex plant point clouds. We proposed a plant point cloud denoising network (PDN) based on point cloud density gradient field learning, which can effectively address the challenges posed by plant point clouds. PDN consists of three main modules: point density feature (PDF) exception module, umbrella operator feature (UOF) computation module, and point density gradient (DG) estimation module. The performance of PDN was evaluated in experiments using point clouds of multiple plant species with noise of different types. Under different levels of Gaussian noise, our method achieved a relative performance improvement of 7.6%-19.3% compared to the state-of-the-art baseline methods, reaching state-of-the-art denoising performance. For noise of different types, the majority of our denoising results outperformed the baseline methods. In addition, our method was 0.5 and 8.6 times faster than the baseline methods when processing point clouds with low and high noise level, respectively. The good robustness, generalization, and computational efficacy of PDN are expected to facilitate the acquisition of high-precision 3D point clouds for various plant species, enhance the versatility of 3D phenotyping methods, improve the accuracy of the measurement of structural phenotypes, and increase the throughput of data processing, therefore facilitate the development of modern breeding research. The source code and the datasets used in this study is available on GitHub at https://github.com/suetme/PDN-plant-denoising-net.

Створення тривимірної моделі культової споруди за даними комбінованих геодезичних знімань

The purpose of the research is to test algorithms for creating a three-dimensional model of an object based on point clouds obtained from combined geodetic surveys (UAV survey and ground laser scanning). The methodology is based on the collection of a cloud of points by ground laser scanning and the acquisition of a cloud of points on the same object by the photogrammetric method based on data taken from a UAV. The formation of the resulting three-dimensional model is performed according to the most reliable data from the point of view of geometric accuracy. The results. Each of the point cloud acquisition methods (UAV or laser scanning) has its advantages for certain parts of the building. Thus, the modeling of the upper part of the building, sloping and complex roofs is best performed based on the data taken from the UAV, vertical walls and the elements placed in them (windows, decoration elements, etc.) – based on point clouds obtained from laser scanning. Scientific novelty and practical significance. The method of processing point clouds obtained from various sources on the basis of free software was tested. The approach discussed in the article has significant potential for use in many branches of the national economy. The performer can combine measurement data from various sources on the basis of free software and prepare them for further export to the necessary software for creating a model.

Reinforcement Learning Based Tactile Sensing for Active Point Cloud Acquisition, Recognition and Localization

Reinforcement Learning Based Tactile Sensing for Active Point Cloud Acquisition, Recognition and Localization

OPTIMIZATION OF VR APPLICATION IN TEXTURING CULTURAL HERITAGE

Abstract. The research question underlying this short essay refers to the possibility of realizing through a well-established workflow a high level of immersivity in the VR representation of Cultural Heritage to be used as a working tool by architects and renovators as well as heritage scholars. At present active or passive 3D scanning techniques are a known reality where the choice of data acquisition system depends both on the final purpose and characteristics of the object to be surveyed. With the maturity achieved in point cloud acquisition, post processing phases and the development of BIM-based systems, the model has given the possibility to become a repository of information related to the existing, to be used for maintenance or renovation processes. If the model can represent the existing with a certain level of detail, it can be assumed that a differentiated Level of Immersivity can be organized depending on the specific needs it intends to fulfil. In the context of renovation or preservation of Cultural Heritage, the potential offered by VR becomes more interesting when it can provide a realistic portrait not only of the geometry, but also of the materiality and state of preservation of the buildings. This research opens new possibilities to develop tools to aid designers and renovators in degradation analysis, intervention projects, and scheduled maintenance. The research question starts analyzing the possibility of creating photorealistic immersive environments that are easy to access, organized based on surveys of existing heritage buildings using specific datasets.

A density-controllable 3D measurement system based on linear profile sensor and universal manipulator

In order to offer a simple and efficient solution for obtaining point clouds, a modular 3D measurement system that incorporates an LPS mounted on a universal manipulator is developed. Unlike the requirement for electronic connections between the LPS and the robot to achieve hard synchronization, we employ a high-low-frequency collaborative strategy for soft synchronization, thereby eliminating the need for complex hardware synchronization modules. Since soft synchronization lacks the high-speed synchronous data acquisition of hard synchronization, we have also proposed an interpolation method on low density data to generate uniform higher-density point clouds, thus enabling a density-controllable point clouds acquisition approach. Experimental results demonstrate the effectiveness and versatility of the system.

Integrated approach for facility management of existing buildings using point cloud segmentation

PurposeFacility management gained profound importance due to the increasing complexity of different systems and the cost of operation and maintenance. However, due to the increasing complexity of different systems, facility managers may suffer from a lack of information. The purpose of this paper is to propose a new facility management approach that links segmented assets to the vital data required for managing facilities.Design/methodology/approachAutomatic point cloud segmentation is one of the most crucial processes required for modelling building facilities. In this research, laser scanning is used for point cloud acquisition. The research utilises region growing algorithm, colour-based region-growing algorithm and Euclidean cluster algorithm.FindingsA case study is worked out to test the accuracy of the considered point cloud segmentation algorithms utilising metrics precision, recall and F-score. The results indicate that Euclidean cluster extraction and region growing algorithm revealed high accuracy for segmentation.Originality/valueThe research presents a comparative approach for selecting the most appropriate segmentation approach required for accurate modelling. As such, the segmented assets can be linked easily with the data required for facility management.

Semantic Segmentation and Roof Reconstruction of Urban Buildings Based on LiDAR Point Clouds

In urban point cloud scenarios, due to the diversity of different feature types, it becomes a primary challenge to effectively obtain point clouds of building categories from urban point clouds. Therefore, this paper proposes the Enhanced Local Feature Aggregation Semantic Segmentation Network (ELFA-RandLA-Net) based on RandLA-Net, which enables ELFA-RandLA-Net to perceive local details more efficiently by learning geometric and semantic features of urban feature point clouds to achieve end-to-end building category point cloud acquisition. Then, after extracting a single building using clustering, this paper utilizes the RANSAC algorithm to segment the single building point cloud into planes and automatically identifies the roof point cloud planes according to the point cloud cloth simulation filtering principle. Finally, to solve the problem of building roof reconstruction failure due to the lack of roof vertical plane data, we introduce the roof vertical plane inference method to ensure the accuracy of roof topology reconstruction. The experiments on semantic segmentation and building reconstruction of Dublin data show that the IoU value of semantic segmentation of buildings for the ELFA-RandLA-Net network is improved by 9.11% compared to RandLA-Net. Meanwhile, the proposed building reconstruction method outperforms the classical PolyFit method.

The Importance of Virtual Reality in the Conservation of Cultural Heritage: An Examination of the Penataran Temple Case Study in Blitar Regency, East Java

Introduction: Visualization of digital archives and applications play a crucial role in preserving cultural heritage. Utilizing efficient 3D scanning, modeling, and deep learning-based reconstruction techniques enables the revelation of hidden parts of Penataran Temple Sample in Blitar Regency, East Java. This research emphasizes the utilization of virtual reality (VR) systems as digital repositories for Penataran Temple Example, facilitating in-depth exploration of this cultural asset. The importance of applications such as guided tours, computer-aided design, and geographic information systems increases in the examination of cultural heritage, offering a deeper and more comprehensive understanding of the preserved cultural wealth. Methods: The methodology used in this research includes photogrammetry, accurate photogrammetric assessment, and the utilization of deep learning. Information obtained from these three avenues is combined to construct a three-dimensional Point cloud, serving as the foundation for the digital repository of Penataran Temple Sample. Virtual reality (VR) platforms enable individuals to observe the temple through both first-person and aerial perspectives, featuring customizable interfaces within the virtual reality setting. Results: The research activities are dedicated to creating a digital repository for Penataran Temple Sample located in Blitar Regency, East Java using 3D Point cloud data. Data acquisition of Point cloud for comprehensive coverage of temple structures and hidden reliefs is achieved through the application of advanced photogrammetric measurements and deep learning technology. A virtual reality (VR) system is developed to offer deep visualization capabilities, including first-person perspectives and aerial views, fostering a profound understanding of the cultural significance and architectural design of the temple. To enhance the realism of the VR experience and enable mapping functions, a 3D simulation engine is incorporated. The utilization of the first-person perspective allows detailed virtual exploration and collision detection, while aerial views contribute to a comprehensive understanding of the temple architecture. Discussion: The digital archive system of Penataran Temple Sample leverages Virtual Reality (VR) technology to integrate 3D Point cloud data from various sources. This system facilitates both first-person perspectives and aerial perspectives, allowing comprehensive examination and in-depth understanding of the temple architectural layout. Accurate 3D data generation is achieved through the utilization of Unmanned Aerial Vehicles (UAVs), photogrammetry technology, and deep learning techniques. Conclusion: The investigation focuses on the utilization of VR technology as a digital repository at Penataran Temple Sample, employing 3D Point cloud methodology. Its primary aim is to provide engaging visual encounters and comprehensive understanding of the architectural complex. Furthermore, this research will encompass detailed 3D measurements and the creation of high-precision VR renderings of the complete temple structures. Additionally, there are opportunities to design VR applications for remote utilization and to produce stereoscopic perspectives through holographic means. The anticipation is that VR will play a significant role in the preservation and interpretation of the heritage of Penataran Temple Sampled in Blitar Regency, East Java, while also demonstrating potential for diverse applications across various platforms and settings.

Cucumber Seedling Segmentation Network Based on a Multiview Geometric Graph Encoder from 3D Point Clouds.

Plant phenotyping plays a pivotal role in observing and comprehending the growth and development of plants. In phenotyping, plant organ segmentation based on 3D point clouds has garnered increasing attention in recent years. However, using only the geometric relationship features of Euclidean space still cannot accurately segment and measure plants. To this end, we mine more geometric features and propose a segmentation network based on a multiview geometric graph encoder, called SN-MGGE. First, we construct a point cloud acquisition platform to obtain the cucumber seedling point cloud dataset, and employ CloudCompare software to annotate the point cloud data. The GGE module is then designed to generate the point features, including the geometric relationships and geometric shape structure, via a graph encoder over the Euclidean and hyperbolic spaces. Finally, the semantic segmentation results are obtained via a downsampling operation and multilayer perceptron. Extensive experiments on a cucumber seedling dataset clearly show that our proposed SN-MGGE network outperforms several mainstream segmentation networks (e.g., PointNet++, AGConv, and PointMLP), achieving mIoU and OA values of 94.90% and 97.43%, respectively. On the basis of the segmentation results, 4 phenotypic parameters (i.e., plant height, leaf length, leaf width, and leaf area) are extracted through the K-means clustering method; these parameters are very close to the ground truth, and the R 2 values reach 0.98, 0.96, 0.97, and 0.97, respectively. Furthermore, an ablation study and a generalization experiment also show that the SN-MGGE network is robust and extensive.

A Calibration Algorithm of 3-D Point Cloud Acquisition System Based on KMPE Cost Function

A Calibration Algorithm of 3-D Point Cloud Acquisition System Based on KMPE Cost Function