Surface Point Cloud Research Articles

Filling the Holes on 3D Heritage Object Surface Based on Automatic Segmentation Algorithm

ABSTRACTReconstructing and processing 3D objects are activities in the research field of computer graphics, image processing and computer vision. The 3D objects are processed based on geometric modelling (a branch of applied mathematics and computational geometry) or machine learning algorithms based on image processing. The computation of geometrical objects includes processing the curves and surfaces, subdivision, simplification, meshing, holes filling or reconstructing the 3D surface's objects on both point cloud data and triangular mesh. While the machine learning methods are developed using deep learning models. With the support of 3D laser scan devices and LiDAR techniques, the obtained dataset is close to the original shape of the real objects. Besides, photography and its application based on modern techniques in recent years help us collect data and process the 3D models more precisely. This article proposes a new method for filling holes on the 3D object's surface based on automatic segmentation. Instead of filling the hole directly as the existing methods, we now subdivide the hole before filling it. The hole is first determined and segmented automatically based on the computation of its local curvature. It is then filled on each part of the hole to match its local curvature shape. The method can work on both 3D point cloud surfaces and triangular mesh surfaces. Compared to the state‐of‐the‐art (SOTA) methods, our proposed method obtained higher accuracy of the reconstructed 3D objects.

Intelligent processing of UAV remote sensing data for building high-precision DEMs in complex terrain: A case study of Loess Plateau in China

The Loess Plateau in China is renowned for its dense gullies and complex terrain, with drastic changes primarily due to soil erosion and human activities, significantly affecting the evolution of the ecological environment. The complex terrains and dense vegetation make precise terrain measurement and modeling challenging. Although the development of Unmanned Aerial Vehicle (UAV) light detection and ranging (LiDAR) scanning and photogrammetry technologies has improved data acquisition precision, relying solely on one remote sensing technology struggles with accurately extracting bare earth information. This study adopted a method that fuses UAV lidar scanning with aerial photogrammetric imagery, generating detailed lidar point cloud data that includes coordinate, reflectance, true color, and texture information to enhance data classifiability and interpretability. Subsequently, a point cloud classification model based on the Transformer architecture (Stratified Transformer) is introduced to intelligently complete the initial ground point cloud extraction in complex gully terrains. Further, to address residual non-ground noise in the initial ground point clouds, a new point cloud classification optimization algorithm (MDD, Multi-scale C2M Distance Difference) is proposed. This algorithm, based on the characteristics of discrete and non-continuous with the ground surface of the noisy point clouds, effectively eliminates the discrete noisy point clouds by analyzing the distances between the point clouds and TINs (Triangular Irregular Networks) of different scales and their differences. This study effectively addresses the technical challenges of ground point cloud extraction in the mixed environment of complex terrain and vegetation, solving the problem of precise terrain measurement and intelligent data processing in complex gully terrains, and offering new technical pathways for detecting geomorphological changes.

A generalized model of cardiac surface motion for evaluating left anterior descending coronary artery dose in left breast cancer radiotherapy.

Retrospective studies indicate that radiation damage to left anterior descending coronary artery (LAD) may be critical for late-stage radiation-induced cardiac morbidity. Developing a method that accurately depicts LAD motion and perform dose assessment is crucial. To construct a generalized cardiac surface motion model for LAD dose assessment in left breast cancer radiotherapy. Cine MRI of 25 cases were divided into training and testing sets for model construction, and five external cases were gathered for generalization validation. Motion prediction from average intensity projection images (AIP) surface point cloud to that of each phase was realized by mapping the relationship between datum points and corresponding points with statistical shape modeling (SSM). Root mean square error (RMSE) for predicted corresponding points and Euclidean distance (ED) for predicted surface point cloud were used to assess model's accuracy. LAD dose assessment for 10 left breast cancer radiotherapy cases was perform by model application. The RMSE in testing cases and external cases were 0.209±0.020mm to 0.841±0.074mm and 0.895±0.093mm to 1.912±0.138mm, respectively; while the ED were 1.399±0.029mm to 1.658±0.100mm, 1.571±0.080mm to 1.779±0.104mm, respectively, proving the generalized model's high accuracy. The volume of LAD characterizing motion range (WPLAD) (2.392±0.639 cm3) was approximately twice that of LAD from superimposed images (SPLAD) (0.927±0.326 cm3) with p<0.05, and the former's Dmax (3582.06±575.92 cGy) was significantly larger than latter's (3222.71±665.37 cGy) (p<0.05). While WPLAD's Dmean (1408.06±413.06 cGy) was slightly smaller than that of SPLAD (1504.15±448.03 cGy), the difference did not reach statistical significance (p>0.05). WPLAD's V20 (23.42%±16.62%) was less than SPLAD's (29.18%±21.07%) with p<0.05, but their comparison in V30 and V40 did not yield statistically significant results. It implies the conventional LAD dose assessment ignores motion impact and may not be justified. The generalized cardiac surface motion model informs LAD dose accurate assessment in left breast cancer radiotherapy.

Parametric indoor 3D reconstruction based on binary image feature point extraction

Abstract In recent years, with the rapid advancement of laser scanning technology, indoor 3D modeling technology has significantly developed, becoming a research hotspot in the field of 3D reconstruction. Addressing the issues of high complexity in decoding point cloud surfaces in traditional indoor reconstruction, this paper proposes an indoor 3D reconstruction method based on binary image feature point extraction. The method utilizes point cloud data for semantic segmentation of the indoor environment to accurately extract the ceiling area, which is then projected onto the XOY plane and converted into a binary image, thereby transforming the 3D problem into a 2D problem. Subsequently, key feature points are identified through feature point detection technology, and systematic model construction is carried out based on the sequence of these feature points and their associated parameters. This method effectively simplifies the complexity of traditional 3D modeling, achieving accurate and parametric indoor 3D reconstruction.

Point cloud data-driven modelling of high-strength steel wire corrosion pits considering orientation features

We propose a corrosion pit modelling approach of high-strength steel wires considering orientation features using surface point cloud data, collected by three-dimensional laser scanning. Corrosion pits were identified through density-based spatial clustering of applications with noise (DBSCAN). The identified corrosion pits could be modelled as a semiellipsoid considering orientation features. The Gaussian copula model was utilised to describe the joint distribution of the geometric parameters of the corrosion pits. This approach will be valuable for simulation of the surface morphology of corroded high-strength steel wires.

Compressing and Recovering Short-Range MEMS-Based LiDAR Point Clouds Based on Adaptive Clustered Compressive Sensing and Application to 3D Rock Fragment Surface Point Clouds.

Short-range MEMS-based (Micro Electronical Mechanical System) LiDAR provides precise point cloud datasets for rock fragment surfaces. However, there is more vibrational noise in MEMS-based LiDAR signals, which cannot guarantee that the reconstructed point cloud data are not distorted with a high compression ratio. Many studies have illustrated that wavelet-based clustered compressive sensing can improve reconstruction precision. The k-means clustering algorithm can be conveniently employed to obtain clusters; however, estimating a meaningful k value (i.e., the number of clusters) is challenging. An excessive quantity of clusters is not necessary for dense point clouds, as this leads to elevated consumption of memory and CPU resources. For sparser point clouds, fewer clusters lead to more distortions, while excessive clusters lead to more voids in reconstructed point clouds. This study proposes a local clustering method to determine a number of clusters closer to the actual number based on GMM (Gaussian Mixture Model) observation distances and density peaks. Experimental results illustrate that the estimated number of clusters is closer to the actual number in four datasets from the KEEL public repository. In point cloud compression and recovery experiments, our proposed approach compresses and recovers the Bunny and Armadillo datasets in the Stanford 3D repository; the experimental results illustrate that our proposed approach improves reconstructed point clouds' geometry and curvature similarity. Furthermore, the geometric similarity increases to 0.9 above in our complete rock fragment surface datasets after selecting a better wavelet basis for each dimension of MEMS-based LiDAR signals. In both experiments, the sparsity of signals was 0.8 and the sampling ratio was 0.4. Finally, a rock outcrop point cloud data experiment is utilized to verify that the proposed approach is applicable for large-scale research objects. All of our experiments illustrate that the proposed adaptive clustered compressive sensing approach can better reconstruct MEMS-based LiDAR point clouds with a lower sampling ratio.

A Multilevel Classification Strategy for the Identification of Discontinuities from 3D Point Clouds of Complicated Rock Surfaces

A Multilevel Classification Strategy for the Identification of Discontinuities from 3D Point Clouds of Complicated Rock Surfaces

A Globally Consistent Merging Method for House Point Clouds Based on Artificially Enhanced Features

In the process of using structured light technology to obtain indoor point clouds, due to the limited field of view of the device, it is necessary to obtain multiple point clouds of different wall surfaces. Therefore, merging the point cloud is necessary to acquire a complete point cloud. However, due to issues such as the sparse geometric features of the wall point clouds and the high similarity of multiple point clouds, the merging effect of point clouds is poor. In this paper, we leverage artificially enhanced features to improve the accuracy of registration in this scenario. Firstly, we design feature markers and present their layout criteria. Then, the feature information of the marker is extracted based on the Color Signature of Histograms of OrienTations (Color-SHOT) descriptor, and coarse registration is realized through the second-order similarity measure matrix. After that, precise registration is achieved using the Iterative Closest Point (ICP) method based on markers and overlapping areas. Finally, the global error of the point cloud registration is optimized by loop error averaging. Our method enables the high-precision reconstruction of integrated home design scenes lacking significant features at a low cost. The accuracy and validity of the method were verified through comparative experiments.

PointSurFace: Discriminative point cloud surface feature extraction for 3D face recognition

Due to the geometric information in the 3D face data, the 3D face recognition methods exhibit better robustness against the physical attacks compared to the 2D recognition methods. In this paper, we propose PointSurFace, a 3D face point cloud recognition method integrated with a specially designed facial surface feature extraction (SurF) module. In PointSurFace, SurF comprehensively combines the coordinate positions, normal vectors, and angle relationships, etc., of each patch in the 3D face point cloud, to explore the explicit local geometric information of 3D faces and generate facial surface features. Subsequently, the surface features are fed into an encoder consisting of four set abstraction (SA) modules and three Inverted Residual MLP (InvResMLP) modules to extract more discriminative 3D facial features, where InvResMLP could suppress the overfitting and reduce information loss in the model training. In addition, PointSurFace utilizes channel fusion operations to integrate position information into the extracted 3D facial features, further enhancing the model performance. Experimental results in 3D facial recognition and verification across multiple datasets demonstrate that PointSurFace achieves the best 3D face recognition performance to date. For example, on Lock3DFace, it achieves the state-of-the-art recognition and verification accuracy of 90.03% and 80.01%, respectively, surpassing the previous methods by 2.02% and 3.19%. The ablation studies evaluate each module of PointSurFace, suggesting that the proposed surface feature extraction module significantly enhances the model performance.

Solutions to enhance the accuracy of UAV’s images for creating extremely large-scale maps in urban area

Abstract Ho Chi Minh City, the largest urban agglomeration in Vietnam, is at the forefront of enabling the government’s efforts and advocating for smart city initiatives in the coming decades. To facilitate this advancement, the implementation of solutions to support the collection, construction, and updating of spatial databases is crucial. These databases serve as the foundational platform for effective urban management practices. Unmanned Aerial Vehicles (UAVs) have witnessed significant growth across various domains, particularly in spatial data collection. However, challenges arise when acquiring and processing UAV images in urban areas. These challenges include orthomosaic distortion in certain regions, insufficient accuracy for creating extremely large-scale maps (e.g., 1:200 or 1:500), and unclear boundaries between adjacent objects. The objective of this paper is to present flight tests conducted at varying altitudes, with different shooting angles, and varying layouts of Ground Control Points (GCPs), both with and without the UAV’s RTK receiver mode. In order to improve the accuracy of UAV’s images, the results suggest that the precision of orthomosaic, Digital Surface Model (DSM) and point clouds in the context of 3D mapping depends on various factors, including: the determination of the optimal flight altitude, optimizing number of GCPs, selecting an appropriate level of image processing, capturing images from varied angles, and using UAVs with RTK or PPK data acquisition modes to mitigate image distortion.

Evaluation of classified ground points from National Agriculture Imagery program photogrammetrically derived point clouds

ABSTRACT Studies have shown that digital surface models and point clouds generated by the United States Department of Agriculture’s National Agriculture Imagery Program (NAIP) can measure basic forest parameters such as canopy height. However, all measured forest parameters from these studies are evaluated using the differences between NAIP digital surface models (DSMs) and available lidar digital terrain models (DTMs). A survey of NAIP point cloud classification and related ground point-generated DTMs has not yet been undertaken. This study applies a Support Vector Machine (SVM) to classifying ground and nonground points from NAIP point clouds for test sites in Wyoming and Arizona, USA. Light detection and ranging (lidar) data from the U.S. Geological Survey 3D Elevation Program (3DEP) are used to validate the classified NAIP ground points and their corresponding DTMs. Comparing height differences between filtered NAIP ground points and 3DEP ground points, the SVM classifier’s results show that the vertical root mean square error value is 1.87 m and 1.69 m for the Wyoming and Arizona sites, respectively. If NAIP point clouds were continuously measured, the resulting availability of medium-resolution DTMs would benefit the application of multitemporal forest health monitoring and DTM generation.

Gap measurement method based on projection lines and convex analysis of 3D points cloud

Abstract Accurate measurement of the gap between the lower surface of the relay and the ground is critical for ensuring the quality of the finished product. Traditional gap measurement methods have some shortcomings, such as low accuracy, poor robustness, and loss of depth clues in obscured areas. In this study, a novel gap measurement method based on computer vision is proposed, which includes a projection line model based on guided filtering and a 3D surface point cloud model based on a three dimensional plane reference.- The relay gap was measured by calculating the projection lines of the upper and lower surfaces of the gap with an error of ±0.016 mm. A 3D point cloud model captures the key features of the underside of the relay through image processing techniques, and combines convex hull and centroid estimation to construct a three-dimensional reference plane for the gap, which could achieve high-precision, real-time measurement of the gap (with an error less than ±0.0087 mm). The experimental measurement results show that the proposed method is better than the SelfConvNet method, which has a high measurement accuracy and strong anti-interference ability, and an accuracy rate of up to 99.5% in factory relay quality inspection experiments.&amp;#xD;

An Induce-on-Boundary Magnetostatic Solver for Grid-Based Ferrofluids

This paper introduces a novel Induce-on-Boundary (IoB) solver designed to address the magnetostatic governing equations of ferrofluids. The IoB solver is based on a single-layer potential and utilizes only the surface point cloud of the object, offering a lightweight, fast, and accurate solution for calculating magnetic fields. Compared to existing methods, it eliminates the need for complex linear system solvers and maintains minimal computational complexities. Moreover, it can be seamlessly integrated into conventional fluid simulators without compromising boundary conditions. Through extensive theoretical analysis and experiments, we validate both the convergence and scalability of the IoB solver, achieving state-of-the-art performance. Additionally, a straightforward coupling approach is proposed and executed to showcase the solver's effectiveness when integrated into a grid-based fluid simulation pipeline, allowing for realistic simulations of representative ferrofluid instabilities.

Measurement method of spherical radius size of differential planetary gear tooth surface based on 3D vision

Abstract In order to accurately measure the spherical radius dimensions of planetary gear tooth surfaces, we propose a non-contact measurement method based on 3D vision technology. Firstly, the point cloud data of the planetary gear captured by the laser 3D profiler is preprocessed, and the tooth surface point cloud is extracted. Next, based on the structural characteristics of the planetary gear tooth surface, a uniform slicing and sampling algorithm is used to extract circular contour points where the tooth surface point cloud intersects with the slicing planes. These points are then projected onto a two-dimensional plane, where an improved RANSAC algorithm is used to precisely fit each circular cross-section. Contour points associated with significant fitting errors are removed based on a set tolerance value. Finally, an improved differential evolution algorithm is used to perform a three-dimensional spherical fitting on the contour points that meet the tolerance criteria, achieving accurate measurement of the spherical radius of the planetary gear tooth surfaces. Experimental results demonstrate that the repeatability error of the proposed method is ±0.02 mm, with a maximum absolute error of 0.026 mm and an average measurement time of 4.76 s. The method exhibits high robustness and measurement accuracy, meeting practical engineering measurement requirements.

HoloGS: Instant Depth-based 3D Gaussian Splatting with Microsoft HoloLens 2

Abstract. In the fields of photogrammetry, computer vision and computer graphics, the task of neural 3D scene reconstruction has led to the exploration of various techniques. Among these, 3D Gaussian Splatting stands out for its explicit representation of scenes using 3D Gaussians, making it appealing for tasks like 3D point cloud extraction and surface reconstruction. Motivated by its potential, we address the domain of 3D scene reconstruction, aiming to leverage the capabilities of the Microsoft HoloLens 2 for instant 3D Gaussian Splatting. We present HoloGS, a novel workflow utilizing HoloLens sensor data, which bypasses the need for pre-processing steps like Structure from Motion by instantly accessing the required input data i.e. the images, camera poses and the point cloud from depth sensing. We provide comprehensive investigations, including the training process and the rendering quality, assessed through the Peak Signal-to-Noise Ratio, and the geometric 3D accuracy of the densified point cloud from Gaussian centers, measured by Chamfer Distance. We evaluate our approach on two self-captured scenes: An outdoor scene of a cultural heritage statue and an indoor scene of a fine-structured plant. Our results show that the HoloLens data, including RGB images, corresponding camera poses, and depth sensing based point clouds to initialize the Gaussians, are suitable as input for 3D Gaussian Splatting.

Research on Lightweight Method of Segment Beam Point Cloud Based on Edge Detection Optimization

In order to reduce the loss of laser point cloud appearance contours by point cloud lightweighting, this paper takes the laser point cloud data of the segment beam of the expressway viaduct as a sample. After comparing the downsampling algorithm from many aspects and angles, the voxel grid method is selected as the basic theory of the research. By combining the characteristics of the normal vector data of the laser point cloud, the top surface point cloud edge data are extracted and the voxel grid method is fused to establish an optimized point cloud lightweighting algorithm. The research in this paper shows that the voxel grid method performs better than the furthest point sampling method and the curvature downsampling method in retaining the top surface data, reducing the calculation time and optimizing the edge contour. Moreover, the average offset of the geometric contour is reduced from 2.235 mm to 0.664 mm by the edge-optimized voxel grid method, which has a higher retention. In summary, the edge-optimized voxel grid method has a better effect than the existing methods in point cloud lightweighting.

SRX Net: A point cloud segmentation framework based on surface representation and X-Net

We propose a novel unified point cloud segmentation framework called SRX Net. This framework is composed of our proposed SurRep Group module and X-Net architecture, reconstructing point cloud segmentation models from two aspects: point cloud surface representation and network architecture. The SurRep Group can model the local surface structure of the point cloud through structures such as the Ellipsoid Surface Representation and Umbrella Surface Representation, providing rich geometric information to the network. The XNet establishes an inverse encoding–decoding path to deepen the modeling of shallow semantics, solving the interference of shallow semantics on deep semantics in prediction in the U-Net architecture. It also introduces a GLFEnhancement module to enhance each point’s perception of global information. As a unified framework, SRX Net can improve the semantic segmentation effects of various existing models. Extensive experimental results show that our model produces more accurate segmentation edges and significantly reduces discrete predicted points. Based on Point Transformer, our SRX-PT Net achieves state-of-the-art performance of 72.1 mIoU on S3DIS, Scannet V2, and WCS3D datasets, respectively.

A subway tunnel image stitching method based on point cloud mapping relationships and high-resolution image

The paper presents an intelligent tunnel 3D laser image acquisition system. The system can quickly acquire the 3D point cloud and tunnel surface image of the whole tunnel section. The paper proposes the subway tunnel image stitching method based on point cloud mapping relationships and high-resolution image. Firstly, the method extracts the homonymous points from the mapped image generated from the 3D point cloud and the camera image based on the maximum information coefficient method. Secondly, the 3D point cloud positions corresponding to the homonymous points are determined. The optimization of camera parameters is achieved based on the gradient descent method. Then, the tunnel point cloud is expanded to generate a gray scale image based on spherical projection. Based on the optimized parameters, the point cloud is projected to the camera image to determine the depth value of each camera image pixel point. Finally, the pixel values of the camera image are projected based on the spherical projection to obtain the stitching result images. This method stitches individual images together to form a more complete tunnel surface image. The larger tunnel surface image helps in identifying tunnel defects.

A Color- and Geometric-Feature-Based Approach for Denoising Three-Dimensional Cultural Relic Point Clouds.

In the acquisition process of 3D cultural relics, it is common to encounter noise. To facilitate the generation of high-quality 3D models, we propose an approach based on graph signal processing that combines color and geometric features to denoise the point cloud. We divide the 3D point cloud into patches based on self-similarity theory and create an appropriate underlying graph with a Markov property. The features of the vertices in the graph are represented using 3D coordinates, normal vectors, and color. We formulate the point cloud denoising problem as a maximum a posteriori (MAP) estimation problem and use a graph Laplacian regularization (GLR) prior to identifying the most probable noise-free point cloud. In the denoising process, we moderately simplify the 3D point to reduce the running time of the denoising algorithm. The experimental results demonstrate that our proposed approach outperforms five competing methods in both subjective and objective assessments. It requires fewer iterations and exhibits strong robustness, effectively removing noise from the surface of cultural relic point clouds while preserving fine-scale 3D features such as texture and ornamentation. This results in more realistic 3D representations of cultural relics.

A Rapid Segmentation Method of Highway Surface Point Cloud Data Based on a Supervoxel and Improved Region Growing Algorithm

Mobile laser scanning (MLS) systems have become an important technology for collecting and measuring road information for highway maintenance and reconstruction services. However, the efficient and accurate extraction of unstructured road surfaces from MLS point cloud data collected on highways is challenging. Specifically, the complex and unstructured characteristics of road surveying point cloud data lead to traditional 3D point cloud segmentation. When traditional 3D point cloud algorithms extract unstructured road surfaces, over-segmentation and under-segmentation often occur, which affects efficiency and accuracy. To solve these problems, this study introduces an enhanced road extraction method that integrates supervoxel and trajectory information into a traditional region growing algorithm. The method involves two main steps: first, a supervoxel data structure is applied to reconstruct the original MLS point cloud data, which diminishes the calculation time of the point cloud feature vector and accelerates the merging speed of a similar region; second, the trajectory information of the vehicle is used to optimize the seed selection strategy of the regio growing algorithm, which improves the accuracy of road surface extraction. Finally, two typical highway section tests (flat road and slope road) were conducted to validate the positioning performance of the proposed algorithm in an MLS point cloud. The results show that, compared with three kinds of traditional road surface segmentation algorithms, our method achieves an average extraction recall and precision of 99.1% and 96.0%, and by calculating the recall and precision, an F1 score of 97.5% can be obtained to evaluate the performance of the proposed method, for both datasets. Additionally, our method exhibits an average road surface extraction time that is 45.0%, 50.3%, and 55.8% faster than those of the other three automated segmentation algorithms.