3D Point Cloud Research Articles

Automated geometric reconstruction and cable force inference for cable-net structures using 3D point clouds

Laser scanning provides an efficient solution to digital twin construction in civil engineering. The complexity and redundancy of large-scale point clouds substantially prolong the labor-intensive model reconstruction process. This paper presents an automated and high-precision geometric reconstruction approach for cable-net structures with a complete workflow from raw points to the extraction of cable shapes and forces. The strategy involves the extraction of coordinates of key cable regions through the characterization of cable geometry and local registration methods, followed by the computation of cable forces and shapes using segmented catenary theory and nonlinear optimization. The approach is validated with a single-curvature cable-net structure, with the accuracy of cable shapes within ±5 mm and errors in cable forces <5%. The method contributes to health monitoring and rapid reconstruction of cable-net structures. The precise geometry and forces will facilitate the creation of the mechanical models reflecting the physical authenticity of structures.

Blind Quality Assessment of Dense 3D Point Clouds with Structure Guided Resampling

Objective quality assessment of three-dimensional (3D) point clouds is essential for the development of immersive multimedia systems in real-world applications. Despite the success of perceptual quality evaluation for 2D images and videos, blind/no-reference metrics are still scarce for 3D point clouds with large-scale irregularly distributed 3D points. Therefore, in this article, we propose an objective point cloud quality index with Structure Guided Resampling (SGR) to automatically evaluate the perceptually visual quality of dense 3D point clouds. The proposed SGR is a general-purpose blind quality assessment method without the assistance of any reference information. Specifically, considering that the human visual system is highly sensitive to structure information, we first exploit the unique normal vectors of point clouds to execute regional pre-processing that consists of keypoint resampling and local region construction. Then, we extract three groups of quality-related features, including (1) geometry density features, (2) color naturalness features, and (3) angular consistency features. Both the cognitive peculiarities of the human brain and naturalness regularity are involved in the designed quality-aware features that can capture the most vital aspects of distorted 3D point clouds. Extensive experiments on several publicly available subjective point cloud quality databases validate that our proposed SGR can compete with state-of-the-art full-reference, reduced-reference, and no-reference quality assessment algorithms.

Executing Ad-Hoc Queries on Large Geospatial Data Sets Without Acceleration Structures

In this case study, we investigate if it is possible to harness the capabilities of modern commodity hardware to perform ad-hoc queries on large raw geospatial data sets. Normally, this requires building an index structure, which is a time-consuming process. We aim to provide means to individual users who receive a new or updated geospatial data set and want to directly start working with it without having to build such an index structure first. To this end, we conduct various experiments on two distinct types of data: 3D building models and point clouds. For the former, we demonstrate that well-known algorithms such as fast string search allow a wide range of queries to be answered in at most a few seconds on data sets with over a million buildings. The usage of progressive indexing additionally improves query run time by more than a factor of two. Regarding point clouds, we achieve similar run times using the popular LAS file format and a query throughput of up to a billion points per second when using a columnar memory layout. The run time of ad-hoc queries is often on par with that of database-driven solutions, sometimes even outperforming them. Considering that ad-hoc queries require no preprocessing, our results show that they are a viable alternative to acceleration structures when working with geospatial data.

Efficient flexible voxel-based two-stage network for 3D object detection in autonomous driving

3D object detection from the LiDAR point cloud plays an important role in autonomous driving. It is difficult to balance inference speed and detection accuracy when performing 3D point cloud object detection due to the large size of point cloud data and its unstructured storage, which makes it difficult to represent its features. To address the challenge, we propose a two-stage point cloud object detector, AFV-RCNN. In stage-1, the attention flexible voxel feature encoding layer is introduced, which utilizes flexible voxels to enhance feature encoding speed and focuses on foreground points to be detected through voxel attention. In stage-2, the multi-level and grid-based multi-scale RoI (Region of Interest) feature fusion module is designed. It directly extracts complete 3D structures from 3D region proposals and focuses on both local and global features through multi-scale partitioning. In the training stage, GHM-C Loss is applied to address the challenges associated with imbalanced target categories and the imbalance between difficult and easy samples in the classification task. We evaluate the model on the public KITTI Dataset and Waymo Open Dataset. The mAP in KITTI for 3D detection is 73.41% and inference on a single GPU reaches 30.0 fps. Compared with other state-of-the-art methods, AFV-RCNN achieves both the inference speed of a single-stage detector and the detection accuracy of a two-stage detector. It ensures higher detection accuracy while efficiently processing the point cloud.

Megaripple stripes in the Qaidam Basin, China: Morphology, grain size and sedimentary characteristics

Megaripple stripes (MRS) are enigmatic aeolian bedform patterns which are composed of crosswind alternating, wind-parallel megaripple corridors (MRCs) and smaller bedform corridors (SBCs). Compared to SBCs, MRCs have taller bedforms, longer wavelengths, and coarser surface sediments. However, their morphology, grain size, and internal structure are poorly understood. Numerous MRS have been identified in the central-southern edge of the Qaidam Basin. Unmanned Aerial Vehicle (UAV) images and 3D cloud point data were used to explore the morphological properties of MRS, and sediment samples were collected for grain size analysis. The sedimentary characteristics were analyzed through sections of MRC, transition zone between MRC and SBC, and SBC. The results show that the downwind and crosswind wavelengths of MRCs, along with the downwind wavelength of SBCs, decrease in the downwind direction. Conversely, the crosswind wavelength of SBCs displays an increasing trend. The crestline of MRS becomes progressively curved along the downwind direction. The average wavelength of MRCs is 8.51 m, with a height of 0.46 m. The average wavelength and height of SBCs are 1.69 m and 0.07 m, respectively. SBCs are smaller and more densely distributed than MRCs. Most values of the Ripple Index (RI) of MRS clustered between 11 and 60. MRCs exhibit a relatively symmetrical cross-sectional profile, while SBCs exhibit comparatively poor symmetry. MRCs are mainly composed of gravel and very fine sand, with accounting for 63.67 % of content in total. In SBCs, the prevalent grain sizes are very coarse sand and fine sand, accounting for 48.42 % of content in total. MRS display poor sorting, presenting a predominantly positive skewness and wide to very wide peaks. The thickness and inclination of laminae in middle layer 1 gradually decrease from MRCs to SBCs. The sedimentary structure of MRS provides insights into the grain migration in downwind and crosswind directions during their formation.

Development of a Precise Tree Structure from LiDAR Point Clouds

Abstract. A precise tree structure that represents the distribution of tree stem, branches, and leaves is crucial for accurately capturing the full representation of a tree. Light Detection and Ranging (LiDAR)-based three-dimensional (3D) point clouds (PCs) capture the geometry of scanned objects including forests stands and individual trees. PCs are irregular, unstructured, often noisy, and contaminated by outliers. Researchers have struggled to develop methods to separate leaves and wood without losing the tree geometry. This paper proposes a solution that employs only the spatial coordinates (x, y, z) of the PC. The new algorithm works as a filtering approach, utilizing multi-scale neighborhood-based geometric features (GFs) e.g., linearity, planarity, and verticality to classify linear (wood) and non-linear (leaf) points. This involves finding potential wood points and coupling them with an octree-based segmentation to develop a tree architecture. The main contributions of this paper are (i) investigating the potential of different GFs to split linear and non-linear points, (ii) introducing a novel method that pointwise classifies leaf and wood points, and (iii) developing a precise 3D tree structure. The performance of the new algorithm has been demonstrated through terrestrial laser scanning PCs. For a Scots pine tree, the new method classifies leaf and wood points with an overall accuracy of 97.9%.

Investigating the Potential of Hyper-Temporal Terrestrial Laser Point Clouds for Monitoring Deciduous Tree Growth

Abstract. Monitoring tree growth processes is relevant for ecological research and understanding the intricate relationship between vegetation and the environment. Time series analyses have revealed a correlation between leaf emergence timing and climate change, with earlier leaf emergence attributed to global warming. While traditional forest inventory methods struggle to quantify growth processes on small scales, terrestrial laser scanning provides a powerful alternative for providing high-resolution 3D information. This study explores the use of high-frequency hyper-temporal terrestrial laser scanning data to quantitatively describe deciduous tree growth, tested on a pedunculate oak (Quercus robur). The research aims to address key questions about detecting leaf growth in hypertemporal terrestrial laser scanning data. Additionally, it explores how 3D tree parameters and point cloud comparisons capture leaf and tree growth throughout the year. Results from M3C2 point cloud analyses indicate that the temporary branch movements correlate with precipitation. Over the year, branch movements were detected to increase with growing distance from the trunk.

High-detail and low-cost underwater inspection of large-scale hydropower dams

Abstract. The article presents a practical method that combines low-cost camera systems with remotely operated vehicles (ROVs) to accomplish a comprehensive but economically feasible underwater survey of large hydropower infrastructures. Typically, inspecting reservoirs entails draining them off to allow for visual inspections, which are time-intensive, pose risks to operators' safety and are associated with generation losses. In this regard, ROVs are a much safer and more efficient alternative to traditional methods. The study was conducted at the Pack reservoir in Austria, where a reference framework was set up using terrestrial laser scanning and checkerboard markings for the above-water components. A ROV equipped with a GoPro camera and lighting system for the underwater recordings has been employed. Via a close-range photogrammetric approach, it was possible to generate 3D point clouds of the submerged infrastructure with a survey-grade accuracy level. Various strategies were explored to perform bundle block adjustment (BBA), among these were strategies where ground control points (GCPs) were used, strategies without the use of GCPs but pre-calibrated initial camera parameters and strategies with a combination of using both GCPs and pre-calibrated camera parameters in the BBA. The deployment of an inspection technique using low-cost sensors that can generate highly detailed three-dimensional models of submerged infrastructure areas is presented and discussed, allowing easy detection and localization for maintenance inspection, all while being cost-effective. The paper strengthens the suggestion of best practices that optimize camera settings, considering the effect of electronic image stabilization, suggesting its avoidance, and using advanced calibration methods.

ESTATE: A Large Dataset of Under-Represented Urban Objects for 3D Point Cloud Classification

Abstract. Cityscapes contain a variety of objects, each with a particular role in urban administration and development. With the rapid growth and implementation of 3D imaging technology, urban areas are increasingly surveyed with high-resolution point clouds. This technical advancement extensively improves our ability to capture and analyse urban environments and their small objects. Deep learning algorithms for point cloud data have shown considerable capacity in 3D object classification but still face problems with generally under-represented objects (such as light poles or chimneys). This paper introduces the ESTATE dataset (https://github.com/3DOM-FBK/ESTATE), which combines available datasets of various sensors, densities, regions, and object types. It includes 13 classes featuring intensity and/or colour attributes. Tests using ESTATE demonstrate that the dataset improves the classification performance of deep learning techniques and could be a game-changer to advance in the 3D classification of urban objects.

HoloGS: Instant Depth-based 3D Gaussian Splatting with Microsoft HoloLens 2

Abstract. In the fields of photogrammetry, computer vision and computer graphics, the task of neural 3D scene reconstruction has led to the exploration of various techniques. Among these, 3D Gaussian Splatting stands out for its explicit representation of scenes using 3D Gaussians, making it appealing for tasks like 3D point cloud extraction and surface reconstruction. Motivated by its potential, we address the domain of 3D scene reconstruction, aiming to leverage the capabilities of the Microsoft HoloLens 2 for instant 3D Gaussian Splatting. We present HoloGS, a novel workflow utilizing HoloLens sensor data, which bypasses the need for pre-processing steps like Structure from Motion by instantly accessing the required input data i.e. the images, camera poses and the point cloud from depth sensing. We provide comprehensive investigations, including the training process and the rendering quality, assessed through the Peak Signal-to-Noise Ratio, and the geometric 3D accuracy of the densified point cloud from Gaussian centers, measured by Chamfer Distance. We evaluate our approach on two self-captured scenes: An outdoor scene of a cultural heritage statue and an indoor scene of a fine-structured plant. Our results show that the HoloLens data, including RGB images, corresponding camera poses, and depth sensing based point clouds to initialize the Gaussians, are suitable as input for 3D Gaussian Splatting.

Extraction of block walls from point clouds measured by Mobile Mapping System

Abstract. To solve the problem of collapsing block walls widely used in Japan, this study proposes a method for extracting block walls using 3D point cloud data measured by the Mobile Mapping System (MMS). Unlike conventional methods, this method identifies block walls based on geometric features without relying on MMS trajectory data or deep learning inference results. In addition, the computational load is low and manual correction can be minimized. In our experiments, we used point cloud data collected in urban areas in Japan and achieved a precision of 0.750, recall of 0.810, and F-measure of 0.779. The results demonstrate the effectiveness of this method for automatic extraction of block walls and rapid assessment of collapse risk and are expected to contribute to safety measures in areas with high seismic risk.

Autonomous robotic drone system for mapping forest interiors

Abstract. During the last decade, the use of drones in forest monitoring and remote sensing has become highly popular. While most of the monitoring tasks take place in high altitudes and open air, in the last few years drones have also gained interest in under-canopy data collection. However, flying under the forest canopy is a complex task since the drone can not use Global Navigation Satellite Systems (GNSS) for positioning and it has to continually avoid obstacles, such as trees, branches, and rocks, on its path. For that reason, drone-based data collection under the forest canopy is still mainly based on manual control by human pilots. Autonomous flying in GNSS-denied obstacle-rich environment has been an actively researched topic in the field of robotics during the last years and various open-sourced methods have been published in the literature. However, most of the research is done purely from the point-of-view of robotics and only a few studies have been published in the boundary of forest sciences and robotics aiming to take steps towards autonomous forest data collection. In this study, a prototype of an autonomous under-canopy drone is developed and implemented utilizing state-of-the-art open-source methods. The prototype is utilizing the EGO-Planner-v2 trajectory planner for autonomous obstacle avoidance and VINS-Fusion for Visual-inertial-odometry based GNSS-free pose estimation. The flying performance of the prototype is evaluated by performing multiple test flights with real hardware in two different boreal forest test plots with medium and difficult densities. Furthermore, the first results of the forest data collecting performance are obtained by post-processing the data collected with a low-cost stereo camera during one test flight to a 3D point cloud and by performing diameter breast at height (DBH) estimation. In the medium-density forest, all seven test flights were successful, but in the difficult test forest, one of eight test flights failed. The RMSE of the DBH estimation was 3.86 cm (12.98 %).

2D Representations of 3D Point Clouds Via the Stereographic Projection with Encryption Applications

Using the topological equivalence between the Riemann sphere S\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\mathbb {S}$$\\end{document} and the extended complex plane C¯=C∪{∞}\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\overline{\\mathbb {C}} = \\mathbb {C} \\cup \\{\\infty \\}$$\\end{document}, where C\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\mathbb {C}$$\\end{document} is the field of complex numbers, we establish 2D-bijective representations of 3D point clouds. Points of 3D point clouds are mapped into the Riemann sphere S\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\mathbb {S}$$\\end{document}, and a stereographic projection is implemented to map the points into the complex plane C\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\mathbb {C}$$\\end{document}. The way the 3D objects are mapped into S\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\mathbb {S}$$\\end{document} may be varied for various applications. To prove the accuracy and efficiency of the proposed 2D representation of 3D objects, we apply this correspondence to 3D point cloud encryption. We utilize chaotic permutations, chaotic circuits, and Latin cubes in addition to the stereographic projection representation to construct our scheme. The permutation steps using chaotic maps and Latin cubes are carried out on the object data points in both S\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\mathbb {S}$$\\end{document} and C¯\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\overline{\\mathbb {C}}$$\\end{document}, while the chaotic circuits are applied to 2D projections of the 3D objects. To the best of our knowledge, no earlier work employed stereographic projections for 3D object encryption. Experimental simulations of this method show high encryption strength and strong confusion and diffusion properties based on quantitative and statistical measures.

Development and Evaluation of a Two-Staged 3D Keypoint Based Workflow for the Co-Registration of Unstructured Multi-Temporal and Multi-Modal 3D Point Clouds

Abstract. Robust and automated point cloud registration methods are required in many geoscience applications using multi-temporal and multi-modal 3D point clouds. Therefore, a 3D keypoint-based coarse registration workflow has been implemented, utilizing the ISS keypoint detector and 3DSmoothNet descriptor. This paper contributes to keypoint-based registration research through variations of the standard workflow proposed in the literature, applying a two-staged strategy of global and local keypoint matching as well as prototypical keypoint projection and fine registration based on ICP. Further, by testing the utilized detector and descriptor on unstructured, multi-temporal and multi-source point clouds with variations in point cloud density, generalization ability is tested outside benchmark data. Therefore, data of the Bøverbreen glacier in Jotunheimen, Norway has been acquired in 2022 and 2023, deploying UAV-based image matching and terrestrial laser scanning. The results show good performance of the implemented robust matching algorithm PROSAC, requiring fewer iterations than the well-known RANSAC approach, but solving the rigid body transformation with TEASER++ is faster and more robust to outliers without demanding pre-knowledge of the data. Further, the results identify the keypoint detection as most limiting factor in speed and accuracy. Summarizing, keypoint-based coarse registration on low density point clouds, applying a global and local matching strategy and transformation estimation using TEASER++ is recommended. Keypoint projection shows potential, increasing number and precision in low density clouds, but has to be more robust. Further research needs to be carried out, focusing on identifying a fast and robust keypoint detector.

Structural Health Monitoring of Bridges with Personal Laser Scanning: Segment-based Analysis of systematic Point Cloud Deformations

Abstract. Bridge structures can be surveyed using a number of different methods. Established are image-based methods using structure from motion by an unmanned aerial vehicle (UAV), terrestrial laser scanning (TLS), or a combination of both methods. Beyond static terrestrial laser scanning, buildings can also be efficiently surveyed using personal laser scanner (PLS) systems. The advantage here is the greater flexibility and increased speed compared to the static method. On the other hand, the accuracy may be more critical, and the resulting point cloud will be more sensitive to systematic global or local deformations under unfavorable measurement conditions. For example, temporary influences can lead to local mapping errors. These include influences such as uneven measurement system motion or non-static, feature-sparse environments. This study investigates the acquisition of 3D point clouds representing the outer shell of a concrete bridge using a PLS system. We demonstrate a method for detecting possible deformations in PLS point clouds using the example of a bridge structure. For this purpose, the reference (TLS) and the PLS point clouds are segmented into individual clusters and a segment-based ICP fine registration is performed. Different RMSE values for the upper road section (0.061 m) and for the pillar segments (0.021 m) as well as different transformation parameters indicate slight displacements in the PLS point cloud. The analysis of the cloud-to-cloud distances showed that there were slight deformations in the Z direction in the area of the road surface. In the lateral direction, no significant residual deviations were found in the area of the bridge pillars.

Using Passive Multi-Modal Sensor Data for Thermal Simulation of Urban Surfaces

Abstract. This paper showcases an integrated workflow hinged on passive airborne multi-modal sensor data for the simulation of the thermal behavior of built-up areas with a focus on urban heat islands. The geometry of the underlying parametrized model, or digital twin, is derived from high-resolution nadir and oblique RGB, near-infrared and thermal infrared imagery. The captured bitmaps get photogrammetrically processed into comprehensive surface models, terrain, dense 3D point clouds and true-ortho mosaics. Building geometries are reconstructed from the projected point sets with procedures presupposing outlining, analysis of roof and fac¸ade details, triangulation, and texturing mapping. For thermal simulation, the composition of the ground is determined using supervised machine learning based on a modified multi-modal DeepLab v3+ architecture. Vegetation is retrieved as individual trees and larger tree regions to be added to the meshed terrain. Building materials are assigned from the available visual, infrared and surface planarity information as well as publicly available references. With actual weather data, surface temperatures can be calculated for any period of time by evaluating conductive, convective, radiative and emissive energy fluxes for triangular layers congruent to the faces of the modeled scene. Results on a sample dataset of the Moabit district in Berlin, Germany, showed the ability of the simulator to output surface temperatures of relatively large datasets efficiently. Compared to the thermal infrared images, several insufficiencies in terms of data and model caused occasional deviations between measured and simulated temperatures. For some of these shortcomings, improvement suggestions within future work are presented.

Measurement of outer diameter parameters of manual grinding cable joints based on 3D point cloud processing

For the problems associated with low measurement accuracy caused by surface shrinkage, uneven grinding, and irregular shapes in manual grinding cable joints, as well as the limitations of existing machine vision measurement techniques for the outer diameter parameters, this paper proposes a non-contact outer diameter parameters measurement method of manual grinding cable joints based on three-dimensional (3D) point cloud processing. Firstly, statistical filtering is used to remove discrete noise, and coordinate transformation preprocessing is conducted to ordered the point cloud. Secondly, a bidirectional point cloud slicing method is proposed based on the cable joint structure and point cloud distribution to preserve the original features for effective measurement. Subsequently, the normalized mean curvature is utilized as the weight of point cloud coordinate averaging to construct key points for surface slices, which reduces the amount of data in the sliced point cloud and improves the matching efficiency. Finally, the key points are matched according to the position of the surface slices, and the distance of point pairs is calculated to complete the measurement of the outer diameter parameters. Experiments and error analysis are conducted on multiple manual grinding 110 kV and 500 kV real cable joints. The absolute error in the measurement of the outer diameter is less than 0.2 mm, with a relative error within 1 %. The absolute error of the XY-axis outer diameter deviation is less than 0.1 mm. The experiments demonstrate that the proposed method exhibits superior performance in cable joint outer diameter measurement.

Self-supervised single-view 3D point cloud reconstruction through GAN inversion

Self-supervised single-view 3D point cloud reconstruction through GAN inversion

Test Input Prioritization for 3D Point Clouds

3D point cloud applications have become increasingly prevalent in diverse domains, showcasing their efficacy in various software systems. However, testing such applications presents unique challenges due to the high-dimensional nature of 3D point cloud data and the vast number of possible test cases. Test input prioritization has emerged as a promising approach to enhance testing efficiency by prioritizing potentially misclassified test cases during the early stages of the testing process. Consequently, this enables the early labeling of critical inputs, leading to a reduction in the overall labeling cost. However, applying existing prioritization methods to 3D point cloud data is constrained by several factors: (1) inadequate consideration of crucial spatial information, and (2) susceptibility to noises inherent in 3D point cloud data. In this article, we propose PCPrior , the first test prioritization approach specifically designed for 3D point cloud test cases. The fundamental concept behind PCPrior is that test inputs closer to the decision boundary of the model are more likely to be predicted incorrectly. To capture the spatial relationship between a point cloud test and the decision boundary, we propose transforming each test (a point cloud) into a low-dimensional feature vector, toward indirectly revealing the underlying proximity between a test and the decision boundary. To achieve this, we carefully design a group of feature generation strategies, and for each test input, we generate four distinct types of features, namely spatial features, mutation features, prediction features, and uncertainty features. Through a concatenation of the four feature types, PCPrior assembles a final feature vector for each test. Subsequently, a ranking model is employed to estimate the probability of misclassification for each test based on its feature vector. Finally, PCPrior ranks all tests based on their misclassification probabilities. We conducted an extensive study based on 165 subjects to evaluate the performance of PCPrior, encompassing both natural and noisy datasets. The results demonstrate that PCPrior outperforms all of the compared test prioritization approaches, with an average improvement of 10.99% to 66.94% on natural datasets and 16.62% to 53% on noisy datasets.

Construction of Three-Dimensional Semantic Maps of Unstructured Lawn Scenes Based on Deep Learning

Traditional automatic gardening pruning robots generally employ electronic fences for the delineation of working boundaries. In order to quickly determine the working area of a robot, we combined an improved DeepLabv3+ semantic segmentation model with a simultaneous localization and mapping (SLAM) system to construct a three-dimensional (3D) semantic map. To reduce the computational cost of its future deployment in resource-constrained mobile robots, we replaced the backbone network of DeepLabv3+, ResNet50, with MobileNetV2 to decrease the number of network parameters and improve recognition speed. In addition, we introduced an efficient channel attention network attention mechanism to enhance the accuracy of the neural network, forming an improved Multiclass MobileNetV2 ECA DeepLabv3+ (MM-ED) network model. Through the integration of this model with the SLAM system, the entire framework was able to generate a 3D semantic point cloud map of a lawn working area and convert it into octree and occupancy grid maps, providing technical support for future autonomous robot operation and navigation. We created a lawn dataset containing 7500 images, using our own annotated images as ground truth. This dataset was employed for experimental purposes. Experimental results showed that the proposed MM-ED network model achieved 91.07% and 94.71% for MIoU and MPA metrics, respectively. Using a GTX 3060 Laptop GPU, the frames per second rate reached 27.69, demonstrating superior recognition performance compared to similar semantic segmentation architectures and better adaptation to SLAM systems.