3D Point Research Articles

Point‐AGM : Attention Guided Masked Auto‐Encoder for Joint Self‐supervised Learning on Point Clouds

AbstractMasked point modeling (MPM) has gained considerable attention in self‐supervised learning for 3D point clouds. While existing self‐supervised methods have progressed in learning from point clouds, we aim to address their limitation of capturing high‐level semantics through our novel attention‐guided masking framework, Point‐AGM. Our approach introduces an attention‐guided masking mechanism that selectively masks low‐attended regions, enabling the model to concentrate on reconstructing more critical areas and addressing the limitations of random and block masking strategies. Furthermore, we exploit the inherent advantages of the teacher‐student network to enable cross‐view contrastive learning on augmented dual‐view point clouds, enforcing consistency between complete and partially masked views of the same 3D shape in the feature space. This unified framework leverages the complementary strengths of masked point modeling, attention‐guided masking, and contrastive learning for robust representation learning. Extensive experiments have shown the effectiveness of our approach and its well‐transferable performance across various downstream tasks. Specifically, our model achieves an accuracy of 94.12% on ModelNet40 and 87.16% on the PB‐T50‐RS setting of ScanObjectNN, outperforming other self‐supervised learning methods.

Stereo and LiDAR Loosely Coupled SLAM Constrained Ground Detection.

In many robotic applications, creating a map is crucial, and 3D maps provide a method for estimating the positions of other objects or obstacles. Most of the previous research processes 3D point clouds through projection-based or voxel-based models, but both approaches have certain limitations. This paper proposes a hybrid localization and mapping method using stereo vision and LiDAR. Unlike the traditional single-sensor systems, we construct a pose optimization model by matching ground information between LiDAR maps and visual images. We use stereo vision to extract ground information and fuse it with LiDAR tensor voting data to establish coplanarity constraints. Pose optimization is achieved through a graph-based optimization algorithm and a local window optimization method. The proposed method is evaluated using the KITTI dataset and compared against the ORB-SLAM3, F-LOAM, LOAM, and LeGO-LOAM methods. Additionally, we generate 3D point cloud maps for the corresponding sequences and high-definition point cloud maps of the streets in sequence 00. The experimental results demonstrate significant improvements in trajectory accuracy and robustness, enabling the construction of clear, dense 3D maps.

Adversarial Unsupervised Domain Adaptation for 3D Semantic Segmentation with 2D Image Fusion of Dense Depth

AbstractUnsupervised domain adaptation (UDA) is increasingly used for 3D point cloud semantic segmentation tasks due to its ability to address the issue of missing labels for new domains. However, most existing unsupervised domain adaptation methods focus only on uni‐modal data and are rarely applied to multi‐modal data. Therefore, we propose a cross‐modal UDA on multi‐modal datasets that contain 3D point clouds and 2D images for 3D Semantic Segmentation. Specifically, we first propose a Dual discriminator‐based Domain Adaptation (Dd‐bDA) module to enhance the adaptability of different domains. Second, given that the robustness of depth information to domain shifts can provide more details for semantic segmentation, we further employ a Dense depth Feature Fusion (DdFF) module to extract image features with rich depth cues. We evaluate our model in four unsupervised domain adaptation scenarios, i.e., dataset‐to‐dataset (A2D2 → SemanticKITTI), Day‐to‐Night, country‐to‐country (USA → Singapore), and synthetic‐to‐real (VirtualKITTI → SemanticKITTI). In all settings, the experimental results achieve significant improvements and surpass state‐of‐the‐art models.

Dual frequency composite pattern temporal phase unwrapping for 3D surface measurement

Phase shifting profilometry (PSP) is widely used in three-dimensional (3D) optical metrology applications such as mechanical engineering, industrial monitoring, computer vision, and biomedicine because of its high measurement accuracy. In PSP, multi-frequency temporal phase unwrapping (TPU) plays a dominant role due to its high-accuracy measurement of surfaces with discontinuities and isolated objects. However, it requires a large number of fringe patterns. To reduce the number of required patterns, a new dual-frequency composite-pattern TPU method is developed, which needs only three patterns. The new method combines two fringe patterns of different frequencies into three composite fringe patterns, reconstructs rough 3D points by demodulating the low-frequency fringe pattern using geometric constraints, and reprojects it to obtain a rough low-frequency absolute phase, which is then refined to correct edge errors. Finally, the high-frequency wrapped phase is unwrapped under the guidance of the refined low-frequency phase, thereby the accurate 3D points are reconstructed based on stereo-vision technology. Experimental results demonstrated that the proposed method can achieve 3D surface measurement with only three images, which greatly improves the measurement efficiency. The proposed method has great application potential for the rapid 3D measurement of discontinuous surfaces and isolated objects.

TreeCompR: Tree competition indices for inventory data and 3D point clouds

Abstract In times of more frequent global change‐type droughts and associated tree mortality events, competition release is one silvicultural measure discussed to have an impact on the resilience of managed forest stands. Understanding how trees compete with each other is therefore crucial, but different measurement options and competition indices (CI) leave users with a difficult choice, as no single competition index has proven universally superior. To help users with the choice and computation of appropriate indices, we present the open‐source TreeCompR package, which handles 3D point clouds and classical forest inventory data, enabling the calculation of both innovative point cloud‐based indices and traditional distance‐dependent indices. It serves as a centralized platform for exploring and comparing different CIs, allowing users to test and select the most suitable CI for their specific research questions within a common interface. To evaluate the package, we used TreeCompR to quantify the competition situation of 307 European beech trees from 13 sites in Central Europe. Based on this dataset, we discuss the interpretation, comparability and sensitivity of the different indices to their parameterization and identify possible sources of uncertainty and ways to minimize them. The compatibility of TreeCompR with different data formats and different data collection methods makes it accessible and useful for a wide range of users, specifically ecologists and foresters. Due to the flexibility in the choice of input formats as well as the emphasis on tidy, well‐structured output, our package can easily be integrated into existing data‐analysis workflows both for 3D point cloud and classical forest inventory data.

CPH-Fmnet: An Optimized Deep Learning Model for Multi-View Stereo and Parameter Extraction in Complex Forest Scenes

The three-dimensional reconstruction of forests is crucial in remote sensing technology, ecological monitoring, and forestry management, as it yields precise forest structure and tree parameters, providing essential data support for forest resource management, evaluation, and sustainable development. Nevertheless, forest 3D reconstruction now encounters obstacles including higher equipment costs, reduced data collection efficiency, and complex data processing. This work introduces a unique deep learning model, CPH-Fmnet, designed to enhance the accuracy and efficiency of 3D reconstruction in intricate forest environments. CPH-Fmnet enhances the FPN Encoder-Decoder Architecture by meticulously incorporating the Channel Attention Mechanism (CA), Path Aggregation Module (PA), and High-Level Feature Selection Module (HFS), alongside the integration of the pre-trained Vision Transformer (ViT), thereby significantly improving the model’s global feature extraction and local detail reconstruction abilities. We selected three representative sample plots in Haidian District, Beijing, China, as the study area and took forest stand sequence photos with an iPhone for the research. Comparative experiments with the conventional SfM + MVS and MVSFormer models, along with comprehensive parameter extraction and ablation studies, substantiated the enhanced efficacy of the proposed CPH-Fmnet model in addressing difficult circumstances such as intricate occlusions, poorly textured areas, and variations in lighting. The test results show that the model does better on a number of evaluation criteria. It has an RMSE of 1.353, an MAE of only 5.1%, an r value of 1.190, and a forest reconstruction rate of 100%, all of which are better than current methods. Furthermore, the model produced a more compact and precise 3D point cloud while accurately determining the properties of the forest trees. The findings indicate that CPH-Fmnet offers an innovative approach for forest resource management and ecological monitoring, characterized by cheap cost, high accuracy, and high efficiency.

Real-Time Semantic Segmentation of 3D LiDAR Point Clouds for Aircraft Engine Detection in Autonomous Jetbridge Operations

This paper presents a study on aircraft engine identification using real-time 3D LiDAR point cloud segmentation technology, a key element for the development of automated docking systems in airport boarding facilities, known as jetbridges. To achieve this, 3D LiDAR sensors utilizing a spinning method were employed to gather surrounding environmental 3D point cloud data. The raw 3D environmental data were then filtered using the 3D RANSAC technique, excluding ground data and irrelevant apron areas. Segmentation was subsequently conducted based on the filtered data, focusing on aircraft sections. For the segmented aircraft engine parts, the centroid of the grouped data was computed to determine the 3D position of the aircraft engine. Additionally, PointNet was applied to identify aircraft engines from the segmented data. Dynamic tests were conducted in various weather and environmental conditions, evaluating the detection performance across different jetbridge movement speeds and object-to-object distances. The study achieved a mean intersection over union (mIoU) of 81.25% in detecting aircraft engines, despite experiencing challenging conditions such as low-frequency vibrations and changes in the field of view during jetbridge maneuvers. This research provides a strong foundation for enhancing the robustness of jetbridge autonomous docking systems by reducing the sensor noise and distortion in real-time applications. Our future research will focus on optimizing sensor configurations, especially in environments where sea fog, snow, and rain are frequent, by combining RGB image data with 3D LiDAR information. The ultimate goal is to further improve the system’s reliability and efficiency, not only in jetbridge operations but also in broader autonomous vehicle and robotics applications, where precision and reliability are critical. The methodologies and findings of this study hold the potential to significantly advance the development of autonomous technologies across various industrial sectors.

Mapping predicted biomass in cereal rye using 3D imaging and geostatistics

Abstract Cover crops are becoming an increasingly important tool for weed suppression. Biomass production in cover crops is one of the most important predictors of weed suppressive ability. A significant challenge for growers is that cover crop growth can be patchy within fields, making biomass estimation difficult. This study tested ground-based structure-from-motion (SfM) for estimating and mapping cereal rye (Secale cereale L.) biomass. SfM generated 3D point clouds from red, green, and blue (RGB) videos collected by a handheld GoPro camera over five fields in North Carolina during the 2022 to 2023 winter season. A model for predicting biomass was generated by relating measured biomass at termination using a density–height index (DH) from point cloud pixel density multiplied by crop height. Overall biomass ranged from 320 to 9,200 kg ha−1, and crop height ranged from 10 to 120 cm. Measured biomass at termination was linearly related to DH (r2 = 0.813) through levels of 9,000 kg ha−1. Based on independent data validation, predicted biomass and measured biomass were linearly related (r2 = 0.713). In the field maps generated by kriging, measured biomass data were autocorrelated at a range of 5.4 to 42.2 m, and predicted biomass data were autocorrelated at a range of 3.4 to 12.0 m. However, the spatial arrangement of high- and low-performing areas was similar for predicted and measured biomass, particularly in fields with greatest patchiness and spatial correlation in biomass values. This study provides proof-of-concept that ground-based SfM can potentially be used to nondestructively estimate and map cover crop biomass production and identify low-performing areas at higher risk for weed pressure and escapes.

Point cloud segmentation method based on an image mask and its application verification

Abstract Accurately perceiving three-dimensional (3D) environments or objects is crucial for the advancement of artificial intelligence (AI) interaction technologies. Currently, various types of sensors are employed to obtain point cloud data for 3D object detection or segmentation tasks. While this multi-sensor approach provides more precise 3D data than monocular or stereo cameras, it is also more expensive. The advent of RGB-D cameras, which provide both RGB images and depth information, addresses this issue.
In this study, we propose a point cloud segmentation method based on image masks. By using an RGB-D camera to capture color and depth images, we generate image masks through object recognition and segmentation. Given the mapping relationship between RGB image pixels and point clouds, these image masks can be further used to extract the point cloud data of the target objects. The experimental results revealed that the average accuracy of target segmentation was 84.78%, which was close to that of PointNet++. Compared with three traditional segmentation algorithms, the accuracy was improved by nearly 23.97%. The running time of our algorithm is reduced by 95.76% compared to the PointNet++ algorithm, which has the longest running time; and by 15.65% compared to the LCCP algorithm, which has the shortest running time among traditional methods. Compared with PointNet++, the segmentation accuracy was improved. This method addressed the issues of low robustness and excessive reliance on manual feature extraction in traditional point cloud segmentation methods, providing valuable support and reference for the accurate segmentation of 3D point clouds.

Unsupervised Clustering-based 3D Static Scene Construction Using LiDAR Channel and Azimuth Angle

Abstract. Cameras are typically used at road intersections to collect data to perform object detection but struggle in low-light and harsh weather. On the other hand, Light Detection and Ranging (LiDAR) is used as a key technology in 3D vision systems. It gives the 3D point cloud, which includes accurate depth information, but its resolution is cost-dependent, with higher resolutions being more expensive. Deep learning-based method requires large, labelled dataset which increases the cost, time and accuracy depending on the model trained on labelled dataset. To perform object detection in point cloud data is a challenging task due to the incomplete representations, data sparsity and unavailability of training data. To overcome this by using an unsupervised approach, it is important to identify the static scene and then detect the moving object. This work incorporated a novel approach to construct static scene using azimuth angle and laser channel information using an unsupervised clustering approach. This work incorporates two modules I) data collection using VLP-16 LiDAR, and II) static scene construction using DBSCAN (density-based spatial clustering and noise) clustering-based approach. Data is collected at a 4-legged intersection and pre-processed to extract aggregated distances corresponding to the unique pair of azimuth angle and laser channel. DBSCAN is used to perform clustering on the aggregated distances, based on the highest silhouette score and lowest intra distance between points in cluster, static points are identified, and static scene constructed. The qualitative evaluation of method demonstrates that the algorithm effectively and accurately filters out background points.

High-precision 3D Modeling of Construction Waste Pile Bodies by Integrating Multi-source Data

Abstract. The instability of construction waste pile bodies, as an increasingly concerning disaster, poses significant risks to the safety of people's lives and property. Currently, there is limited research on high-precision 3D modeling techniques for construction waste pile bodies, which significantly hinders the accuracy and reliability of early detection and risk assessment of pile body instability. Therefore, constructing high-precision 3D models of construction waste pile bodies using multi-source data plays a crucial role in improving the accuracy and timeliness of early warning systems for pile body instability, offering significant theoretical research and practical application value. Building 3D models of construction waste pile bodies solely based on unmanned aerial vehicle (UAV) oblique photography data faces challenges, such as various degrees of data voids and insufficient model completeness, and few methods currently address the construction of 3D models of construction waste pile bodies through the fusion of multi-source data. This paper attempts to use the Iterative Closest Point (ICP) algorithm, combining SLAM laser point cloud data with UAV oblique photography data to fill data voids, and utilizes the fused point cloud to reconstruct the triangulation network, achieving the transformation from 3D point cloud models to 3D surface models. On the basis of texture mapping, a high-precision 3D model of the construction waste pile body is constructed. The research results show that the 3D model of the construction waste pile body, integrated with multi-source data, has a planar error of 0.0187m and an elevation error of 0.0368m, meeting the corresponding model accuracy requirements. It can more realistically restore the fine 3D features of the construction waste pile body, effectively compensating for the shortcomings of single-source data in 3D modeling of construction waste pile bodies, providing a new method for the 3D model reconstruction of construction waste pile bodies, and offering effective data support for construction waste research.

Assessing Photo-Voltaic Potential in Urban Environments: A Comparative Study between Aerial Photogrammetry and LiDAR Technologies

Abstract. Urban development and population growth have significantly increased energy demands, predominantly met by non-renewable resources, negatively impacting nature and climate. Consequently, there has been a global shift towards renewable energy sources, with solar energy being widely adopted. Photovoltaic (PV) potential measures the usable electricity from solar energy using PV technology. With conventional methods, estimating PV potential involves converting 3D point cloud data to 2.5D elevation models, which can affect the accuracy of the estimation. A research gap exists in using adequate 3D point cloud datasets for PV potential estimation considering roof surface area and the surface normal vectors. This study compares aerial photogrammetry and aerial LiDAR point clouds for PV potential estimation against the DSM-based approach. Accurate PV potential estimation must consider solar incidence, roof area, azimuth, tilt angles, and PV efficiency. Traditional 2.5D methods often overlook crucial azimuth and tilt data, limiting the accuracy of the PV estimation. Converting 3D data to 2.5D may result in information loss, while 3D analyses offer higher accuracy. To investigate the mentioned gaps, this research aims to evaluate the capabilities of photogrammetry and LiDAR for urban PV potential estimation, highlighting their feasibility and accuracy over 2.5D methods.

Low-Cost Thermal Point Clouds of Indoor Environments

Abstract. The integration of low-cost thermal sensors with Apple Smart Devices supports the generation of 3D point clouds that include temperature indicators. This generates a new perspective for the study of buildings, allowing for fast and reliable examination of physical building structures. To the best of our knowledge, this study is the first to demonstrate the use of affordable sensors for 3D thermal point cloud generation. The study involved capturing data from the LiDAR and thermal sensors, followed by an extrinsic calibration process to align the datasets. Subsequently, the point cloud was segmented based on different acquisition poses of the device and finally, the thermal data was projected onto the 3D model, integrating temperature information with spatial coordinates. Our results demonstrate the effectiveness of the approach for three-dimensional point cloud generation in indoor environments, highlighting significant thermal variations and enabling thermal mapping of building structures. Furthermore, our findings underscore the feasibility of employing low-cost sensors for generating detailed thermal models, opening possibilities for widespread adoption in various building analysis applications. This approach provides a comprehensive and cost-effective solution for building monitoring, democratizing access to advanced evaluation tools.

A Specialized Pipeline for Efficient and Reliable 3D Semantic Model Reconstruction of Buildings from Indoor Point Clouds.

Recent advances in laser scanning systems have enabled the acquisition of 3D point cloud representations of scenes, revolutionizing the fields of Architecture, Engineering, and Construction (AEC). This paper presents a novel pipeline for the automatic generation of 3D semantic models of multi-level buildings from indoor point clouds. The architectural components are extracted hierarchically. After segmenting the point clouds into potential building floors, a wall detection process is performed on each floor segment. Then, room, ground, and ceiling extraction are conducted using the walls 2D constellation obtained from the projection of the walls onto the ground plan. The identification of the openings in the walls is performed using a deep learning-based classifier that separates doors and windows from non-consistent holes. Based on the geometric and semantic information from previously detected elements, the final model is generated in IFC format. The effectiveness and reliability of the proposed pipeline are demonstrated through extensive experiments and visual inspections. The results reveal high precision and recall values in the extraction of architectural elements, ensuring the fidelity of the generated models. In addition, the pipeline's efficiency and accuracy offer valuable contributions to future advancements in point cloud processing.

The Performance of the Optical Flow Field based Dense Image Matching for UAV Imagery

Abstract. With the rapid development of sensing platforms, unmanned aerial vehicle (UAV)-based mapping has become increasingly popular because of its economic efficiency and flexibility, especially for providing 3D information to support urban growth monitoring and change detection to meet sustainable development goals (SDGs). This paper presents an improved optical flow field-based dense matching algorithm (OFFDM) for low-altitude UAV images based on the Ph.D. thesis of Yuan (Yuan, 2018). First, high-precision seed points were used to compute the optical flow field within stereo pairs, effectively minimizing redundant calculations during the fine-matching phase. Second, a fine-matching approach, integrating multiple constraints, was applied to refine the coarse matching results based on the optical flow field. Extensive dense matching experiments on UAV low-altitude aerial imagery assessed the performance of OFFDIM across four dimensions: 3D point cloud visualization, matching success rate, precision, and reliability. Extensive experiments on low-altitude UAV imagery, characterized by a resolution of 7cm per pixel over a 10,608×8,608 pixel dimension and a 60% forward overlap, evaluate the OFFDM's efficacy. The quantitative evaluation revealed that the proposed method achieved an accuracy of ±0.7 pixels in image coordinates and ±20 cm on the ground, with a matching success rate exceeding 97%. The processing time was approximately 272 seconds for handling one single stereo pair. When compared to the widely adopted PMVS algorithm, known for its effectiveness in dense matching for UAV images, the proposed method demonstrated higher completeness and improved matching efficiency by more than five times. These results demonstrated that the proposed approach is more suitable for dense matching on UAV imagery-based high-precision 3D spatial data extraction, supporting global mapping tasks more effectively.

DSM Reconstruction from Uncalibrated Multi-View Satellite Stereo Images by RPC Estimation and Integration

In this paper, we propose a 3D Digital Surface Model (DSM) reconstruction method from uncalibrated Multi-view Satellite Stereo (MVSS) images, where Rational Polynomial Coefficient (RPC) sensor parameters are not available. While recent investigations have introduced several techniques to reconstruct high-precision and high-density DSMs from MVSS images, they inherently depend on the use of geo-corrected RPC sensor parameters. However, RPC parameters from satellite sensors are subject to being erroneous due to inaccurate sensor data. In addition, due to the increasing data availability from the internet, uncalibrated satellite images can be easily obtained without RPC parameters. This study proposes a novel method to reconstruct a 3D DSM from uncalibrated MVSS images by estimating and integrating RPC parameters. To do this, we first employ a structure from motion (SfM) and 3D homography-based geo-referencing method to reconstruct an initial DSM. Second, we sample 3D points from the initial DSM as references and reproject them to the 2D image space to determine 3D–2D correspondences. Using the correspondences, we directly calculate all RPC parameters. To overcome the memory shortage problem while running the large size of satellite images, we also propose an RPC integration method. Image space is partitioned to multiple tiles, and RPC estimation is performed independently in each tile. Then, all tiles’ RPCs are integrated into the final RPC to represent the geometry of the whole image space. Finally, the integrated RPC is used to run a true MVSS pipeline to obtain the 3D DSM. The experimental results show that the proposed method can achieve 1.455 m Mean Absolute Error (MAE) in the height map reconstruction from multi-view satellite benchmark datasets. We also show that the proposed method can be used to reconstruct a geo-referenced 3D DSM from uncalibrated and freely available Google Earth imagery.

Updating assembly parameters for spacecraft assembly process state changes: based on data fusion method

Thermal protection systems (TPSs) are important components of reusable spacecraft, and their assembly quality has a crucial impact on flight safety. Owing to the complex assembly process and variable states of spacecraft thermal protection systems, assembly parameters may vary under different assembly states. Therefore, to obtain assembly parameters accurately and efficiently under different assembly states, in this study, 3D point cloud data and fiber optic sensor data were fused to develop an assembly parameter update method for assembly process state changes. Firstly, based on the measured data of thermal protection components and load-bearing structure, the gap, flush and matching parameters solution model are proposed. Secondly, to address the deformation problem of the load-bearing structure caused by changes in assembly status, a fusion method based on laser scanning and sensor detection was devised to achieve deformation prediction of the assembly structure during the assembly process. Thirdly, based on the assembly parameter solution model and point cloud prediction model, a constraint-based assembly parameter optimisation model was established, and an improved quantum particle swarm optimisation (LQPSO) algorithm was employed to achieve assembly parameter updates oriented toward changes in assembly status. Finally, an experimental system for array-based thermal protection structure simulation was established to validate the proposed method. The results show that the proposed parameter update method can achieve ideal results for different assembly state simulation components.

Weakly supervised 3D point cloud semantic segmentation for architectural heritage using teacher-guided consistency and contrast learning

Point cloud semantic segmentation is significant for managing and protecting architectural heritage. Currently, fully supervised methods require a large amount of annotated data, while weakly supervised methods are difficult to transfer directly to architectural heritage. This paper proposes an end-to-end teacher-guided consistency and contrastive learning weakly supervised (TCCWS) framework for architectural heritage point cloud semantic segmentation, which can fully utilize limited labeled points to train network. Specifically, a teacher-student framework is designed to generate pseudo labels and a pseudo label dividing module is proposed to distinguish reliable and ambiguous point sets. Based on it, a consistency and contrastive learning strategy is designed to fully utilize supervision signals to learn the features of point clouds. The framework is tested on the ArCH dataset and self-collected point cloud, which demonstrates that the proposed method can achieve effective semantic segmentation of architectural heritage using only 0.1 % of annotated points.

Multi-view high-dynamic-range 3D reconstruction and point cloud quality evaluation based on dual-frame difference images

A high dynamic range can easily lead to image saturation, making it a challenge for structured light 3D reconstruction. The article proposes a multi-view 3D topography measurement system, which consists of dual projectors, a single camera, and a high-precision rotary platform. The system utilizes single-frame images to achieve high-dynamic-range surface adaptive exposure. Subsequently, it proposes a method that combines two-frame differential images with multi-view imaging to identify highly reflective regions and complete point cloud holes. This approach addresses the issue of visual blind spots caused by shadows and local high reflectivity due to the occlusion of the measured object’s geometric features. Finally, the system employs deep learning to evaluate the quality of the high-dynamic-range point cloud data. Comparative experiments show that optimal exposure in single-frame images can achieve better imaging quality and shorten capture time. The dual-frame difference image algorithm can identify high-reflection areas and complete the point cloud data. The point cloud quality evaluation model based on IT-PCQA demonstrates the effectiveness of the proposed method for high-dynamic-range 3D reconstruction.

A Method for Tracking an Underwater Pipeline from Stereo Images Using an Autonomous Underwater Vehicle

The problem of tracking an underwater pipeline (UP) is considered in the context of an inspection mission. It is assumed that the diameter of the cylindrical pipe is known. To solve this problem, a tracking method is proposed, based on searching and calculating the center line of the UP, and determining the relative position of the autonomous underwater robot (AUV) and UP in the coordinate system of the AUV camera. Unlike well-known analogues, in which the solution is based on recognizing and constructing the boundaries of the UP in images, the proposed method searches for the true position of the center line of the UP on a set of possible options by checking their veracity. Possible options for the spatial position of the search centerline of the current UP section are generated by varying the direction of the beam in the horizontal and vertical plane. The starting point of the ray is the end point of the centerline of the previous section. The veracity criterion is to check that the 3D point features constructed in the scene belong to the cylindrical surface of the UP. The generation and matching of pointfeatures in images of a stereo pair is carried out using the SURF detector. The choice of the correct direction of the center line of the current UP section is ultimately made by voting. The end point of the center line of the current section is determined taking into account the calculated common visibility area with the previous section of the UP. To evaluate the effectiveness of the method, computational experiments were carried out on virtual scenes. The effectiveness was assessed by the accuracy of UP localization (in the AUV coordinate system) and by the speed of calculations in comparison with: a) the first version of this method, based on the use of a vectorized form of images; b) analogues using the Canny and Hough Transform detectors. The stability of the method to the accumulation of navigation accuracy errors during long-term tracking of UPs was also assessed.