Plane Segmentation Algorithm Research Articles

A robotic polishing trajectory planning method for TBCs of aero-engine turbine blade using measured point cloud

PurposeThe application of thermal barrier coatings (TBCs) allows aero-engine blades to operate at higher temperatures with higher efficiency. The preparation of the TBCs increases the surface roughness of the blade, which impacts the thermal cycle life and thermal insulation performance of the coating. To reduce the surface roughness of blades, particularly the blades with small size and complex curvature, this paper aims to propose a method for industrial robot polishing trajectory planning based on on-site measuring point cloud.Design/methodology/approachThe authors propose an integrated robotic polishing trajectory planning method using point cloud processing technical. At first, the acquired point cloud is preprocessed, which includes filtering and plane segmentation algorithm, to extract the blade body point cloud. Then, the point cloud slicing algorithm and the intersection method are used to create a preliminary contact point set. Finally, the Douglas–Peucker algorithm and pose frame estimation are applied to extract the tool-tip positions and optimize the tool contact posture, respectively. The resultant trajectory is evaluated by simulation and experiment implementation.FindingsThe target points of trajectory are not evenly distributed on the blade surface but rather fluctuate with surface curvature. The simulated linear and orientation speeds of the robot end could be relatively steady over 98% of the total time within 20% reduction of the rest time. After polishing experiments, the coating roughness on the blade surface is reduced dramatically from Ra 7–8 µm to below Ra 1.0 µm. The removal of the TBCs is less than 100 mg, which is significantly less than the weight of the prepared coatings. The blade surface becomes smoothed to a mirror-like state.Originality/valueThe research on robotic polishing of aero-engine turbine blade TBCs is worthwhile. The real-time trajectory planning based on measuring point cloud can address the problem that there is no standard computer-aided drawing model and the geometry and size of the workpiece to be processed differ. The extraction and optimization of tool contact points based on point cloud features can enhance the smoothness of the robot movement, stability of the polishing speed and performance of the blade surface after polishing.

Automatic coarse registration of point clouds using plane contour shape descriptor and topological graph voting

Registration is a basic yet crucial task in point cloud processing. Obtaining robust as well as accurate features and correspondences is critical to feature-based point cloud registration. Aiming at structured urban scenes, this paper employs plane as primitive for feature extraction and proposes an automatic coarse registration algorithm of pairwise point clouds based on plane contour shape descriptor and topological graph voting. It contains three major steps: First, in order to improve the efficiency and accuracy of plane extraction, an efficient voxel-based plane segmentation algorithm is applied, then a RANSAC plane fitting method is used to obtain high precision plane parameters. Second, we propose a plane contour shape descriptor, hole inner angle combined with triangular centroid distance (HIA-TCD), for obtaining corresponding planes in two laser scans, which is robust to the contour shape matching in the urban scene. Finally, according to the position relationship between plane correspondences, a topological graph is built, and an edge voting strategy is proposed to eliminate potential unreliable plane correspondences, then the optimal matched planes are selected to estimate the rigid transformation. The algorithm is tested by three sets of large-scale outdoor and two sets of indoor scene data, and the experimental results show that the presented algorithm can effectively register the pairwise point cloud. In the outdoor scene, the rotation errors are less than 0.3°, and the translation errors are less than 0.1 m. In the indoor scene, the rotation errors are less than 0.2°, and the translation errors are less than 0.1 m. In addition, our algorithm is superior to the state-of-the-art in efficiency.

A Disparity Refinement Algorithm for Satellite Remote Sensing Images Based on Mean-Shift Plane Segmentation

Objects in satellite remote sensing image sequences often have large deformations, and the stereo matching of this kind of image is so difficult that the matching rate generally drops. A disparity refinement method is needed to correct and fill the disparity. A method for disparity refinement based on the results of plane segmentation is proposed in this paper. The plane segmentation algorithm includes two steps: Initial segmentation based on mean-shift and alpha-expansion-based energy minimization. According to the results of plane segmentation and fitting, the disparity is refined by filling missed matching regions and removing outliers. The experimental results showed that the proposed plane segmentation method could not only accurately fit the plane in the presence of noise but also approximate the surface by plane combination. After the proposed plane segmentation method was applied to the disparity refinement of remote sensing images, many missed matches were filled, and the elevation errors were reduced. This proved that the proposed algorithm was effective. For difficult evaluations resulting from significant variations in remote sensing images of different satellites, the edge matching rate and the edge matching map are proposed as new stereo matching evaluation and analysis tools. Experiment results showed that they were easy to use, intuitive, and effective.

Robust 3-D Plane Segmentation From Airborne Point Clouds Based on Quasi-A-Contrario Theory

Three-dimensional (3-D) plane segmentation has been and continues to be a challenge in 3-D point cloud processing. The current methods typically focus on the planar subsets separation but ignore the requirement of the precise plane fitting. We propose a quasi-a-contrario theory-based plane segmentation algorithm, which is capable of dealing with point clouds of severe noise level, low density, and high complexity robustly. The main proposition is that the final plane can be composed of basic planar subsets with high planar accuracy. We cast planar subset extraction from the point set as a geometric rigidity measuring problem. The meaningfulness of the planar subset is estimated by the number of false alarms (NFA), which can be used to eliminate false-positive effectively. Experiments were conducted to analyze both the planar subset extraction and the 3-D plane segmentation. The results show that the proposed algorithms perform well in terms of accuracy and robustness compared with state-of-art methods. Experimental datasets, results, and executable program of the proposed algorithm are available at https://skyearth.org/publication/project/QTPS.

A Novel OpenMVS-Based Texture Reconstruction Method Based on the Fully Automatic Plane Segmentation for 3D Mesh Models

The Markov Random Field (MRF) energy function, constructed by existing OpenMVS-based 3D texture reconstruction algorithms, considers only the image label of the adjacent triangle face for the smoothness term and ignores the planar-structure information of the model. As a result, the generated texture charts results have too many fragments, leading to a serious local miscut and color discontinuity between texture charts. This paper fully utilizes the planar structure information of the mesh model and the visual information of the 3D triangle face on the image and proposes an improved, faster, and high-quality texture chart generation method based on the texture chart generation algorithm of the OpenMVS. This methodology of the proposed approach is as follows: (1) The visual quality on different visual images of each triangle face is scored using the visual information of the triangle face on each image in the mesh model. (2) A fully automatic Variational Shape Approximation (VSA) plane segmentation algorithm is used to segment the blocked 3D mesh models. The proposed fully automatic VSA-based plane segmentation algorithm is suitable for multi-threaded parallel processing, which solves the VSA framework needed to manually set the number of planes and the low computational efficiency in a large scene model. (3) The visual quality of the triangle face on different visual images is used as the data term, and the image label of adjective triangle and result of plane segmentation are utilized as the smoothness term to construct the MRF energy function. (4) An image label is assigned to each triangle by the minimizing energy function. A texture chart is generated by clustering the topologically-adjacent triangle faces with the same image label, and the jagged boundaries of the texture chart are smoothed. Three sets of data of different types were used for quantitative and qualitative evaluation. Compared with the original OpenMVS texture chart generation method, the experiments show that the proposed approach significantly reduces the number of texture charts, significantly improves miscuts and color differences between texture charts, and highly boosts the efficiency of VSA plane segmentation algorithm and OpenMVS texture reconstruction.

Robust segmentation and localization of structural planes from photogrammetric point clouds in construction sites

Plane segmentation is a simple yet essential processing step for using 3D point clouds in applications such as temporal data registration and object modeling. The performance of traditional plane segmentation algorithms, especially when addressing photogrammetric point clouds, is significantly limited by factors including the errors of the point positions, noise contamination, and unevenly distributed density, resulting in a low segmentation efficiency and poor segmentation outcome. This paper proposes a robust plane segmentation algorithm for addressing photogrammetric point clouds of construction site scenarios. The implementation of the proposed method includes three major steps: preprocessing, segmentation of the point clouds, and extraction of the planes. The preprocessing is designed to suppress noise and filter outliers, and downsample the dataset. In the subsequent segmentation step, we propose a bottom-up approach utilizing the voxelized structure and facet-based global clustering to obtain planar segments using an automatic and unsupervised clustering of the planar facets selected from the voxelized points. In the last step, the points of the planar segments are extracted using a parametric model-based plane fitting. A 3D Hough transform-based method, constrained by an orientated bounding box, is developed to generate the initial model of the planes. The obtained plane models are then localized by the oriented bounding box and optimized by a local clustering-based plane refinement. The experimental results using synthetic datasets and a low-cluttered laser scanning dataset confirm that the proposed method is superior to the traditional algorithms in key criteria including calculation time, extraction precision, and recall. For real applications on two datasets of different construction sites, the proposed method could segment the majority of the structural planes from the points, with accurate model parameters, sizes, and locations estimated.

Automatic “Ground Truth” Annotation and Industrial Workpiece Dataset Generation for Deep Learning

In industry, it is becoming common to detect and recognize industrial workpieces using deep learning methods. In this field, the lack of datasets is a big problem, and collecting and annotating datasets in this field is very labor intensive. The researchers need to perform dataset annotation if a dataset is generated by themselves. It is also one of the restrictive factors that the current method based on deep learning cannot expand well. At present, there are very few workpiece datasets for industrial fields, and the existing datasets are generated from ideal workpiece computer aided design (CAD) models, for which few actual workpiece images were collected and utilized. We propose an automatic industrial workpiece dataset generation method and an automatic ground truth annotation method. Included in our methods are three algorithms that we proposed: a point cloud based spatial plane segmentation algorithm to segment the workpieces in the real scene and to obtain the annotation information of the workpieces in the images captured in the real scene; a random multiple workpiece generation algorithm to generate abundant composition datasets with random rotation workpiece angles and positions; and a tangent vector based contour tracking and completion algorithm to get improved contour images. With our procedures, annotation information can be obtained using the algorithms proposed in this paper. Upon completion of the annotation process, a json format file is generated. Faster R-CNN (Faster R-convolutional neural network), SSD (single shot multibox detector) and YOLO (you only look once: unified, real-time object detection) are trained using the datasets proposed in this paper. The experimental results show the effectiveness and integrity of this dataset generation and annotation method.

A Fast and Accurate Planar-Feature-Based Global Scan Registration Method

Scan registration is the basis of LiDAR-based mapping and localization. When a good initial guess is available, classical local registration algorithms such as iterative closet point (ICP) and normal distributions transform (NDT) can be utilized. However, scan registration without odometer or inertial measurement unit (IMU) is still an open problem. After reviewing the existing methods, we summarize a modular planar-feature-based registration framework. Based on the idea of modular design, we present a novel global scan registration method based on planar features. A modified renormalization-based algorithm considering the sensor noise model is derived and used for accurate plane fitting and covariance computing. An efficient adaptive plane segmentation algorithm is employed to extract planar patches which is based on voxel growing. A novel projected-image-based method is utilized for fast plane attribute calculation. The attributes of individual planes and relation constrains between the planes are employed to robustly build plane correspondences. The plane-plane singular value decomposition (SVD) method is used to estimate the transform, and normalization of the point coordinates is adopted to further improve accuracy. A consensus metric is introduced to determine plane correspondences and transform solution updating, and to select the optimal solution using the distance from the centroid of one plane to its conjugate plane. Extensive comparative experiments were performed on four datasets covering different structured scenarios to evaluate the performance of the proposed registration method. The experimental results demonstrate that the proposed method achieves a better balance between accuracy and efficiency, and has more stable and robust performance across different scenarios compared to all tested state-of-the-art algorithms.

An Efficient Encoding Voxel-Based Segmentation (EVBS) Algorithm Based on Fast Adjacent Voxel Search for Point Cloud Plane Segmentation

Plane segmentation is a basic yet important process in light detection and ranging (LiDAR) point cloud processing. The traditional point cloud plane segmentation algorithm is typically affected by the number of point clouds and the noise data, which results in slow segmentation efficiency and poor segmentation effect. Hence, an efficient encoding voxel-based segmentation (EVBS) algorithm based on a fast adjacent voxel search is proposed in this study. First, a binary octree algorithm is proposed to construct the voxel as the segmentation object and code the voxel, which can compute voxel features quickly and accurately. Second, a voxel-based region growing algorithm is proposed to cluster the corresponding voxel to perform the initial point cloud segmentation, which can improve the rationality of seed selection. Finally, a refining point method is proposed to solve the problem of under-segmentation in unlabeled voxels by judging the relationship between the points and the segmented plane. Experimental results demonstrate that the proposed algorithm is better than the traditional algorithm in terms of computation time, extraction accuracy, and recall rate.

Unsupervised object region proposals for RGB-D indoor scenes

In this paper, we present a novel unsupervised framework for automatically generating bottom up class independent object candidates for detection and recognition in cluttered indoor environments. Utilizing raw depth map from active sensors such as Kinect, we propose a novel plane segmentation algorithm for dividing an indoor scene into predominant planar regions and non-planar regions. Based on this partition, we are able to effectively predict object locations and their spatial extensions. Our approach automatically generates object proposals considering five different aspects: Non-planar Regions (NPR), Planar Regions (PR), Detected Planes (DP), Merged Detected Planes (MDP) and Hierarchical Clustering (HC) of 3D point clouds. Object region proposals include both bounding boxes and instance segments. Our approach achieves very competitive results and is even able to outperform supervised state-of-the-art algorithms on the challenging NYU-v2 RGB-Depth dataset. In addition, we apply our approach to the most recently released large scale RGB-Depth dataset from Princeton University – “SUN RGBD”, which utilizes four different depth sensors. Its consistent performance demonstrates a general applicability of our approach.