Point Cloud Features Research Articles

A selective harvesting robot for cherry tomatoes: Design, development, field evaluation analysis

AbstractWith the aging population and increasing labor costs, traditional manual harvesting methods have become less economically efficient. Consequently, research into fully automated harvesting using selective harvesting robots for cherry tomatoes has become a hot topic. However, most of the current research is focused on individual harvesting of large tomatoes, and there is less research on the development of complete systems for harvesting cherry tomatoes in clusters. The purpose of this study is to develop a harvesting robot system capable of picking tomato clusters by cutting their fruit‐bearing pedicels and to evaluate the robot prototype in real greenhouse environments. First, to enhance the grasping stability, a novel end‐effector was designed. This end‐effector utilizes a cam mechanism to achieve asynchronous actions of cutting and grasping with only one power source. Subsequently, a visual perception system was developed to locate the cutting points of the pedicels. This system is divided into two parts: rough positioning of the fruits in the far‐range view and accurate positioning of the cutting points of the pedicels in the close‐range view. Furthermore, it possesses the capability to adaptively infer the approaching pose of the end‐effector based on point cloud features extracted from fruit‐bearing pedicels and stems. Finally, a prototype of the tomato‐harvesting robot was assembled for field trials. The test results demonstrate that in tomato clusters with unobstructed pedicels, the localization success rates for the cutting points were 88.5% and 83.7% in the two greenhouses, respectively, while the harvesting success rates reached 57.7% and 55.4%, respectively. The average cycle time to harvest a tomato cluster was 24 s. The experimental results prove the potential for commercial application of the developed tomato‐harvesting robot and through the analysis of failure cases, discuss directions for future work.

MMFG: Multimodal-based Mutual Feature Gating 3D Object Detection

To address the problem that image and point cloud features are fused in a coarse fusion way and cannot achieve deep fusion, this paper proposes a multimodal 3D object detection architecture based on a mutual feature gating mechanism. First, since the feature aggregation approach based on the set abstraction layer cannot obtain fine-grained features, a point-based self-attention mechanism module is designed. This module is added to the extraction branch of point cloud features to achieve fine-grained feature aggregation while maintaining accurate location information. Second, a new gating mechanism is designed for the deep fusion of image and point cloud. Deep fusion is achieved by mutual feature weighting between the image and the point cloud. The newly fused features are then fed into a feature refinement network to extract classification confidence and 3D target bounding boxes. Finally, a multi-scale detection architecture is proposed to obtain a more complete object shape. The location-based encoding feature algorithm is also designed to focus the interest points in the region of interest adaptively. The whole architecture shows outstanding performance on the KITTI3D and nuSenece datasets, especially at the difficult level. It shows that the framework solves the problem of low detection rates in LiDAR mode due to the low number of surface points obtained from distant objects.

PSVMLP: Point and Shifted Voxel MLP for 3D deep learning

We propose a high-performance 3D feature extraction deep learning network based on point cloud and shifted voxel, named Point and Shifted Voxel MLP (PSVMLP). The main component of PSVMLP is simple Multi-Layer Perceptron (MLP) structure. PSVMLP achieves effective extraction of multi-scale features from 3D data. Specifically, we combine point cloud and voxel-based feature extraction methods. In voxel representation learning, we propose a wide-range geometric feature extraction method based on axial shifting operations and simple MLP structure. The axial shifting operations allow shifting voxels in the depth, height, and width directions, capturing more geometric information. In point cloud representation learning, we use simple MLP structure to extract local features, and we also extract global features by combining transformer structure. By combining point cloud and voxel feature extraction methods, we obtain rich feature representations from different scales, enhancing the model’s expressive power and generalization performance. Applying our designed model to basic geometric feature learning tasks, we achieve excellent results. Despite being built primarily on a simple MLP framework, our model demonstrates remarkable performance on both shape classification and shape part segmentation tasks. Our code is available at https://github.com/hitxraz/psvmlp.

Ground segmentation based point cloud feature extraction for 3D LiDAR SLAM enhancement

Point cloud preprocessing lays the foundation for the realization of autonomous vehicles (AVs) as it is the backbone of 3D LiDAR simultaneous localization and mapping (SLAM). Matching feature points selection based on multiple classifiers from preprocessing techniques may significantly increase the chances of the good matching result, and thus reduces drift error accumulation. In this paper, a series of point cloud preprocessing and feature extraction methods were proposed, where LiDAR sensor is used only. Experiments indicate that our ground point segmentation algorithm is efficient, comparable to state-of-the-art methods, and even outperforms the general approaches when measured with certain metrics. Improvement in extracting edge features with vertical clustering can ensure stability and geometrical characteristics of features. With the implementation of the proposed point cloud preprocessing techniques on well-known pose estimation framework such as LeGO-LOAM, higher accuracy with the reduction in both rotation and translation error in most dataset sequences is achieved. Finally, the proposed algorithm is examined and evaluated via KITTI, Semantic-KITTI, and our own VLP-16 campus datasets.

3D Object Detection under Urban Road Traffic Scenarios Based on Dual-Layer Voxel Features Fusion Augmentation.

To enhance the accuracy of detecting objects in front of intelligent vehicles in urban road scenarios, this paper proposes a dual-layer voxel feature fusion augmentation network (DL-VFFA). It aims to address the issue of objects misrecognition caused by local occlusion or limited field of view for targets. The network employs a point cloud voxelization architecture, utilizing the Mahalanobis distance to associate similar point clouds within neighborhood voxel units. It integrates local and global information through weight sharing to extract boundary point information within each voxel unit. The relative position encoding of voxel features is computed using an improved attention Gaussian deviation matrix in point cloud space to focus on the relative positions of different voxel sequences within channels. During the fusion of point cloud and image features, learnable weight parameters are designed to decouple fine-grained regions, enabling two-layer feature fusion from voxel to voxel and from point cloud to image. Extensive experiments on the KITTI dataset demonstrate the significant performance of DL-VFFA. Compared to the baseline network Second, DL-VFFA performs better in medium- and high-difficulty scenarios. Furthermore, compared to the voxel fusion module in MVX-Net, the voxel feature fusion results in this paper are more accurate, effectively capturing fine-grained object features post-voxelization. Through ablative experiments, we conducted in-depth analyses of the three voxel fusion modules in DL-VFFA to enhance the performance of the baseline detector and achieved superior results.

TriClsNet: Surface Reconstruction via Graph-based Triangle Classification

In this paper, we introduce TriClsNet, a novel learning-based network that reconstructs surfaces by reframing the triangle classification problem as a graph node classification problem. An improved graph-based triangle classification module is employed to aggregate information from neighboring triangles, effectively leveraging local neighborhood information and enhancing triangle classification accuracy. Additionally, a self-supervised learning branch is incorporated to predict point cloud normals, aiding our network in better learning local point cloud features. Furthermore, a new loss function is designed to guide our network in effective multi-task learning, encompassing both graph node classification and normal prediction. Comparative experimental results on ShapeNet demonstrate that our method can efficiently perform surface reconstruction, outperforming existing methods in the aspects of preserving surface details, reducing holes, and generalization.

A Novel High-Precision Railway Obstacle Detection Algorithm Based on 3D LiDAR.

This article presents a high-precision obstacle detection algorithm using 3D mechanical LiDAR to meet railway safety requirements. To address the potential errors in the point cloud, we propose a calibration method based on projection and a novel rail extraction algorithm that effectively handles terrain variations and preserves the point cloud characteristics of the track area. We address the limitations of the traditional process involving fixed Euclidean thresholds by proposing a modulation function based on directional density variations to adjust the threshold dynamically. Finally, using PCA and local-ICP, we conduct feature analysis and classification of the clustered data to obtain the obstacle clusters. We conducted continuous experiments on the testing site, and the results showed that our system and algorithm achieved an STDR (stable detection rate) of over 95% for obstacles with a size of 15 cm × 15 cm × 15 cm in the range of ±25 m; at the same time, for obstacles of 10 cm × 10 cm × 10 cm, an STDR of over 80% was achieved within a range of ±20 m. This research provides a possible solution and approach for railway security via obstacle detection.

Method for Generating Indoor 3D Scene Graphs Based on Instance Features and Relationship Encoding

Abstract. A 3D scene graph is a compact and explicit representation in scene analysis. In today’s 3D scene graph prediction methods, the feature encoding method of nodes and edges is relatively simple, which essentially hinders the network from fully learning 3D point cloud features. In this paper, we propose a 3D scene graph task framework that fully expresses node and edge features, trying to meet the requirements of fully utilizing point cloud features to achieve high-precision prediction. Experimental results show that with the help of the new representation method, the prediction performance of 3D scene graphs has been significantly improved.

Improved pass-through dimensionality reduction for point cloud contour extraction

Obtaining a point cloud model that considers the overall information of large steel components as well as the local information of bolt holes is an important approach for the virtual assembly of large steel structures. A point cloud extraction method to improve the uniformity of the dimensionality reduction distribution is presented. Firstly, point cloud data characterising the overall information of a large-size component and local information of the bolt-hole group are obtained by combining vertical three-dimensional (3D) laser scanning and handheld 3D laser scanning. The point cloud is then triaxially equidistant and reduced in dimension based on improved straight-pass filtering and the corner point cloud is extracted using a planar point cloud distribution uniformity algorithm. Finally, the point cloud is restored to the same space to complete the contour extraction of the point cloud. The accuracy of the contour extraction method was verified by conducting point cloud feature extraction tests using standard components. Compared with conventional feature extraction, the method provides targeted local feature extraction for bars with a certain regularity of geometric configuration, reducing the time required for feature extraction and providing a brief database for the virtual assembly of steel joist beams.

A Point Cloud Segmentation Method for Pigs from Complex Point Cloud Environments Based on the Improved PointNet++

In animal husbandry applications, segmenting live pigs in complex farming environments faces many challenges, such as when pigs lick railings and defecate within the acquisition environment. The pig’s behavior makes point cloud segmentation more complex because dynamic animal behaviors and environmental changes must be considered. This further requires point cloud segmentation algorithms to improve the feature capture capability. In order to tackle the challenges associated with accurately segmenting point cloud data collected in complex real-world scenarios, such as pig occlusion and posture changes, this study utilizes PointNet++. The SoftPool pooling method is employed to implement a PointNet++ model that can achieve accurate point cloud segmentation for live pigs in complex environments. Firstly, the PointNet++ model is modified to make it more suitable for pigs by adjusting its parameters related to feature extraction and sensory fields. Then, the model’s ability to capture the details of point cloud features is further improved by using SoftPool as the point cloud feature pooling method. Finally, registration, filtering, and extraction are used to preprocess the point clouds before integrating them into a dataset for manual annotation. The improved PointNet++ model’s segmentation ability was validated and redefined with the pig point cloud dataset. Through experiments, it was shown that the improved model has better learning ability across 529 pig point cloud data sets. The optimal mean Intersection over Union (mIoU) was recorded at 96.52% and the accuracy at 98.33%. This study has achieved the automatic segmentation of highly overlapping pigs and pen point clouds. This advancement enables future animal husbandry applications, such as estimating body weight and size based on 3D point clouds.

Environmental-structure-perception-based adaptive pose fusion method for LiDAR-visual-inertial odometry

Environmental-structure-perception-based adaptive pose fusion method for LiDAR-visual-inertial odometry

Robust point cloud registration for map-based autonomous robot navigation

Robust point cloud registration for map-based autonomous robot navigation

A novel model-based welding trajectory planning method for identical structural workpieces

Welding robots have been widely used with the development of manufacturing industry. At present, welding trajectory planning and programming of welding robots are performed separately for similar (structural) workpieces with different dimensions or deformations. Besides, the welding torch pose planning using conventional robot programming methods is time-consuming for workpieces with complex welding trajectories. To realize automatic planning of the welding torch pose for similar workpieces and improve the programming efficiency of welding robots, we propose a novel model-based welding trajectory planning method for welding robots. We create a basic model (point set) containing information on welding seam trajectory planning for similar workpieces. The welding seam trajectory extraction of similar workpieces is achieved by the non-rigid point set registration of the feature point cloud of similar workpieces with the basic model. Then, the welding torch pose planning is implemented by the geometric projection method of the local point cloud of the welding seam trajectory. Finally, the results of offline experiments and online experiments demonstrate the effectiveness of the proposed method.

A region feature fusion network for point cloud and image to detect 3D object

AbstractSensor fusion is very important for collaborative intelligent systems. A regional feature fusion network called ReFuNet for detecting 3D Object is proposed. It is difficult to detect distant or small objects accurately for the sparsity of LiDAR point cloud. The LiDAR point cloud and camera image information to solve the problem of point cloud sparsity is used, which can integrate image‐rich semantic information to enhance point cloud features. Also, the authors’ ReFuNet method segments the possible areas of objects by the results of 2D image detection. A cross‐attention mechanism adaptively fuses image and point cloud features within the areas. Then, the authors’ ReFuNet uses fused features to predict the 3D bounding boxes of objects. Experiments on the KITTI 3D object detection dataset showed that the authors’ proposed fusion method effectively improved the performance of 3D object detection.

HALNet: Partial Point Cloud Registration Based on Hybrid Attention and Deep Local Features.

Point cloud registration is an important task in computer vision and robotics which is widely used in 3D reconstruction, target recognition, and other fields. At present, many registration methods based on deep learning have better registration accuracy in complete point cloud registration, but partial registration accuracy is poor. Therefore, a partial point cloud registration network, HALNet, is proposed. Firstly, a feature extraction network consisting mainly of adaptive graph convolution (AGConv), two-dimensional convolution, and convolution block attention (CBAM) is used to learn the features of the initial point cloud. Then the overlapping estimation is used to remove the non-overlapping points of the two point clouds, and the hybrid attention mechanism composed of self-attention and cross-attention is used to fuse the geometric information of the two point clouds. Finally, the rigid transformation is obtained by using the fully connected layer. Five methods with excellent registration performance were selected for comparison. Compared with SCANet, which has the best registration performance among the five methods, the RMSE(R) and MAE(R) of HALNet are reduced by 10.67% and 12.05%. In addition, the results of the ablation experiment verify that the hybrid attention mechanism and fully connected layer are conducive to improving registration performance.

SelFLoc: Selective feature fusion for large-scale point cloud-based place recognition

Point cloud-based place recognition is crucial for mobile robots and autonomous vehicles, especially when the global positioning sensor is not accessible. LiDAR points are scattered on the surface of objects and buildings, which have strong shape priors along different axes. To enhance message passing along particular axes, Stacked Asymmetric Convolution Block (SACB) is designed, which is one of the main contributions in this paper. Comprehensive experiments demonstrate that asymmetric convolution and its corresponding strategies employed by SACB can contribute to the more effective representation of point cloud feature. On this basis, Selective Feature Fusion Block (SFFB), which is formed by stacking point- and channel-wise gating layers in a predefined sequence, is proposed to selectively boost salient local features in certain key regions, as well as to align the features before fusion phase. SACBs and SFFBs are combined to construct a robust and accurate architecture for point cloud-based place recognition, which is termed SelFLoc. Comparative experimental results show that SelFLoc achieves the state-of-the-art (SOTA) performance on the Oxford and other three in-house benchmarks with an improvement of 1.6 absolute percentages on mean average recall@1.

LGD: A fast place recognition method based on the fusion of local and global descriptors

Place recognition, which is also known as loop detection, has been one of the main problems in the field of robot navigation, and it is crucial in realizing high-performance simultaneous localization and mapping (SLAM). Most of the existing methods directly construct local or global descriptors with point clouds, ignoring the offset of different visits in the same place. To overcome the current limitations, this study emphasizes the importance of point cloud features in improving the environment recognition precision and proposes an innovative framework that combines local features that are spatially invariant and prominent descriptive global features as a representation of a laser scanner frame. The feature center of the laser point cloud is obtained by computing local feature points with spatial invariance, which improves the offset between the original point cloud and the feature center. Furthermore, a coarse-to-fine hierarchical recognition strategy is developed to improve the efficiency of place recognition. An evaluation with different offsets and synthesis is conducted on the publicly available KITTI and self-developed CHDloop datasets. The experimental results show that the proposed method can improve time efficiency by more than 18% compared to the SC and ISC, and there are also significant improvements in the recall rate and accuracy in slight lane-change scenarios. Moreover, the proposed method can obtain rich lane-change loops in moderate and large lane-change scenarios where the SC and ISC fail to obtain effective loops. The proposed method can be used as a lightweight and interpretable approach for place recognition, which not only can be quickly deployed in any SLAM framework but can also efficiently handle more complex place recognition issues in urban environments.

SSF: Sparse point cloud object detection based on self-adaptive voxel encoding and focal-sparse convolution

Point cloud object detection is gradually playing a key role in autonomous driving tasks. To address the issue of insensitivity to sparse objects in point cloud object detection, we have made improvements to the voxel encoding and 3D backbone network of the PVRCNN++. We have introduced adaptive pooling operations during voxel feature encoding to expand the point cloud information within each voxel, followed by the utilization of multi-layer perceptrons to extract richer point cloud features. On the 3D backbone network, we have employed adaptive sparse convolution operations to make the backbone network’s channel count more flexible, allowing it to accommodate a wider range of input data types. Furthermore, we have integrated Focal Loss to tackle the issue of class imbalance in detection tasks. Experimental results on the public KITTI dataset demonstrate significant improvements over the PVRCNN++, particularly in pedestrian and bicycle detection tasks. Specifically, we have observed 1% increase in detection accuracy for pedestrians and 2.1% improvement for bicycles. Our detection performance also surpasses that of other comparative detection algorithms.

Point-Sim: A Lightweight Network for 3D Point Cloud Classification

Analyzing point clouds with neural networks is a current research hotspot. In order to analyze the 3D geometric features of point clouds, most neural networks improve the network performance by adding local geometric operators and trainable parameters. However, deep learning usually requires a large amount of computational resources for training and inference, which poses challenges to hardware devices and energy consumption. Therefore, some researches have started to try to use a nonparametric approach to extract features. Point-NN combines nonparametric modules to build a nonparametric network for 3D point cloud analysis, and the nonparametric components include operations such as trigonometric embedding, farthest point sampling (FPS), k-nearest neighbor (k-NN), and pooling. However, Point-NN has some blindness in feature embedding using the trigonometric function during feature extraction. To eliminate this blindness as much as possible, we utilize a nonparametric energy function-based attention mechanism (ResSimAM). The embedded features are enhanced by calculating the energy of the features by the energy function, and then the ResSimAM is used to enhance the weights of the embedded features by the energy to enhance the features without adding any parameters to the original network; Point-NN needs to compute the similarity between each feature at the naive feature similarity matching stage; however, the magnitude difference of the features in vector space during the feature extraction stage may affect the final matching result. We use the Squash operation to squeeze the features. This nonlinear operation can make the features squeeze to a certain range without changing the original direction in the vector space, thus eliminating the effect of feature magnitude, and we can ultimately better complete the naive feature matching in the vector space. We inserted these modules into the network and build a nonparametric network, Point-Sim, which performs well in 3D classification tasks. Based on this, we extend the lightweight neural network Point-SimP by adding some trainable parameters for the point cloud classification task, which requires only 0.8 M parameters for high performance analysis. Experimental results demonstrate the effectiveness of our proposed algorithm in the point cloud shape classification task. The corresponding results on ModelNet40 and ScanObjectNN are 83.9% and 66.3% for 0 M parameters—without any training—and 93.3% and 86.6% for 0.8 M parameters. The Point-SimP reaches a test speed of 962 samples per second on the ModelNet40 dataset. The experimental results show that our proposed method effectively improves the performance on point cloud classification networks.

Point cloud registration method for indoor depth sensor acquisition system based on dual graph computation with irregular shape factors

AbstractThe registration performance determines the widespread indoor application of 3D models acquired by depth sensors. Many advanced registration methods lack comprehensive feature aggregation and poor generalization capabilities, which improves the mismatching ratio. Here, a dual graph network is proposed by incorporating irregular shape factors to make point cloud features more expressive. At first, we transform point cloud sets into the stellar graph within the local neighbourhood of each point. The deep feature and shape factor of each point are combined in the directional‐connected irregular projection space. Subsequently, the combined features are modelled as the second graph. By the attention mechanism computation, feature information is continuously aggregated with intra‐graph and inter‐graph. Finally, a loss function is utilized to confirm point correspondence and perform the registration through singular value decomposition. Extensive experiments validate that the proposed point cloud registration method achieves state‐of‐the‐art performance.