Object Detection In Point Clouds Research Articles

Gradient-based sparse voxel attacks on point cloud object detection

Point cloud object detection is crucial for a variety of applications, including autonomous driving and robotics. Voxel-based representation for 3D point clouds has drawn significant attention due to their efficiency and effectiveness. Recent studies have revealed the vulnerability of deep learning models to adversarial attacks, while considerably less attention is paid to the robustness of voxel-based point cloud object detectors. Existing adversarial attacks on the point cloud data involve generating fake obstacles, removing objects or producing fake predictions. Despite the demonstrated success, these approaches have three limitations. First, manipulating point data, which was originally designed for point-based representation, is inapplicable to voxel-based representation. Second, existing works that modified points in the hold scene yield redundant perturbations. Third, the evaluation primarily performed on small-scale datasets, such as KITTI, does not scale well. To address these limitations, we propose a gradient-based sparse voxel attack (GSVA) algorithm for voxel-based 3D point cloud object detectors. Two novel frameworks, i.e., re-voxelization-based voxel attack framework and light voxel attack framework, successfully modify voxel-based representation instead of raw points. In addition to KITTI, extensive experiments on large-scale datasets including nuScenes and Waymo Open Dataset demonstrate the favorable attack performance (with mAP decrease by 86.2%∼99.5%) and the slight perturbation costs (with lowest modification rate of 3.5%) of our voxel attack method over the state-of-the-art approaches.

DopplerPTNet: Object Detection Network with Doppler Velocity Information for FMCW LiDAR Point Cloud

Abstract In the field of autonomous driving, LiDAR plays a crucial role in perception and detection. LiDAR based on Time-of-Flight (ToF) mode can only provide three-dimensional spatial coordinate information of point clouds. In point cloud object detection, the limited feature information of spatial coordinates to some extent restricts the further optimization and improvement of algorithm detection performance. However, LiDAR based on Frequency-Modulated Continuous-Wave (FMCW) mode can not only obtain the three-dimensional spatial coordinates of point clouds, but also directly measure the Doppler velocity information of points, effectively compensating for the limitation of relying solely on spatial coordinate information for object recognition. Therefore, based on the CARLA simulator, we construct the first FMCW LiDAR point cloud object detection simulation dataset, FMCWLidDet. What’s more, a novel 4D object detection algorithm, DopplerPTNet, is proposed based on the direct acquisition of point Doppler velocity information by FMCW LiDAR. The algorithm solves the problem of single spatial coordinate information feature in existing 3D object detection algorithms, which makes it difficult to further improve detection accuracy. The dataset is available at https://github.com/xilight123/FMCW-LiDAR-object-detection-dataset.

Improved 3D Object Detection Based on PointPillars

Despite the recent advancements in 3D object detection, the conventional 3D point cloud object detection algorithms have been found to exhibit limited accuracy for the detection of small objects. To address the challenge of poor detection of small-scale objects, this paper adopts the PointPillars algorithm as the baseline model and proposes a two-stage 3D target detection approach. As a cutting-edge solution, point cloud processing is performed using Transformer models. Additionally, a redefined attention mechanism is introduced to further enhance the detection capabilities of the algorithm. In the first stage, the algorithm uses PointPillars as the baseline model. The central concept of this algorithm is to transform the point cloud space into equal-sized columns. During the feature extraction stage, when the features from all cylinders are transformed into pseudo-images, the proposed algorithm incorporates attention mechanisms adapted from the Squeeze-and-Excitation (SE) method to emphasize and suppress feature information. Furthermore, the 2D convolution of the traditional backbone network is replaced by dynamic convolution. Concurrently, the addition of the attention mechanism further improves the feature representation ability of the network. In the second phase, the candidate frames generated in the first phase are refined using a Transformer-based approach. The proposed algorithm applies channel weighting in the decoder to enhance channel information, leading to improved detection accuracy and reduced false detections. The encoder constructs the initial point features from the candidate frames for encoding. Meanwhile, the decoder applies channel weighting to enhance the channel information, thereby improving the detection accuracy and reducing false detections. In the KITTI dataset, the experimental results verify the effectiveness of this method in small objects detection. Experimental results show that the proposed method significantly improves the detection capability of small objects compared with the baseline PointPillars. In concrete terms, in the moderate difficulty detection category, cars, pedestrians, and cyclists average precision (AP) values increased by 5.30%, 8.1%, and 10.6%, respectively. Moreover, the proposed method surpasses existing mainstream approaches in the cyclist category.

A study on 3D LiDAR-based point cloud object detection using an enhanced PointPillars network

Abstract The PointPillar target detection algorithm is a mainstream 3D lidar point cloud target detection algorithm that has a fast response speed but low detection accuracy. Addressing the problem of the low detection accuracy of the PointPillar target detection network, we propose an improved PointPillar target detection algorithm that integrates an attention mechanism. The algorithm first introduces the attention mechanism and strengthens the feature extraction module based on PointPillar to realize the amplification of the local information in the three scale feature maps and to better extract the more important feature information. Then, our algorithm adds an anchor free type detector head to further optimize the detector head module. The experimental results show that the optimized PointPillar target detection algorithm has achieved good test results in the KITTI data set. Under medium difficulty, the AOS mode mAP reaches 79.76%, the 3D mode mAP reaches 82.03%, and the BEV mode mAP reaches 82.30%. Compared with that of other point cloud target detection algorithms, the detection accuracy of our algorithm is improved by approximately 10%.

Intracranial aneurysm detection based on 3D point cloud object detection method

Intracranial aneurysm detection based on 3D point cloud object detection method

Efficient flexible voxel-based two-stage network for 3D object detection in autonomous driving

3D object detection from the LiDAR point cloud plays an important role in autonomous driving. It is difficult to balance inference speed and detection accuracy when performing 3D point cloud object detection due to the large size of point cloud data and its unstructured storage, which makes it difficult to represent its features. To address the challenge, we propose a two-stage point cloud object detector, AFV-RCNN. In stage-1, the attention flexible voxel feature encoding layer is introduced, which utilizes flexible voxels to enhance feature encoding speed and focuses on foreground points to be detected through voxel attention. In stage-2, the multi-level and grid-based multi-scale RoI (Region of Interest) feature fusion module is designed. It directly extracts complete 3D structures from 3D region proposals and focuses on both local and global features through multi-scale partitioning. In the training stage, GHM-C Loss is applied to address the challenges associated with imbalanced target categories and the imbalance between difficult and easy samples in the classification task. We evaluate the model on the public KITTI Dataset and Waymo Open Dataset. The mAP in KITTI for 3D detection is 73.41% and inference on a single GPU reaches 30.0 fps. Compared with other state-of-the-art methods, AFV-RCNN achieves both the inference speed of a single-stage detector and the detection accuracy of a two-stage detector. It ensures higher detection accuracy while efficiently processing the point cloud.

Multi-Scale Feature Fusion Point Cloud Object Detection Based on Original Point Cloud and Projection

Multi-Scale Feature Fusion Point Cloud Object Detection Based on Original Point Cloud and Projection

Enhancing small object detection in point clouds with self-attention voting network

Enhancing small object detection in point clouds with self-attention voting network

SSF: Sparse point cloud object detection based on self-adaptive voxel encoding and focal-sparse convolution

Point cloud object detection is gradually playing a key role in autonomous driving tasks. To address the issue of insensitivity to sparse objects in point cloud object detection, we have made improvements to the voxel encoding and 3D backbone network of the PVRCNN++. We have introduced adaptive pooling operations during voxel feature encoding to expand the point cloud information within each voxel, followed by the utilization of multi-layer perceptrons to extract richer point cloud features. On the 3D backbone network, we have employed adaptive sparse convolution operations to make the backbone network’s channel count more flexible, allowing it to accommodate a wider range of input data types. Furthermore, we have integrated Focal Loss to tackle the issue of class imbalance in detection tasks. Experimental results on the public KITTI dataset demonstrate significant improvements over the PVRCNN++, particularly in pedestrian and bicycle detection tasks. Specifically, we have observed 1% increase in detection accuracy for pedestrians and 2.1% improvement for bicycles. Our detection performance also surpasses that of other comparative detection algorithms.

Three-Dimensional Point Cloud Object Detection Based on Feature Fusion and Enhancement

With the continuous emergence and development of 3D sensors in recent years, it has become increasingly convenient to collect point cloud data for 3D object detection tasks, such as the field of autonomous driving. But when using these existing methods, there are two problems that cannot be ignored: (1) The bird’s eye view (BEV) is a widely used method in 3D objective detection; however, the BEV usually compresses dimensions by combined height, dimension, and channels, which makes the process of feature extraction in feature fusion more difficult. (2) Light detection and ranging (LiDAR) has a much larger effective scanning depth, which causes the sector to become sparse in deep space and the uneven distribution of point cloud data. This results in few features in the distribution of neighboring points around the key points of interest. The following is the solution proposed in this paper: (1) This paper proposes multi-scale feature fusion composed of feature maps at different levels made of Deep Layer Aggregation (DLA) and a feature fusion module for the BEV. (2) A point completion network is used to improve the prediction results by completing the feature points inside the candidate boxes in the second stage, thereby strengthening their position features. Supervised contrastive learning is applied to enhance the segmentation results, improving the discrimination capability between the foreground and background. Experiments show these new additions can achieve improvements of 2.7%, 2.4%, and 2.5%, respectively, on KITTI easy, moderate, and hard tasks. Further ablation experiments show that each addition has promising improvement over the baseline.

3D Point Cloud Object Detection Method Based on Multi-Scale Dynamic Sparse Voxelization.

Perception plays a crucial role in ensuring the safety and reliability of autonomous driving systems. However, the recognition and localization of small objects in complex scenarios still pose challenges. In this paper, we propose a point cloud object detection method based on dynamic sparse voxelization to enhance the detection performance of small objects. This method employs a specialized point cloud encoding network to learn and generate pseudo-images from point cloud features. The feature extraction part uses sliding windows and transformer-based methods. Furthermore, multi-scale feature fusion is performed to enhance the granularity of small object information. In this experiment, the term "small object" refers to objects such as cyclists and pedestrians, which have fewer pixels compared to vehicles with more pixels, as well as objects of poorer quality in terms of detection. The experimental results demonstrate that, compared to the PointPillars algorithm and other related algorithms on the KITTI public dataset, the proposed algorithm exhibits improved detection accuracy for cyclist and pedestrian target objects. In particular, there is notable improvement in the detection accuracy of objects in the moderate and hard quality categories, with an overall average increase in accuracy of about 5%.

AFRNet: Anchor-Free Object Detection Using Roadside LiDAR in Urban Scenes

In urban settings, roadside infrastructure LiDAR is a ground-based remote sensing system that collects 3D sparse point clouds for the traffic object detection of vehicles, pedestrians, and cyclists. Current anchor-free algorithms for 3D point cloud object detection based on roadside infrastructure face challenges related to inadequate feature extraction, disregard for spatial information in large 3D scenes, and inaccurate object detection. In this study, we propose AFRNet, a two-stage anchor-free detection network, to address the aforementioned challenges. We propose a 3D feature extraction backbone based on the large sparse kernel convolution (LSKC) feature set abstraction module, and incorporate the CBAM attention mechanism to enhance the large scene feature extraction capability and the representation of the point cloud features, enabling the network to prioritize the object of interest. After completing the first stage of center-based prediction, we propose a refinement method based on attentional feature fusion, where fused features incorporating raw point cloud features, voxel features, BEV features, and key point features are used for the second stage of refinement to complete the detection of 3D objects. To evaluate the performance of our detection algorithms, we conducted experiments using roadside LiDAR data from the urban traffic dataset DAIR-V2X, based on the Beijing High-Level Automated Driving Demonstration Area. The experimental results show that AFRNet has an average of 5.27 percent higher detection accuracy than CenterPoint for traffic objects. Comparative tests further confirm that our method achieves high accuracy in roadside LiDAR object detection.

Equal Emphasis on Data and Network: A Two-Stage 3D Point Cloud Object Detection Algorithm with Feature Alignment

Three-dimensional object detection is a pivotal research topic in computer vision, aiming to identify and locate objects in three-dimensional space. It has wide applications in various fields such as geoscience, autonomous driving, and drone navigation. The rapid development of deep learning techniques has led to significant advancements in 3D object detection. However, with the increasing complexity of applications, 3D object detection faces a series of challenges such as data imbalance and the effectiveness of network models. Specifically, in an experiment, our investigation revealed a notable discrepancy in the LiDAR reflection intensity within a point cloud scene, with stronger intensities observed in proximity and weaker intensities observed at a distance. Furthermore, we have also noted a substantial disparity in the number of foreground points compared to the number of background points. Especially in 3D object detection, the foreground point is more important than the background point, but it is usually downsampled without discrimination in the subsequent processing. With the objective of tackling these challenges, we work from both data and network perspectives, designing a feature alignment filtering algorithm and a two-stage 3D object detection network. Firstly, in order to achieve feature alignment, we introduce a correction equation to decouple the relationship between distance and intensity and eliminate the attenuation effect of intensity caused by distance. Then, a background point filtering algorithm is designed by using the aligned data to alleviate the problem of data imbalance. At the same time, we take into consideration the fact that the accuracy of semantic segmentation plays a crucial role in 3D object detection. Therefore, we propose a two-stage deep learning network that integrates spatial and spectral information, in which a feature fusion branch is designed and embedded in the semantic segmentation backbone. Through a series of experiments on the KITTI dataset, it is proven that the proposed method achieves the following average precision (AP_R40) values for easy, moderate, and hard difficulties, respectively: car (Iou 0.7)—89.23%, 80.14%, and 77.89%; pedestrian (Iou 0.5)—52.32%, 45.47%, and 38.78%; and cyclist (Iou 0.5)—76.41%, 61.92%, and 56.39%. By emphasizing both data quality optimization and efficient network architecture, the performance of the proposed method is made comparable to other state-of-the-art methods.

Language guided 3D object detection in point clouds for MEP scenes

AbstractIn recent years, contrastive language‐image pre‐training (CLIP) has gained popularity for processing 2D data. However, the application of cross‐modal transferable learning to 3D data remains a relatively unexplored area. In addition, high‐quality, labelled point cloud data for Mechanical, Electrical, and Plumbing (MEP) scenarios are in short supply. To address this issue, the authors introduce a novel object detection system that employs 3D point clouds and 2D camera images, as well as text descriptions as input, using image‐text matching knowledge to guide dense detection models for 3D point clouds in MEP environments. Specifically, the authors put forth the proposition of a language‐guided point cloud modelling (PCM) module, which leverages the shared image weights inherent in the CLIP backbone. This is done with the aim of generating pertinent category information for the target, thereby augmenting the efficacy of 3D point cloud target detection. After sufficient experiments, the proposed point cloud detection system with the PCM module is proven to have a comparable performance with current state‐of‐the‐art networks. The approach has 5.64% and 2.9% improvement in KITTI and SUN‐RGBD, respectively. In addition, the same good detection results are obtained in their proposed MEP scene dataset.

3D point cloud object detection algorithm based on Transformer

In response to the difficulty in deploying anchor box based methods in 3D object detection due to the increase in spatial dimensions, this paper studies a point cloud object detection algorithm based on set prediction. This article proposes a Transformer based 3D point cloud object detection algorithm, and combines the characteristics of point clouds in autonomous driving scenarios to propose an improved spatial modulation attention and heat map initialization strategy for training acceleration and query initialization, achieving good detection performance in shallow networks. This article compares it with other algorithms on the KITTI dataset, and the results show that our algorithm has reached an advanced level in performance. We also conducted ablation experiments on the main components of the algorithm to verify the contribution of each module to the detection effect.

Optimized voxel transformer for 3D detection with spatial-semantic feature aggregation

In this paper, we propose a novel 3D object detection model that leverages the advantages of the Voxel Transformer (VoTr) and the Confident IoU-Aware Single-Stage Object Detector (CIA-SSD) to address the challenges of detecting objects in 3D point clouds. Our model adopts the VoTr as its backbone, which enables long-range interactions between voxels via a self-attention mechanism. This overcomes the limitations of conventional voxel-based 3D detectors, which struggle to capture sufficient contextual information due to their restricted receptive fields. Our model also integrates the sparse voxel module and the submanifold voxel module, which efficiently process empty and non-empty voxel positions, effectively handling the natural sparsity and abundance of non-empty voxels. Moreover, inspired by the CIA-SSD design, our model incorporates the Spatial-Semantic Feature Aggregation (SSFA) module, which allows for the adaptive fusion of high-level abstract semantic features and low-level spatial features, ensuring accurate predictions of bounding boxes and classification confidence. Furthermore, based on the IoU-aware confidence rectification module, which refines the alignment between confidence scores and localization accuracy, we devise an Optimized RPN (Region Proposal Network) Detection Head module as a dense head to further predict the IoU loss and improve the accuracy. In this paper, we combine two state-of-the-art techniques to provide a precise and efficient solution for 3D object detection in point clouds. We evaluate our model on the KITTI dataset1 and achieve 76.56 % accuracy in terms of AP3D (%) Hard.

3D Object Detection Using Multiple-Frame Proposal Features Fusion.

Object detection is important in many applications, such as autonomous driving. While 2D images lack depth information and are sensitive to environmental conditions, 3D point clouds can provide accurate depth information and a more descriptive environment. However, sparsity is always a challenge in single-frame point cloud object detection. This paper introduces a two-stage proposal-based feature fusion method for object detection using multiple frames. The proposed method, called proposal features fusion (PFF), utilizes a cosine-similarity approach to associate proposals from multiple frames and employs an attention weighted fusion (AWF) module to merge features from these proposals. It allows for feature fusion specific to individual objects and offers lower computational complexity while achieving higher precision. The experimental results on the nuScenes dataset demonstrate the effectiveness of our approach, achieving an mAP of 46.7%, which is 1.3% higher than the state-of-the-art 3D object detection method.

Graph Neural Network and Spatiotemporal Transformer Attention for 3D Video Object Detection from Point Clouds.

Previous works for LiDAR-based 3D object detection mainly focus on the single-frame paradigm. In this paper, we propose to detect 3D objects by exploiting temporal information in multiple frames, i.e., the point cloud videos. We empirically categorize the temporal information into short-term and long-term patterns. To encode the short-term data, we present a Grid Message Passing Network (GMPNet), which considers each grid (i.e., the grouped points) as a node and constructs a k-NN graph with the neighbor grids. To update features for a grid, GMPNet iteratively collects information from its neighbors, thus mining the motion cues in grids from nearby frames. To further aggregate the long-term frames, we propose an Attentive Spatiotemporal Transformer GRU (AST-GRU), which contains a Spatial Transformer Attention (STA) module and a Temporal Transformer Attention (TTA) module. STA and TTA enhance the vanilla GRU to focus on small objects and better align the moving objects. Our overall framework supports both online and offline video object detection in point clouds. The evaluation results on the challenging nuScenes benchmark show the superior performance of our method, achieving 1st on the leaderboard without any bells and whistles, by the time the paper is submitted.

3D Point Cloud Object Detection Algorithm Based on Temporal Information Fusion and Uncertainty Estimation

In autonomous driving, LiDAR (light detection and ranging) data are acquired over time. Most existing 3D object detection algorithms propose the object bounding box by processing each frame of data independently, which ignores the temporal sequence information. However, the temporal sequence information is usually helpful to detect the object with missing shape information due to long distance or occlusion. To address this problem, we propose a temporal sequence information fusion 3D point cloud object detection algorithm based on the Ada-GRU (adaptive gated recurrent unit). In this method, the feature of each frame for the LiDAR point cloud is extracted through the backbone network and is fed to the Ada-GRU together with the hidden features of the previous frames. Compared to the traditional GRU, the Ada-GRU can adjust the gating mechanism adaptively during the training process by introducing the adaptive activation function. The Ada-GRU outputs the temporal sequence fusion features to predict the 3D object in the current frame and transmits the hidden features of the current frame to the next frame. At the same time, the label uncertainty of the distant and occluded objects affects the training effect of the model. For this problem, this paper proposes a probability distribution model of 3D bounding box coordinates based on the Gaussian distribution function and designs the corresponding bounding box loss function to enable the model to learn and estimate the uncertainty of the positioning of the bounding box coordinates, so as to remove the bounding box with large positioning uncertainty in the post-processing stage to reduce the false positive rate. Finally, the experiments show that the methods proposed in this paper improve the accuracy of the object detection without significantly increasing the complexity of the algorithm.

A Lightweight Model for 3D Point Cloud Object Detection

With the rapid development of deep learning, more and more complex models are applied to 3D point cloud object detection to improve accuracy. In general, the more complex the model, the better the performance and the greater the computational resource consumption it has. However, complex models are incompatible for deployment on edge devices with restricted memory, so accurate and efficient 3D point cloud object detection processing is necessary. Recently, a lightweight model design has been proposed as one type of effective model compression that aims to design more efficient network computing methods. In this paper, a lightweight 3D point cloud object detection network architecture is proposed. The core innovation of the proposal consists of a lightweight 3D sparse convolution layer module (LW-Sconv module) and knowledge distillation loss. Firstly, in the LW-Sconv module, factorized convolution and group convolution are applied to the standard 3D sparse convolution layer. As the basic component of the lightweight 3D point cloud object detection network proposed in this paper, the LW-Sconv module greatly reduces the complexity of the network. Then, the knowledge distillation loss is used to guide the training of the lightweight network proposed in this paper to further improve the detection accuracy. Finally, extensive experiments are performed to verify the algorithm proposed in this paper. Compared with the baseline model, the proposed model can reduce the FLOPs and parameters by 3.7 times and 7.9 times, respectively. The lightweight model trained with knowledge distillation loss achieves comparable accuracy to the baseline. Experiments show that the proposed method greatly reduces the model complexity while ensuring detection accuracy.