3D Point Cloud Classification Research Articles

Robust Structured Declarative Classifiers for Point Clouds

Deep neural networks for 3D point cloud classification, such as PointNet, have been demonstrated to be vulnerable to adversarial attacks. Current adversarial defenders often learn to denoise the (attacked) point clouds by reconstruction, and then feed them to the classifiers as input. In contrast to the literature, we propose a novel bilevel optimization framework for robust point cloud classification, where the internal optimization can effectively defend the adversarial attacks as denoising and the external optimization can learn the classifiers accordingly. As a demonstration, we further propose an effective and efficient instantiation of our approach, namely, Lattice Point Classifier (LPC), based on structured sparse coding in the permutohedral lattice and 2D convolutional neural networks (CNNs) that integrates both internal and external optimization problems into end-to-end trainable network architectures. We demonstrate state-of-the-art robust point cloud classification performance on ModelNet40 and ScanNet under seven different attackers. Our demo code is available at: https://github.com/Zhang-VISLab .

FMP: Enhancing 3D Point Cloud Classification With Functional Multi-Head Pooling to Surpass PointNet

FMP: Enhancing 3D Point Cloud Classification With Functional Multi-Head Pooling to Surpass PointNet

DualMLP: a two-stream fusion model for 3D point cloud classification

DualMLP: a two-stream fusion model for 3D point cloud classification

3D Point Cloud Classification Method Based on Multiple Attention Mechanism and Dynamic Graph Convolution

In order to solve the problem of uneven density and the low classification accuracy of 3D point cloud, a 3D point cloud classification method fuses multi-attention machine is proposed. It is principally based on the traditional point cloud dynamic graph convolution classification network, into multiple attention mechanisms, including self-attention, spatial attention and channel attention mechanisms. The self-attention mechanism can reduce the dependence on irrelevant points while aligning point clouds, and input the processed point cloud into the classification network. Then the missing geometric information in the classification network is compensated by the integration of spatial and channel attention mechanisms. The experimental results on the public data set ModelNet40 indicate that compared with the DGCNN classification network, the improved network model improves the classification accuracy of the data set by 0.5 % and the average accuracy by 0.9 %. Meantime, the classification accuracy outstrips other contrast classification algorithms.

A new self-augment CNN for 3D point cloud classification and segmentation

A new self-augment CNN for 3D point cloud classification and segmentation

POINT-WISE CLASSIFICATION OF HIGH-DENSITY UAV-LIDAR DATA USING GRADIENT BOOSTING MACHINES

Abstract. Point-wise classification of 3D point clouds is a challenging task in point cloud processing, whereas, in particular, its application to high-density point clouds needs special attention because a large number of point clouds affect computational efficiency negatively. Although deep learning based models have been gaining popularity in recent years and have reached state-of-the-art results in accuracy for point-wise classification, their requirements of the high number of training samples and computational resources make those models inefficient for high-density 3D point clouds. However, traditional machine learning classifiers require less training samples, so they are capable of reducing computational requirements, even considering the latest machine learning classifiers, particularly in ensemble learning of gradient boosting machines, the results can compete with deep learning models. In this study, we are studying the point-wise classification of high-density UAV LiDAR data and focusing on efficient feature extraction and a recent state-of-the-art gradient boosting machine learning classifier, LightGBM. Our proposed framework includes the following steps: at first, we are using point cloud sampling for creating sub-sampled point clouds, then we are calculating the features based on those scales implemented on GPU. Finally, we are using the LightGBM classifier for training and testing. For the evaluation of our framework, we used a publicly available benchmark dataset, Hessigheim 3D. According to the results, we achieved an overall accuracy of 87.59% and an average F1 score of 75.92%. Our framework has promising results and scores closer to deep learning models. However, more distinctive features are required to obtain more accurate results.

Research on 3D Point Cloud Classification Based on Density-Based Spatial Clustering of Algorithm with Noise

The classification of three-dimensional point clouds is a complex task because of its disorder and uneven density. This paper proposes that in the point-cloud preprocessing stage, the Density-Based Spatial Clustering of Algorithm with Noise (DBSCAN) is added to cluster the three-dimensional point cloud, then the clustering results are extracted through the PointNet deep learning network to extract the characteristics of the local area, thus outputting the classification results of the point cloud. This method can not only reflect the feature distribution of point cloud in three-dimensional space, but also can be divided into several classes according to the different shape features of point cloud. Verified in the ModelNet10 and ModelNet40 point cloud dataset, the classification accuracy of this method on both ModelNet10 and ModelNet40 can reach more than 92.5%.

Improving point cloud classification and segmentation via parametric veronese mapping

Deep learning based 3D point cloud classification and segmentation has achieved remarkable success. Existing methods are usually implemented in the original space with 3D coordinates as inputs. However, we find that point networks taking only information of first-order coordinates hardly learn geometric features of higher order, such as point cloud normals or poses. In this study, we propose to map the input point clouds into a non-linear space to facilitate networks learning and leveraging high-order features. Firstly, we design the Parametric Veronese Mapping (PVM) function which automatically learns to map point clouds into a non-linear space. As a result, the mapped point clouds are enriched with high-order elements and maintain the basic point set properties as in the original 3D space. We can then exploit existing networks to learn high-order features from mapped point clouds. Secondly, we contribute a two-stage transformation learning module that modifies the previous one-stage module to better leverage high-order features for aligning point clouds in the projective space. Finally, an interaction module is designed to learn more discriminative features by aggregating information from both the original and projective space. Extensive experiments demonstrate that our method successfully improves the ability of most existing networks to learn high-order features and thus contributing to more accurate classification and segmentation. Moreover, the resulting models show stronger robustness to affine transformations and real-world perturbations.

3D SCENE RECONSTRUCTION AND DIGITIZATION METHOD FOR MIXED REALITY SYSTEMS

Mixed reality systems are a promising direction of research that opens up great opportunities forinteraction with virtual objects in the real world. Like any promising direction, mixed reality has a number of unresolved problems. One of these problems is the synthesis of natural lighting conditions for virtual objects, including the correct light interaction of virtual objects with the real world. Since virtual and real objects are located in different spaces, it is difficult to ensure their correct interaction. To create digital copies of realworld objects, machine learning tools and neural network technologies are employed. These methods are successfully used in computer vision for space orientation and environment reconstruction. As a solution, it is proposed to transfer all objects into the same information space: virtual space. This makes it possible to solve most of the problems associated with visual discomfort caused by the unnatural light interaction of real and virtual objects. Thus, the basic idea of the method is to recognize physical objects from point clouds and replace these objects with virtual CAD models. In other words, it implies semantic analysis of a scene and classification of objects with their subsequent transformation into polygonal models. In this study, we use competitive neural network architectures, which can provide state-of-the-art results. The test experiments are carried out on Semantic3D, ScanNet, and S3DIS, which are currently the largest datasets with point clouds that represent indoor scenes. For semantic segmentation and classification of 3D point clouds, we use the PointNeXt architecture based on PointNet, as well as modern methods of data augmentation in the process of learning. For geometry reconstruction, the Soft Rasterizer differentiable rendering method and the Total3Understanding neural network are considered.

Deep 3D point cloud classification and segmentation network based on GateNet

Deep 3D point cloud classification and segmentation network based on GateNet

3D Point Cloud Classification with ACGAN-3D and VACWGAN-GP

Machine learning and deep learning techniques are widely used to make sense of 3D point cloud data which became ubiquitous and important due to the recent advances in 3D scanning technologies and other sensors. In this work, we propose two networks to predict the class of the input 3D point cloud: 3D Auxiliary Classifier Generative Adversarial Network (ACGAN-3D) and Versatile Auxiliary Conditional Wasserstein Generative Adversarial Network with Gradient Penalty (VACWGAN-GP). Unlike other classifiers, we are able to enlarge the limited data set with the data produced by generative models. We consequently aim to increase the success of the model by training it with more data. As suggested by the conventional ACGAN models, in addition to the real dataset, we train the Discriminator with synthetic data generated by the Generator using the class label. By doing so, we ensure that Discriminator can discriminate between the real data and the synthetic data. Thus, as the training evolves, the Generator is trained to produce more realistic synthetic data, which in turn forces Discriminator to classify or discriminate better. Defined originally on 2D images, our ACGAN-3D modifies this conventional ACGAN model in order to classify 3D point clouds by updating the neural network layers. Our second model VACWGAN-GP, on the other hand, demonstrates similar abilities with more stable training by replacing its Discriminator with Critic and by modifying its loss function. In this model, we managed to merge Wasserstein GAN-GP with conditional GAN in order to improve the classifier?s performance. The proposed models ACGAN-3D and VACWGAN-GP were tested extensively on 3D datasets and comparisons with the other state-of-the-art studies have revealed our clear advantages on various aspects. While ACGAN-3D can be preferred with its compact design, our second method VACWGAN-GP stands out for higher performance.

Privacy protection for 3D point cloud classification based on an optical chaotic encryption scheme.

In allusion to the privacy and security problems in 3D point cloud classification, a novel privacy protection method for 3D point cloud classification based on optical chaotic encryption scheme is proposed and implemented in this paper for the first time. The mutually coupled spin-polarized vertical-cavity surface-emitting lasers (MC-SPVCSELs) subject to double optical feedback (DOF) are studied to generate optical chaos for permutation and diffusion encryption process of 3D point cloud. The nonlinear dynamics and complexity results demonstrate that the MC-SPVCSELs with DOF have high chaotic complexity and can provide tremendously large key space. All the test-sets of ModelNet40 dataset containing 40 object categories are encrypted and decrypted by the proposed scheme, and then the classification results of 40 object categories for original, encrypted, and decrypted 3D point cloud are entirely enumerated through the PointNet++. Intriguingly, the class accuracies of the encrypted point cloud are nearly all equal to 0.0000% except for the plant class with 100.0000%, indicating the encrypted point cloud cannot be classified and identified. The decryption class accuracies are very close to the original class accuracies. Therefore, the classification results verify that the proposed privacy protection scheme is practically feasible and remarkably effective. Additionally, the encryption and decryption results show that the encrypted point cloud images are ambiguous and unrecognizable, while the decrypted point cloud images are identical to original images. Moreover, this paper improves the security analysis via analyzing 3D point cloud geometric features. Eventually, various security analysis results validate that the proposed privacy protection scheme has high security level and good privacy protection effect for 3D point cloud classification.

Exploiting Manifold Feature Representation for Efficient Classification of 3D Point Clouds

In this paper, we propose an efficient point cloud classification method via manifold learning based feature representation. Different from conventional methods, we use manifold learning algorithms to embed point cloud features for better considering the geometric continuity on the surface. Then, the nature of point cloud can be acquired in low dimensional space, and after being concatenated with features in the original three-dimensional (3D) space, both the capability of feature representation and the classification network performance can be improved. We explore three traditional manifold algorithms (i.e., Isomap, Locally-Linear Embedding, and Laplacian eigenmaps) in detail, and finally, we select the Locally-Linear Embedding (LLE) algorithm due to its low complexity and locality consistency preservation. Furthermore, we propose a neural network based manifold learning (NNML) method to implement manifold learning based non-linear projection. Experiments demonstrate that the proposed two manifold learning methods can obtain better performances than the state-of-the-art methods, and the obtained mean class accuracy (mA) and overall accuracy (oA) can reach 91.4% and 94.4%, respectively. Moreover, because of the improved feature learning capability, the proposed NNML method can also have better classification accuracy on models with prominent geometric shapes. To further demonstrate the advantages of PointManifold, we extend it as a plug and play method for point cloud classification task, which can be directly used with existing methods and gain a significant improvement.

Residual learning with annularly convolutional neural networks for classification and segmentation of 3D point clouds

Analysis of point clouds through deep convolutional neural networks is an active area of research due to their massive real-world applications including autonomous driving, indoor navigation, robotics, virtual/augmented reality, unmanned aerial vehicles, and drone technology. However, capturing the fine-grained geometric and semantic properties for the underlying recognition task with raw unstructured point cloud is highly challenging due to the lack of explicit neighborhood relationship and sparsity among the points. In this paper, we have introduced a deep, hierarchical 3D point based architecture for object classification and part segmentation that is able to learn robust geometric features which remain invariant to both the geometry and orientation of the local patches. The proposed architecture consists of multiple layers of sampling, concentric annular convolution, pooling, and residual feature propagation blocks. In the skip connections of our deep residual design, we propose to use a combination of linear projection shortcut and nonlinear ReLU group normalization shortcut with batch normalization, to improve both the optimization landscape and the representational power. Our network achieves on par or even better than state-of-the-art results on synthetic and real-world object classification (i.e., ModelNet40 and ScanObjectNN) and part segmentation (i.e., ShapeNet-part) benchmark datasets. The implementation and results have been made available at …https://github.com/Rabbia-Hassan/Deep_Annular_Residual_Feature_Learning_for_3dPointCloudsGitHub-link

GN-CNN: A Point Cloud Analysis Method for Metaverse Applications

Metaverse applications often require many new 3D point cloud models that are unlabeled and that have never been seen before; this limited information results in difficulties for data-driven model analyses. In this paper, we propose a novel data-driven 3D point cloud analysis network GN-CNN that is suitable for such scenarios. We tackle the difficulties with a few-shot learning (FSL) approach by proposing an unsupervised generative adversarial network GN-GAN to generate prior knowledge and perform warm start pre-training for GN-CNN. Furthermore, the 3D models in the Metaverse are mostly acquired with a focus on the models’ visual appearances instead of the exact positions. Thus, conceptually, we also propose to augment the information by unleashing and incorporating local variance information, which conveys the appearance of the model. This is realized by introducing a graph convolution-enhanced combined multilayer perceptron operation (CMLP), namely GCMLP, to capture the local geometric relationship as well as a local normal-aware GeoConv, namely GNConv. The GN-GAN adopts an encoder–decoder structure and the GCMLP is used as the core operation of the encoder. It can perform the reconstruction task. The GNConv is used as the convolution-like operation in GN-CNN. The classification performance of GN-CNN is evaluated on ModelNet10 with an overall accuracy of 95.9%. Its few-shot learning performance is evaluated on ModelNet40, when the training set size is reduced to 30%, the overall classification accuracy can reach 91.8%, which is 2.5% higher than Geo-CNN. Experiments show that the proposed method could improve the accuracy in 3D point cloud classification tasks and under few-shot learning scenarios, compared with existing methods such as PointNet, PointNet++, DGCNN, and Geo-CNN, making it a beneficial method for Metaverse applications.

Adaptive neighbourhood recovery method for machine learning based 3D point cloud classification

ABSTRACT 3D scene analysis and classification by automatically assigning semantic labels to 3D points has become a major concern in remote sensing, photogrammetry and computer vision. Due to the high redundancy of point clouds and unbalanced sampling density, it is difficult to recover appropriately sized neighbourhoods to extract local features from 3D point clouds, which will decrease the classification performance. Existing point cloud neighborhood recovery methods usually use fixed-size neighbourhoods, where the shape structure of the point cloud and the distribution of the input points are usually disregarded. To avoid the above shortcomings, an adaptive 3D point cloud neighbourhood recovery method is proposed, which fully considers the correlation between point clouds and the surface variation of local neighbourhoods in the neighbourhood recovery process, and recovers dynamically sized optimal neighbourhoods for point clouds. The performance of our proposed method is evaluated on the Oakland dataset and the Vaihingen dataset with various modern classifiers. Experiments demonstrate that our proposed method achieves a superior classification result.

Adversarial Attacks and Defenses on 3D Point Cloud Classification: A Survey

Adversarial Attacks and Defenses on 3D Point Cloud Classification: A Survey

Dual Transformer for Point Cloud Analysis

Feature representation learning is a key component in 3D point cloud analysis. However, the powerful convolutional neural networks (CNNs) cannot be applied due to the irregular structure of point clouds. Therefore, following the tremendous success of transformer in natural language processing and image understanding tasks, in this paper, we present a novel point cloud representation learning architecture, named Dual Transformer Network (DTNet), which mainly consists of Dual Point Cloud Transformer (DPCT) module. Specifically, by aggregating the well-designed point-wise and channel-wise self-attention models simultaneously, DPCT module can capture much richer contextual dependencies semantically from the perspective of position and channel. With the DPCT model as a fundamental component, we construct the DTNet for performing 3D point cloud analysis in an end-to-end manner. Extensive quantitative and qualitative experiments on publicly available benchmarks demonstrate the effectiveness of our transformer framework for the tasks of 3D point cloud classification, segmentation and visual object affordance understanding, achieving highly competitive performance in comparison with the state-of-the-art approaches. The source code and trained models are available at <uri>https://github.com/hxf-666/DT</uri>.

Uni3DA: Universal 3D Domain Adaptation for Object Recognition

Traditional 3D point cloud classification tasks focus on training a classifier in the closed-set scenario, where training and test data have the same label set and the same data distribution. In this work, we focus on a more challenging and realistic scenario in 3D point cloud classification task: universal domain adaptation (UniDA), where 1) data distributions for training and test data are different; and 2) for given label sets of training data and test data, they may have the shared classes and keep the private classes respectively, introducing an extra label set discrepancy. To solve UniDA problem, researchers have designed many methods based on 2D image datasets. However, due to the difficulty in capturing discriminative local geometric structures brought by the unordered and irregular 3D point cloud data, we cannot directly deploy the existing methods based on 2D image datasets to the 3D scenarios. To address UniDA in 3D scenarios, we develop a 3D universal domain adaptation framework, which consists of three modules: Self-Constructed Geometric (SCG) module, Local-to-Global Hypersphere Reasoning (LGHR) module and Self-Supervised Boundary Adaptation (SBA) module. SCG and LGHR generate the discriminative representation, which is used to acquire domain-invariant knowledge for training and test data. SBA is designed to automatically recognize whether a given label is from the shared label set or private label set, and adapts training and test data from the shared label set. To our best knowledge, this work is the first exploration of UniDA for 3D scenarios. Extensive experiments on public 3D point cloud datasets verify that the proposed method outperforms the existing UniDA methods.

Interpreting Hidden Semantics in the Intermediate Layers of 3D Point Cloud Classification Neural Network

Interpreting Hidden Semantics in the Intermediate Layers of 3D Point Cloud Classification Neural Network