Irregular Point Research Articles

Extracting geometric and semantic point cloud features with gateway attention for accurate 3D object detection

3D object detection using point clouds has received a lot of attention in autonomous vehicles, robotics, and virtual reality. However, feature learning for 3D object detection from point cloud is very challenging due to the irregularity and sparsity of 3D point cloud data. Grid-based methods convert irregular point clouds into regular 2D views or 3D voxels and then use 2D CNN or 3D CNN for feature learning, but the point cloud transformation process will inevitably cause quantization loss. Point-based methods use the PointNet network to directly learn the features of the point cloud, but the semantic information obtained by PointNet may be incomplete. To address the above issues, we propose a novel Gateway Attention-based Point Set Abstraction 3D object detector (GAPSA) to learn geometric and semantic point cloud features. Specifically, the framework utilizes set abstraction downsampling points and performs local feature extraction on the sampling points through the proposed gateway attention pooling module to learn more discriminative point cloud features. Given the high-quality 3D proposals generated by attention-based backbone networks, we design a RoI multi-pooling head to adaptively learn features for sparse points of interest within proposals, so as to encode richer contextual information and obtain fine-grained features to accurately estimate object confidence and location. Compared with advanced point-based 3D object detectors, experimental results demonstrate that our attention-based point set abstraction 3D object detector has the best detection performance on KITTI and NuScenes datasets. The code is available at https://github.com/liuhuaijjin/GAPSA.

A Regularized Projection-Based Geometry Compression Scheme for LiDAR Point Cloud

Due to the ability to depict large-scale 3D scenes, point clouds acquired by the Light Detection And Ranging (LiDAR) devices have played an indispensable role in various fields. The growing data amount of point cloud, however, brings huge challenges to existing point cloud processing networks. Developing point cloud compression algorithms has become an active research area in recent years. Representative compression frameworks include the MPEG Geometry-based Point Cloud Compression (G-PCC) standard in which a dedicated profile is designed for spinning LiDAR point clouds. In that design, prior knowledge of the LiDAR device is used to project points to nodes in a predictive structure which better reflects the spatial correlation of LiDAR point clouds. In this paper, an analysis has been conducted to explain the observed irregular point distribution in the predictive structure. A regularized projection algorithm is then proposed to construct a reliable prediction relationship in the predictive structure. Simplified geometry prediction techniques are further proposed based on the regularized projection pattern. Experimental results show that an average BD-rate gain of 18% can be achieved with lower encoding runtime if compared with MPEG G-PCC.

Sensitivity and historic behavior for continuous maps on Baire metric spaces

AbstractWe introduce a notion of sensitivity with respect to a continuous real-valued bounded map which provides a sufficient condition for a continuous transformation, acting on a Baire metric space, to exhibit a Baire generic subset of points with historic behavior (also known as irregular points). The applications of this criterion recover, and extend, several known theorems on the genericity of the irregular set, in addition to yielding a number of new results, including information on the irregular set of geodesic flows, in both negative and non-positive curvature, and semigroup actions.

A FFT accelerated fourth order finite difference method for solving three-dimensional elliptic interface problems

In this paper, a fourth order augmented matched interface and boundary (AMIB) method is proposed for solving a three-dimensional elliptic interface problem which involves a smooth material interface inside a cuboid domain. On the boundary of the cuboid domain, the fourth order AMIB method can handle different types of boundary conditions, including Dirichlet, Neumann, Robin and their mix combinations in fictitious value generation. Moreover, zero-padding solutions are introduced so that the fast Fourier transform (FFT) algorithm is still valid near the boundary. In dealing with the interior interface, a fourth order ray-casting matched interface and boundary (MIB) scheme is proposed, which enforces the jump conditions along the normal direction for calculating fictitious values. Comparing with the existing MIB scheme, the ray-casting scheme naturally bypasses the corner issue and becomes more robust in handling complex geometry. Based on fictitious values generated near interface and boundary, the fourth order central difference can be corrected at various irregular points including corner points, by introducing Cartesian derivative jumps as auxiliary variables. This gives rise to an enlarged linear system, which can be efficiently solved by the Schur complement procedure together with the FFT inversion of the discrete Laplacian. Extensive numerical experiments have been carried to test the proposed ray-casting AMIB method for numerical accuracy, efficiency, and robustness in corner treatment. The numerical results demonstrate that the ray-casting AMIB scheme not only maintains a fourth order of accuracy in treating various interfaces and boundaries for both solutions and solution gradients, but also attains an overall efficiency on the order of O(n3log⁡n) for a n×n×n uniform grid.

Improved stencil selection for meshless finite difference methods in 3D

We introduce a geometric stencil selection algorithm for Laplacian in 3D that significantly improves octant-based selection considered earlier. The goal of the algorithm is to choose a small subset from a set of irregular points surrounding a given point that admits an accurate numerical differentiation formula. The subset serves as an influence set for the numerical approximation of the Laplacian in meshless finite difference methods using either polynomial or kernel-based techniques. Numerical experiments demonstrate a competitive performance of this method in comparison to the finite element method and to other selection methods for solving the Dirichlet problems for the Poisson equation on several STL models. Discretization nodes for these domains are obtained either by 3D triangulations or from Cartesian grids or Halton quasi-random sequences.

Uni3DA: Universal 3D Domain Adaptation for Object Recognition

Traditional 3D point cloud classification tasks focus on training a classifier in the closed-set scenario, where training and test data have the same label set and the same data distribution. In this work, we focus on a more challenging and realistic scenario in 3D point cloud classification task: universal domain adaptation (UniDA), where 1) data distributions for training and test data are different; and 2) for given label sets of training data and test data, they may have the shared classes and keep the private classes respectively, introducing an extra label set discrepancy. To solve UniDA problem, researchers have designed many methods based on 2D image datasets. However, due to the difficulty in capturing discriminative local geometric structures brought by the unordered and irregular 3D point cloud data, we cannot directly deploy the existing methods based on 2D image datasets to the 3D scenarios. To address UniDA in 3D scenarios, we develop a 3D universal domain adaptation framework, which consists of three modules: Self-Constructed Geometric (SCG) module, Local-to-Global Hypersphere Reasoning (LGHR) module and Self-Supervised Boundary Adaptation (SBA) module. SCG and LGHR generate the discriminative representation, which is used to acquire domain-invariant knowledge for training and test data. SBA is designed to automatically recognize whether a given label is from the shared label set or private label set, and adapts training and test data from the shared label set. To our best knowledge, this work is the first exploration of UniDA for 3D scenarios. Extensive experiments on public 3D point cloud datasets verify that the proposed method outperforms the existing UniDA methods.

Robust Ellipse Fitting Based on Maximum Correntropy Criterion With Variable Center

The presence of radically irregular data points (RIDPs), which are referred to as the subset of measurements that represents no or little information, can significantly degrade the performance of ellipse fitting methods. We develop an ellipse fitting method that is robust to RIDPs based on the maximum correntropy criterion with variable center (MCC-VC), where an adaptable Laplacian kernel is used. For single ellipse fitting, we formulate a non-convex optimization problem and divide it into two subproblems, one to estimate the kernel bandwidth and the other the kernel center. We design sufficiently accurate convex approximation to each subproblem that will lead to computationally efficient closed-form solutions. The two subproblems are solved in an alternate manner until convergence is reached. We also investigate coupled ellipses fitting. While there exist multiple ellipses fitting methods in the literature, we develop a coupled ellipses fitting method by exploiting the underlying special structure, where the associations between the data points and ellipses are absent in the problem. The proposed method first introduces an association vector for each data point and then formulates a non-convex mixed-integer optimization problem to establish the data associations, which is approximately solved by relaxing it into a second-order cone program. Using the estimated data associations, we then extend the proposed single ellipse fitting method to accomplish the final coupled ellipses fitting. The proposed method is shown to perform significantly better than the existing methods using both simulated data and real images.

An Energy-Efficient Stream-Based FPGA Implementation of Feature Extraction Algorithm for LiDAR Point Clouds With Effective Local-Search

Feature extraction is a fundamental and essential step in light detection and ranging (LiDAR) based simultaneously localization and mapping (SLAM) algorithms. Considering the run-time requirement of feature extraction and the stringent battery constraint in smart vehicles, it is a great challenge to develop fast and highly energy-efficient feature extraction implementation for massive point clouds. Unfortunately, existing implementations not only fail to exploit the available parallelism but also fail to make full use of the local information to optimize the computations. To solve the issue, we propose three novel techniques to achieve a fast and energy-efficient FPGA implementation of the feature extraction algorithm with effective local search. First, we propose a low-complexity projection method and a column-scanning scheduler to organize the irregular and sparse point cloud into a well-organized point cloud matrix. Second, based on the point cloud matrix, we exploit its local information and propose a high-parallel method to detect the coarse-grain feature points. Third, we propose a high-parallel conditional priority queue to progressively and evenly select the fine-grain feature points. Experimental results on the KITTI dataset show that our method implemented on the ZCU104 FPGA board achieves the best accuracy and reaches 584 frames per second (FPS) for the feature extraction of a 64-laser LiDAR’s point cloud. Moreover, our proposal achieves the best energy efficiency, which is on average 11.7 times and 9.0 times higher than the state-of-the-art implementations on the GPU and FPGA platforms, respectively.

Masked Autoencoders for 3D Point Cloud Self-supervised Learning

Self-supervised learning methods are able to learn latent features from unlabeled data samples by designing pre-text tasks, and hence have attracted a great deal of interest in terms of sample-efficient learning. As a promising scheme of self-supervised learning, masked autoencoding has significantly advanced natural language processing and computer vision, but has not been introduced to point cloud yet. In this paper, we propose a novel scheme of masked autoencoders for 3D point cloud self-supervised learning, addressing the special challenges posed by point cloud, including leakage of location information and uneven information density. Concretely, we divide the input point cloud into irregular point patches and randomly mask them at a high ratio. Then, a standard Transformer-based autoencoder, with an asymmetric design and a shifting mask tokens operation, learns high-level latent features from unmasked point patches, aiming to reconstruct the masked point patches. Extensive experiments show that our approach is efficient during pre-training and generalizes well on various downstream tasks. Apart from the proposed method, we will also introduce potential directions for 3D point cloud self-supervised learning, including improvements for masked autoencoding, developments for point cloud scene understanding, etc.

On some new boundary properties of normal harmonic functions

In this paper, some boundary properties of normal harmonic functions defined in the unit circle D are investigated. Similar results for meromorphic and holomorphic functions defined in D were considered in the works of Professor V. I. Gavrilov and other authors. Next, we consider the question of the existence of irregular points for harmonic functions

Progressive generation of 3D point clouds with hierarchical consistency

Generating 3D point cloud directly from latent prior (e.g., Gaussian distribution) plays a vital role in the representation learning and data augmentation in 3D vision tasks. Since point cloud is formed by irregular points, the generation process of point cloud requires rich semantic information, yet few studies are devoted to it. In this paper, we recast this generation task as a progressive learning problem to model the two-level hierarchy of distributions and address it by proposing a novel model called hierarchical consistency variational autoencoder (HC-VAE). This framework introduces a hierarchical consistent mechanism (HCM) to model the shape consistency and the pointwise representation consistency in a complementary manner. Specifically, we propose a stackable encoder-decoder framework and constrain the generation quality progressively to ensure that the underlying shape and fine-grained parts can be reconstructed with high fidelity. Additionally, given the progressively generated intermediate point cloud instances, a hierarchical-positive contrastive loss is introduced to learn the point-distribution-free instance representations to avoid explicitly parametrizing the distribution of points in a shape. In this way, our model suffices to generate diverse, high-resolution, and uniform point cloud instances. Extensive experimental results demonstrate that the proposed method achieves state-of-the-art performance in point cloud generation.

Solving Biharmonic Equations with Tri-Cubic C1 Splines on Unstructured Hex Meshes

Unstructured hex meshes are partitions of three spaces into boxes that can include irregular edges, where n≠4 boxes meet along an edge, and irregular points, where the box arrangement is not consistent with a tensor-product grid. A new class of tri-cubic C1 splines is evaluated as a tool for solving elliptic higher-order partial differential equations over unstructured hex meshes. Convergence rates for four levels of refinement are computed for an implementation of the isogeometric Galerkin approach applied to Poisson’s equation and the biharmonic equation. The ratios of error are contrasted and superior to an implementation of Catmull-Clark solids. For the trivariate Poisson problem on irregularly partitioned domains, the reduction by 24 in the L2 norm is consistent with the optimal convergence on a regular grid, whereas the convergence rate for Catmull-Clark solids is measured as O(h3). The tri-cubic splines in the isogeometric framework correctly solve the trivariate biharmonic equation, but the convergence rate in the irregular case is lower than O(h4). An optimal reduction of 24 is observed when the functions on the C1 geometry are relaxed to be C0.

Local multifractality in urban systems—the case study of housing prices in the greater Paris region

Even though the study of fractal and multifractal properties has now become an established approach for statistical urban data analysis, the accurate multifractal characterisation of smaller, district-scale spatial units is still a somewhat challenging task. The latter issue is key for understanding complex spatial correlations within urban regions while the methodological challenge can be mainly attributed to inhomogeneous data availability over their territories. We demonstrate how the approach proposed here for the multifractal analysis of irregular marked point processes is able to estimate local self-similarity and intermittency exponents in a satisfactory manner via combining methods from classical multifractal and geographical analysis. With the aim of emphasizing general applicability, we first introduce the procedure on synthetic data using a multifractal random field as mark superposed on two distinct spatial distributions. We go on to illustrate the methodology on the example of home prices in the greater Paris region, France. In the context of complex urban systems, our findings proclaim the need for separately tackling processes on the geolocation (support) and any attached value (mark, e.g. home prices) of geospatial data points in an attempt to fully describe the phenomenon under observation. In particular, the results are indicators of the strength of global and local spatial dependency in the housing price structure and how these build distinct layered patterns within and outside of the municipal boundary. The derived properties are of potential urban policy and strategic planning relevance for the timely identification of local vulnerabilities while they are also intended to be combinable with existing price indices in the regional economics context.

A Goodness-of-Identifiability Criterion for Parametric Statistical Models

Abstract This note sets out a goodness-of-identifiability criterion. This criterion quantifies the identifying strength of a parametric statistical model. Unlike the qualitative criterion for identifiability based only on the Fisher matrix, it applies to both regular and irregular points of the Fisher matrix. Unlike the qualitative criterion based only on the Hellinger distance, it quantifies set-identification. JEL classification numbers: C10, C50. Keywords: Parametric statistical model, Identifiability, Fisher matrix, Hellinger distance.

Spatial Gap-Filling of GK2A Daily Sea Surface Temperature (SST) around the Korean Peninsula Using Meteorological Data and Regression Residual Kriging (RRK)

Satellite remote sensing can measure large ocean surface areas, but the infrared-based sea surface temperature (SST) might not be correctly calculated for the pixels under clouds, resulting in missing values in satellite images. Early studies for the gap-free raster maps of satellite SST were based on spatial interpolation using in situ measurements. In this paper, however, an alternative spatial gap-filling method using regression residual kriging (RRK) for the Geostationary Korea Multi-Purpose Satellite-2A (GK2A) daily SST was examined for the seas around the Korean Peninsula. Extreme outliers were first removed from the in situ measurements and the GK2A daily SST images using multi-step statistical procedures. For the pixels on the in situ measurements after the quality control, a multiple linear regression (MLR) model was built using the selected meteorological variables such as daily SST climatology value, specific humidity, and maximum wind speed. The irregular point residuals from the MLR model were transformed into a residual grid by optimized kriging for the residual compensation for the MLR estimation of the null pixels. The RRK residual compensation method improved accuracy considerably compared with the in situ measurements. The gap-filled 18,876 pixels showed the mean bias error (MBE) of −0.001 °C, the mean absolute error (MAE) of 0.315 °C, the root mean square error (RMSE) of 0.550 °C, and the correlation coefficient (CC) of 0.994. The case studies made sure that the gap-filled SST with RRK had very similar values to the in situ measurements to those of the MLR-only method. This was more apparent in the typhoon case: our RRK result was also stable under the influence of typhoons because it can cope with the abrupt changes in marine meteorology.

Deep Learning for LiDAR Point Cloud Classification in Remote Sensing.

Point clouds are one of the most widely used data formats produced by depth sensors. There is a lot of research into feature extraction from unordered and irregular point cloud data. Deep learning in computer vision achieves great performance for data classification and segmentation of 3D data points as point clouds. Various research has been conducted on point clouds and remote sensing tasks using deep learning (DL) methods. However, there is a research gap in providing a road map of existing work, including limitations and challenges. This paper focuses on introducing the state-of-the-art DL models, categorized by the structure of the data they consume. The models’ performance is collected, and results are provided for benchmarking on the most used datasets. Additionally, we summarize the current benchmark 3D datasets publicly available for DL training and testing. In our comparative study, we can conclude that convolutional neural networks (CNNs) achieve the best performance in various remote-sensing applications while being light-weighted models, namely Dynamic Graph CNN (DGCNN) and ConvPoint.

An efficient explicit jump high-order compact immersed interface approach for transient incompressible viscous flows

In the present work, we propose a novel hybrid explicit jump immersed interface approach in conjunction with a higher-order compact scheme for simulating transient complex flows governed by the streamfunction-vorticity (ψ-ζ) formulation of the Navier–Stokes (N–S) equations for incompressible viscous flows. A new strategy has been adopted for the jump conditions at the irregular points across the interface using Lagrangian interpolation on a Cartesian grid. This approach, which starts with the discretization of parabolic equations with discontinuities in the solutions, source terms, and the coefficients across the interface, can easily be accommodated into simulating flow past bluff bodies immersed in the flow. The superiority of the approach is reflected by the reduced magnitude and faster decay of the errors in comparison to other existing methods. It is seen to handle several fluid flow problems having practical implications in the real world very efficiently, including flows involving multiple and moving bodies. This includes the flow past a stationary circular and a twenty-four edge cactus cylinder, flows past two tandem cylinders, where in one situation both are fixed and in another, one of them is oscillating transversely with variable amplitude in time. To the best of our knowledge, the last two examples have been tackled for the first time by such an approach employing the ψ-ζ formulation in finite difference set-up. The extreme closeness of our computed solutions with the existing numerical and experimental results exemplifies the accuracy and the robustness of the proposed approach.

Sparse 3D Point Cloud Parallel Multi-Scale Feature Extraction and Dense Reconstruction with Multi-Headed Attentional Upsampling

Three-dimensional (3D) point clouds have a wide range of applications in the field of 3D vision. The quality of the acquired point cloud data considerably impacts the subsequent work of point cloud processing. Due to the sparsity and irregularity of point cloud data, processing point cloud data has always been challenging. However, existing deep learning-based point cloud dense reconstruction methods suffer from excessive smoothing of reconstruction results and too many outliers. The reason for this is that it is not possible to extract features for local and global features at different scales and provide different levels of attention to different regions in order to obtain long-distance dependence for dense reconstruction. In this paper, we use a parallel multi-scale feature extraction module based on graph convolution and an upsampling method with an added multi-head attention mechanism to process sparse and irregular point cloud data to obtain extended point clouds. Specifically, a point cloud training patch with 256 points is inputted. The PMS module uses three residual connections in the multi-scale feature extraction stage. Each PMS module consists of three parallel DenseGCN modules with different size convolution kernels and different averaging pooling sizes. The local and global feature information of the augmented receptive field is extracted efficiently. The scale information is obtained by averaging the different pooled augmented receptive fields. The scale information was obtained using the different average pooled augmented receptive fields. The upsampling stage uses an upsampling rate of r=4, The self-attentive features with a different focus on different point cloud data regions obtained by fusing different weights make the feature representation more diverse. This operation avoids the bias of one attention, and each focuses on extracting valuable fine-grained feature information. Finally, the coordinate reconstruction module obtains 1024 dense point cloud data. Experiments show that the proposed method demonstrates good evaluation metrics and performance and is able to obtain better visual quality. The problems of over-smoothing and excessive outliers are effectively mitigated, and the obtained sparse point cloud is more dense.

DSACNN: Dynamically local self-attention CNN for 3D point cloud analysis

The point cloud is a common 3D representation widely applied in CAX engineering due to its simple data representation and rich semantic information. However, discrete and unordered 3D data structures make it difficult for point clouds to understand semantic information and make them unsuitable for applying standard operators. In this paper, to enhance machine perception of 3D semantic information, we propose a novel approach that can not only directly process point cloud data by a novel convolution-like operator but also dynamically pay attention to local semantic information. First, we design a novel dynamic local self-attention mechanism that can dynamically and flexibly focus on top-level information of the receptive field to learn and understand subtle features. Second, we propose a dynamic self-attention learning block, which adopts the proposed dynamic local self-attention learning convolution operation to directly deal with disordered and irregular point clouds to learn global and local point features while dynamically learning the important local semantic information. Third, the proposed operation can be compatibly applied as an independent component in popular architectures to improve the perception of local semantic information. Numerous experiments demonstrate the advantage of our method for point cloud tasks on datasets from both CAD data and scan data of complex real-world scenes.

이기종 CAD 시스템에서 설계된 엘보우 파이프 모델의 통합을 위한 부분 토러스 재건

For constructing various geometric shapes of ship components, computer-aided design (CAD) models need to be integrated into shipbuilding CAD systems. As package models provided by CAD solution vendors are designed using mechanical CAD systems, complex triangular meshes have a serious impact on the modeling performance. In particular, reconstructing the elbow pipe represented by triangular meshes in a shipbuilding model is a difficult problem. We present a method to reconstruct elbow primitives for the Plant Design Management System (PDMS) from triangular meshes using mesh segmentation and elbow parameter extraction methods. The mesh segmentation method divides the mesh of irregular and minimal 3D points into segments for elbow parameter extraction based on curvature and face normal vector. The proposed method could also enable to extract parameters for the elbow pipe by using the geometrical features of the partial torus. As a result of the experiment, the average reconstruction difference is 2.278㎜, which showed that the proposed method can be applied to integration of elbow pipe models designed in heterogeneous cad systems.