Incomplete Shapes Research Articles

Learning-Based Intrinsic Reflectional Symmetry Detection.

Reflectional symmetry is a ubiquitous pattern in nature. Previous works usually solve this problem by voting or sampling, suffering from high computational cost and randomness. In this article, we propose a learning-based approach to intrinsic reflectional symmetry detection. Instead of directly finding symmetric point pairs, we parametrize this self-isometry using a functional map matrix, which can be easily computed given the signs of Laplacian eigenfunctions under the symmetric mapping. Therefore, we manually label the eigenfunction signs for a variety of shapes and train a novel neural network to predict the sign of each eigenfunction under symmetry. Our network aims at learning the global property of functions and consequently converts the problem defined on the manifold to the functional domain. By disentangling the prediction of the matrix into separated bases, our method generalizes well to new shapes and is invariant under perturbation of eigenfunctions. Through extensive experiments, we demonstrate the robustness of our method in challenging cases, including different topology and incomplete shapes with holes. By avoiding random sampling, our learning-based algorithm is over 20 times faster than state-of-the-art methods, and meanwhile, is more robust, achieving higher correspondence accuracy in commonly used metrics.

KT-Net: Knowledge Transfer for Unpaired 3D Shape Completion

Unpaired 3D object completion aims to predict a complete 3D shape from an incomplete input without knowing the correspondence between the complete and incomplete shapes. In this paper, we propose the novel KTNet to solve this task from the new perspective of knowledge transfer. KTNet elaborates a teacher-assistant-student network to establish multiple knowledge transfer processes. Specifically, the teacher network takes complete shape as input and learns the knowledge of complete shape. The student network takes the incomplete one as input and restores the corresponding complete shape. And the assistant modules not only help to transfer the knowledge of complete shape from the teacher to the student, but also judge the learning effect of the student network. As a result, KTNet makes use of a more comprehensive understanding to establish the geometric correspondence between complete and incomplete shapes in a perspective of knowledge transfer, which enables more detailed geometric inference for generating high-quality complete shapes. We conduct comprehensive experiments on several datasets, and the results show that our method outperforms previous methods of unpaired point cloud completion by a large margin. Code is available at https://github.com/a4152684/KT-Net.

Robust Symmetry Prediction with Multi-Modal Feature Fusion for Partial Shapes

In geometry processing, symmetry research benefits from global geometric features of complete shapes, but the shape of an object captured in real-world applications is often incomplete due to the limited sensor resolution, single viewpoint, and occlusion. Different from the existing works predicting symmetry from the complete shape, we propose a learning approach for symmetry prediction based on a single RGB-D image. Instead of directly predicting the symmetry from incomplete shapes, our method consists of two modules, i.e., the multi-modal feature fusion module and the detection-by-reconstruction module. Firstly, we build a channel-transformer network (CTN) to extract cross-fusion features from the RGB-D as the multi-modal feature fusion module, which helps us aggregate features from the color and the depth separately. Then, our self-reconstruction network based on a 3D variational auto-encoder (3D-VAE) takes the global geometric features as input, followed by a prediction symmetry network to detect the symmetry. Our experiments are conducted on three public datasets: ShapeNet, YCB, and ScanNet, we demonstrate that our method can produce reliable and accurate results.

Deformable Pyramid R-CNN for 3D object detection (ChinaMM2022)

3D object detector performance has gradually improved, but it still faces huge challenges for accurate detection from point clouds due to incomplete shapes and sparse point clouds. To solve this problem, recent two-stage approaches usually divide the region of interest (RoI) into regular grids and aggregate the internal features of the RoI within a certain radius for refinement proposals. However, for objects with sparse points, only the features inside the RoI cannot satisfy the need for accurate detection. In view of this objective fact, we present a new two-stage 3D object detection framework from point clouds to 3D voxels inside and outside RoI, named Deformable Pyramid R-CNN. Specially, we firstly propose the Voxel Feature Pyramid, which dynamically selects multi-layers of 3D features based on the sparsity of non-empty voxels within the RoI for proposal refinement. Secondly, a new Deformable Voxel RoI Pooling is introduced to abstract rich context information from the voxels of interest outside RoI, to better align semantic information by deformation grid points set with multiple receptive fields for accurate detection. Our method achieves progressive performance in both KITTI Dataset and Waymo Open Dataset. Especially on KITTI Dataset, our method outperforms PV-RCNN by 0.47%, 1.63%, 1.34% on val set easy, moderate and hard levels for car detection, respectively. Code will be available at https://github.com/june2678/DP-RCNN.

SELF-SUPERVISED ADVERSARIAL SHAPE COMPLETION

Abstract. The goal of this paper is 3D shape completion: given an incomplete instance of a known category, hallucinate a complete version of it that is geometrically plausible. We develop an adversarial framework that makes it possible to learn shape completion in a self-supervised fashion, only from incomplete examples. This is enabled by a discriminator network that rejects incomplete shapes, via a loss function that separately assesses local sub-regions of the generated example and accepts only regions with sufficiently high point count. This inductive bias against empty regions forces the generator to output complete shapes. We demonstrate the effectiveness of this approach on synthetic data from ShapeNet and ModelNet, and on a real mobile mapping dataset with nearly 9’000 incomplete cars. Moreover, we apply it to the KITTI autonomous driving dataset without retraining, to highlight its ability to generalise to different data characteristics.

Effect of rolling parameters on forming quality of flat cross wedge rolling thread shafts

The flat cross wedge rolling is an advanced forming process, which is used to produce thread shafts. In this work, a finite element model of flat cross wedge rolling thread shafts was established by the DEFORM-3D, and the effect of the forming angle, spreading angle, helix angle and flank angle on forming quality are investigated. The research results show that the forming angle, spreading angle, helix angle have a significant influence on the position of the die wedge-in point. The angle designed wrongly makes the die wedge-in point unable to be aligned correctly, causing defects such as slipping in the process, repeated rolling of formed thread, thin or incomplete tooth shapes. The flank angle mainly makes an impact on the axial force. Reducing the right flank angle or increasing the left flank angle can effectively improve the imbalance of axial force to obtain high-quality thread shafts. Through the experiment of flat cross wedge rolling thread shafts, the data comparison demonstrates that the error of the external dimension of formed parts is small, proving the reliability of the rolling parameter selection.

Spherical Harmonic Energy Over Gaussian Sphere for Incomplete 3D Shape Retrieval

Spherical harmonic (SH) has shown excellent advantages in terms of accuracy and efficiency in the retrieval of complete three-dimensional shapes. However, since the spherical function directly takes the shape centroid as the global reference point, the SH features depend heavily on the central position. In this context, the features become no longer reliable in the query of incomplete shapes that may cause erratic centroid. In this work, we propose a novel shape descriptor, namely spherical harmonic energy over the Gaussian sphere (SHE-GS), especially for the incomplete shape retrieval. Firstly, all unit normal vectors on the shape surface are mapped to points on a Gaussian sphere, which has the constant center. Secondly, kernel density estimation is used to establish a Gaussian Sphere Model (GSM) to describe the density change of these mapping points. Finally, the shape descriptor is generated by applying an SH transformation on the model. According to the way of GSM being regarded as a surface model or a volume model, we have separately designed two specific algorithm implementations. Experimental results, on two engineering shape sets containing artificially defective shapes, indicate that the proposed method outperforms other traditional methods also defined in the sphere space for incomplete shape retrieval. The superiority is verified for both similar objects in the same category or the single specific object of query.

A novel character segmentation-reconstruction approach for license plate recognition

Developing an automatic license plate recognition system that can cope with multiple factors is challenging and interesting in the current scenario. In this paper, we introduce a new concept called partial character reconstruction to segment characters of license plates to enhance the performance of license plate recognition systems. Partial character reconstruction is proposed based on the characteristics of stroke width in the Laplacian and gradient domain in a novel way. This results in character components with incomplete shapes. The angular information of character components determined by PCA and the major axis are then studied by considering regular spacing between characters and aspect ratios of character components in a new way for segmenting characters. Next, the same stroke width properties are used for reconstructing the complete shape of each character in the gray domain rather than in the gradient domain, which helps in improving the recognition rate. Experimental results on benchmark license plate databases, namely, MIMOS, Medialab, UCSD data, Uninsbria data Challenged data, as well as video databases, namely, ICDAR 2015, YVT video, and natural scene data, namely, ICDAR 2013, ICDAR 2015, SVT, MSRA, show that the proposed technique is effective and useful.

ALIGNet

The process of aligning a pair of shapes is a fundamental operation in computer graphics. Traditional approaches rely heavily on matching corresponding points or features to guide the alignment, a paradigm that falters when significant shape portions are missing. These techniques generally do not incorporate prior knowledge about expected shape characteristics, which can help compensate for any misleading cues left by inaccuracies exhibited in the input shapes. We present an approach based on a deep neural network, leveraging shape datasets to learn a shape-aware prior for source-to-target alignment that is robust to shape incompleteness. In the absence of ground truth alignments for supervision, we train a network on the task of shape alignment using incomplete shapes generated from full shapes for self-supervision. Our network, called ALIGNet , is trained to warp complete source shapes to incomplete targets, as if the target shapes were complete, thus essentially rendering the alignment partial-shape agnostic . We aim for the network to develop specialized expertise over the common characteristics of the shapes in each dataset, thereby achieving a higher-level understanding of the expected shape space to which a local approach would be oblivious. We constrain ALIGNet through an anisotropic total variation identity regularization to promote piecewise smooth deformation fields, facilitating both partial-shape agnosticism and post-deformation applications. We demonstrate that ALIGNet learns to align geometrically distinct shapes and is able to infer plausible mappings even when the target shape is significantly incomplete. We show that our network learns the common expected characteristics of shape collections without over-fitting or memorization, enabling it to produce plausible deformations on unseen data during test time.

Archetypal shapes based on landmarks and extension to handle missing data

Archetype and archetypoid analysis are extended to shapes. The objective is to find representative shapes. Archetypal shapes are pure (extreme) shapes. We focus on the case where the shape of an object is represented by a configuration matrix of landmarks. As shape space is not a vectorial space, we work in the tangent space, the linearized space about the mean shape. Then, each observation is approximated by a convex combination of actual observations (archetypoids) or archetypes, which are a convex combination of observations in the data set. These tools can contribute to the understanding of shapes, as in the usual multivariate case, since they lie somewhere between clustering and matrix factorization methods. A new simplex visualization tool is also proposed to provide a picture of the archetypal analysis results. We also propose new algorithms for performing archetypal analysis with missing data and its extension to incomplete shapes. A well-known data set is used to illustrate the methodologies developed. The proposed methodology is applied to an apparel design problem in children.

Damage detection based on optimized incomplete mode shape and frequency

For the purpose of structural health monitoring, a damage detection method combined with optimum sensor placement is proposed in this paper. The back sequential sensor placement (BSSP) algorithm is introduced to optimize the sensor locations with the aim of maximizing the 2-norm of information matrix, since the EI method is not suitable for optimum sensor placement based on eigenvector sensitivity analysis. Structural damage detection is carried out based on the respective advantages of mode shape and frequency. The optimized incomplete mode shapes yielded from the optimal sensor locations are used to localize structural damage. After the potential damage elements have been preliminarily identified, an iteration scheme is adopted to estimate the damage extent of the potential damage elements based on the changes in the frequency. The effectiveness of this method is demonstrated using a numerical example of a 31-bar truss structure.

Multimodality liver registration of Open-MR and CT scans.

Multimodality registration of liver CT and MRI scans is challenging due to large initial misalignment, non-uniform MR signal intensity in the liver parenchyma, incomplete liver shapes in Open-MR scans and non-rigid deformations of the organ. An automated method was developed to register liver CT and open-MRI scans. A hybrid registration algorithm was developed which incorporates both rigid and non-rigid methods. First, large misalignment of input CT and Open-MR images was compensated by intensity-based registration. Maximum intensity projections (MIPs) of CT and MR data were registered in 2D, and the corresponding rigid transform parameters were used to align 3D images in axial, coronal and sagittal planes. Use of MIP projections compensates for intensity inhomogeneities inherent in the Open-MR data. A bounding box of MIP images defines an ROI which removes outliers and copes with incomplete MR data. Next, principal components analysis (PCA) was used to align MR and CT data datasets. The corresponding translation and rotation parameters were then used to increase the global registration accuracy. A modified TPS-RPM point-based non-rigid algorithm was used to accommodate local liver deformations. Surface points on the liver and branching points of the portal veins were input as landmarks to TPS-RPM method. Incorporating vascular branching points improves registration since tumors are usually found near vessels, so greater weight was given to branching points compared with surface points. The automated registration algorithm was compared with both rigid and non-rigid methods. Quantitative evaluation was performed using modified Hausdorff distance and overlap measure. The mean modified Hausdorff distances of liver and tumor were decreased from 23.53 and 40.03 mm to 9.38 and 8.88 mm, respectively. The mean overlap measures of liver and tumor were increased from 39 and 0 % to 78 and 27 %, respectively. Statistical analysis of the outcomes resulted in a p value less than 5 %. MIP-PCA-based rigid multimodality CT-MRI registration of liver scans compensates for large misalignment of input images even when the data are incomplete. A modified TPS-RPM algorithm, in which vascular points are emphasized over surface points, successfully handled local deformations.

Shape Segmentation by Approximate Convexity Analysis

We present a shape segmentation method for complete and incomplete shapes. The key idea is to directly optimize the decomposition based on a characterization of the expected geometry of a part in a shape. Rather than setting the number of parts in advance, we search for the smallest number of parts that admit the geometric characterization of the parts. The segmentation is based on an intermediate-level analysis, where first the shape is decomposed into approximate convex components, which are then merged into consistent parts based on a nonlocal geometric signature. Our method is designed to handle incomplete shapes, represented by point clouds. We show segmentation results on shapes acquired by a range scanner, and an analysis of the robustness of our method to missing regions. Moreover, our method yields results that are comparable to state-of-the-art techniques evaluated on complete shapes.

Eigen-feature analysis of weighted covariance matrices for LiDAR point cloud classification

The features used in the separation of different objects are important for successful point cloud classification. Eigen-features from a covariance matrix of a point set with the sample mean are commonly used geometric features that can describe the local geometric characteristics of a point cloud and indicate whether the local geometry is linear, planar, or spherical. However, eigen-features calculated by the principal component analysis of a covariance matrix are sensitive to LiDAR data with inherent noise and incomplete shapes because of the non-robust statistical analysis. To obtain reliable eigen-features from LiDAR data and to improve classification accuracy, we introduce a method of analyzing local geometric characteristics of a point cloud by using a weighted covariance matrix with a geometric median. Each point is assigned a weight to represent its spatial contribution in the weighted principal component analysis and to estimate the geometric median which can be regarded as a localized center of a shape. In the experiments, qualitative and quantitative analyses on airborne LiDAR data and simulated point clouds show a clear improvement of the proposed method compared with the standard eigen-features. The classification accuracy is improved by 1.6–4.5% using a supervised classifier.

Stratified Generalized Procrustes Analysis

Generalized procrustes analysis computes the best set of transformations that relate matched shape data. In shape analysis the transformations are usually chosen as similarities, while in general statistical data analysis other types of transformation groups such as the affine group may be used. Generalized procrustes analysis has a nonlinear and nonconvex formulation. The classical approach alternates the computation of a so-called reference shape and the computation of transformations relating this reference shape to each shape datum in turn. We propose the stratified approach to generalized procrustes analysis. It first uses the affine transformation group to analyze the data and then upgrades the solution to the sought after group, whether Euclidean or similarity. We derive a convex formulation for each of these two steps, and efficient practical algorithms that gracefully handle missing data (incomplete shapes). Extensive experimental results show that our approaches perform well on simulated and real data. In particular our closed-form solution gives very accurate results for generalized procrustes analysis of Euclidean data.

Damage detection strategy for reticulated structures based on incomplete strain mode

Damage detection based on strain responses of vibration is highly attractive for monitoring long-span reticulated structures. However, there are a lot of structure members in reticulated structures and it is impossible to install strain sensors in each member. Therefore, how to locate and quantify damages with the incomplete mode shapes obtained from few strain sensors is a challenge topic. A new strategy, named incomplete strain mode damage detection (ISMDD) strategy, is proposed in this paper. In the strategy, the distribution of the strain sensors in the reticulated structures can be optimized through sensitive analysis on strain mode perturbation matrix, which can be obtained by perturbation theory. Mode assurance criterion (MAC) value is applied in damage location, and the members with relative large MAC values are defined as damage members. In addition, damage index obtained by solving the perturbation equation is used for damage quantification. Numerical analysis on a long-span reticulated structure, including damage location and quantification for single- and multi-member damages, detection for different damage quantity, the effect analysis of sensor quantity, are performed to verify the effectiveness of the proposed ISMDD strategy. It can be shown from the analysis that the ISMDD strategy is effective in damage location and quantification for both single- and multi-member damages. And the quantity of strain sensors has effect on damage location, but has no obvious influence on damage quantification. Additionally, the anti-noise pollution ability analysis of the ISMDD strategy is carried out, which shows that the ISMDD strategy has excellent anti-noise pollution ability for both single- and multi-damaged members.

Boundary Smoothing via Symmetry Transforms

This paper proposes a smoothing technique for shape based on a perturbation analysis of the underlying medial axis, and by iteratively removing skeletal branches. While medial axis regularization usually aims at dealing with its sensitivity in the context of applications such as recognition, we use it here to smooth shape. The approach overcomes a basic drawback with current smoothing techniques, namely, the rounding of perceptually salient corners. The key idea is that the instability of the skeleton under a family of deformations can be dealt with via a symmetry transform which is applied to all related symmetry branches so as to effect an appropriate change in the shape. This leads to the notion of a splice transform which removes a branch coupled with simultaneous changes in the skeleton. In contrast to point-based pruning techniques which use local salience and which cannot construct coarse-scale corners, our approach constructs and preserves coarse-scale curvature extrema, leading to a “scale-discrete scale-space” for shape. The approach is also applicable to the smoothing of incomplete shapes represented as a set of curve segments.

Growth phenomena in the surface layer and step generation from the crystal edges

A possible mechanism of growth-step generation from the edges of faceted crystals obtained from experimental observations with KDP crystals is described. It suggests that growth from the crystal edges is initiated by the deviation of the edges from their crystallographic orientation and formation of incomplete shapes of singular facets. The conditions for formation of the incomplete faceted shapes during dislocation growth are considered. It is shown that the process of step generation from the edges can be determined by the mutual positions of the vicinal slopes on the adjacent faces.