Shape Correspondence Research Articles

Semantic isometry for 3-D shape correspondence

Isometry has been widely used in corresponding problem with pose variation. However, most existing methods are causing wrong correspondences due to the ambiguity of geodesic distance. This article introduces the semantic isometry to three-dimensional shape correspondence and proposes a new framework called detection–recognition–correspondence. The idea of semantic isometry is to embed the semantic information and statistical learning through sparse correspondence for better performance. Feature point detection is first utilized to extract the salient feature point of the three-dimensional shapes. Then, instead of finding correspondence pairs directly by minimizing the isometric error of the detected feature points, the semantic labels of these feature points are recognized using the support vector machine. The semantic label is used to perform a priority-driven isometric correspondence.The highly reliable corresponding pairs are then obtained to serve as the further constraint in the following corresponding process. During the experiments, the robustness of the proposed algorithm is verified by different kinds of three-dimensional dynamic models, including some very challenging data with pose variation and missing parts. Moreover, the proposed framework can greatly improve the corresponding accuracy over the existing state-of-the-art algorithms.

Group-wise shape correspondence of variable and complex objects

Group-wise shape correspondence of variable and complex objects

Unsupervised Learning of 3-D Local Features From Raw Voxels Based on a Novel Permutation Voxelization Strategy.

Effective 3-D local features are significant elements for 3-D shape analysis. Existing hand-crafted 3-D local descriptors are effective but usually involve intensive human intervention and prior knowledge, which burdens the subsequent processing procedures. An alternative resorts to the unsupervised learning of features from raw 3-D representations via popular deep learning models. However, this alternative suffers from several significant unresolved issues, such as irregular vertex topology, arbitrary mesh resolution, orientation ambiguity on the 3-D surface, and rigid and slightly nonrigid transformation invariance. To tackle these issues, we propose an unsupervised 3-D local feature learning framework based on a novel permutation voxelization strategy to learn high-level and hierarchical 3-D local features from raw 3-D voxels. Specifically, the proposed strategy first applies a novel voxelization which discretizes each 3-D local region with irregular vertex topology and arbitrary mesh resolution into regular voxels, and then, a novel permutation is applied to permute the voxels to simultaneously eliminate the effect of rotation transformation and orientation ambiguity on the surface. Based on the proposed strategy, the permuted voxels can fully encode the geometry and structure of each local region in regular, sparse, and binary vectors. These voxel vectors are highly suitable for the learning of hierarchical common surface patterns by stacked sparse autoencoder with hierarchical abstraction and sparse constraint. Experiments are conducted on three aspects for evaluating the learned local features: 1) global shape retrieval; 2) partial shape retrieval; and 3) shape correspondence. The experimental results show that the learned local features outperform the other state-of-the-art 3-D shape descriptors.

Angular Versus Curved Shapes: Correspondences and Emotional Processing.

The present work aimed to systematically examine sensory and higher level correspondences to angular and curved shapes. Participants matched angular and curved abstract shapes to sensory experiences in five different modalities as well as to emotion, gender, and name attributes presented as written labels (Study 1) and real experiences (Study 2). The results demonstrated nonarbitrary mapping of angular and curved shapes to attributes from all basic sensory modalities (vision, audition, gustation, olfaction, and tactation) and higher level attributes (emotion, gender, and name). Participants associated curved shapes with sweet taste, quiet or calm sound, vanilla smell, green color, smooth texture, relieved emotion, female gender, and wide-vowel names. In contrast, they associated angular shapes with sour taste, loud or dynamic sound, spicy or citrus smell, red color, rough texture, excited or surprise emotion, male gender, and narrow-vowel names. These prevalent correspondences were robust across different shape pairs as well as all sensory and higher level attributes, presented as both verbal labels and real sensory experiences. The second goal of this research was to examine the relationship between the shape correspondences and individual differences in emotional processing, assessed by self-report and performance measures. The results suggest that heightened emotional ability is associated with making shape attributions that go along with the found prevalent trends.

Deformation-driven shape correspondence via shape recognition

Many approaches to shape comparison and recognition start by establishing a shape correspondence. We "turn the table" and show that quality shape correspondences can be obtained by performing many shape recognition tasks. What is more, the method we develop computes a fine-grained, topology-varying part correspondence between two 3D shapes where the core evaluation mechanism only recognizes shapes globally. This is made possible by casting the part correspondence problem in a deformation-driven framework and relying on a data-driven "deformation energy" which rates visual similarity between deformed shapes and models from a shape repository. Our basic premise is that if a correspondence between two chairs (or airplanes, bicycles, etc.) is correct, then a reasonable deformation between the two chairs anchored on the correspondence ought to produce plausible , "chair-like" in-between shapes. Given two 3D shapes belonging to the same category, we perform a top-down, hierarchical search for part correspondences. For a candidate correspondence at each level of the search hierarchy, we deform one input shape into the other, while respecting the correspondence, and rate the correspondence based on how well the resulting deformed shapes resemble other shapes from ShapeNet belonging to the same category as the inputs. The resemblance, i.e., plausibility, is measured by comparing multi-view depth images over category-specific features learned for the various shape categories. We demonstrate clear improvements over state-of-the-art approaches through tests covering extensive sets of man-made models with rich geometric and topological variations.

Counterclockwise maxillomandibular advancement surgery and disc repositioning: can condylar remodeling in the long-term follow-up be predicted?

This study investigated predictive risk factors of condylar remodeling changes after counterclockwise maxillomandibular advancement (CCW-MMA) and disc repositioning surgery. Forty-one female patients (75 condyles) treated with CCW-MMA and disc repositioning had cone beam computed tomography (CBCT) scans taken pre-surgery, immediately after surgery, and at an average 16 months post-surgery. Pre- and post-surgical three-dimensional models were superimposed using automated voxel-based registration on the cranial base to evaluate condylar displacements after surgery. Regional registration was performed to assess condylar remodeling in the follow-up period. Three-dimensional cephalometrics, shape correspondence (SPHARM-PDM), and volume measurements were applied to quantify changes. Pearson product–moment correlations and multiple regression analysis were performed. Highly statistically significant correlation showed that older patients were more susceptible to overall condylar volume reduction following CCW-MMA and disc repositioning (P≤0.001). Weak but statistically significant correlations were observed between condylar remodeling changes in the follow-up period and pre-surgical facial characteristics, magnitude of the surgical procedure, and condylar displacement changes. After CCW-MMA and disc repositioning, the condyles moved mostly downwards and medially, and were rotated medially and counterclockwise; displacements in the opposite direction were correlated with a greater risk of condylar resorption. Moreover, positional changes with surgery were only weakly associated with remodeling in the follow-up period, suggesting that other risk factors may play a role in condylar resorption.

Automatic construction of statistical shape models using deformable simplex meshes with vector field convolution energy

Background In the active shape model framework, principal component analysis (PCA) based statistical shape models (SSMs) are widely employed to incorporate high-level a priori shape knowledge of the structure to be segmented to achieve robustness. A crucial component of building SSMs is to establish shape correspondence between all training shapes, which is a very challenging task, especially in three dimensions.Methods We propose a novel mesh-to-volume registration based shape correspondence establishment method to improve the accuracy and reduce the computational cost. Specifically, we present a greedy algorithm based deformable simplex mesh that uses vector field convolution as the external energy. Furthermore, we develop an automatic shape initialization method by using a Gaussian mixture model based registration algorithm, to derive an initial shape that has high overlap with the object of interest, such that the deformable models can then evolve more locally. We apply the proposed deformable surface model to the application of femur statistical shape model construction to illustrate its accuracy and efficiency.ResultsExtensive experiments on ten femur CT scans show that the quality of the constructed femur shape models via the proposed method is much better than that of the classical spherical harmonics (SPHARM) method. Moreover, the proposed method achieves much higher computational efficiency than the SPHARM method.ConclusionsThe experimental results suggest that our method can be employed for effective statistical shape model construction.

The Discriminative Power of Shape an Empirical Study in Time Series Matching.

Shape provides significant discriminating power in time series matching of visual or geometric data as required in many important applications in graphics and vision. The well established dynamic time warping (DTW) algorithm and its variants do this matching by determining a non-linear time mapping to minimise euclidean distances between corresponding time-warped points. However the shape of curves is not considered. In this paper, we present a new shape-aware algorithm which uses time and shape correspondence (TSC) at increasing levels of detail to define a similarity measure with an norm to aggregate the results, making it robust to noise and missing data. The norm is implicitly regularised using a shape-based error. Through extensive experiments we empirically show that our algorithm outperforms existing state of the art algorithms, works more effectively with high dimensional data, and handles noise and missing data better. We demonstrate its versatile applicability and comparative performance using a large in-house created gait data base, an action data base from Microsoft, exercise action data from a local company, a large public time series data base from University of California, Riverside and hand movement in quaternion stream data format.

Structure-oriented networks of shape collections

We introduce a co-analysis technique designed for correspondence inference within large shape collections. Such collections are naturally rich in variation, adding ambiguity to the notoriously difficult problem of correspondence computation. We leverage the robustness of correspondences between similar shapes to address the difficulties associated with this problem. In our approach, pairs of similar shapes are extracted from the collection, analyzed and matched in an efficient and reliable manner, culminating in the construction of a network of correspondences that connects the entire collection. The correspondence between any pair of shapes then amounts to a simple propagation along the minimax path between the two shapes in the network. At the heart of our approach is the introduction of a robust, structure-oriented shape matching method. Leveraging the idea of projective analysis, we partition 2D projections of a shape to obtain a set of 1D ordered regions, which are both simple and efficient to match. We lift the matched projections back to the 3D domain to obtain a pairwise shape correspondence. The emphasis given to structural compatibility is a central tool in estimating the reliability and completeness of a computed correspondence, uncovering any non-negligible semantic discrepancies that may exist between shapes. These detected differences are a deciding factor in the establishment of a network aiming to capture local similarities. We demonstrate that the combination of the presented observations into a co-analysis method allows us to establish reliable correspondences among shapes within large collections.

Unsupervised 3D Local Feature Learning by Circle Convolutional Restricted Boltzmann Machine.

Extracting local features from 3D shapes is an important and challenging task that usually requires carefully designed 3D shape descriptors. However, these descriptors are hand-crafted and require intensive human intervention with prior knowledge. To tackle this issue, we propose a novel deep learning model, namely circle convolutional restricted Boltzmann machine (CCRBM), for unsupervised 3D local feature learning. CCRBM is specially designed to learn from raw 3D representations. It effectively overcomes obstacles such as irregular vertex topology, orientation ambiguity on the 3D surface, and rigid or slightly non-rigid transformation invariance in the hierarchical learning of 3D data that cannot be resolved by the existing deep learning models. Specifically, by introducing the novel circle convolution, CCRBM holds a novel ring-like multi-layer structure to learn 3D local features in a structure preserving manner. Circle convolution convolves across 3D local regions via rotating a novel circular sector convolution window in a consistent circular direction. In the process of circle convolution, extra points are sampled in each 3D local region and projected onto the tangent plane of the center of the region. In this way, the projection distances in each sector window are employed to constitute a novel local raw 3D representation called projection distance distribution (PDD). In addition, to eliminate the initial location ambiguity of a sector window, the Fourier transform modulus is used to transform the PDD into the Fourier domain, which is then conveyed to CCRBM. Experiments using the learned local features are conducted on three aspects: global shape retrieval, partial shape retrieval, and shape correspondence. The experimental results show that the learned local features outperform other state-of-the-art 3D shape descriptors.

Symmetry aware embedding for shape correspondence

In this paper, we present symmetry-aware embedding for shape correspondence, which is robust against symmetric (left–right) flips and rotational (front–back) flips. Unlike previous embedding approaches that embed surfaces into a high-dimensional space, our technique is based on a low dimensional (3D) embedding. Our method can solve left–right flips by finding a 3D rigid transformation between two embedding surfaces without reflections. Using the global reflectional symmetry plane to align two surfaces, we can further reduce the problem to that of finding the rotation that corrects the signs of the front–back and up–down directions (there are four possible solutions). We exploit this simple problem formulation and alleviate the front–back flips, by explicitly comparing the front and back of embedding surfaces based on the global and local extrinsic shape characteristics. Consequently, reasonably accurate point-to-point correspondences can be established simply by performing the nearest neighbor search in our embedding space. Experimental results based on a shape correspondence benchmark showed that our method produces stable matching results.

Congenital and acquired mandibular asymmetry: Mapping growth and remodeling in 3 dimensions.

Disordered craniofacial development frequently results in definitive facial asymmetries that can significantly impact a person's social and functional well-being. The mandible plays a prominent role in defining facial symmetry and, as an active region of growth, commonly acquires asymmetric features. Additionally, syndromic mandibular asymmetry characterizes craniofacial microsomia (CFM), the second most prevalent congenital craniofacial anomaly (1:3000 to 1:5000 live births) after cleft lip and palate. We hypothesized that asymmetric rates of mandibular growth occur in the context of syndromic and acquired facial asymmetries. To test this hypothesis, a spherical harmonic-based shape correspondence algorithm was applied to quantify and characterize asymmetries in mandibular growth and remodeling in 3 groups during adolescence. Longitudinal time points were automatically registered, and regions of the condyle and posterior ramus were selected for growth quantification. The first group (n=9) had a diagnosis of CFM, limited to Pruzansky-Kaban type I or IIA mandibular deformities. The second group (n=10) consisted of subjects with asymmetric, nonsyndromic dentofacial asymmetry requiring surgical intervention. A control group (n=10) of symmetric patients was selected for comparison. A linear mixed model was used for the statistical comparison of growth asymmetry between the groups. Initial mandibular shape and symmetry displayed distinct signatures in the 3 groups (P<0.001), with the greatest asymmetries in the condyle and ramus. Similarly, mandibular growth had unique patterns in the groups. The dentofacial asymmetry group was characterized by significant asymmetry in condylar and posterior ramal remodeling with growth (P<0.001). The CFM group was characterized by asymmetric growth of the posterior ramus (P<0.001) but relatively symmetric growth of the condyles (P=0.47). Forms of CFM are characterized by active and variable growth of the dysplastic side, which has a distinct pattern from other disorders of mandibular growth.

Mesh Convolutional Restricted Boltzmann Machines for Unsupervised Learning of Features With Structure Preservation on 3-D Meshes.

Discriminative features of 3-D meshes are significant to many 3-D shape analysis tasks. However, handcrafted descriptors and traditional unsupervised 3-D feature learning methods suffer from several significant weaknesses: 1) the extensive human intervention is involved; 2) the local and global structure information of 3-D meshes cannot be preserved, which is in fact an important source of discriminability; 3) the irregular vertex topology and arbitrary resolution of 3-D meshes do not allow the direct application of the popular deep learning models; 4) the orientation is ambiguous on the mesh surface; and 5) the effect of rigid and nonrigid transformations on 3-D meshes cannot be eliminated. As a remedy, we propose a deep learning model with a novel irregular model structure, called mesh convolutional restricted Boltzmann machines (MCRBMs). MCRBM aims to simultaneously learn structure-preserving local and global features from a novel raw representation, local function energy distribution. In addition, multiple MCRBMs can be stacked into a deeper model, called mesh convolutional deep belief networks (MCDBNs). MCDBN employs a novel local structure preserving convolution (LSPC) strategy to convolve the geometry and the local structure learned by the lower MCRBM to the upper MCRBM. LSPC facilitates resolving the challenging issue of the orientation ambiguity on the mesh surface in MCDBN. Experiments using the proposed MCRBM and MCDBN were conducted on three common aspects: global shape retrieval, partial shape retrieval, and shape correspondence. Results show that the features learned by the proposed methods outperform the other state-of-the-art 3-D shape features.

A Multi-Model Approach Using Statistical Index and Information Criteria to Evaluate the Adequacy of the Model Geometry in a Fissured Carbonate Aquifer (Italy)

A conceptual model related to a mountain aquifer that is characterized by a lack of data of hydrogeological parameters and boundary conditions, which were based on a single available observational dataset used for calibration, was studied using numerical models. For the first time, a preliminary spatial-temporal analysis has been applied to the study area in order to evaluate the real extension of the aquifer studied. The analysis was based on four models that were characterized by an increasing degree of complexity using a minimum of two zones and a maximum of five zones, which consequently increased the number of adjustable parameters from a minimum of 10 to a maximum of 22, calibrated using the parameter estimation code PEST. Statistical index and information criteria were calculated for each model, which showed comparable results; the information criteria indicated that the model with the low number of adjustable parameters was the optimal model. A comparison of the simulated and observed spring hydrographs showed a good shape correspondence but a general overestimation of the discharge, which indicated a good fit with the rainfall time series and a probably incorrect extension of the aquifer structure: the recharge contributes more than half of the total outflow at the springs but is not able to completely feed the springs.

Anisotropic Diffusion Descriptors

AbstractSpectral methods have recently gained popularity in many domains of computer graphics and geometry processing, especially shape processing, computation of shape descriptors, distances, and correspondence. Spectral geometric structures are intrinsic and thus invariant to isometric deformations, are efficiently computed, and can be constructed on shapes in different representations. A notable drawback of these constructions, however, is that they areisotropic, i.e., insensitive to direction. In this paper, we show how to construct direction‐sensitive spectral feature descriptors usinganisotropic diffusionon meshes and point clouds. The core of our construction are directed local kernels acting similarly to steerable filters, which are learned in a task‐specific manner. Remarkably, while being intrinsic, our descriptors allow to disambiguate reflection symmetries. We show the application of anisotropic descriptors for problems of shape correspondence on meshes and point clouds, achieving results significantly better than state‐of‐the‐art methods.

Statistical shape analysis of the human spleen geometry for probabilistic occupant models

Statistical shape models are an effective way to create computational models of human organs that can incorporate inter-subject geometrical variation. The main objective of this study was to create statistical mean and boundary models of the human spleen in an occupant posture. Principal component analysis was applied to fifteen human spleens in order to find the statistical modes of variation, mean shape, and boundary models. A landmark sliding approach was utilized to refine the landmarks to obtain a better shape correspondence and create a better representation of the underlying shape contour. The first mode of variation was found to be the overall volume, and it accounted for 69% of the total variation. The mean model and boundary models could be used to develop probabilistic finite element (FE) models which may identify the risk of spleen injury during vehicle collisions and consequently help to improve automobile safety systems.

MaterialCloning: Acquiring Elasticity Parameters from Images for Medical Applications.

We present a practical approach for automatically estimating the material properties of soft bodies from two sets of images, taken before and after deformation. We reconstruct 3D geometry from the given sets of multiple-view images; we use a coupled simulation-optimization-identification framework to deform one soft body at its original, non-deformed state to match the deformed geometry of the same object in its deformed state. For shape correspondence, we use a distance-based error metric to compare the estimated deformation fields against the actual deformation field from the reconstructed geometry. The optimal set of material parameters is thereby determined by minimizing the error metric function. This method can simultaneously recover the elasticity parameters of multiple types of soft bodies using Finite Element Method-based simulation (of either linear or nonlinear materials undergoing large deformation) and particle-swarm optimization methods. We demonstrate this approach on real-time interaction with virtual organs in patient-specific surgical simulation, using parameters acquired from low-resolution medical images. We also highlight the results on physics-based animation of virtual objects using sketches from an artist's conception.

Shape localization, quantification and correspondence using Region Matching Algorithm

We propose a method for local, region-based matching of planar shapes, especially as those shapes that change over time. This is a problem fundamental to medical imaging, specifically the comparison over time of mammograms. The method is based on the non-emergence and non-enhancement of maxima, as well as the causality principle of integral invariant scale space. The core idea of our Region Matching Algorithm (RMA) is to divide a shape into a number of “salient” regions and then to compare all such regions for local similarity in order to quantitatively identify new growths or partial/complete occlusions. The algorithm has several advantages over commonly used methods for shape comparison of segmented regions. First, it provides improved key-point alignment for optimal shape correspondence. Second, it identifies localized changes such as new growths as well as complete/partial occlusion in corresponding regions by dividing the segmented region into sub-regions based upon the extrema that persist over a sufficient range of scales. Third, the algorithm does not depend upon the spatial locations of mammographic features and eliminates the need for registration to identify salient changes over time. Finally, the algorithm is fast to compute and requires no human intervention. We apply the method to temporal pairs of mammograms in order to detect potentially important differences between them.

Deformation-driven topology-varying 3D shape correspondence

We present a deformation-driven approach to topology-varying 3D shape correspondence. In this paradigm, the best correspondence between two shapes is the one that results in a minimal-energy, possibly topology-varying, deformation that transforms one shape to conform to the other while respecting the correspondence. Our deformation model, called GeoTopo transform , allows both geometric and topological operations such as part split, duplication, and merging, leading to fine-grained and piecewise continuous correspondence results. The key ingredient of our correspondence scheme is a deformation energy that penalizes geometric distortion, encourages structure preservation, and simultaneously allows topology changes. This is accomplished by connecting shape parts using structural rods , which behave similarly to virtual springs but simultaneously allow the encoding of energies arising from geometric, structural, and topological shape variations. Driven by the combined deformation energy, an optimal shape correspondence is obtained via a pruned beam search. We demonstrate our deformation-driven correspondence scheme on extensive sets of man-made models with rich geometric and topological variation and compare the results to state-of-the-art approaches.

Three-dimensional regional displacement after surgical-orthodontic correction of Class III malocclusion.

To investigate how displacements of maxillo-mandibular structures are associated with each other at splint removal and 1 year post-surgery following 1-jaw and 2-jaw surgeries for correction of Class III malocclusion. Fifty patients who underwent surgical correction with maxillary advancement only (n = 25) or combined with mandibular setback (n = 25) were prospectively enrolled in this study. Cone-beam computed tomographies were taken pre-surgery, at splint removal and at 1 year post-surgery. Three-dimensional cranial base superimpositions and shape correspondence were used to measure the outcomes from pre-surgery to splint removal (surgical changes) and splint removal to 1 year post-surgery (post-surgical adaptations). Pearson's correlation coefficients were used to evaluate the association between the regional displacements. Both surgery groups presented mandibular clockwise rotation with surgery and post-surgical adaptive counterclockwise rotation. In patients treated with maxillary advancement only, the surgical changes of the maxilla were significantly correlated with chin changes. The amount and direction of chin autorotation were significantly correlated with right and left ramus autorotation. Right and left condylar displacements were significantly correlated. One year post-surgery, adaptive displacements and bone remodeling of both rami were correlated with the chin and condylar changes. For the 2-jaw group, the few correlations between the positional and remodeling changes in the anatomic regions of interest observed due to the surgery were different than those observed after post-surgical adaptations, suggesting that these changes occurred independently. Our results indicate that surgical displacements and post-surgical adaptations are often correlated in one-jaw surgery and are, in general, independent in two-jaw surgery.