Kendall Shape Space Research Articles

A Gaussian kernel for Kendall’s space of [formula omitted]-D shapes

In this paper, we develop an approach to exploit kernel methods with data lying on the m-D Kendall shape space. When data arise in a finite-dimensional curved Riemannian manifold, as in this case, the usual Euclidean computer vision and machine learning algorithms must be treated carefully. A good approach is to use positive definite kernels on manifolds to embed the manifold with its corresponding metric in a high-dimensional reproducing kernel Hilbert space, where it is possible to utilize algorithms developed for linear spaces. Different Gaussian kernels can be found in the literature on the 2-D Kendall shape space to perform this embedding. The main novelty of this work is to provide a Gaussian kernel for the m-D Kendall shape space. This new Kernel coincides in the case m=2 with the Gaussian kernels most widely used in the Kendall planar shape space and allows to define an embedding of the m-D Kendall shape space into a reproducible kernel Hilbert space. As far as we know, the complexity of the m-D Kendall shape space has meant that this embedding has not been addressed in the literature until now. This methodology will be tested on a machine learning problem with a simulated and a real data set.

Shape correspondence for cel animation based on a shape association graph and spectral matching

We present an effective spectral matching method based on a shape association graph for finding region correspondences between two cel animation keyframes. We formulate the correspondence problem as an adapted quadratic assignment problem, which comprehensively considers both the intrinsic geometric and topology of regions to find the globally optimal correspondence. To simultaneously represent the geometric and topological similarities between regions, we propose a shape association graph (SAG), whose node attributes indicate the geometric distance between regions, and whose edge attributes indicate the topological distance between combined region pairs. We convert topological distance to geometric distance between geometric objects with topological features of the pairs, and introduce Kendall shape space to calculate the intrinsic geometric distance. By utilizing the spectral properties of the affinity matrix induced by the SAG, our approach can efficiently extract globally optimal region correspondences, even if shapes have inconsistent topology and severe deformation. It is also robust to shapes undergoing similarity transformations, and compatible with parallel computing techniques.

KSS-ICP: Point Cloud Registration based on Kendall Shape Space.

Point cloud registration is a popular topic that has been widely used in 3D model reconstruction, location, and retrieval. In this paper, we propose a new registration method, KSS-ICP, to address the rigid registration task in Kendall shape space (KSS) with Iterative Closest Point (ICP). The KSS is a quotient space that removes influences of translations, scales, and rotations for shape feature-based analysis. Such influences can be concluded as the similarity transformations that do not change the shape feature. The point cloud representation in KSS is invariant to similarity transformations. We utilize such property to design the KSS-ICP for point cloud registration. To tackle the difficulty to achieve the KSS representation in general, the proposed KSS-ICP formulates a practical solution that does not require complex feature analysis, data training, and optimization. With a simple implementation, KSS-ICP achieves more accurate registration from point clouds. It is robust to similarity transformation, non-uniform density, noise, and defective parts. Experiments show that KSS-ICP has better performance than the state-of-the-art. Code1 and executable files2 are made public.

Geometric Deep Neural Network Using Rigid and Non-Rigid Transformations for Landmark-Based Human Behavior Analysis.

Deep learning architectures, albeit successful in most computer vision tasks, were designed for data with an underlying Euclidean structure, which is not usually fulfilled since pre-processed data may lie on a non-linear space. In this article, we propose a geometric deep learning approach using rigid and non-rigid transformations, named KShapenet, for 2D and 3D landmark-based human motion analysis. Landmark configuration sequences are first modeled as trajectories on Kendall's shape space and then mapped to a linear tangent space. The resulting structured data are then input to a deep learning architecture, which includes a layer that optimizes over rigid and non-rigid transformations of landmark configurations, followed by a CNN-LSTM network. We apply KShapenet to 3D human landmark sequences for action and gait recognition, and 2D facial landmark sequences for expression recognition, and demonstrate the competitiveness of the proposed approach with respect to state-of-the-art.

Geodesics in the TPS Space

In shape analysis, the interpolation of shapes’ trajectories is often performed by means of geodesics in an appropriate Riemannian Shape Space. Over the past several decades, different metrics and shape spaces have been proposed, including Kendall shape space, LDDMM based approaches, and elastic contour, among others. Once a Riemannian space is chosen, geodesics and parallel transports can be used to build splines or piecewise geodesics paths. In a recent paper, we introduced a new Riemannian shape space named TPS Space based on the Thin Plate Spline interpolant and characterized by an appropriate metric and parallel transport rule. In the present paper, we further explore the geometry of the TPS Space by characterizing the properties of its geodesics. Several applications show the capability of the proposed formulation to conserve important physical properties of deformation, such as local strains and global elastic energy.

Nonparametric k-Sample Test on Shape Spaces with Applications to Mitochondrial Shape Analysis

Abstract An important hypothesis in animal cell biology is that an animal’s acute exercise regimen affects some subcellular structures, for example mitochondrial morphology, in its muscle tissue. This paper investigates that hypothesis using a nonparametric metric-based energy test for comparing mitochondrial populations. It explores two shape spaces—the elastic shape space and Kendall shape space—and five corresponding shape metrics on these spaces. The results overwhelmingly point to the statistical significance of the effect of an acute exercise regimen on the shape of SS-type mitochondria. Although past studies based on specific morphological features derived from mitochondria failed to detect this significance. In this analysis, a potentially significant factor is cell membership and a k-sample generalization of the energy test—the DISCO test shows that the cell effect is indeed significant. The energy test cannot be applied directly due to the hierarchical structure of the distance matrix. We propose a compression method to remove the significant cell effect while testing for the exercise effect. With this compression, only the elastic scaled metric shows statistical significance of the exercise factor in this more complicated scenario. This result is because the elastic-scaled metric is more sensitive to subtle changes in mitochondrial shapes caused by acute exercise.

Caricature Expression Extrapolation Based on Kendall Shape Space Theory.

Facial expression editing plays a fundamental role in facial expression generation and has been widely applied in modern film productions and computer games. While the existing 2-D caricature facial expression editing methods are mostly realized by expression interpolation from the original image to the target image, expression extrapolation has rarely been studied before. In this article, we propose a novel expression extrapolation method for caricature facial expressions based on the Kendall shape space, in which the key idea is to introduce a representation for the 3-D expression model to remove rigid transformations, such as translation, scaling, and rotation, from the Kendall shape space. Built upon the proposed representation, the 2-D caricature expression extrapolation process can be controlled by the 3-D model reconstructed from the input 2-D caricature image and the exaggerated expressions of the caricature images generated based on the extrapolated expression of a 3-D model that is robust to facial poses in the Kendall shape space; this 3-D model can be calculated with tools such as exponential mapping in Riemannian space. The experimental results demonstrate that our method can effectively and automatically extrapolate facial expressions in caricatures with high consistency and fidelity. In addition, we derive 3-D facial models with diverse expressions and expand the scale of the original FaceWarehouse database. Furthermore, compared with the deep learning method, our approach is based on standard face datasets and avoids the construction of complicated 3-D caricature training sets.

Smoothing Splines on Riemannian Manifolds, with Applications to 3D Shape Space

AbstractThere has been increasing interest in statistical analysis of data lying in manifolds. This paper generalizes a smoothing spline fitting method to Riemannian manifold data based on the technique of unrolling, unwrapping and wrapping originally proposed by Jupp and Kent for spherical data. In particular, we develop such a fitting procedure for shapes of configurations in general m-dimensional Euclidean space, extending our previous work for two-dimensional shapes. We show that parallel transport along a geodesic on Kendall shape space is linked to the solution of a homogeneous first-order differential equation, some of whose coefficients are implicitly defined functions. This finding enables us to approximate the procedure of unrolling and unwrapping by simultaneously solving such equations numerically, and so to find numerical solutions for smoothing splines fitted to higher dimensional shape data. This fitting method is applied to the analysis of some dynamic 3D peptide data.

Shape correspondence based on Kendall shape space and RAG for 2D animation

We introduce a 2D vectorized shape correspondence method based on Kendall shape space and region adjacency graph. We regard shape correspondence in 2D animation as a weighted bipartite graph matching problem and optimize it by means of various weights, including geometric attribute, local topological information, and global topological information in 2D animation drawing. The measuring method in Kendall shape space is introduced to determine similarity between two regions and a local adjacent region matching method is presented to associate the vicinity of the corresponding region by utilizing local topology information. Ultimately, we propose a globally optimal shape matching method, which exploits the global topological information to get the final result of the shape correspondence. Our approach can efficiently match associated regions that have exaggerated deformation and unstable topology between two shapes in 2D animation. It is also robust to the similarity transformation of shapes.

Heat Kernel Approximation on Kendall Shape Space

The heat kernel on Kendall shape subspaces is approximated by an expansion. The Kendall space is useful for representing the shapes associated to collections of landmarks’positions. The Minakshisundaram-Pleijel recursion formulas are used in order to calculate the closed-form approximations of the first and second coefficients of the heat kernel expansion. Prior to the exploitation of the recursion scheme, the expression of the Laplace-Beltrami operator is adapted to the targeted space using geodesic spherical and angular coordinates.

3D Facial Similarity Measurement and Its Application in Facial Organization

We propose a novel framework for 3D facial similarity measurement and its application in facial organization. The construction of the framework is based on Kendall shape space theory. Kendall shape space is a quotient space that is constructed by shape features. In Kendall shape space, the shape features can be measured and is robust to similarity transformations. In our framework, a 3D face is represented by the facial feature landmarks model (FFLM), which can be regarded as the facial shape features. We utilize the geodesic in Kendall shape space to represent the FFLM similarity measurement, which can be regarded as the 3D facial similarity measurement. The FFLM similarity measurement is robust to facial expressions, head poses, and partial facial data. In our experiments, we compute the distance between different FFLMs in two public facial databases: FRGC2.0 and BosphorusDB. On average, we achieve a rank-one facial recognition rate of 98%. Based on the similarity results, we propose a method to construct the facial organization. The facial organization is a hierarchical structure that is achieved from the facial clustering by FFLM similarity measurement. Based on the facial organization, the performance of face searching in a large facial database can be improved obviously (about 400% improvement in experiments).

Generalized Partially Linear Models on Riemannian Manifolds

SummaryWe introduce generalized partially linear models with covariates on Riemannian manifolds. These models, like ordinary generalized linear models, are a generalization of partially linear models on Riemannian manifolds that allow for scalar response variables with error distribution models other than a normal distribution. Partially linear models are particularly useful when some of the covariates of the model are elements of a Riemannian manifold, because the curvature of these spaces makes it difficult to define parametric models. The model was developed to address an interesting application: the prediction of children's garment fit based on three-dimensional scanning of their bodies. For this reason, we focus on logistic and ordinal models and on the important and difficult case where the Riemannian manifold is the three-dimensional case of Kendall's shape space. An experimental study with a well-known three-dimensional database is carried out to check the goodness of the procedure. Finally, it is applied to a three-dimensional database obtained from an anthropometric survey of the Spanish child population. A comparative study with related techniques is carried out.

Integrando análisis morfométricos y filogenéticos: de la sistemática fenética a la morfometría filogenética

Antecedentes y Objetivos: Se han acumulado métodos cuantitativos para el uso de mediciones lineales y coordenadas Cartesianas de puntos en análisis de la variación morfométrica. A diferencia de revisiones previas, aquí se enfatizan las bases teóricas de los espacios matemáticos y del morfoespacio de un carácter taxonómico. El objetivo de esta revisión es suministrar elementos conceptuales para una comprensión básica de los métodos morfométricos y estadísticos útiles en la sistemática bajo un enfoque filogenético. Métodos: Los datos morfométricos se están aplicando en estudios de biología comparativa, usando las filogenias como referencia. En contraste, las aplicaciones de la morfometría en sistemática han sido con el objetivo de agrupar y distinguir grupos taxonómicos fenéticamente con la similitud total. Bajo un enfoque filogenético, los datos morfométricos también se pueden usar para el estudio de la variación de caracteres taxonómicos, la identidad de los estados y la inferencia de filogenias. Los grupos taxonómicos debieran ser propuestos a partir de grupos monofiléticos descubiertos con métodos filogenéticos. Resultados clave: Se presentan las bases de la teoría de la morfometría, geometría vectorial, el espacio de Kendall, la distancia Procrustes, proyección de espacios tangenciales y construcción de hipercubos del morfoespacio. Se revisan conceptos estadísticos útiles para la aplicación de los Análisis de Componentes Principales y los Análisis de Variables Canónicas en sistemática. Particularmente se dan recomendaciones y ejemplos sobre el uso de mediciones lineales y coordenadas de puntos en análisis morfométricos para la identificación de especies, la variación de caracteres taxonómicos y en la inferencia de filogenias y clasificación. Conclusiones: El amplio acervo actual de métodos brinda la oportunidad de integrar los datos morfométricos para descubrir homología táxica y homología transformacional. Esto implica un cambio epistemológico necesario para transitar de aplicaciones bajo una sistemática fenética, a la integración de los análisis morfométricos como parte de la investigación filogenética.

Dictionary Learning for Two-Dimensional Kendall Shapes

We propose a novel sparse dictionary learning method for planar shapes in the sense of Kendall, namely configurations of landmarks in the plane considered up to similitudes. Our shape dictionary method provides a good trade-off between algorithmic simplicity and faithfulness with respect to the nonlinear geometric structure of Kendall's shape space. Remarkably, it boils down to a classical dictionary learning formulation modified using complex weights. Existing dictionary learning methods extended to nonlinear spaces map the manifold either to a reproducing kernel Hilbert space or to a tangent space. The first approach is unnecessarily heavy in the case of Kendall's shape space and causes the geometrical understanding of shapes to be lost, while the second one induces distortions and theoretical complexity. Our approach does not suffer from these drawbacks. Instead of embedding the shape space into a linear space, we rely on the hyperplane of centered configurations, including preshapes from which shapes are defined as rotation orbits. In this linear space, the dictionary atoms are scaled and rotated using complex weights before summation. Furthermore, our formulation is more general than Kendall's original one: it applies to discretely defined configurations of landmarks as well as continuously defined interpolating curves. We implemented our algorithm by adapting the method of optimal directions combined to a Cholesky-optimized order recursive matching pursuit. An interesting feature of our shape dictionary is that it produces visually realistic atoms, while guaranteeing reconstruction accuracy. Its efficiency can mostly be attributed to a clear formulation of the framework with complex numbers. We illustrate the strong potential of our approach for the characterization of datasets of shapes up to similitudes and the analysis of patterns in deforming two-dimensional shapes.

Ethnicity classification by the 3D Discrete Landmarks Model measure in Kendall shape space

Knowledge of human ethnicity constitutes important biometric information. An automated ethnicity classification is a good first step in facial analysis. However, most ethnicity classification methods require a complex feature extraction and model training process. We propose a novel ethnicity classification method based on the analysis of facial landmarks in Kendall shape space. Facial features with different relative positions have a close relationship with ethnicity. Facial landmarks can represent positions of facial features. We build a Discrete Landmarks Model (DLM) based on facial landmarks and construct an ethnicity classification model based on the DLM analysis. The clear advantages of our method are that it is fully automated; requires no complex data preprocessing, feature extraction or a complex training process; results in a fast and accurate classification process. We estimate the effectiveness of our method experimentally, using public databases such as Texas3D, FRGC2.0, BU-3DFE and BU-4DFE. On average, our method can achieve a 95% ethnicity classification rate with each classification attempt in 2.0 s.

3D facial expression modeling based on facial landmarks in single image

Facial expression modeling is important for many applications such as human emotional analysis and facial animation. Generally, facial expression modeling from single 2D facial image is difficult. Different head poses and scales of facial data in images affect the accuracy of the modeling results. We propose a new 3D facial expression modeling method which is based on facial landmarks from single image. Using the facial landmarks, expression modeling can be processed in Kendall shape space. The Kendall shape space is mathematic space, the facial expression modeling process in Kendall shape space can be regarded as a geodesic path search between different faces. The modeling result is more accurate. The 3D facial expression modeling result is convenient to obtain from 2D facial image with different head poses. In experiments, we show the 3D facial expression modeling performance by our method, which include expression editing and evaluation in public facial database: JAFFE, LFW, Helen and RAF-DB.

Joint Segmentation and Shape Regularization With a Generalized Forward-Backward Algorithm.

This paper presents a method for the simultaneous segmentation and regularization of a series of shapes from a corresponding sequence of images. Such series arise as time series of 2D images when considering video data, or as stacks of 2D images obtained by slicewise tomographic reconstruction. We first derive a model where the regularization of the shape signal is achieved by a total variation prior on the shape manifold. The method employs a modified Kendall shape space to facilitate explicit computations together with the concept of Sobolev gradients. For the proposed model, we derive an efficient and computationally accessible splitting scheme. Using a generalized forward-backward approach, our algorithm treats the total variation atoms of the splitting via proximal mappings, whereas the data terms are dealt with by gradient descent. The potential of the proposed method is demonstrated on various application examples dealing with 3D data. We explain how to extend the proposed combined approach to shape fields which, for instance, arise in the context of 3D+t imaging modalities, and show an application in this setup as well.

Intrinsic Polynomials for Regression on Riemannian Manifolds

We develop a framework for polynomial regression on Riemannian manifolds. Unlike recently developed spline models on Riemannian manifolds, Riemannian polynomials offer the ability to model parametric polynomials of all integer orders, odd and even. An intrinsic adjoint method is employed to compute variations of the matching functional, and polynomial regression is accomplished using a gradient-based optimization scheme. We apply our polynomial regression framework in the context of shape analysis in Kendall shape space as well as in diffeomorphic landmark space. Our algorithm is shown to be particularly convenient in Riemannian manifolds with additional symmetry, such as Lie groups and homogeneous spaces with right or left invariant metrics. As a particularly important example, we also apply polynomial regression to time-series imaging data using a right invariant Sobolev metric on the diffeomorphism group. The results show that Riemannian polynomials provide a practical model for parametric curve regression, while offering increased flexibility over geodesics.

Hierarchical bayesian modeling, estimation, and sampling for multigroup shape analysis.

This paper proposes a novel method for the analysis of anatomical shapes present in biomedical image data. Motivated by the natural organization of population data into multiple groups, this paper presents a novel hierarchical generative statistical model on shapes. The proposed method represents shapes using pointsets and defines a joint distribution on the population's (i) shape variables and (ii) object-boundary data. The proposed method solves for optimal (i) point locations, (ii) correspondences, and (iii) model-parameter values as a single optimization problem. The optimization uses expectation maximization relying on a novel Markov-chain Monte-Carlo algorithm for sampling in Kendall shape space. Results on clinical brain images demonstrate advantages over the state of the art.

Affine-invariant Shape Recognition Using Grassmann Manifold

Traditional Kendall shape space theory is only applied to similar transform. However, geometric transforms of the object in the imaging process should be represented by affine transform at most situations. We analyze the nonlinear geometry structure of the affine invariant shape space and propose an affine-invariant shape recognition algorithm based on Grassmann manifold geometry. Firstly, we compute the mean shape and covariance for every shape class in the train sets. Then, we construct their norm probability models on the tangent space at each mean shape. Finally, we compute the maximum likelihood class according to the measured object and prior learned shape models. We use the proposed algorithm to recognize shapes in standard shape dataset and real images. Experiment results on MPEG-7 shape dataset show that our recognition algorithm outperforms the algorithm based on Procrustean metric in traditional Kendall shape space theory. Experiment results on real images also show that the proposed algorithm exhibits higher capacity to affine transform than the Procrustean metric based algorithm and can recognize object classes with higher posterior probability.