Kendall Shape Research Articles

A Gaussian kernel for Kendall’s space of [formula omitted]-D shapes

In this paper, we develop an approach to exploit kernel methods with data lying on the m-D Kendall shape space. When data arise in a finite-dimensional curved Riemannian manifold, as in this case, the usual Euclidean computer vision and machine learning algorithms must be treated carefully. A good approach is to use positive definite kernels on manifolds to embed the manifold with its corresponding metric in a high-dimensional reproducing kernel Hilbert space, where it is possible to utilize algorithms developed for linear spaces. Different Gaussian kernels can be found in the literature on the 2-D Kendall shape space to perform this embedding. The main novelty of this work is to provide a Gaussian kernel for the m-D Kendall shape space. This new Kernel coincides in the case m=2 with the Gaussian kernels most widely used in the Kendall planar shape space and allows to define an embedding of the m-D Kendall shape space into a reproducible kernel Hilbert space. As far as we know, the complexity of the m-D Kendall shape space has meant that this embedding has not been addressed in the literature until now. This methodology will be tested on a machine learning problem with a simulated and a real data set.

Shape correspondence for cel animation based on a shape association graph and spectral matching

We present an effective spectral matching method based on a shape association graph for finding region correspondences between two cel animation keyframes. We formulate the correspondence problem as an adapted quadratic assignment problem, which comprehensively considers both the intrinsic geometric and topology of regions to find the globally optimal correspondence. To simultaneously represent the geometric and topological similarities between regions, we propose a shape association graph (SAG), whose node attributes indicate the geometric distance between regions, and whose edge attributes indicate the topological distance between combined region pairs. We convert topological distance to geometric distance between geometric objects with topological features of the pairs, and introduce Kendall shape space to calculate the intrinsic geometric distance. By utilizing the spectral properties of the affinity matrix induced by the SAG, our approach can efficiently extract globally optimal region correspondences, even if shapes have inconsistent topology and severe deformation. It is also robust to shapes undergoing similarity transformations, and compatible with parallel computing techniques.

Geometric Deep Neural Network Using Rigid and Non-Rigid Transformations for Landmark-Based Human Behavior Analysis.

Deep learning architectures, albeit successful in most computer vision tasks, were designed for data with an underlying Euclidean structure, which is not usually fulfilled since pre-processed data may lie on a non-linear space. In this article, we propose a geometric deep learning approach using rigid and non-rigid transformations, named KShapenet, for 2D and 3D landmark-based human motion analysis. Landmark configuration sequences are first modeled as trajectories on Kendall's shape space and then mapped to a linear tangent space. The resulting structured data are then input to a deep learning architecture, which includes a layer that optimizes over rigid and non-rigid transformations of landmark configurations, followed by a CNN-LSTM network. We apply KShapenet to 3D human landmark sequences for action and gait recognition, and 2D facial landmark sequences for expression recognition, and demonstrate the competitiveness of the proposed approach with respect to state-of-the-art.

KSS-ICP: Point Cloud Registration based on Kendall Shape Space.

Point cloud registration is a popular topic that has been widely used in 3D model reconstruction, location, and retrieval. In this paper, we propose a new registration method, KSS-ICP, to address the rigid registration task in Kendall shape space (KSS) with Iterative Closest Point (ICP). The KSS is a quotient space that removes influences of translations, scales, and rotations for shape feature-based analysis. Such influences can be concluded as the similarity transformations that do not change the shape feature. The point cloud representation in KSS is invariant to similarity transformations. We utilize such property to design the KSS-ICP for point cloud registration. To tackle the difficulty to achieve the KSS representation in general, the proposed KSS-ICP formulates a practical solution that does not require complex feature analysis, data training, and optimization. With a simple implementation, KSS-ICP achieves more accurate registration from point clouds. It is robust to similarity transformation, non-uniform density, noise, and defective parts. Experiments show that KSS-ICP has better performance than the state-of-the-art. Code1 and executable files2 are made public.

Geodesics in the TPS Space

In shape analysis, the interpolation of shapes’ trajectories is often performed by means of geodesics in an appropriate Riemannian Shape Space. Over the past several decades, different metrics and shape spaces have been proposed, including Kendall shape space, LDDMM based approaches, and elastic contour, among others. Once a Riemannian space is chosen, geodesics and parallel transports can be used to build splines or piecewise geodesics paths. In a recent paper, we introduced a new Riemannian shape space named TPS Space based on the Thin Plate Spline interpolant and characterized by an appropriate metric and parallel transport rule. In the present paper, we further explore the geometry of the TPS Space by characterizing the properties of its geodesics. Several applications show the capability of the proposed formulation to conserve important physical properties of deformation, such as local strains and global elastic energy.

Nonparametric k-Sample Test on Shape Spaces with Applications to Mitochondrial Shape Analysis

Abstract An important hypothesis in animal cell biology is that an animal’s acute exercise regimen affects some subcellular structures, for example mitochondrial morphology, in its muscle tissue. This paper investigates that hypothesis using a nonparametric metric-based energy test for comparing mitochondrial populations. It explores two shape spaces—the elastic shape space and Kendall shape space—and five corresponding shape metrics on these spaces. The results overwhelmingly point to the statistical significance of the effect of an acute exercise regimen on the shape of SS-type mitochondria. Although past studies based on specific morphological features derived from mitochondria failed to detect this significance. In this analysis, a potentially significant factor is cell membership and a k-sample generalization of the energy test—the DISCO test shows that the cell effect is indeed significant. The energy test cannot be applied directly due to the hierarchical structure of the distance matrix. We propose a compression method to remove the significant cell effect while testing for the exercise effect. With this compression, only the elastic scaled metric shows statistical significance of the exercise factor in this more complicated scenario. This result is because the elastic-scaled metric is more sensitive to subtle changes in mitochondrial shapes caused by acute exercise.

Caricature Expression Extrapolation Based on Kendall Shape Space Theory.

Facial expression editing plays a fundamental role in facial expression generation and has been widely applied in modern film productions and computer games. While the existing 2-D caricature facial expression editing methods are mostly realized by expression interpolation from the original image to the target image, expression extrapolation has rarely been studied before. In this article, we propose a novel expression extrapolation method for caricature facial expressions based on the Kendall shape space, in which the key idea is to introduce a representation for the 3-D expression model to remove rigid transformations, such as translation, scaling, and rotation, from the Kendall shape space. Built upon the proposed representation, the 2-D caricature expression extrapolation process can be controlled by the 3-D model reconstructed from the input 2-D caricature image and the exaggerated expressions of the caricature images generated based on the extrapolated expression of a 3-D model that is robust to facial poses in the Kendall shape space; this 3-D model can be calculated with tools such as exponential mapping in Riemannian space. The experimental results demonstrate that our method can effectively and automatically extrapolate facial expressions in caricatures with high consistency and fidelity. In addition, we derive 3-D facial models with diverse expressions and expand the scale of the original FaceWarehouse database. Furthermore, compared with the deep learning method, our approach is based on standard face datasets and avoids the construction of complicated 3-D caricature training sets.

Smoothing Splines on Riemannian Manifolds, with Applications to 3D Shape Space

AbstractThere has been increasing interest in statistical analysis of data lying in manifolds. This paper generalizes a smoothing spline fitting method to Riemannian manifold data based on the technique of unrolling, unwrapping and wrapping originally proposed by Jupp and Kent for spherical data. In particular, we develop such a fitting procedure for shapes of configurations in general m-dimensional Euclidean space, extending our previous work for two-dimensional shapes. We show that parallel transport along a geodesic on Kendall shape space is linked to the solution of a homogeneous first-order differential equation, some of whose coefficients are implicitly defined functions. This finding enables us to approximate the procedure of unrolling and unwrapping by simultaneously solving such equations numerically, and so to find numerical solutions for smoothing splines fitted to higher dimensional shape data. This fitting method is applied to the analysis of some dynamic 3D peptide data.

New Directional Residuals to Treat Shape Changes Using Spherical Regression Models

One of the important problems in the statistical shape analysis context is to predict the shape change. Rather than taking the change in terms of time, we confine ourselves to the deformation of the objects represented by a regression-type model. We properly combine the shape definition in terms of the Kendall and Bookstein shape coordinate systems to break down the problem on the sphere. This is achieved through the triangulation of objects; a very popular technique in geometrical mathematics. A novel idea on tracing the residuals of the spherical regression is then proposed, enabling us to invoke the well-known spherical distributions, including von Mises–Fisher density, to make the statistical inference. New directional residuals not only lie on the sphere but also help to easily simulate the spherical responses. The performance of our proposed method is highlighted via running a simulation study as well as analysing a real data set.

Shape correspondence based on Kendall shape space and RAG for 2D animation

We introduce a 2D vectorized shape correspondence method based on Kendall shape space and region adjacency graph. We regard shape correspondence in 2D animation as a weighted bipartite graph matching problem and optimize it by means of various weights, including geometric attribute, local topological information, and global topological information in 2D animation drawing. The measuring method in Kendall shape space is introduced to determine similarity between two regions and a local adjacent region matching method is presented to associate the vicinity of the corresponding region by utilizing local topology information. Ultimately, we propose a globally optimal shape matching method, which exploits the global topological information to get the final result of the shape correspondence. Our approach can efficiently match associated regions that have exaggerated deformation and unstable topology between two shapes in 2D animation. It is also robust to the similarity transformation of shapes.

Heat Kernel Approximation on Kendall Shape Space

The heat kernel on Kendall shape subspaces is approximated by an expansion. The Kendall space is useful for representing the shapes associated to collections of landmarks’positions. The Minakshisundaram-Pleijel recursion formulas are used in order to calculate the closed-form approximations of the first and second coefficients of the heat kernel expansion. Prior to the exploitation of the recursion scheme, the expression of the Laplace-Beltrami operator is adapted to the targeted space using geodesic spherical and angular coordinates.

3D Facial Similarity Measurement and Its Application in Facial Organization

We propose a novel framework for 3D facial similarity measurement and its application in facial organization. The construction of the framework is based on Kendall shape space theory. Kendall shape space is a quotient space that is constructed by shape features. In Kendall shape space, the shape features can be measured and is robust to similarity transformations. In our framework, a 3D face is represented by the facial feature landmarks model (FFLM), which can be regarded as the facial shape features. We utilize the geodesic in Kendall shape space to represent the FFLM similarity measurement, which can be regarded as the 3D facial similarity measurement. The FFLM similarity measurement is robust to facial expressions, head poses, and partial facial data. In our experiments, we compute the distance between different FFLMs in two public facial databases: FRGC2.0 and BosphorusDB. On average, we achieve a rank-one facial recognition rate of 98%. Based on the similarity results, we propose a method to construct the facial organization. The facial organization is a hierarchical structure that is achieved from the facial clustering by FFLM similarity measurement. Based on the facial organization, the performance of face searching in a large facial database can be improved obviously (about 400% improvement in experiments).

Generalized Partially Linear Models on Riemannian Manifolds

SummaryWe introduce generalized partially linear models with covariates on Riemannian manifolds. These models, like ordinary generalized linear models, are a generalization of partially linear models on Riemannian manifolds that allow for scalar response variables with error distribution models other than a normal distribution. Partially linear models are particularly useful when some of the covariates of the model are elements of a Riemannian manifold, because the curvature of these spaces makes it difficult to define parametric models. The model was developed to address an interesting application: the prediction of children's garment fit based on three-dimensional scanning of their bodies. For this reason, we focus on logistic and ordinal models and on the important and difficult case where the Riemannian manifold is the three-dimensional case of Kendall's shape space. An experimental study with a well-known three-dimensional database is carried out to check the goodness of the procedure. Finally, it is applied to a three-dimensional database obtained from an anthropometric survey of the Spanish child population. A comparative study with related techniques is carried out.

A Central Limit Theorem for extrinsic antimeans and estimation of Veronese–Whitney means and antimeans on planar Kendall shape spaces

This article is concerned with random objects in the complex projective space ℂPk−2. It is shown that the Veronese–Whitney (VW) antimean, which is the extrinsic antimean of a random point on ℂPk−2 relative to the VW-embedding, is given by the point on ℂPk−2 represented by the eigenvector corresponding to the smallest eigenvalue of the expected mean of the VW-embedding of the random point, provided this eigenvalue is simple. We also derive a CLT for extrinsic sample antimeans, and an asymptotic χ2-distribution of an appropriately studentized statistic, based on the extrinsic antimean, which in the particular case of a VW-embedding is then used to construct nonparametric bootstrap confidence regions for the VW-antimean planar Kendall shape. Simulations studies and an application to medical imaging are illustrating the proposed methodology.

Integrando análisis morfométricos y filogenéticos: de la sistemática fenética a la morfometría filogenética

Antecedentes y Objetivos: Se han acumulado métodos cuantitativos para el uso de mediciones lineales y coordenadas Cartesianas de puntos en análisis de la variación morfométrica. A diferencia de revisiones previas, aquí se enfatizan las bases teóricas de los espacios matemáticos y del morfoespacio de un carácter taxonómico. El objetivo de esta revisión es suministrar elementos conceptuales para una comprensión básica de los métodos morfométricos y estadísticos útiles en la sistemática bajo un enfoque filogenético. Métodos: Los datos morfométricos se están aplicando en estudios de biología comparativa, usando las filogenias como referencia. En contraste, las aplicaciones de la morfometría en sistemática han sido con el objetivo de agrupar y distinguir grupos taxonómicos fenéticamente con la similitud total. Bajo un enfoque filogenético, los datos morfométricos también se pueden usar para el estudio de la variación de caracteres taxonómicos, la identidad de los estados y la inferencia de filogenias. Los grupos taxonómicos debieran ser propuestos a partir de grupos monofiléticos descubiertos con métodos filogenéticos. Resultados clave: Se presentan las bases de la teoría de la morfometría, geometría vectorial, el espacio de Kendall, la distancia Procrustes, proyección de espacios tangenciales y construcción de hipercubos del morfoespacio. Se revisan conceptos estadísticos útiles para la aplicación de los Análisis de Componentes Principales y los Análisis de Variables Canónicas en sistemática. Particularmente se dan recomendaciones y ejemplos sobre el uso de mediciones lineales y coordenadas de puntos en análisis morfométricos para la identificación de especies, la variación de caracteres taxonómicos y en la inferencia de filogenias y clasificación. Conclusiones: El amplio acervo actual de métodos brinda la oportunidad de integrar los datos morfométricos para descubrir homología táxica y homología transformacional. Esto implica un cambio epistemológico necesario para transitar de aplicaciones bajo una sistemática fenética, a la integración de los análisis morfométricos como parte de la investigación filogenética.

Dictionary Learning for Two-Dimensional Kendall Shapes

We propose a novel sparse dictionary learning method for planar shapes in the sense of Kendall, namely configurations of landmarks in the plane considered up to similitudes. Our shape dictionary method provides a good trade-off between algorithmic simplicity and faithfulness with respect to the nonlinear geometric structure of Kendall's shape space. Remarkably, it boils down to a classical dictionary learning formulation modified using complex weights. Existing dictionary learning methods extended to nonlinear spaces map the manifold either to a reproducing kernel Hilbert space or to a tangent space. The first approach is unnecessarily heavy in the case of Kendall's shape space and causes the geometrical understanding of shapes to be lost, while the second one induces distortions and theoretical complexity. Our approach does not suffer from these drawbacks. Instead of embedding the shape space into a linear space, we rely on the hyperplane of centered configurations, including preshapes from which shapes are defined as rotation orbits. In this linear space, the dictionary atoms are scaled and rotated using complex weights before summation. Furthermore, our formulation is more general than Kendall's original one: it applies to discretely defined configurations of landmarks as well as continuously defined interpolating curves. We implemented our algorithm by adapting the method of optimal directions combined to a Cholesky-optimized order recursive matching pursuit. An interesting feature of our shape dictionary is that it produces visually realistic atoms, while guaranteeing reconstruction accuracy. Its efficiency can mostly be attributed to a clear formulation of the framework with complex numbers. We illustrate the strong potential of our approach for the characterization of datasets of shapes up to similitudes and the analysis of patterns in deforming two-dimensional shapes.

Ethnicity classification by the 3D Discrete Landmarks Model measure in Kendall shape space

Knowledge of human ethnicity constitutes important biometric information. An automated ethnicity classification is a good first step in facial analysis. However, most ethnicity classification methods require a complex feature extraction and model training process. We propose a novel ethnicity classification method based on the analysis of facial landmarks in Kendall shape space. Facial features with different relative positions have a close relationship with ethnicity. Facial landmarks can represent positions of facial features. We build a Discrete Landmarks Model (DLM) based on facial landmarks and construct an ethnicity classification model based on the DLM analysis. The clear advantages of our method are that it is fully automated; requires no complex data preprocessing, feature extraction or a complex training process; results in a fast and accurate classification process. We estimate the effectiveness of our method experimentally, using public databases such as Texas3D, FRGC2.0, BU-3DFE and BU-4DFE. On average, our method can achieve a 95% ethnicity classification rate with each classification attempt in 2.0 s.

Sparse Coding of Shape Trajectories for Facial Expression and Action Recognition.

The detection and tracking of human landmarks in video streams has gained in reliability partly due to the availability of affordable RGB-D sensors. The analysis of such time-varying geometric data is playing an important role in the automatic human behavior understanding. However, suitable shape representations as well as their temporal evolution, termed trajectories, often lie to nonlinear manifolds. This puts an additional constraint (i.e., nonlinearity) in using conventional Machine Learning techniques. As a solution, this paper accommodates the well-known Sparse Coding and Dictionary Learning approach to study time-varying shapes on the Kendall shape spaces of 2D and 3D landmarks. We illustrate effective coding of 3D skeletal sequences for action recognition and 2D facial landmark sequences for macro- and micro-expression recognition. To overcome the inherent nonlinearity of the shape spaces, intrinsic and extrinsic solutions were explored. As main results, shape trajectories give rise to more discriminative time-series with suitable computational properties, including sparsity and vector space structure. Extensive experiments conducted on commonly-used datasets demonstrate the competitiveness of the proposed approaches with respect to state-of-the-art.

3D facial expression modeling based on facial landmarks in single image

Facial expression modeling is important for many applications such as human emotional analysis and facial animation. Generally, facial expression modeling from single 2D facial image is difficult. Different head poses and scales of facial data in images affect the accuracy of the modeling results. We propose a new 3D facial expression modeling method which is based on facial landmarks from single image. Using the facial landmarks, expression modeling can be processed in Kendall shape space. The Kendall shape space is mathematic space, the facial expression modeling process in Kendall shape space can be regarded as a geodesic path search between different faces. The modeling result is more accurate. The 3D facial expression modeling result is convenient to obtain from 2D facial image with different head poses. In experiments, we show the 3D facial expression modeling performance by our method, which include expression editing and evaluation in public facial database: JAFFE, LFW, Helen and RAF-DB.

Detecting statistical differences between average glaucoma indices of the eye recovered from virtual 3D reconstructions of the eye’s optic nerve head using 2D stereo images

In this article, we revisit a Shape-based Glaucoma Index previously examined in Derado et al. (2004) using Scanning Confocal Laser (Heidelberg Retina) Tomographic (HRT) images. Here, we examine the use of 2D stereographic images for detecting glaucomatous shape change. Similar to Derado et al. (2004), we are interested in detecting such change from the same Rhesus monkeys experimental data. In contrast to Derado et al. (2004), in this study we use 2D stereographic images that were taken on the same sample of Rhesus monkeys before and after receiving an eye injection, inducing glaucoma-like Intra-Ocular Pressure (IOP) in one of their eyes, while keeping the other eye as control. Once we retrieve the 3D Kendall Shape reconstruction for the stereographic images of the Rhesus monkeys eye using Agisoft PhotoScan, we summarize the shape by selecting three matched corresponding biological landmarks on the neural-rim and the fourth matched landmark as the point of maximum depth. Clinicians believes that the ratio depth to diameter of the eye cup provides useful measure of shape change. We calculate the glaucoma index, (G), a normalized depth of the eye cup of Rhesus monkeys before and after receiving an injection. Parametric and nonparametric analyses were performed. Both procedures detected a positive mean glaucoma index increase in the medically induced glaucomatous eye. Our analyses supports a similar conclusion obtained in Derado et al. (2004) using HRT images.