Flexible Shape Model Research Articles

Localized Statistical Shape Models for Large-Scale Problems With Few Training Data.

Statistical shape models have been successfully used in numerous biomedical image analysis applications where prior shape information is helpful such as organ segmentation or data augmentation when training deep learning models. However, training such models requires large data sets, which are often not available and, hence, shape models frequently fail to represent local details of unseen shapes. This work introduces a kernel-based method to alleviate this problem via so-called model localization. It is specifically designed to be used in large-scale shape modeling scenarios like deep learning data augmentation and fits seamlessly into the classical shape modeling framework. Relying on recent advances in multi-level shape model localization via distance-based covariance matrix manipulations and Grassmannian-based level fusion, this work proposes a novel and computationally efficient kernel-based localization technique. Moreover, a novel way to improve the specificity of such models via normalizing flow-based density estimation is presented. The method is evaluated on the publicly available JSRT/SCR chest X-ray and IXI brain data sets. The results confirm the effectiveness of the kernelized formulation and also highlight the models' improved specificity when utilizing the proposed density estimation method. This work shows that flexible and specific shape models from few training samples can be generated in a computationally efficient way by combining ideas from kernel theory and normalizing flows. The proposed method together with its publicly available implementation allows to build shape models from few training samples directly usable for applications like data augmentation.

A Marked Poisson Process Driven Latent Shape Model for 3D Segmentation of Reflectance Confocal Microscopy Image Stacks of Human Skin.

Segmenting objects of interest from 3D data sets is a common problem encountered in biological data. Small field of view and intrinsic biological variability combined with optically subtle changes of intensity, resolution, and low contrast in images make the task of segmentation difficult, especially for microscopy of unstained living or freshly excised thick tissues. Incorporating shape information in addition to the appearance of the object of interest can often help improve segmentation performance. However, the shapes of objects in tissue can be highly variable and design of a flexible shape model that encompasses these variations is challenging. To address such complex segmentation problems, we propose a unified probabilistic framework that can incorporate the uncertainty associated with complex shapes, variable appearance, and unknown locations. The driving application that inspired the development of this framework is a biologically important segmentation problem: the task of automatically detecting and segmenting the dermal-epidermal junction (DEJ) in 3D reflectance confocal microscopy (RCM) images of human skin. RCM imaging allows noninvasive observation of cellular, nuclear, and morphological detail. The DEJ is an important morphological feature as it is where disorder, disease, and cancer usually start. Detecting the DEJ is challenging, because it is a 2D surface in a 3D volume which has strong but highly variable number of irregularly spaced and variably shaped "peaks and valleys." In addition, RCM imaging resolution, contrast, and intensity vary with depth. Thus, a prior model needs to incorporate the intrinsic structure while allowing variability in essentially all its parameters. We propose a model which can incorporate objects of interest with complex shapes and variable appearance in an unsupervised setting by utilizing domain knowledge to build appropriate priors of the model. Our novel strategy to model this structure combines a spatial Poisson process with shape priors and performs inference using Gibbs sampling. Experimental results show that the proposed unsupervised model is able to automatically detect the DEJ with physiologically relevant accuracy in the range 10- 20 μm .

3-D Facial Landmark Localization With Asymmetry Patterns and Shape Regression from Incomplete Local Features.

We present a method for the automatic localization of facial landmarks that integrates nonrigid deformation with the ability to handle missing points. The algorithm generates sets of candidate locations from feature detectors and performs combinatorial search constrained by a flexible shape model. A key assumption of our approach is that for some landmarks there might not be an accurate candidate in the input set. This is tackled by detecting partial subsets of landmarks and inferring those that are missing, so that the probability of the flexible model is maximized. The ability of the model to work with incomplete information makes it possible to limit the number of candidates that need to be retained, drastically reducing the number of combinations to be tested with respect to the alternative of trying to always detect the complete set of landmarks. We demonstrate the accuracy of the proposed method in the face recognition grand challenge database, where we obtain average errors of approximately 3.5 mm when targeting 14 prominent facial landmarks. For the majority of these our method produces the most accurate results reported to date in this database. Handling of occlusions and surfaces with missing parts is demonstrated with tests on the Bosphorus database, where we achieve an overall error of 4.81 and 4.25 mm for data with and without occlusions, respectively. To investigate potential limits in the accuracy that could be reached, we also report experiments on a database of 144 facial scans acquired in the context of clinical research, with manual annotations performed by experts, where we obtain an overall error of 2.3 mm, with averages per landmark below 3.4 mm for all 14 targeted points and within 2 mm for half of them. The coordinates of automatically located landmarks are made available on-line.

Hierarchical Object Parsing from Structured Noisy Point Clouds

Object parsing and segmentation from point clouds are challenging tasks because the relevant data is available only as thin structures along object boundaries or other features, and is corrupted by large amounts of noise. To handle this kind of data, flexible shape models are desired that can accurately follow the object boundaries. Popular models such as active shape and active appearance models (AAMs) lack the necessary flexibility for this task, while recent approaches such as the recursive compositional models make model simplifications to obtain computational guarantees. This paper investigates a hierarchical Bayesian model of shape and appearance in a generative setting. The input data is explained by an object parsing layer which is a deformation of a hidden principal component analysis (PCA) shape model with Gaussian prior. The paper also introduces a novel efficient inference algorithm that uses informed data-driven proposals to initialize local searches for the hidden variables. Applied to the problem of object parsing from structured point clouds such as edge detection images, the proposed approach obtains state-of-the-art parsing errors on two standard datasets without using any intensity information.

Gabor feature constrained statistical model for efficient landmark localization and face recognition

Feature extraction and classification using Gabor wavelets have proven to be successful in computer vision and pattern recognition. Gabor feature-based Elastic Bunch Graph Matching (EBGM), which demonstrated excellent performance in the FERET evaluation test, has been considered as one of the best algorithms for face recognition due to its robustness against expression, illumination and pose variations. However, EBGM involves considerable computational complexity in its rigid and deformable matching process, preventing its use in many real-time applications. This paper presents a new Constrained Profile Model (CPM), in cooperation with Flexible Shape Model (FSM) to form an efficient localization framework. Through Gabor feature constrained local alignment, the proposed method not only avoids local minima in landmark localization, but also circumvents the exhaustive global optimization. Experiments on CAS-PEAL and FERET databases demonstrated the effectiveness and efficiency of the proposed method.

Flexible Skew-Symmetric Shape Model for Shape Representation, Classification, and Sampling

Skewness of shape data often arises in applications (e.g., medical image analysis) and is usually overlooked in statistical shape models. In such cases, a Gaussian assumption is unrealistic and a formulation of a general shape model which accounts for skewness is in order. In this paper, we present a novel statistical method for shape modeling, which we refer to as the flexible skew-symmetric shape model (FSSM). The model is sufficiently flexible to accommodate a departure from Gaussianity of the data and is fairly general to learn a "mean shape" (template), with a potential for classification and random generation of new realizations of a given shape. Robustness to skewness results from deriving the FSSM from an extended class of flexible skew-symmetric distributions. In addition, we demonstrate that the model allows us to extract principal curves in a point cloud. The idea is to view a shape as a realization of a spatial random process and to subsequently learn a shape distribution which captures the inherent variability of realizations, provided they remain, with high probability, within a certain neighborhood range around a mean. Specifically, given shape realizations, FSSM is formulated as a joint bimodal distribution of angle and distance from the centroid of an aggregate of random points. Mean shape is recovered from the modes of the distribution, while the maximum likelihood criterion is employed for classification.

Separating shape and pose variations

We have integrated a coupled-view Flexible Shape Model (FSM) and the Linear Combination of Views (LCV) technique to form an Integrated Shape and Pose Model (ISPM) of a 3D deformable object. The ISPM describes the object's intrinsic shape variations via the coupled-view FSM, while modelling its extrinsic pose variations via the LCV technique, all in a 2D image context. This enables us to model pose and shape independently in 2D. Our main contribution is the pose-alignment algorithm that separates pose and shape information. Some preliminary results are provided.

Segmentation of 2-D and 3-D objects from MRI volume data using constrained elastic deformations of flexible Fourier contour and surface models

This paper describes a new model-based segmentation technique combining desirable properties of physical models (snakes), shape representation by Fourier parametrization, and modelling of natural shape variability. Flexible parametric shape models are represented by a parameter vector describing the mean contour and by a set of eigenmodes of the parameters characterizing the shape variation. Usually the segmentation process is divided into an initial placement of the mean model and an elastic deformation restricted to the model variability. This, however, leads to a separation of biological variation due to a global similarity transform from small-scale shape changes originating from elastic deformations of the normalized model contours only. The performance can be considerably improved by building shape models normalized with respect to a small set of stable landmarks (AC-PC in our application) and by explaining the remaining variability among a series of images with the model flexibility. This way the image interpretation is solved by a new coarse-to-fine segmentation procedure based on the set of deformation eigenmodes, making a separate initialization step unnecessary. Although straightforward, the extension to 3-D is severely impeded by difficulties arising during the generation of a proper surface parametrization for arbitrary objects with spherical topology. We apply a newly developed surface parametrization which achieves a uniform mapping between object surface and parameter space. The 3-D procedure is demonstrated by segmenting deep structures of the human brain from MR volume data.

Automatic tracking, coding and reconstruction ofhuman faces, using flexible appearance models

The use of flexible shape and flexible grey-level models for representing variations in the appearance of human faces is described. These models are controlled by a small number of parameters which can be used for coding and reconstructing a face image.

Image search using trained flexible shape models

This paper describes a technique for building compact models of the shape and appearance of flexible objects seen in two-dimensional images. The models are derived from the statistics of sets of images of example objects with ‘landmark’ points labelled on each object. Each model consists of a flexible shape template, describing how the landmark points can vary, and a statistical model of the expected grey levels in regions around each point. Such models have proved useful in a wide variety of applications. We describe how the models can be used in local image search and give examples of their application.