Point Distribution Model Research Articles

Robust deformable image registration using prior shape information for atlas to patient registration

Statistical atlases enable the individualization of atlas information for patient specific applications such as surgical planning. In this paper, a statistical atlas comprising a point distribution model defined on the vertices of a tetrahedral mesh is registered to a subject’s computed tomography scan of the human pelvis. The approach consists of a volumetric deformable registration method augmented to maintain the topology of the atlas mesh after deformation as well as incorporating the dominant three-dimensional shape modes in the atlas. Experimental results demonstrate that incorporation of the statistical shape atlas helps to stabilize the registration and improves robustness and registration accuracy.

3-D Face Detection, Landmark Localization, and Registration Using a Point Distribution Model

<para xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> We present an accurate and robust framework for detecting and segmenting faces, localizing landmarks, and achieving fine registration of face meshes based on the fitting of a facial model. This model is based on a 3-D Point Distribution Model (PDM) that is fitted without relying on texture, pose, or orientation information. Fitting is initialized using candidate locations on the mesh, which are extracted from low-level curvature-based feature maps. Face detection is performed by classifying the transformations between model points and candidate vertices based on the upper-bound of the deviation of the parameters from the mean model. Landmark localization is performed on the segmented face by finding the transformation that minimizes the deviation of the model from the mean shape. Face registration is obtained using prior anthropometric knowledge and the localized landmarks. The performance of face detection is evaluated on a database of faces and non-face objects where we achieve an accuracy of 99.6%. We also demonstrate face detection and segmentation on objects with different scale and pose. The robustness of landmark localization is evaluated with noisy data and by varying the number of shapes and model points used in the model learning phase. Finally, face registration is compared with the traditional Iterative Closest Point (ICP) method and evaluated through a face retrieval and recognition framework on the GavabDB dataset, where we achieve a recognition rate of 87.4% and a retrieval rate of 83.9%. </para>

MATCHING PARAMETERIZED SHAPES BY NONPARAMETRIC BELIEF PROPAGATION

In this paper we generalize the belief propagation based rigid shape matching algorithm to a nonparametric belief propagation based on parameterized shape matching. We construct a local-global shape descriptor based cost function to compare the distances among landmarks in each data set, which is equivalent to the Hamiltonian of a spin glass. The constructed cost function is immune to rigid transformations, therefore the parameterized shape matching can be achieved by searching for the optimal shape parameter and the correspondence assignment that minimize the cost function. The optimization procedure is then approximated by a Monte Carlo simulation based MAP estimation on a graphical model, i.e. the nonparametric belief propagation. Experiments on a principal component analysis (PCA) based point distribution model (PDM) of the proximal femur illustrate the effects of two key factors, the topology of the graphical model and the renormalization of the shape parameters of the parameterized shape. Other factors that can influence its performance and its computational complexity are also discussed.

A generative model for graph matching and embedding

This paper shows how to construct a generative model for graph structure through the embedding of the nodes of the graph in a vector space. We commence from a sample of graphs where the correspondences between nodes are unknown ab initio. We also work with graphs where there may be structural differences present, i.e. variations in the number of nodes in each graph and their edge structure. We characterise the graphs using the heat-kernel, and this is obtained by exponentiating the Laplacian eigensystem with time. The idea underpinning the method is to embed the nodes of the graphs into a vector space by performing a Young-Householder decomposition of the heat-kernel into an inner product of node co-ordinate matrices. The co-ordinates of the nodes are determined by the eigenvalues and eigenvectors of the Laplacian matrix, together with a time-parameter which can be used to scale the embedding. Node correspondences are located by applying Scott and Longuet-Higgins algorithm to the embedded nodes. We capture variations in graph structure using the covariance matrix for corresponding embedded point positions. We construct a point-distribution model for the embedded node positions using the eigenvalues and eigenvectors of the covariance matrix. We show how to use this model to both project individual graphs into the eigenspace of the point position covariance matrix and how to fit the model to potentially noisy graphs to reconstruct the Laplacian matrix. We illustrate the utility of the resulting method for shape analysis using data from the Caltech–Oxford and COIL databases.

Cardiac Motion Recovery via Active Trajectory Field Models

Cardiovascular researchers are constantly developing new and innovative medical imaging technologies, striving to improve the understanding, diagnosis, and treatment of cardiovascular dysfunction. Combining these sophisticated imaging methods with advancements in image understanding via computational intelligence will continue to advance the frontier of cardiovascular medicine. Recently, researchers have turned to a new class of tissue motion imaging techniques, including displacement encoding with stimulated echoes (DENSE) in cardiac magnetic resonance (cMR) imaging, to directly quantify cardiac displacement and produce accurate spatiotemporal measurements of myocardial strain, twist, and torsion. The associated analysis of DENSE cMR and other tissue motion imagery, however, represents a major bottleneck in the study of intramyocardial mechanics. In the computational intelligence area of deformable models, this paper develops an automated motion recovery technique termed active trajectory field models (ATFMs) geared toward these new motion imaging protocols, offering quantitative physiological measurements without the pains of manual analyses. This novel generative deformable model exploits both image information and prior knowledge of cardiac motion, utilizing a point distribution model derived from a training set of myocardial trajectory fields to automatically recover cardiac motion from a noisy image sequence. The effectiveness of the ATFM method is demonstrated by quantifying myocardial motion in 2-D short-axis murine DENSE cMR image sequences both before and after myocardial infarction, producing results comparable to existing semiautomatic analysis methods.

Moments Added Statistical Shape Model for Boundary Extraction

In this paper, we propose a method for extracting an object boundary from a low-quality image such as an infrared one. To take full advantage of a training set, the overall shape is modeled by incorporating statistical characteristics of moments into the point distribution model (PDM). Furthermore, a differential equation for the moment of overall shape is derived for shape refinement, which leads to accurate and rapid deformation of a boundary template toward real object boundary. The simulation results show that the proposed method has better performance than conventional boundary extraction methods.

A 2D/3D correspondence building method for reconstruction of a patient-specific 3D bone surface model using point distribution models and calibrated X-ray images

Constructing a 3D bone surface model from a limited number of calibrated 2D X-ray images (e.g. 2) and a 3D point distribution model is a challenging task, especially, when we would like to construct a patient-specific surface model of a bone with pathology. One of the key steps for such a 2D/3D reconstruction is to establish correspondences between the 2D images and the 3D model. This paper presents a 2D/3D correspondence building method based on a non-rigid 2D point matching process, which iteratively uses a symmetric injective nearest-neighbor mapping operator and 2D thin-plate splines based deformations to find a fraction of best matched 2D point pairs between features extracted from the X-ray images and those extracted from the 3D model. The estimated point pairs are then used to set up a set of 3D point pairs such that we turn a 2D/3D reconstruction problem to a 3D/3D one, whose solutions are well studied. Incorporating this 2D/3D correspondence building method, a 2D/3D reconstruction scheme combining a statistical instantiation with a regularized shape deformation has been developed. Comprehensive experiments on clinical datasets and on images of cadaveric femurs with both non-pathologic and pathologic cases are designed and conducted to evaluate the performance of the 2D/3D correspondence building method as well as that of the 2D/3D reconstruction scheme. Quantitative and qualitative evaluation results are given, which demonstrate the validity of the present method and scheme.

Automatic Construction of 3D-ASM Intensity Models by Simulating Image Acquisition: Application to Myocardial Gated SPECT Studies

Active shape models bear a great promise for model-based medical image analysis. Their practical use, though, is undermined due to the need to train such models on large image databases. Automatic building of point distribution models (PDMs) has been successfully addressed and a number of autolandmarking techniques are currently available. However, the need for strategies to automatically build intensity models around each landmark has been largely overlooked in the literature. This work demonstrates the potential of creating intensity models automatically by simulating image generation. We show that it is possible to reuse a 3D PDM built from computed tomography (CT) to segment gated single photon emission computed tomography (gSPECT) studies. Training is performed on a realistic virtual population where image acquisition and formation have been modeled using the SIMIND Monte Carlo simulator and ASPIRE image reconstruction software, respectively. The dataset comprised 208 digital phantoms (4D-NCAT) and 20 clinical studies. The evaluation is accomplished by comparing point-to-surface and volume errors against a proper gold standard. Results show that gSPECT studies can be successfully segmented by models trained under this scheme with subvoxel accuracy. The accuracy in estimated LV function parameters, such as end diastolic volume, end systolic volume, and ejection fraction, ranged from 90.0% to 94.5% for the virtual population and from 87.0% to 89.5% for the clinical population.

Methods to automatically build Point Distribution Models for objects like hand palms and faces represented in images

In this work we developed methods to automatically extract significant points of objects like hand palms and faces represented in images that can be used to build Point Distribution Models automatically. These models are further used to segment the modelled objects in new images, through the use of Active Shape Models or Active Appearance Models. These models showed to be efficient in the segmentation of objects, but had as drawback the fact that the labelling of the landmark points was usually manually made and consequently time consuming. Thus, in this paper we describe some methods capable to extract significant points of objects like hand palms and compare the segmentation results in new images. Keyword: Feature Points; Image Analysis; Point Distribution Models; Segmentation and Object Recognition.

Pedestrian Detection and Tracking Using a Mixture of View-Based Shape–Texture Models

This paper presents a robust multicue approach to the integrated detection and tracking of pedestrians in a cluttered urban environment. A novel spatiotemporal object representation is proposed, which combines a generative shape model and a discriminative texture classifier, both of which are composed of a mixture of pose-specific submodels. Shape is represented by a set of linear subspace models, which is an extension of point distribution models, with shape transitions being modeled by a first-order Markov process. Texture, i.e., the shape-normalized intensity pattern, is represented by a manifold that is implicitly delimited by a set of pattern classifiers, whereas texture transition is modeled by a random walk. Direct 3-D measurements that are provided by a stereo system are further incorporated into the observation density function. We employ a Bayesian framework based on particle filtering to achieve integrated object detection and tracking. Large-scale experiments that involve pedestrian detection and tracking from a moving vehicle demonstrate the benefit of the proposed approach.

Perfusion weighted color maps for enhanced visualization of myocardial infarction by MSCT: preliminary experience

The reliable detection of myocardial perfusion defects and myocardial infarction (MI) is of great interest in the comprehensive workup of coronary artery disease. The aim of this study was to optimize the ability of contrast-enhanced cardiac multislice spiral computed tomography (MSCT) for detecting hypoperfused myocardium as surrogate marker of MI using a newly developed post-processing technique. First a model-based software tool for semi-automated detection of the long axis of the left ventricle and assignment of left-ventricular segments was developed using a region growing algorithm and a point distribution model. To visualize changes of the myocardial contrast enhancement pattern color coding was performed after spreading of the attenuation values. 15 patients (12 men, mean age 57 +/- 15 years) with a history of MI underwent cardiac MSCT (16 x 0.75 mm, 120 kV, 550 mA s(eff.), 100 ml Iopromide) and contrast enhanced delayed enhanced magnetic resonance imaging (DE-MRI) after administration of 0.2 mmol Gd-DTPA/kg/bodyweight as reference standard. Presence of infarction was assessed from MSCT, post-processed MSCT images and DE-MRI using a 17-segment model of the left ventricle. On DE-MRI MI was present in 78/255 myocardial segments. From conventional MSCT images MI was detected in 63/255 segments (5 false positive; sensitivity 74.4%; specificity: 97.1%), while on post-processed images MI was assigned to 74/255 segments (6 false positive; sensitivity 87.2%; specificity: 96.6%). Agreement between DE-MRI and conventional MSCT images for detecting MI was kappa = 0.756. Using post-processed images agreement improved to kappa = 0.850. MSCT detection of hypoperfused myocardium as surrogate for MI can be improved using dedicated post processing algorithms.

MULTIMODAL BIOMETRICS BY FACE AND HAND IMAGES TAKEN BY A CELL PHONE CAMERA

This paper presents a multimodal approach for a biometrics verification system. It is based on face and hand images captured by a cell phone. The algorithm includes all parts that are required for face and hand verification, such as feature extraction, classification and authentication. To find local facial features, such as eyes, mouth and nose, we apply a point distribution model and active shape models. We use the same system to find distinctive points in hand geometry. The face feature vector is constructed by applying a Gabor filter to the image and extracting the key points found by an active shape model. The palm feature vector contains characteristics of the hand geometry features. A support vector machine (SVM) is applied to verify the identity of the user. One SVM machine is built for each person in the database to distinguish that person from others. To test the algorithm we built our own database containing face and hand images taken by a cell phone camera. The database contains 480 frontal face images and 120 hand images of 30 persons (16 face images and 4 hand images per person).

Toward Pose-Invariant 2-D Face Recognition Through Point Distribution Models and Facial Symmetry

This paper proposes novel ways to deal with pose variations in a 2-D face recognition scenario. Using a training set of sparse face meshes, we built a point distribution model and identified the parameters which are responsible for controlling the apparent changes in shape due to turning and nodding the head, namely the pose parameters. Based on them, we propose two approaches for pose correction: 1) a method in which the pose parameters from both meshes are set to typical values of frontal faces, and 2) a method in which one mesh adopts the pose parameters of the other one. Finally, we obtain pose corrected meshes and, taking advantage of facial symmetry, virtual views are synthesized via Thin Plate Splines-based warping. Given that the corrected images are not embedded into a constant reference frame, holistic methods are not suitable for feature extraction. Instead, the virtual faces are fed into a system that makes use of Gabor filtering for recognition. Unlike other approaches that warp faces onto a mean shape, we show that if only pose parameters are modified, client specific information remains in the warped image and discrimination between subjects is more reliable. Statistical analysis of the authentication results obtained on the XM2VTS database confirm the hypothesis. Also, the CMU PIE database is used to assess the performance of the proposed methods in an identification scenario where large pose variations are present, achieving state-of-the-art results and outperforming both research and commercial techniques.

Assessment of adolescent body perception: Development and characterization of a novel tool for morphing images of adolescent bodies

We developed a computer-based method of distorting adolescent body images, which incorporates the covariation between body parts found during growth and sexual maturation. An adolescent body-shape database (AdoBSD) and adolescent body morphing tool (AdoBMT) are described; the AdoBSD comprises real (n=320) and morphed (n approximately 41,000) images (front and side view) of 160 adolescents (9-17 years). We used a point distribution model, based upon principal components analysis, to characterize the covariation between predefined body tag-points manually positioned on the body images and to morph the body images in a realistic manner. Eight principal components (PCs) were found to characterize 96.3% of the covariation between body tag-point. Application of the PCs to the body images resulted in the manipulation of body parts including shoulder width, waist, hip, belly, thigh and calf sizes. The AdoBMT and AdoBSD may be used to investigate changes in body perception during adolescence, and the role of body perception in adolescent obesity and eating disorders. The AdoBSD is available to the research community (www.brainbody.nottingham.ac.uk).

Human Face Image Searching System Using Sketches

This paper reports a human face image searching system using sketches. A two-phase method, namely, sketch-to-mug-shot matching and human face image searching using relevance feedback, is designed and developed. In the sketch-to-mug-shot matching phase, we have developed a facial feature matching algorithm using local and global features. A point distribution model is employed to represent local facial features while the global feature consists of a set of the geometrical relationship between facial features. It is found that the performance of the sketch-to-mug-shot matching is good if the sketch image looks like the mug shot image in the database. However, in some situations, it is hard to construct a sketch that looks like the photograph. To overcome this limitation, this paper makes use of the concept of ldquohuman-in-the-looprdquo and proposes a human face image searching algorithm using relevance feedback in the second phase. Positive and negative samples will be collected from the user. A feedback algorithm that employs subspace linear discriminant analysis for online learning of the optimal projection for face representation is then designed and developed. The proposed system has been evaluated using the FERET database and a Japanese database with hundreds of individuals. The results are encouraging.

A new scheme for automated 3D PDM construction using deformable models

This paper describes a novel scheme to build a 3D Point Distribution Model (PDM) from a set of segmented training volumetric images. This approach is based on a deformable model algorithm to find correspondences across a set of surfaces/samples. It selects one sample as the template, and then deforms the template to approximate all other samples. These approximations carry the correspondences from the template to all other samples. The challenge is that a single template cannot guarantee accurate approximations to all other samples. The proposed solution is first to select the template sample, which brings the most accurate approximations to others among all samples. For each sample, which is not approximated accurately by the template, a “bridge” sample is chosen so that the bridge approximates accurately the current sample and the template approximates accurately to the bridge. The correspondences are then carried over from the template to the current sample via the “bridge”. A PDM is then constructed from the set of template’s approximations to all samples. This method is applied to construct four PDMs from 3D human brain Magnetic Resonance Images (MRIs). The four 3D PDMs constructed show considerable improvement on the approximation accuracy as compared to that constructed by adapting arbitrary templates. This improvement is important, as the approximation accuracy is the major concern of the deformable model-based approaches for the construction of PDMs.

TH‐C‐M100F‐04: A Deformable Lung Tumor Tracking Method in Fluoroscopic Video Using Active Shape Models

Purpose: Elastic tumor deformation and different intra‐ and inter‐fractional tumor paths between inhalation and exhalation were observed for some lung patients. For high accuracy and possibly 100% duty cycle dose delivery, we propose and evaluate a noninvasive method to fluoroscopically track location and shape variations of lung tumors with different types of deformations. Method and Materials: During a fluoroscopic simulation, lung tumor contours in one complete respiratory period are manually drawn by an expert. Each contour is described by 65 landmarks. The respiratory period is divided into 9 phases and a Point Distribution Models (PDM) statistically describing typical tumor shape variations are built for each phase. When tracking starts, the breathing phase for an incoming frame is first determined by the respiratory signal generated simultaneously from diaphragm motion and the PDM for this frame is also found. Starting from an initial estimate of the tumor contour, the Active Shape Models algorithm searches the area near each landmark and finds a better location. Based on the shifts found for these landmarks, the initial estimate is deformed within a certain range of typical shape variations found in PDM and also rigidly transformed to match the shifts. The new generated contour iteratively updates the previous contour estimate until no significant difference appears between two consecutive iterations or a user defined number of iterations is reached. Results: Tumors demonstrating distinct types of deformations in fluoroscopic videos were well tracked. All the landmarks of the tracked objects were manually revised by an expert using a GUI tool. The average magnitude of the deviation between the tracked and revised results was within 2 mm for 95% the landmarks and within 3mm for all landmarks. Conclusion: This method affords precise tracking of lung tumor location and deformation and may be used for real‐time tracking or DMLC radiotherapy.

Multiscale 3-D Shape Representation and Segmentation Using Spherical Wavelets

This paper presents a novel multiscale shape representation and segmentation algorithm based on the spherical wavelet transform. This work is motivated by the need to compactly and accurately encode variations at multiple scales in the shape representation in order to drive the segmentation and shape analysis of deep brain structures, such as the caudate nucleus or the hippocampus. Our proposed shape representation can be optimized to compactly encode shape variations in a population at the needed scale and spatial locations, enabling the construction of more descriptive, nonglobal, nonuniform shape probability priors to be included in the segmentation and shape analysis framework. In particular, this representation addresses the shortcomings of techniques that learn a global shape prior at a single scale of analysis and cannot represent fine, local variations in a population of shapes in the presence of a limited dataset. Specifically, our technique defines a multiscale parametric model of surfaces belonging to the same population using a compact set of spherical wavelets targeted to that population. We further refine the shape representation by separating into groups wavelet coefficients that describe independent global and/or local biological variations in the population, using spectral graph partitioning. We then learn a prior probability distribution induced over each group to explicitly encode these variations at different scales and spatial locations. Based on this representation, we derive a parametric active surface evolution using the multiscale prior coefficients as parameters for our optimization procedure to naturally include the prior for segmentation. Additionally, the optimization method can be applied in a coarse-to-fine manner. We apply our algorithm to two different brain structures, the caudate nucleus and the hippocampus, of interest in the study of schizophrenia. We show: 1) a reconstruction task of a test set to validate the expressiveness of our multiscale prior and 2) a segmentation task. In the reconstruction task, our results show that for a given training set size, our algorithm significantly improves the approximation of shapes in a testing set over the Point Distribution Model, which tends to oversmooth data. In the segmentation task, our validation shows our algorithm is computationally efficient and outperforms the Active Shape Model algorithm, by capturing finer shape details.

GAMEs: Growing and adaptive meshes for fully automatic shape modeling and analysis

This paper presents a new framework for shape modeling and analysis, rooted in the pattern recognition theory and based on artificial neural networks. Growing and adaptive meshes (GAMEs) are introduced: GAMEs combine the self-organizing networks which grow when require (SONGWR) algorithm and the Kohonen's self-organizing maps (SOMs) in order to build a mesh representation of a given shape and adapt it to instances of similar shapes. The modeling of a surface is seen as an unsupervised clustering problem, and tackled by using SONGWR (topology-learning phase). The point correspondence between point distribution models is granted by adapting the original model to other instances: the adaptation is seen as a classification task and performed accordingly to SOMs (topology-preserving phase). We thoroughly evaluated our method on challenging synthetic datasets, with different levels of noise and shape variations. Finally, we describe its application to the analysis of a challenging medical dataset. Our method proved to be reproducible, robust to noise, and capable of capturing real variations within and between groups of shapes.

A method to calculate coverage probability from uncertainties in radiotherapy via a statistical shape model

In this paper we describe a technique that may be used to model the geometric uncertainties that accrue during the radiotherapy process. Using data from in-treatment cone beam CT scans, we simultaneously analyse non-uniform observer delineation variability and organ motion together with patient set-up errors via the creation of a point distribution model (PDM). We introduce a novel method of generating a coverage probability matrix, that may be used to determine treatment margins and calculate uncertainties in dose, from this statistical shape model. The technique does not assume rigid body motion and can extrapolate shape variability in a statistically meaningful manner. In order to construct the PDM, we generate corresponding surface points over a set of delineations. Correspondences are established at a set of points in parameter space on spherically parameterized and canonical aligned outlines. The method is demonstrated using rectal delineations from serially acquired in-treatment cone beam CT image volumes of a prostate patient (44 image volumes total), each delineated by a minimum of two observers (maximum six). Two PDMs are constructed, one with set-up errors included and one without. We test the normality assumptions of the PDMs and find the distributions to be Gaussian in nature. The rectal PDM variability is in general agreement with data in the literature. The two resultant coverage probability matrices show differences as expected.