Deformable Shape Model Research Articles

WE-AB-BRA-05: Fully Automatic Segmentation of Male Pelvic Organs On CT Without Manual Intervention

Purpose: We aim to develop a fully automatic tool for accurate contouring of major male pelvic organs in CT images for radiotherapy without any manual initialization, yet still achieving superior performance than the existing tools. Methods: A learning-based 3D deformable shape model was developed for automatic contouring. Specifically, we utilized a recent machine learning method, random forest, to jointly learn both image regressor and classifier for each organ. In particular, the image regressor is trained to predict the 3D displacement from each vertex of the 3D shape model towards the organ boundary based on the local image appearance around the location of this vertex. The predicted 3D displacements are then used to drive the 3D shape model towards the target organ. Once the shape model is deformed close to the target organ, it is further refined by an organ likelihood map estimated by the learned classifier. As the organ likelihood map provides good guideline for the organ boundary, the precise contouring Result could be achieved, by deforming the 3D shape model locally to fit boundaries in the organ likelihood map. Results: We applied our method to 29 previously-treated prostate cancer patients, each with one planning CT scan. Compared with manually delineated pelvic organs, our method obtains overlap ratios of 85.2%±3.74% for the prostate, 94.9%±1.62% for the bladder, and 84.7%±1.97% for the rectum, respectively. Conclusion: This work demonstrated feasibility of a novel machine-learning based approach for accurate and automatic contouring of major male pelvic organs. It shows the potential to replace the time-consuming and inconsistent manual contouring in the clinic. Also, compared with the existing works, our method is more accurate and also efficient since it does not require any manual intervention, such as manual landmark placement. Moreover, our method obtained very similar contouring results as the clinical experts. Project is partially support by a grant from NCI 1R01CA140413

Image Processing in Radiation Therapy

This article reviews Image Processing in Radiation Therapy by Kristy Brock Editor: Series Editor: William R. Hendee , Boca Raton, FL, 2013. Hardcover 286 pp. Price: $103.96. ISBN: 9781439830178.

Non-Rigid Structure-From-Motion on Degenerate Deformations With Low-Rank Shape Deformation Model

Non-rigid structure-from-motion (NRSfM) is the process of recovering time-varying 3D structures and poses of a deformable object from an uncalibrated monocular video sequence. Currently, most NRSfM algorithms utilize a non- degenerate assumption for non-rigid object deformations whereby the 3D structures of a non-rigid object can be assumed to be a linear combination of basis shapes with full rank three. Unfortunately, this assumption will produce extra degrees-of-freedom when the non-rigid object has some degenerate deformations with shape bases of rank less than three. These extra degrees-of-freedom will yield spurious shape deformations due to non-negligible noise in real applications, which will cause substantial reconstruction errors. To solve this problem, we propose a low-rank shape deformation model to represent 3D structures of degenerate deformations. When modeling degenerate deformations, the proposed model exploits the rank-deficient nature of degenerate deformations in addition to the low-rank property of non-rigid objects' trajectories, thus providing a more accurate and compact representation compared with existing models. Based on this model, we formulate the NRSfM problem as two coherent optimization problems. These problems are solved with iterative non-linear optimization algorithms. Experiments on synthetic and motion capture data are conducted. The results exhibit the significant advantages of our approach over state-of-the-art NRSfM algorithms for the 3D recovery of non-rigid objects with degenerate deformations.

Parametric active contour model using Gabor balloon energy for texture segmentation

Active contour models (ACM) as deformable shape models are one of the popular methods in object detection and image segmentation. This article presents a robust texture-based segmentation method using parametric ACM. In the proposed method, the energy function of the parametric ACM is modified by adding texture-based balloon energy, so the accurate detection and segmentation of textured object in textured background would be achieved. In this study, texture features of contour, object, and background points are calculated by Gabor filter bank. Then, comparing the calculated texture features of contour points and target object obtains movement direction of the balloon, whereupon active contour curves are shrunk or expanded to make the contour fit to object boundaries. The comparison between our proposed segmentation method and the ACM based on the directional Walsh– Hadamard features, fast adaptive color snake model, and parametric texture model based on joint statistics of complex Wavelet coefficients, indicates that our method is more effective, accurate, and faster for texture image segmentation especially when the textures are irregular or texture direction of object and background is similar.

Modeling and Estimation of Shape Deformation for Topology-Preserving Object Tracking

In this work we propose a topology-preserving framework for tracking deforming objects in a sequence of video frames. The shape of the object is modeled as obtained through the action of a group of diffeomorphisms on the initial object boundary. We formulate a state-space model for the diffeomorphic deformation of the object and implement a particle filter on this shape space to estimate the state of the shape in each video frame. We use a practical method for sampling diffeomorphic shapes in which we generate deformations via flows of finitely generated vector fields. Based on the observations and the proposed samples we obtain an approximate estimate for the posterior distribution of the shape.

Online tracking of deformable objects under occlusion using dominant points

This paper deals with tracking of deformable objects in the presence of occlusion using dominant point representation of the boundary contour. A novel nonintegral time propagation model for propagating the dominant points is proposed. It uses an initial guess generated from a linear operation and an analytical conjugate gradient approach for online robust learning of the shape deformation and motion model. A scheme is presented to automatically detect and correct the region of large local deformation. In order to deal with occlusion, admissible restrictions on deformation and motion of the object are automatically determined. The proposed method overcomes the need of offline learning and learns the deformation and motion model of the object using very few initial frames of the input video. The performance of the method is demonstrated using varieties of videos of different objects.

Multi-Kernel Appearance Model

Automatic facial landmarking is a crucial prerequisite of many applications dedicated to face analysis. In this paper we describe a two-step method. In a first step, each landmark position in the image is predicted independently. To achieve fast and accurate localizations, we implement detectors based on a two-stage classifier and we use multiple kernel learning algorithms to combine multi-scale features. In a second step, to increase the robustness of the system, we introduce spatial constraints between landmarks. To this end, parameters of a deformable shape model are optimized using the first step outputs through a Gauss–Newton algorithm. Extensive experiments have been carried out on different databases (PIE, LFPW, Cohn-Kanade, Face Pix and BioID), assessing the accuracy and the robustness of the proposed approach. They show that the proposed algorithm is not significantly affected by small rotations, facial expressions or natural occlusions and can be favorably compared with the current state of the art landmarking systems.

Reality based modeling and simulation of gallbladder shape deformation using variational methods

Accurate soft tissue deformation modeling is important for realistic surgical simulation. The aim of this study is to develop a reality-based gallbladder model and to determine material constants that represent gallbladder wall mechanical properties. Mechanical experiments on porcine gallbladder were performed to investigate tissue deformation, and an exponential strain energy function was used to describe the nonlinear stress-strain behavior of the gallbladder wall. A new volumetric function based upon the exponential strain energy function was proposed to model the gallbladder organ. A genetic algorithm was used to identify the material parameters of the proposed biomechanical model from the experimental data. The material constants of the exponential strain energy model were determined based on the experimental data. Deformation simulation and haptic rendering using the proposed gallbladder model were presented. Comparison between deformation predicted by the proposed model and that of the experimental data on gallbladder wall and gallbladder organ tissues demonstrates the applicability of this reality-based variational method for deformation simulation. An accurate soft tissue deformation model was developed using material constants identified for gallbladder. The model is suitable for interactive haptic rendering and deformation simulation. This model has potential applications for simulation of other hollow organs.

Sparse Representation Shape Models

It is well-known that, during shape extraction, enrolling an appropriate shape constraint model could effectively improve locating accuracy. In this paper, a novel deformable shape model, Sparse Representation Shape Models (SRSM), is introduced. Rather than following commonly utilized statistical shape constraints, our model constrains shape appearance based on a morphological structure, the convex hull of aligned training samples, i.e., only shapes that could be linearly represented by aligned training samples with the sum of coefficients equal to one, are defined as qualified. This restriction strictly controls shape deformation modes to reduce extraction errors and prevent extremely poor outputs. This model is realized based on sparse representation, which ensures during shape regularization the maximum valuable shape information could be reserved. Besides, SRSM is interpretable and hence helpful to further understanding applications, such as face pose recognition. The effectiveness of SRSM is verified on two publicly available face image datasets, the FGNET and the FERET.

The active geometric shape model: A new robust deformable shape model and its applications

We present a novel approach for fitting a geometric shape in images. Similar to active shape models and active contours, a force field is used in our approach. But the object to be detected is described with a geometric shape, represented by parametric equations. Our model associates each parameter of this geometric shape with a combination of integrals (summations in the discrete case) of the force field along the contour. By iteratively updating the shape parameters according to these integrals, we are able to find the optimal fit of the shape in the image. In this paper, we first explore simple cases such as fitting a line, circle, ellipse or cubic spline contour using this approach. Then we employ this technique to detect the cross-sections of subarachnoid spaces containing cerebrospinal fluid (CSF) in phase-contrast magnetic resonance (PC-MR) images, where the object of interest can be described by a distorted ellipse. The detection results can be further used by an s–t graph cut to generate a segmentation of the CSF structure. We demonstrate that, given a properly configured geometric shape model and force field, this approach is robust to noise and defects (disconnections and non-uniform contrast) in the image. By using a geometric shape model, this approach does not rely on large training datasets, and requires no manual labeling of the training images as is needed when using point distribution models.

Approche hybride combinant champs de Markov et modèle statistique de forme pour l’extraction des contours de la prostate en IRM

Recent progress in magnetic resonance imaging (MRI) has enabled new prostate cancer diagnosis techniques. The newest challenges in this field are to enhance image-based tumours detection. In such a context, the extraction of prostate's contours is a crucial step in the interpretation of MR images, and is usually carried out by an expert radiologist. This is though a tedious time consuming task, especially in 3D images (like CT and MRI). In addition, manual delineation is not reproducible because of differences between observers. In this paper, we introduce a novel method for automatic segmentation of prostate MRI that could help physicians in extracting 3D outlines of the gland. First a deformable shape model is used to obtain a first segmentation. The latter is refined using intensity information and Markov Random Fields modelling of regions. We use the Iterative Conditional Mode for optimising voxels’ labelling according to a Maximum A Posteriori criterion. Results from evaluation on patients’ data show that the method is satisfyingly accurate, fast and robust which makes it suitable for use in a clinical context. A multicentric validation and transfer to the industry would bring the contributions of this method to clinical routine and help improving diagnosis of prostate cancer.

A Hierarchical Connection Graph Algorithm for Gable-Roof Detection in Aerial Image

In this letter, we present a hierarchical connection graph (HCG) algorithm based on a self-avoiding polygon (SAP) model for detecting and extracting gable roofs from aerial imagery. The SAP model is a deformable shape model that is capable of representing gable roofs of various shapes and appearances. The model is composed of a sequence of roof-corner templates that are connected into a SAP, which serves as a flexible shape prior. An energy function that combines features from three channels (corner, boundary, and interior area) is defined over the sequence to quantify the variability in appearances of gable roofs. To infer the most probable state of the corner sequence for an input image, we use an efficient algorithm-called HCG algorithm. The algorithm converts the solution space of a SAP model into a directed graph (which we call “HCG”) and searches for the best path using dynamic programming (DP). It is efficient for two reasons: 1) By constructing an HCG, the algorithm can quickly prune out a large amount of invalid solutions using only geometric constraints, which are inexpensive to compute, and 2) by employing DP, the algorithm decomposes the searching problem into smaller overlapping subproblems and reuses energy scores, which are expensive to compute. Experimental results on a set of challenging gable roofs show that our algorithm has good performance and is computationally effective.

Assessment and stenting of tracheal stenosis using deformable shape models

This work presents a decision support system for the assessment of tracheal stenosis. In the proposed method, a statistical shape model of healthy tracheas is registered to a 3D CT image of a patient with tracheal stenosis. The registration yields an estimation of the shape of the patient's trachea as if stenosis was not present. From this point, the extent and the severity of the stenosis is assessed and stent parameters are obtained automatically. The method was extensively evaluated on simulation as well on real data and the results showed that it is accurate and fast enough to be used in the clinical setting.

A Bayesian Approach to Automated Creation of Tactile Facial Images

Portrait photos (facial images) play important social and emotional roles in our life. This type of visual media is unfortunately inaccessible by users with visual impairment. This paper proposes a systematic approach for automatically converting human facial images into a tactile form that can be printed on a tactile printer and explored by a user who is blind. We propose a deformable Bayesian Active Shape Model (BASM), which integrates anthropometric priors with shape and appearance information learnt from a face dataset. We design an inference algorithm under this model for processing new face images to create an input-adaptive face sketch. Further, the model is enhanced by input-specific details through semantic-aware processing. We report experiments on evaluating the accuracy of face alignment using the proposed method, with comparison with other state-of-the-art results. Furthermore, subjective evaluations of the produced tactile face images were performed by 17 persons including six visually-impaired users, confirming the effectiveness of the proposed approach in conveying via haptics vital visual information in a face image.

SU-FF-I-90: A Clinical Evaluation of the M-Rep-Based Automatic Prostate Segmentation

Purpose: To evaluate the clinical application readiness of a statistically trainable deformable shape model, called an m‐rep, for automatic segmentation of the prostate from CTimages by comparison with manual contouring. Method and Materials: ConStruct, an in‐house automatic prostate segmentation tool featuring m‐rep (medial representations) models, was used in this study. This study is to test the robustness of the m‐rep models for prostate treatment planning segmentation in routine use. For each patient case, an m‐rep was initialized and automatically deformed to segment the prostate. Ground truth was taken to be the contour set used for treatment planning. Two radiation oncologists each manually created an additional contour set. The average distance between m‐rep segmentation (or manual segmentation) and the ground truth was calculated. Retrospective clinical validation was also carried out in all cases. Two radiation oncologists blindly scored all the contour sets including the ground truth. Score 1: perfect contour—no corrections needed; 2: minor correction—less than one third of the slices require correction; 3: major correction—more efficient than redrawing all the contours from scratch; 4: not acceptable. Results: Averaged over five patients, the average distances between m‐rep, set 1 and 2 manual contour sets and the ground truth are 0.51, 0.45, and 0.42 cm, respectively. The average scores for m‐rep, set 1, set 2 contour sets, and the ground truth are 2.4, 2.4, 2.0, and 2.1, respectively. Conclusion: The qualitative and clinical validations were performed on m‐rep‐based automatic segmentation by comparing it to manual contouring. M‐reps appeared to produce reasonable and acceptable prostate contour sets when compared to those generated by clinicians. We propose that m‐reps be used to improve clinical efficiency by automatically contouring the prostate as a starting shape and then manually editing when needed. Further validation on a larger patient cohort is indicated.

Metamorphs: Deformable Shape and Appearance Models

This paper presents a new deformable modeling strategy aimed at integrating shape and appearance in a unified space. If we think traditional deformable models as active contours or evolving curve fronts, the new deformable shape and appearance models we propose are deforming disks or volumes. Each model has not only boundary shape but also interior appearance. The model shape is implicitly embedded in a higher dimensional space of distance transforms, thus represented by a distance map image. In this way, both shape and appearance of the model are defined in the pixel space. A common deformation scheme, the Free Form Deformations (FFD), parameterizes warping deformations of the volumetric space in which the model is embedded in, hence deforming both model boundary and interior simultaneously.

Activity Representation Using 3D Shape Models

We present a method for characterizing human activities using 3D deformable shape models. The motion trajectories of points extracted from objects involved in the activity are used to build models for each activity, and these models are used for classification and detection of unusual activities. The deformable models are learnt using the factorization theorem for nonrigid 3D models. We present a theory for characterizing the degree of deformation in the 3D models from a sequence of tracked observations. This degree, termed as deformation index (DI), is used as an input to the 3D model estimation process. We study the special case of ground plane activities in detail because of its importance in video surveillance applications. We present results of our activity modeling approach using videos of both high-resolution single individual activities and ground plane surveillance activities.

SU‐FF‐I‐58: A Software Toolkit for Multi‐Image Registration and Segmentation in IGRT and ART

Purpose: Extracting information from images for image‐guided and adaptive radiation therapy usually involves segmentation and/or registration with a reference image. The purpose of this research was to develop and make freely available a software toolkit for research use that combines several registration and segmentation methods and is applicable to problems involving multiple sequential images. Method and Materials: Two automatic approaches have been integrated with standard interactive methods. One automatic approach solves PDEs describing visco‐elastic flow to register a set of target images with a reference image. Segmentations can then be transferred from the reference image to target images via corresponding deformation fields. The second method segments images via Bayesian posterior optimization of a 3D deformable shape model called an m‐rep. User efficiency has been improved by the ability to queue, view, work with, and compare multiple images and segmentations during a single session. The software was developed in C++ and runs on Windows and Linux. Results: The GUI, including a window with panes for standard orthogonal 2D views linked to a second 3D window, preserves a common look and feel for all three approaches. A set of treatment images of the same patient initially are rigidly registered to a reference image. The images are then segmented by the methods of choice. Unsatisfactory automatic results are edited with conventional methods. Registered images and segmentations are output in DICOM RT format. Conclusion: The toolkit combines multiple approaches for image registration and segmentation in a system specifically designed to support research on IGRT and ART. A cost‐free license to download and use the software is available at http://planunc.radonc.unc.edu.Research supported by NCI/NCRR R01 RR 01861501.

WE‐C‐I‐609‐07: On Constructing Priors and Likelihoods for Deformable Shape Models

Purpose: Explicit deformable shape models (DSMs) can be used in a Bayesian statistical framework to provide a priori information for posterior optimization to match the DSM against a target image for automatic segmentation. In this approach a DSM is initialized in the target image and undergoes a series of deformations to closely match the target object. Deformation is driven by optimizing an objective function with terms for geometric typicality (prior) and model‐to‐image match (likelihood). The purpose of this work was to develop strategy, methodology, and tools for constructing the geometric prior and intensity likelihood for a particular form of DSM called m‐reps. Method and Materials: Geometric truth is defined for an object of interest by a statistically significant collection of expert human segmentations of training images. M‐reps are fit to the human drawn contours by minimizing the distance between the surfaces of the m‐rep and the contours under added conditions that lead to positional correspondence across training cases. The geometry of the resulting set of training m‐reps is analyzed in non‐Euclidean space using an approach called principal geodesic analysis (PGA) to yield a set of eigenmodes that define the geometric prior. The intensity likelihood is constructed by registering each training m‐rep with the corresponding gray scale image and collecting regional intensity information that is statistically characterized over all training cases. The intensity information can be in several forms including linear profiles and regional histograms. Results: PGA produces modes that include natural deformations such as local twisting, bending, bulging, and constricting. Unlike analysis in Euclidean space, improper shapes are avoided. The form of the intensity prior can be customized to each object of interest for optimal performance. Conclusion: These methods are powerful, robust and generalizable to other DSMs. Conflict of Interest: The presenting author has a financial interest in Morphormics, Inc.

Asymmetry analysis of deformable hippocampal model using the principal component in schizophrenia.

The hippocampus is thought to play an important role in learning and memory processing, and impairments in memory, attention, and decision making are found commonly in schizophrenia. Although many studies have reported decreases in hippocampal volume in the left hemisphere in schizophrenia, regionally specific hippocampal volume loss has not been revealed consistently using volume analysis. Recently, many studies have analyzed shape asymmetry using 3-D models; however, inconsistent results have been reported, mainly due to methodologic differences. We therefore used an active, flexible, deformable shape model for surface parameterization, and compared shape asymmetry based on principal component analysis (PCA) in the hippocampi of schizophrenic patients with those of the normal controls. Although the overall pattern of the statistical results did not change according to the number of principal components, the reconstructed results based on six major components were much more distinguishable. Although the left hemispheric hippocampal volume was larger than the right hemispheric was in this study, the difference was not significant. In shape asymmetry analysis, the right hemisphere hippocampus was bilaterally larger than the left hemisphere hippocampus was in the head of the superior CA1 and smaller in the tail and head of the inferior CA1. The asymmetry in the schizophrenia group was statistically smaller than that in the control group through reduction of the left hemisphere hippocampus volume.