Deformable Template Research Articles

A nonlinear elastic deformable template for soft structure segmentation: application to the heart segmentation in MRI

This paper proposes a nonlinear 3D deformable model for the image segmentation of soft structures. The template is modelled as an elastic body which is deformed by forces derived from the image. It relies on a template, which is a topological, geometrical and material model of the structure to segment. This model is based on the nonlinear three-dimensional elasticity problem with a boundary condition of pure traction. In addition, the applied forces are successive, as they depend on the displacements. For computations, an incremental algorithm is proposed to minimize the global energy of template deformation. Sufficient conditions of the convergence for the incremental algorithm are given. Finally, a discrete algorithm using the finite element method is presented and evaluated on synthetic images and actual MR images of mouse hearts.

A proposal of improved lip contour extraction method using deformable template matching and its application to dental treatment

AbstractStudies of facial parts have been conducted in recent years in various fields of medical science and psychology, including studies of the lip contour and location. In fact, in the treatment of cleft lip patients in oral surgery, it is desirable to extract the precise lip contour and to identify its location on the face in order to plan the plastic surgery procedure and to follow up the site after surgery. Aiming at applications to dental treatment, in this study we attempt to extend the lip contour extraction procedure based on the deformable template to the case of a template with left– right asymmetry. An algorithm is also proposed for adjusting the hue and value thresholds in order to improve extraction accuracy and reduce the burden on the user. The proposal is evaluated using sample images of normal subjects and cleft lip patients. It is shown that the shape and location of the lip can be extracted precisely and automatically by the proposed method. In connection with application to dental clinics, we present an example of a quantitative evaluation of the lip shape by using the parameters extracted by the proposed method. © 2007 Wiley Periodicals, Inc. Syst Comp Jpn, 38(5): 80– 89, 2007; Published online in Wiley InterScience (www.interscience.wiley.com). DOI 10.1002/scj.10668

POP: Patchwork of Parts Models for Object Recognition

We formulate a deformable template model for objects with an efficient mechanism for computation and parameter estimation. The data consists of binary oriented edge features, robust to photometric variation and small local deformations. The template is defined in terms of probability arrays for each edge type. A primary contribution of this paper is the definition of the instantiation of an object in terms of shifts of a moderate number local submodels--parts--which are subsequently recombined using a patchwork operation, to define a coherent statistical model of the data. Object classes are modeled as mixtures of patchwork of parts POP models that are discovered sequentially as more class data is observed. We define the notion of the support associated to an instantiation, and use this to formulate statistical models for multi-object configurations including possible occlusions. All decisions on the labeling of the objects in the image are based on comparing likelihoods. The combination of a deformable model with an efficient estimation procedure yields competitive results in a variety of applications with very small training sets, without need to train decision boundaries--only data from the class being trained is used. Experiments are presented on the MNIST database, reading zipcodes, and face detection.

Dynamic quantization for belief propagation in sparse spaces

Graphical models provide an attractive framework for modeling a variety of problems in computer vision. The advent of powerful inference techniques such as belief propagation (BP) has recently made inference with many of these models tractable. Even so, the enormous size of the state spaces required for some applications can create a heavy computational burden. Pruning is a standard technique for reducing this burden, but since pruning is irreversible it carries the risk of greedily deleting important states, which can subsequently result in gross errors in BP. To address this problem, we propose a novel extension of pruning, which we call dynamic quantization (DQ), that allows BP to adaptively add as well as subtract states as needed. We examine DQ in the context of graphical-model based deformable template matching, in which the state space size is on the order of the number of pixels in an image. The combination of BP and DQ yields deformable templates that are both fast and robust to significant occlusions, without requiring any user initialization. Experimental results are shown on deformable templates of planar shapes. Finally, we argue that DQ is applicable to a variety of graphical models in which the state spaces are sparsely populated.

Towards a Coherent Statistical Framework for Dense Deformable Template Estimation

SummaryThe problem of estimating probabilistic deformable template models in the field of computer vision or of probabilistic atlases in the field of computational anatomy has not yet received a coherent statistical formulation and remains a challenge. We provide a careful definition and analysis of a well-defined statistical model based on dense deformable templates for grey level images of deformable objects. We propose a rigorous Bayesian framework for which we prove asymptotic consistency of the maximum a posteriori estimate and which leads to an effective iterative estimation algorithm of the geometric and photometric parameters in the small sample setting. The model is extended to mixtures of finite numbers of such components leading to a fine description of the photometric and geometric variations of an object class. We illustrate some of the ideas with images of handwritten digits and apply the estimated models to classification through maximum likelihood.

Continuous Medial Representation for Anatomical Structures

The m-rep approach pioneered by Pizer et al. (2003) is a powerful morphological tool that makes it possible to employ features derived from medial loci (skeletons) in shape analysis. This paper extends the medial representation paradigm into the continuous realm, modeling skeletons and boundaries of three-dimensional objects as continuous parametric manifolds, while also maintaining the proper geometric relationship between these manifolds. The parametric representation of the boundary-medial relationship makes it possible to fit shape-based coordinate systems to the interiors of objects, providing a framework for combined statistical analysis of shape and appearance. Our approach leverages the idea of inverse skeletonization, where the skeleton of an object is defined first and the object's boundary is derived analytically from the skeleton. This paper derives a set of sufficient conditions ensuring that inverse skeletonization is well-posed for single-manifold skeletons and formulates a partial differential equation whose solutions satisfy the sufficient conditions. An efficient variational algorithm for deformable template modeling using the continuous medial representation is described and used to fit a template to the hippocampus in 87 subjects from a schizophrenia study with sub-voxel accuracy and 95% mean overlap.

Image segmentation using voronoi polygons and MCMC, with application to muscle fibre images

We investigate a Bayesian method for the segmentation of muscle fibre images. The images are reasonably well approximated by a Dirichlet tessellation, and so we use a deformable template model based on Voronoi polygons to represent the segmented image. We consider various prior distributions for the parameters and suggest an appropriate likelihood. Following the Bayesian paradigm, the mathematical form for the posterior distribution is obtained (up to an integrating constant). We introduce a Metropolis–Hastings algorithm and a reversible jump Markov chain Monte Carlo algorithm (RJMCMC) for simulation from the posterior when the number of polygons is fixed or unknown. The particular moves in the RJMCMC algorithm are birth, death and position/colour changes of the point process which determines the location of the polygons. Segmentation of the true image was carried out using the estimated posterior mode and posterior mean. A simulation study is presented which is helpful for tuning the hyperparameters and to assess the accuracy. The algorithms work well on a real image of a muscle fibre cross-section image, and an additional parameter, which models the boundaries of the muscle fibres, is included in the final model.

A robust lane detection and tracking method based on computer vision

This paper presents a robust method designed to detect and track a road lane from images provided by an on-board monocular monochromatic camera. The proposed lane detection approach makes use of a deformable template model to the expected lane boundaries in the image, a maximum a posteriori formulation of the lane detection problem, and a Tabu search algorithm to maximize the posterior density. The model parameters completely determine the position of the host vehicle within the lane, its heading direction and the local structure of the lane ahead. Based on the lane detection result in the first frame of the image sequence, a particle filter, having multiple hypotheses capability and performing nonlinear filtering, is used to recursively estimate the lane shape and the vehicle position in the sequence of consecutive images. Experimental results reveal that the proposed lane detection and tracking method is robust against broken lane markings, curved lanes, shadows, strong distracting edges, and occlusions in the captured road images.

Construction Method of Three-Dimensional Deformable Template Models for Tree-Shaped Organs

In this paper, we propose a construction method of three-dimensional deformable models that represent tree-shaped human organs, such as bronchial tubes, based on results obtained by statistically analyzing the distributions of bifurcation points in the tree-shaped organs. The models are made to be used as standard templates of tree-shaped organs in medical image recognition, and are formed by control points that can be uniquely identified as structural elements of organs such as bifurcation tracheae in bronchial tubes. They can be transfigured based on the statistical validity of relationships between the control points. The optimal state of that transfiguration is determined within the framework of energy minimization. Experimental results from bronchial tubes are shown on actual CT images.

Caenorhabditis elegans Egg-Laying Detection and Behavior Study Using Image Analysis

Egg laying is an important phase of the life cycle of the nematode Caenorhabditis elegans (C. elegans). Previous studies examined egg-laying events manually. This paper presents a method for automatic detection of egg-laying onset using deformable template matching and other morphological image analysis techniques. Some behavioral changes surrounding egg-laying events are also studied. The results demonstrate that the computer vision tools and the algorithm developed here can be effectively used to study C. elegans egg-laying behaviors. The algorithm developed is an essential part of a machine-vision system for C. elegans tracking and behavioral analysis.

The bi-elliptical deformable contour and its application to automated tongue segmentation in Chinese medicine

Automated tongue image segmentation, in Chinese medicine, is difficult due to two special factors: 1) there are many pathological details on the surface of the tongue, which have a large influence on edge extraction; 2) the shapes of the tongue bodies captured from various persons (with different diseases) are quite different, so they are impossible to describe properly using a predefined deformable template. To address these problems, in this paper, we propose an original technique that is based on a combination of a bi-elliptical deformable template (BEDT) and an active contour model, namely the bi-elliptical deformable contour (BEDC). The BEDT captures gross shape features by using the steepest decent method on its energy function in the parameter space. The BEDC is derived from the BEDT by substituting template forces for classical internal forces, and can deform to fit local details. Our algorithm features fully automatic interpretation of tongue images and a consistent combination of global and local controls via the template force. We apply the BEDC to a large set of clinical tongue images and present experimental results.

A Survey of Elastic Matching Techniques for Handwritten Character Recognition

This paper presents a survey of elastic matching (EM) techniques employed in handwritten character recognition. EM is often called deformable template, flexible matching, or nonlinear template matching, and defined as the optimization problem of two-dimensional warping (2DW) which specifies the pixel-to-pixel correspondence between two subjected character image patterns. The pattern distance evaluated under optimized 2DW is invariant to a certain range of geometric deformations. Thus, by using the EM distance as a discriminant function, recognition systems robust to the deformations of handwritten characters can be realized. In this paper, EM techniques are classified according to the type of 2DW and the properties of each class are outlined. Several topics around EM, such as the category-dependent deformation tendency of handwritten characters, are also discussed.

Automated 3-D Reconstruction of the Surface of Live Early-Stage Amphibian Embryos

Although three-dimensional (3-D) reconstructions of the surfaces of live embyos are vital to understanding embryo development, morphogenetic tissue movements and other factors have prevented the automation of this task. Here, we report an integrated set of software algorithms that overcome these challenges, making it possible to completely automate the reconstruction of embryo surfaces and other textured surfaces from multiview images. The process involves: 1) building accurate point correspondences using a robust deformable template block matching algorithm; 2) removing outliers using fundamental matrix calculations in conjunction with a RANSAC algorithm; 3) generating 3-D point clouds using a bundle adjustment algorithm that includes camera position and distortion corrections; 4) meshing the point clouds into triangulated surfaces using a Tight Cocone algorithm that produces water tight models; 5) refining surfaces using midpoint insertion and Laplacian smoothing algorithms; and 6) repeating these steps until a measure of convergence G, the rms difference between successive reconstructions, is below a specified threshold. Reconstructions were made of 2.2-mm diameter, neurulation-stage axolotl (amphibian) embryos using 44 multiview images collected with a robotic microscope. A typical final model (sixth iteration) contained 3787 points and 7562 triangles and had an error measure of G = 5.9 microm.

Image recognition via deformable templates

High quality handwritten numeral recognition is currently seen as an essential capability in the area of office automation. Statistical algorithms applied to this task can have a very significant impact because they offer the possibility of generalizing the information contained in a training set to new examples, without requiring an extensive programming effort tailored to specific hand-written styles. In this thesis, we apply the deformable probabilistic template approach to a problem of recognizing handwritten digits. Unlike previous published works in this area, the template is fed directly with images, rather than feature vectors, thus demonstrating the ability of Bayesian models to deal with large amounts of global and high-level information.

Representation and detection of deformable shapes

We describe some techniques that can be used to represent and detect deformable shapes in images. The main difficulty with deformable template models is the very large or infinite number of possible nonrigid transformations of the templates. This makes the problem of finding an optimal match of a deformable template to an image incredibly hard. Using a new representation for deformable shapes, we show how to efficiently find a global optimal solution to the nonrigid matching problem. The representation is based on the description of objects using triangulated polygons. Our matching algorithm can minimize a large class of energy functions, making it applicable to a wide range of problems. We present experimental results of detecting shapes in medical images and images of natural scenes. Our method does not depend on initialization and is very robust, yielding good matches even in images with high clutter. We also consider the problem of learning a nonrigid shape model for a class of objects from examples. We show how to learn good models while constraining them to be in the form required by the matching algorithm.

Deformable Template과 GA를 이용한 얼굴 인식 및 아바타 자동 생성

본 논문에서는 아바타를 자동으로 생성하기 위한 컬러 이미지 상에서의 얼굴, 눈, 입술 윤곽선 검출 기법을 제안하였다. 제안된 기법에서는 먼저 조명의 영향을 최대한 배제하기 위하여 HSI 색상 모델을 사용하였고 I 정보를 제외한 HS 평면상에서 피부색을 정의하고 이를 이용하여 입력된 이미지로부터 피부 영역을 검출하였다. 그리고 변형가능 템플릿과 유전자 알고리즘을 이용하여 얼굴, 눈, 입의 윤곽선을 검출하였다. 여기서 변형가능 템플릿은 B-spline 곡선과 컨트롤 포인트 벡터로 이루어지며, 이것은 다양한 얼굴, 눈, 입술 모양의 표현을 가능하게 한다. 또 유전자 알고리즘은 자연계의 진화와 선택원리를 응용한 매우 효율적인 탐색 알고리즘이다 다음으로, 검출된 얼굴과 각 요소들의 윤곽선과 퍼지 C-평균 군집화를 이용하여 아바타를 생성하게 된다. 퍼지 C-평균 군집화는 얼굴색을 일정한 수로 단순화하는 과정에서 사용하였다. 결과적으로, 이와 같은 기법을 이용하여 기존의 정해진 이미지를 가지고 표현하던 아바타와는 달리 사용자의 특성을 표현할 수 있는 아바타를 자동으로 생성할 수 있다. This paper proposes the method to detect contours of a face, eyes and a mouth in a color image for making an avatar automatically. First, we use the HSI color model to exclude the effect of various light condition, and we find skin regions in an input image by using the skin color is defined on HS-plane. And then, we use deformable templates and Genetic Algorithm(GA) to detect contours of a face, eyes and a mouth. Deformable templates consist of B-spline curves and control point vectors. Those can represent various shape of a face, eyes and a mouth. And GA is very useful search procedure based on the mechanics of natural selection and natural genetics. Second, an avatar is created automatically by using contours and Fuzzy C-means clustering(FCM). FCM is used to reduce the number of face color As a result, we could create avatars like handmade caricatures which can represent the user's identity, differing from ones generated by the existing methods.

Fitting and Manipulating Freeform Shapes Using Templates

Finding effective and efficient tools for complex freeform shape modification continues to be a challenging problem in computer graphics and computer-aided design. Although current approaches give reasonable results, their computation time and complexity often prevent their further development in more complex cases, especially in reusing an existing design. In this paper, for a better control of existing freeform shapes, deformable freeform feature templates are introduced. By the advantage of a small number of intrinsic parameters, a given freeform shape can be quickly approximated by one of the deformable templates. The deformable templates are further developed to track and match complex freeform shapes, resulting in extendable templates. With mappings, the original shape and the approximated template are associated. Thus, further shape manipulations can be conducted effectively using high-level intrinsic shape parameters. Experiments were carried out to verify the proposed algorithms. It is also described how the matching and manipulating techniques can be applied in computer graphics and computer-aided design applications.

A non‐parametric 2D deformable template classifier

AbstractWe introduce an interactive segmentation method for a sea floor survey. The method is based on a deformable template classifier and is developed to segment data from an echo sounder post‐processor called RoxAnn.RoxAnn collects two different measures for each observation point, and in this 2D feature space the ship‐master will be able to interactively define a segmentation map, which is refined and optimized by the deformable template algorithms.The deformable templates are defined as two‐dimensional vector‐cycles. Local random transformations are applied to the vector‐cycles, and stochastic relaxation in a Bayesian scheme is used. In the Bayesian likelihood a class density function and its estimate hereof is introduced, which is designed to separate the feature space.The method is verified on data collected in Øresund, Scandinavia. The data come from four geographically different areas. Two areas, which are homogeneous with respect to bottom type, are used for training of the deformable template classifier, and the classifier is applied to two areas, which are heterogeneous with respect to bottom type.The classification results are good with a correct classification percent above 94 per cent for the bottom type classes, and show that the deformable template classifier can be used for interactive on‐line sea floor segmentation of RoxAnn echo sounder data. Copyright © 2004 John Wiley & Sons, Ltd.

Analysis and synthesis of textured motion: particles and waves

Natural scenes contain a wide range of textured motion phenomena which are characterized by the movement of a large amount of particle and wave elements, such as falling snow, wavy water, and dancing grass. In this paper, we present a generative model for representing these motion patterns and study a Markov chain Monte Carlo algorithm for inferring the generative representation from observed video sequences. Our generative model consists of three components. The first is a photometric model which represents an image as a linear superposition of image bases selected from a generic and overcomplete dictionary. The dictionary contains Gabor and LoG bases for point/particle elements and Fourier bases for wave elements. These bases compete to explain the input images and transfer them to a token (base) representation with an O(10(2))-fold dimension reduction. The second component is a geometric model which groups spatially adjacent tokens (bases) and their motion trajectories into a number of moving elements--called "motons." A moton is a deformable template in time-space representing a moving element, such as a falling snowflake or a flying bird. The third component is a dynamic model which characterizes the motion of particles, waves, and their interactions. For example, the motion of particle objects floating in a river, such as leaves and balls, should be coupled with the motion of waves. The trajectories of these moving elements are represented by coupled Markov chains. The dynamic model also includes probabilistic representations for the birth/death (source/sink) of the motons. We adopt a stochastic gradient algorithm for learning and inference. Given an input video sequence, the algorithm iterates two steps: 1) computing the motons and their trajectories by a number of reversible Markov chain jumps, and 2) learning the parameters that govern the geometric deformations and motion dynamics. Novel video sequences are synthesized from the learned models and, by editing the model parameters, we demonstrate the controllability of the generative model.

Estimating blood vessel areas in ultrasound images using a deformable template model

We consider the problem of obtaining interval estimates of vessel areas from ultrasound images of cross sections through the carotid artery. Robust and automatic estimates of the cross sectional area is of medical interest and of help in diagnosing atherosclerosis, which is caused by plaque deposits in the carotid artery. We approach this problem by using a deformable template to model the blood vessel outline, and use recent developments in ultrasound science to model the likelihood. We demonstrate that by using an explicit model for the outline, we can easily adjust for an important feature in the data: strong edge reflections called specular reflection. The posterior is challenging to explore, and naive standard MCMC algorithms simply converge too slowly. To obtain an efficient MCMC algorithm we make extensive use of computational efficient Gaussian Markov random fields, and use various block sampling constructions that jointly update large parts of the model.