Segmentation Of Image Sequences Research Articles

A geodesic deformable model for automatic segmentation of image sequences applied to radiation therapy

Organ motion should be taken into account for image-guided fractionated radiotherapy. A deformable segmentation and registration method was developed for inter-and intra-fraction organ motion planning and evaluation. Energy minimizing active models were synthesized for tracking a set of organs delineated by regions of interest (ROI) in radiotherapy treatment. The initial model consists of a surface deformed to match the ROI contour by geometrical properties, following a heat flow model. The deformable segmentation model was tested using a Shepp-Logan head CT simulation, and different quantitative metrics were applied such as ROC analysis, Jaccard index, Dice coefficient and Hausdorff distance. Experimental evaluation of automated versus manual segmentation was done for the cardiac, thoracic and pelvic regions. The method has been quantitatively validated, obtaining an average of 93.3 and 99.2% for the sensitivity and specificity, respectively, 90.79% for the Jaccard index, 95.15% for the Dice coefficient and 0.96% mm for the Hausdorff distance. Model-based deformable segmentation was developed and tested for image-guided radiotherapy treatment planning. The method is efficient, robust and has sufficient accuracy for 2D CT data without markers.

Applications of Computed Tomography Angiography (CTA) in Neuroimaging

A novel image processing strategy is detailed for simultaneous measurement of tumor perfusion and neovascular morphology parameters from a sequence of dynamic contrast-enhanced ultrasound (DCE-US) images. After normalization and tumor segmentation, a global time–intensity curve describing contrast agent flow was analyzed to derive surrogate measures of tumor perfusion (i.e., peak intensity, time-to-peak intensity, area under the curve, wash-in rate, wash-out rate). A maximum intensity image was generated from these same segmented image sequences, and each vascular component was skeletonized via a thinning algorithm. This skeletonized data set and collection of vessel segments were then investigated to extract parameters related to the neovascular network and physical architecture (i.e., vessel-to-tissue ratio, number of bifurcations, vessel count, average vessel length and tortuosity). An efficient computation of local perfusion parameters was also introduced and operated by averaging time–intensity curve data over each individual neovascular segment. Each skeletonized neovascular segment was then color-coded by these local measures to produce a parametric map detailing spatial properties of tumor perfusion. Longitudinal DCE-US image data sets were collected in six patients diagnosed with invasive breast cancer using a Philips iU22 ultrasound system equipped with a L9-3 transducer and Definity contrast agent. Patients were imaged using US before and after contrast agent dosing at baseline and again at weeks 6, 12, 18 and 24 after treatment started. Preliminary clinical results suggested that breast tumor response to neoadjuvant chemotherapy may be associated with temporal and spatial changes in DCE-US-derived parametric measures of tumor perfusion. Moreover, changes in neovascular morphology parametric measures may also help identify any breast tumor response (or lack thereof) to systemic treatment. Breast cancer management from early detection to therapeutic monitoring is currently undergoing profound changes. Novel imaging techniques that are sensitive to the unique biological conditions of each individual tumor represent valuable tools in the pursuit of personalized medicine.

Remote Sensing Image Sequence Segmentation Based on the Modified Fuzzy C-means

Remote sensing image with characteristics of multiple gray level, more informative, fuzzy boundary, complex target structure and so on, there is no completely reliable model to guide the remote sensing image segmentation. In response to these issues, the article presents a remote sensing image sequence segmentation method based on improved FCM (fuzzy c-means) algorithm. The color space selects the lower relevance of HSI (hue, saturation, intensity) and adopts standard covariance matrix-the Mahalanobis distance formula, which is more suitable for the use of remote sensing image. It can solve the initial centers selection problems of fuzzy C-means clustering algorithm by the use of ECM. By using the partition of S component, it can divide the image into high S regions and low S regions. We can do FCM segmentation respectively with H component and I component of these two parts. The segmentation results can be achieved after the merger. The program experimental result shows that this method will enable FCM to converge to global optimal solution with less iteration, and has good stability and robustness. It has good effect on improving the accuracy of threshold segmentation and efficiency for remote sensing images, which can be used for content-based remote sensing image retrieval systems.

Rocket image sequence segmentation algorithm combined with edge detection and improved Otsu algorithm

To satisfy machine vision based rocket tracking servo system's requirements for rocket target segmentation,a segmentation method combined with Prewitt edge detection operator with decaying factor and improved Otsu algorithm was proposed. Based on the rocket's morphological characteristics,a nonlinear filter de-noising method was presented to denoise and remove the non-rocket edge. Simulation results show that the proposed method has 96.5% successful segmentation rate for rocket target,and can effectively segment rocket image sequence in real-time.

Watershed from propagated markers: An interactive method to morphological object segmentation in image sequences

In this paper, it is introduced an interactive method to object segmentation in image sequences, by combining classical morphological segmentation with motion estimation – the watershed from propagated markers. In this method, the objects are segmented interactively in the first frame and the mask generated by its segmentation provides the markers that will be used to track and segment the object in the next frame. Besides the interactivity, the proposed method has the following important characteristics: generality, rapid response and progressive manual edition. This paper also introduces a new benchmark to do quantitative evaluation of assisted object segmentation methods applied to image sequences. The evaluation is done according to several criteria such as the robustness of segmentation and the easiness to segment the objects through the sequence.

Joint Motion Estimation and Layer Segmentation in Transparent Image Sequences—Application to Noise Reduction in X-Ray Image Sequences

This paper is concerned with the estimation of the motions and the segmentation of the spatial supports of the different layers involved in transparent X-ray image sequences. Classical motion estimation methods fail on sequences involving transparent effects since they do not explicitly model this phenomenon. We propose a method that comprises three main steps: initial block-matching for two-layer transparent motion estimation, motion clustering with 3D Hough transform, and joint transparent layer segmentation and parametric motion estimation. It is validated on synthetic and real clinical X-ray image sequences. Secondly, we derive an original transparent motion compensation method compatible with any spatiotemporal filtering technique. A direct transparent motion compensation method is proposed. To overcome its limitations, a novel hybrid filter is introduced which locally selects which type of motion compensation is to be carried out for optimal denoising. Convincing experiments on synthetic and real clinical images are also reported.

A motion-based approach for segmenting dynamic textures

Characterisation of visual processes consisting of multiple dynamic textures is of vital importance to computer vision research, with a diverse set of applications in the field of robot navigation and remote monitoring applications, etc. In the current literature, however, studies are mostly limited to characterisation of single dynamic textures. In this paper, we aim to address the problem of segmenting image sequences consisting of multiple dynamic textures. More precisely, we separate image segments having different characteristic motion patterns – a key attribute of individual dynamic textures. Experimental results demonstrate the ability of the proposed technique by segmenting a wide variety of multiple dynamic texture image sequences.

Viscosity solutions for geodesic active contour under geometrical conditions

This paper addresses the issue of introducing geometrical constraints into segmentation processes in image analysis. This question has emerged from the analysis of classical tools: indeed, usual methods such as deformable models, fast marching prove to be fruitless when image data are missing or of poor quality or when the image owns homogeneous-textured regions. To cope with these hindrances, the idea developed hereafter consists of integrating geometrical constraints in the modelling to make the segmentation process easier to handle. We have devised a geodesic active contour-based model, in which we are trying to determine a curve that best approaches the given points (geometrical data) while detecting the object under consideration (see also Apparato et al., Segmentation of medical image sequence under constraints: application to non-invasive assessment of pulmonary arterial hypertension, Int. J. Comput. Math. 5(2004), pp. 527–536 and Gout and Vieira-Testé C. Gout and S. Vieira-Testé, An algorithm for segmentation under interpolation conditions using deformable models, Int. J. Comput. Math. 80(1) (2003), pp. 47–54 where other approaches are given for this kind of problems). In this paper the main results concern the study of the existence and uniqueness of the viscosity solution of this problem (following three different approaches). We also give numerical examples on real data sets.

Variational Segmentation of Image Sequences Using Region-Based Active Contours and Deformable Shape Priors

In this paper we address the problem of segmentation in image sequences using region-based active contours and level set methods. We propose a novel method for variational segmentation of image sequences containing nonrigid, moving objects. The method is based on the classical Chan-Vese model augmented with a novel frame-to-frame interaction term, which allow us to update the segmentation result from one image frame to the next using the previous segmentation result as a shape prior. The interaction term is constructed to be pose-invariant and to allow moderate deformations in shape. It is expected to handle the appearance of occlusions which otherwise can make segmentation fail. The performance of the model is illustrated with experiments on synthetic and real image sequences.

Cooperative Object Segmentation and Behavior Inference in Image Sequences

In this paper, we propose a general framework for fusing bottom-up segmentation with top-down object behavior inference over an image sequence. This approach is beneficial for both tasks, since it enables them to cooperate so that knowledge relevant to each can aid in the resolution of the other, thus enhancing the final result. In particular, the behavior inference process offers dynamic probabilistic priors to guide segmentation. At the same time, segmentation supplies its results to the inference process, ensuring that they are consistent both with prior knowledge and with new image information. The prior models are learned from training data and they adapt dynamically, based on newly analyzed images. We demonstrate the effectiveness of our framework via particular implementations that we have employed in the resolution of two hand gesture recognition applications. Our experimental results illustrate the robustness of our joint approach to segmentation and behavior inference in challenging conditions involving complex backgrounds and occlusions of the target object.

The rigidity in fibrin gels as a contributing factor to the dynamics of in vitro vascular cord formation

While the formation of vascular cords in in vitro angiogenesis assay is commonly used to test the angiogenic properties of many molecular or cellular components, an extensive characterisation of the dynamics of this process is still lacking. Up to now, quantitative studies only focused on the resulting capillary structures characterised through static morphometric approaches. We therefore propose in this paper a rather extensive characterisation aiming to identify different stages in the dynamics of this process, through the investigation of the influence of the rigidity of the fibrin extracellular matrix on the growth of the vascular cords. Using time lapse videomicroscopy, the time evolution of relevant morphodynamical parameters has been considered both at the cell level and at the cell population level. At the cell level, a trajectography analysis of individual cells observed in different locations of the growing network has been conducted and analysed using a random walk model. From image sequence analysis and segmentation i.e. extraction of the boundaries of the lacunae formed through matrix degradation and cell tractions, the evolution of the lacunae surface has been precisely quantified, revealing different phases and transitions in the growth patterns. Our results indicate that the rigidity of the extracellular fibrin matrix strongly influences the different stages, i.e. the dynamics of the angiogenic process. Consequently, optimal rigidity conditions for the formation of stable vascular cord networks could be identified in the context of our experiments.

Dynamical statistical shape priors for level set-based tracking

In recent years, researchers have proposed introducing statistical shape knowledge into level set-based segmentation methods in order to cope with insufficient low-level information. While these priors were shown to drastically improve the segmentation of familiar objects, so far the focus has been on statistical shape priors which are static in time. Yet, in the context of tracking deformable objects, it is clear that certain silhouettes (such as those of a walking person) may become more or less likely over time. In this paper, we tackle the challenge of learning dynamical statistical models for implicitly represented shapes. We show how these can be integrated as dynamical shape priors in a Bayesian framework for level set-based image sequence segmentation. We assess the effect of such shape priors "with memory" on the tracking of familiar deformable objects in the presence of noise and occlusion. We show comparisons between dynamical and static shape priors, between models of pure deformation and joint models of deformation and transformation, and we quantitatively evaluate the segmentation accuracy as a function of the noise level and of the camera frame rate. Our experiments demonstrate that level set-based segmentation and tracking can be strongly improved by exploiting the temporal correlations among consecutive silhouettes which characterize deforming shapes.

Object-based MPEG-2 video indexing and retrieval in a collaborative environment

In this paper, an object-based video retrieval methodology for search in large, heterogeneous video collections is presented. The proposed approach employs a real-time, compressed-domain, unsupervised algorithm for the segmentation of image sequences to spatiotemporal objects. For the resulting objects, MPEG-7 compliant low-level descriptors describing their color, shape, position and motion characteristics are extracted. These are automatically associated using a fuzzy C-means algorithm with appropriate intermediate-level descriptors, which are part of a simple vocabulary termed object ontology. Combined with a relevance feedback mechanism, this scheme allows the qualitative definition of the high-level concepts the user queries for (semantic objects, each represented by a keyword) and relations between them, facilitating the retrieval of relevant video segments. Furthermore, it allows the collaborative construction of a knowledge base by accumulating the information contributed to the system during feedback by different users. Thus, it enables faster and more accurate retrieval of commonly requested keywords or semantic objects. Experimental results in the context of a collaborative environment demonstrate the efficiency of the proposed video indexing and retrieval scheme.

Integrating region and boundary information for spatiallycoherent object tracking

The problem of segmenting image sequences based on 2D motion has been under study for many years now. Most early approaches were either region-based, doing some sort of robust motion estimation, or boundary-based, preferring instead to track the bounding contours of the moving image region. In this paper, we explore an approach based on a synergy between these two previous approaches. For example, while motion constraints are often in violation of their underlying assumptions at region boundaries, image edges are a rich source of information. The approach we propose uses feed-forward to use region-based information to propagate boundary estimates, feedback to use boundaries to improve motion estimation, and finally uses motion-based warping to compare image appearance between frames in order to provide additional information for the boundary estimation process. We show results from an implementation in which a hierarchical, layered-motion estimation using parametric models is coupled with a distance-transform based active contour. The system is shown to provide stable and accurate segmentation in sequences with background motion, and multiple moving objects. Quantitative measures are proposed and reported for these sequences. Finally, a modification is detailed which allows the system to incorporate a Condensation algorithm tracker, but without requiring off-line learning in advance.

Integration of deformable contours and a multiple hypotheses Fisher color model for robust tracking in varying illuminant environments

In this paper, we propose a new technique to perform figure-ground segmentation in image sequences of moving objects under varying illumination conditions. Unlike most of the algorithms that adapt color, there is not the assumption of smooth change of the viewing conditions. To cope with this, we propose the use of a new colorspace that maximizes the foreground/background class separability based on the ‘Linear Discriminant Analysis’ method. Moreover, we introduce a technique that formulates multiple hypotheses about the next state of the color distribution (some of these hypotheses take into account small and gradual changes in the color model and others consider more abrupt and unexpected variations) and the hypothesis that generates the best object segmentation is used to remove noisy edges from the image. This simplifies considerably the final step of fitting a deformable contour to the object boundary, thus allowing a standard snake formulation to successfully track non-rigid contours. In the same manner, the contour estimate is used to correct the color model. The integration of color and shape is done in a stage called ‘sample concentration’, introduced as a final step to the well-known CONDENSATION algorithm

Context-based segmentation of image sequences

We describe an algorithm for context-based segmentation of visual data. New frames in an image sequence (video) are segmented based on the prior segmentation of earlier frames in the sequence. The segmentation is performed by adapting a probabilistic model learned on previous frames, according to the content of the new frame. We utilize the maximum a posteriori version of the EM algorithm to segment the new image. The Gaussian mixture distribution that is used to model the current frame is transformed into a conjugate-prior distribution for the parametric model describing the segmentation of the new frame. This semisupervised method improves the segmentation quality and consistency and enables a propagation of segments along the segmented images. The performance of the proposed approach is illustrated on both simulated and real image data.

Space-time image sequence analysis: object tunnels and occlusion volumes

We address the issue of image sequence analysis jointly in space and time. While typical approaches to such an analysis consider two image frames at a time, we propose to perform this analysis jointly over multiple frames. We concentrate on spatiotemporal segmentation of image sequences and on analysis of occlusion effects therein. The segmentation process is three-dimensional (3-D); we search for a volume carved out by each moving object in the image sequence domain, or "object tunnel," a new space-time concept. We pose the problem in variational framework by using only motion information (no intensity edges). The resulting formulation can be viewed as volume competition, a 3-D generalization of region competition. We parameterize the unknown surface to be estimated, but rather than using an active-surface approach, we embed it into a higher dimensional function and apply the level-set methodology. We first develop simple models for the detection of moving objects over static background; no motion models are needed. Then, in order to improve segmentation accuracy, we incorporate motion models for objects and background. We further extend the method by including explicit models for occluded and newly exposed areas that lead to "occlusion volumes," another new space-time concept. Since, in this case, multiple volumes are sought, we apply a multiphase variant of the level-set method. We present various experimental results for synthetic and natural image sequences.

View Invariance for Human Action Recognition

This paper presents an approach for viewpoint invariant human action recognition, an area that has received scant attention so far, relative to the overall body of work in human action recognition. It has been established previously that there exist no invariants for 3D to 2D projection. However, there exist a wealth of techniques in 2D invariance that can be used to advantage in 3D to 2D projection. We exploit these techniques and model actions in terms of view-invariant canonical body poses and trajectories in 2D invariance space, leading to a simple and effective way to represent and recognize human actions from a general viewpoint. We first evaluate the approach theoretically and show why a straightforward application of the 2D invariance idea will not work. We describe strategies designed to overcome inherent problems in the straightforward approach and outline the recognition algorithm. We then present results on 2D projections of publicly available human motion capture data as well on manually segmented real image sequences. In addition to robustness to viewpoint change, the approach is robust enough to handle different people, minor variabilities in a given action, and the speed of aciton (and hence, frame-rate) while encoding sufficient distinction among actions.

A probabilistic approach for foreground and shadow segmentation in monocular image sequences

This paper presents a novel method of foreground and shadow segmentation in monocular indoor image sequences. The models of background, edge information, and shadow are set up and adaptively updated. A Bayesian network is proposed to describe the relationships among the segmentation label, background, intensity, and edge information. A maximum a posteriori—Markov random field estimation is used to boost the spatial connectivity of segmented regions.

Semi-automatic foreground/background segmentation of motion picture images and image sequences

Segmentation of images into foreground (an actor) and background is required for many motion picture special effects. To produce these shots, the unwanted background must be removed so that none of it appears in the final composite shot. The standard approach requires the background to be a blue screen. Systems that are capable of segmenting actors from more natural backgrounds have been proposed, but many of these are not readily adaptable to the resolution involved in motion picture imaging. An algorithm is presented that requires minimal human interaction to segment motion picture resolution images. Results from this algorithm are quantitatively compared with alternative approaches. Adaptations to the algorithm, which enable segmentation even when the foreground is lit from behind, are described. Segmentation of image sequences normally requires manual creation of a separate hint image for each frame of a sequence. An algorithm is presented that generates such hint images automatically, so that only a single input is required for an entire sequence. Results are presented that show that the algorithm successfully generates hint images where an alternative approach fails.