Image Sequence Analysis Research Articles

Qualitative Motion of Point-like Objects

In this paper, we introduce a framework to represent the course of motion of point-like objects qualitatively, i.e. by making only as many distinctions as necessary in a given context. The need to describe the course of motion arises in many applications, such as robotics and motion planning, automatic control and surveillance systems, image sequence analysis and indexing, and animation. We use a bottom-up approach that starts from basic descriptional units for orientation, length and velocity—integrated in the concept of qualitative motion vectors (QMVs)—and the operations on them. In order to describe sequences of QMVs we explore qualitative means of obtaining ordering information and of classifying significant turns. We then move on to issues of generalization discerning global, local and ‘turn-based’ variants. Finally, we develop techniques for identifying higher-level motion patterns such as closed loops and spirals.

Four-dimensional factor analysis of confocal image sequences (4D-FAMIS) to detect and characterize low copy numbers of human papillomavirus DNA by FISH in HeLa and SiHa cells.

Visualization and localization of specific DNA sequences were performed by fluorescence in situ hybridization, confocal laser scanning microscopy (CLSM), and four-dimensional factor analysis of biomedical image sequences (4D-FAMIS). HeLa and SiHa cells containing, respectively 20-50 and 1-2 copies per cell of human papillomavirus (HPV) DNA type 18 and 16 integrated in cellular DNA were used as models. HPV-DNA was identified using DNA probes containing the whole genome of HPV-DNA type 18 or 16, and DNA-DNA hybrids were revealed by alkaline phosphatase and Fast Red. Cell nuclei were counterstained with thiazole orange (TO) or TOTO-iodide. 4D image sequences were obtained using successive dynamic or spectral sequences of images on different optical sections from CLSM. The location of fluorescent signals within the preparations was determined by FAMIS. This original method summarizes image sequences into a reduced number of images called factor images, and curves called factors. Factors estimate different individual physical behaviours in the sequence such as extinction velocity, spectral patterns and depth emission profiles. Factor images correspond to spatial distributions of the different factors. We distinguished between Fast Red and nucleus stainings in HPV-DNA hybridization signals by taking into account differences in their extinction velocities (fluorescence decay rate) or spectral patterns, and in their focus (depth emission profiles). In HeLa cells, factor images showed that Fast-Red-stained targets could be distinguished from nucleus stainings, and were located on different focal planes of the nuclei. In SiHa cells, 4D-FAMIS determined as few as 1-2 copies per cell of HPV-DNA type 16 located in continuous focal planes. Therefore, 4D-FAMIS, together with CLSM, made the detection and characterization of low copy numbers of genes in whole cells possible.

Image Sequence Analysis via Partial Differential Equations

This article deals with the problem of restoring and motion segmenting noisy image sequences with a static background. Usually, motion segmentation and image restoration are considered separately i...

An Efficient Parallel Algorithm for Computing the Gaussian Convolution of Multi-dimensional Image Data

In this paper, we propose a parallel convolution algorithm for estimating the partial derivatives of 2D and 3D images on distributed-memory MIMD architectures. Exploiting the separable characteristics of the Gaussian filter, the proposed algorithm consists of multiple phases such that each phase corresponds to a separated filter. Furthermore, it exploits both the task and data parallelism, and reduces communication through data redistribution. We have implemented the proposed algorithm on the Intel Paragon and obtained a substantial speedup using more than 100 processors. The performance of the algorithm is also evaluated analytically. The analytical results confirming with the experimental results indicate that the proposed algorithm scales very well with the problem size and number of processors. We have also applied our algorithm to the design and implementation of an efficient parallel scheme for the 3D surface tracking process. Although our focus is on 3D image data, the algorithm is also applicable to 2D image data, and can be useful for a myriad of important applications including medical imaging, magnetic resonance imaging, ultrasonic imagery, scientific visualization, and image sequence analysis.

Estimation of global motion parameters by complex linear regression

Global motion is very likely to occur in image sequences analysis. For example, it arises if the observer is moving during the sequence acquisition (ego-motion). Our aim is to get a simple method to estimate in a reliable may a set of parameters that can take into account the presence of a global motion component, using only local information. The novelty of our approach is in regarding spatial shift, change of scale, and rotation (corresponding to usual camera effects such as pan and zoom) as a two-dimensional (2-D) Doppler effect. The mathematical treatment is carried on in the complex plane, so that the results can be easily deduced as an extension of the one-dimensional (1-D) case; in this way, we obtain simple expressions, well suited for a practical realization of the estimate. The method has been experimentally validated by both real pictures with a synthetic motion and real image sequences.

HOS-based method of global motion estimation for noisy image sequences

An investigation is presented into a new region-recursive higher-order statistics (HOS) based method for global motion estimation which uses a simplified affine motion model (SLM) of a region. Compared to the similar SOS method, the presented method is is better suited to the analysis of complex noisy image sequences contaminated by correlated gaussian noise (CGN).

Determining Motion Parameters for V ehicle-Mounted Camera Using Focus of Expansion

Camera's motion parameters play a vital role in the research of autonomous vehicle's vision navigation and dynamic image sequences analysis. We propose here a fast motion parameters' estimation method with the use of stationary camera's Focus of Expansion (FOE). Contrary to general methods like using optical flow and feature matching, our approach is able to determine camera's rotation and translation parameters by using only 3 pairs of matching points. A fast matching algorithm of feature points extracted by color gradient edge detector (CGD) and localized contour sequences (LCS) is also presented in this paper. Experiments on image sequences of synthetic and real road scenes showed the of fectiveness and precision of our approach.

An adaptable time-delay neural-network algorithm for image sequence analysis

In this letter we present an algorithm based on a time-delay neural network with spatio-temporal receptive fields and adaptable time delays for image sequence analysis. Our main result is that tedious manual adaptation of the temporal size of the receptive fields can be avoided by employing a novel method to adapt the corresponding time delay and related network structure parameters during the training process.

Independent component analysis of natural image sequences yields spatio-temporal filters similar to simple cells in primary visual cortex.

Simple cells in the primary visual cortex process incoming visual information with receptive fields localized in space and time, bandpass in spatial and temporal frequency, tuned in orientation, and commonly selective for the direction of movement. It is shown that performing independent component analysis (ICA) on video sequences of natural scenes produces results with qualitatively similar spatio-temporal properties. Whereas the independent components of video resemble moving edges or bars, the independent component filters, i.e. the analogues of receptive fields, resemble moving sinusoids windowed by steady Gaussian envelopes. Contrary to earlier ICA results on static images, which gave only filters at the finest possible spatial scale, the spatio-temporal analysis yields filters at a range of spatial and temporal scales. Filters centred at low spatial frequencies are generally tuned to faster movement than those at high spatial frequencies.

Quantitative analysis of the local rates of growth of dicot leaves at a high temporal and spatial resolution, using image sequence analysis

Summary A new technique is presented for quantitative mapping of dicot leaf growth at high spatial and temporal resolution, at a speed making online‐mapping feasible. Time lapse video sequences of growing leaves are captured by a personal computer (PC) with a frame‐grabber board and a standard CCD camera, and evaluated using algorithms that have been recently developed to analyse motion in dynamic image sequences. Growth can be detected at under 1% per hour, with a time resolution of minutes and a spatial resolution of a few millimeters. The new technique has been verified by comparing it with classical approaches to map integrated growth. Diurnal courses of leaf growth of Ricinus communis and tobacco are presented to demonstrate the localised character of growth in leaves. Expansion growth is restricted to the base of the leaf and is restricted to a few hours at the end of the night and the start of the day. The high resolution of the method is illustrated by mapping the responses to step‐changes in leaf turgor. A 3 bar turgor jump led to a rapid transient expansion over the entire length of the leaf that was partially reversed when the turgor was relaxed.

Integrating Spatio-temporal Information in Image Sequence Analysis to Enforce Consistency of Interpretation

We present a technique of image-sequence analysis which ensures consistency of interpretation in terms of the classification of segmented regions, both spatially within an image and temporally through a sequence of forward-looking infrared images taken from a low-flying aircraft. The technique is based on classical relaxation labeling techniques but is novel in that it treats segmented regions assingle entitiesthat have both spatial and temporal neighbors. This enables classification probabilities both to be propagated through the sequence and to be used in the minimization of the Kullback entropy of information. Sample results are shown from the application of the method to a 12-s sequence of infrared images that have been segmented using co-occurrence-based techniques. For completeness, the paper also describes the whole analysis algorithm including segmentation, texture analysis using discrete Hermite functions, and classification using neural network techniques.© British Crown copyright 1998. Published with the permission of the Defence Evaluation and Research Agency on behalf of the Controller of HMSO.

On-line analysis of echocardiographic image sequences

This paper presents a semiautomatic system for the interactive analysis of echocardiographic image sequences, able to provide useful information to cardiologists. The proposed approach combines well known techniques for the detection of left ventricular boundaries with the computation of optical flow. The initial detection of the cavity contour is based on an improved balloon model, with automatic tuning of parameters and optional model-based constraints. The computation of optical flow is performed with a fast correlation technique and the contour tracking is obtained combining motion information provided by the optical flow with model-based constraints and/or a snake-based regularization. The system is able to follow the motion of the ventricular boundaries precisely, to provide several quantitative features of the heartbeat and a dynamic representation of systolic and diastolic motion. Preliminary experimental results are presented and commented on with particular attention to their clinical relevance. Furthermore, a distributed implementation of the system suitable for remote examination analysis has been realized and applied to echo images of the carotid artery.

Learning identity with radial basis function networks

Radial basis function (RBF) networks are compared with other neural network techniques on a face recognition task for applications involving identification of individuals using low-resolution video information. The RBF networks are shown to exhibit useful shift, scale and pose ( y-axis head rotation) invariance after training when the input representation is made to mimic the receptive field functions found in early stages of the human vision system. In particular, representations based on difference of Gaussian (DoG) filtering and Gabor wavelet analysis are compared. Extensions of the techniques to the case of image sequence analysis are described and a time delay (TD) RBF network is used for recognising simple movement-based gestures. Finally, we discuss how these techniques can be used in real-life applications that require recognition of faces and gestures using low-resolution video images.

Statistical distribution of factors and factor images in factor analysis of medical image sequences

From a time or energy image sequence, factor analysis of medical image sequences (FAMIS) estimates factors, representing kinetics or spectra in a given physiological compartment, and associated factor images, showing the compartments corresponding to each curve. In this paper, we show that the statistical properties of factor images and associated factors can be determined using a well known result from elementary probability theory. Numerical experiments are conducted to demonstrate that the variance observed in factor images can be predicted when the statistical properties of the original data are known. It is shown how these theoretical results can be used to relax the non-negativity constraints during FAMIS oblique analysis and to improve the quantitative interpretation of the factor images by associating a confidence interval with each pixel value.

Motion-based analysis and segmentation of image sequences using 3-D scene models

In this paper we present an algorithm for automatic extraction and tracking of multiple objects from a video sequence. Our approach is model-based in the sense that we first use a robust structure-from-motion algorithm to identify multiple objects and to recover initial 3-D-shape models. Then, these models are used to identify and track the objects over multiple frames of the video sequence. The procedure starts with recovering a dense depth map of the scene using two frames at the beginning of the sequence, and representing the scene as a 3-D wire frame computed from the depth map. Texture extracted from the video frames is mapped onto the model. Once the initial models are available we use a linear and low-complexity algorithm to recover the motion parameters and scene structure of the objects for the subsequent frames. Combining the new estimates of depth and the initially computed 3-D models into an unstructured set of 3-D points with associated color information, we obtain updates of the 3-D scene description for each additional frame. We show that the usage of a 3-D scene model is suitable to analyze complex scenes with several objects. In our experimental results, we apply the approach presented in this paper to the problem of video sequence segmentation, object tracking, and video object plane (VOP) generation. We separate the video sequences into different layers of depth and combine the information from multiple frames to a compact and complete description of these layers.

Direct incremental model-based image motion segmentation for video analysis

Dynamic analysis of image sequences is an important task in object-oriented video applications. It often relies on the segmentation of each image of the sequence into region entities of apparent homogeneous motion. In this paper, we present an original motion segmentation algorithm based on 2D polynomial motion models, a multiresolution robust estimator to compute these motion models, and appropriate local observations supplying both motion relevant information and their reliability. Motion segmentation is formulated as a contextual statistical labeling problem exploiting multiscale Markov random field (MRF) models. One of its main features is that it avoids time consuming alternate iterations between motion model estimation and spatial support identification. An original detection step allows us to estimate and to update the number of required motion models, and thus to handle the appearance of new objects. Numerous experiments performed with real indoor and outdoor image sequences demonstrate the efficiency of the method.

Extensive operators in partition lattices for image sequence analysis

This paper deals with the class of extensive operators in the lattice of partitions. These operators are merging techniques. They can be used as filtering tools or as segmentation algorithms. In the first case, they are known as connected operators and, in the second case, they are region growing techniques. This paper discusses the basic elements that have to be defined to create a merging algorithm: merging order, merging criterion and region model. This analysis highlights the similarity and differences between a filtering tool, such as a connected operator, and a segmentation algorithm. Taking benefit from the filtering and segmentation viewpoints, we propose a general merging algorithm that can be used to create new connected operators (in particular self-dual operators) and efficient segmentation algorithms (robust criteria and efficient implementation).

Analysis of image sequences in fluorescence videomicroscopy of stationary objects.

Fluorescence videomicroscopy allows the temporal behavior of biological specimens to be studied at the cellular level. We describe two types of methods that can be used for extracting quantitative information from image sequences: the modeling approach, which is mainly local, and multivariate statistical analysis, which provides a global approach. The potentials for use of these two methods are illustrated through a simulation example and actual examples dealing with the study of chloride secretion by airway epithelial cells. We define some guidelines for making a choice between these two approaches, bearing in mind that a blend of these two methodologies is also possible.

Cooperative matching paradigm for the analysis of stereo image sequences

Cooperative matching paradigm for the analysis of stereo image sequences

Intracardiac echocardiography: computerized detection of left ventricular borders.

A semi-automated method for two- and three-dimensional analysis of intracardiac echocardiography (ICE) images and image sequences is reported based on detection of epicardial and endocardial borders using graph searching. The border detection method was applied to 50 ICE images acquired in vivo in five dogs and to 108 images in 16 volumetric ICE image sequences from eight cadaveric pig hearts. The ICE images from the in vivo study showed good correlation between computer-detected and observer-defined left ventricular (LV) cavity areas and epicardial areas (r = 0.99, y = 0.98x + 0.43 [cm2]; r = 0.99, y = 0.98x + 1.11 [cm2]; respectively). In the cadaveric hearts, the LV volumes were determined with the volume measurement error of 7.6 +/- 7.7% and 11.3 +/- 11.2% for the aortic valve and mitral valve image sequences, respectively. Our method facilitates an accurate and computationally efficient approach for the quantitative assessment of ICE image data in 2D and 3D.