Local Motion Estimation Research Articles

An integrated architecture for adaptive image stabilization in zooming operation

This paper presents a novel architecture for integrating digital stabilizer with the camera zooming process. The proposed architecture scheme allows integration for both optical and digital zooming operation scenarios. The stabilizer adjusts its operational parameters from the current optical zooming status for a refined local motion estimation. Subsequently, the global motion compensation vectors, produced by the digital stabilizer, are sent to the digital zooming system, where the image compensation is merged within the interpolation process. Experimental results indicate that the proposed architecture can improve not only the quantitative performance of digital stabilization but also the computational efficiency when zooming is employed.

Interlaced-to-progressive conversion using adaptive projection-based global and representative local motion estimation

We propose a motion-compensation-based deinterlacing algorithm using global and representative local motion estimation. The proposed algorithm first divides an entire image into five regions of interest (ROIs) according to the temporally predicted motion type (i.e., global or local) and the spatial position. One of them is for global motion estimation and the others are for local motion estimation. Then, dominant motions of respective ROIs are found by adaptive projection approach. The adaptive projection method not only estimates dominant local motions with low computational cost, but also ensures consistent global motion estimation. Using the estimated motion vectors, adaptive two-field bidirectional motion compensation is performed. The arbitration rules, measuring the reliability of motion compensation accurately, produce high-quality deinterlaced frames by effectively combining the results of motion compensation and the stable intrafield deinterlacing. Experimental results show that the proposed deinterlacing algorithm provides better image quality than the existing algorithms in both subjective and objective measures.

New Block-Based Motion Estimation for Sequences with Brightness Variation and Its Application to Static Sprite Generation for Video Compression

In this brief, a new local motion estimator is proposed which can accurately estimate motion activities under varying strong brightness conditions. The proposed estimator makes use of a new block division technique which manages practically to get rid of the adverse influence caused by brightness changes between frames. We also propose a new static sprite coding system using the proposed local motion estimator. The system is characterized not only with the features of accurate motion estimation under varying brightness conditions, but also possesses the capability of coding the brightness variability of the background scene using a single layered sprite image. Experimental results show that the resulting static sprite coding system improves the PSNR by 6.32 dB as compared with the conventional static sprite coding system when the background scenes of the video sequences involve strong brightness variations in the spatial and time domains.

Probabilistic Global Motion Estimation Based on Laplacian Two-Bit Plane Matching for Fast Digital Image Stabilization

Digital image stabilization (DIS) is a technique to prevent images captured by a handheld camera from temporal fluctuation. This paper proposes a new DIS algorithm that reduces the computation time while preserving the accuracy of the algorithm. To reduce the computation time, an image is transformed by a Laplacian operation and then converted into two one-bit spaces, called L+ and L- spaces. The computation time is reduced because only two-bits-per-pixel are used while the accuracy is maintained because the Laplacian operation preserves the edge information which can be efficiently used for the estimation of camera motion. Either two or four subimages in the corners of an image frame are selected according to the type of the image and five local motion vectors with their probabilities to be a global motion vector are derived for each subimage. The global motion vector is derived from these local motion vectors based on their probabilities. Experimental results show that the proposed algorithm achieves a similar or better accuracy than a conventional DIS algorithm using a local motion estimation based on a full-search scheme and MSE criterion while the complexity of the proposed algorithm is much less than the conventional algorithm.

Motion estimation of magnetic resonance cardiac images using the Wigner-Ville and hough transforms

Myocardial motion analysis and quantification is of utmost importance for analyzing contractile heart abnormalities and it can be a symptom of a coronary artery disease. A fundamental problem in processing sequences of images is the computation of the optical flow, which is an approximation of the real image motion. This paper presents a new algorithm for optical flow estimation based on a spatiotemporal-frequency (STF) approach. More specifically it relies on the computation of the Wigner-Ville distribution (WVD) and the Hough Transform (HT) of the motion sequences. The latter is a well-known line and shape detection method that is highly robust against incomplete data and noise. The rationale of using the HT in this context is that it provides a value of the displacement field from the STF representation. In addition, a probabilistic approach based on Gaussian mixtures has been implemented in order to improve the accuracy of the motion detection. Experimental results in the case of synthetic sequences are compared with an implementation of the variational technique for local and global motion estimation, where it is shown that the results are accurate and robust to noise degradations. Results obtained with real cardiac magnetic resonance images are presented.

Disambiguating Visual Motion by Form-Motion Interaction—a Computational Model

The neural mechanisms underlying motion segregation and integration still remain unclear to a large extent. Local motion estimates often are ambiguous in the lack of form features, such as corners or junctions. Furthermore, even in the presence of such features, local motion estimates may be wrong if they were generated near occlusions or from transparent objects. Here, a neural model of visual motion processing is presented that involves early stages of the cortical dorsal and ventral pathways. We investigate the computational mechanisms of V1-MT feedforward and feedback processing in the perception of coherent shape motion. In particular, we demonstrate how modulatory MT-V1 feedback helps to stabilize localized feature signals at, e.g. corners, and to disambiguate initial flow estimates that signal ambiguous movement due to the aperture problem for single shapes. In cluttered environments with multiple moving objects partial occlusions may occur which, in turn, generate erroneous motion signals at points of overlapping form. Intrinsic-extrinsic region boundaries are indicated by local T-junctions of possibly any orientation and spatial configuration. Such junctions generate strong localized feature tracking signals that inject erroneous motion directions into the integration process. We describe a simple local mechanism of excitatory form-motion interaction that modifies spurious motion cues at T-junctions. In concert with local competitive-cooperative mechanisms of the motion pathway the motion signals are subsequently segregated into coherent representations of moving shapes. Computer simulations demonstrate the competency of the proposed neural model.

An oblique effect for local motion: Psychophysics and natural movie statistics

Human perception of visual motion is thought to involve two stages--estimation of local motion (i.e., of small features) and global motion (i.e., of larger objects)--identified with cortical areas V1 and MT, respectively. We asked if poor discrimination of oblique compared to cardinal directions (the oblique effect for motion; OEM) reflects a deficit in local or in global motion processing. We used an equivalent noise (EN) paradigm--where one measures direction discrimination thresholds in the presence of directional variability--to quantify local and global limits. We report that the OEM diminishes with increasing directional variability, indicating that global motion processing (the number of local motion signals pooled) is equal across all directions and that the OEM is attributable to anisotropies in local motion processing. To investigate the origin of this effect, we measured local motion statistics from natural movies (filmed from the point of view of a walking observer). This analysis reveals that the distribution of local directional energy on the oblique directions tends to be broader, and frequently more asymmetric, than on the cardinal directions. If motion detectors are optimized to deal with our visual world then such anisotropies likely explain the local nature of the OEM.

Video de-interlacing by adaptive 4-field global/local motion compensated approach

A de-interlacing algorithm using adaptive 4-field global/local motion compensated approach is presented. It consists of block-based directional edge interpolation, same-parity 4-field motion detection, global/local motion estimation and compensation. The edges are sharper when the directional edge interpolation is adopted. The same parity 4-field motion detection and the 4-field local motion estimation detect the static areas and fast motion by four reference fields, and the global motion estimation detects the camera panning and zooming motions. The global and local motion compensation recover the interlaced videos to the progressive ones. Experimental results show that the peak signal-to-noise ratio of our proposed algorithm is 2/spl sim/3 dB higher than that of previous studies and attain the best quality of subjective view.

Kinematical models to reduce the effect of skin artifacts on marker-based human motion estimation

The estimation of the skeletal motion obtained from marker-based motion capture systems is known to be affected by significant bias caused by skin movement artifacts, which affects joint center and rotation axis estimation. Among different techniques proposed in the literature, that based on rigid body model, still the most used by commercial motion capture systems, can smooth only part of the above effects without eliminating their main components. In order to sensibly improve the accuracy of the motion estimation, a novel technique, named local motion estimation (LME), is proposed. This rests on a recently described approach that, using virtual humans and extended Kalman filters, estimates the kinematical variables directly from 2D measurements without requiring the 3D marker reconstruction. In this paper, we show how such method can be extended to include the computation of the local marker displacement due to skin artifacts. The 3D marker coordinates, expressed in the corresponding local reference coordinate frames, are inserted into the state vector of the filter and their dynamics is automatically estimated, with adequate accuracy, without assuming any particular deformation function. Simulated experiments of lower limb motion, involving systematic mislocations (5, 10, 20 mm) and random errors of the marker coordinates and joint center locations (±5, ±10, ±15 mm), have shown that artifact motion can be substantially decoupled from the global skeletal motion with an effective increase of the accuracy wrt standard techniques. In particular, the comparison between the nominal kinematical variables and the one recovered from markers attached to the skin surface proved LME to be sensibly superior (50% in the worse condition) to the methods imposing marker-bone rigidity. In conclusion, while requiring further validation on real movement data, we argue that the proposed method can constitute an appropriate approach toward the improvement of the human motion estimation.

Simultaneous localization and mapping with stereovision

This paper presents a simultaneous localization and mapping approach based on an extended kalman filter, using only a set of non-registered stereovision image pairs. Invariant image point features are used to detect 3D landmarks and to provide local motion estimates. The 3D coordinates of each point feature are computed by stereovision, and an interest point matching algorithm solves the data association problem. The estimation of the errors on the local motion estimations, on the landmark initialisations and on the landmark observations are depicted, and results with low altitude aerial images are presented.

Integration of form and motion within a generative model of visual cortex

One of the challenges faced by the visual system is integrating cues within and across processing streams for inferring scene properties and structure. This is particularly apparent in the inference of object motion, where psychophysical experiments have shown that integration of motion signals, distributed across space, must also be integrated with form cues. This has led several to conclude that there exist mechanisms which enable form cues to ‘veto’ or completely suppress ambiguous motion signals. We describe a probabilistic approach which uses a generative network model for integrating form and motion cues using the machinery of belief propagation and Bayesian inference. We show, using computer simulations, that motion integration can be mediated via a local, probabilistic representation of contour ownership, which we have previously termed ‘direction of figure’. The uncertainty of this inferred form cue is used to modulate the covariance matrix of network nodes representing local motion estimates in the motion stream. We show with results for two sets of stimuli that the model does not completely suppress ambiguous cues, but instead integrates them in a way that is a function of their underlying uncertainty. The result is that the model can account for the continuum of bias seen for motion coherence and perceived object motion in psychophysical experiments.

Shot Partitioning Based Recognition of TV Commercials

Digital video applications exploit the intrinsic structure of video sequences. In order to obtain and represent this structure for video annotation and indexing tasks, the main initial step is automatic shot partitioning. This paper analyzes the problem of automatic TV commercials recognition, and a new algorithm for scene break detection is then introduced. The structure of each commercial is represented by the set of its key-frames, which are automatically extracted from the video stream. The particular characteristics of commercials make commonly used shot boundary detection techniques obtain worse results than with other video content domains. These techniques are based on individual image features or visual cues, which show significant performance lacks when they are applied to complex video content domains like commercials. We present a new scene break detection algorithm based on the combined analysis of edge and color features. Local motion estimation is applied to each edge in a frame, and the continuity of the color around them is then checked in the following frame. By separately considering both sides of each edge, we rely on the continuous presence of the objects and/or the background of the scene during each shot. Experimental results show that this approach outperforms single feature algorithms in terms of precision and recall.

Motion segmentation based on motion/brightness integration and oscillatory correlation

A segmentation method based on the integration of motion and brightness is proposed for image sequences. The method is composed of two parallel pathways that process motion and brightness, respectively. Inspired by the visual system, the motion pathway has two stages. The first stage estimates local motion at locations with reliable information. The second stage performs segmentation based on local motion estimates. In the brightness pathway, the input scene is segmented into regions based on brightness distribution. Subsequently, segmentation results from the two pathways are integrated to refine motion estimates. The final segmentation is performed in the motion network based on refined estimates. For segmentation, locally excitatory globally inhibitory oscillator network (LEGION) architecture is employed whereby the oscillators corresponding to a region of similar motion/brightness oscillate in synchrony and different regions attain different phases. Results on synthetic and real image sequences are provided, and comparisons with other methods are made.

A new fast local motion estimation algorithm using global motion

A new fast local motion estimation (ME) method is proposed for the two-stage estimation algorithm of global and local motions. The local motion part is extracted from the block MV (Motion Vector) used in the global ME and it is used as an initial MV in the local ME, resulting in a remarkable reduction of the search range for the local ME.

An efficient coding method for the local motion vector by using global motion

A new fast local motion estimation (ME) method is proposed for the two-stage estimation algorithm of global and local motions. The local motion part is extracted from the block motion used in the global ME and it is used as an initial motion vector in the local ME, resulting in a remarkable reduction of the search range for the local ME.

An efficient motion estimator with application to medical image registration

Image registration is a very important problem in computer vision and medical image processing. Numerous algorithms for registering single and multi-modal image data have been reported in these areas. Robustness as well as computational efficiency are prime factors of importance in image data registration. In this paper, a robust/reliable and efficient algorithm for estimating the transformation between two image data sets of a patient taken from the same modality over time is presented. Estimating the registration between two image data sets is formulated as a motion-estimation problem. We use a hierarchical optical flow motion model which allows for both global as well as local motion between the data sets. In this hierarchical motion model, we represent the flow field with a B -spline basis which implicitly incorporates smoothness constraints on the field. In computing the motion, we minimize the expectation of the squared differences energy function numerically via a modified Newton iteration scheme. The main idea in the modified Newton method is that we precompute the Hessian of the energy function at the optimum without explicitly knowing the optimum. This idea is used for both global and local motion estimation in the hierarchical motion model. We present examples of motion estimation on synthetic and real data (from a patient acquired during pre- and post-operative stages) and compare the performance of our algorithm with that of competing ones.

An efficient motion estimator with application to medical image registration

Image registration is a very important problem in computer vision and medical image processing. Numerous algorithms for registering single and multi-modal image data have been reported in these areas. Robustness as well as computational efficiency are prime factors of importance in image data registration. In this paper, a robust/reliable and efficient algorithm for estimating the transformation between two image data sets of a patient taken from the same modality over time is presented. Estimating the registration between two image data sets is formulated as a motion-estimation problem. We use a hierarchical optical flow motion model which allows for both global as well as local motion between the data sets. In this hierarchical motion model, we represent the flow field with a B-spline basis which implicitly incorporates smoothness constraints on the field. In computing the motion, we minimize the expectation of the squared differences energy function numerically via a modified Newton iteration scheme. The main idea in the modified Newton method is that we precompute the Hessian of the energy function at the optimum without explicitly knowing the optimum. This idea is used for both global and local motion estimation in the hierarchical motion model. We present examples of motion estimation on synthetic and real data (from a patient acquired during pre- and post-operative stages) and compare the performance of our algorithm with that of competing ones.

Surface Recovery from Range Images Using Curvature and Motion Consistency

A key problem in the recovery of scene descriptions from multiple views is the fusion of information from different vantage points. The contribution of this paper is a set of algorithms for reconstructing surfaces obtained from overlapping range images in a common frame of reference. Surfaces are assumed to be piecewise-smooth but not necessarily rigid. Motion parameters, rotations and translations that describe correspondence between views, are recovered locally under the assumption that the curvature structure at a point on a surface varies slowly under transformation. The recovery problem can thus be posed as finding the set of motion parameters that preserves curvature across two views. We show that an appropriate similarity functional can be devised that is convex in the vicinity of the true motion parameters. This leads to an efficient local algorithm that recovers motion parameters in gradient descent fashion. Fusion of information from different viewponts is accomplished by applying local motion estimates to map data points between frames. However, because these estimates are determined locally, they are subject to the usual effects of noise and quantization error. To increase the robustness of this reconstruction procedure the additional constraint of motion consistency is introduced, that variations in the velocities of adjacent regions are also piecewise-smooth. This is cast as a second local minimization problem which seeks to find the set of motion parameters that minimizes differences in the relative positions and orientations at adjacent points. The resulting algorithm serves to smooth out local perturbations and blend adjacent surface patches. In contrast to global rigid body motion approaches, our procedure for reconstructing surfaces from different viewpoints is tolerant of local errors in correspondence and can accommodate objects that are articulated.

3-D motion estimation and wireframe adaptation including photometric effects for model-based coding of facial image sequences

Proposes a novel formulation where 3D global and local motion estimation and the adaptation of a generic wireframe model to a particular speaker are considered simultaneously within an optical flow based framework including the photometric effects of the motion. We use a flexible wireframe model whose local structure is characterized by the normal vectors of the patches which are related to the coordinates of the nodes. Geometrical constraints that describe the propagation of the movement of the nodes are introduced, which are then efficiently utilized to reduce the number of independent structure parameters. A stochastic relaxation algorithm has been used to determine optimum global motion estimates and the parameters describing the structure of the wireframe model. Results with both simulated and real facial image sequences are provided. >

Reversible interframe compression of medical images: a comparison of decorrelation methods

The authors investigate whether data representing medical image sequences can be compressed more efficiently by taking into account the temporal correlation between the sequence frames. The standard of comparison is intraframe HINT, the best-known reversible decorrelation method for 2-D images. In interframe decorrelation, a distinction is made between extrapolation- and interpolation-based methods, and methods based on local motion estimation, block motion estimation, and unregistered decorrelation. These distinctions give six classes of interframe decorrelation methods, all of which are described. The methods are evaluated by applying them to sequences of coronary X-ray angiograms, ventricle angiograms, and liver scintigrams, as well as to a (nonmedical) videoconferencing image sequence. For the medical image sequences: (1) interpolation-based methods are superior to extrapolation-based methods; (2) estimation of interframe motion is not advantageous for image compression; (3) interframe compression yields entropies comparable to intraframe HINT at higher computational costs; and (4) two methods, unregistered extrapolation and interpolation, are nonetheless possibly interesting alternatives to intraframe HINT.