Global Motion Estimation Research Articles

Video stabilisation using local salient feature in particle filter framework

A novel digital video stabilisation approach that uses salient features and particle filter for global motion estimation is proposed in this paper. In this approach, the local salient features are first gained by the improved K-means clustering method using the features obtained from the Scale Invariant Feature Transform SIFT feature points extracted from the down-sampled images, and then the salient regions are located in the video frames. The trajectory of the features extracted from the salient regions is used to estimate global motion between frames. A new similarity function called SRMSE Salient Region Mean Square Error is proposed in particle filter framework to reduce the computational cost. Motion compensation yields stabilised video sequences using the estimates. Experimental results demonstrate that the proposed algorithm has good performance and improves the efficiency significantly.

Non-linear factorised dynamic shape and appearance models for facial expression analysis and tracking

Facial expressions exhibit non-linear shape and appearance deformations with variations in different people and expressions. The authors present a non-linear factorised shape and appearance model for facial expression analysis and tracking. The novel non-linear factorised generative model of facial expressions, using conceptual manifold embedding and empirical kernel maps, provides accurate facial expression shape and appearance. It preserves non-linear facial deformations based on the configuration, face style and expression type. The proposed model supports tasks, such as facial expression recognition, person identification and global and local facial motion tracking. Given a sequence of images, temporal embedding, expression type and person identification parameters are iteratively estimated for facial expression analysis. The authors combine global facial motion estimation and local facial deformation estimation for large global and subtle local facial motion tracking. The authors employ local facial motion deformation estimation using a thin-plate spline for subtle facial motion tracking. The global shape and appearance model provides appearance templates for the estimation of local deformation. Experimental results using Cohen-Kanade AU-coded facial expressions demonstrate facial expression recognition using estimated personal style parameter, and facial deformation tracking using global and local facial motion estimation.

A novel video salient object extraction method based on visual attention

This paper addresses a novel approach to automatically extract video salient objects based on visual attention mechanism and seeded object growing technique. First, a dynamic visual attention model to capture the object motions by global motion estimation and compensation is constructed. Through combining it with a static attention model, a saliency map is formed. Then, with a modified inhibition of return (MIOR) strategy, the winner-take-all (WTA) neural network is used to scan the saliency map for the most salient locations selected as attention seeds. Lastly, the particle swarm optimization (PSO) algorithm is employed to grow the attention objects modeled by Markov random field (MRF) from the seeds. Experiments verify that our presented approach could extract both of stationary and moving salient objects efficiently.

Fast feature-based video stabilization without accumulative global motion estimation

This paper presents a novel digital video stabilization approach that provides both efficiency and robustness. In this approach, features of each frame are first detected by the oriented features from accelerated segment test (FAST) method, and then the corresponding features between consecutive frames are matched by a very fast binary descriptor which is based on the rotated binary robust independent elementary features (BRIEF). The oriented FAST combined with the rotated BRIEF, which is called ORB, is very efficient in feature detection and matching, and can be used to speed up the motion estimation without any hardware acceleration. In addition, an improved motion smoothing method is proposed to smooth affine model based motion parameters without accumulative global motion estimation. Unlike the conventional method, the proposed method uses unstable input frames and stabilized output frames instead of original input frames to estimate motion parameters directly, allowing for more desirable motion parameters. Experiments with a variety of videos demonstrate that the proposed approach is both efficient and robust.

A Qualitative Approach to Camera Motion Classification

This paper proposed a robust and hierarchical camera motion classification approach based on integrated histogram. First, a motion direction histogram is built, and the information entropy of histogram is employed to classify camera motion operations into scaling operation and non-scaling operation. And then, static and translation operations are distinguished by distribu- tion information of histogram, meanwhile the direction of translation operation is also identified. Experimental results show that the new approach can efficiently achieve global motion estimation.

A subsampling-predictor associated approach for fast global motion estimation

Global Motion Estimation (GME) has many important roles in numerous applications, such as video compression, image stabilization, video-object segmentation, and etc. One well-known GME method is the gradient-based technique. This method uses optimization techniques, like the Levenberg–Marquardt algorithm, to minimize estimation error. Such algorithms require an initial value for the initializing step. In this paper, we propose a simple and reliable GME structure with a new predictor. This structure uses a three-step search and a predictor for the initializing step. It is also incorporated with a fast GME method that uses pixel subsampling. This incorporation reduces the computational complexity of GME without a significant loss of accuracy. This structure has less computational complexity and similar accuracy versus common methods.

Robust video stabilization based on particle filtering with weighted feature points

Camera global motion estimation is critical to the success of video stabilization. This paper presents an effective and robust feature based motion estimation method. In the proposed approach, feature points are collected from input video sequences based on Speeded Up Robust Features (SURF). Random Samples Consensus (RANSAC) is used to remove local motion vectors and incorrect correspondences. In the global motion estimation, a particle filter is used to estimate the weight of feature points, solving the issue of Different Depth of Field (DDOF) for feature points. Then, the weighted least square (WLS) algorithm is applied to obtain the global motion estimation. Finally, a Kalman filter estimates the intentional motion, and the unintentional motion is compensated to obtain stable video sequences. Experimental results show that the proposed algorithm has the characteristics of high precision and good robustness.

A Low-Complexity Image Registration Algorithm for Global Motion Estimation

An important recent application of image registration is the estimation of global motion parameters for object-based video coding. However, the main disadvantage of standard approaches to global motion estimation (GME) is the increased computational complexity with the higher degree of motion models when compared to block-based local motion estimation approaches. In this paper, we propose a new low complexity GME algorithm. In our proposed algorithm, full-precision images are replaced with 1 bit-per-pixel images which allows many of the arithmetic operations in the standard GME approach to be replaced with logic operations. Experimental results show that our proposed algorithm achieves the same registration accuracy as the standard GME approach but with significantly reduced computational complexity. Our results also demonstrate the superior performance of the proposed algorithm when compared with previously proposed low-complexity GME approaches.

Macroblock Classification Method for Video Applications Involving Motions

In this paper, a macroblock classification method is proposed for various video processing applications involving motions. Based on the analysis of the Motion Vector field in the compressed video, we propose to classify Macroblocks of each video frame into different classes and use this class information to describe the frame content. We demonstrate that this low-computation-complexity method can efficiently catch the characteristics of the frame. Based on the proposed macroblock classification, we further propose algorithms for different video processing applications, including shot change detection, motion discontinuity detection, and outlier rejection for global motion estimation. Experimental results demonstrate that the methods based on the proposed approach can work effectively on these applications.

Robust Real-Time-Constrained Estimation of Respiratory Motion for Interventional MRI on Mobile Organs

Real-time magnetic resonance imaging is a promising tool for image-guided interventions. For applications such as thermotherapy on moving organs, a precise image-based compensation of motion is required in real time to allow quantitative analysis, retrocontrol of the interventional device, or determination of the therapy endpoint. Reduced field-of-view imaging represents a promising way to improve spatial and/or temporal resolution. However, it introduces new challenges for target motion estimation, since structures near the target may appear transiently due to the respiratory motion and the limited spatial coverage. In this paper, a new image-based motion estimation method is proposed combining a global motion estimation with a novel optical flow approach extending the initial Horn and Schunck (H&S) method by an additional regularization term. This term integrates the displacement of physiological landmarks into the variational formulation of the optical flow problem. This allowed for a better control of the optical flow in presence of transient structures. The method was compared to the same registration pipeline employing the H&S approach on a synthetic dataset and in vivo image sequences. Compared to the H&S approach, a significant improvement (p<0.05) of the Dice's similarity criterion computed between the reference and the registered organ positions was achieved.

Decomposition of Dynamic Textures Using Morphological Component Analysis

The research context of this paper is dynamic texture analysis and characterization. Many dynamic textures can be modeled as large scale propagating wavefronts and local oscillating phenomena. After introducing a formal model for dynamic textures, the morphological component analysis (MCA) approach with a well-chosen dictionary is used to retrieve the components of dynamic textures. We define two new strategies for adaptive thresholding in the MCA framework, which greatly reduce the computation time when applied on videos. Tests on real image sequences illustrate the efficiency of the proposed method. An application to global motion estimation is proposed and future prospects are finally exposed.

基于仿射变换的快速全局运动估计算法

Global Motion Estimation is an important tool widely used in computer vision,video processing,and other fields.To reduce the computational complexity of global motion estimation,a global motion estimation method based on the pixel gradient strategy was proposed.With regard to global motion models,six-parameter affine model is adopted because of its reasonable tradeoff between complexity and accuracy.In order to accelerate the algorithm,two-level pyramid is utilized in the calculation.It uses those gradient bigger pixels in the calculation of every pyramid level,and combines Gauss-Newton algorithm.The extensive experiments show that the proposed method is efficient and robust in terms of both computational complexity and accuracy.

Resolution Enhancement by Incorporating Segmentation-based Optical Flow Estimation

In this paper, the problem of recovering a high-resolution frame from a sequence of low-resolution frames is considered. High-resolution reconstruction process highly depends on image registration step. Typical resolution enhancement techniques use global motion estimation technique. However, in general, video frames cannot be related through global motion due to the arbitrary individual pixel movement between frame pairs. To overcome this problem, we propose to employ segmentation-based optical flow estimation technique for motion estimation with a modified model for frame alignment. To do that, we incorporate the segmentation with the optical flow estimation in two-stage optical flow estimation. In the first stage, a reference image is segmented into homogeneous regions. In the second stage, the optical flow is estimated for each region rather than pixels or blocks. Then, the frame alignment is accomplished by optimizing the cost function that consists of L1-norm of the difference between the interpolated low-resolution (LR) frames and the simulated LR frames. The experimental results demonstrate that using segmentation-based optical flow estimation in motion estimation step with the modified alignment model works better than other motion models such as affine, and conventional optical flow motion models.

A Joint Approach to Global Motion Estimation and Motion Segmentation From a Coarsely Sampled Motion Vector Field

In many content-based video processing systems, the presence of moving objects limits the accuracy of global motion estimation (GME). On the other hand, the inaccuracy of global motion parameter estimates affects the performance of motion segmentation. In this paper, we introduce a procedure for simultaneous object segmentation and GME from a coarsely sampled (i.e., block-based) motion vector (MV) field. The procedure starts with removing MV outliers from the MV field, and then performs GME to obtain an estimate of global motion parameters. Using these estimates, global motion is removed from the MV field, and moving region segmentation is performed on this compensated MV field. MVs in the moving regions are treated as outliers in the context of GME in the next round of processing. Iterating between GME and motion segmentation helps improve both GME and segmentation accuracy. Experimental results demonstrate the advantage of the proposed approach over state-of-the-art methods on both synthetic motion fields and MVs from real video sequences.

Algorithms for the Digital Restoration of Torn Films

This paper presents algorithms for the digital restoration of films damaged by tear. As well as causing local image data loss, a tear results in a noticeable relative shift in the frame between the regions at either side of the tear boundary. This paper describes a method for delineating the tear boundary and for correcting the displacement. This is achieved using a graph-cut segmentation framework that can be either automatic or interactive when automatic segmentation is not possible. Using temporal intensity differences to form the boundary conditions for the segmentation facilitates the robust division of the frame. The resulting segmentation map is used to calculate and correct the relative displacement using a global-motion estimation approach based on motion histograms. A high-quality restoration is obtained when a suitable missing-data treatment algorithm is used to recover any missing pixel intensities.

Moving Region Segmentation From Compressed Video Using Global Motion Estimation and Markov Random Fields

In this paper, we propose an unsupervised segmentation algorithm for extracting moving regions from compressed video using global motion estimation (GME) and Markov random field (MRF) classification. First, motion vectors (MVs) are compensated from global motion and quantized into several representative classes, from which MRF priors are estimated. Then, a coarse segmentation map of the MV field is obtained using a maximum a posteriori estimate of the MRF label process. Finally, the boundaries of segmented moving regions are refined using color and edge information. The algorithm has been validated on a number of test sequences, and experimental results are provided to demonstrate its advantages over state-of-the-art methods.

Real-Time Affine Global Motion Estimation Using Phase Correlation and its Application for Digital Image Stabilization

We propose a fast and robust 2D-affine global motion estimation algorithm based on phase-correlation in the Fourier-Mellin domain and robust least square model fitting of sparse motion vector field and its application for digital image stabilization. Rotation-scale-translation (RST) approximation of affine parameters is obtained at the coarsest level of the image pyramid, thus ensuring convergence for a much larger range of motions. Despite working at the coarsest resolution level, using subpixel-accurate phase correlation provides sufficiently accurate coarse estimates for the subsequent refinement stage of the algorithm. The refinement stage consists of RANSAC based robust least-square model fitting for sparse motion vector field, estimated using block-based subpixel-accurate phase correlation at randomly selected high activity regions in finest level of image pyramid. Resulting algorithm is very robust to outliers such as foreground objects and flat regions. We investigate the robustness of the proposed method for digital image stabilization application. Experimental results show that the proposed algorithm is capable of estimating larger range of motions as compared to another phase correlation method and optical flow algorithm.

Global motion estimation–based method for nighttime video enhancement

In order to efficiently enhance the dark nighttime videos, the high-quality daytime information of the same scene is often introduced to help the enhancement. However, due to camera motion, the introduced daytime may not have exactly the same scene of the nighttime videos. Thus, the final fused moving objects may not produce reasonable results. In this paper, we make the following two contributions: 1. we propose a global motion estimation-based scheme to address the problem of scene differences between daytime and nighttime videos. 2. Based on this, we further propose an improved framework for nighttime video enhancement which can efficiently recover the unreasonable enhancement results due to scene differences. Experimental results show the effectiveness of the proposed algorithm.

Automatic, robust global motion estimation using clustering

Global motion estimation (GME) is a vital part of many video compression and computer vision applications. However, the large moving foreground objects that are present in many video scenes make the task of GME more challenging. In this paper, we propose an automatic, efficient, and robust approach for GME that addresses the issue of large foreground objects. The proposed GME algorithm is based on two key ideas: a new clustering technique, to automate the initial segmentation of background and foreground blocks, and a modified Lorentzian estimator, to reduce the impact of any remaining foreground blocks on the GME process. We also apply an up-sampling technique to the estimated motion parameters to remove any errors caused by under-sampling during the warping process. These ideas provide a significant improvement in performance when combined into a common framework. Simulation results and analyses demonstrate the improved performance of our proposed algorithm over other state-of-the-art methods.

Precise and accurate decimal number recognition using Global Motion Estimation

Precise and accurate automatic recognition of decimal numbers is essential for many applications. Motion Estimation (ME) is a basic component in any video compression technique used to account for the temporal redundancy in image sequences. Global Motions are often modelled by parametric transformations of two dimensional images. The process of estimating the motion parameters is called Global Motion Estimation (GME). In this paper, we propose a new way of using GME for the purpose of off-line machine-print decimal digit. We show that the proposed approach is able to achieve very high recognition rates.