Affine Motion Model Research Articles

A Robust hierarchical motion estimation algorithm in noisy image sequences in the bispectrum domain

The present study describes a robust hierarchical motion estimation algorithm in noisy image sequences using the bispectrum. The motion can be characterized by an affine model and the parameters of an affine motion model are estimated by means third-order auto-bispectrum and cross-bispectrum measures. The basic components of this framework to obtain motion vectors are (i) pyramid construction, (ii) motion estimation and (iii) coarse-to-fine refinement. The entire motion is decomposed as a global and a local motion field, which helps accurately obtain high resolution estimates for the local motion field. Simulation results are presented and compared to those obtained from the phase correlation algorithm. The results demonstrate that the proposed method is more suited than the phase correlation algorithm to analyses complex noisy image sequences. On the other hand, our method produces smoother displacement vector field with a more accurate measure of object motion in different signal-to-noise ratio scenarios.

Motion Analysis with the Radon Transform on Log-Polar Images

Image projections provide an effective way of describing image contents or estimate particular kinds of motion. However, most (if not all) of previous literature on projections has been done on Cartesian images. In contrast, the work described in this paper is aimed at exploring how projections can be defined on log-polar images and how they perform in estimating motion. In the proposed algorithm, a set of projection signals is computed in two consecutive frames. Then, 1D affine motion between each pair of corresponding projection signals is estimated. Finally, 2D image affine motion is derived from the set of estimated 1D motion parameters, using a 2D-1D motion mapping model (MMM). A reduced, 5-parameter, affine motion model can be estimated with this MMM. The algorithm is implemented in both, log-polar and Cartesian images. Synthetic motion is used for a careful analysis of the strengths and weaknesses of the algorithm. The comparison of the results with log-polar and Cartesian images reveal that the limitations of the approach are due to the MMM, rather than to the inherent difficulties and distortions introduced by the space-variant nature of log-polar images. Another significant finding is that Cartesian images require much more pixels than log-polar images to get comparable estimation performance.

Encoding of Affine Motion Vectors

An affine motion model provides better motion representation than a translational motion model. Therefore, it is a good candidate for advanced video compression algorithms, requiring higher compression efficiency than current algorithms. One disadvantage of the affine motion model is the increased number of motion vector parameters, therefore increased motion vector bit rate. We develop and analyze several simulation based approaches of entropy coding for orthonormalized affine motion vector (AMV) coefficients, by considering various context-types and coders. In our work we expand the traditional idea of a context type by introducing four new context types. We compare our method of contexts-type and coder selection with context quantization. The best of our contexts-type and coder solutions produces 4% to 15% average AMV bit-rate reductions over the original VLC approach. For more difficult content AMV bit rate reduction up to 26% is reported.

Video frame rate up conversion under inconsistent camera motion

In this paper, we address the problem of video frame rate up-conversion (FRC) in the compressed domain. FRC is often recognized as video temporal interpolation. This problem is very challenging when targeted for video sequences with inconsistent camera and object motion, such as sports videos. A novel compressed domain motion compensation scheme is presented and applied in this paper, aiming at up-sampling frame rates in sports videos. MPEG-2 encoded motion vectors (MVs) are utilized as inputs in the proposed algorithm. The decoded MVs undergo a cumulative spatiotemporal interpolation. An iterative rejection scheme based on the dense motion vector field (MVF) and the generalized affine motion model is exploited to detect global camera motion. Subsequently, the foreground object separation is performed by additionally examining the temporal consistency of the output of iterative rejections. This consistency check process helps coalesce the resulting foreground blocks and weed out the unqualified blocks. Finally, different compensation strategies for the camera and object motions are applied to interpolate the new frames. Illustrative examples are provided to demonstrate the efficacy of the proposed approach. Experimental results are compared with the popular block and non-block based frame interpolation approaches.

Complexity accuracy tradeoffs of Lie operators in motion estimation

Lie operators are known to be effective in detecting very small degrees of object motions such as scaling, rotations and deformations. In this paper, we apply multiple Lie operators in combination to increase the accuracy of the conventional translation-only motion estimation. We propose the following three motion estimation methods using Lie operators with varying computational complexities, including the serial search, iterative search and the dynamic programming like search methods. We seek to study the tradeoffs allowed by the different combinations of Lie operators between the improved accuracy and the extra computational complexity associated with estimating the motion parameters. Both analytical and experimental results show that the proposed Lie-operator approaches can offer significant increases in the accuracy of the motion estimation with only low to moderate increase in the computational complexity. We also demonstrate that the iterative search method based on Lie operators has much lower complexity than motion search using an affine motion model for small motion parameters, while providing accuracy improvements very close to those attainable by the affine model approach.

Recognition of Dynamic Video Contents With Global Probabilistic Models of Visual Motion

The exploitation of video data requires methods able to extract high-level information from the images. Video summarization, video retrieval, or video surveillance are examples of applications. In this paper, we tackle the challenging problem of recognizing dynamic video contents from low-level motion features. We adopt a statistical approach involving modeling, (supervised) learning, and classification issues. Because of the diversity of video content (even for a given class of events), we have to design appropriate models of visual motion and learn them from videos. We have defined original parsimonious global probabilistic motion models, both for the dominant image motion (assumed to be due to the camera motion) and the residual image motion (related to scene motion). Motion measurements include affine motion models to capture the camera motion and low-level local motion features to account for scene motion. Motion learning and recognition are solved using maximum likelihood criteria. To validate the interest of the proposed motion modeling and recognition framework, we report dynamic content recognition results on sports videos.

A genetic algorithm for optical flow estimation

This paper illustrates a new optical flow estimation technique that builds upon a genetic algorithm (GA). First, the current frame is segmented into generic shape regions, using only luminance and color information. For each region, a two-parameter motion model is estimated using a GA. The fittest individuals identified at the end of this step are used to initialize the population of the second step of the algorithm, which estimates a six-parameter affine motion model, again using a GA. The proposed method is compared with a multi-resolution version of the well-known Lucas–Kanade differential algorithm. Our simulations demonstrate that, with respect to Lucas–Kanade, it significantly reduces the energy of the motion-compensated residual error.

Analysis of residual coronary artery motion for breath hold and navigator approaches using real‐time coronary MRI

Coronary artery MRI methods utilize breath holds, or diaphragmatic navigators, to compensate for respiratory motion. To increase image quality and navigator (NAV) gating efficiency, slice tracking is used, with more sophisticated affine motion models recently introduced. This study assesses the extent of remaining coronary artery motion in free breathing NAV and single and multi breath hold coronary artery MRI. Additionally, the effect of the NAV gating window size was examined. To visualize and measure the respiratory induced motion, an image containing a coronary artery cross section was acquired at each heartbeat. The amount of residual coronary artery displacement was used as a direct measure for the performance of the respiratory motion correction method. Free breathing studies with motion compensation (slice tracking with 5 mm gating window) had a similar amount of residual motion (0.76+/-0.17 mm) as a single breath hold (0.52+/-0.20 mm) and were superior to multiple breath holds (1.22+/-0.60 mm). Affine NAV methods allowed for larger gating windows ( approximately 10 mm windows) with similar residual motion (0.74+/-0.17 mm). In this healthy adult cohort (N=10), free-breathing NAV methods offered respiratory motion suppression similar to a single breath hold.

A robust real-time video stabilization algorithm

The acquisition of digital video usually suffers from undesirable camera jitters due to unstable camera motions. In this paper, we propose a robust real-time video stabilization algorithm that alleviates the undesirable jitter motions from the unstable video to produce a stabilized video. In the proposed algorithm, we first compute the sparse optical flow vectors between successive frames, followed by estimating the camera motion by fitting the computed optical flow vectors to a simplified affine motion model with a robust trimmed least squares method. Then the computed camera motion parameters are smoothed temporally to reduce the motion fluctuations by using a regularization method. Finally, we transform all frames in the video sequence based on the original and smoothed camera motions to obtain a stabilized video. Experimental results are given to demonstrate the stabilization performance and the efficiency of the proposed algorithm.

Respiratory motion compensation with tracked internal and external sensors during CT-guided procedures

This paper investigates the possibility of using the motion of a patient's anterior surface in combination with a motion model to compensate for internal respiratory motion during tracked biopsies. Position data from two electromagnetically tracked sensors, one placed on the patient's sternum, the other incorporated into a biopsy needle, were acquired during a liver biopsy. The data were used to evaluate the correlation between the position measurements of the two sensors and to derive an affine motion model to assess respiratory motion compensation for image-guided interventional procedures. The correlation reached up to 94% for ranges of steady respiration. The residual motion of the internal sensor after compensation is reduced by a factor of approximately four.

Prospective correction of affine motion for arbitrary MR sequences on a clinical scanner

The concept of prospective 3D affine motion correction was generalized, based on the Bloch equations, for signal excitation and sampling using arbitrary MR sequences. The technique was implemented on a clinical MRI scanner for Cartesian, radial, and spiral imaging sequences, as well as for 2D spatially selective RF excitation pulses. A patient-specific motion model steered by real-time navigators was employed to account for the additional degrees of freedom provided by the affine motion model. Different navigator concepts (multiple spatial and temporal navigators, quadratic navigators and other motion sensors) were investigated, with the aim of improving the correlation between navigator information and the motion model. Experiments on moving phantoms are presented to prove the technical feasibility of the approach. In vivo experiments on coronary MRA and renal MRI show the potential of the method for cardiac and abdominal applications hampered by respiratory motion.

Respiratory motion compensation with tracked internal and external sensors during CT guided procedures

Position data from two electromagnetically tracked sensors, one placed on the patient's sternum, the other incorporated into a biopsy needle, were acquired during a liver biopsy. The data were used to evaluate the correlation between the position measurements of the two sensors and to derive an affine motion model to assess the possibility of using an external sensor for respiratory motion compensation for image-guided interventional procedures.

Motion compensation for interventional navigation on 3D static roadmaps based on an affine model and gating

Current cardiac interventions are performed under 2D fluoroscopy, which comes along with well-known burdens to patients and physicians, such as x-ray exposure and the use of contrast agent. Furthermore, the navigation on complex structures such as the coronaries is complicated by the use of 2D images in which the catheter position is only visible while the contrast agent is introduced. In this work, a new method is presented, which circumvents these drawbacks and enables the cardiac interventional navigation on motion-compensated 3D static roadmaps. For this, the catheter position is continuously reconstructed within a previously acquired 3D roadmap of the coronaries. The motion compensation makes use of an affine motion model for compensating the respiratory motion and compensates the motion due to cardiac contraction by gating the catheter position. In this process, only those positions which have been acquired during the rest phase of the heart are used for the reconstruction. The method necessitates the measurement of the catheter position, which is done by using a magnetic tracking system. Nevertheless, other techniques, such as image-based catheter tracking, can be applied. This motion compensation has been tested on a dynamic heart phantom. The evaluation shows that the algorithm can reconstruct the catheter position on the 3D static roadmap precisely with a residual motion of 1.0 mm and less.

Curvature-based algorithms for nonrigid motion and correspondence estimation

We present a novel technique for utilizing the Gaussian curvature information in 3D nonrigid motion estimation in the absence of known correspondence. Differential-geometric constraints derived in the paper allow one to estimate parameters of the local affine motion model given the values of Gaussian curvature before and after motion. These constraints can be further combined with the previously known constraints based on the unit normals before and after motion. Our experiments demonstrate that the resulting hybrid algorithm is more accurate than each of its constituents and more accurate than the classical ICP algorithm. We also present a technique for curvilinear orthogonalization of quadratic Monge patches that is essential in our derivation and useful in other applications.

Fast two-frame multiscale dense optical flow estimation using discrete wavelet filters.

A multiscale algorithm with complexity O(N) (where N is the number of pixels in one image) using wavelet filters is proposed to estimate dense optical flow from two frames. Hierarchical image representation by wavelet decomposition is integrated with differential techniques in a new multiscale framework. It is shown that if a compactly supported wavelet basis with one vanishing moment is carefully selected, hierarchical image, first-order derivative, and corner representations can be obtained from the wavelet decomposition. On the basis of this result, three of the four components of the wave let decomposition are employed to estimate dense optical flow with use of only two frames. This overcomes the "flattening-out" problem in traditional pyramid methods, which produce large errors when low-texture regions become flat at coarse levels as a result of blurring. A two-dimensional affine motion model is used to formulate the optical flow problem as a linear system, with all resolutions simultaneously (i.e., coarse-and-fine) rather than the traditional coarse-to-fine approach, which unavoidably propagates errors from the coarse level. This not only helps to improve the accuracy but also makes the hardware implementation of our algorithm simple. Experiments on different types of image sequences, together with quantitative and qualitative comparisons with several other optical flow methods, are given to demonstrate the effectiveness and the robustness of our algorithm.

Investigation on convergence behaviors of the differential method for 2-D motion parameter estimation

We investigate convergence behaviors of the differential method for estimation of 2-D motion parameters. While the differential method has been widely researched for motion compensated predictive video coding, little attention has been paid to its convergence properties. Based on the nonseparable exponential covariance image model, we derive theoretical estimates of update terms for translational and affine motion models. The effects of noise, spatial correlation, choice of spatial gradient measures, and the size of a region, are quantitatively analyzed in relation to the convergence speed. Some empirical results are presented to verify the analysis.

A New Method for Region-Based Depth Ordering in a Video Sequence: Application to Frame Interpolation

Region-based interpolation schemes for video sequences have grown in popularity with the introduction of the new object-based representation of video such as in the MPEG4 standard. A new method of a region-based interpolation of missing frames in video sequences for a rate up-conversion is proposed. The method allows an interpolation of both original video frames and segmentation maps for object-based content manipulation. The method uses a 2 1 2 D model of video scene content corresponding to the depth ordering of plane motion-homogeneous regions along the depth axis. A method for determining the depth order of regions in a frame is developed based on a motion analysis in occluding areas and an optimal graph search. This depth order is used for a motion-compensated interpolation based on an affine motion model. Results in real video sequences in CCIR601 format for up-conversion of video rate are obtained, showing a good quality compared to a B-frame interpolation scheme of MPEG 1,2 standards.

Four-dimensional affine registration models for respiratory-gated PET

The heart position shifts considerably due to motion associated with the respiratory cycle, and this motion can degrade the image quality of cardiac-gated positron emission tomography (PET) studies. One method to combat this motion-induced blur is a respiratory-gated acquisition followed by recombination of registered image volumes using a rigid-body motion assumption; however, nonrigid deformation of the heart from respiratory motion may reduce the effectiveness of this procedure. We have investigated a 12-parameter global affine motion model for registration of different respiratory gates in an end-diastolic cardiac PET sequence. To obtain robust estimates of motion, a four-dimensional registration model was devised that encouraged smoothly varying motion between adjacent respiratory time frames. Registration parameters were iteratively calculated using a cost function that combined a least squares voxel difference measure with a penalty obtained from a prediction prior. The prior was calculated from adjacent time frames assuming constant velocity and an affine model. After registration, the principal extension ratios were calculated to measure the degree of nonrigid motion. In data from ten subjects, extension ratios of over 5% were common, indicating that an affine model may provide better registrations and in turn, better motion-corrected composite volumes than could a technique restricted to the six-parameter rigid body assumption.

Performance of an advanced video codec on a general-purpose processor with media ISA extensions

This paper analyses the performance of the state-of-the-art media ISA (instruction set architecture) extensions in a general-purpose processor, when executing a video encoder based on an affine motion model. In addition to SIMD (single instruction multiple data) fixed-point instructions, these ISA extensions include SIMD floating-point instructions, special-purpose SIMD fixed-point instructions, and cacheability control instructions. In this study, eight time-consuming kernels of the video encoder were hand-optimized, using instructions in all four instruction categories of these media ISA extensions (the FLP version). These kernels were also hand-optimized using only SIMD fixed-point ISA extensions, without special-purpose instructions (the FXP version). For the FLP version, this study resulted in an average kernel-level speedup of 1.37X and an application-level speedup of 1.11X, compared to the FXP version, and an application-level speedup of 3.41X, compared to the C version.

HOS-based method of global motion estimationfor noisy image sequences

An investigation is presented into a new region-recursive higher-order statistics (HOS) based method for global motion estimation which uses a simplified affine motion model (SLM) of a region. Compared to the similar SOS method, the presented method is is better suited to the analysis of complex noisy image sequences contaminated by correlated gaussian noise (CGN).