Frequency-domain Motion Estimation Research Articles

Frequency domain subpixel registration using HOG phase correlation

We present a novel frequency-domain image registration technique, which employs histograms of oriented gradients providing subpixel estimates. Our method involves image filtering using dense Histogram of oriented gradients (HOG), which provides an advanced representation of the images coping with real-world registration problems such as non-overlapping regions and small deformations. The proposed representation retains the orientation information and the corresponding weights in a multi-dimensional representation. Furthermore, due to the overlapping local contrast normalization characteristic of HOG, the proposed histogram of oriented gradients - phase correlation (HOG-PC) method improves significantly the estimated motion parameters in small size blocks. Experiments using sequences with and without ground truth including both global and local/multiple motions demonstrate that the proposed method outperforms the state-of-the-art in frequency-domain motion estimation, in the shape of phase correlation, in terms of subpixel accuracy and motion compensation prediction for a range of test material, block sizes and motion scenarios.

High speed on-chip multiple cosine transform generator

Speeding up video encoding process is an important issue nowadays due to the high demand of increasing the video transmission speed for real-time video applications. Many algorithms and hardware implementations have been proposed for achieving such demand using pixel domain motion estimation. Moreover, much savings in computations and hardware complexity can be achieved using frequency domain motion estimation (FDME). The key issues in the FDME encoder is to generate four different transforms that are used for deciding the best match motion vector in the motion estimation (ME) process. The related transformed-discrete cosine transform (RT-DCT) generator is responsible for generating such transforms in the FDME encoder. In this paper, a fast RT-DCT generator is proposed to speed up the FDME encoding time by reducing its complexity. That makes the proposed RT-DCT appropriated for real-time video applications with limited resources, such as mobile internet device, cellular phones, personal digital assistant (PDA), and ultra-mobile personal computer. Moreover, the proposed architecture does not scarify the resolution. By adding the proposed RT-DCT generator to the whole FDME system, implementation and simulation results show that the system performs ME for SDTV video sequence at 371 MHz.

High Speed Transform Coding on Chip for Wireless Video Surveillance Systems

Streaming video quality dependents in part upon the video encoding process. The Frequency Domain Motion Estimation process is commonly used to accelerate the entire video encoding process. It is used to reduce the encoding time by increasing the Motion Estimation Time Saving. The Combinational-Discrete Cosine Transform consumes most of the time of the Frequency Domain Motion Estimation video encoder. Its function is to generate four transform coefficients to be used by the Frequency Domain Motion Estimation process. Generating such transforms slows down the encoding speed. Consequently, Frequency Domain Motion Estimation may not be suitable for online video applications. Saving time in the Combinational-Discrete Cosine Transform generator will considerably speed up the Frequency Domain Motion Estimation process. This paper proposes a fast Combinational-Discrete Cosine Transform architecture. The proposed architecture achieves up to 50% and 39% savings in both area and power consumption, respectively, compared to recent state-of-the-art Combinational-Discrete Cosine Transform architectures. The proposed architecture can perform up to 60 fps motion estimation over a 4CIF video at 123.21 MHz when integrated in the frequency domain encoder. It can be generalized to be used even for higher video resolution transmission such as HDTV broadcasting and video surveillance systems.

An Efficient Adaptive High Speed Manipulation Architecture for Fast Variable Padding Frequency Domain Motion Estimation

Motion estimation (ME) consumes up to 70% of the entire video encoder's computations and is, therefore, the main encoding-time consuming process. Discrete cosine transform (DCT)-based phase correlation along with dynamic padding (DP) are the recently evolved frequency domain ME (FDME) techniques that promise to efficiently reduce the computational complexity of the ME process. DP uses dynamic padding thresholds to select the proper search area size according to a pre-estimated set of motion vectors (MVs). The main drawbacks of using conventional DP in the frequency domain are two-fold. First, the dynamic thresholds need to be estimated in the pixel (IDCT) domain which increases complexity. Second, the mismatched transformed search area is formed from different successive transformed blocks, which would lead to an inaccurate ME if the search area is not manipulated. In this paper, an efficient low complexity algorithm and high speed architecture are proposed to implement an adaptive manipulation unit engine (MUE). The MUE, the main module of the FDME system, adaptively decides the padding size and forges a matched transformed search area from the successive transformed blocks. Additionally, the proposed utilized dynamic thresholds are efficiently estimated in the frequency domain (FD). The MUE architecture is presented with two different design implementations trading off the VLSI design parameters. Implementation and simulation results project that the proposed MUE, when integrated in a whole FDME system, can perform ME for 60 fps of 4CIF video at 172 MHz.

Fast mode decision for the H.264/AVC video coding standard based on frequency domain motion estimation

The H.264 video coding standard achieves high performance compression and image quality at the expense of increased encoding complexity. Consequently, several fast mode decision and motion estimation techniques have been developed to reduce the computational cost. These approaches successfully reduce the computational time by reducing the image quality and/or increasing the bitrate. In this paper we propose a novel fast mode decision and motion estimation technique. The algorithm utilizes preprocessing frequency domain motion estimation in order to accurately predict the best mode and the search range. Experimental results show that the proposed algorithm significantly reduces the motion estimation time by up to 97%, while maintaining similar rate distortion performance when compared to the Joint Model software.

Sub-Hexagonal Phase Correlation for Motion Estimation

We present a novel frequency-domain motion estimation technique, which operates on hexagonal images and employs the hexagonal Fourier transform. Our method involves image sampling on a hexagonal lattice followed by a normalised hexagonal cross-correlation in the frequency domain. The term subpixel (or subcell) is defined on a hexagonal grid in order to achieve floating point registration. Experiments using both artificially induced motion and actual motion demonstrate that the proposed method outperforms the state-of-the-art in frequency-domain motion estimation operating on a square lattice, in the shape of phase correlation, in terms of subpixel accuracy for a range of test material and motion scenarios.

Fast Variable Padding Motion Estimation Using Smart Zero Motion Prejudgment Technique for Pixel and Frequency Domains

Motion estimation (ME) plays an important role in modern video coders since it consumes approximately 60-80% of the entire encoder's computations. In this paper, three novel techniques are proposed to effectively speed up the ME process. First, a smart prediction technique for effectively deciding an initial search center is proposed. Second, a zero motion prejudgment technique is proposed to accurately decide whether the pre-estimated ISC can be considered as a best match motion vector (MV) and consequently save the required computations for the MV refinement process. Finally, a variable padding pixels ME technique is proposed to adaptively determine the number of padding pixels required for the search window for more computational cost savings. The three techniques are combined and applied to the block-based ME for a superior computational complexity savings in the ME process. The performance of the proposed techniques is tested in both the pixel domain ME and the frequency domain ME in terms of their quantitative visual quality (peak signal-to-noise ratio, PSNR), their computational complexity, and their bit rate. Experimental results demonstrate that the proposed fast ME technique is able to achieve approximately a 99.4% reduction in ME time compared to the conventional full search block-based ME (FSSBB-ME) with negligible degradation in both the PSNR and the bit rate. Additionally, the experimental results prove the effectiveness of the proposed techniques if they are combined with any block-based ME technique such as the fast extended diamond enhanced predictive zonal search. Experimental results also demonstrate that there is at least an additional savings of 72% in ME time using the conventional discrete cosine transform phase correlation ME (DCT-PC-ME) in the frequency domain compared to the conventional FSBB-ME technique in pixel domain. Compared to the conventional DCT-PC-ME, applying the proposed novel techniques to the DCT-PC-ME saves up to 89% in ME time.

Estimation of Subpixel Motion Using Bispectrum

Motion estimation techniques are widely used in today's video processing systems. The frequently used techniques are frequency-domain motion estimation methods, most notably phase correlation (PC). If the image frames are corrupted by Gaussian noises, then cross-correlation and related techniques do not work well. In this paper, however, we have studied this topic from a viewpoint different from the above. Our scheme is based on the bispectrum method for sub-pixel motion estimation of noisy image sequences. Experimental results show that our proposed method performs significantly better than PC technique.

Performance study of gradient correlation for sub-pixel motion estimation in the frequency domain

The authors present a performance study of gradient correlation in the context of the estimation of interframe motion in video sequences. The method is based on the maximisation of the spatial gradient cross-correlation function, which is computed in the frequency domain and therefore can be implemented by fast transformation algorithms. Enhancements to the baseline gradient-correlation algorithm are presented which further improve performance, especially in the presence of noise. A comparative performance study is also presented, which demonstrates that the proposed method outperforms state-of-the-art methods in frequency-domain motion estimation, in the shape of phase correlation, in terms of sub-pixel accuracy for a range of test material and motion scenarios.

Estimation of sub-pixel motion using gradient cross-correlation

A computationally efficient scheme suitable for the estimation of sub-pixel motion in video sequences is presented. The scheme is based on the maximisation of the cross-correlation function between spatial gradient information derived from a pair of images. The proposed scheme is highly accurate, outperforms established frequency-domain motion estimation methods and is considerably more immune to noise.