Motion Estimation Process Research Articles

Flexible Complexity Control Solution for Transform Domain Wyner-Ziv Video Coding

Most Wyner-Ziv (WZ) video coding solutions in the literature focus on improving the coding efficiency. Recently, a few papers have addressed problems related to the complexity distribution of WZ video coding by sharing the motion estimation process between the encoder and decoder. However, these methods turn out to significantly increase the computational complexity of the encoding process, due to the presence of motion estimation at the encoder, which is less appealing in WZ coding applications, since their primary requirement is a low-encoding complexity. To address this problem, we propose a different approach to complexity control based on the adaptive selection between WZ and intra coding in the WZ frames. This solution is more flexible than the previous ones in that it can be implemented either at the encoder or decoder. The proposed intra mode selection algorithm exploits the spatial and temporal coherency in the transform domain Wyner-Ziv video coding (TDWZ), in order to achieve complexity control between the WZ encoder and decoder. The experimental results illustrate that the proposed algorithm not only effectively distributes the computational complexity over the encoder and decoder, but also retains the low-complexity feature at the encoder. This should make it attractive for a large number of real video applications of the WZ video coding paradigm. Moreover, the coding efficiency of the conventional TDWZ codec without intra mode decision is improved by up to 2 dB by the proposed intra mode selection algorithm.

Simple drift error-resilient H.264/AVC encoder for fast video transcoding using DCT coefficients

As described in this paper, an encoder that generates drift-error-resilient coded bitstreams is proposed. The proposed drift-error-resilient bitstreams are generated using the TotalCoeff or the total amount of non-zero DCT coefficients to estimate the drift-error distortion. This estimated drift error distortion is then examined in the motion estimation process. Results show that our proposed method can decrease drift error effectively compared to the original encoder and that it can achieve comparable PSNR to a conventional drift-error-resilient encoder. The main advantage of the proposed method compared to a conventional drift-error-resilient encoder is that, comparable PSNR performance, lower complexity and memory usage can be achieved without implementing an open-loop transcoder at the encoder feedback loop.

Cogeneration of Fast Motion Estimation Processors and Algorithms for Advanced Video Coding

This paper presents a flexible and scalable motion estimation processor capable of supporting the processing requirements for high-definition (HD) video using the H.264 Advanced Video Codec, which is suited for FPGA implementation. Unlike most previous work, our core is optimized to execute all existing fast block matching algorithms, which we show to match or exceed the inter-frame prediction performance of traditional full-search approaches at the HD resolutions commonly in use today. Using our development tools, such algorithms can be described using a C-style syntax which is compiled into our custom instruction set. We show that different HD sequences exhibit different characteristics which necessitate a flexible and configurable solution when targeting embedded applications. This is supported in our core and toolset by allowing designers to modify the number of functional units to be instantiated. All processor instances remain binary compatible so recompilation of the motion estimation algorithm is not required. Due to this optimization process, it is possible to match the processing requirements of the selected motion estimation algorithm to the hardware microarchitecture leading to a very efficient implementation.

Effect of scaling on the area and performance of the H.264/AVC full-search fractional motion estimation algorithm on field-programmable gate arrays

Fractional motion estimation (FME) is an important part of the H.264/AVC video encoding standard. The algorithm can significantly increase the compression ratio of video encoders while preserving high video quality. The full-search FME algorithm, however, is computationally expensive and can consist of over 45% of the total motion estimation process. To maximise the performance and efficiency of FME implementations on field-programmable gate arrays (FPGAs), one needs to efficiently exploit the inherent parallelism in the algorithm. The authors investigate the scalability of the full-search FME algorithm on FPGAs and also implemented six scaled versions of the algorithm on Xilinx Virtex-5 FPGAs. The authors found that scaling the algorithm vertically within a 4×4 sub-block is more efficient than scaling horizontally across several sub-blocks. It is shown that, with four reference frames, the best vertically scaled design can achieve 96 frames-per-second (fps) performance while encoding full 1920×1088 progressive HDTV video, and the design only consumes 25.5 K LUTS and 28.7 K registers.

Fast Adaptive Hierarchical Prediction Algorithm for H.264/AVC Scalable Video Coding

The objective of scalable video coding (SVC) is to enable the generation of a unique bitstream that can adapt to various bit-rates, transmission channels and display capabilities. The scalability is categorised in terms of temporal, spatial, and quality. In order to improve encoding efficiency, the SVC scheme incorporates inter-layer prediction mechanisms to complement the H.264/AVC very refined Motion Estimation (ME) and mode decision processes. However, this further increases the overall encoding complexity of the scalable coding standard. In this paper several conditional probabilities are established relating motion estimation characteristics and the mode distribution at different layers of the H.264/SVC. An evaluation of these probabilities is used to structure a low-complexity prediction algorithm for Group of Pictures (GOP) in H.264/SVC, reducing computational complexity whilst maintaining similar RD performance. When compared to the JSVM software, the proposed algorithm achieves a significant reduction of encoding time, with a negligible average PSNR loss and bit-rate increase in temporal, spatial and SNR scalability. Experiments are conducted to provide a comparison between our method and recently developed fast mode selection algorithms. These demonstrate the proposed method achieves appreciable time savings for scalable spatial and scalable quality video coding, while maintaining similar PSNR and bit rate.

A Memory Hierarchy Model Based on Data Reuse for Full-Search Motion Estimation on High-Definition Digital Videos

The motion estimation is the most complex module in a video encoder requiring a high processing throughput and high memory bandwidth, mainly when the focus is high-definition videos. The throughput problem can be solved increasing the parallelism in the internal operations. The external memory bandwidth may be reduced using a memory hierarchy. This work presents a memory hierarchy model for a full-search motion estimation core. The proposed memory hierarchy model is based on a data reuse scheme considering the full search algorithm features. The proposed memory hierarchy expressively reduces the external memory bandwidth required for the motion estimation process, and it provides a very high data throughput for the ME core. This throughput is necessary to achieve real time when processing high-definition videos. When considering the worst bandwidth scenario, this memory hierarchy is able to reduce the external memory bandwidth in 578 times. A case study for the proposed hierarchy, using32×32search window and8×8block size, was implemented and prototyped on a Virtex 4 FPGA. The results show that it is possible to reach 38 frames per second when processing full HD frames (1920×1080pixels) using nearly 299 Mbytes per second of external memory bandwidth.

An efficient compression scheme for 4-D medical images using hierarchical vector quantization and motion compensation

This paper proposes an efficient compression scheme for compressing time-varying medical volumetric data. The scheme uses 3-D motion estimation to create a homogenous preprocessed data to be compressed by a 3-D image compression algorithm using hierarchical vector quantization. A new block distortion measure, called variance of residual (VOR), and three 3-D fast block matching algorithms are used to improve the motion estimation process in term of speed and data fidelity. The 3-D image compression process involves the application of two different encoding techniques based on the homogeneity of input data. Our method can achieve a higher fidelity and faster decompression time compared to other lossy compression methods producing similar compression ratios. The combination of 3-D motion estimation using VOR and hierarchical vector quantization contributes to the good performance.

An Efficient Adaptive High Speed Manipulation Architecture for Fast Variable Padding Frequency Domain Motion Estimation

Motion estimation (ME) consumes up to 70% of the entire video encoder's computations and is, therefore, the main encoding-time consuming process. Discrete cosine transform (DCT)-based phase correlation along with dynamic padding (DP) are the recently evolved frequency domain ME (FDME) techniques that promise to efficiently reduce the computational complexity of the ME process. DP uses dynamic padding thresholds to select the proper search area size according to a pre-estimated set of motion vectors (MVs). The main drawbacks of using conventional DP in the frequency domain are two-fold. First, the dynamic thresholds need to be estimated in the pixel (IDCT) domain which increases complexity. Second, the mismatched transformed search area is formed from different successive transformed blocks, which would lead to an inaccurate ME if the search area is not manipulated. In this paper, an efficient low complexity algorithm and high speed architecture are proposed to implement an adaptive manipulation unit engine (MUE). The MUE, the main module of the FDME system, adaptively decides the padding size and forges a matched transformed search area from the successive transformed blocks. Additionally, the proposed utilized dynamic thresholds are efficiently estimated in the frequency domain (FD). The MUE architecture is presented with two different design implementations trading off the VLSI design parameters. Implementation and simulation results project that the proposed MUE, when integrated in a whole FDME system, can perform ME for 60 fps of 4CIF video at 172 MHz.

Adaptive search range adjustment scheme for fast motion estimation in AVC/H.264

AVC/H.264 supports the use of multiple reference frames (e.g., 5 frames in AVC/H.264) for motion estimation (ME), which demands a huge computational complexity in ME. We propose an adaptive search range adjustment scheme to reduce the computational complexity of ME by reducing the search range of each reference frame-from the (t-1)'th frame to the (t-5)'th frame-for each macroblock. Based on the statistical analysis that the 16×16 mode type is dominantly selected rather than the other block partition mode types, the proposed method reduces the search range of the remaining ME process in the given reference frame according to the motion vector (MV) position of the 16×16 block ME. In the case of the (t-1)'th frame, the MV position of the 8×8 block ME-in addition to that of 16×16 block ME-is also used for the search range reduction to sub-block partition mode types of the 8×8 block. The experimental results show that the proposed method reduces about 50% and 65% of the total encoding time over CIF/SIF and full HD test sequences, respectively, without any noticeable visual degradation, compared to the full search method of the AVC/H.264 encoder.

High-speed Motion Estimation Architecture for Real-time Video Transmission

Motion estimation (ME) process consumes up to 70% of the total encoding time of video transmission. Because it has a high coding efficiency and it is very scalable, the full search (FS) algorithm is considered to be the most popular ME algorithm. However, the main drawback of the FS algorithm is that it is computationally intensive. For this reason, FS is rarely used for real-time video coding. This paper proposes a simple and fast adaptive search window size (ASWS) algorithm that eliminates a significant amount of computations from the conventional FS algorithm. This is achieved by dynamically reducing the required search area for each reference block. Simulation results show that more than 94% of the candidate blocks are eliminated by our algorithm without significant loss in visual quality. This paper also presents an efficient and high-speed ME engine (MEE) architecture for the proposed ASWS algorithm. The MEE efficiently reuses the search area data to minimize the memory I/O while fully utilizing the available hardware resources. A smart processing element design along with an innovative data scheduling scheme allows the search area data to flow both horizontally and vertically, whereas the current block data remain stationary. This allows the proposed architecture a simple and highly regular dataflow through the core. Simulation results show that for a search range of [-16,+15] and a block size of 16×16, the proposed architecture performs the ME for 60 fps of 4CIF video at 100 MHz and easily outperforms many FS architectures.

Motion estimation using two-stage predictive search algorithms based on joint spatio-temporal correlation information

Motion estimation is one of the major problems in developing a real-time software-based video codec since it has high search complexity. In the motion estimation process, the motion field of the current block can generally be tracked from the motion fields of the neighboring blocks in the spatial and temporal directions. In this paper, two efficient fast motion estimation algorithms with a two-stage predictive search based on joint spatio-temporal correlations are proposed to reduce the search complexity. In the first stage, a rough search from the given motion vectors associated with six spatially and temporally correlated blocks attempts to find a starting point of the adequate search range that is closer to the global optimum. In the second stage, block-based gradient descent search ( Liu & Feig, 1996) and predictive partial search (proposed) algorithms are used for fine search to elaborately search the adequate range from the starting point for the best motion vector. Simulation results demonstrate that our algorithms effectively reduce the average number of checked points to only 1.55% and 0.78% as compared to the full search method and yield a great performance improvement in terms of computational complexity, PSNR and bit rates as compared to full search and some well-known fast search methods.

Multi-standard reconfigurable motion estimation processor for hybrid video codecs

This study presents a programmable and configurable motion estimation (ME) processor capable of performing ME across several state-of-the-art video codecs that include multiple tools to improve the accuracy of the calculated motion vectors. The core can be programmed using a C-style syntax optimised to implement arbitrary block matching algorithms and configured with different execution units depending on the selected codec, the available inter-coding options and required performance. This flexibility means that the core can support the latest video codecs such as H.264, VC-1 and AVS at high-definition resolutions and frame rates. The configuration and programming phases are supported by an integrated development environment that includes a compiler and profiling tools enabling a designer without specific hardware knowledge to optimise the microarchitecture for the selected codec standard and motion search technique leading to a highly efficient implementation.

Fast Motion Estimation Scheme for H.264/AVC Using Adaptive Single and Multiple Reference Picture Selections

In video coding standard of H.264/AVC, motion estimation (ME) using multiple reference pictures improves compression efficiency considerably. However, the motion estimation includes a large amount of useless computations regardless of picture contents. This paper proposes a fast motion estimation algorithm that removes the worthless computations in the motion estimation process with switching single and multiple reference pictures adaptively. The proposed algorithm classifies blocks into valid and invalid blocks for the multiple references based ME, and removes the unnecessary computations by applying a single reference based ME to the invalid blocks. In order to evaluate the performance of the proposed algorithm, measures with respects to image quality, bit rate, and motion estimation time are compared with those of the scheme in a reference software JM 9.5. The simulation results show that the proposed algorithm can considerably save 39% on average motion estimation time while keeping the image quality (0.025dB degradation) and the bit rate (0.8% decrease) as good as the reference algorithm.

Speckle reduction in optical coherence tomography imaging by affine-motion image registration

Signal-degrading speckle is one factor that can reduce the quality of optical coherence tomography images. We demonstrate the use of a hierarchical model-based motion estimation processing scheme based on an affine-motion model to reduce speckle in optical coherence tomography imaging, by image registration and the averaging of multiple B-scans. The proposed technique is evaluated against other methods available in the literature. The results from a set of retinal images show the benefit of the proposed technique, which provides an improvement in signal-to-noise ratio of the square root of the number of averaged images, leading to clearer visual information in the averaged image. The benefits of the proposed technique are also explored in the case of ocular anterior segment imaging.

Two fast multi-point search algorithms for high quality motion estimation in high resolution videos

In this paper, we present two new algorithms focusing on a high quality fast motion estimation process for high definition video coding. Both algorithms provide more efficiency to avoid falling into local minima in fast motion estimation when compared to diamond search (DS) algorithm. They were called multi-point diamond search (MPDS) and dynamic multi-point diamond search (DMPDS). The multi-point search could be done in a serial or parallel approach. In the parallel approach, the penalties in the performance are minimal near the impressive gain in final quality. The MPDS and DMPDS algorithms were implemented and evaluated on ten HD 1080p video sequences. The results show an average quality gain, in comparison with original DS, about 3.9 dB and 4.55 dB for MPDS and DMPDS respectively.

Activity-Based Motion Estimation Scheme for H.264 Scalable Video Coding

This paper proposes a motion estimation scheme to reduce the computational complexity of multilayer motion estimation for scalable video coding. Based on the result of the motion estimation of the lower resolution layer referred to as base layer, we developed a new approach for exploring the search range of the enhancement layer with high coding efficiency. This approach is based on the activity defined as the absolute difference between the motion vector predictor and the final motion vector. Based on the correlation of the activities between neighboring layers, an inter-layer activity model was developed using a curve-fitted linear equation to exploit the activity in the base layer for deciding the search center and the search range of the enhancement layer. Each activity pair in the neighboring layers is used to associate the relevant macroblock to one of two groups; boundary region and interior region. The base-layer motion vector predictor is basically selected over all the activity regions; for each activity region, the proposed motion estimation algorithm decides whether to include the median motion vector predictor or not. Minimal sufficient search range is also decided from the inter-layer activity prediction factor that is adjusted to the given sequence. The proposed scheme reduced the execution time of motion estimation by 99.26% at the cost of 1.56% bit-rate increase and 0.048 dB peak signal-to-noise ratio (PSNR) decrease on average compared with the conventional full-search algorithm. The fast full-search block matching algorithm can also be incorporated to obtain the extra CPU time reduction in the motion estimation process. By adopting the fast full-search block matching algorithm (FFSBMA) in JSVM reference software, the CPU time was reduced by up to 91.84% and the memory bandwidth was reduced by 90% at the sacrifice of 1.27% bit-rate increase and 0.041 dB PSNR decrease on average compared with the FFSBMA only.

RD Optimized Bandwidth Efficient Motion Estimation and Its Hardware Design With On-Demand Data Access

Data bandwidth dominates the performance and power consumption in the video encoder design. In which a low bandwidth and bandwidth aware motion estimation design enables smooth and better video quality as well as lower power consumption on data accesses. This paper proposes a bandwidth efficient motion estimation and its hardware implementation to deal with the bandwidth issues. First, an on-demand data access mechanism is proposed to acquire the reference data according to the video content for motion estimation process and thus can avoid unnecessary reference data loading. Furthermore, the available bandwidth constraint is properly modeled into our proposed rate distortion optimization framework to efficiently use the data bandwidth. Simulation results show that our proposed algorithm not only allocates proper data bandwidth for motion estimation according to video content but also saves 79.15% data bandwidth demand with 0.03 dB PSNR drop and 2.50% bitrate increase in maximum for 4 CIF resolution sequences, when compared to the fully data reuse full search motion estimation which reuses the overlapped reference data to avoid unnecessary data reloading. In addition, under the available data bandwidth constraint, our proposed algorithm can achieve 2.43%, 0.08%, and 0.20% BD-bitrate saving with 0.17 dB, 0.01 dB, and 0.01 dB BD-PSNR increase on average for high, median, and low motion sequences when compared to the full search motion estimation algorithm. The resulted design only needs 75.27 K gate counts when running at 23 MHz operating frequency for 4 CIF at 30 frames/s with 90 nm CMOS process due to its search range independent buffer design.

Reconfigurable system-on-a-chip motion estimation architecture for multi-standard video coding

A new domain-specific, reconfigurable system-on-a-chip (SoC) architecture is proposed for video motion estimation. This has been designed to cover most of the common block-based video coding standards, including MPEG-2, MPEG-4, H.264, WMV-9 and AVS. The architecture exhibits simple control, high throughput and relatively low hardware cost when compared with existing circuits. It can also easily handle flexible search ranges without any increase in silicon area and can be configured prior to the start of the motion estimation process for a specific standard. The computational rates achieved make the circuit suitable for high-end video processing applications, such as HDTV. Silicon design studies indicate that circuits based on this approach incur only a relatively small penalty in terms of power dissipation and silicon area when compared with implementations for specific standards. Indeed, the cost/performance achieved exceeds that of existing but specific solutions and greatly exceeds that of general purpose field programmable gate array (FPGA) designs.

Processing of Angular Motion and Gravity Information Through an Internal Model

The vestibular organs in the base of the skull provide important information about head orientation and motion in space. Previous studies have suggested that both angular velocity information from the semicircular canals and information about head orientation and translation from the otolith organs are centrally processed in an internal model of head motion, using the principles of optimal estimation. This concept has been successfully applied to model behavioral responses to classical vestibular motion paradigms. This study measured the dynamic of the vestibuloocular reflex during postrotatory tilt, tilt during the optokinetic afternystagmus, and off-vertical axis rotation. The influence of otolith signal on the VOR was systematically varied by using a series of tilt angles. We found that the time constants of responses varied almost identically as a function of gravity in these paradigms. We show that Bayesian modeling could predict the experimental results in an accurate and consistent manner. In contrast to other approaches, the Bayesian model also provides a plausible explanation of why these vestibulooculo motor responses occur as a consequence of an internal process of optimal motion estimation.

Fast encoding algorithm avoiding repetition of motion estimation in scalable video coding

Proposed is a fast encoding algorithm which avoids the repetition of the motion estimation process in the current scalable video coding (SVC) codec. Experimental results show that the proposed algorithm can reduce encoding time by 29.69–37.80%, while the coding performance of the SVC codec adopting the proposed algorithm is exactly the same as that of the original one.