Variable Block Size Motion Estimation Research Articles

High Speed Parallel SAD Architecture Implementation on FPGA for HEVC encoder

Video compression is a very complex and time consuming task which generally pursuit high performance. Motion Estimation (ME) process in any video encoder is responsible to primarily achieve the colossal performance which contributes to significant compression gain. Summation of Absolute Difference (SAD) is widely applied as distortion metric for ME process. With the increase in block size to 64×64 for real time applications along with the introduction of asymmetric mode motion partitioning(AMP) in High Efficiency Video Encoding (HEVC)causes variable block size motion estimation very convoluted. This results in increase in computational time and demands for significant requirement of hardware resources. In this paper parallel SAD hardware circuit for ME process in HEVC is propound where parallelism is used at various levels. The propound circuit has been implemented using Xilinx Virtex-5 FPGA for XC5VLX20T family. Synthesis results shows that the propound circuit provides significant reduction in delay and increase in frequency in comparison with results of other parallel architectures

A Hardware-Efficient Block Matching Algorithm and Its Hardware Design for Variable Block Size Motion Estimation in Ultra-High-Definition Video Encoding

Variable block size motion estimation has contributed greatly to achieving an optimal interframe encoding, but involves high computational complexity and huge memory access, which is the most critical bottleneck in ultra-high-definition video encoding. This article presents a hardware-efficient block matching algorithm with an efficient hardware design that is able to reduce the computational complexity of motion estimation while providing a sustained and steady coding performance for high-quality video encoding. A three-level memory organization is proposed to reduce memory bandwidth requirement while supporting a predictive common search window. By applying multiple search strategies and early termination, the proposed design provides 1.8 to 3.7 times higher hardware efficiency than other works. Furthermore, on-chip memory has been reduced by 96.5% and off-chip bandwidth requirement has been reduced by 39.4% thanks to the proposed three-level memory organization. The corresponding power consumption is only 198mW at the highest working frequency of 500MHz. The proposed design is attractive for high-quality video encoding in real-time applications with low power consumption.

Recongfigurable Hardware-Friendly Early Termination Mechanism in Motion Estimation for HEVC

Motion estimation with a quad-tree variable block size is the driver for the high performance of HEVC in video compression. However, tremendous sum of absolute difference (SAD) computations are needed for every single block size, resulting in huge memory access and power consumption. This work proposes an early termination mechanism at the hardware level that is suitable for quad-tree variable block size motion estimation without degrading video quality. The simulations show reductions in memory access to fetch pixels of about 34.03% and 51.98% saving of SAD computations. This reduction results in reduced power consumption. The simplicity of the algorithm makes it a hardware friendly mechanism, which could be adopted in any motion estimation hardware design.

Energy efficient processing of motion estimation for embedded multimedia systems

Visual sensor networks require low power compression techniques of large amount of video data in each camera node due to the energy-constrained and bandwidth-limited environments. In this paper, energy-efficient architecture for Variable Block Size Motion Estimation is proposed to fully utilize dynamic partial reconfiguration capability of programmable hardware fabric in distributed embedded vision processing nodes. Partial reconfiguration of FPGA is exploited to support run-time reconfiguration of the proposed modular hardware architecture for motion estimation. According to the required search range, hardware reconfiguration is performed adaptively to reduce the hardware resources and power consumption. A reconfigurable ME ranging from simple 1-D to a complex 2-D Sum of Absolute Differences (SAD) array to perform full search block matching is selected in order to support different search window size. The implemented scalable SAD array can provide different resolutions and frame rates for real time applications with multiple reconfigurable regions.

An improved 3D wavelet-based scalable video coding codec for MC-EZBC

With the rapid growth of modern multimedia applications, 3D wavelet-based scalable video coding (SVC) codec has received considerable attention lately because of its high coding performance and flexibility in bitstream scalability. It combines the motion-compensated temporal filtering (MCTF) together with the spatial decomposition to produce an embedded bitstream offering various levels of video quality over the heterogeneous networks. However, in the existing 3D wavelet-based SVC schemes, where the block types for block matching algorithms are limited, weighting matrices for block-wise motion compensation are fixed, and variations in activities of temporal subbands are not considered in the selection of the Lagrange multiplier for mode decision. In this paper, our major contribution is to provide some recent extensions to the well-known scalable subband/wavelet video codec Motion-Compensated Embedded Zero Block Coding (MC-EZBC) using three novel and content adaptive algorithms. Firstly, the enhanced hierarchical variable size block matching (Enhanced HVSBM) algorithm is proposed for the variable block size motion estimation. Then, the rate-distortion optimization (RDO) based adaptive Lagrange multiplier selection model for mode decision is presented. Finally, we introduce the adaptive weighting matrices design for overlapped block motion compensation (OBMC). Experimental results show that all the three proposed algorithms significantly improve the overall coding performance of MC-EZBC. Comparisons with other popular wavelet-based SVC codecs demonstrate the effectiveness of our improved codec in terms of both video quality assessment and computational complexity.

Hardware Efficient Architecture with Variable Block Size for Motion Estimation

Video coding standards such as MPEG-x and H.26x incorporate variable block size motion estimation (VBSME) which is highly time consuming and extremely complex from hardware implementation perspective due to huge computation. In this paper, we have discussed basic aspects of video coding and studied and compared existing architectures for VBSME. Various architectures with different pixel scanning pattern give a variety of performance results for motion vector (MV) generation, showing tradeoff between macroblock processed per second and resource requirement for computation. Aim of this paper is to design VBSME architecture which utilizes optimal resources to minimize chip area and offer adequate frame processing rate for real time implementation. Speed of computation can be improved by accessing 16 pixels of base macroblock of size 4 × 4 in single clock cycle using z scanning pattern. Widely adopted cost function for hardware implementation known as sum of absolute differences (SAD) is used for VBSME architecture with multiplexer based absolute difference calculator and partial summation term reduction (PSTR) based multioperand adders. Device utilization of proposed implementation is only 22k gates and it can process 179 HD (1920 × 1080) resolution frames in best case and 47 HD resolution frames in worst case per second. Due to such higher throughput design is well suitable for real time implementation.

Complexity reduction methods for fast motion estimation in HEVC

Motion estimation is one of the most demanding and complex tools in block based video encoders. Variable block size motion estimation (ME) and multiple reference frames in H.264/AVC make motion estimation even more complex and time consuming. In HEVC, the complexity is even higher since there are more block sizes. This paper presents an analysis of various tools involved in some fast ME algorithms and proposes some improvements to them in order to achieve a novel fast hybrid algorithm. The proposed algorithm has been tested with HEVC reference software. Simulation results show that the algorithm achieves up to 44.7% decrease in ME complexity when compared to the fast ME algorithm (Test Zone Search or TZSearch) and up to 99% reduction in ME complexity compared to full search algorithm with negligible loss in PSNR and bitrate.

A speed FPGA hardware accelerator based FSBMA-VBSME used in H.264/AVC

Image and video processing applications represent major challenge concerning real-time embedded systems. In video coding, adjacent frames are similar; this correlation can be exploited to reduce the amount of data to be transmitted, in this case reducing temporal redundancies. Actually, H.264/AVC is the most popular standard; the high performance that offers magnifies the difficulty of a real-time implementation. This complexity is mainly related to the operation of the motion estimation and requires high computational power. This paper presents an efficient hardware implementation of integer motion estimation for H.264/AVC encoder. The considered methodology is based on full search block matching algorithm for its regular algorithm implementation. The proposed architecture enables variable block size motion estimation and computes 41 motion vectors values (MVs) resulted from each 16 × 16 bloc and its derived sub-blocks. The proposed architecture calculates the best MV using a parallel process composed of three processor modules and a set of comparators three values. Implementation results based on field-programmable gate arrays devices uses Xilinx Virtex7 XC7VX550T show performance characteristics like low latency reduced up to 80 %, high processing speed reaching 443 MHz of frequency. The processing capacity is up to 1920 × 1088 HD video streams with a search range of 48 × 48.

Efficient Motion and Disparity Estimation Optimization for Low Complexity Multiview Video Coding

The use of variable block-size motion estimation (ME), disparity estimation (DE), and multiple reference frames selection aims to improve the coding efficiency of multiview video coding (MVC), however, this is at the cost of high computational complexity of these advanced coding techniques, which are not suitable for real-time video broadcasting applications. In this paper, we propose an efficient ME and DE algorithm for reducing the computational complexity of MVC. Firstly, according to the characteristics of the coded block pattern and rate distortion (RD) cost, an early DIRECT mode decision algorithm is proposed. Then, based on the characteristics of the initial search point in the ME/DE process and the observation that the best point is center-biased, an early ME/DE termination strategy is proposed. If the ME/DE early termination is not satisfied, the ME/DE search window will be reduced by applying the optimal theory. At last, two block matching search strategies are proposed to predict the best point for the ME/DE. Experimental results show that the proposed algorithm can achieve 50.05% to 77.61%, 64.83% on average encoding time saving. Meanwhile, the RD performance degradation is negligible. Especially, the proposed algorithm can be applied to not only the odd views but also the even views.

Efficient fast mode decision using mode complexity for multi-view video coding

The variable block-size motion estimation (ME) and disparity estimation (DE) are adopted in multi-view video coding (MVC) to achieve high coding efficiency. However, much higher computational complexity is also introduced in coding system, which hinders practical application of MVC. An efficient fast mode decision method using mode complexity is proposed to reduce the computational complexity. In the proposed method, mode complexity is firstly computed by using the spatial, temporal and inter-view correlation between the current macroblock (MB) and its neighboring MBs. Based on the observation that direct mode is highly possible to be the optimal mode, mode complexity is always checked in advance whether it is below a predefined threshold for providing an efficient early termination opportunity. If this early termination condition is not met, three mode types for the MBs are classified according to the value of mode complexity, i.e., simple mode, medium mode and complex mode, to speed up the encoding process by reducing the number of the variable block modes required to be checked. Furthermore, for simple and medium mode region, the rate distortion (RD) cost of mode 16×16 in the temporal prediction direction is compared with that of the disparity prediction direction, to determine in advance whether the optimal prediction direction is in the temporal prediction direction or not, for skipping unnecessary disparity estimation. Experimental results show that the proposed method is able to significantly reduce the computational load by 78.79% and the total bit rate by 0.07% on average, while only incurring a negligible loss of PSNR (about 0.04 dB on average), compared with the full mode decision (FMD) in the reference software of MVC.

Bi-Directional Trajectory Tracking With Variable Block-Size Motion Estimation for Frame Rate Up-Convertor

This paper presents bi-directional trajectory tracking with variable block-size motion estimation for frame rate up-conversion (FRUC) based on the algorithm/architecture co-exploration (AAC) design methodology. Due to concurrent exploration in both algorithm and architecture domains, the designed system requires fewer computations and lowers hardware cost, but is capable of enhancing the accuracy of motion vectors (MVs) by allowing MV refinement from coarse-grained to fine-grained. In addition, a method that uses multiple block candidates tracked by bi-directional MVs for interpolation is also presented to improve visual quality. Benefiting from AAC, we can extract architectural information at algorithmic design phase to determine the most feasible architecture; then, the proposed algorithm can be mapped onto target platform smoothly. The proposed FRUC system, which is capable of converting the frame rate from 60 fps up to 120 fps at full HD (1920 <formula formulatype="inline" xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex Notation="TeX">$\times$</tex> </formula> 1080) resolution, was successfully implemented and verified on a field-programmable gate array. This FRUC system's performance has not only been shown to surpass state-of-art alternatives in algorithmic performance, but its hardware cost is less than the comparable works described in the literature.

A low-power VLSI implementation for fast full-search variable block size motion estimation

Variable block size motion estimation (VBSME) is becoming the new coding technique in H.264/AVC. This article presents a low-power VLSI implementation for VBSME, which employs a fast full-search block-matching algorithm to reduce power consumption, while preserving the optimal motion vectors (MVs). The fast full-search algorithm is based on the comparison of the current minimum sum of absolute difference (SAD) to a conservative lower bound so that unnecessary SAD calculations can be eliminated. We first experimentally determine the specific conservative lower bound of SAD and then implement the fast full-search algorithm in FPGA and 0.18 µm CMOS technology. To the best of our knowledge, this is the first time that a fast full-search block-matching algorithm is explored to reduce power consumption in the context of VBSME and implemented in hardware. Experiment results show that the proposed design can save power consumption by 45% compared to conventional VBSME designs that give optimal MV based on the full-search algorithms.

Bit Serial Architecture for Variable Block Size Motion Estimation

H.264/AVC is the latest video coding standard adopting variable block size, quarter-pixel accuracy and motion vector prediction and multi- reference frames for motion estimations. These new features result in higher computation requirements than that for previous coding standards.The computational complexity of motion estimation is about 60% in the H.264/AVC encoder. In this paper most significant bit (MSB first) arithmetic based bit serial Variable Block Size Motion Estimation (VBSME) hardware architecture is proposed. MSB first bit serial architecture main feature is, its early termination SAD computation compared to normal bit serial architectures. With this early termination technique, number computations are reduced drastically. Hence power consumption is also less compared to parallel architectures. An efficient bit serial processing element is proposed and developed 2D architecture for processing of 4x4 block in parallel .Inter connect structure is developed in such way that data reusability is achieved between PEs. Two types of adder trees are employed for variable block size SAD calculation with less number of adders. The proposed architecture can generate up to 41 motion vectors (MVs) for each macroblock. The inter connection complexity between PEs reduced drastically compared to parallel architectures. The architecture supports processing of SDTV (640x480) with 30fps at 172.8 MHz for search range (+8, -7). We could reduce 14% of computations by using early termination technique.

An Adaptive Motion Estimation Architecture for H.264/AVC

We introduce a variable block size motion estimation architecture that is adaptive to the full search (FS) and the three-step search (3SS) algorithms. Early termination, intensive data reuse, pipel...

Adapting hierarchical bidirectional inter prediction on a GPU-based platform for 2D and 3D H.264 video coding

The H.264/AVC video coding standard introduces some improved tools in order to increase compression efficiency. Moreover, the multi-view extension of H.264/AVC, called H.264/MVC, adopts many of them. Among the new features, variable block-size motion estimation is one which contributes to high coding efficiency. Furthermore, it defines a different prediction structure that includes hierarchical bidirectional pictures, outperforming traditional Group of Pictures patterns in both scenarios: single-view and multi-view. However, these video coding techniques have high computational complexity. Several techniques have been proposed in the literature over the last few years which are aimed at accelerating the inter prediction process, but there are no works focusing on bidirectional prediction or hierarchical prediction. In this article, with the emergence of many-core processors or accelerators, a step forward is taken towards an implementation of an H.264/AVC and H.264/MVC inter prediction algorithm on a graphics processing unit. The results show a negligible rate distortion drop with a time reduction of up to 98% for the complete H.264/AVC encoder.

Fast encoding techniques for Multiview Video Coding

Multiview Video Coding (MVC) is a technique that permits efficient compression of multiview video. MVC uses variable block size motion and disparity estimation for block matching. This requires an exhaustive search process that involves all possible macroblock partition sizes. We analyze the time complexity of MVC and the methods that have been proposed to speed up motion and disparity estimation. We then propose two new methods: Previous Disparity Vector Disparity Estimation (PDV-DE) and Stereo-Motion Consistency Constraint Motion and Disparity Estimation (SMCC-MDE). PDV-DE exploits the correlation between temporal levels and disparity vectors to speed up the disparity estimation process while SMCC-MDE exploits the geometrical relationship of consecutive frame pairs to speed up motion and disparity estimation. We build a complete low complexity MVC encoding solution that combines our two methods with complementary previous methods to speed up motion and disparity search. We evaluate the complexity of our solution in terms of encoding time and number of search points. Our experimental results show that our solution can reduce the encoding time and number of search points of the standard MVC implementation (JMVM 6.0) using the fast TZ search mode up to 93.7% and 96.9%, respectively, with negligible degradation in the rate-distortion performance. Compared to the best published results, this is an improvement of up to 11% and 7%, respectively.

Power Efficient Sum of Absolute Difference Algorithms for video Compression

Video Compression (VC) is one of the resource hungry key element in video communication and is commonly achieved using Motion Estimation (ME). In this paper we proposed power efficient one bit full adder and one of the simple and easy metric called Sum of Absolute Difference (SAD) for estimating the motion vectors in motion estimation. SAD is primarily used to detect motion in the video sub system. Here we proposed power efficient 4X4 and 8X8 SAD architectures. The proposed 4X4 SAD proves that 29%, 63.23% and 61.31% improvement in leakage power, dynamic power and total power respectively as compared with existing 4X4 SAD. Similarly the proposed 8X8 SAD which proves that 57%, 46.16% and 46.78% improvement in Leakage Power (LP), Dynamic Power (DP) and Total Power (TP) respectively as compared with that of existing 8X8 SAD at the gate level. The designs are implemented in ASIC methodology using cadence tools. Keywords - SAD, ME, VC, CSA, DSP, LP, DP, TP, FPGA etc. I. INTRODUCTION Many recent multimedia applications of digital video systems such as video conferencing, video-on- demand, video-phone, distance learning and digital TV, object tracking and many more have become popular products because of their convenience. Such applications require video compression with ever higher compression ratios, better visual quality and high bandwidth. The high efficiency video compression commonly uses the efficient hardware architectures. In general the development of hardware architectures are designed to form the integrated circuits which allows parallel processing of data from various sub blocks of the architectures, however hardware architectures suffers from limitations such as algorithm flexibility due to timing dependencies, which arises from the dataflow of various blocks of the architecture. Thus the development of good architecture of video codec in integrated circuits is very important. The customization of video codec is the video compression in the modern state of the art real time Digital Signal Processing (DSP) systems. Video compression is one of the techniques in video processing system to reduce resource usage. The two primary challenges addressed during video compression are:  Limited Network bandwidth.  Limited Storage capacity. Hence the two important metrics of a video encoder are low computational complexity & low power hardware implementation. Present day world, compression ratio plays the major role in the field of video processing. The motion in the video scene will reduce the efficiency of the compression ratio. Hence the motion estimation field has seen the highest research topic and interested issue in the past a few decades. In short the motion estimation means the estimation of the displacement (or velocity) of image structures from one frame to another in a time sequence of 2-D images of the video in order to achieve video compression in video coding. The efficiency of the compression ratio can be increased by exploiting the similarities between the video frames. The simple metric system is the SAD algorithm, where the absolute differences between the corresponding elements are added up. There are varieties of video coding standards in the video processing systems; the modern video coding standard is H.264/AVC. This coding standard uses the Variable Block Size Motion Estimation (VBSME), in this standard; the computation requirements are much higher than the previous coding standards such as H.263/ MPEG-IV. In H.264/AVC, each picture frame is divided into many macro blocks.

A low-power oriented architecture for H.264 variable block size motion estimation based on a resource sharing scheme

In the Advanced Video Coding (AVC) standard, motion estimation (ME) adopts many new features to increase the coding performances such as block matching algorithm (BMA), motion vector prediction (MVP) and variable block size motion estimation (VBSME). However, VBSME is utilized in the MPEG4-AVC/H.264 standard which leads to high computational complexity and data dependency that make the hardware implementation very complex.This paper proposes a flexible VLSI architecture for full-search VBSME (FSVBSME), allowing the partitioning of the source frames into sixteen 4×4 sub-blocks and using a MVP scheme. A clock gating technique based on a distributed control unit is used for power saving. The proposed architecture was designed by Synopsys Design Compiler with 0.13μm CMOS standard cell library. Under a clock frequency of 500MHz, it allows a power consumption of about 131mW. Our VLSI architecture, compared with contemporary ones, can offer higher processing speed, lower power consumption, lower latency and lower gate count complexity.

Analysis and Design of Low-Cost Bit-Serial Architectures for Motion Estimation in H.264/AVC

Variable block-size motion estimation (VBSME) process occupies a major part of computation of an H.264 encoder, which is usually accelerated by bit-parallel hardware architectures with large I/O bit width to meet real-time constrains. However, such kind of architectures increase the area overhead and pin count, and therefore will not be suitable for area-constrained electronic consumer designs such as small portable multimedia devices. This paper addresses this problem by proposing two area efficient least significant bit (LSB) bit-serial architectures with small pin numbers. Both designs take advantage of data reusing technique in different ways for sum of absolute differences (SAD) computation and reading reference pixels, leading to a considerable reduction of memory bandwidth. The first architecture propagates the partial SAD and sum results and broadcasts the reference pixel rows whereas the second design reuse the SAD of small blocks and has a reconfigurable reference buffer leading to a better memory bandwidth when using hardware parallelism. The proposed designs benefit from several optimization techniques including an efficient serial absolute difference architecture, word length reduction by parallelism, bit truncation, mode filtering, and macroblock (MB) level subsampling, which significantly enhance their performances in terms of silicon area, throughput, latency, and power consumption. The first and second designs can support full search VBSME of 720?×?480 video with 30 frames per second (fps), two reference frames, and [?16, 15] search range at a clock frequency of 414 MHz with 29.28 k and 31.5 k gates, respectively.

Effective Search Point Reduction Algorithm and its VLSI Design for HDTV H.264/AVC Variable Block Size Motion Estimation

Variable block size motion estimation (VBSME) in H.264/AVC has greatly led to achieve an optimal inter frame encoding. However, the computation burden of the VBSME becomes the bottleneck of the H.264/AVC encoders. The conventional architecture in hardware realization is hard to adopt a fast software algorithm suitable to reduce the VBSME computation burden. Therefore, this paper presents a search point reduction (SPR) algorithm with an efficient hardware design, able to decrease the motion estimation time while maintaining the coding performance of H.264. The effectiveness of the proposed method is compared with those of existing methods with respect to chip area, operation frequency, and throughput rate. The proposed SPR algorithm increases the coding speed by around 90%; with a peak signal-to-noise ratio drop of less than 0.1 dB than that achieved by the JM reference software. The proposed SPR algorithm can operate at 200 MHz with 191 k gate count, which supports high-definition television 1280 720 format.