Variable Block Size Motion Estimation Research Articles

Early SKIP mode decision for MVC using inter-view correlation

In the joint multiview video model (JMVM) proposed by JVT, the variable block-size motion estimation (ME) and disparity estimation (DE) have been employed to determine the best coding mode for each macroblock (MB). These give a high coding efficiency for multiview video coding (MVC), however, they cause a very high computational complexity in encoding system. This paper proposes to reduce the complexities of the ME and DE processes with an early SKIP mode decision algorithm based on the analysis of prediction mode distribution regarding the corresponding MBs in the neighbor view. In this method, the mode decision procedures of most of MBs can be early terminated, and thus much of computation for ME and DE can be greatly reduced. Simulation results demonstrate that our algorithm can achieve computational saving of 46–57% (depending on the tested sequences) with no significant loss of rate-distortion performance.

Low-Power H.264 Video Compression Architectures for Mobile Communication

This paper presents a method to reduce the computation and memory access for variable block size motion estimation (ME) using pixel truncation. Previous work has focused on implementing pixel truncation using a fixed-block-size (16 times 16 pixels) ME. However, pixel truncation fails to give satisfactory results for smaller block partitions. In this paper, we analyze the effect of truncating pixels for smaller block partitions and propose a method to improve the frame prediction. Our method is able to reduce the total computation and memory access compared to conventional full-search method without significantly degrading picture quality. With unique data arrangement, the proposed architectures are able to save up to 53% energy compared to the conventional full-search architecture. This makes such architectures attractive for H.264 application in future mobile devices.

Adaptive Variable Block-Size Early Motion Estimation Termination Algorithm for H.264/AVC Video Coding Standard

<para xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> The variable block-size motion estimation (ME) process is the H.264/AVC encoder's most time-consuming function. This letter proposes to reduce the complexity of the ME process with an early termination algorithm that features an adaptive threshold based on the statistical characteristics of rate-distortion (RD) cost regarding current block and previously processed blocks and modes. In this method, most motion searches can be stopped early, with a large number of search points saved. A region-based search is also suggested to further reduce the computation required for full search ME. A search point reduction scheme for the fast motion estimation of H.264/AVC is also introduced, and the experimental results illustrate how the proposed method reduces ME times for full search and fast motion estimation by about 77 and 31%, respectively, despite the insignificant degradation of RD performance. </para>

High performance hardware architectures for one bit transform based motion estimation

Motion estimation (ME) is the most computationally intensive part of video compression and video enhancement systems. One bit transform (1BT) based ME algorithms have low computational complexity. Therefore, in this paper, we propose high performance systolic hardware architectures for 1BT based fixed block size (FBS) and variable block size (VBS) ME. The proposed ME hardware architectures perform full search ME for 4 Macroblocks in parallel and they are faster than the 1BT based ME hardware reported in the literature. They use less on-chip memory than the previous 1BT based ME hardware by using a novel data reuse scheme and memory organization. The proposed VBS ME hardware is the first 1BT based ME hardware implementing VBS ME. The proposed hardware architectures are implemented in Verilog HDL. The FBS ME and VBS ME hardware consume %34 and %49 of the slices in a Xilinx XC2VP30-7 FPGA, respectively. They can work at 113 MHz in the same FPGA and are capable of processing 49 1920times1080 full High Definition frames per second. Therefore, they can be used in consumer electronics products that require real-time video processing or compression.

A novel VLSI architecture for full-search variable block-size motion estimation

Variable block-size motion estimation (VBSME) has become an important technique in H.264/AVC to improve video quality. In this paper, we propose a scalable VLSI architecture for VBSME in H.264/AVC based on a full-search motion estimation algorithm. A new scan order is introduced to re-use the sum of absolute differences (SAD) values of smaller sub-blocks on an as-early-as-possible basis, thus the complexity of the required hardware resources, such as registers, multiplexers, and controls is reduced. It also spreads the timing for the final SAD outputs so that the number of output buses is reduced. The architecture is flexible and scalable with regard to the size of the searching windows and PE arrays. Compared to the conventional approaches, the architecture shows higher throughput rate with less hardware. After logic synthesis using DongbuAnam 0.18 mum standard cell library, the number of gates is 39K (16 PEs) in two-input equivalent NAND gates and the maximum operating clock frequency is 416 MHz (256 fps@CIF).

An Analog Processor Array Implementing Interconnect-Efficient Reference Data Shift and SAD/SSD Extraction for Motion Estimation

A cellular analog processor array for use in variable block-size motion estimation with a new simple method for shifting reference image data is presented. The new shift method leads to a greatly reduced number of neighborhood connections for each cell of the array, and allows for all shifts within the [8,8] search area to be performed in a single step, with simple digital controls. The new shift circuitry, together with some other cell and system level optimizations, reduces silicon area and array layout complexity, enabling faster and more efficient parallel full search motion estimation hardware. A 32 × 32 cell parallel analog test array for reference-shift with a maximum block-size of 16 × 16, as well as absolute value/quadratic processing for variable block-size analog motion estimation (AME) has been designed in a 0.13 µm CMOS technology.

Scalable high-throughput variable block size motion estimation architecture

Variable block size (VBS) motion compensated prediction (MCP) provides substantial rate-distortion performance gain over conventional fixed-block-size MCP and is a key feature of the H.264/AVC video coding standard. VBS–MCP requires the encoder to perform VBS motion estimation (VBSME), a computationally complex operation. In this paper, we propose a high motion vector throughput full-search VBSME architecture. High performance is achieved by performing parallel computations for multiple pixels within a macroblock, as well as computing several candidate motion vector (MV) positions in parallel. Two implementations of the architecture are examined, a four pixel-parallel implementation, and a higher performance 16 pixel-parallel implementation. A high degree of scalability is achieved by allowing for a variable length processing element array, where more processing elements yields a higher degree of candidate MV parallelism. The proposed architecture achieves a throughput exceeding current full-search VBSME architectures.

A Low Bandwidth Integer Motion Estimation Module for MPEG-2 to H.264 Transcoding

A low-bandwidth Integer Motion Estimation (IME) module is proposed for MPEG-2 to H.264 transcoding. Based on bandwidth reduction method proposed in Ref. 1), a ping-pang memory control scheme combined with Partial Sum of Absolute Differences (SAD) Variable Block Size Motion Estimation (VBSME) architecture are realized. Experiment results show bandwidth of the proposed architecture is 70.6% of H.264 regular IME (Level C+ scheme, 2 Macro Block (MB) stitched vertically), while the on-chip memory size is 11.7% of that.

Variable Block Size Motion Vector Retrieval Schemes for H.264 Inter Frame Error Concealment

With the ubiquitous application of Internet and wireless networks, H.264 video communication becomes more and more common. However, due to the high-efficiently predictive coding and the variable length entropy coding, it is more sensitive to transmission errors. The current error concealment (EC) scheme, which utilizes the spatial and temporal correlations to conceal the corrupted region, produces unsatisfied boundary artifacts. In this paper, first we propose variable block size error concealment (VBSEC) scheme inspired by variable block size motion estimation (VBSME) in H.264. This scheme provides four EC modes and four sub-block partitions. The whole corrupted macro-block (MB) will be divided into variable block size adaptively according to the actual motion. More precise motion vectors (MV) will be predicted for each sub-block. Then MV refinement (MVR) scheme is proposed to refine the MV of the heterogeneous sub-block by utilizing three step search (TSS) algorithm adaptively. Both VBSEC and MVR are based on our directional spatio-temporal boundary matching algorithm (DSTBMA). By utilizing these schemes, we can reconstruct the corrupted MB in the inter frame more accurately. The experimental results show that our proposed scheme can obtain better objective and subjective EC quality, respectively compared with the boundary matching algorithm (BMA) adopted in the JM11.0 reference software, spatio-temporal boundary matching algorithm (STBMA) and other comparable EC methods.

Towards an H.264/AVC HW/SW Integrated Solution: An Efficient VBSME Architecture

This paper presents an efficient real-time variable block size motion estimation architecture. The proposed architecture provides motion vectors for each 16 times 16 block and its 40 sub-blocks. The proposed architecture is a single-instruction multiple-data architecture integrated with embedded SRAMs on one chip. The architecture has been prototyped using Xilinx Virtex-4 XC4VSX35-10 field-programmable gate array. It processes 30-CIF fps using 71-MHz clock frequency. Its maximum clock frequencyuency is 187.7 MHz and the maximum throughput is 20 4CIF fps. The prototyped architecture has 175 k gates and 18 kbits embedded SRAM.

Edge Block Detection and Motion Vector Information Based Fast VBSME Algorithm

Compared with previous standards, H.264/AVC adopts variable block size motion estimation (VBSME) and multiple reference frames (MRF) to improve the video quality. Full search motion estimation algorithm (FS), which calculates every search candidate in the search window for 7 block type with multiple reference frames, consumes massive computation power. Mathematical analysis reveals that the aliasing problem of subsampling algorithm comes from high frequency signal components. Moreover, high frequency signal components are also the main issues that make MRF algorithm essential. As we know, a picture being rich of texture must contain lots of high frequency signals. So based on these mathematical investigations, two fast VBSME algorithms are proposed in this paper, namely edge block detection based subsampling method and motion vector based MRF early termination algorithm. Experiments show that strong correlation exists among the motion vectors of those blocks belonging to the same macroblock. Through exploiting this feature, a dynamically adjustment of the search ranges of integer motion estimation is proposed in this paper. Combing our proposed algorithms with UMHS almost saves 96–98% Integer Motion Estimation (IME) time compared to the exhaustive search algorithm. The induced coding quality loss is less than 0.8% bitrate increase or 0.04 dB PSNR decline on average.

Variable step search fast motion estimation for H.264/AVC video coder

In this paper, we propose a variable step search fast motion estimation algorithm using local statistics of neighboring motion vectors. Using the degree of correlation between neighboring motion vectors, motion search range of each macroblock is adaptively adjusted to reduce unnecessary search points. Based on the determined search range, motion vector is obtained by variable step search step. Experimental results show that comparing to fast full spiral search, fixed step, and fast variable block size motion estimation algorithms, the motion search points and computing time of proposed algorithm are dramatically reduced with acceptable loss in coding performance.

Content-Aware Fast Motion Estimation for H.264/AVC

The key to high performance in video coding lies on efficiently reducing the temporal redundancies. For this purpose, H.264/AVC coding standard has adopted variable block size motion estimation on multiple reference frames to improve the coding gain. However, the computational complexity of motion estimation is also increased in proportion to the product of the reference frame number and the intermode number. The mathematical analysis in this paper reveals that the prediction errors mainly depend on the image edge gradient amplitude and quantization parameter. Consequently, this paper proposes the image content based early termination algorithm, which outperforms the original method adopted by JVT reference software, especially at high and moderate bit rates. In light of rate-distortion theory, this paper also relates the homogeneity of image to the quantization parameter. For the homogenous block, its search computation for futile reference frames and intermodes can be efficiently discarded. Therefore, the computation saving performance increases with the value of quantization parameter. These content based fast algorithms were integrated with Unsymmetrical-cross Multihexagon-grid Search (UMHexagonS) algorithm to demonstrate their performance. Compared to the original UMHexagonS fast matching algorithm, 26.14–54.97% search time can be saved with an average of 0.0369 dB coding quality degradation.

A high-performance reconfigurable VLSI architecture for vbsme in H.264

VBSME (variable block size motion estimation) is adopted in the MPEG-4 AVC/H.264 standard. In order to increase the hardware utilization for VBSME with FSBMA (full search block matching algorithm), this paper proposed a new high-performance reconfigurable VLSI architecture to support meander-like scan format for a high data reuse of search area. The architecture can support the three data flows of the scan format through a reconfigurable computing array and a memory of the search area. The computing array can achieve 100% processing element (PE) utilization and can reuse the smaller blocks' SADs to calculate 41 motion vectors (MVs) of a 16X16 block in parallel. The design is implemented with TSMC 0.18 mum CMOS technology. Under a clock frequency of 180 MHz, the architecture allows the real-time processing of 1280 x 720 at 45 fps in a search range [-16, +16].

Fast Motion Estimation for H.264/AVC in Walsh–Hadamard Domain

In this paper, we propose a fast full-pel variable block size motion estimation algorithm called fast Walsh search in variable block size (FWS-VBS). FWS-VBS employs a new error measurement defined in Walsh-Hadamard domain, which is called partial sum-of-absolute difference, to identify likely mismatches. Mismatches are rejected by thresholding method and the thresholds are determined adaptively to cater for different activity levels in each block. Early termination techniques are employed to further reduce the number of candidates and modes to be searched of each block. Experimental results show that FWS-VBS performs equally well to the exhaustive full search algorithm and requires only about 10% of the computation time.

Reconfigurable Variable Block Size Motion Estimation Architecture for Search Range Reduction Algorithm

Variable Block Size Motion Estimation (VBSME) costs a lot of computation during video coding. Search range reduction algorithm is widely used to reduce computational cost of motion estimation. Current VBSME designs are not suitable for this algorithm. This paper proposes a reconfigurable design of VBSME which can be efficiently used with search range reduction algorithm. While using proposed design, n×m reference MBs form an MB array which can be processed in parallel, n and m can be configured according to the new search range shape calculated by algorithm. In this way, the parallelism of proposed design is very flexible and can be adapted to any search range shape. The hardware resource is also fully used while performing VBSME. There are two primary reconfigurable modules in this design: PEGA (PE Group Array) and SAD comparator. By using TSMC 0.18μm standard cell library, the implementation results show that the hardware cost of design which uses 16 PEGs (PE Groups) is about 179K Gates, the clock frequency is 167MHz.

A Parallel Memory System for Variable Block-Size Motion Estimation Algorithms

This paper proposes an efficient parallel memory system for algorithms applied in fixed and variable block-size motion estimation (VBSME). The proposed system is implemented by a novel combination of two parallel memory architectures. The distribution of data among the memory modules is modified over contemporary approaches and the optimized address computation unit enables a rapid address generation for accessed memory locations. Furthermore, the introduced data permutation scheme organizes data efficiently for storage and retrieval. The proposed system enables up to 4 X speedup in data storage and retrieves data up to 55% faster for VBSME compared with the reference implementations. With a 0.18- mum CMOS technology, the proposed memory addressing and data permutation scheme can be clocked at 980 MHz operating frequency with a cost of less than 6 kgates. On FPGA, the system can operate at 200 MHz with less than 700 logic elements. The results show that the proposed system is applicable to real-time VBSME at HDTV resolution.

Parallel Improved HDTV720p Targeted Propagate Partial SAD Architecture for Variable Block Size Motion Estimation in H.264/AVC

One hardware efficient and high speed architecture for variable block size motion estimation (VBSME) in H.264 is presented in this paper. By improving the pipeline structure and processing element (PE) circuits, the system latency and hardware cost is reduced, which makes this structure more hardware efficient than the original Propagate Partial SAD architecture. For small and middle frame size picture's coding, the proposed structure can save 12.1% hardware cost compared with original Propagate Partial SAD structure. In the case of HDTV, since small inter modes trivially contribute to the coding quality, we remove modes below 8 × 8 in our design. By adopting mode reduction technique, when the set number of PE array is less than 8, the proposed mode reduction based Propagate Partial SAD structure can work at faster clock speed and consume less hardware cost than widely used SAD Tree architecture. It is more robust to the high speed timing constraint when parallel processing is considered. With TSMC 0.18μm technology in worst work conditions (1.62 V, 125°C), its peak throughput of 8-set PE array structure is 720p@30 Hz with 128 × 64 search range and 5 reference frames. 12 k gates hardware cost can be reduced by our design compared with the parallel SAD Tree architecture.

Adaptive Search Range Algorithms for Variable Block Size Motion Estimation in H.264/AVC

Comparing with search pattern motion estimation (ME) algorithms, adaptive search range (ASR) algorithms are more fundamental, regular and flexible. In variable block size motion estimation (VBSME), ASR algorithms can be applied whether on a whole frame (frame level), or on an entire macroblock which includes up to forty-one blocks (macroblock level), or just on a single block (block level). In the other hand, in H.264/AVC, not the motion vectors (MVs) but the motion vector differences (MVDs) are coded and the median motion vector predictors (median-MVPs) are used to place the search centers. In this sense, it can be thought that the search windows (SWs) are centered at the positions pointed by median-MVPs, the search ranges (SRs) play the role of limiting MVDs. Thus it is reasonable for considering using MVDs to predict SRs. In this paper, one of the MB level and two of the block level, at all three MVD based SR prediction algorithms are proposed. VBSME based experiments are carried out to assess the proposed algorithms. Comparisons between the proposed three algorithms and the previously proposed one given in [8] are done in terms of encoding quality and computational complexity.

Efficient Reference Frame Selector for H.264

This paper proposes a simple yet effective mechanism to select proper reference frames for H.264 motion estimation. Unlike traditional video codecs, H.264 permits more than one reference frame for increased precision in motion estimation. However, motion estimation is complicated by variable block-size motion estimation, which requires significant encoding complexity to identify the best inter-coding. Our smart selection mechanism selects suitable reference frames by means of a simple test, and only the selected frames will be searched further in the variable block size motion estimation. One major advantage of our mechanism is that it enables working with any existing motion search algorithms developed for the traditional single reference frame. Experimental results demonstrate the effectiveness of our proposed algorithm.