Motion Estimation In Video Coding Research Articles

Taylor rate-distortion trade-off and adaptive block search for HEVC encoding

The advancement in high-efficiency video coding (HEVC) is adapted for defining the subsequent generation compression model to offer efficient compression without affecting the image quality. The HEVC offers improved performance than the existing compression models. Accordingly, this work develops an approach for video compression by proposing weighted entropy coding and adaptive block search-based rate-distortion (R-D) trade-off, named Taylor R-D trade-off. The adaptive block search R-D trade-off, by integrating the hexagon-based tree search algorithm (HBTSA), along with the Taylor R-D trade-off for initiating the block search process of motion estimation in video coding, which selects the optimal block. Initially, the frames are extorted from the input video. Then, the video frames are divided into macroblocks to perform the adaptive block search. Further, the suitable blocks are selected and given to the encoding process by weighted context-adaptive binary arithmetic coding (CABAC) that employs a weighted entropy function to persist the video quality after the compression. The results analysis shows that the proposed HBTSA method has improved PSNR and SSIM values of 42.717dB, and 0.991, respectively, using football, coast guard, garden, and tennis videos.

Multi-level rate-constrained successive elimination algorithm tailored to suboptimal motion estimation in HEVC

In the context of motion estimation for video coding, successive elimination algorithms (SEAs) significantly reduce the number of candidates evaluated during motion estimation without altering the resulting optimal motion vector. Nevertheless, SEA is often only used in conjunction with exhaustive search algorithms (e.g., full search). In this paper, we combine the multi-level successive elimination algorithm (ML-SEA) and the rate-constrained successive elimination algorithm (RCSEA) and show that they can be advantageously applied to suboptimal search algorithms. We demonstrate that the savings brought about by the new multi-level RCSEA (ML-RCSEA) outweigh the pre-computational costs of this approach for the Test Zonal (TZ) Search algorithm found in the HM reference encoder. We propose a novel multi-level composition pattern for performing RCSEA on an asymmetric partitioning. We introduce a double-check mechanism for RCSEA, and show that on average, it avoids computing 71% of motion vector (MV) costs. We also apply the proposed ML-RCSEA to bi-predictive refinement search and leverage a cost-based search ordering to remove 56% of error metric computations, on average. When compared to the HM reference encoder, our experiments show that the proposed solution reduces the TZ Search time by approximately 45%, contributing to an average encoding time reduction of about 7%, without increasing the Bjøntegaard delta rate (BD-Rate).

Cost-Based Search Ordering for Rate-Constrained Motion Estimation Applied to HEVC

In the context of motion estimation for video coding, combining a successive elimination algorithm (SEA) with a motion estimation algorithm reduces the number of computed cost functions without impact on rate or distortion. The SEAs use the sum of absolute differences to eliminate motion vector candidates in the search area that cannot improve the current minimum. The novelty in this paper is that instead of relying on a static geometric pattern (i.e., like a spiral), we proposed a dynamic algorithm that creates a cost-based search orderings. A dynamic cost-based search ordering not only improves elimination but also allows for early termination which removes, on average, 61% of the block-matching loop iterations performed by the rate-constrained successive elimination algorithm (RCSEA). Our experiments show that the proposed solution is 5 times faster than the high efficiency video coding (HEVC) HM encoder software in full search mode with a 0.02% impact on BD-Rate. This is twice the speed of the HEVC HM software encoder using only the RCSEA.

A study and analysis on block matching algorithms for motion estimation in video coding

In video coding system, block matching algorithms are used to identify the matching blocks in digital video sequence for motion estimation. Motion estimation is used to discover temporal redundancy in the digital video sequence. This paper shows the analysis of popular existing algorithms for motion estimation. The analysis is based on the number of search points/macroblocks and speed-up ratio. The endeavor of this revision is to present the successive researchers with some productive information in design of block matching algorithms.

Motion Estimation in Video Coding & SSIM and CIR Comparison between Adaptive Search Algorithms

An adaptive search order calculations are introduced to accelerate the square or hexagon movement estimation in advanced video coding. As indicated by the movement incline, a table of the versatile inquiry request is characterized. For each seeking emphasis, a superior pursuit request is determined and after that the best coordinated square can be found in the early hunt stage. Some test results exhibit the computational point of interest of the proposed enhanced calculation when contrasted with past calculations. Block motion estimation and compensation are played a major role in the video compression to reduce the temporal redundancies of the input videos. Assortment of square pursuit example is created in the writing to coordinate the pieces with decreased calculation multifaceted nature however without influencing the visual quality. In this paper, we have talked about the precious stone, square and hexagon look design with versatile request to discover the piece movement estimation. These inquiry examples are created as adaptive order square hexagon (AOSH) look calculation to locate the best coordinating piece without much considering vast number of hunt focuses. Likewise, the seeking capacity is defined as exchange off standard where, digression weighted capacity is recently created to assess the coordinating point. The AOSH seek calculation and digression weighted exchange off paradigm is successfully connected to piece estimation process with the point of enhancing the visual quality and compressive execution. The goal of video pressure is accepted for the proposed strategy utilizing three recordings to be specific, football, garden and tennis. The quantitative execution of the proposed technique and existing strategies are broke down utilizing Structural Similarity Index (SSIM) and Compression Improvement Ratio (CIR). The outcomes demonstrate that the AOSH technique got the great visual quality and compressive execution than the past strategies.

Homography-based block motion estimation for video coding of PTZ cameras

Due to the constrained movement of pan-tilt-zoom (PTZ) cameras, two frames in the video sequences captured by such cameras can be geometrically related by a relationship (homography). This geometric relationship is helpful for reducing the spatial redundancy in video coding. In this paper, by exploiting the homography between two frames with optical flow tracking algorithm, we propose a novel homography-based search (HBS) algorithm for block motion estimation in coding the sequences captured by PTZ cameras. In addition, adaptive thresholds are adopted in our method to classify different kinds of blocks. Compared with other traditional fast algorithms, the proposed HBS algorithm is proved to be more efficient for the sequences captured by PTZ cameras. And compared to our previous work in ICME (Cui et al., 2011), which only deals with pan-tilt (PT) camera and calculates the homography with mechanical devices, in this extended work we compute the homography by using information on images instead.

Novel two-layer motion estimation for video coding

In this paper, we proposed a novel Two-layer Motion Estimation (TME) which searches motion vectors on two layers with partial distortion measures in order to reduce the overwhelming computational complexity of Motion Estimation (ME) in video coding. A layer is an image which is derived from the reference frame such that the sum of a block of pixels in the reference frame determines the point of a layer. It has been noticed on different video sequences that many motion vectors on the layers are the same as those searched on the reference frame. The proposed TME performs a coarse search on the first layer to identify the small region in which the best candidate block is likely to be positioned and then perform local refined search on the next layer to pick the best candidate block in the located small area. The key feature of TME is its flexibility of mixing with any fast search algorithm. Experimental results on a wide variety of video sequences show that the proposed algorithm has achieved both fast speed and good motion prediction quality when compared to well known as well as the state-of-the-art fast block matching algorithms.

Improved Method to Select the Lagrange Multiplier for Rate-Distortion Based Motion Estimation in Video Coding

The motion estimation (ME) process used in the H.264/AVC reference software is based on minimizing a cost function that involves two terms (distortion and rate) that are properly balanced through a Lagrangian parameter, usually denoted as λmotion. In this paper we propose an algorithm to improve the conventional way of estimating λmotion and, consequently, the ME process. First, we show that the conventional estimation of λmotion turns out to be significantly less accurate when ME-compromising events, which make the ME process to perform poorly, happen. Second, with the aim of improving the coding efficiency in these cases, an efficient algorithm is proposed that allows the encoder to choose between three different values of λmotion for the Inter 16x16 partition size. To be more precise, for this partition size, the proposed algorithm allows the encoder to additionally test λmotion=0 and λmotion arbitrarily large, which corresponds to minimum distortion and minimum rate solutions, respectively. By testing these two extreme values, the algorithm avoids making large ME errors. The experimental results on video segments exhibiting this type of ME-compromising events reveal an average rate reduction of 2.20% for the same coding quality with respect to the JM15.1 reference software of H.264/AVC. The algorithm has been also tested in comparison with a state-of-the-art algorithm called context adaptive Lagrange multiplier. Additionally, two illustrative examples of the subjective performance improvement are provided.

깊이 카메라를 이용한 움직임 추정 방법

동영상 부호화에서 움직임 추정은 구현 복잡도에 가장 크게 영향을 미친다. 본 논문에서는 깊이 카메라와 색상 카메라를 동시에 이용하여 움직임 추정의 복잡도를 줄이는 방법을 제안한다. 깊이 카메라로부터 얻어진 거리 정보로 동영상내 객체 정보를 얻고, 비슷한 거리에 있는 화소들을 그룹화하여 동일한 객체로 라벨링을 수행한다. 라벨링된 현재 및 참조 화면내에서 움직임 추정 블록을 세부분(배경, 객체내부, 경계)의 탐색영역으로 구분하여 적응적으로 판단한다. 현재 블록이 객체내부영역이면 참조 화면에서 객체내부영역에만 움직임 추정을 탐색하고, 배경영역이면 블록은 참조 화면에서 배경영역에만 탐색한다. 모의실험을 바탕으로 전역탐색방법에 비하여 제안된 방법은 움직임 추정 차신호가 동일하게 유지되면서 탐색 복잡도가 크게 줄어듬을 확인할 수 있다. Motion estimation in video coding greatly affects implementation complexity. In this paper, a reducing method of the complexity in motion estimation is proposed by using both the depth and color cameras. We obtain object information with video sequence from distance information calculated by depth camera, then perform labeling for grouping pixels within similar distances as the same object. Three search regions (background, inside-object, boundary) are determined adaptively for each of motion estimation blocks within current and reference pictures. If a current block is the inside-object region, then motion is searched within the inside-object region of reference picture. Also if a current block is the background region, then motion is searched within the background region of reference picture. From simulation results, we can see that the proposed method compared to the full search method remains the almost same as the motion estimated difference signal and significantly reduces the searching complexity.

A Novel Search Algorithm Based on Particle Swarm Optimization and Simplex Method for Block Motion Estimation

To improve the performance of motion estimation in video coding, we propose a novel search algorithm, which utilizes the global search ability of particle swarm optimization (PSO) and the local search ability of simplex method (SM).According to the center-biased and temporal-spatial correlation features of motion vector and the global randomness of PSO, the fixed and random points are selected as the initial individual. Then, block matching process is executed by the updating of the position and velocity of individual. In order to accelerate the convergence of PSO and improve the precision of local search, mutation operator and simplex method are used. Meanwhile, the proposed algorithm intelligently uses early termination strategies based on the feature of static macro block. Experimental results demonstrate that the proposed algorithm has better PSNR values than existing fast block matching algorithms especially for violent motion with negligible increase of computational complexity.

A New Multiplication-Free Block Matching Criterion

In block matching, the matching criterion plays a pivotal role in both matching accuracy and computational complexity. Mean squared error (MSE) is one most widely accepted benchmark for matching accuracy. To avoid multiplication operations for simpler implementation, the sum of absolute difference (SAD) is normally taken as a substitute to approximate MSE. In this paper, we first examine statistically the quantitative deviation of SAD from MSE in terms of maximum and average deviations. To minimize the average deviation, a new measure, namely, weighted sum of absolute difference (WSAD), is proposed in a two-element case first, for which an optimal weight is obtained theoretically. The weight is approximately adapted to 1/2 so that the weighting operation can be simplified to a shift operation, and the resulting multiplication-free WSAD also shows better matching accuracy with a smaller deviation from MSE than SAD. The two-element WSAD is further extended to a multi-element case in a multilevel pyramid structure. The proposed WSAD is experimentally validated by applying it in block motion estimation for video coding, showing better rate-distortion performance than SAD.

An 8.6 mW 25 Mvertices/s 400-MFLOPS 800-MOPS 8.91 mm$^{2}$ Multimedia Stream Processor Core for Mobile Applications

<para xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> For the demands of mobile multimedia applications, a stream processor core is designed with 8.91 <formula formulatype="inline"><tex Notation="TeX">${\hbox {mm}}^{2}$</tex></formula> area in 0.18<formula formulatype="inline"><tex Notation="TeX">$\ \mu{\hbox {m}}$</tex></formula> CMOS technology at 50 MHz. Several techniques and architectures are proposed to achieve high performance with low power consumption. First of all, an optimized core pipeline is designed with 2-issue VLIW architecture to achieve the processing capability of 400 MFLOPS or 800 MOPS. In addition, adaptive multi-thread scheme can increase the performance by increasing hardware utilization, and the proposed configurable memory array architecture can reduce off-chip memory accessing frequency by caching both input data and output results. Furthermore, for graphics applications, a geometry-content-aware technique called early-rejection-after-transformation is proposed to remove redundant operations for invisible triangles. As for video applications, the proposed video accelerating instruction set can support motion estimation for video coding. Experimental results show that 86% power reduction and more than ten times speedup of the VLIW architecture can be achieved with the proposed techniques to provide the processing speed of 25 Mvertices/s and power consumption of 8.6 mW. Moreover, CIF (352 <formula formulatype="inline"><tex Notation="TeX">$\times$</tex></formula> 288) 30 fps video encoding with the search range of {H[<formula formulatype="inline"> <tex Notation="TeX">$-$</tex></formula>24,24), V[<formula formulatype="inline"> <tex Notation="TeX">$-$</tex></formula>16,16]} is also supported by the proposed stream processor. By supporting both video and graphics functions, this highly efficient, high performance, and low power processor core is applicable to multimedia mobile devices. </para>

Integer-pixel motion estimation H.264/AVC accelerator architecture with optimal memory management

This paper presents an improved accelerator core for H.264/AVC video-coding motion estimation. The proposed hardware architecture meets the integer-pixel, full-search block-matching algorithm requirements with an optimal memory management and an effective data-path. Performance characteristics like low latency, high processing speed and efficiency near 100% are achieved without a high control overhead. The core calculates the 41 best motion vectors using a pipeline process. It is composed of a systolic 16 × 16 processor elements array, a sum of absolute differences adder tree and a Lagrangian rate/distortion cost optimizer. Implementation results based on FPGA devices in a system on chip (SoC) structure using VHDL are included.

Algorithm/Architecture Co-Design of 3-D Spatio&amp;ndash;Temporal Motion Estimation for Video Coding

This paper presents a new spatio-temporal motion estimation algorithm and its VLSI architecture for video coding based on algorithm and architecture co-design methodology. The algorithm consists of the new strategies of spatio-temporal motion vector prediction, modified one-at-a-time search scheme, and multiple update paths derived from optimization theory. The hardware specification is for high-definition video coding. We applied the ME algorithm to H.264 reference software. Our algorithm surpasses recently published research and achieves close performance to full search. The VLSI implementation proves the low cost feature of our algorithm. The algorithm and architecture co-design concept is highly emphasized in this paper. We provide some quantitative example to show the necessity of algorithm and architecture co-design

Efficient multilevel successive elimination algorithms for block matching motion estimation

The authors present fast algorithms to reduce the computations of block matching algorithms, for motion estimation in video coding. Efficient multilevel successive elimination algorithms are based on the multilevel successive elimination. Efficient multilevel successive elimination algorithms consist of four algorithms. The first algorithm is given by the sum of absolute difference between the sum norms of sub-blocks in a multilevel successive elimination algorithm (MSEA) using the partial distortion elimination technique. By using the first algorithm, computations of MSEA can be reduced further. In the second algorithm, the sum of absolute difference (SAD) is calculated adaptively from large value to small value according to the absolute difference values between pixels of blocks. By using the second algorithm, the partial distortion elimination in the SAD calculation can occur early, therefore the computations of MSEA can be reduced. The second algorithm is useful not only with MSEA, but also with all kinds of block matching algorithms. In the third algorithm, the elimination level of the MSEA can be estimated. Accordingly, the computations of the MSEA related to the level lower than the estimated level can be reduced. The fourth algorithm is, first of all, to search the motion vector over the half sampled search points. At the second search, the authors search the unsampled search points around the tested search points where the motion vector may exist from the first search results. The motion estimation accuracy of the fourth algorithm is nearly 100% and the computations can be greatly reduced.

A multilevel successive elimination algorithm for block matching motion estimation

An efficient algorithm is proposed to reduce the computation cost of block matching algorithms for motion estimation in video coding. Based on a new insight in block matching algorithms, we extend the successive elimination algorithm to a multilevel case. By using the sum norms of the blocks and the subblocks, tighter and tighter decision boundaries can be obtained for eliminating the search positions. The efficiency of the proposed algorithm combined with the full search algorithm and several fast search algorithms is verified by simulation results.

Optimised feature map finite-state vector quantisation for image coding

An optimised feature map finite-state vector quantisation (referred to as optimised FMFSVQ) is presented for image coding. Based on the block-based gradient descent search algorithm used for motion estimation in video coding, the optimised FMFSVQ system finds a neighbourhood-based optimal codevector for each input vector by extending the associated state codebook stage by stage, thus rendering each state quantiser a variable rate vector quantisation. The optimised FMFSVQ system can be interpreted as a cascade of a finite-state vector quantiser and classified vector quantisers. Furthermore, an adaptive optimised FMFSVQ is obtained. Experiments demonstrate the superior rate-distortion performance of the adaptive optimised FMFSVQ compared with the original adaptive FMFSVQ and the memoryless vector quantisation.

An integrated approach to block based motion estimation for video coding

In video communication systems motion compensation techniques allow one to remove temporal redundancy among frames in order to achieve reductions in the bit rate. Block based approaches are widely used for such a task, since they offer a good balance between performance and complexity; however, the resulting motion fields still present spatial and temporal redundancies, the exploitation of which would allow a greater bit rate reduction. The motion estimation method proposed attains such a goal, maintaining the block based partitioning of the resulting field. It uses a spatio-temporal prediction followed by a lossy coding of the residual field by vector quantization. The prediction is based on autocompensation, enhanced with the aid of some additional parameters so that it provides a stationary error field. The statistical distribution of such a residual field allows vector quantization to perform efficiently, even when the method is applied to video sequences with extremely varying motion activities. Both the prediction and the vector quantization simply use an index for each block to code its motion vector. Finally, the indexes are spatially organized in two single images that are coded with a region based technique.

A block-based gradient descent search algorithm for block motion estimation in video coding

A block-based gradient descent search (BBGDS) algorithm is proposed in this paper to perform block motion estimation in video coding. The BBGDS evaluates the values of a given objective function starting from a small centralized checking block. The minimum within the checking block is found, and the gradient descent direction where the minimum is expected to lie is used to determine the search direction and the position of the new checking block. The BBGDS is compared with full search (FS), three-step search (TSS), one-at-a-time search (OTS), and new three-step search (NTSS). Experimental results show that the proposed technique provides competitive performance with reduced computational complexity.

Efficient block motion estimation using integral projections

Several efficient techniques have previously been proposed to reduce the computational burden of block matching for motion estimation in video coding. The goal is efficient motion estimation with minimal error in the motion-compensated predicted image. We present a block motion estimation scheme which is based on matching of integral projections of motion blocks with those of the search area in the previous frame. Like many other techniques, ours operates in a sequence of decreasing search radii, but it performs an exhaustive search at each level of the hierarchy. The projection method is much less computationally costly than block matching and has a prediction accuracy of competitive quality with both full block matching and other efficient techniques. Our algorithm also takes advantage of the similarity of motion vectors in adjacent blocks in typical imagery by subsampling the motion vector field. It has the added advantage of allowing parallel computation of vertical and horizontal displacements.