Complexity Of Motion Estimation Research Articles

An efficient hardware architecture of integer motion estimation based on early termination and data reuse for Versatile video coding

The integer motion estimation (IME) involves high computational complexity and large amount of computation data due to the variable block sizes, and it is one of the most critical bottlenecks in video coding. The traditional algorithm for integer motion estimation loses search accuracy when simplifying the motion search process, or it becomes complex to pursue search accuracy, which is not conducive to video encoding and transmission. This paper presents a data reuse algorithm based on search window division, block splitting, the arrangement of search points in the Test Zone (TZ) search algorithm and block division. which divides pixel blocks of different sizes into several 8x8 pixel blocks and then reuses the overlapping 8x8 pixel blocks between adjacent search points of the Test Zone (TZ) search algorithm to reduce hardware resource consumption. By comparing the Sum of Absolute Differences (SAD) values of left and top pixel blocks with current pixel block, the integer motion estimation (IME) can be terminated in advance, reducing the computation complexity of integer motion estimation by over 70 %. Furthermore, the paper also presents a hardware architecture based on the data reuse algorithm, which can process 7680 × 4320@60fps videos at an operating clock frequency of 182.99 MHz, with lower resource consumption compared to similar hardware designs.

Distributed soft video transmission based on hybrid digital and analog scheme

This study presents a novel distributed soft video delivery scheme using hybrid digital and analog framework, which realizes a relatively lightweight encoder as well as good robustness. Specifically, after applying scalar quantization to the interframes, the scaled analog information (quantization error) and the scaled digital information (quantized source) are superimposed and then transmitted. In this way, the proposed scheme directly delivers pseudo-analog symbols over the orthogonal frequency division multiplexing (OFDM) channels and involves no complicated digital codec. Moreover, we utilize cross-frame correlation to implement a distributed paradigm which further reduces the encoder complexity since the complex motion estimation and compensation algorithms are transferred to the decoder. Accordingly, the power distortion optimization scheme, which resolve the allocation of power and the parameters optimization to achieve minimum transmission distortion, is proposed. To solve it, first, we formulate the power distortion expressions regarding quantization parameters and power allocation coefficients. Subsequently, we divide the problem into two sub-problems based on the fast coordinate descent method and further propose a greedy iterative algorithm to optimize them. We also develop a data-driven optimization algorithm based on deep learning that reduces the additional delay brought by the iterative optimization method. At the receiver, we estimate the quantization output and quantization error by the modified linear least squares estimation with the virtual noise variance. Based on the simulation results, the proposed framework has a better performance in terms of peak signal-to-noise ratio and structural similarity than the relevant cutting-edge schemes while maintaining good robustness.

Real-Time Video Deraining via Global Motion Compensation and Hybrid Multi-Scale Temporal Correlations

The current video deraining algorithms mainly use adjacent frames to optimize the target frame information. However, they only consider the inter-frame temporal correlations of a uniform scale between frames, ignoring the inter-frame temporal correlations of different scales. In addition, the high computational cost is another drawback of the current video deraining algorithms. To this end, we propose a novel aggregation network that explores the inter-frame multi-scale temporal correlations for video deraining with the small computational cost. First, we construct a hybrid multi-scale feature extraction structure in the network to increase the receptive field of multi-scale features. For similar rain streaks at adjacent frames with different scales, a hybrid multi-scale residual block (HMSRB) is proposed to explore the complementary and redundant information at the temporal dimension to characterize the target frame. At the same time, we introduce an improved global context module (GCM) to avoid the complex motion estimation and motion compensation (ME&MC) operation as in previous video deraining approaches, while reducing the calculation complexity. Finally, a fusion block is utilized to adaptively merge the extracted features. Experiments demonstrate that our proposed network is proved to be more efficient and effective than the existing algorithms.

Parallel spiral search algorithm applied to integer motion estimation

Thanks to its flexible coding structure, high-efficiency video coding (HEVC) can save more coding bit rates than the previous standard, H.264. However, it also increases the complexity of integer-pixel motion estimation (IME). To speed up the encoding process, we propose a parallel spiral search (PSS) algorithm, which features the following characteristics and advantages. First, the proposed algorithm is hardware-friendly. PSS has both a fix search order that cuts the correlation between search points and a high data reuse level that facilitates the pipeline application in hardware implementation. Second, the PSS algorithm processes all prediction units (PU) blocks in parallel, which speeds up the RD calculation. Finally, the early termination strategy is proposed to end the search for unnecessary search points and further reduce search time. Experimental results show that the proposed algorithm outperforms other popular hardware-oriented IME algorithms in terms of coding speed, with the same loss of RD performance. Compared with the default full search algorithm (FSA) in the HEVC test model HM-16.7, the proposed algorithm achieves average time saving ratio of up to 92.55%, with BD-PSNR loss of 0.056 dB and an increase by 1.38% in terms of BD-BR.

Prediction-biased diamond search algorithm: a new approach to reduce motion estimation complexity

Prediction-biased diamond search algorithm: a new approach to reduce motion estimation complexity

Bidirectional Temporal-Recurrent Propagation Networks for Video Super-Resolution

Recently, convolutional neural networks have made a remarkable performance for video super-resolution. However, how to exploit the spatial and temporal information of video efficiently and effectively remains challenging. In this work, we design a bidirectional temporal-recurrent propagation unit. The bidirectional temporal-recurrent propagation unit makes it possible to flow temporal information in an RNN-like manner from frame to frame, which avoids complex motion estimation modeling and motion compensation. To better fuse the information of the two temporal-recurrent propagation units, we use channel attention mechanisms. Additionally, we recommend a progressive up-sampling method instead of one-step up-sampling. We find that progressive up-sampling gets better experimental results than one-stage up-sampling. Extensive experiments show that our algorithm outperforms several recent state-of-the-art video super-resolution (VSR) methods with a smaller model size.

Complexity reduction for HEVC encoder using one-dimensional filtering based constrained one-bit transform

This paper presents a one-dimensional-filtering-based constrained one-bit transformation (1D-C1BT)-based motion estimation to reduce the complexity of the High Efficiency Video Coding (HEVC) encoder. The proposed 1D-C1BT requires integer arithmetic additions, subtraction and shift operations to convert full resolution video frames to two bit depth 1D-C1BT frames. By embedding the 1D-C1BT in the HEVC encoder, the motion estimation (ME) complexity of the HEVC encoder is greatly reduced in terms of Motion Estimation Time (MET) and total Encoding Time (ET) in uni-prediction and bi-prediction respectively. From the experimental results, it is shown that the 1D-C1BT accompanying full search (FS) and test zonal search (TZS) algorithms greatly reduces MET and ET as compared to the FS and TZS algorithms of the HEVC encoder respectively. To further reduce the intricacy of the HEVC encoder in the fast search mode, star-diamond (SD) search accompanying 1D-C1BT is proposed. The 1D-C1BT accompanying SD further reduces MET and ET with small increment in Bjontegaard Delta Rate and small decrement in Bjontegaard Delta Peak Signal to Noise Ratio, compared to the TZS algorithm of the HEVC encoder. In the FS mode of the HEVC encoder, the proposed 1D-C1BT, accompanying FS, reduces MET by 86.00% and ET by 83.94%, with BD-Rate of 0.1452% and BD-PSNR of 0.2635% in low delay-P main profile. The proposed ME algorithms, 1D-C1BT accompanying full search (FS + 1D-C1BT), 1D-C1BT accompanying test zonal search (TZS + 1D-C1BT) and 1D-C1BT accompanying star diamond search (SD + 1D-C1BT) greatly reduce ME complexity in all the three configurations (low delay-P, low delay-B and random access main profile) of the HEVC encoder.

Complexity reduction of test zonal search for fast motion estimation in uni-prediction of High Efficiency Video Coding

The complexity of motion estimation in the High Efficiency Video Coding (HEVC) standard is very high as it uses a number of prediction block sizes. The test zonal search (TZS) mechanism is used as motion estimation algorithm in the fast search mode of HM encoder, the reference software for HEVC. In this paper, we present schemes for reducing the complexity of the TZS algorithm for uni-prediction in HEVC. The proposed mechanisms help in reducing complexity of grid, raster and refinement search stages of TZS. The performance of the proposed mechanisms is tested independently and also combined in the fast search mode of HM-16.18. The motion estimation time and the number of search points in the combined algorithm are reduced by 68.14% and 77.10% with a BD-rate of 0.30% and a BD-PSNR of $$-0.007$$ dB in comparison with the original fast search mode in the low-delay P main profile. The proposed complexity reduction schemes are compared with those adopted in other recently proposed fast motion estimation algorithms and the performance of the proposed scheme is found to be superior in terms of the reduction in motion estimation time.

Hexagon Based Compressed Diamond Algorithm for motion estimation and its dedicated VLSI system for HD videos

Motion estimation (ME) is crucial to the performance of a video codec as it recognizes and mitigates temporal redundancies between consecutive frames of a video sequence. It is one of the most computationally intensive blocks in the video encoder. Block matching algorithm (BMA) is considered as one of the best approaches for performing ME due to its ease of implementation and efficiency. Several ME algorithms are based on fixed search patterns while the others are adaptive in nature and exploit the correlation present between neighboring macroblocks of a video frame. This paper introduces an efficient ME algorithm named Hexagon Based Compressed Diamond Algorithm (HCDA) that tries to reduce the complexity of ME by lowering the number of search points in the process. The algorithm uses an early termination technique and an adaptive search pattern that can uniformly deal with slow and fast motion content in a video. The proposed HCDA uses dual threshold technique based on the available spatial information. This further helps in terms of clock cycles or hardware resources during VLSI implementation. Moreover, VLSI design for HCDA can explore the adaptive nature of the algorithm and the adopted search methodology is handled by interleaved memory organization that enhances the operational speed. Supportive data re-use scheme with effective block for SAD computation contributes to a fair trade-off between speed and on-chip area. The designs can work at a maximum frequency of 262 MHz to process 76 HD (1280 × 720) frames per second. The corresponding gate count comes out to be 35.8 K gate equivalent. The encouraging experimental results obtained under various defined metrics indicate suitability of the proposed hardware architecture for its inclusions in portable battery powered consumer video devices.

A New Rate-Complexity-Distortion Model for Fast Motion Estimation Algorithm in HEVC

In the high efficiency video coding (HEVC) standard, motion estimation (ME) adopts a quadtree coding structure and a larger search range to improve the coding performance. These advanced coding tools, however, dramatically increase the computational complexity. To accelerate ME, fast methods have been proposed that reduce ME complexity at the expense of rate-distortion (R-D) performance. However, none of these methods can claim their tradeoff to be optimal. In this paper, we propose an optimal fast motion estimation (FME) algorithm based on an analytical model of rate-complexity-distortion (R-C-D). We extend the traditional R-D model by introducing the ME complexity into it, which enables us to explicitly express the R-D performance under different complexity budgets. Based on the R-C-D model, the proposed FME finds the R-C-D optimized search ranges for some representative prediction units (PUs), which are then extended or refined dynamically according to motion characteristics for neighboring PUs. The proposed FME enables an R-D performance close to that of full search. When compared with the default FME method in the reference software of HEVC, the proposed fast algorithm can reduce the complexity by over $80\%$ while improving the R-D performance. Furthermore, our proposed FME algorithm is hardware friendly, as regular data flow enables high data reuse efficiency.

A Hardware-Efficient Block Matching Algorithm and Its Hardware Design for Variable Block Size Motion Estimation in Ultra-High-Definition Video Encoding

Variable block size motion estimation has contributed greatly to achieving an optimal interframe encoding, but involves high computational complexity and huge memory access, which is the most critical bottleneck in ultra-high-definition video encoding. This article presents a hardware-efficient block matching algorithm with an efficient hardware design that is able to reduce the computational complexity of motion estimation while providing a sustained and steady coding performance for high-quality video encoding. A three-level memory organization is proposed to reduce memory bandwidth requirement while supporting a predictive common search window. By applying multiple search strategies and early termination, the proposed design provides 1.8 to 3.7 times higher hardware efficiency than other works. Furthermore, on-chip memory has been reduced by 96.5% and off-chip bandwidth requirement has been reduced by 39.4% thanks to the proposed three-level memory organization. The corresponding power consumption is only 198mW at the highest working frequency of 500MHz. The proposed design is attractive for high-quality video encoding in real-time applications with low power consumption.

Complexity reduction for HEVC encoder using multiplication free one-bit transformation

In high-efficiency video coding (HEVC), motion estimation (ME) complexity is high. The ME complexity is greatly reduced by implementing a multiplication free one-bit transform (MF-1BT) in the HEVC encoder. Compared with the other one-bit transforms, the computational load of MF-1BT is less. The implementation of MF-1BT in a HEVC encoder greatly reduces the ME complexity in terms of ME time (MET) and total encoding time (ET). Experimental results show that the MF-1BT along with full search reduces the ME complexity in terms of percentage reduction in MET and ET by 89.03% and 87.53%, respectively, as compared with a conventional full search algorithm with a Bjontegaard delta-rate (BD-rate) of 0.1095% and Bjontegaard delta-peak signal-to-noise ratio (BD-PSNR) of 0.1311% in low delay-P main profile. The performance comparison of test zonal search (TZS) along with MF-1BT is evaluated by comparing it with the TZS algorithm in the fast search mode of the HEVC encoder. Hexagon and diamond search algorithm along with MF-1BT is also proposed in the fast search mode of the HEVC encoder to further reduce the MET and ET compared with the TZS algorithm of the HEVC encoder.

Fast Integer Motion Estimation With Bottom-Up Motion Vector Prediction for an HEVC Encoder

Although advanced motion vector prediction (AMVP) modes based on motion estimation (ME) are selected significantly less due to the merge mode newly adopted in the high-efficiency video coding (HEVC), integer ME (IME) still occupies a large amount of computation in HEVC because the HEVC supports a highly flexible block partitioning structure. Introduction of the merge mode in HEVC substantially affects the optimal search algorithm of IME. Therefore, this gives a chance to reduce the computational complexity of ME with marginal drop of the compression efficiency. This paper proposes a new IME algorithm that significantly reduces its search ranges. Computational complexity of the proposed algorithm is reduced without serious degradation in coding efficiency by obtaining additional accurate motion vector predictors (MVPs) in a bottom-up order and searching narrow regions around multiple MVPs. These bottom-up MVPs are obtained from the prediction units (PUs) in the coding units (CUs) in the hierarchically lower level, which share the pixel area either completely or partially with the current PU. Although the encoder should perform the proposed IME in a bottom-up order from the smallest CUs to the larger CUs, several other processes, such as fractional ME and merges mode are performed in a top-down order to exploit several fast algorithms already adopted in the HEVC test model encoder. To keep compatibility with HEVC standard, the bottom-up MVPs are utilized for only IME. The bitstream is generated using standard AMVP. According to the simulation results, it is confirmed that the proposed algorithm improves the accuracy of MVPs, which leads to the reduction of IME computational complexity up to 81.95% on average with the Bjontegaard delta bitrate of 0.39%.

Fast adaptive motion estimation algorithm and its efficient VLSI system for high definition videos

Motion estimation (ME) plays an important part in the functioning of the video codec by identifying and reducing the temporal redundancies in between successive frames of a video sequence. Block matching algorithm (BMA) has been accepted as one of the finest approaches for motion estimation due to its efficiency and ease of implementation. This paper presents a new and improved iterative and adaptive search strategy for block-based motion estimation along with its efficient hardware implementation. Since it is expected that there will be more demand for streaming video services on mobile devices, designing fine tuning algorithm with dedicated efficient hardware would provide significant benefits. The present motion estimation algorithm is adaptive in nature that takes into consideration the motion content of the current frame while predicting the motion vector. The adaptive nature of the search eases the complexity of motion estimation and the algorithm makes use of the correlation present among the motion vectors of the neighboring blocks to lower the number of search position. Traditionally, such adaptive algorithms are executed by CPU cores running a software stack. Since software involves a significant amount of overheads like fetching into cache, branches, stalls etc., the efficiency of the proposed algorithm can be overshadowed by the hardware platform. To avoid this, compact hardware architecture was developed which stands ahead of other existing architectures as shown in comparison. The VLSI design for the proposed algorithm presented in this work deals with the generation of the adaptive search pattern and use of interleaved memory organization fasten the operational speed. A profitable data re-use scheme and involvement of minimum processing elements required for parallelization reduce the on-chip area. Working at a frequency of 243 MHz, the proposed design can process 66 720p HD (1280 × 720) frames in one second consuming an area of 38.2 K gate equivalent. Hence, the proposed design can be incorporated in video codecs to be used in commercial devices like camcorders, smart phones and other portable, battery-powered video consumer devices. The proposed research method achieves significantly improved results both in terms of algorithmic metrics (PSNR) as well hardware performance (speed, area).

Adaptive Order Cross–Square–Hexagonal search and fuzzy tangential-weighted trade-off for H.264 in motion estimation

In recent years, various algorithms have been proposed to attain low computational complexity in motion estimation of the image sequence coding based on block matching. This paper presents an Adaptive Order Cross–Square–Hexagon (AOCSH) search algorithm, which employs a smaller cross-shaped pattern before the first step of a square pattern and replaces the square-shaped pattern with the hexagon search patterns in subsequent steps. The proposed search patterns aid in finding the best matching block, without much consideration of the vast number of search points. Here, fuzzy tangent-weighted function is also proposed to evaluate the matching points using the rate and the distortion parameters. The proposed methods are effectively applied to the block estimation process to handle the objectives of visual quality and distortion. The performance of the proposed AOCSH approach is compared to the existing methods, such as AOSH, H.264 and elastic models, using the structural similarity index (SSIM) and the peak signal to noise ratio (PSNR). From the analysis, it can be seen that the proposed approach attains the maximum SSIM of 0.99 and maximum PSNR of 40. 92 dB with reduced computation time of 3.28 s.

One-dimensional filtering based two-bit transform and its efficient hardware architecture for fast motion estimation

Video encoding is one of the most emerging application in many consumer electronics devices such as smartphones, tablets, cameras and camcorders in order to reduce required memory size for recording and utilized bandwidth for transmission purposes. The processing and battery power required for real-time video encoding is an important challenge for this kind of devices. Thus, efficient video encoding hardware architectures are required. Motion estimation process consumes most of the resources in a typical video encoder. Hence, low-complexity motion estimation approaches with efficient hardware implementations are investigated in the literature. In this paper, a novel low-complexity motion estimation approach based on one-dimensional filtering of input frames to construct two bit-planes for matching is presented. An embedded hardware architecture is also presented in order to demonstrate feasibility of the seamless integration into the state of the art consumer electronics devices. The, binarization stage of filtering based low- complexity motion estimation approaches are implemented in hardware first time in the literature. Experimental results show that the proposed motion estimation method and its hardware architecture provides an efficient alternative to existing low-complexity ME methods.

Dynamic initial search pattern defined on Cartesian product of neighboring motion vectors for fast block-based motion estimation

Block-based motion estimation is widely used in video compression for reducing the temporal data redundancy. However, it is still a main problem to effectively reduce the computational complexity of motion estimation. The median predictor is usually used for initial search center prediction, however it is not always accurate enough, especially for fast motion sequences. In this paper, a novel dynamic initial search pattern algorithm for fast block-based motion estimation is proposed. Based on the observation that the components of the current motion vector are very similar to the corresponding components of its neighboring motion vectors, Cartesian product of neighboring motion vectors is introduced to generate the proposed dynamic initial search pattern (DISP). And then the cross search pattern is employed to search for the best matching block. The number of search points of the proposed DISP is adaptive to the neighboring correlation of the current block. In fact, the proposed DISP can be considered as a generalization of median prediction scheme and it performs better in capturing the best matching block than median prediction. Experiment results show that the proposed DISP method with small cross search pattern can save about 1.71 search points on average compared with adaptive rood pattern search (ARPS) algorithm and can achieve the similar PSNR to full search (FS) algorithm by combining large cross search pattern.

Feedback-Free Binning Design for Mobile Wyner-Ziv Video Coding: An Operational Duality between Source Distortion and Channel Capacity

Most mobile video applications require the encoder to have low complexity. Wyner-Ziv (WZ) video coding removes complex motion estimation from the encoder, and provides error resilience from the embedded channel coding module. WZ video coding is regarded as a promising encoder for wireless video systems. Most WZ video coding based on channel codes is a practical implementation of the binning schemes. In this work, we present a novel two-tier binning scheme that consists of the inner and outer structure, for improving rate-distortion performance. First, we develop a Raptor coding with side information to construct the inner binning structure, which provides a lower rate. Second, for the outer binning, we model the WZ video coding architecture as a multi-access channel, so that we can exploit the property of channel capacity. Third, we exploit the duality property of WZ video coding. Based on such a property, both the primal and dual solutions are subsequently provided in this study. For the primal problem of distortion minimization, we develop dynamic programming to find the optimal binning policy, whereas for the dual problem of capacity maximization, we devise a near sum-capacity binning algorithm. The objective is to lower the coding rate with lower complexity. Experimental results showed that when compared with the state-of-the-art coding, the decoding performance and the quality of our proposed method were respectively enhanced. Besides, we observed that the decoding distortion was reduced through the proposed outer binning, while the proposed inner binning based on Raptor coding by jointly considering side information (SI) lead to a low bitrate when a target decoding quality was specified. Such findings have substantiated the effectiveness of our method.

Adaptive search range control in H.265/HEVC with error propagation resilience and hierarchical adjustment

In the state-of-the-art video coding standards H.26 4/AVC and H.265/HEVC, adaptive search range (ASR) control has achieved a significant performance in reducing the computational complexity of motion estimation. Nevertheless, the application of ASR in adjacent coding units may bring error propagation, and the ASR values may vary subject to different layers in hierarchical prediction structure. Here we propose two separate algorithms for ASR in H.265/HEVC: an error propagation resilience algorithm that eliminates the rate-distortion (RD) loss due to error propagation, and a hierarchical range adjustment algorithm that adjusts the ASR values between hierarchical layers. Simulation results show that our methods greatly improve the RD performances of traditional ASR works whilst keeping almost intact complexity reductions.

Motion Classification-Based Fast Motion Estimation for High-Efficiency Video Coding

High efficiency video coding (HEVC), the latest video coding standard, is becoming popular due to its excellent coding performance. However, the significant gain in performance is achieved at the cost of substantially higher encoding complexity than its precedent H.264/AVC, in which motion estimation (ME) is the most time-consuming module that effectively removes temporal redundancy. Test zone search (TZS) is adopted as the default fast ME method in the reference software of HEVC; however, its computational complexity is still too high for real-time applications. Several fast ME algorithms have been recently proposed to further reduce ME complexity; however, these approaches typically lead to non-negligible performance loss. To address this problem, this paper proposes a motion classification-based fast ME algorithm. By exploring the motion relationship of neighboring blocks and the coding cost characteristic, the prediction unit (PU) is first categorized into one of three classes, namely, motion-smooth PU, motion-medium PU and motion-complex PU. Then different search strategies are carefully designed for PUs of each class according to their respective motion and content characteristics. Furthermore, a fast search priority-based partial internal termination scheme is presented to rapidly skip impossible positions that speeds up cost computation during the ME process. Extensive experimental results demonstrate that the proposed algorithm achieves as much as 12.47% and 20.25% reductions in total encoder complexity when compared with TZS under low delay P and random access configuration, respectively, with negligible rate-distortion degradation; thus, it outperforms state-of-the-art fast ME algorithms in terms of both coding performance and complexity reduction.