Integer Motion Estimation Research Articles

A fast and efficient data reuse scheme for HEVC Integer Motion Estimation hardware architecture

ABSTRACT High-Efficiency Video Coding (HEVC) is a video compression standard designed to improve video compression efficiency compared to its predecessor, Advanced Video Coding (AVC). HEVC provides approximately twice the compression efficiency compared to the AVC. In the HEVC system, the most computational complexity module is the Motion Estimation (ME). ME module helps reduce redundancy by compensating for motion during video compression. However, the computational complexity makes it a bottleneck in the design of high-resolution video encoders. This paper proposes an efficient architecture for the Integer ME (IME) module of the HEVC encoder. The proposed architecture introduces an efficient memory usage scheme to support the Full Search motion search algorithm. The full pipeline architecture includes 4096 Processing Elements (PE) and an adder tree to compute the Sum of Absolute Difference (SAD) for every Prediction Unit (PU) partition. The proposed architecture was designed and implemented on Xilinx Virtex-7 XC7VX550T FPGA. Our design achieved up to 275 FPS and approximately 10 FPS at 4 K video for the Search Regions of 32 × 16 and 128 × 128 pixels, respectively.

An efficient hardware architecture of integer motion estimation based on early termination and data reuse for Versatile video coding

The integer motion estimation (IME) involves high computational complexity and large amount of computation data due to the variable block sizes, and it is one of the most critical bottlenecks in video coding. The traditional algorithm for integer motion estimation loses search accuracy when simplifying the motion search process, or it becomes complex to pursue search accuracy, which is not conducive to video encoding and transmission. This paper presents a data reuse algorithm based on search window division, block splitting, the arrangement of search points in the Test Zone (TZ) search algorithm and block division. which divides pixel blocks of different sizes into several 8x8 pixel blocks and then reuses the overlapping 8x8 pixel blocks between adjacent search points of the Test Zone (TZ) search algorithm to reduce hardware resource consumption. By comparing the Sum of Absolute Differences (SAD) values of left and top pixel blocks with current pixel block, the integer motion estimation (IME) can be terminated in advance, reducing the computation complexity of integer motion estimation by over 70 %. Furthermore, the paper also presents a hardware architecture based on the data reuse algorithm, which can process 7680 × 4320@60fps videos at an operating clock frequency of 182.99 MHz, with lower resource consumption compared to similar hardware designs.

Fraction Execution Resolver Using a Hybrid Multi-CPU/GPU Encoding Scheme

Modern video coding standards make use of sub-pixel motion estimation to improve the video quality and reduce the bitrate. It is known that the fraction motion estimation (FME) part follows the integer motion estimation (IME) and adds an extra computational overhead due to the interpolation and the additional motion searches. In this paper, we propose a fraction execution resolver (FER) algorithm that lets the encoder skip the fraction part when specific criteria are met by introducing a preliminary fast test decision point (pFTDP) function for the IME part. If the pFTDP returns zero motion vectors (MVs) and the displacement search area center is also zero, then the fraction part is skipped. The pFTDP decision maker is executed only once, when a 2N × 2N block is first met, while all subsequent blocks follow this initial decision either by receiving the necessary MVs and RD from the pFTDP function or by using the precalculated IME values from the GPU kernel. For our experiments, we use a multithreaded CPU environment that also makes use of GPUs only for the integer part. Our evaluations provide a greater than 1600% encoding time saving at its peak in comparison with the default HEVC sequential mode and ideally a saving of greater than 2286% for still video frame sequences. The total average speedup for both Class A and Class B video sequences is ×13.45. The gain of the FER itself is more than ×3.9 compared with the same multithreaded setup environment. The PSNR and bitrate overhead observed are proportional to the tiling scheme used and are more related to the way CABAC works internally. The FER’s negative effects on coding efficiency are proven to be negligible. A balance between speed and quality achieved by using a lower tiling pattern is shown to minimize the negative effects of the encoding scheme pattern. The experimental results confirm the validity of our motivation, namely, that we can benefit from a software fraction execution resolver without any extra hardware costs. The gain is further increased when video sequences have more static blocks than others.

Design and exploration of low-power SAD architectures using approximate compressors for Integer Motion Estimation

Integer motion estimation performs a pivotal role in achieving video compression by exploiting the temporal redundancy of video sequences. However, the motion estimation’s computational complexity is huge and requires much computational power in real-time. On the other hand, portable mobile devices which operate on battery power have minimal computational capabilities. Since the video encoding can tolerate error up to a specific limit, the hardware complexity can be reduced by approximating the motion estimator’s fundamental modules. The major contributor to the power consumption in the motion estimator is the sum of the absolute difference (SAD) module. This paper presents SAD architectures implemented using new approximate 8:2 compressors to reduce the computational complexity. Thus, leading to a reduction in the hardware complexity and power consumption while maintaining the coding efficiency. Our proposed approximate SAD architecture achieves up to 56.7% and 49.8% savings in power compared to exact compressor-based SAD architectures and existing approximate SAD architectures respectively.

FastInter360: A Fast Inter Mode Decision for HEVC 360 Video Coding

This paper presents FastInter360, a fast inter mode decision algorithm for accelerating the encoding of ERP 360 videos. The development of FastInter360 involves an in-depth and comprehensive set of evaluations performed to understand the differences in the encoder’s behavior when encoding 360 and conventional videos. These evaluations showed that due to the texture distortions resulting from projection, the encoder presents a specific behavior when encoding 360 videos, making it more likely to use a recurrent set of encoding modes when processing 360 videos. Besides, the coding efficiency is less sensible to approximations in some encoding steps depending on the frame region. FastInter360 is then proposed to reduce the encoding complexity by exploiting these differences. FastInter360 comprises three algorithms that accelerate the encoding by performing early decision by SKIP mode, reducing integer motion estimation search range, and adjusting fractional motion estimation precision. Furthermore, each of these algorithms behaves according to distortion intensity, performing greater complexity reduction in more distorted regions. When employed altogether, these algorithms compose FastInter360, which is able to achieve an average complexity reduction of 22.84% with a coding efficiency loss of 0.652% BD-BR, on average, making FastInter360 competitive with literature works.

Design and Implementation of an Efficient Multi-Pattern Motion Estimation Search Algorithm for HEVC/H.265

Motion Estimation (ME) is the most computationally intensive block in high efficiency video coding (HEVC) due to its complex partition schemes. It consumes a large amount of power in the encoder. So designing a ME module on hardware platform imposes significant challenges. Many hardware oriented integer ME (IME) algorithms have been proposed in the literature to meet the challenges but suffer from coding efficiency degradation. In this paper we propose a low complexity IME algorithm and its hardware implementation. The proposed algorithm has been shown for two different pattern structures (PS) with 38 search points. It achieves bjontegaard bitrate (BD-BR) of 0.5% decrease and 0.077% increase for two PSs compared to test zone search (TZS) in HM 16.8. It results 8.725% and 9.072% saving in encoding time respectively. The proposed architecture supports all the HEVC partitions. It provides real time encoding of 4096 <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\times $ </tex-math></inline-formula> 2160 @60 fps operated at the maximum frequency of 353 MHz with 1591 K NAND equivalent gate count and 8.32 kB on chip memory using 90 nm technology library. Therefore, it can be used in consumer electronics applications such as security surveillance, video recording systems etc. that require high efficient HEVC encoder.

A hybrid hardware oriented motion estimation algorithm for HEVC/H.265

High Efficiency Video Coding (HEVC) is the latest video coding standard that supports high resolution videos by providing approximately twice the compression efficiency as compared to its previous standard H.264. Motion Estimation (ME) in HEVC is the most computation-intensive block as a result it becomes a bottleneck in the design of the encoder while implementing video applications on various computing platforms such as general purpose and embedded processors. So developing computational efficient architectures on Field Programmable Gate Array (FPGA) and Application Specific Integrated Circuit (ASIC) platforms is inevitable. This paper proposes a fast hybrid search pattern algorithm and its hardware architecture for encoding UHD videos. The proposed Integer ME (IME) algorithm requires an average of 11.19% less encoding time than the default Test Zone Search (TZS) algorithm in HM reference software with compromising decrement in PSNR and increment in bit rate. The proposed architecture is implemented in both FPGA and ASIC platform with TSMC 90 nm technology library. It consumed 32-33% of resources in Virtex-7 FPGA and 2784.4 K equivalent gate count (in terms of NAND ) and 18 kB of memory, respectively. The results show that maximum frequency of the proposed architecture is 162 MHz and a total power consumption is 463.4 mW. The architecture provides a maximum throughput of 2.78 Gpixels/sec due to it process $$32\times 32$$ CU comparatively much less clock cycles (59.5) as compared to the state-of-the-art literature . Further, it supports 8K UHD $$(8192\times 4320)$$ @ 78 fps.

RETRACTED: FPGA-Based Motion Estimation Algorithm Optimization

The new international video compression standards (HEVC) are referred to as video motion detection and efficient physical education video encoding. HEVC is determined by calculating the increasing complexity of video compression efficiency of the remarkable possible. Motion estimation is the most complicated arithmetic unit video encoder. Integer motion estimation and HEVC 70%, accounted for, and the video encoder's complexity calculates fractional motion estimation. High-Level Synthesis (HLS) tools algorithms have been successfully used in digital signal processing on FPGA. According to the actual requirements of the project, it is set as sports video management, target detection, video acquisition and target tracking. In order to adapt the smoothing time, the hardware configure ration consists of two parts: the intermediate results of the final optical flow vector and the optical flow calculating module generates the derivative calculation to adapt the smoothing time. Run built processor, software. FPGA chip, designed for directing data flow management and hardware components. Recommend using a first interpolation HEVC FPGA implementation scores and Vivado HLS motion estimation. The sports video target detection and tracking system designed in this paper analyzes the scene to be detected, further analyzes the moving target trajectory, and tracks the target according to the trajectory.

Fast motion estimation algorithm for HEVC video codec

Subject of Research. The paper considers motion estimation as the most difficult and resource-consuming process in the high efficiency video coding standard. A fast algorithm is proposed that skips the fractional-pixel motion estimation. Method. The fractional-pixel motion estimation, including the half-pixel and quarter-pixel motion estimation, is used as a refinement process for integer-pixel motion estimation to provide better coding efficiency. The proposed algorithm involves skipping the fractional motion estimation for prediction blocks if the block with 2N × 2N dimension, which contains more information than the other block sizes, is encoded with integer motion estimation. The decision can also be made in reference frames, so that motion estimation at half- and/or quarter-pixel levels can be skipped to reduce encoding time. Main Results. The proposed algorithm is implemented in HM-16.20 reference software. Several test sequences recommended by the International Telecommunication Union (ITU) are used to evaluate its effectiveness. Experimental results show that the proposed fast algorithm reduces the encoding time by an average of 64 % with a performance loss of less than 1 %. Practical Relevance. The proposed fast algorithm provides a reduction in coding time with a slight loss in performance. It can replace an existing unit in the H.265 standard speed up the video compression process.

Fast and energy-efficient approximate motion estimation architecture for real-time 4 K UHD processing

Approximate computing techniques exploit the characteristics of error-tolerant applications either to provide faster implementations of their computational structures or to achieve substantial improvements in terms of energy efficiency. In video encoding, the motion estimation (ME) stage, including the Integer ME (IME) and the Fractional ME (FME) steps, is the most computational intensive task and it is highly resilient to controlled losses of accuracy. In accordance, this article proposes the exploitation of approximate computing techniques to implement energy efficient dedicated hardware structures targeting the motion estimation stage of current video encoders. The designed ME architecture supports IME and FME and is able to real-time process 4 K UHD videos (3840 × 2160 pixels) at 30 frames per second, while dissipating 108.92 mW. When running at its maximum operation frequency, the architecture can process 8 K UHD videos (7680 × 4320 pixels) at 120 frames per second. The solution described in this article presents the highest throughput and the highest energy efficiency among all state-of-the-art compared works, showing that the use of approximate computing is a promising solution when implementing video encoders in dedicated hardware.

Fast Hardware-Based IME With an Idle Cycle and Computational Redundancy Reduction

Extensive efforts have been made to design hardware-based integer motion estimation (IME) that is much faster than software-based IME but suffers from the degradation in the coding efficiency. This is because the strategy for previous efforts was a simple algorithmic modification of the fast IME to facilitate the given hardware design at the expense of coding efficiency. This paper proposes a novel hardware design of the IME that not only offers real-time processing capability but also provides a flexible tradeoff between computational complexity and coding efficiency. First, a prediction unit (PU) loop unrolling scheme is proposed to solve the pipeline stall problem owing to the nature of fast IME algorithms such as the test zone search (TZS). It reduces idle cycles by 89.24%. Next, to further reduce the computational complexity of the TZS algorithm, a computational redundancy among PUs within a coding unit is reduced through a search step synchronization and search point sharing scheme. Thus, the computational complexity is reduced by 72.25%. The proposed schemes eliminate the inefficiency of hardware design; thus, they do not suffer from serious degradation in the coding efficiency. Consequently, the proposed hardware-based IME processes $7680\times4320$ videos at 30 frames per second while increasing the Bjontegaard delta bitrate by only 0.90% on average. The hardware design is synthesized using a 65 nm general purpose CMOS technology, and its gate count is 268.5K at an operating clock frequency of 500 MHz.

A Novel Parallel Motion Estimation Design and Implementation on GPU

The development of high-resolution video mounts a serious challenge to the previous video coding standard. The appearance of the new generation standards greatly relieves the dilemma but increases the coding complexity dramatically. Motion estimation is considered as the module with a relatively high computational complexity. In this paper, a parallel motion estimation implementation is proposed, which includes pre-motion estimation, integer motion estimation, and fractional motion estimation. They are highly accelerated on GPU based on AVS2, which is one of the new generation standards. A rapid mapping table algorithm is introduced to improve the efficiency of data access. In addition, a quasi-integral-graph algorithm is designed to calculate SAD or SATD efficiently for blocks of different sizes. The two novel techniques can effectively improve the utilization and efficiency of threads and exploit the characteristics of GPU. The experimental results show that the proposed parallel method can effectively accelerate the motion estimation.

An adaptive hash-based search for integer motion estimation in SCC

An adaptive hash-based search for integer motion estimation in SCC

Fast Integer Motion Estimation With Bottom-Up Motion Vector Prediction for an HEVC Encoder

Although advanced motion vector prediction (AMVP) modes based on motion estimation (ME) are selected significantly less due to the merge mode newly adopted in the high-efficiency video coding (HEVC), integer ME (IME) still occupies a large amount of computation in HEVC because the HEVC supports a highly flexible block partitioning structure. Introduction of the merge mode in HEVC substantially affects the optimal search algorithm of IME. Therefore, this gives a chance to reduce the computational complexity of ME with marginal drop of the compression efficiency. This paper proposes a new IME algorithm that significantly reduces its search ranges. Computational complexity of the proposed algorithm is reduced without serious degradation in coding efficiency by obtaining additional accurate motion vector predictors (MVPs) in a bottom-up order and searching narrow regions around multiple MVPs. These bottom-up MVPs are obtained from the prediction units (PUs) in the coding units (CUs) in the hierarchically lower level, which share the pixel area either completely or partially with the current PU. Although the encoder should perform the proposed IME in a bottom-up order from the smallest CUs to the larger CUs, several other processes, such as fractional ME and merges mode are performed in a top-down order to exploit several fast algorithms already adopted in the HEVC test model encoder. To keep compatibility with HEVC standard, the bottom-up MVPs are utilized for only IME. The bitstream is generated using standard AMVP. According to the simulation results, it is confirmed that the proposed algorithm improves the accuracy of MVPs, which leads to the reduction of IME computational complexity up to 81.95% on average with the Bjontegaard delta bitrate of 0.39%.

Energy and Rate-Aware Design for HEVC Motion Estimation Based on Pareto Efficiency

This paper presents a high-throughput energy and rate-aware hardware design for the Motion Estimation (ME) according to the High Efficiency Video Coding (HEVC) standard. The hardware design implements a modified Test Zone Search (TZS) algorithm to perform Integer Motion Estimation (IME) as well as the Fractional Motion Estimation (FME) defined by the HEVC standard. Based on evaluations with the HEVC Reference Software, a complexity-reduction strategy was adopted in the developed architecture that mainly consists of supporting only the 8x8, 16x16, 32x32, and 64x64 Prediction Unit (PU) sizes rather than using the 24 possible PU sizes. The architecture allows an external control unit selects a subset of these four PU sizes according to the energy and rate targets for a specific application. The possible operation points were determined based on Pareto Efficiency. The architecture was described in VHDL, and the synthesis results for ASIC 45nm Nangate standard cells show that the developed architecture can process at least 53 frames per second (fps) considering Ultra-High Definition (UHD) 4320p videos. When an average-case of processing is considered, the architecture is able to process 112fps at UHD 4320p resolution.

A Hardware-Oriented IME Algorithm for HEVC and Its Hardware Implementation

High Efficiency Video Coding (HEVC), the latest video coding standard, aims to provide coding performance that is much superior to that of its predecessor, H.264, especially for high definition video. To fulfill this goal, the inter-prediction unit (PU) partitions of HEVC are more complex, and the search range of motion estimation (ME) is much larger. As a result, ME becomes a bottleneck in the design of the HEVC inter predictor. In response to this challenge, we developed a hardware-oriented integer ME algorithm and the related hardware implementation. Our proposed algorithm led to a decrease in terms of the Bjontegaard Delta rate when compared with the HEVC test model 15.0. The corresponding hardware solution benefitted from 2-D data reuse supported by horizontal and vertical reference SRAMs, on-chip memory reduction supported by $4\times4$ block compression, and a low-power sum of absolute difference (SAD) tree supported by PU-level chip selection. When adopting a $32\times32$ SAD tree, the minimum and maximum required working frequency for 4K $\times2\text{K}$ at 30 frames/s videos was [375, 500] MHz. These results demonstrated that our proposed solution offered desirable improvement in both coding speed and coding performance.

Three-level pipelined multi-resolution integer motion estimation engine with optimized reference data sharing search for AVS

Integer motion estimation (IME), which acts as a key component in video encoder, is to remove temporal redundancies by searching the best integer motion vectors for dynamic partition blocks in a macro-block (MB). Huge memory bandwidth requirements and unbearable computational resource demanding are two key bottlenecks in IME engine design, especially for large search window (SW) cases. In this paper, a three-level pipelined VLSI architecture design is proposed, where efficiently integrates the reference data sharing search (RDSS) into multi-resolution motion estimation algorithm (MMEA). First, a hardware-friendly MMEA algorithm is mapped into three-level pipelined architecture with neglected coding quality loss. Second, sub-sampled RDSS coupled with Level C + are adopted to reduce on-chip memory and bandwidth at the coarsest and middle level. Data sharing between IME and fractional motion estimation (FME) is achieved by loading only a local predictive SW at the finest level. Finally, the three levels are parallelized and pipelined to guarantee the gradual refinement of MMEA and the hardware utilization. Experimental results show that the proposed architecture can reach a good balance among complexity, on-chip memory, bandwidth, and the data flow regularity. Only 320 processing elements (PE) within 550 cycles are required for IME search, where the SW is set to 256 × 256. Our architecture can achieve 1080P@30 fps real-time processing at the working frequency of 134.6 MHz, with 135 K gates and 8.93 KB on-chip memory.

Acceleration of the integer motion estimation in JEM through pre-analysis techniques

ITU-T Video Coding Expert Group and ISO/IEC Moving Picture Expert Group are studying the potential need for standardization of the future video coding technology with a compression capability that significantly exceeds that of the current High Efficiency Video Coding (HEVC) standard, including its current extensions. Both groups are working together on this exploration activity in a collaboration effort known as Joint Video Exploration Team to evaluate compression technology designs proposed by their experts in this area. Preliminary results show that the new model achieves 25% bitrate reduction, but at a cost of extremely high computational complexity (11 $$\times {}$$ ) with respect to HEVC. This paper proposes a pre-analysis algorithm designed to extract motion information of a frame, which is later used in the Motion Estimation (ME) module to speed up the encoder, showing that around 27% of the reference frames can be skipped and that more than 62% of the time is saved in the integer ME operation with a negligible impact of 0.11% in BD-rate.

A Novel Data Reuse Method to Reduce Demand on Memory Bandwidth and Power Consumption For True Motion Estimation

Motion estimation (ME) is a kernel algorithm in many video applications. Full search integer ME (FSIME) can find the best result but it usually takes plenty of time. Traditionally, only intra-frame data reuse is considered for FSIME. In this paper, a new inter-frame data reuse method is proposed to further utilize inter-frame data reuse for true ME. ME in frame rate up-conversion (FRUC-ME), a kind of true ME, is used as a case study. For FRUC-ME with the new inter-frame data reuse method, a frame is loaded to the on-chip buffer only once instead of twice and used for two interpolated frames. Two levels of the new method are proposed, Inter-D and new Inter-E, which give a good tradeoff between off-chip memory bandwidth and on-chip buffer size. New data access order is used to implement the new data reuse method. The proposed data reuse method (Inter-D) demands less off-chip memory bandwidth than its intra-frame counterpart (Intra-D), and the off-chip memory traffic and power consumption are both reduced by 37.5% for FRUC-ME.

A hardware-oriented concurrent TZ search algorithm for High-Efficiency Video Coding

High-Efficiency Video Coding (HEVC) is the latest video coding standard, in which the compression performance is double that of its predecessor, the H.264/AVC standard, while the video quality remains unchanged. In HEVC, the test zone (TZ) search algorithm is widely used for integer motion estimation because it effectively searches the good-quality motion vector with a relatively small amount of computation. However, the complex computation structure of the TZ search algorithm makes it difficult to implement it in the hardware. This paper proposes a new integer motion estimation algorithm which is designed for hardware execution by modifying the conventional TZ search to allow parallel motion estimations of all prediction unit (PU) partitions. The algorithm consists of the three phases of zonal, raster, and refinement searches. At the beginning of each phase, the algorithm obtains the search points required by the original TZ search for all PU partitions in a coding unit (CU). Then, all redundant search points are removed prior to the estimation of the motion costs, and the best search points are then selected for all PUs. Compared to the conventional TZ search algorithm, experimental results show that the proposed algorithm significantly decreases the Bjøntegaard Delta bitrate (BD-BR) by 0.84%, and it also reduces the computational complexity by 54.54%.