Full Search Motion Estimation Research Articles

GPU-Based Hierarchical Motion Estimation for High Efficiency Video Coding

Motion estimation (ME) plays a crucial role in removing the temporal redundancy for video compression. However, during the encoding process a substantial computational burden is imposed by ME due to the exhaustive evaluations of possible candidates within the searching window. In view of the increasing computing capacity of GPU, we propose a GPU-based low delay parallel ME scheme for high efficiency video coding (HEVC). In particular, considering the quadtree coding structure of HEVC, we achieve the parallelization in a hierarchical way by optimizing the ME process in a coding tree unit (CTU), prediction unit (PU), and motion vector (MV) layers. Specifically, in the CTU layer, a novel motion vector predictor determination scheme is proposed to alleviate the side effects of inaccurate MV prediction due to the removal of the CTU-level dependency. In the PU layer, a novel indexing table is particularly designed to realize an efficient cost derivation strategy. As such, the cost of each PU can be computed in a convenient and efficient manner. In an MV layer, we propose a compact descriptor to represent MV and its corresponding cost as a whole, such that the redundant branches can be further avoided in the searching process. With such an optimization strategy, the proposed scheme can completely save the encoding time for ME on CPU. Experimental results demonstrate that the proposed scheme can achieve 41% encoding time savings with the ME acceleration up to 12.7 times, and the incurred BD-BR loss is only 0.52% on average. Moreover, further experimental results show that the proposed GPU-based ME can achieve up to 200 times acceleration compared to the full search ME on CPU.

Protection of Ultrasound Image Sequence: Employing Motion Vector Reversible Watermarking

In healthcare information systems, medical data is very important for diagnosis. Most of the health institutions store their patients’ data on third-party servers. Therefore, its security is very important, since the advent of advanced multimedia and communication technology, whereby digital contents are manipulated, copied, and duplicated without leaving any trace. In this paper, a reversible watermarking technique is applied to the patients’ data (ultrasound image sequence). Since the traditional watermarking schemes can experience some permanent distortions that are not acceptable in the medical application. Thus, a reversible watermarking technique has been used, which can not only secure the ultrasound image sequence but also restore the original sequence back. For watermark embedding, the magnitude and phase angles of motion vectors of the image sequence are used that are obtained by using Full-Search block-based motion estimation algorithm. Before applying the motion estimation algorithm and watermark embedding, the histogram pre-processing is performed to avoid underflow/overflow. Unlike other state-of-the-art watermarking schemes that are reported in the last decades, the experimental results show that the proposed algorithm is simple, provides a much larger embedding capacity and better quality of the watermarked image sequence.

Novel architecture and implementation of low power oriented full search block motion estimation

High efficient video coding (HEVC) succeeds dual coding efficiency development contrasted to its predecessor H.264/MPEG-4 AVC. Though, it experiences high computational complexity caused by its quad-tree structure in motion estimation (ME). The full search ME (FSME) procedure can fundamentally raise the video encoder compression ratio (CR) while maintaining high video quality, but it is also fundamentally expensive and can comprise of more than 45% of the entire motion estimation method. The consecutive rate-constrained elimination technique (CRCE) safely removes contender motion vectors while saving the best aspirant chosen by the full search block motion estimation (FSBME) method. However, these methods have been failed to consume less power. This paper proposes low power oriented full search block based motion estimation (LP-FSBME) algorithm. The proposed algorithm decomposes every block into subsequent sub-pixel blocks by computing the sum of absolute difference (SAD) values of sub-pixel search locations utilizing the SAD benefits of neighboring integer pixel search areas for best motion vector (MV) and it diminishes the computational complexity. The proposed method hardware design is implemented with Verilog Hardware Description Language (Verilog-HDL) and synthesized by Xilinx Virtex 7 FPGA XC7VX1140T device with speed grade 1 in Xilinx software version 14.5. The proposed method is checked videos occupied 754 K gate counts, 4410 Look Up Tables (LUTs), 3312 Registers, maximum occupied frequency is 204 MHz, maximum power consumption is 40.14 mW in 1080 pixels @ 30 frames per second.

Energy Efficiency Effects of Vectorization in Data Reuse Transformations for Many-Core Processors—A Case Study †

Thread-level and data-level parallel architectures have become the design of choice in many of today’s energy-efficient computing systems. However, these architectures put substantially higher requirements on the memory subsystem than scalar architectures, making memory latency and bandwidth critical in their overall efficiency. Data reuse exploration aims at reducing the pressure on the memory subsystem by exploiting the temporal locality in data accesses. In this paper, we investigate the effects on performance and energy from a data reuse methodology combined with parallelization and vectorization in multi- and many-core processors. As a test case, a full-search motion estimation kernel is evaluated on Intel® CoreTM i7-4700K (Haswell) and i7-2600K (Sandy Bridge) multi-core processors, as well as on an Intel® Xeon PhiTM many-core processor (Knights Landing) with Streaming Single Instruction Multiple Data (SIMD) Extensions (SSE) and Advanced Vector Extensions (AVX) instruction sets. Results using a single-threaded execution on the Haswell and Sandy Bridge systems show that performance and EDP (Energy Delay Product) can be improved through data reuse transformations on the scalar code by a factor of ≈3× and ≈6×, respectively. Compared to scalar code without data reuse optimization, the SSE/AVX2 version achieves ≈10×/17× better performance and ≈92×/307× better EDP, respectively. These results can be improved by 10% to 15% using data reuse techniques. Finally, the most optimized version using data reuse and AVX512 achieves a speedup of ≈35× and an EDP improvement of ≈1192× on the Xeon Phi system. While single-threaded execution serves as a common reference point for all architectures to analyze the effects of data reuse on both scalar and vector codes, scalability with thread count is also discussed in the paper.

Constructive Synthesis of Memory-Intensive Accelerators for FPGA From Nested Loop Kernels

Field-programmable gate arrays are ideal hosts to custom accelerators for signal, image, and data processing but demand manual register transfer level design if high performance and low cost are desired. High-level synthesis reduces this design burden but requires manual design of complex on-chip and off-chip memory architectures, a major limitation in applications such as video processing. This paper presents an approach to resolve this shortcoming. A constructive process is described that can derive such accelerators, including on- and off-chip memory storage from a C description such that a user-defined throughput constraint is met. By employing a novel statement-oriented approach, dataflow intermediate models are derived and used to support simple approaches for on-/off-chip buffer partitioning, derivation of custom on-chip memory hierarchies and architecture transformation to ensure user-defined throughput constraints are met with minimum cost. When applied to accelerators for full search motion estimation, matrix multiplication, Sobel edge detection, and fast Fourier transform, it is shown how real-time performance up to an order of magnitude in advance of existing commercial HLS tools is enabled whilst including all requisite memory infrastructure. Further, optimizations are presented that reduce the on-chip buffer capacity and physical resource cost by up to 96% and 75%, respectively, whilst maintaining real-time performance.

High Performance Architecture of Motion Estimation Algorithm for Video Compression

Motion estimation (ME) is a highly computationally intensive operation in video compression. Efficient ME architectures are proposed in the literature. This paper presents an efficient low computational complexity systolic architecture for full search block matching ME (FSBME) algorithm. The proposed architecture is based on one-bit transform-based full search (FS) algorithm. The proposed ME hardware architectures perform FS ME for four macroblocks (MBs) in parallel. The proposed hardware architecture is implemented in VHDL. The FSBME hardware consumes 34% of the slices in a Xilinx Vertex XC6vlx240T FPGA device with a maximum frequency of 133[Formula: see text]MHz and is capable of processing full high definition (HD) ([Formula: see text]) frames at a rate of 60 frames per second.

An Efficient Full Search Block Motion Estimation using Trace and Sum of Off- Diagonal based Match and FPGA Implementation

H.264/AVC offer many coding tools that give higher compression ratio compared to earlier standards. These tools increase the computational complexity of the motion estimation. In this paper, we have presented a computationally efficient method to speed up the search process by minimizing the computational load on general full search. The proposed method is having two phases, wherein the first phase involves search for the nearly matching block using the trace and sum of off-diagonal elements. Only upon match, the Sum of Absolute Differences (SAD)is calculated for best motion prediction in the second phase and it is only for limited number of blocks. The trace and off-diagonal sum matching requires O(2n) computations and O(n 2 ) computations are required in case of full search. Hence, the proposed method reduces 75% to 80% of SAD computations of full search motion estimation. This leads to significant improvement in coding efficiency compared to the existing motion estimation techniques without much degradation in video quality. FPGA implementation result shows that, the design can work at the maximum clock frequency of 420 MHz with the power consumption of 34.86mW

An efficient multi-layer reference frame motion estimation for video coding

Most of the fast search motion estimation algorithms reduce the computational complexity of motion estimation (ME) greatly by checking only a few search points inside the search area. In this paper, we propose a new algorithm--multi-layer motion estimation (MME) which reduces the computational complexity of each distortion measure instead of reducing the number of search points. The conventional fast search motion estimation algorithms perform ME on the reference frame with full distortion measure; on the contrary, the MME performs ME on the layers with partial distortion measures to enhance the computational speed of ME. A layer is an image which is derived from the reference frame; each macro-pixel value in the layer represents the sum of the values of the corresponding pixels in the reference frame. A hierarchical quad-tree structure is employed in this paper to construct multiple layers from the reference frame. Experimental results on different video sequences show evidence that many motion vectors have been found similar both in the reference frame and the layers. The effectiveness of the proposed MME algorithm is compared with that of some state-of-the-art fast block matching algorithms with respect to speed and motion prediction quality. Experimental results on a wide variety of video sequences show that the proposed algorithm outperforms the other popular conventional fast search motion estimation algorithms computationally while maintaining the motion prediction quality very close to the full-search algorithm. Moreover, the proposed algorithm can achieve a maximum of 97.99 % speed-improvement rate against the fast full-search motion estimation algorithms which are based on hierarchical block matching process. The proposed MME performs the motion estimation on the layers by using three types of search patterns. The derivation of these search patterns exploits the characteristic of the center-biased motion vector distribution and that of less intensive block distortion measurement of the layers.

High performance architecture for real-time HDTV broadcasting

A novel full search motion estimation co-processor architecture design is presented in this paper. The proposed architecture efficiently reuses search area data to minimize memory I/O while fully utilizing the hardware resources. A smart processing element (PE) and an efficient simple internal memory are the main components of the proposed co-processor. An efficient algorithm is used for loading both the current block and the search area inside the PE array. The search area data flow horizontally while the current block data are stationary. As a result, the speed of the co-processor is improved in terms of the throughput and the operating frequency compared to the state-of-the-art techniques. A smart local memory and PE design guarantees a simple and a regular data flow. The design of the local memory is implemented using only registers and a simple counter. This simplifies the design by avoiding the use of complicated addressing to write or read into/from the local memory. The proposed architecture is implemented using both the FPGA and the ASIC flow design tools. For a search range of 32 × 32 and block size of 16 × 16, the architecture can perform motion estimation for 30 fps of HDTV video at 350 MHz and easily outperforms many fast full search architectures.

Nonlinear Transform for Robust Dense Block-Based Motion Estimation

We present a noniterative multiresolution motion estimation strategy, involving block-based comparisons in each detail band of a Laplacian pyramid. A novel matching score is developed and analyzed. The proposed matching score is based on a class of nonlinear transformations of Laplacian detail bands, yielding 1-bit or 2-bit representations. The matching score is evaluated in a dense full-search motion estimation setting, with synthetic video frames and an optical flow data set. Together with a strategy for combining the matching scores across resolutions, the proposed method is shown to produce smoother and more robust estimates than mean square error (MSE) in each detail band and combined. It tolerates more of nontranslational motion, such as rotation, validating the analysis, while providing much better localization of the motion discontinuities. We also provide an efficient implementation of the motion estimation strategy and show that the computational complexity of the approach is closely related to the traditional MSE block-based full-search motion estimation procedure.

The efficient optimal and suboptimal motion estimation algorithms

The new fast full-search motion estimation algorithm for optimal motion estimation is proposed in this paper. This algorithm presents fast computing method that calculates the tighter boundaries faster by exploiting the computational redundancy. The proposed algorithm, first, determines the possible motion vectors (PMVs) that are not rejected by the first two tighter boundaries to facilitate the prediction of the best initial motion vector (IMV) for the follow-up search. Thereafter, an optimal motion vector (OMV) will be traced out in the PMV set. The IMV greatly helps in the early rejection of impossible candidate blocks while tracing the OMV. Experimental results show that the proposed algorithm outperforms the other previous optimal motion estimation algorithms in reducing the number of computations on several video sequences. The proposed algorithm achieves about 156–456 speed-up gain over full search on several video sequences. But, the state-of-the-art algorithms such as adaptive MSEA and Winner Update algorithm with Integral image algorithms can achieve only about 72–382 speed-up gain over full search on the same video sequences. Finally, the proposed new fast full-search motion estimation algorithm is modified to suboptimal motion estimation algorithm, resulting only in a trivial average peak signal-to-noise ratio drop of about 0.2 dB, but it achieves a very fast computational speed.

A low-power VLSI implementation for fast full-search variable block size motion estimation

Variable block size motion estimation (VBSME) is becoming the new coding technique in H.264/AVC. This article presents a low-power VLSI implementation for VBSME, which employs a fast full-search block-matching algorithm to reduce power consumption, while preserving the optimal motion vectors (MVs). The fast full-search algorithm is based on the comparison of the current minimum sum of absolute difference (SAD) to a conservative lower bound so that unnecessary SAD calculations can be eliminated. We first experimentally determine the specific conservative lower bound of SAD and then implement the fast full-search algorithm in FPGA and 0.18 µm CMOS technology. To the best of our knowledge, this is the first time that a fast full-search block-matching algorithm is explored to reduce power consumption in the context of VBSME and implemented in hardware. Experiment results show that the proposed design can save power consumption by 45% compared to conventional VBSME designs that give optimal MV based on the full-search algorithms.

Reduced bit low power VLSI architectures for motion estimation

Low power and real time very large scale integration (VLSI) architectures of motion estimation (ME) algorithms for mobile devices and applications are presented. The power reduction is achieved by devising a novel correction recovery mechanism based on algorithms which allow the use of reduced bit sum of absolute difference (RBSAD) metric for calculating matching error and conversion to full resolution sum of absolute difference (SAD) metric whenever necessary. Parallel and pipelined architectures for high throughput of full search ME corresponding to both the full resolution SAD and the generalized RBSAD algorithm are synthesized using Xilinx Synthesis Tools (XST), where the ME designs based on reduced bit (RB) algorithms demonstrate the reduction in power consumption up to 45% and/or the reduction in area up to 38%.

Parallelization of Full Search Motion Estimation Algorithm for Parallel and Distributed Platforms

This work presents an efficient method to map the Full Search algorithm for Motion Estimation (ME) onto General Purpose Graphic Processing Unit (GPGPU) architectures using Compute Unified Device Ar...

Improved mode decision algorithm based on residues and early zero block detection in H.264/AVC

Video compression standard H.264/AVC outperforms previous standards in terms of coding efficiency but at the cost of higher computational complexity. In H.264/AVC, the variable block size full motion estimation is the most time-consuming operation. This paper presents a method to reduce the complexity of motion estimation in two stages. The first stage exploits the similarities between frames for early SKIP mode decision for a macroblock (MB) based upon a criteria formulated on the basis of the statistics of the frame difference residues. MBs that fail to qualify for the SKIP mode in the first stage spills over to the second stage where mode decision depends upon the number of zero blocks (ZB) in the MB. The study of the full search motion estimation on different sequences show that there is a strong dependence between the number of ZBs in a MB and the likelihood of a particular mode being selected. The proposed algorithm utilizes this relationship for early mode decision for a MB. The algorithm is evaluated using a wide range of test sequences from different classes. Experimental results show that the proposed algorithm gives considerable saving in encoding time and search points in the range of 36–87%. Furthermore, despite the reduction in computational complexity, the coding efficiency (picture quality and bitrate) in the proposed method is comparable to the H.264/AVC standard software Joint Model (JM12.4).

A Memory Hierarchy Model Based on Data Reuse for Full-Search Motion Estimation on High-Definition Digital Videos

The motion estimation is the most complex module in a video encoder requiring a high processing throughput and high memory bandwidth, mainly when the focus is high-definition videos. The throughput problem can be solved increasing the parallelism in the internal operations. The external memory bandwidth may be reduced using a memory hierarchy. This work presents a memory hierarchy model for a full-search motion estimation core. The proposed memory hierarchy model is based on a data reuse scheme considering the full search algorithm features. The proposed memory hierarchy expressively reduces the external memory bandwidth required for the motion estimation process, and it provides a very high data throughput for the ME core. This throughput is necessary to achieve real time when processing high-definition videos. When considering the worst bandwidth scenario, this memory hierarchy is able to reduce the external memory bandwidth in 578 times. A case study for the proposed hierarchy, using32×32search window and8×8block size, was implemented and prototyped on a Virtex 4 FPGA. The results show that it is possible to reach 38 frames per second when processing full HD frames (1920×1080pixels) using nearly 299 Mbytes per second of external memory bandwidth.

Algorithm and Architecture Co-Design of Hardware-Oriented, Modified Diamond Search for Fast Motion Estimation in H.264/AVC

In this paper, we present a new hardware-oriented, modified diamond search (HMDS) algorithm, for fast integer pel, motion estimation in H.264/AVC. We also present our co-designed, low power very large scale integration (VLSI) architecture for HMDS. The goal of HMDS is to enable the support of high quality video on low power mobile devices and low bit rate applications which typically use H.264/AVC baseline profile at levels 1-2. Our experiments use standard test sequences ranging from QCIF to high-definition 1280 × 720p video. The proposed VLSI architecture is prototyped as an field-programable gate array (FPGA)-based field programable system-on-chip. Our results show that HMDS on average has better rate-distortion performance and speedup, compared to previous state-of-the-art fast motion estimation algorithms, while its losses compared to full search motion estimation, are insignificant. Our prototyped architecture is more hardware-efficient than previous FPGA-based architectures in terms of power consumption, area, throughput, and memory utilization. We also show that its performance in terms of maximum frequency, minimum frequency, transistor count, and power consumption are comparable to that of state-of-the-art architectures implemented on application-specific integrated circuits.

Parallelized Block Match Algorithm on Multi-core Processors

In order to increase the coding efficiency of H.264/AVC, this paper proposed a parallelized approach of full search (FS) algorithm for motion estimation on Graphic Processor Unit (GPU) using computing unified device architecture (CUDA). By utilizing the independence among different MBs, we are able to take the full advantage of the computational power of CUDA and speed up the FS motion estimation by a large scale. Experimental results show that our implementation on CUDA can be as much as 50 times faster than existing CPU implementations.

RD Optimized Bandwidth Efficient Motion Estimation and Its Hardware Design With On-Demand Data Access

Data bandwidth dominates the performance and power consumption in the video encoder design. In which a low bandwidth and bandwidth aware motion estimation design enables smooth and better video quality as well as lower power consumption on data accesses. This paper proposes a bandwidth efficient motion estimation and its hardware implementation to deal with the bandwidth issues. First, an on-demand data access mechanism is proposed to acquire the reference data according to the video content for motion estimation process and thus can avoid unnecessary reference data loading. Furthermore, the available bandwidth constraint is properly modeled into our proposed rate distortion optimization framework to efficiently use the data bandwidth. Simulation results show that our proposed algorithm not only allocates proper data bandwidth for motion estimation according to video content but also saves 79.15% data bandwidth demand with 0.03 dB PSNR drop and 2.50% bitrate increase in maximum for 4 CIF resolution sequences, when compared to the fully data reuse full search motion estimation which reuses the overlapped reference data to avoid unnecessary data reloading. In addition, under the available data bandwidth constraint, our proposed algorithm can achieve 2.43%, 0.08%, and 0.20% BD-bitrate saving with 0.17 dB, 0.01 dB, and 0.01 dB BD-PSNR increase on average for high, median, and low motion sequences when compared to the full search motion estimation algorithm. The resulted design only needs 75.27 K gate counts when running at 23 MHz operating frequency for 4 CIF at 30 frames/s with 90 nm CMOS process due to its search range independent buffer design.

Specification of hierarchical-model-based fast quarter-pixel motion estimation

We propose a robust and fast quarter-pixel motion estimation algorithm. This algorithm is an advanced version of the previously proposed model-based quarter-pixel motion estimation (MBQME). MBQME has many advantages in computational complexity, running speed, and hardware implementations. But it has the problem that it does not find the quarter-pixel positions that locate beyond the half-pixel positions. That is one of limitations of model-based motion estimation methods, and it leads to both peak-SNR degradation and bit-rate increase. To solve this problem, we propose a hierarchical mathematical model with minimum interpolations. Through this model, we can determine a motion vector at every quarter-pixel point, which is perfectly compatible with the quarter-pixel motion estimation method within international video coding standards such as MPEG-4 and H.264/AVC. The simulation results show that the proposed method yields almost the same or even better peak-SNR performance than that of full-search quarter-pixel motion estimation, with much lower computational complexity.