Motion Estimation Hardware Research Articles

An efficient versatile video coding motion estimation hardware

Versatile Video Coding (VVC) is the latest video coding standard. It provides higher compression efficiency than the previous video coding standards at the cost of significant increase in computational complexity. Motion estimation (ME) is the most time consuming and memory intensive module in VVC encoder. Therefore, in this paper, we propose an efficient VVC ME hardware. It is the first VVC ME hardware in the literature. It has real time performance with small hardware area. This efficiency is achieved by using a 64 × 64 systolic processing element array to support maximum coding tree unit (CTU) size of 128 × 128 and by using a novel memory-based sum of absolute differences (SAD) adder tree to calculate SADs of 128 × 128 CTUs. The proposed VVC ME hardware reduces memory accesses significantly by using an efficient data reuse method. It can process up to 30 4 K (3840 × 2160) video frames per second.

Design and Implementation of Gray-Coded Bit-Plane Based Reconfigurable Motion Estimation Architecture Using Binary Content Addressable Memory for Video Encoder

Motion Estimation (ME) is the most power consuming module in the video encoder due to its high computational complex operations. So designing an efficient ME hardware without losing coding performance is a major challenge. This paper proposes a low-bit-depth ME technique based on Gray-Coded bit-planes and its hardware implementation using Binary Content Addressable Memory (BCAM). The proposed method significantly reduces the computational burden due to its low-bit-depth representation. The novel BCAM based ME hardware provides faster results because of its on-chip memory computation without compromising other performance parameters. It can process 8K @53.71 fps operated at maximum frequency of 155 MHz with 152.78K NAND equivalent gate count using 90 nm technology library.

Recongfigurable Hardware-Friendly Early Termination Mechanism in Motion Estimation for HEVC

Motion estimation with a quad-tree variable block size is the driver for the high performance of HEVC in video compression. However, tremendous sum of absolute difference (SAD) computations are needed for every single block size, resulting in huge memory access and power consumption. This work proposes an early termination mechanism at the hardware level that is suitable for quad-tree variable block size motion estimation without degrading video quality. The simulations show reductions in memory access to fetch pixels of about 34.03% and 51.98% saving of SAD computations. This reduction results in reduced power consumption. The simplicity of the algorithm makes it a hardware friendly mechanism, which could be adopted in any motion estimation hardware design.

Optimization of motion estimation hardware using charged system optimization with low power multipliers for H.264 encoders

Motion estimation has a critical role to play in the transmission and compression of digital video. As it has a complexity in terms of computation that is inherent it is a big challenge for the implementation of real time codes. Recently there have been studies on block matching algorithms (BMA) for bringing down the complexity in the computation and has also got better attention in many algorithms that are very effective for very large scale integration (VLSI) systems for bringing down the complexity. These BMA algorithms are relatively easy to implement which can give solutions that are suboptimal even if the entire search has not been performed. The entire search is high in terms of cost of computation that prevents it from being applied in the systems of VLSI. A novel and fast motion estimation based algorithm known as charged system search (CSS) and full search algorithm (FSA) for multipliers of low power and for H.264 encoders is proposed. The CSS is used in all fields of optimization and is well suited for most domains and does not require any gradient information. The results have proved that proposed FSA has better performance.

High Performance Architecture of Motion Estimation Algorithm for Video Compression

Motion estimation (ME) is a highly computationally intensive operation in video compression. Efficient ME architectures are proposed in the literature. This paper presents an efficient low computational complexity systolic architecture for full search block matching ME (FSBME) algorithm. The proposed architecture is based on one-bit transform-based full search (FS) algorithm. The proposed ME hardware architectures perform FS ME for four macroblocks (MBs) in parallel. The proposed hardware architecture is implemented in VHDL. The FSBME hardware consumes 34% of the slices in a Xilinx Vertex XC6vlx240T FPGA device with a maximum frequency of 133[Formula: see text]MHz and is capable of processing full high definition (HD) ([Formula: see text]) frames at a rate of 60 frames per second.

Kinect Posture Reconstruction Based on a Local Mixture of Gaussian Process Models.

Depth sensor based 3D human motion estimation hardware such as Kinect has made interactive applications more popular recently. However, it is still challenging to accurately recognize postures from a single depth camera due to the inherently noisy data derived from depth images and self-occluding action performed by the user. In this paper, we propose a new real-time probabilistic framework to enhance the accuracy of live captured postures that belong to one of the action classes in the database. We adopt the Gaussian Process model as a prior to leverage the position data obtained from Kinect and marker-based motion capture system. We also incorporate a temporal consistency term into the optimization framework to constrain the velocity variations between successive frames. To ensure that the reconstructed posture resembles the accurate parts of the observed posture, we embed a set of joint reliability measurements into the optimization framework. A major drawback of Gaussian Process is its cubic learning complexity when dealing with a large database due to the inverse of a covariance matrix. To solve the problem, we propose a new method based on a local mixture of Gaussian Processes, in which Gaussian Processes are defined in local regions of the state space. Due to the significantly decreased sample size in each local Gaussian Process, the learning time is greatly reduced. At the same time, the prediction speed is enhanced as the weighted mean prediction for a given sample is determined by the nearby local models only. Our system also allows incrementally updating a specific local Gaussian Process in real time, which enhances the likelihood of adapting to run-time postures that are different from those in the database. Experimental results demonstrate that our system can generate high quality postures even under severe self-occlusion situations, which is beneficial for real-time applications such as motion-based gaming and sport training.

An Advanced Parallel FPGA Architecture for Bi-Directional Motion Estimation

Motion estimation (ME) and motion compensation (MC) are the key elements for frame rate up-conversion (FRUC) system. Fast and accurate motion estimation is the premise of high quality motion compensation. Unlike conventional unidirectional motion estimation which brings holes, overlaps and blocking artifacts, the bi-directional motion estimation does not produce any overlapped pixel or hole in the interpolated frames. As a result, the bi-directional motion estimation has better performance than conventional unidirectional motion estimation. This paper presents an efficient FPGA architecture targeting bi-directional motion estimation hardware implementation. This proposed architecture can achieve real-time processing for 1280x720@60Hz under 200MHz operating frequency. The design is described in Verilog HDL, verified in Virtex5 FPGA platform. Experimental results show that the proposed architecture has high performance and low cost for bi-directional motion estimation algorithm. This architecture can be used for video post-processing system.

A low energy adaptive motion estimation hardware for H.264 multiview video coding

Multiview Video Coding (MVC) is the process of efficiently compressing stereo (2 views) or multiview video signals. The improved compression efficiency achieved by H.264 MVC comes with a significant increase in computational complexity. Temporal prediction and inter-view prediction are the most computationally intensive parts of H.264 MVC. Therefore, in this paper, we propose novel techniques for reducing the amount of computations performed by temporal and inter-view predictions in H.264 MVC. The proposed techniques reduce the amount of computations performed by temporal and inter-view predictions significantly with very small PSNR loss and bitrate increase. We also propose a low energy adaptive H.264 MVC motion estimation hardware for implementing the temporal and inter-view predictions including the proposed computation reduction techniques. The proposed hardware is implemented in Verilog HDL and mapped to a Xilinx Virtex-6 FPGA. The FPGA implementation is capable of processing 30∗8=240 frames per second of CIF (352×288) size 8 view video sequence or 30∗2=60 frames per second of VGA (640×480) size stereo (2 views) video sequence. The proposed techniques reduce the energy consumption of this hardware significantly.

Bit Serial Architecture for Variable Block Size Motion Estimation

H.264/AVC is the latest video coding standard adopting variable block size, quarter-pixel accuracy and motion vector prediction and multi- reference frames for motion estimations. These new features result in higher computation requirements than that for previous coding standards.The computational complexity of motion estimation is about 60% in the H.264/AVC encoder. In this paper most significant bit (MSB first) arithmetic based bit serial Variable Block Size Motion Estimation (VBSME) hardware architecture is proposed. MSB first bit serial architecture main feature is, its early termination SAD computation compared to normal bit serial architectures. With this early termination technique, number computations are reduced drastically. Hence power consumption is also less compared to parallel architectures. An efficient bit serial processing element is proposed and developed 2D architecture for processing of 4x4 block in parallel .Inter connect structure is developed in such way that data reusability is achieved between PEs. Two types of adder trees are employed for variable block size SAD calculation with less number of adders. The proposed architecture can generate up to 41 motion vectors (MVs) for each macroblock. The inter connection complexity between PEs reduced drastically compared to parallel architectures. The architecture supports processing of SDTV (640x480) with 30fps at 172.8 MHz for search range (+8, -7). We could reduce 14% of computations by using early termination technique.

Synthesis and comparison of low-power high-throughput architectures for SAD calculation

Video applications are increasingly present in consumer electronic devices which require low-power and low-energy consumption. Sum of Absolute Differences (SAD) is the most used distortion metric in video coding implementation and consumes a relative large area in the motion estimation hardware. This paper presents the standard-cells synthesis and a comprehensive analysis of various parallel hardware architectures alternatives for SAD calculation, focusing on different design constraints such as high-performance (maximum throughput) and the tradeoff between high-performance and low-power dissipation (namely an isoperformance target). Low-power techniques supported by commercial standard-cells tools are exercised in this design, such as clock gating, multi-threshold (VT) and a combination of slow and fast standard-cells. We achieved significant power reduction for the architectures with lower frequencies and higher parallelism, slow cells and mainly with only one pipeline stage.

Implementation of weighted constrained one-bit transformation based fast motion estimation

An efficient architecture that implements motion estimation based on a combination of diamond search algorithm and weighted constrained one-bit transformation is presented. The component of any motion estimation hardware that deals with the memory access consumes considerable power. This paper reports efforts at exploiting the overlap of search data among various search locations to reduce the number of memory accesses. The proposed architecture can reduce total power consumption by up to 23% compared to a recently reported architecture. Moreover, only 9 processing elements are used in this architecture, as against 16 processing elements used in conventional one bit transformation based motion estimation hardware. This in turn substantially reduces the total area of the architecture. As the proposed architecture is area efficient in addition to consuming relatively less power, its potential application lies in portable consumer video playback systems typically operated by batteries.

H.264 비디오 코덱을 위한 고속 움직임 예측기의 하드웨어 구조

(bcdP*H.264/AVC ef dP6ghi)2RjkXWl*m(motion estimation, ME) &n o. IP(intellectual property) KpETHq. m *rs(buffer), PU tk(processing unit array), SAD uv(SAD selector), MV wx(motion vector generator) yDExlzq. PU tk*16=PUExlz, {{PU*16=PE(processing element)Ek|}zq. . &n~e€i/ ‚8Yƒ„ k †T]‡ˆ‰tŠk‹o]?Œ q*ŽSAD)2FG‘’“”•k–2Fzq*kq. T m *Altera ?FPGAeStatixEP3SE80F1152C2dP3%$9?ŒH, ` &n—˜F*446.43MHzk™q. š›PT *1080p _œ` 50fpsEV8Fzq. AbstractIn this paper, we proposed a new hardware architecture for motion estimation (ME) which is the most time-consuming unit among H.264 algorithms and designed to the type of intellectual property (IP). The proposed ME hardware consists of buffer, processing unit (PU) array, SAD (sum of absolute difference) selector, and motion vector (MVgenerator). PU array is composed of 16 PUs and each PU consists of 16 processing elements (PUs). The main characteristics of the proposed hardware are that current and reference frames are re-used to reduce the number of access to the external memory and that there is no clock loss during SAD operation. The implemented ME hardware occupies 3% hardware resources of StatixEP3SE80F1152C2 which is a FPGA of Altera Inc. and can operate at up to 446.43MHz. Therefore it can process up to 50 frames of 1080p in a second.Keyword : Data reuse, H.264/AVC standard, Motion Estimation, memory access reduction, Inter predictionBbc

A Hardware-Efficient Multi-Resolution Block Matching Algorithm and its VLSI Architecture for High Definition MPEG-Like Video Encoders

High throughput, heavy bandwidth requirement, huge on-chip memory consumption, and complex data flow control are major challenges in high definition integer motion estimation hardware implementation. This paper proposes an efficient very large scale integration architecture for integer multi-resolution motion estimation based on optimized algorithm. There are three major contributions in this paper. First, this paper proposes a hardware friendly multi-resolution motion estimation algorithm well-suited for high definition video encoder. Second, parallel processing element (PE) array structure is proposed to implement three-level hierarchical motion estimation, only 256PEs are enough for one reference frame real-time high definition motion estimation by efficient PE reuse. Third, efficient on-chip reference pixel buffer sharing mechanism between integer and fractional motion estimation is proposed with almost 50% SRAM saving and memory bandwidth reduction. The proposed multi-resolution motion estimation algorithm reached a good balance between complexity and performance with rate distortion optimized variable block size motion estimation support. Also, we have achieved moderate logic circuit and on-chip SRAM consumption. The proposed architecture is well-suited for all MPEG-like video coding standards such as H.264, audio video coding standard, and VC-1.

High performance hardware architectures for one bit transform based single and multiple reference frame motion estimation

Motion Estimation (ME) is the most computationally intensive part of video compression and video enhancement systems. One bit transform (1BT) based ME algorithms have low computational complexity. Therefore, in this paper, we propose high performance systolic hardware architectures for 1BT based fixed block size (FBS) single reference frame (SRF) ME, variable block size (VBS) SRF ME, and multiple reference frame (MRF) ME. The proposed FBS-SRF ME hardware performs full search ME for 4 Macroblocks in parallel and it is faster than the 1BT based ME hardware reported in the literature. In addition, it uses less on-chip memory than the previous 1BT based ME hardware by using a novel data reuse scheme and memory organization. The proposed VBS-SRF ME hardware is also faster and uses less on-chip memory than previous 1BT based VBS-SRF ME hardware. The proposed MRF ME hardware is the first 1BT based MRF ME hardware in the literature. In order to trade-off ME performance and computational complexity, the proposed MRF ME hardware is designed as reconfigurable in order to statically configure the number and selection of reference frames based on the application requirements. The proposed hardware architectures are implemented in Verilog HDL. They are capable of processing 83 1920×1080 full High Definition frames per second. Therefore, they can be used in consumer electronics products that require real-time video processing or compression.

Efficient Hardware Implementations of Low Bit Depth Motion Estimation Algorithms

In this paper, we present efficient hardware implementation of multiplication free one-bit transform (MF1BT) based and constraint one-bit transform (C-1BT) based motion estimation (ME) algorithms, in order to provide low bit-depth representation based full search block ME hardware for real-time video encoding. We used a source pixel based linear array (SPBLA) hardware architecture for low bit depth ME for the first time in the literature. The proposed SPBLA based implementation results in a genuine data flow scheme which significantly reduces the number of data reads from the current block memory, which in turn reduces the power consumption by at least 50% compared to conventional 1BT based ME hardware architecture presented in the literature. Because of the binary nature of low bit-depth ME algorithms, their hardware architectures are more efficient than existing 8 bits/pixel representation based ME architectures.

High performance hardware architectures for one bit transform based motion estimation

Motion estimation (ME) is the most computationally intensive part of video compression and video enhancement systems. One bit transform (1BT) based ME algorithms have low computational complexity. Therefore, in this paper, we propose high performance systolic hardware architectures for 1BT based fixed block size (FBS) and variable block size (VBS) ME. The proposed ME hardware architectures perform full search ME for 4 Macroblocks in parallel and they are faster than the 1BT based ME hardware reported in the literature. They use less on-chip memory than the previous 1BT based ME hardware by using a novel data reuse scheme and memory organization. The proposed VBS ME hardware is the first 1BT based ME hardware implementing VBS ME. The proposed hardware architectures are implemented in Verilog HDL. The FBS ME and VBS ME hardware consume %34 and %49 of the slices in a Xilinx XC2VP30-7 FPGA, respectively. They can work at 113 MHz in the same FPGA and are capable of processing 49 1920times1080 full High Definition frames per second. Therefore, they can be used in consumer electronics products that require real-time video processing or compression.

An Analog Processor Array Implementing Interconnect-Efficient Reference Data Shift and SAD/SSD Extraction for Motion Estimation

A cellular analog processor array for use in variable block-size motion estimation with a new simple method for shifting reference image data is presented. The new shift method leads to a greatly reduced number of neighborhood connections for each cell of the array, and allows for all shifts within the [8,8] search area to be performed in a single step, with simple digital controls. The new shift circuitry, together with some other cell and system level optimizations, reduces silicon area and array layout complexity, enabling faster and more efficient parallel full search motion estimation hardware. A 32 × 32 cell parallel analog test array for reference-shift with a maximum block-size of 16 × 16, as well as absolute value/quadratic processing for variable block-size analog motion estimation (AME) has been designed in a 0.13 µm CMOS technology.

An all binary sub-pixel motion estimation approach and its hardware architecture

Motion estimation (ME) is the most computationally intensive part of a video coding system. Therefore it is very important to reduce its computational complexity. In this paper, a novel all-binary approach for reducing the computational complexity of sub-pixel accurate ME is proposed. An efficient hardware architecture for the proposed all-binary sub-pixel accurate motion estimation approach is also presented. The proposed hardware architecture has significantly low hardware complexity and therefore very low power consumption. It can process 720p video frames at 30 fps in a pipelined fashion together with the integer ME hardware. Therefore, it can be used in real-time low power video coding systems required by many mobile consumer electronics devices.

Scalable VLSI Architecture for Variable Block Size Integer Motion Estimation in H.264/AVC

Because of the data correlation in the motion estimation (ME) algorithm of H.264/AVC reference software, it is difficult to implement an efficient ME hardware architecture. In order to make parallel processing feasible, four modified hardware friendly ME workflows are proposed in this paper. Based on these workflows, a scalable full search ME architecture is presented, which has following characteristics: (1) The sum of absolute differences (SAD) results of 4 × 4 sub-blocks is accumulated and reused to calculate SADs of bigger sub-blocks. (2) The number of PE groups is configurable. For a search range of M × N pixels, where M is width and N is height, up to M PE groups can be configured to work in parallel with a peak processing speed of N × 16 clock cycles to fulfill a full search variable block size ME (VBSME). (3) Only conventional single port SRAM is required, which makes this architecture suitable for standard-cell-based implementation. A design with 8 PE groups has been realized with TSMC 0.18 μm CMOS technology. The core area is 2.13 mm × 1.60 mm and clock frequency is 228 MHz in typical condition (1.8 V, 25°C).