High Efficiency Video Coding Encoder Research Articles

Hardware Efficient Integrated In-loop Filter for HEVC Encoder

The deblocking filter (DF) and the sample adaptive offset (SAO) filter, which aids in enhancing the subjective quality of the image, make up the in-loop filter of the high-efficiency video coding (HEVC) encoder and decoder. The in-loop filter significantly increases the computational load on the HEVC encoder. It is challenging to design an in-loop filter on hardware that can handle intensive computations while using the least amount of on-chip memory, taking external memory traffic and dependencies simultaneously delivering high throughput to support Ultra HD video applications. The proposed design employs the following strategies to address these issues. This work proposes an address generation technique for pipelined horizontal and vertical filtering in DF, that avoids a transpose buffer which otherwise is required. This enables easy pipelining and parallelization thus improving throughput while reducing the on-chip memory utilization. A simplified SAO filter with parallel-pipelined processing is included in the design. These features enable the design to support ultra-HD 7680 × 4320 @ 40 fps video applications. The proposed hardware architecture has a total gate count of 7.73 K LUTs and 2.8 K slice registers, and it is implemented on a 28 nm field programmable gate array (FPGA) platform.

Variable Block Size Fractional Motion Estimation Hardware Architecture for VVC and HEVC Standards

The ever-increasing growth in digital video consumption motivates the research and development of new video coding standards to improve coding efficiency. Among the several new tools that the Versatile Video Coding (VVC) standard introduced lies the Quadtree Plus Multi-Type Tree (QT+MTT), which improves upon the quadtree of High Efficiency Video Coding (HEVC) in terms of coding efficiency. Nevertheless, such improvements come at the cost of increased complexity. Hence, to meet the real-time and energy efficiency requirements, embedded devices must adopt techniques to effectively reduce the VVC complexity, including customized hardware accelerators. This work presents a low-energy hardware architecture design for the Fractional Motion Estimation (FME), one of the most critical steps of HEVC and VVC encoders. The proposed architecture supports the Switchable Interpolation Filter (SIF) of VVC and was synthesized to enable Variable Block Size (VBS) from 8×8 up to 128×128 sized blocks so that it can perform the FME of most partitions of the QT+MTT from VVC. The results show increases of at most 4.52% in area and 10.79% in total power for the extra hardware needed to support VBS, and up to 9.15% and 15.72%, in area and power, respectively, for supporting up to 128×128 block sizes.

Pre-encoding based temporal dependent rate–distortion optimization for HEVC

Rate–distortion optimization (RDO) is a crucial technique in block-based hybrid video encoders, which determines the coding option for a certain coding unit to achieve optimal rate–distortion (R–D) performance. However, the current RDO method, which is applied in High Efficiency Video Coding (HEVC), is far from being globally optimal as it ignores the interaction among coding options. In fact, the current coding option not only determines the R–D point of the coded unit but also influences the achievable R–D​ performance of the subsequent coding units due to the spatial–temporal dependencies among the block/frame optimizations. In this paper, we first investigate the R–D characteristics, including operational R–D curves and R–D dependencies, in HEVC through several simulation experiments. It is shown that under the hierarchical fixed QP setting in the HEVC encoder, the temporal R–D dependence dominates the coding process, and the distortion of the reference frame has a strong impact on that of coded frames while less influence on rate. We then introduce distortion effect factors to quantitatively measure the temporal R–D​ dependency. The distortion effect factor at frame level reflects the correlation degree between the current coding frame and the previous coded frames, and the one at block level reflects the influence degree that the distortion of a block to that of subsequent blocks which refer to it in temporal domain. Finally, a pre-encoding based temporal dependent RDO is proposed, which adaptively adjusts quantization parameters and Lagrange multipliers according to the distortion effect factors. Experimental results show that under configurations of low-delay B and P frames, the proposed method can achieve 5.4% and 5.5% bit rate savings on average, respectively, with around 7% encoding complexity increment, compared with the original HEVC encoder.

HEVC extension for phase hologram compression.

Compressing digital holograms have growing attention nowadays due to their huge amount of original data sizes. Although many progresses have been reported for full-complex holograms, the coding performance for phase-only holograms (POHs) has been quite limited so far. In this paper, we present a very efficient compression method for POHs. It extends the conventional video coding standard HEVC (High Efficiency Video Coding) in such a way that the standard can be able to compress not only the natural images but also the phase images effectively. First, we suggest a proper way to calculate differences, distances and clipped values for phase signals by considering the inherent periodicity of phases. Then, some of the HEVC encoding and decoding processes are modified accordingly. The experimental results show that the proposed extension significantly outperforms the original HEVC for POH video sequences; specifically, average BD-rate reductions of 63.3% and 65.5% are achieved in phase domain and numerical reconstruction domain, respectively. It is worth mentioning that the modified encoding & decoding processes are very minimal and also applicable to the VVC (Versatile Video Coding), which is a successor of the HEVC.

Deep learning-based video coding optimisation of H.265

Today's multi-media applications need high video quality with low bitrates. However, it is restricted in its capacity to provide higher quality than earlier coding methods. Deep learning (DL) approaches for video coding have shown compression capacities equal to or better than traditional methods, including high-efficiency video coding (HEVC) methods. The trade-off between compression efficiency and encoding/decoding complexity, optimisation for perceptual nature of semantic dependability, specialisation, and universality, the federalised layout of various deep toolkits, etc. remains unclear. HEVC encoding is more efficient than previous standards. Improved efficiency is driven by intra image prediction, which incorporates more prior directions (35 modes) than previous standards. Its high efficiency comes from balancing encoder complexity and dependability. This article presents DL, which uses a convolutional neural network to predict the best model with the least rate-distortion (RD) and further promotes study into deep learning video coding (DLVC).

Comparative Objective Analysis of video quality Between H.265/HEVC and H.264/AVC

This paper shows the difference in video quality between two compressed videos using H.264 AVC (Advanced Video Coding) and H.265 HEVC (High Efficiency Video Coding) encoders. To evaluate video completely it should be prepared video files that have a variety of bit rates and content. There are many video quality assessment methods. We can divide the min to subjective and objective methods. Subjective are conducted by a human perception and objective are conducted by a computer software which is calculating the video quality. All of these methods have theirs advantages and disadvantages. To generate compressed videos from the original video FFmpeg (Fast Forward-moving picture experts group) converter has been used. MSU-VQMT (Moscow State University’s Video Quality Measure- mentTool) was used to perform comparative objective analysis of video quality. Delta, MSE (mean square Error), MSAD (Mean Sum of Absolute Difference), PSNR (Peak Signal-to-Noise Ratio), and SSIM (Structural Similarity Index Measure) metrics were measured. The result from FFmpeg shows that the size of the compressed video using the H.265 codec has been decreased by 50% compared to the compressed video using the H.264 codec. The comparison of metrics shows that delta, MSAD, PSNR, and SSIM values of H.265 encoded video was decreased, while Delta and MSE value was increased compared to H.264 encoded Video. That’s mean the overall video quality was decreased but the video size was enhanced.

Transformer-Based Data-Driven Video Coding Acceleration for Industrial Applications

With the exploding development of edge intelligence and smart industry, deep learning-based intelligent industrial solutions are promptly applied in the manufacturing process. Many intelligent industrial solutions such as automatic manufacturing inspection are computer vision based and require fast and efficient video encoding techniques so that video streams can be processed as quickly as possible either at the edge cluster or over the cloud. As one of the most popular video coding standards, the high efficiency video coding (HEVC) standard has been applied to various industrial scenes. However, HEVC brings not only a higher compression rate but also a significant increase in encoding complexity, which hinders its practical application in industrial scenarios. Fortunately, a large amount of video coding data makes it possible to accelerate the encoding process in the industry. To speed up the video coding process in some industrial scenes, this paper proposes a data-driven fast approach for coding tree unit (CTU) partitioning in HEVC intracoding. First, we propose a method to represent the partition result of a CTU as a column vector of length 21. Then, we employ lots of encoding data produced in normal industry scenes to train transformer models used to predict the partitioning vector of the CTU. Finally, the final partitioning structure of the CTU is generated from the partitioning vector after a postprocessing operation and used by an industrial encoder. Compared with the original HEVC encoder used by some industrial applications, experiment results show that our approach achieves 58.77% encoding time reduction with 3.9% bit rate loss, which indicates that our data-driven approach for video coding has great capacity working in industrial applications.

High-Level Synthesis Implementation of an Embedded Real-Time HEVC Intra Encoder on FPGA for Media Applications

High Efficiency Video Coding (HEVC) is the key enabling technology for numerous modern media applications. Overcoming its computational complexity and customizing its rich features for real-time HEVC encoder implementations, calls for automated design methodologies. This article introduces the first complete High-Level Synthesis (HLS) implementation for HEVC intra encoder on FPGA. The C source code of our open-source Kvazaar HEVC encoder is used as a design entry point for HLS that is applied throughout the whole encoder design process, from data-intensive coding tools like intra prediction and discrete transforms to more control-oriented tools such as context-adaptive binary arithmetic coding (CABAC). Our prototype is run on Nokia AirFrame Cloud Server equipped with 2.4 GHz dual 14-core Intel Xeon processors and two Intel Arria 10 PCIe FPGA accelerator cards with 40 Gigabit Ethernet. This proof-of-concept system is designed for hardware-accelerated HEVC encoding and it achieves real-time 4K coding speed up to 120 fps. The coding performance can be easily scaled up by adding practically any number of network-connected FPGA cards to the system. These results indicate that our HLS proposal not only boosts development time, but also provides previously unseen design scalability with competitive performance over the existing FPGA and ASIC encoder implementations.

Design and Implementation of Deep Depth Decision Algorithm for Complexity Reduction in High Efficiency Video Coding (HEVC)

High efficiency video (HEVC) coding made its mark as a codec which compress with low bit rate than its preceding codec that is H.264, but the factor that stop HEVC from many applications is its complex encoding procedure. The rate distortion optimisation (RDO) cost calculation in HEVC consume complex calculations. In this paper, we propose a method to cross out the issue of complex calculations by replacing the traditional inter-prediction procedure of brute force search for RDO by a deep convolutional neural network to predict and perform this process. In the first step, the modelling of the deep depth decision algorithm is done with optimum specifications using convolutional neural network (CNN). In the next step, the model is designed and trained with dataset and validated. The trained model is tested by pipelining it to the original HEVC encoder to check its performance. We also evaluate the efficiency of the model by comparing the average time of encoding for various resolution video input. The testing is done with mutually independent input to maintain the accuracy of the system. The system shows a substantial saving in encoding time that proves the complexity reduction in HEVC.

Design and Implementation of an Efficient Multi-Pattern Motion Estimation Search Algorithm for HEVC/H.265

Motion Estimation (ME) is the most computationally intensive block in high efficiency video coding (HEVC) due to its complex partition schemes. It consumes a large amount of power in the encoder. So designing a ME module on hardware platform imposes significant challenges. Many hardware oriented integer ME (IME) algorithms have been proposed in the literature to meet the challenges but suffer from coding efficiency degradation. In this paper we propose a low complexity IME algorithm and its hardware implementation. The proposed algorithm has been shown for two different pattern structures (PS) with 38 search points. It achieves bjontegaard bitrate (BD-BR) of 0.5% decrease and 0.077% increase for two PSs compared to test zone search (TZS) in HM 16.8. It results 8.725% and 9.072% saving in encoding time respectively. The proposed architecture supports all the HEVC partitions. It provides real time encoding of 4096 <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\times $ </tex-math></inline-formula> 2160 @60 fps operated at the maximum frequency of 353 MHz with 1591 K NAND equivalent gate count and 8.32 kB on chip memory using 90 nm technology library. Therefore, it can be used in consumer electronics applications such as security surveillance, video recording systems etc. that require high efficient HEVC encoder.

A novel high-efficiency holography image compression method, based on HEVC, Wavelet, and nearest-neighbor interpolation

One of the critical challenges facing 3D video systems and images such as holography lies in their compression technique. High-efficiency video coding (HEVC) has emerged as one of the leading schemes to address this challenge. In this article, a novel method based on wavelet transform is presented to improve HEVC, particularly in digital holography systems (object plane). In this regard, wavelet and resizing are included in the coding process, while extra HEVC decoders and encoders are added to predict and decrease errors in the target. Simulation results reveals that the proposed algorithm reduces Bjøntegaard-Delta (BD) bitrate 17.5% (based on average BD-Rate values) compared to the original HEVC (H.265) scheme while maintaining signal fidelity and even enhancing it slightly. We observe an increased BD-peak-signal-to-noise ratio (BD-PSNR) in real and imaginary parts of digital holograms of high rate quantization values up to 1.1 dB.

Low-Complexity Rate-Distortion Optimization for HEVC Encoders

Typical high efficiency video coding (HEVC) encoders use rate-distortion optimization (RDO) to select the best coding parameters among numerous candidates to achieve high compression efficiency. However, calculating the rate-distortion cost for each coding parameter entails high computational complexity, which limits the real-time application of these encoders. In this study, time-consuming processes, such as reconstruction and entropy coding, are removed during the calculation of the full RD cost to reduce the complexity of the encoder. Instead, RD costs are estimated for transformed coefficients by using the RD cost model, which is made up of a distortion model based on the distribution features of the transformed coefficients, a residual rate model based on the characteristics of the entropy coding module, and a simple and convenient header rate model. In addition to the full RD cost, typical encoders also adopt fast RD cost, which is composed of the sum of absolute transform differences (SATD) and header bit. To further reduce encoding complexity, three simplified SATD (SSATD) methods based on DCT and DST features are proposed to serve as substitutes for SATD in fast RD cost, thus reducing the computational complexity of SATD calculation. Experimental results show that 35.53%, 14.57%, and 14.37% of coding time can be reduced respectively under AI, RA and LDP configurations with negligible RD performance loss when applying the proposed RD cost model and the SSATD methods to HM16.7. Besides, the proposed methods also can reduce 4%-9% of encoding time and exhibits the same RD performance as state-of-the-art methods.

An optimal weighted HEVC coding for video compression

In recent years, the applications of multimedia are rising in greedy mode and hence the amount of video transactions are also increasing exponentially. It has been shouted a great demands for researchers to develop an effective models for video encoding & decoding, compression, bandwidth allocation, transmission channel and congestion control. This research work mainly focusses on to develop an efficient video compression standard algorithm named as High Efficiency Video Coding (HEVC). It proposes a new improved Iterative based propagation update in water wave Optimization Algorithm (IPU-WWO) that optimize the weights in adopted HEVC encoding. The proposed HEVC produces double the ratio of data compression at the similar level of quality of the video and considerably enhanced video quality with same bit rate compared with Advanced Video Coding (AVC). The performance of proposed IPU-WWO is compared with other conventional methods like Artificial Bee Colony (ABC), Firefly (FF), Particle Swarm Optimization (PSO) and Genetic Algorithm (GA) with respect to Peak Signal to Noise Ratio (PSNR). The experimental results show that proposed method produces better compression rate of 3.03%, 4.98%, 6.82%, and 8.38% compared with PSO, GA, FF, and ABC, respectively with high PSNR for the block size of 4. Moreover, the proposed method outperforms the state-of-the-art fast encoding algorithms for in terms of compression and performance.

Regressive rate-distortion trade-off with weighted entropy coding for HEVC encoding

High-Efficiency Video Coding (HEVC) is the standard employed for the subsequent compressing of the video without degrading the quality of the image. HEVC renders effective performance compared to the existing compression standards as the encoding time is minimal. With the concern for quality and video compression, this paper proposes a modified version for HEVC encoding using the Ordered Tree-based Hex-Octagon based block Search and Rate-Distortion trade-off (OrTHO-Search-based RD) for motion estimation and weighted Context-Adaptive Binary Arithmetic Coding (CABAC). The proposed OrTHO-Search-based RD-dependent HEVC renders a good-quality video after compression with half the compression standard when compared with the other existing compression standards. In the motion estimation block, the OrTHO-Search is employed with a new RD trade-off in such a way that the Conditional Autoregressive Value at Risk concept modifies the existing RD trade-off measure. The bit rates of the proposed method are reduced with effective coding. The experimentation and analysis of the methods are performed using four videos from CIPR SIF Sequences and one video from Xiph.org Video Test Media datasets and the analysis based on Peak Signal-to-Noise Ratio (PSNR), Structural Similarity Index (SSIM), reveals that the proposed method acquired the maximal PSNR of 45.1132 dB, maximal SSIM of 0. 9918, and minimal computational time of 0.2135 min.

Co‐design implementation of High Efficiency Video Coding standard encoder on Zynq MPSoC

SummaryStatistical analysis of High Efficiency Video Coding (HEVC) encoder reveals that in the motion compensation block, the interpolation filter consumes more than 30% in the encoder time in comparison with other blocks. In this paper, we start with an optimized hardware implementation of the interpolation filter on field‐programmable gate array (FPGA) based on Xilinx setup environment. In a second step, a Hardware/Software (HW/SW) co‐design implementation of HM16.7 encoder is performed on Zynq MPSoC platform to evaluate the proposed interpolation filter IP in terms of total encoder run‐time, taking advantages of both processing units (quad‐core ARM Cortex TM‐A53 processor and Programmable Logic FPGA component) available on the Zynq MPSoC. The proposed architecture of luma and chroma filters was simulated and synthesized on Xilinx XCZU7EV‐2FFVC1156 FPGA at 250‐MHz clock frequency. The synthesis results present an optimized power consumption of 3.308 mW for higher resolutions (2560 × 1600 and 1920 × 1080) at 50 fps with the use of just 1% of the FPGA resources. The experimental results of the co‐design implementation of HEVC encoder present a speedup of 2 times (41% in PeopleOnStreet sequence) in terms of processing time compared to the software alone implementation, with a an increase of 0.51% of bit rate and a very small degradation of peak signal‐to‐noise ratio (PSNR) (0.01%).

Taylor rate-distortion trade-off and adaptive block search for HEVC encoding

The advancement in high-efficiency video coding (HEVC) is adapted for defining the subsequent generation compression model to offer efficient compression without affecting the image quality. The HEVC offers improved performance than the existing compression models. Accordingly, this work develops an approach for video compression by proposing weighted entropy coding and adaptive block search-based rate-distortion (R-D) trade-off, named Taylor R-D trade-off. The adaptive block search R-D trade-off, by integrating the hexagon-based tree search algorithm (HBTSA), along with the Taylor R-D trade-off for initiating the block search process of motion estimation in video coding, which selects the optimal block. Initially, the frames are extorted from the input video. Then, the video frames are divided into macroblocks to perform the adaptive block search. Further, the suitable blocks are selected and given to the encoding process by weighted context-adaptive binary arithmetic coding (CABAC) that employs a weighted entropy function to persist the video quality after the compression. The results analysis shows that the proposed HBTSA method has improved PSNR and SSIM values of 42.717dB, and 0.991, respectively, using football, coast guard, garden, and tennis videos.

A Fast Encoding Scheme for High Efficiency Video Coding

Background:High-Efficiency Video Coding (HEVC) is a recent video compression standard. It provides better compression performance compared to its predecessor, H.264/AVC. However, the computational complexity of the HEVC encoder is much higher than that of H.264/AVC encoder. This makes HEVC less attractive to be used in real-time applications and in devices with limited resources (e.g., low memory, low processing power, etc.). The increased computational complexity of HEVC is partly due to its use of a variable size Transform Unit (TU) selection algorithm which successively performs transform operations using transform units of different sizes before selecting the optimal transform unit size. In this paper, a fast transform unit size selection method is proposed to reduce the computational complexity of an HEVC encoder.Methods:Bayesian decision theory is used to predict the size of the TU during encoding. This is done by exploiting the TU size decisions at a previous temporal level and by modeling the relationship between the TU size and the Rate-Distortion (RD) cost values.Results:Simulation results show that the proposed method achieves a reduction of the encoding time of the latest HEVC encoder by 16.21% on average without incurring any noticeable compromise on its compression efficiency. The algorithm also reduces the number of transform operations by 44.98% on average.Conclusion:In this paper, a novel fast TU size selection scheme for HEVC is proposed. The proposed technique outperforms both the latest HEVC reference software, HM 16.0, as well as other state-of-the-art techniques in terms of time-complexity. The compression performance of the proposed technique is comparable to that of HM 16.0.

Complexity reduction for HEVC encoder using one-dimensional filtering based constrained one-bit transform

This paper presents a one-dimensional-filtering-based constrained one-bit transformation (1D-C1BT)-based motion estimation to reduce the complexity of the High Efficiency Video Coding (HEVC) encoder. The proposed 1D-C1BT requires integer arithmetic additions, subtraction and shift operations to convert full resolution video frames to two bit depth 1D-C1BT frames. By embedding the 1D-C1BT in the HEVC encoder, the motion estimation (ME) complexity of the HEVC encoder is greatly reduced in terms of Motion Estimation Time (MET) and total Encoding Time (ET) in uni-prediction and bi-prediction respectively. From the experimental results, it is shown that the 1D-C1BT accompanying full search (FS) and test zonal search (TZS) algorithms greatly reduces MET and ET as compared to the FS and TZS algorithms of the HEVC encoder respectively. To further reduce the intricacy of the HEVC encoder in the fast search mode, star-diamond (SD) search accompanying 1D-C1BT is proposed. The 1D-C1BT accompanying SD further reduces MET and ET with small increment in Bjontegaard Delta Rate and small decrement in Bjontegaard Delta Peak Signal to Noise Ratio, compared to the TZS algorithm of the HEVC encoder. In the FS mode of the HEVC encoder, the proposed 1D-C1BT, accompanying FS, reduces MET by 86.00% and ET by 83.94%, with BD-Rate of 0.1452% and BD-PSNR of 0.2635% in low delay-P main profile. The proposed ME algorithms, 1D-C1BT accompanying full search (FS + 1D-C1BT), 1D-C1BT accompanying test zonal search (TZS + 1D-C1BT) and 1D-C1BT accompanying star diamond search (SD + 1D-C1BT) greatly reduce ME complexity in all the three configurations (low delay-P, low delay-B and random access main profile) of the HEVC encoder.

Computational complexity reduction in HEVC intra prediction using dual tree complex wavelet transform and holoentropy

High efficiency video coding (HEVC) is the newest video codec to increases the coding efficiency of its ancestor H.264/Advance Video Coding at the cost of highly increased computational complexity. In this paper, a novel method using Dual tree Complex Wavelet Transform is proposed to reduce the computational time consumption in HEVC Encoding. DTCWT provides better directional selectivity. By using DTCWT, total intra prediction modes are reduced from 35 modes to 7 modes including DC and Planer mode. The encoding process in HEVC system is performed using clustered entropy computing, which distinguishes the video information has useful outliers. The pixel variations under varying frames are clustered based on the interestingness and the outliers are removed using an advanced entropy principle called as holoentropy. Compared to the current state of the art algorithms, the experimental results show that this scheme is computationally simple and achieves superior reconstructed video quality at less computational complexity.

Speed-Distortion Optimization: Tradeoffs in Open-Source HEVC Encoding

<italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">The high-efficiency video coding (HEVC) video compression standard, which drives a significant increase in coding efficiency, is an essential technology for delivering ultrahigh-definition content in the high dynamic range (HDR). This improved coding efficiency comes at a significant cost in computational complexity. Adaptive streaming and content-adaptive encoding significantly increase the compute required for encoding for the additional encodes and analysis required to support the corresponding use cases. All these factors combined require a careful analysis of tradeoffs to avoid exorbitant infrastructure costs of transition to HEVC. This paper discusses how the throughput of an HEVC encoder can be optimized to enable a cost-efficient distributed encoding system running on commodity hardware. We chose the popular open-source ×265 video encoder for this study due to its wide adoption. We first examine the ×265 presets, which combine various performance and encoding efficiency tradeoffs into a single operational point. We then detail some additional options that enable additional analysis on a selective subset of cases, such as per frame or per coding tree unit; these options are representative of several such options that are not exposed via the presets of ×265. Specifically, we discuss options pertaining to selective in-loop filtering and dynamically retargeting rate-distortion optimization (RDO) tools depending on the quality fluctuations encountered at runtime. We present tradeoff results for real-world 4K HDR content to demonstrate the applicability of these coding tools to real content used for over-the-top (OTT) and live streaming</i> .