Complexity Of High Efficiency Video Coding Research Articles

A Method to Reduce the Intra-Frame Prediction Complexity of HEVC Based on D-CNN

Among a series of video coding standards jointly developed by ITU-T, VCEG, and MPEG, high-efficiency video coding (HEVC) is one of the most widely used video coding standards today. Therefore, it is still necessary to further reduce the coding complexity of HEVC. In the HEVC standard, a flexible partitioning procedure entitled “quad-tree partition” is proposed to significantly improve the coding efficiency, which, however, leads to high coding complexity. To reduce the coding complexity of the intra-frame prediction, this paper proposes a scheme based on a densely connected convolution neural network (D-CNN) to predict the partition of coding units (CUs). Firstly, a densely connected block was designed to improve the efficiency of the CU partition by fully extracting the pixel features of CTU. Then, efficient channel attention (ECA) and adaptive convolution kernel size were applied to a fast CU partition for the first time to capture the information of the D-CNN convolution channels. Finally, a threshold optimization strategy was formulated to select the best threshold for each depth to further balance the computation complexity of video coding and the performance of RD. The experimental results show that the proposed method reduces the encoding time of HEVC by 60.14%, with a negligible reduction in RD performance, which is better than the existing fast partitioning methods.

SVM based approach for complexity control of HEVC intra coding

The High Efficiency Video Coding (HEVC) is adopted by various video applications in recent years. Because of its high computational demand, controlling the complexity of HEVC is of paramount importance to appeal to the varying requirements in many applications, including power-constrained video coding, video streaming, and cloud gaming. Most of the existing complexity control methods are only capable of considering a subset of the decision space, which leads to low coding efficiency. While the efficiency of machine learning methods such as Support Vector Machines (SVM) can be employed for higher precision decision making, the current SVM-based techniques for HEVC provide a fixed decision boundary which results in different coding complexities for different video content. Although this might be suitable for complexity reduction, it is not acceptable for complexity control. This paper proposes an adjustable classification approach for Coding Unit (CU) partitioning, which addresses the mentioned problems of complexity control. Firstly, a novel set of features for fast CU partitioning is designed using image processing techniques. Then, a flexible classification method based on SVM is proposed to model the CU partitioning problem. This approach allows adjusting the performance-complexity trade-off, even after the training phase. Using this model, and a novel adaptive thresholding technique, an algorithm is presented to deliver video encoding within the target coding complexity, while maximizing the coding efficiency. Experimental results justify the superiority of this method over the state-of-the-art methods, with target complexities ranging from 20% to 100%.

Prediction mode grouping and coding bits grouping based on texture complexity for Fast HEVC intra-coding

HEVC (High Efficiency Video Coding) as one of the newest international video coding standard, can achieve about 50% bit rate reduction compared with H.264/AVC (Advanced Video Coding) with the same perceptual quality by the use of flexible CTU (coding tree unit) structure, but at the same time, it also dramatically adds its computational complexity for HEVC. To reduce the computational complexity, a fast intra-prediction mode and CU (Coding Unit) size decision algorithm based on prediction mode and coding bits grouping is presented for HEVC intra-encoding in this paper. The contribution of this paper lies in the fact that we successfully use the prediction mode grouping and coding bits grouping technologies to rapidly realize the optimal prediction mode and size decision for the current CU, thus saving much computation complexity for HEVC. Specifically, in our scheme, first, we use grouping technology to group 35 intra-prediction modes into 5 subsets of candidate modes list according to the texture complexity of current PU (Prediction Unit), and each subset only contains 11 intra-prediction modes, which can greatly reduce the traversing and calculating number of candidate mode in RMD (Rough Mode Decision); second, we use coding bits grouping technology to quickly judge whether the current CU needs to be further divided on the basis of the studying of texture complexity in the current CU, which can reduce many unnecessary prediction and partition operations for the current CU; at last we use the fast intra-mode prediction and CU size decision algorithm above to quickly realize the optimal encoding for the current CU in HEVC. As a result, the high computational complexity in HEVC intra-encoding can be efficiently reduced by our proposed scheme. And the simulation results of our experiments show that our proposed fast intra-coding algorithm based on prediction mode and coding bit grouping in this paper can reduce about 49.10% computational complexity on average only at a cost of 0.92% bit rate increase and 0.065 db PSNR decline compared with the standard reference HM16.1algorithm under all-intra-configuration.

Convolutional neural network based low complexity HEVC intra encoder

Video coding is one of the key technologies of visual sensors. As the state-of-art video coding standard, High Efficiency Video Coding (HEVC) achieves a significant high compression ratio for video. However, it also introduces heavy computational complexity, leading to challenges in application of visual sensors. To reduce the complexity of HEVC intra encoder, this paper proposed a one-stage decision method of CU/PU partition and prediction mode for intra coding. First, the potential factors that may related to the corresponding decisions in CU/PU are explored. Based on this, a one-stage decision network (OSDN) structure is specially designed to determine these decisions. Consequently, the complexity of HEVC intra coding can be drastically reduced by avoiding the brute-force search. Then, OSDN is embedded into the HEVC reference software HM 15.0. Thresholds are set to let the encoder switch between OSDN and the original implementation in HEVC to obtain the final decisions. The experimental results show that the proposed method can reduce 73.69% intra encoding time with 0.1673 dB BD-PSNR loss on average. In addition, the trade-off between RD performance degradation and complexity reduction can be controlled by thresholds.

HEVC Intra Mode Selection Using Benford’s Law

In this paper, Benford’s law (BF) is analyzed and its feasibility is presented to solve the complexity issue of high-efficiency video coding (HEVC). Secondly, the intra modes selected for the current block are modeled using BF’s curve to identify the most important intra modes. Finally, the intra mode computation process is terminated when the probability of these intra modes does not follow BF’s curve. This termination results in an early mode decision algorithm that reduces the complexity of HEVC. Experimental results on HEVC’s dataset show that the proposed algorithm reduces the complexity of the HEVC by 32.77% and the increase in BD-BR (Bjontegaard delta bit rate) is only 1.62%.

Fast prediction mode selection and CU partition for HEVC intra coding

Compared with the predecessor H.264/advanced video coding, high-efficiency video coding (HEVC) is a new video coding standard with nearly double coding efficiency under the same coding quality. However, the computing complexity of HEVC increases sharply. To solve this problem, a fast algorithm for intra prediction mode selection based on mode grouping is proposed by reducing the number of modes entering rough mode decision. Moreover, the dual support vector machine is proposed to efficiently select the coding unit (CU) size, which is based on texture features of CU and sub-CUs including content complexity and direction complexity. By using the new CU size selection algorithm, the encoder can confirm the split for complex CU and terminate the split for simple CU in advance, so as to reduce the computing complexity of CU size selection. The experimental results show that by employing the two fast algorithms in intra coding, it can save 42.80% encoding time, with 0.98% increment in bit rate and 0.018 dB loss of peak-signal-to-noise ratio of luminance, compared with the reference software x265-1.7.

Pure intra mode decision in HEVC using optimized firefly algorithm

High Efficiency Video Coding (HEVC) is the latest encoder that increased the intra modes from 9 to 35 to efficiently handle the contents of the video. The HEVC’s test model (HM) selects the optimal intra mode using the brute force method which increases the complexity of HEVC. This work, firstly, investigates the feasibility of firefly algorithm (FFA) due to its exploration and exploitation characteristics to expedite the intra mode decision in HEVC. Secondly, a novel objective function is formulated for FFA to efficiently compute the brightness for intra modes in FFA. Thirdly, the parameters of FFA are made dynamic to adjust according to the contents of video sequences. Simulation results demonstrate that the nature inspired algorithm, FFA, pays off by saving a minimum of 27% of the total coding time on average and doesn’t sacrifice quality by limiting Bjontegaard delta bit rate (BD-BR) increase to only 0.98% on average.

Low complexity high efficiency coding of light fields using ensemble classifiers

Light field images can be efficiently compressed using standard video codecs, such as the High Efficiency Video Coding (HEVC). However, the huge amount of data combined with the high computational complexity of HEVC, poses limitations on high-speed light field capturing and storage. This paper presents a contribution for low complexity encoding of light fields, in different formats using HEVC, based on a Random Forests ensemble classifier. Optimal features for training the classifier are found through a score fusion based approach. Using the HEVC still image profile, the proposed method gives speed-up of 56.23% for sub-aperture images. For pseudo video format, the proposed method outperforms others available in the literature, yielding an average speed-up of 62.18%, 56.54% and 44.73% for Random Access, Low-delay Main and All-Intra profiles respectively, with negligible decrease in RD performance. These are novel results in fast coding of light fields, which are useful for further research and benchmarking.

Fast CU partition-based machine learning approach for reducing HEVC complexity

With the development of video coding technology, the high efficiency video coding (HEVC) provides better coding efficiency compared to its predecessors H.264/AVC. HEVC improves rate distortion (RD) performance significantly with increased encoding complexity. Due to the adoption of a large variety of coding unit (CU) sizes, at RD optimization level, the quadtree partition of the CU consumes a large proportion of the encoding complexity. Hence, the computational complexity cost remains a critical issue that must be properly considered in the optimization task. In this paper, two machine learning-based fast CU partition method for inter-mode HEVC are proposed, to optimize the complexity allocation at CU level. First, we propose an online support vector machine (SVM)-based fast CU algorithm for reducing HEVC complexity. The later was trained in an online way. Second, a deep convolutional neural network (CNN) is designed to predict the CU partition, in which large-scale training database including substantial CU partition data is considered. Experimental results demonstrate that the proposed online SVM can achieve a time saving of 52.28% with a degradation of 1.928% in the bitrate (BR). However, the proposed deep CNN can reduce the encoding time by 53.99% with 0.195% BR degradation. Compared to the state-of-the art, the two proposed approaches outperform the related works in terms of both RD performance and complexity reduction at inter-mode.

A Fast Intra Mode Prediction Decision Algorithm Based on Neighborhood Grouping for HEVC

Abstract High Efficiency Video Coding (HEVC) employs quadtree coding tree unit (CTU) structure to improve its coding efficiency, but at the same time, it requires a very high computational complexity due to its exhaustive calculating process to find the optimal prediction mode for the current CU (Coding Unit). Aiming to solve the problem, a fast intra mode prediction decision algorithm based on neighborhood grouping is presented for HEVC in this article. The contribution of this article lies in the fact that the authors successfully use the neighborhood grouping technology to rapidly find the optimal prediction mode for the current CU, which can significantly reduce computation complexity for HEVC. Specifically, they use the correlative information of adjacent angle prediction mode in their first scheme to quickly reduce the number of Rough Mode Decision (RMD); then they use the correlation between Most Probable Mode (MPM) and the first candidate mode selected as the optimal prediction mode to quickly determine the number of candidate modes; at last, they quickly find the optimal prediction mode with the minimum Rate Distortion Cost (RDCost) by using neighborhood grouping technology or direct calculation approach based on the number of candidate modes. As a result, their proposed algorithm can efficiently solve the problem above. The simulation results show that our proposed intra mode prediction decision algorithm based on neighborhood grouping in this article can reduce about 22.3% computational complexity on average only at a cost of 0.03% bit rate increase and 0.26% PSNR decline compared with the standard reference HM16.1 algorithm.

Mass center direction-based decision method for intraprediction in HEVC standard

Increasing the prediction modes and recursive quad-tree structure, can be the reasons for high compression efficiency in high-efficiency video coding (HEVC) standard. For the same reasons, the computational complexity of HEVC is increased compared with the previous standards. One of the parts that impose a high computational load is the intraprediction unit. In this paper, an improved intraprediction mode decision method is presented to accelerate intracoding of HEVC. A mass center direction-based approach is used to find correlation direction of pixels which can effectively eliminate the prediction modes that have low chance to be selected as the best intramode from the rate-distortion optimization computations. According to the founded correlation direction for 4 × 4 blocks, the depth range of coding units can also be reduced to accelerate the intracoding further. The performance of the proposed method evaluated with different test sequences proposed by the joint collaborative team on video coding. The coding results indicate that the proposed method reduces the encoding time significantly as compared with the HM-16.19 reference software with negligible loss of peak signal-to-noise ratio quality.

Fast video encoding based on random forests

Machine learning approaches have been increasingly used to reduce the high computational complexity of high-efficiency video coding (HEVC), as this is a major limiting factor for real-time implementations, due to the decision process required to find optimal coding modes and partition sizes for the quad-tree data structures defined by the standard. This paper proposes a systematic approach to reduce the computational complexity of HEVC based on an ensemble of online and offline Random Forests classifiers. A reduced set of features for training the Random Forests classifier is proposed, based on the rankings obtained from information gain and a wrapper-based approach. The best model parameters are also obtained through a consistent and generalizable method. The proposed Random Forests classifier is used to model the coding unit and transform unit-splitting decision and the SKIP-mode prediction, as binary classification problems, taking advantage from the combination of online and offline approaches, which adapts better to the dynamic characteristics of video content. Experimental results show that, on average, the proposed approach reduces the computational complexity of HEVC by 62.64% for the random access (RA) profile and 54.57% for the low-delay (LD) main profile, with an increase in BD-Rate of 2.58% for RA and 2.97% for LD, respectively. These results outperform the previous works also using ensemble classifiers for the same purpose.

Fast mode decision method based on edge feature for HEVC inter‐prediction

High-efficiency video coding (HEVC) is an international video coding standard. It can reduce bit rate by almost 50% when preceding state-of-the-art standard H.264/advanced video coding (AVC) with the same objective quality but increase about 40% encoding complexity. The computation complexity of HEVC largely relies on the inter-prediction, which consumes about 60-70% of the total encoding time. This study proposes a fast mode decision method for HEVC inter-prediction. Since current coding unit (CU) is the basic coding zone for HEVC encoding process, the authors apply the edge detection method with Sobel operator to extract the edge information inside CU. It is based on the threshold-oriented means to compare those edge values and then predict the best partition mode for this CU. Furthermore, they define a weighting factor to adopt various resolutions of test sequence from wide quarter video graphics array (WQVGA) to 1600 p. The experimental results show that the proposed algorithm can reduce computational complexity by 50% on average compared with HEVC reference software, and appear to have almost the same coding performance.

Fast intra coding based on online learning for high efficiency video coding

The state-of-art high efficiency video coding (HEVC) standard attracts much attention as its excellent performances in video compression in recent years. However the inherent coding complexity of HEVC makes it difficult for real-time application. A fast algorithm for HEVC intra coding based on online learning is proposed in this paper. During learning coding, the partition threshold of coding unit (CU) is obtained by learning the random sample set which conforms to Poisson probability distribution. Meanwhile the threshold of rate distortion cost is also obtained by online learning. After that, the acquired thresholds are used for early skip of CU depth or early termination of CU partition in the stage of fast coding. Anchoring the HM16 test model, the proposed algorithm achieves an important improvement of average time-saving 46.02%. In addition, it is also demonstrated that the proposed algorithm gains higher time efficiency when video sequence is encoded with higher compression efficiency.

Reducing Complexity of HEVC: A Deep Learning Approach.

High Efficiency Video Coding (HEVC) significantly reduces bit-rates over the preceding H.264 standard but at the expense of extremely high encoding complexity. In HEVC, the quad-tree partition of coding unit (CU) consumes a large proportion of the HEVC encoding complexity, due to the brute-force search for rate-distortion optimization (RDO). Therefore, this paper proposes a deep learning approach to predict the CU partition for reducing the HEVC complexity at both intra-and inter-modes, which is based on convolutional neural network (CNN) and long-and short-term memory (LSTM) network. First, we establish a large-scale database including substantial CU partition data for HEVC intra-and inter-modes. This enables deep learning on the CU partition. Second, we represent the CU partition of an entire coding tree unit (CTU) in the form of a hierarchical CU partition map (HCPM). Then, we propose an early-terminated hierarchical CNN (ETH-CNN) for learning to predict the HCPM. Consequently, the encoding complexity of intra-mode HEVC can be drastically reduced by replacing the brute-force search with ETH-CNN to decide the CU partition. Third, an early-terminated hierarchical LSTM (ETH-LSTM) is proposed to learn the temporal correlation of the CU partition. Then, we combine ETH-LSTM and ETH-CNN to predict the CU partition for reducing the HEVC complexity at inter-mode. Finally, experimental results show that our approach outperforms other state-of-the-art approaches in reducing the HEVC complexity at both intra-and inter-modes.

Optimisation of HEVC motion estimation exploiting SAD and SSD GPU‐based implementation

The new High-Efficiency Video Coding (HEVC) standard doubles the video compression ratio compared to the previous H.264/AVC at the same video quality and without any degradation. However, this important performance is achieved by increasing the encoder computational complexity. That's why HEVC complexity is a crucial subject. The most time consuming and the most intensive computing part of HEVC is the motion estimation based principally on the sum of absolute differences (SAD) or the sum of square differences (SSD) algorithms. For these reasons, the authors proposed an implementation of these algorithms on a low cost NVIDIA GPU (graphics processing unit) using the Fermi architecture developed with Compute Unified Device Architecture language. The proposed algorithm is based on the parallel-difference and the parallel-reduction process. The investigational results show a significant speed-up in terms of execution time for most 64 × 64 pixel blocks. In fact, the proposed parallel algorithm permits a significant reduction in the execution time that reaches up to 56.17 and 30.4%, compared to the CPU, for SAD and SSD algorithms, respectively. This improvement proves that parallelising the algorithm with the new proposed reduction process for the Fermi-GPU generation leads to better results. These findings are based on a static study that determines the PU percentage utilisation for each dimension in the HEVC. This study shows that the larger PUs are the most utilised in temporal levels 3 and 4, which attain 84.56% for class E. This improvement is accompanied by an average peak signal-to-noise ratio loss of 0.095 dB and a decrease of 0.64% in terms of BitRate.

An Adaptive Quad-Tree Depth Range Prediction Mechanism for HEVC

A next-generation video coding standard High Efficiency Video Coding (HEVC) provides higher video quality and lower compression bit rate but leads to very high encoding complexity, especially in the quad-tree-based coding tree unit partitioning process. To reduce the computational complexity of HEVC, in this paper, we propose an adaptive quad-tree depth range prediction mechanism. First, the proposed mechanism defines the similar region flag to distinguish between the similar region and the non-similar region. Then, two algorithms, the similar region depth range prediction algorithm and the non-similar region depth range prediction algorithm, are proposed. The similar region depth range prediction algorithm estimates the features of the similar region based on the coding unit depth of this region. The optimal depth of this region can be predicted. The non-similar region depth range prediction algorithm can skip low probability tree nodes based on the depth correlation coefficient, which is calculated based on scene content change. Both the similar region depth range prediction algorithm and the non-similar region depth range prediction algorithm show more than 90% predictive accuracy. Experimental results show that under random access configuration and low delay configuration, the proposed mechanism can yield 28.17% and 32.99% computational complexity reduction with negligible rate distortion performance loss, respectively, compared with HM16.9. The results show that the proposed mechanism is expected to be applied in real-time environments.

Fast Intra-Mode and CU Size Decision for HEVC

The latest video coding standard High Efficiency Video Coding (HEVC) achieves about a 50% bit-rate reduction compared with H.264/AVC under the same perceptual video quality. For intra coding, a coding unit (CU) is recursively divided into a quadtree-based structure from the largest CU 64 × 64 to the smallest CU 8 × 8. Also, up to 35 intra-prediction modes are allowed. These two techniques improve the intra-coding performance significantly. However, the encoding complexity increases several times compared with H.264/AVC intra coding. In this paper, fast intra-mode decision and CU size decision are proposed to reduce the complexity of HEVC intra coding while maintaining the rate-distortion (RD) performance. For fast intra-mode decision, a gradient-based method is proposed to reduce the candidate modes for rough mode decision and RD optimization. For fast CU size decision, the homogenous CUs are early terminated first. Then two linear support vector machines that employ the depth difference and HAD cost ratio (and RD cost ratio) as features are proposed to perform the decisions of early CU split and early CU termination for the rest of the CUs. Experimental results show that the proposed fast intra-coding algorithm achieves about a 54% encoding time reduction on average with only a 0.7% BD-rate increase for the HEVC reference software HM 14.0 under all-intra configuration.

High Bit-Depth Medical Image Compression With HEVC.

Efficient storing and retrieval of medical images has direct impact on reducing costs and improving access in cloud-based health care services. JPEG 2000 is currently the commonly used compression format for medical images shared using the DICOM standard. However, new formats such as high efficiency video coding (HEVC) can provide better compression efficiency compared to JPEG 2000. Furthermore, JPEG 2000 is not suitable for efficiently storing image series and 3-D imagery. Using HEVC, a single format can support all forms of medical images. This paper presents the use of HEVC for diagnostically acceptable medical image compression, focusing on compression efficiency compared to JPEG 2000. Diagnostically acceptable lossy compression and complexity of high bit-depth medical image compression are studied. Based on an established medically acceptable compression range for JPEG 2000, this paper establishes acceptable HEVC compression range for medical imaging applications. Experimental results show that using HEVC can increase the compression performance, compared to JPEG 2000, by over 54%. Along with this, a new method for reducing computational complexity of HEVC encoding for medical images is proposed. Results show that HEVC intra encoding complexity can be reduced by over 55% with negligible increase in file size.

A fast inter-frame encoding scheme using the edge information and the spatiotemporal encoding parameters for HEVC

High Efficiency Video Coding (HEVC), as a novel video coding standard, has shown a better coding efficiency than all existing standards, such as H.264/AVC. It adopts a lot of new efficient coding tools, the most important one is the new hierarchical structures which include the coding unit (CU), prediction unit (PU) and transform unit (TU). However, the rate-distortion (RD) optimization process for all CUs, PUs, and TUs cause large computational costs. In this paper, a fast Inter-frame encoding scheme using the edge information and the spatiotemporal encoding parameters is proposed to reduce the encoder complexity of HEVC, which consists of a fast all 2 N × 2 N modes decision method, a fast CU depth level decision method and a fast PU mode decision method. This scheme uses edge information to express the structure complexity and uses the difference of edge information between current CU and its spatiotemporal CUs to express the edge similarity (ES) in one frame and the edge movement (EM) between two adjacent frames. And then, utilizes ES and EM as assistant parameters cooperate with CU depth levels and PU mode RD costs of spatiotemporal CUs to accomplish the early termination of CU split and the PU mode selection. The experimental results show that the proposed fast inter-frame encoding method can significantly reduce the computational costs with negligible RD loss. There are 53.7 and 54.9% encoding time savings on average, but only with average 1.5 and 1.8% Bjontegaard difference bitrate (BDBR) losses for various test sequences under random access and low delay conditions, respectively.