Coding Unit Depth Decision Research Articles

Fast CU Depth Decision for HEVC Using Neural Networks

This paper presents a coding unit (CU) depth-decision algorithm using neural networks to reduce the computational overhead of High Efficiency Video Coding (HEVC). The coding tree unit (CTU) of HEVC has a quad-tree structure, and its computational complexity is considerably high because it searches an optimal CU depth from the upper to lower depth recursively and exhaustively. In the proposed method, neural networks are used to predict the CTU depth. A database for neural networks is constructed, which considers both the image and encoding properties of the CU. It consists of the image data representing the image value of the CU, the vector data based on the encoding information of the CU, and the labels indicating whether the CU is divided. By using both properties of the CU, high test accuracy can be achieved. It is completely separated from the test sequence used for encoding and can be configured to use a sequence with various resolutions, motions, and contents for diverse CUs. We also design a neural-network architecture and perform training. The architecture consists of the convolution and pooling layers for analyzing the image property of the CU. The feature map is concatenated with the vector data and trained by fully connected layers in order to analyze the encoding property of the CU. Finally, a fast CU depth-decision algorithm is designed based on the trained neural networks. When the result of the neural network inference with the current CU depth is non-split, the operation on the lower CU depth is skipped. The experimental results show that the proposed method can reduce the computational overhead by 61.77% on average, and by a maximum of 73.45% with 3.91% Bjøntegaard-Difference-bitrate (BD rate) degradation.

Early CU Depth Decision and Reference Picture Selection for Low Complexity MV-HEVC

The Multi-View extension of High Efficiency Video Coding (MV-HEVC) has improved the coding efficiency of multi-view videos, but this comes at the cost of the extra coding complexity of the MV-HEVC encoder. This coding complexity can be reduced by efficiently reducing time-consuming encoding operations. In this work, we propose two methods to reduce the encoder complexity. The first one is Early Coding unit Splitting (ECS), and the second is the Efficient Reference Picture Selection (ERPS) method. In the ECS method, the decision of Coding Unit (CU) splitting for dependent views is made on the CU splitting information obtained from the base view, while the ERPS method for dependent views is based on selecting reference pictures on the basis of the temporal location of the picture being encoded. Simulation results reveal that our proposed methods approximately reduce the encoding time by 58% when compared with HTM (16.2), the reference encoder for MV-HEVC.

Hierarchical complexity control algorithm for HEVC based on coding unit depth decision

The next-generation High Efficiency Video Coding (HEVC) standard reduces the bit rate by 44% on average compared to the previous-generation H.264 standard, resulting in higher encoding complexity. To achieve normal video coding in power-constrained devices and minimize the rate distortion degradation, this paper proposes a hierarchical complexity control algorithm for HEVC on the basis of the coding unit depth decision. First, according to the target complexity and the constantly updated reference time, the coding complexity of the group of pictures layer and the frame layer is allocated and controlled. Second, the maximal depth is adaptively assigned to the coding tree unit (CTU) on the basis of the correlation between the residual information and the optimal depth by establishing the complexity-depth model. Then, the coding unit smoothness decision and adaptive low bit threshold decision are proposed to constrain the unnecessary traversal process within the maximal depth assigned by the CTU. Finally, adaptive upper bit threshold decision is used to continue the necessary traversal process at a larger depth than the maximal depth of allocation to guarantee the quality of important coding units. Experimental results show that our algorithm can reduce the encoding time by up to 50%, with notable control precision and limited performance degradation. Compared to state-of-the-art algorithms, the proposed algorithm can achieve higher control accuracy.

Efficient coding mode and partition decision for screen content intra coding

In order to tackle the exclusive characteristic in the screen content video, new coding tools such as intra block copy mode and palette mode are adopted in the screen content coding (SCC) based on high efficiency video coding (HEVC) standard. It results in high coding complexity for SCC and is hard to satisfy real time applications. An efficient intra coding algorithm for screen content video is proposed in this paper. Early skipping or early termination of coding unit (CU) partition and mode prediction is applied to speed up intra coding by exploiting the correlated information among luminance, gradient and neighboring CUs. Furthermore, the correlated information of hit rate and coding bit is revealed and properly used for fast CU depth decision. Experimental results demonstrate that the proposed algorithm obtains significant time saving with ignorable compression loss in comparison with several recent algorithms.

An Efficient H.264/AVC to HEVC Transcoder for Real-Time Video Communication in Internet of Vehicles

Because of the co-existing of H.264/AVC and high efficiency video coding standard (HEVC) in the coming long period, video transcoding technology has become an essential part of multimedia communication in the field of the Internet of Vehicles (IoV). However, due to the huge computational complexity of re-encoding processes, traditionally cascaded transcoders greatly increase the computing burden of the embedded devices and impact the real-time capability of transportation communication systems. In order to address this problem, a fast transcoding solution is proposed in this paper. First, we exploit the mapping relationship among H.264/AVC decoding information and HEVC coding unit (CU) depth decision and prediction unit (PU) mode decision. Then, a three-output classification model is built for CU depth decision processes, and a two-output classification model is built for PU mode selection processes by using support vector machine method. Finally, the models are applied into the cascaded transcoder to accelerate the re-encoding process. The experimental results show that our proposal averagely achieves up to 53.7% and 52.3% complexity reductions under Lowdelay_P_main and Randomaccess_main configurations, respectively, with the negligible rate-distortion degradation, which show a great potential in improving the transcoding efficiency in the real-time video communication system of IoV.

Adaptive Inter CU Depth Decision for HEVC Using Optimal Selection Model and Encoding Parameters

High efficiency video coding adopts a new hierarchical coding structure, including coding unit (CU), prediction unit (PU), and transform unit to achieve higher coding efficiency than its predecessor H.264/AVC high profile. However, its hierarchical unit partitioning strategy leads to huge computational complexity. In this paper, an adaptive inter CU depth decision algorithm is proposed, which exploits both temporal correlation of CU depth and available encoding parameters. An optimal selection model of CU depth is established to estimate the range of candidate CU depth by exploiting the temporal correlation of CU depth among current CU and temporally co-located CUs. To reduce the accumulated errors in the process of CU depth prediction, the maximum depth of the co-located CUs and the coded block flag (CBF) of the current CU are used. Moreover, PU size and CBF information are also used to decide the maximum depth for the current CU. Experimental results show that the proposed CU depth decision approach reduces 56.3% and 51.5% average encoding time, and the Bjontegaard delta bit rate increases only 1.3% and 1.1% for various test sequences under the random access and the low delay B conditions, respectively.

HEVC 인트라 부호화를 위한 특징점 기반의 고속 CU Depth 결정

The High Efficiency Video Coding (MPEG-H HEVC/ITU-T H.265) is the newest video coding standard which has the quadtree-structured coding unit (CU). The quadtree-structure splits a CU adaptively, and its optimum CU depth can be determined by rate-distortion optimization. Such HEVC encoding requires very high computational complexity for CU depth decision. Motivated that the blob detection, which is a well-known algorithm in computer vision, detects keypoints in pictures and decision of CU depth needs to consider high frequency energy distribution, in this paper, we propose to utilize these keypoints for fast CU depth decision. Experimental results show that 20% encoding time can be saved with only slightly increasing BDBR by 0.45% on all intra case.

Low Complexity HEVC Encoder for Visual Sensor Networks.

Visual sensor networks (VSNs) can be widely applied in security surveillance, environmental monitoring, smart rooms, etc. However, with the increased number of camera nodes in VSNs, the volume of the visual information data increases significantly, which becomes a challenge for storage, processing and transmitting the visual data. The state-of-the-art video compression standard, high efficiency video coding (HEVC), can effectively compress the raw visual data, while the higher compression rate comes at the cost of heavy computational complexity. Hence, reducing the encoding complexity becomes vital for the HEVC encoder to be used in VSNs. In this paper, we propose a fast coding unit (CU) depth decision method to reduce the encoding complexity of the HEVC encoder for VSNs. Firstly, the content property of the CU is analyzed. Then, an early CU depth decision method and a low complexity distortion calculation method are proposed for the CUs with homogenous content. Experimental results show that the proposed method achieves 71.91% on average encoding time savings for the HEVC encoder for VSNs.

Machine learning-based coding unit depth decisions for flexible complexity allocation in high efficiency video coding.

In this paper, we propose a machine learning-based fast coding unit (CU) depth decision method for High Efficiency Video Coding (HEVC), which optimizes the complexity allocation at CU level with given rate-distortion (RD) cost constraints. First, we analyze quad-tree CU depth decision process in HEVC and model it as a three-level of hierarchical binary decision problem. Second, a flexible CU depth decision structure is presented, which allows the performances of each CU depth decision be smoothly transferred between the coding complexity and RD performance. Then, a three-output joint classifier consists of multiple binary classifiers with different parameters is designed to control the risk of false prediction. Finally, a sophisticated RD-complexity model is derived to determine the optimal parameters for the joint classifier, which is capable of minimizing the complexity in each CU depth at given RD degradation constraints. Comparative experiments over various sequences show that the proposed CU depth decision algorithm can reduce the computational complexity from 28.82% to 70.93%, and 51.45% on average when compared with the original HEVC test model. The Bjøntegaard delta peak signal-to-noise ratio and Bjøntegaard delta bit rate are -0.061 dB and 1.98% on average, which is negligible. The overall performance of the proposed algorithm outperforms those of the state-of-the-art schemes.

A fast CU depth decision mechanism for HEVC

In order to reduce the computational complexity of High Efficiency Video Coding (HEVC), a fast Coding Unit (CU) depth decision mechanism is proposed by utilizing the spacial correlations in the sequence frame. The proposed mechanism includes an adaptive CU depth range determination and a CU depth comparison algorithm. CU depth range is determined according to the distribution of CU depths in the same sequence. Thus, the Rate Distortion (RD) cost calculations at the CU depths outside of the range can be skipped. Furthermore, by comparing the depths of neighbor CUs with the current CU depth, the RD cost calculations at the current CU depth may be further skipped. Experimental results show that the proposed mechanism achieves a significant reduction of the computational complexity by comparing with the original HEVC, while still maintaining high video quality.

Novel Intermode Prediction Algorithm for High Efficiency Video Coding Encoder

The joint collaborative team on video coding (JCT-VC) is developing the next-generation video coding standard which is called high efficiency video coding (HEVC). In the HEVC, there are three units in block structure: coding unit (CU), prediction unit (PU), and transform unit (TU). The CU is the basic unit of region splitting like macroblock (MB). Each CU performs recursive splitting into four blocks with equal size, starting from the tree block. In this paper, we propose a fast CU depth decision algorithm for HEVC technology to reduce its computational complexity. In2N×2N PU, the proposed method compares the rate-distortion (RD) cost and determines the depth using the compared information. Moreover, in order to speed up the encoding time, the efficient merge SKIP detection method is developed additionally based on the contextual mode information of neighboring CUs. Experimental result shows that the proposed algorithm achieves the average time-saving factor of 44.84% in the random access (RA) at Main profile configuration with the HEVC test model (HM) 10.0 reference software. Compared to HM 10.0 encoder, a small BD-bitrate loss of 0.17% is also observed without significant loss of image quality.

Spatio-temporal correlation-based fast coding unit depth decision for high efficiency video coding

The exhaustive block partition search process in high efficiency video coding (HEVC) imposes a very high computational complexity on test module of HEVC encoder (HM). A fast coding unit (CU) depth algorithm using the spatio-temporal correlation of the depth information to fasten the search process is proposed. The depth of the coding tree unit (CTU) is predicted first by using the depth information of the spatio-temporal neighbor CTUs. Then, the depth information of the adjacent CU is incorporated to skip some specific depths when encoding the sub-CTU. As compared with the original HM encoder, experimental results show that the proposed algorithm can save more than 20% encoding time on average for intra-only, low-delay, low-delay P slices, and random access cases with almost the same rate-distortion performance.