Coding Unit Partition Research Articles

CU Partition Mode Decision for HEVC Hardwired Intra Encoder Using Convolution Neural Network.

The intensive computation of High Efficiency Video Coding (HEVC) engenders challenges for the hardwired encoder in terms of the hardware overhead and the power dissipation. On the other hand, the constrains in hardwired encoder design seriously degrade the efficiency of software oriented fast coding unit (CU) partition mode decision algorithms. A fast algorithm is attributed as VLSI friendly, when it possesses the following properties. First, the maximum complexity of encoding a coding tree unit (CTU) could be reduced. Second, the parallelism of the hardwired encoder should not be deteriorated. Third, the process engine of the fast algorithm must be of low hardware- and power-overhead. In this paper, we devise the convolution neural network based fast algorithm to decrease no less than two CU partition modes in each CTU for full rate-distortion optimization (RDO) processing, thereby reducing the encoder's hardware complexity. As our algorithm does not depend on the correlations among CU depths or spatially nearby CUs, it is friendly to the parallel processing and does not deteriorate the rhythm of RDO pipelining. Experiments illustrated that, an averaged 61.1% intraencoding time was saved, whereas the Bjøntegaard-Delta bit-rate augment is 2.67%. Capitalizing on the optimal arithmetic representation, we developed the high-speed [714 MHz in the worst conditions (125 °C, 0.9 V)] and low-cost (42.5k gate) accelerator for our fast algorithm by using TSMC 65-nm CMOS technology. One accelerator could support HD1080p at 55 frames/s real-time encoding. The corresponding power dissipation was 16.2 mW at 714 MHz. Finally, our accelerator is provided with good scalability. Four accelerators fulfill the throughput requirements of UltraHD-4K at 55 frames/s.

Online learning early skip decision method for the HEVC Inter process using the SVM‐based Pegasos algorithm

High efficient video coding (HEVC), the latest coding standard, has an encoding complexity much higher compared with H.264/advanced video coding (AVC). The greater efficiency in HEVC is obtained at much greater computational cost compared with AVC. A fast coding unit (CU) splitting algorithm is proposed for the HEVC encoder, which early terminates the CU partitioning process based on an adaptive classification model. This model is generated by an online learning method based on a Pegasos (primal estimated sub-gradient solver for SVM) algorithm. The proposed method is implemented over the HEVC reference implementation on its version 16.7. Experimental results show that the proposed method reduces the computational complexity of HEVC encoder to 35% without any loss, resulting in a 1% of Bjøntegaard Delta-rate gain in the low delay B configuration without any offline training phase.

Hole-filling map-based coding unit size decision for dependent views in three-dimensional high-efficiency video coding

The three-dimensional high-efficiency video coding (3-D-HEVC) is an emerging compression standard for multiview video plus depth data. In addition to the quad-tree coding structure inherited from HEVC, some tools are integrated, which significantly improve the coding efficiency but also result in remarkably high computational complexity. We propose a fast coding unit (CU) size decision algorithm for both depth and texture components in dependent views, where hole-filling maps created through view synthesis are utilized. First, after coding the base view, warp it onto each dependent view via depth image based rendering, during which hole-filling maps are generated. Then for depth in dependent views, CU splitting can be early terminated considering the disoccluded information from hole-filling maps; for texture in dependent views, combining the disoccluded information and the interview correlations, the CU partitioning process can also be accelerated. Experimental results show that the proposed algorithm can achieve on average 54.3% time reduction, with a negligible Bjontegaard delta bitrate increase of 0.15% on synthesized views, and a 0.05% increase on all the coded plus synthesized views compared with the original encoding scheme in a 3-D-HEVC test model.

Fast depth map coding based on virtual view quality

Multi-view video plus depth (MVD) format is a three-dimensional (3D) video representation. The depth map in MVD provides the scene distance information and is used to synthesize the virtual view with texture image through Depth Image Based Rendering (DIBR) technique. In this paper, we first present a virtual view distortion estimation (VVDE) scheme to estimate the synthesized virtual view distortion induced by depth map compression. Then based on the synthesized virtual view distortion, we propose region based Quantization Parameter (QP) adjustment scheme to improve the perceived virtual view quality, fast VVDE scheme in depth map coding Rate Distortion Optimization (RDO) to improve the coding efficiency and fast Coding Unit (CU) partition scheme to reduce the depth map coding complexity. Experimental results demonstrate that the proposed algorithm can achieve 0.50dB gain with 47.37% coding time reduction.

Machine learning based fast H.264/AVC to HEVC transcoding exploiting block partition similarity

Video transcoding is to convert one compressed video stream to another. In this paper, a fast H.264/AVC to High Efficiency Video Coding (HEVC) transcoding method based on machine learning is proposed by considering the similarity between compressed streams, especially the block partition correlations, to reduce the computational complexity. This becomes possible by constructing three-level binary classifiers to predict quad-tree Coding Unit (CU) partition in HEVC. Then, we propose a feature selection algorithm to get representative features to improve predication accuracy of the classification. In addition, we propose an adaptive probability threshold determination scheme to achieve a good trade-off between low coding complexity and high compression efficiency during the CU depth prediction in HEVC. Extensive experimental results demonstrate the proposed transcoder achieves complexity reduction of 50.2% and 49.2% on average under lowdelay P main and random access configurations while the rate-distortion degradation is negligible. The proposed scheme is proved more effective as comparing with the state-of-the-art benchmarks.

Fast CU Partitioning Algorithm for HEVC Using an Online-Learning-Based Bayesian Decision Rule

High Efficiency Video Coding (HEVC) is the state-of-the-art video coding standard. It adopts a hierarchical quad-tree-based coding unit (CU) partitioning structure that is flexible in various texture and motion characteristics of a video signal. However, the exhaustive partitioning process for finding optimal CU partitions requires a dramatic increase in computational complexity of the HEVC encoder compared with previous video coding standards. In this paper, a fast CU partitioning algorithm is proposed for HEVC encoder, which early on terminates the CU partitioning process based on the Bayesian decision rule using joint online and offline learning. An online learning method is first presented based on the minimum error Bayesian decision rule using a training picture selection method with scene change detection. Next, a joint online and offline learning method is presented, which additionally trains the loss of decision making of the proposed method based on the minimum risk Bayesian decision rule. The proposed method is implemented on an HEVC test software 15.0. Experimental results show that the proposed method reduces the computational complexity of HEVC encoder to 53.6% on an average with a 0.71% acceptable Bjøntegaard delta bitrate loss in random access configuration. For other configurations, 48.4%, 48.5%, and 54.2% encoding time saving are obtained on an average for low delay, low delay-P, and all intra-configurations, respectively.

Adaptive Fast Quadtree Level Decision Algorithm for H.264 to HEVC Video Transcoding

High Efficiency Video Coding (HEVC) was developed by the Joint Collaborative Team on Video Coding to replace the current H.264/Advanced Video Coding (AVC) standard, which has dominated digital video services in all segments of the domestic and professional markets for over ten years. Therefore, there is a lot of legacy content encoded with H.264/AVC, and an efficient video transcoding from H.264/AVC to HEVC will be needed to enable gradual migration to HEVC. In terms of rate-distortion (RD) performance, HEVC roughly doubles the RD compression performance of H.264/AVC at the expense of a high computational cost. HEVC adopts a quadtree-based coding unit (CU) block partitioning structure that is flexible in adapting various texture characteristics of images. However, this causes a dramatic increase in computational complexity due to the necessity of finding the best CU partitions. This paper presents an adaptive fast quadtree level decision algorithm that is designed to exploit the information gathered at the H.264/AVC decoder in order to make faster decisions on CU splitting in HEVC using a Naïve-Bayes probabilistic classifier that is determined by a supervised data mining process. The experimental results show that the proposed algorithm can achieve a good tradeoff between coding efficiency and complexity compared with the anchor transcoder; moreover, it outperforms other related works available in the literature.

HEVC Encoding Optimization Using Multicore CPUs and GPUs

Although the High Efficiency Video Coding (HEVC) standard significantly improves the coding efficiency of video compression, it is unacceptable even in offline applications to spend several hours compressing 10 s of high-definition video. In this paper, we propose using a multicore central processing unit (CPU) and an off-the-shelf graphics processing unit (GPU) with 3072 streaming processors (SPs) for HEVC fast encoding, so that the speed optimization does not result in loss of coding efficiency. There are two key technical contributions in this paper. First, we propose an algorithm that is both parallel and fast for the GPU, which can utilize 3072 SPs in parallel to estimate the motion vector (MV) of every prediction unit (PU) in every combination of the coding unit (CU) and PU partitions. Furthermore, the proposed GPU algorithm can avoid coding efficiency loss caused by the lack of a MV predictor (MVP). Second, we propose a fast algorithm for the CPU, which can fully utilize the results from the GPU to significantly reduce the number of possible CU and PU partitions without any coding efficiency loss. Our experimental results show that compared with the reference software, we can encode high-resolution video that consumes 1.9% of the CPU time and 1.0% of the GPU time, with only a 1.4% rate increase.

Fast coding unit partitioning method based on edge detection for HEVC intra-coding

High efficiency video coding (HEVC) standard is the latest video coding standard generation. It employs powerful coding tools to obtain improved compression efficiency. To better exploit the redundancies, HEVC adopts a very flexible quad-tree coding structure, allowing the encoder to use a block partition that matches the image features. This exhaustive technique may achieve a higher coding efficiency; however, it induces a significant computational complexity in the encoding engine. This paper proposes a new texture parameter for classifying digital videos as a first contribution and then introduces an efficient coding unit (CU) partitioning algorithm based on the early defined texture parameter in order to speed up the encoding process. In fact, the proposed technique is based on edge detection by performing SOBEL filtering in order to decide the appropriate CU size. Compared to the original HEVC, the average execution time-saving is about 31 % while maintaining almost the same output video quality.

Coding unit partition prediction technique for fast video encoding in HEVC

In this paper, a prediction technique for the best coding unit (CU) partition tree is proposed to avoid the repeated computation on the variable-sized prediction units (PU) and transform units (TU)...

Fast three‐dimensional video coding encoding algorithms based on edge information of depth map

The high-efficiency video coding (HEVC) standard for three-dimensional (3D) video, which is called 3D-HEVC, improves the compression efficiency for high resolution 3D video sequences. In this study, three coding time reduction algorithms, namely fast coding unit partition, fast prediction unit (PU) partition and search range reduction, are proposed for inter-prediction of 3D-HEVC by referring to the depth edge information. The pixels in one object should have the same motion characteristics. By detecting depth edge, we can easily classify the regional pixels if they are in the same object. Thus, in the encoding stage, the sizes of coding and PUs can be determined by the proposed fast coding unit partition and fast PU partition methods, respectively. Furthermore, the range of motion search can be effectively reduced by the proposed search range reduction method. Experimental results show that the proposed depth edge-based motion search algorithms can reduce the coding time of inter-prediction by up to 62% on average with negligible quality degradation.

Feature-based fast coding unit partition algorithm for high efficiency video coding

High Efficiency Video Coding (HEVC), which is the newest video coding standard, has been developed for the efficient compression of ultra high definition videos. One of the important features in HEVC is the adoption of a quad-tree based video coding structure, in which each incoming frame is represented as a set of non-overlapped coding tree blocks (CTB) by variable-block sized prediction and coding process. To do this, each CTB needs to be recursively partitioned into coding unit (CU), predict unit (PU) and transform unit (TU) during the coding process, leading to a huge computational load in the coding of each video frame. This paper proposes to extract visual features in a CTB and uses them to simplify the coding procedure by reducing the depth of quad-tree partition for each CTB in HEVC intra coding mode. A measure for the edge strength in a CTB, which is defined with simple Sobel edge detection, is used to constrain the possible maximum depth of quad-tree partition of the CTB. With the constrained partition depth, the proposed method can reduce a lot of encoding time. Experimental results by HM10.1 show that the average time-savings is about 13.4% under the increase of encoded BD-Rate by only 0.02%, which is a less performance degradation in comparison to other similar methods. All Rights Reserved © 2015 Universidad Nacional Autonoma de Mexico, Centro de Ciencias Aplicadas y Desarrollo Tecnologico. This is an open access item distributed under the Creative Commons CC License BY-NC-ND 4.0.

상위깊이의 Sub-CU 부호화 정보를 이용한 HEVC의 고속 PU 결정 기법

HEVC(High Efficiency Video Coding)-'./012345678(CU: Coding Unit)9:CU; ?@8(PU: Prediction Unit) A !, B-CD$EFG)/CU 45679PU H1 # IFGJKLMBNONPQ-ERSTUV WXYZA- ZQ=. [ ;<-$STU \]^_$`  =abZ PU H1!# $%N c =. def% LCUsub-CBF(Coded Block Flag)  Agh'CU;<PU H1 7$# AgPU ijN! =. kef% LRl(Intra) ?@! AgCUsub-Intra B-CDm N Agh';<Rl?@Nno =. p q% FG-Rl?@! Agh'CUsub-CBF Irs AgA;<Rl?@Nno =. tu#v c% LHM 14.0 wm::1.2%, 0.11%, 0.9%BD-rate YZ;31.4%, 2.5%, 23.4%^x\]Mv ONPQy=. cz3Z % LRx{Rl?@;/ |-}FG#~Ag/ PQF€, ‚1.9%BD-rate YZ;34.2%^x\] Oy=.AbstractHEVC (High Efficiency Video Coding) achieves high coding efficiency by employing a quadtree-based coding unit (CU) block partitioning structure and various prediction units (PUs), and the determination of the best CU partition structure and the best PU mode based on rate-distortion (R-D) cost. However, the computation complexity of encoding also dramatically increases. In this paper, to reduce such encoding computational complexity, we propose three fast PU mode decision methods based on encoding information of upper depth as follows. In the first method, the search of PU mode of the current CU is early terminated based on the sub-CBF (Coded Block Flag) of upper depth. In the second method, the search of intra prediction modes of PU in the current CU is skipped based on the sub-Intra R-D cost of upper depth. In the last method, the search of intra prediction modes of PU in the lower depth’s CUs is skipped based on the sub-CBF of the current depth’s CU. Experimental results show that the three proposed methods reduce the computational complexity of HM 14.0 to 31.4%, 2.5%, and 23.4% with BD-rate increase of 1.2%, 0.11%, and 0.9%, respectively. The three methods can be applied in a combined way to be applied to both of inter prediction and intra prediction, which results in the complexity reduction of 34.2% with 1.9% BD-rate increase. Keywords : HEVC, fast encoding, CU, PU, fast mode decision, sub-CBF ( (Regular Paper) 20 2, 20153(JBE Vol. 20, No. 2, March 2015)http://dx.doi.org/10.5909/JBE.2015.20.2.340ISSN 2287-9137 (Online) ISSN 1226-7953 (Print)

A Novel Fast CU Encoding Scheme Based on Spatiotemporal Encoding Parameters for HEVC Inter Coding

Recently, a new video coding standard, High Efficiency Video Coding (HEVC), has shown greatly improved coding efficiency by adopting hierarchical structures of coding unit (CU), prediction unit (PU), and transform unit (TU). To best achieve the coding efficiency, the best combinations of CU, PU, and TU must be found in the sense of the minimum rate-distortion (R-D) costs. Owing to this, a large computational complexity occurs. Among these CU, PU, and TU, the determination of CU sizes most significantly affects the R-D performance of HEVC encoders, which causes large computational costs in operation with PU and TU size determinations. In spite of recent works in the complexity reduction of HEVC encoders, most of the research has focused on the complexity reduction with fast CU split in intra slice coding and with early TU split in both intra and inter slice. In this paper, we propose a fast and an efficient CU encoding scheme based on the spatiotemporal encoding parameters of HEVC encoders, which consists of an improved early CU SKIP detection method and a fast CU split decision method. For the current CU block under encoding, the proposed scheme utilizes sample-adaptive-offset parameters as the spatial encoding parameter to estimate the texture complexity that affects the CU partition. In addition, the motion vectors, TU size, and coded block flag information are used as the temporal encoding parameters to estimate the temporal complexity that also affects the CU partition. The proposed scheme effectively utilizes the spatiotemporal encoding parameters that are the byproducts during the encoding process of HEVC without additionally required computation. The proposed novel fast CU encoding scheme significantly reduces the total encoding time with negligible RD-performance loss. The experimental results show that the proposed scheme achieves the total encoding time savings of average 49.6% and 42.7% only with average 1.4% and 1.0% bit-rate losses for various test sequences under random access and low delay B conditions, respectively. The proposed scheme has an advantage on the implementation for parallel processing in pipeline structures of HEVC encoders due to its independency with neighboring CU blocks.

Texture-intensity-based fast coding unit size determination method for high-efficiency video coding intracoding

The quad-tree-based picture partition scheme in high-efficiency video coding (HEVC) provides sig- nificant coding efficiency improvements, particularly for high-resolution videos. However, its flexible block par- titions and numerous intraprediction modes result in a substantial increase in computational complexity compared with previous video coding standards because of the necessity of determining the best coding unit (CU) partitions. A fast HEVC intracomputation method that uses image textural feature information instead of the traditional rate-distortion (R-D) cost value is proposed. This information is used to control the CU partition at the exterior of the xCompressCU function in the HEVC model coding framework. The method is comprised of two key steps: (1) simulation of the CU partition of the current frame image and calculation of the strength value of the textural features of each sub-block; and (2) disregard of some CU partition depths in the xCompressCU function, with no R-D cost calculation being required in the corresponding prediction mode. By leveraging these two steps, the computational complexity of the encoder is effectively lowered. Experimental results indicate that the proposed method reduces the coding time computational complexity of the current HEVC model by approx- imately 39.08% with negligible loss of coding efficiency. © 2014 SPIE and IS&T (DOI: 10.1117/1.JEI.23.5.053005)

A fast inter-prediction algorithm for HEVC based on temporal and spatial correlation

In HEVC, the structure of coding unit (CU) and prediction unit (PU) is defined, which brings about higher coding efficiency than H.264/AVC. However, the rate distortion (RD) cost calculations of all depths of CUs and partition modes have yielded tremendous coding computational complexity. In order to reduce the complexity, a fast inter-prediction algorithm is proposed based on temporal and spatial correlations in this paper. In the proposed algorithm, the optimal partition mode in HEVC is selected based on its occurrence probability among all the partition modes and the similarity of the CU segmentation and partition mode between two adjacent frames is counted. Based on the optimal mode and similarity of CU segmentation and partition mode, at most two partition modes are evaluated for each CU to save computational complexity. In addition, the spatial correlation of CU segmentation and partition mode between the corresponding located (co-located) CU and its four surrounding CUs are analyzed. Based on this spatial correlation, only the optimal partition mode is evaluated to save the computational complexity for the deeper depths of CUs of the current CU. Simulation results show that the proposed algorithm achieves 65 % coding time reduction with negligible loss in coding efficiency and peak signal-to-noise ratio (PSNR), compared to previous fast mode decision algorithm.

A Highly Parallel Framework for HEVC Coding Unit Partitioning Tree Decision on Many-core Processors

High Efficiency Video Coding (HEVC) uses a very flexible tree structure to organize coding units, which leads to a superior coding efficiency compared with previous video coding standards. However, such a flexible coding unit tree structure also places a great challenge for encoders. In order to fully exploit the coding efficiency brought by this structure, huge amount of computational complexity is needed for an encoder to decide the optimal coding unit tree for each image block. One way to achieve this is to use parallel computing enabled by many-core processors. In this paper, we analyze the challenge to use many-core processors to make coding unit tree decision. Through in-depth understanding of the dependency among different coding units, we propose a parallel framework to decide coding unit trees. Experimental results show that, on the Tile64 platform, our proposed method achieves averagely more than 11 and 16 times speedup for 1920x1080 and 2560x1600 video sequences, respectively, without any coding efficiency degradation.

Just Noticeable Difference Based Fast Coding Unit Partition in HEVC Intra Coding

As an ongoing video compression standard, High Efficiency Video Coding (HEVC) has achieved better rate distortion performance than H.264, but it also leads to enormous encoding complexity. In this paper, we propose a novel fast coding unit partition algorithm in the intra prediction of HEVC. Firstly, instead of the time-consuming rate distortion optimization for coding mode decision, just-noticeable-difference (JND) values can be exploited to partition the coding unit according to human visual system characteristics. Furthermore, coding bits in HEVC can also be considered as assisted information to refine the partition results. Compared with HEVC test model HM10.1, the experimental results show that the fast intra mode decision algorithm provides over 28% encoding time saving on average with comparable rate distortion performance.

Fast CU Splitting and Pruning for Suboptimal CU Partitioning in HEVC Intra Coding

High Efficiency Video Coding (HEVC), a new video coding standard currently being established, adopts a quadtree-based Coding Unit (CU) block partitioning structure that is flexible in adapting various texture characteristics of images. However, this causes a dramatic increase in computational complexity compared to previous video coding standards due to the necessity of finding the best CU partitions. In this paper, a fast CU splitting and pruning method is presented for HEVC intra coding, which allows for significant reduction in computational complexity with small degradations in rate-distortion (RD) performance. The proposed fast splitting and pruning method is performed in two complementary steps: 1) early CU split decision and 2) early CU pruning decision. For CU blocks, the early CU splitting and pruning tests are performed at each CU depth level according to a Bayes decision rule method based on low-complexity RD costs and full RD costs, respectively. The statistical parameters for the early CU split and pruning tests are periodically updated on the fly for each CU depth level to cope with varying signal characteristics. Experimental results show that our proposed fast CU splitting and pruning method reduces the computational complexity of the current HM to about 50% in encoding time with only 0.6% increases in BD rate.

Fast inter-mode decision algorithm for high-efficiency video coding based on similarity of coding unit segmentation and partition mode between two temporally adjacent frames

High-efficiency video coding (HEVC) introduces a flexible hierarchy of three block structures: coding unit (CU), prediction unit (PU), and transform unit (TU), which have brought about higher coding efficiency than the current national video coding standard H.264/advanced video coding (AVC). HEVC, however, simultaneously requires higher computational complexity than H.264/AVC, although several fast inter-mode decisions were proposed in its development. To further reduce this complexity, a fast inter-mode decision algorithm is proposed based on temporal correlation. Because of the distinct difference of inter-prediction block between HEVC and H.264/AVC, in order to use the temporal correlation to speed up the inter prediction, the correlation of inter-prediction between two adjacent frames needs to be analyzed according to the structure of CU and PU in HEVC. The probabilities of all the partition modes in all sizes of CU and the similarity of CU segmentation and partition modes between two adjacent frames are tested. The correlation of partition modes between two CUs with different sizes in two adjacent frames is tested and analyzed. Based on the characteristics tested and analyzed, at most, two prior partition modes are evaluated for each level of CU, which reduces the number of rate distortion cost calculations. The simulation results show that the proposed algorithm further reduces coding time by 33.0% to 43.3%, with negligible loss in bitrate and peak signal-to-noise ratio, on the basis of the fast inter-mode decision algorithms in current HEVC reference software HM7.0.