Prediction Unit Research Articles

Temporal feature vector for video analysis and retrieval in high efficiency video coding compressed domain

A new feature vector based on motion structure of video sequences is proposed which can be used independently or along with motion vectors in video analysis and retrieval applications. The proposed feature vector employs size of prediction units (PUs) in P and B frames of high efficiency video coding (HEVC) coded videos. The feature vector can be extracted by partial decoding of compressed video in the same decoding stage as motion vectors. Video retrieval application is selected to evaluate the performance of the new feature vector. In experimental evaluations, the proposed temporal feature vector independently shows on average 12% improvement in performance against motion vectors and by using it along with motion vectors, the performance of motion vector improves on average by 32% in average normalised modified retrieval rank (ANMRR) measure.

Inter Mode Accelerating Decision Algorithm for HEVC

In this paper, proposes a fast inter mode decision algorithm for HEVC encoder. According to the similarity, between the current coding unit (CU) depth and the CU of the spatio-temporal and the previous depth CU corresponding to the prediction unit (PU) mode, to reduce the current PU mode traversal range. Compared with HEVC test model (HM)16.9, the result shows that the algorithm can reduce the coding time by 48.97% on average and ensure the output bit rate of the coding is less and the peak signal to noise ratio(PSNR) is basically unchanged.

Accelerating HEVC Encoding Using Early-Split

The increase in coding efficiency and complexity of high efficiency video coding (HEVC) over H.264 is due to, among other factors, the time needed to find the optimal partition structure among the more flexible encoding modes for the coding units (CUs) and prediction units (PUs). Although many classification-based algorithms have been proposed to expedite the partition decision, the features that can be acquired from current HEVC encoding order are not sufficient to minimize the loss in coding efficiency. In this letter, we proposed an early-split (ES) order for HEVC CU-level encoding, where the encoder checks the split mode before the nonsquare PU partition modes and utilizes the encoding output of the subCUs to expedite subsequent encoding. Experiments show that the proposed algorithm can save 48% of encoding time on average with only about 0.8% loss in coding performance.

Adaptive Fractional-Pixel Motion Estimation Skipped Algorithm for Efficient HEVC Motion Estimation

High-Efficiency Video Coding (HEVC) efficiently addresses the storage and transmit problems of high-definition videos, especially for 4K videos. The variable-size Prediction Units (PUs)--based Motion Estimation (ME) contributes a significant compression rate to the HEVC encoder and also generates a huge computation load. Meanwhile, high-level encoding complexity prevents widespread adoption of the HEVC encoder in multimedia systems. In this article, an adaptive fractional-pixel ME skipped scheme is proposed for low-complexity HEVC ME. First, based on the property of the variable-size PUs--based ME process and the video content partition relationship among variable-size PUs, all inter-PU modes during a coding unit encoding process are classified into root-type PU mode and children-type PU modes. Then, according to the ME result of the root-type PU mode, the fractional-pixel ME of its children-type PU modes is adaptively skipped. Simulation results show that, compared to the original ME in HEVC reference software, the proposed algorithm reduces ME encoding time by an average of 63.22% while encoding efficiency performance is maintained.

Efficient CU and PU Decision Based on Neural Network and Gray Level Co-Occurrence Matrix for Intra Prediction of Screen Content Coding

Screen content coding (SCC) significantly improves the screen content compression efficiency over the high efficient video coding but at the cost of extremely huge computational complexity. The flexible quad-tree coding unit (CU) partitioning structure and the newly-introduced SCC intra modes are largely responsible for the high computational complexity. In order to meet the challenge of huge computational complexity of SCC, we propose an efficient CU and prediction unit (PU) decision based on neural network (NN) and gray level co-occurrence matrix (GLCM) for SCC intra prediction. The proposed efficient SCC intra prediction algorithm contains three stages, including NN-based CU classification model, efficient PU mode decision based on classification (EPMD), and efficient CU size decision based on GLCM and spatiotemporal correlations (ECSD). Consequently, the computational complexity of SCC intra prediction can be drastically reduced by replacing the brute-force search with EPMD and ECSD to decide the optimal combination of CU size and PU mode. In addition, an online updating method of weighted factor is introduced to cope with different characteristics of test sequences. In order to achieve a good tradeoff between complexity reduction and rate distortion (RD) performance, extensive experiments are conducted to select the optimal threshold for ECSD. The experimental results show that in comparison with the original SCC reference software, the proposed algorithm can reduce 49.33% intra coding time with 1.36% BDBR increase and 0.13 dB BDPSNR decrease. Meanwhile, the proposed algorithm outperforms eight state-of-the-art algorithms in terms of computational complexity reduction and RD performance.

Efficient Video Coding Using Visual Sensitive Information for HEVC Coding Standard

The latest high efficiency video coding (HEVC) standard introduces a large number of inter-mode block partitioning modes. The HEVC reference test model (HM) uses partially exhaustive tree-structured mode selection, which still explores a large number of prediction unit (PU) modes for a coding unit (CU). This impacts on encoding time rise which deprives a number of electronic devices having limited processing resources to use various features of HEVC. By analyzing the homogeneity, residual, and different statistical correlation among modes, many researchers speed-up the encoding process through the number of PU mode reduction. However, these approaches could not demonstrate the similar rate-distortion (RD) performance with the HM due to their dependency on existing Lagrangian cost function (LCF) within the HEVC framework. In this paper, to avoid the complete dependency on LCF in the initial phase, we exploit visual sensitive foreground motion and spatial salient metric (FMSSM) in a block. To capture its motion and saliency features, we use the dynamic background and visual saliency modeling, respectively. According to the FMSSM values, a subset of PU modes is then explored for encoding the CU. This preprocessing phase is independent from the existing LCF. As the proposed coding technique further reduces the number of PU modes using two simple criteria (i.e., motion and saliency), it outperforms the HM in terms of encoding time reduction. As it also encodes the uncovered and static background areas using the dynamic background frame as a substituted reference frame, it does not sacrifice quality. Tested results reveal that the proposed method achieves 32% average encoding time reduction of the HM without any quality loss for a wide range of videos.

Detection of Double Compression for HEVC Videos With Fake Bitrate

Bitrate is an important criterion for digital video quality. Forgers prefer to up-convert bitrate of videos without improving the video quality at all, especially for videos over the Internet. Therefore, proposing an effective algorithm to expose fake bitrate videos becomes an important issue in digital video forensics. In this paper, a novel method based on prediction unit (PU) partition types is proposed to detect high efficient video coding (HEVC) videos with fake bitrate. PU partition type is one unique syntactic unit of HEVC and could be reflected by compression history, while it has not been addressed in the topic of fake bitrate detection. The proposed method adopts support vector machine classifiers with feature vectors formed by histograms of PU partition types in the first P-frame in each group of pictures (GOPs). The performance of the algorithm is compared with state-of-the-art algorithms on videos of various resolutions and bitrates. The experimental results show that the proposed method can identify the fake bitrate videos with high accuracy, and is robust to frame-deletion, copy-paste, and shifted GOP structure attacks.

A Fast HEVC Encoding Method Using Depth Information of Collocated CUs and RD Cost Characteristics of PU Modes

The latest video coding standard, high efficiency video coding (HEVC), aims to achieve better coding efficiency than the H.264/AVC standard. To improve the coding performance of HEVC, tools and structures are included that also increase the computational complexity of encoding, especially for the mode decisions in the coding unit (CU), which is a structural element in the HEVC. For each CU size, the HEVC encoder performs mode decision and motion estimation using all prediction unit (PU) types and incurs massive computational burdens. In this paper, we investigate the depth correlation between the current and the collocated CU to avoid irrelevant CU procedure and truncate some PU predictions as a result. We also investigate the relationship of the rate-distortion (RD) costs after the Merge/SKIP prediction. By analyzing different CU sizes and quantization parameters, we build a mathematical model to represent the relationship for the RD costs after the Merge/SKIP prediction with an adaptive termination of the CU procedure. In addition, the relationship of the RD costs after the Merge/SKIP prediction and 2N $\boldsymbol {\times }2\text{N}$ mode is also explored. Search range reduction is used to further speed up our algorithm. Experimental results show that our algorithm can reduce the coding time by up to 80%, and provide average coding time savings of 46%, 45%, and 52%, for low delay B, low delay P, and random access configurations, respectively. Moreover, the proposed algorithm is able to maintain coding performance similar to HM 11.0. The proposed scheme outperforms previous work in terms of both the coding speed and the RD performance.

A hardware-oriented concurrent TZ search algorithm for High-Efficiency Video Coding

High-Efficiency Video Coding (HEVC) is the latest video coding standard, in which the compression performance is double that of its predecessor, the H.264/AVC standard, while the video quality remains unchanged. In HEVC, the test zone (TZ) search algorithm is widely used for integer motion estimation because it effectively searches the good-quality motion vector with a relatively small amount of computation. However, the complex computation structure of the TZ search algorithm makes it difficult to implement it in the hardware. This paper proposes a new integer motion estimation algorithm which is designed for hardware execution by modifying the conventional TZ search to allow parallel motion estimations of all prediction unit (PU) partitions. The algorithm consists of the three phases of zonal, raster, and refinement searches. At the beginning of each phase, the algorithm obtains the search points required by the original TZ search for all PU partitions in a coding unit (CU). Then, all redundant search points are removed prior to the estimation of the motion costs, and the best search points are then selected for all PUs. Compared to the conventional TZ search algorithm, experimental results show that the proposed algorithm significantly decreases the Bjøntegaard Delta bitrate (BD-BR) by 0.84%, and it also reduces the computational complexity by 54.54%.

Fast mode decision and PU size decision algorithm for intra depth coding in 3D-HEVC

In 3D video extension of High-Efficiency-Video-Coding (3D-HEVC), intra depth coding involves huge computational complexity for the added depth modeling modes (DMMs) and the new complex processing flow. This paper proposes an algorithm including the fast mode and prediction unit (PU) size decisions to accelerate the intra depth coding. For the mode decision, after a modification of the processing flow, three strategies are adopted: (1) evaluation results for intra conventional modes (ICMs) are utilized to determine whether DMMs are skipped; (2) with a predicted direction from ICMs, a golden ratio method is adopted to simplify the DMMs searching; and (3) fast Planar mode decision is made by analyzing the relationship of DMMs and Planar mode. For the fast PU size decision, evaluation costs of ICMs and DMMs, with combination of some trained thresholds, are used to make decisions. Experimental results show the overall algorithm reduces 63.00% computational complexity, with 1.04% BD-rate increasing.

Binary and Multi-Class Learning Based Low Complexity Optimization for HEVC Encoding

High Efficiency Video Coding (HEVC) improves the compression efficiency at the cost of high computational complexity by using the quad-tree coding unit (CU) structure and variable prediction unit (PU) modes. To minimize the HEVC encoding complexity while maintaining its compression efficiency, a binary and multi-class support vector machine (SVM)-based fast HEVC encoding algorithm is presented in this paper. First, the processes of recursive CU decision and PU selection in HEVC are modeled as hierarchical binary classification and multi-class classification structures. Second, according to the two classification structures, the CU decision and PU selection are optimized by binary and multi-class SVM, i.e., the CU and PU can be predicted directly via classifiers without intensive rate distortion (RD) cost calculation. In particular, to achieve better prediction performance, a learning method is proposed to combine the off-line machine learning (ML) mode and on-line ML mode for classifiers based on a multiple reviewers system. Additionally, the optimal parameters determination scheme is adopted for flexible complexity allocation under a given RD constraint. Experimental results show that the proposed method can achieve 68.3%, 67.3%, and 65.6% time saving on average while the values of Bjontegaard delta peak signal-to-noise ratio are −0.093dB, −0.091dB, and −0.094dB and the values of Bjontegaard delta bit rate are 4.191%, 3.842%, and 3.665% under low delay $P$ main, low delay main, and random access configurations, respectively, when compared with the HEVC test model version HM 16.5. Meanwhile, the proposed method outperforms the state-of-the-art fast coding algorithms in terms of complexity reduction and RD performance.

Adaptive Inter CU Depth Decision for HEVC Using Optimal Selection Model and Encoding Parameters

High efficiency video coding adopts a new hierarchical coding structure, including coding unit (CU), prediction unit (PU), and transform unit to achieve higher coding efficiency than its predecessor H.264/AVC high profile. However, its hierarchical unit partitioning strategy leads to huge computational complexity. In this paper, an adaptive inter CU depth decision algorithm is proposed, which exploits both temporal correlation of CU depth and available encoding parameters. An optimal selection model of CU depth is established to estimate the range of candidate CU depth by exploiting the temporal correlation of CU depth among current CU and temporally co-located CUs. To reduce the accumulated errors in the process of CU depth prediction, the maximum depth of the co-located CUs and the coded block flag (CBF) of the current CU are used. Moreover, PU size and CBF information are also used to decide the maximum depth for the current CU. Experimental results show that the proposed CU depth decision approach reduces 56.3% and 51.5% average encoding time, and the Bjontegaard delta bit rate increases only 1.3% and 1.1% for various test sequences under the random access and the low delay B conditions, respectively.

Fast intra coding based on CU size decision and direction mode decision for HEVC

High Efficiency Video Coding (HEVC) adopts a quad-tree structured coding unit (CU) and more prediction unit (PU) modes are used in all CU depth levels. These improvements bring better coding performance, but the exhaustive search process for optimal CU and PU selection causes higher computational complexity than earlier standards. To speed up the encoder, this paper proposes a fast intra coding algorithm. Firstly, a fast CU size decision method is introduced, which selects different depth decision methods for each largest coding unit. Secondly, a fast direction mode decision method is proposed. It first compares the direction modes of the parent unit and most probable modes (MPMs) list, and then the first direction mode of rate-distortion optimization (RDO) list is used to early terminate the RDO process. Experimental results show that compared with HM 10.0, the proposed algorithm achieves 47% time reduction, whereas the value of BDBR is only 0.7%.

An Adaptive Mode Decision Algorithm Based on Video Texture Characteristics for HEVC Intra Prediction

The latest High Efficiency Video Coding (HEVC) standard could achieve the highest coding efficiency compared with the existing video coding standards. To improve the coding efficiency of the intra frame, a quad-tree-based variable block size coding structure that is flexible to adapt to various texture characteristics of images and up to 35 intra-prediction modes for each prediction unit (PU) is adopted in HEVC. However, the computational complexity is increased dramatically because all the possible combinations of the mode candidates are calculated in order to find the optimal rate distortion cost using the Lagrange multiplier. To alleviate the encoder computational load, this paper proposes an adaptive mode decision algorithm based on texture complexity and direction for HEVC intra prediction. First, an adaptive coding unit selection algorithm according to each depth levels' texture complexity is presented to filter out unnecessary coding block. Then, the original redundant mode candidates for each PU are reduced according to its texture direction. The simulation results show that the proposed algorithm could reduce around 56% encoding time on average while maintaining the encoding performance efficiently with only a 1.0% increase in BD-rate compared with the test model HM16 of HEVC.

Unimodal Stopping Model-Based Early SKIP Mode Decision for High-Efficiency Video Coding

High-efficiency video coding (HEVC) can greatly improve coding efficiency compared with the prior video coding standard H.264/AVC by adopting advanced hierarchical coding structures such as coding unit (CU), prediction unit (PU), and transform unit. For each CU, an exhaustive mode decision strategy is adopted to achieve the best rate distortion (RD) cost, which simultaneously results in enormous computational complexity. In this paper, an early SKIP mode decision algorithm is proposed for the HEVC encoder to speed up the process of mode decision. Each CU size is categorized into either rare used or frequent used by exploiting the correlation of CU depth, which is estimated from the temporally colocated CUs. For the rare-used CU size, the SKIP mode is directly selected as the optimal mode and the remaining mode decision process is early terminated. For the frequent-used CU size, a unimodal stopping model is designed for its early SKIP mode decision by exploiting both hierarchical mode structure and RD cost property. Experimental results show that the proposed early SKIP mode decision method achieves average 58.5% and 54.8% encoding time savings, while the Bjontegaard Delta bit rate only increases average 0.8% and 0.8% for various test sequences under the random access and the low delay B conditions, respectively.

Motion-Homogeneous-Based Fast Transcoding Method From H.264/AVC to HEVC

With the popularity of high-efficiency video coding (HEVC) standard, a video server usually transcodes a video stream to HEVC for its higher compression ratio. In this paper, a fast H.264/advanced video coding (AVC) to HEVC transcoding method is proposed. In the HEVC encoding procedure, a coding unit (CU), which is a motion-homogeneous block, is first checked based on the analysis of the decoded information from H.264/AVC bit stream. Then, for motion-homogeneous blocks, CU depth and the corresponding prediction unit (PU) mode's early termination strategies are proposed based on the CU size and corresponding prior statistical knowledge. For non-motion-homogeneous blocks, a corresponding PU mode's early termination strategy is also proposed. Experimental results demonstrate the effectiveness of the proposed method.

An efficient information hiding method based on motion vector space encoding for HEVC

As the newest video coding standard, high efficiency video coding (HEVC) has great potential as a new information hiding carrier. This paper proposes an efficient information hiding method based on motion vector space encoding for HEVC encoding process. In this method, the mapping relationship between motion vector set and the points in the motion vector space is defined. The motion vector components from the N/2 prediction units (PUs) with smallest size in a coding tree unit (CTU) are selected as the secret information carriers. Each N secret bits are converted to a 2N + 1-ary number. By modifying at most one element in the set of N motion vector components, the mapping value of the set in the motion vector space can be equal to the 2N + 1-ary number. In this way, information hiding is realized. Since at most one element is changed and the N/2 PUs with smallest size are selected, this method contributes to excellent transparency of steganography and anti-steganalysis performance with high embedding efficiency. To the best of our knowledge, this is the first information hiding method based on motion vector for HEVC. Experimental results verify that the proposed method is practicable and has better performance than two typical embedding rules of information hiding based on motion vector.

Motion Classification-Based Fast Motion Estimation for High-Efficiency Video Coding

High efficiency video coding (HEVC), the latest video coding standard, is becoming popular due to its excellent coding performance. However, the significant gain in performance is achieved at the cost of substantially higher encoding complexity than its precedent H.264/AVC, in which motion estimation (ME) is the most time-consuming module that effectively removes temporal redundancy. Test zone search (TZS) is adopted as the default fast ME method in the reference software of HEVC; however, its computational complexity is still too high for real-time applications. Several fast ME algorithms have been recently proposed to further reduce ME complexity; however, these approaches typically lead to non-negligible performance loss. To address this problem, this paper proposes a motion classification-based fast ME algorithm. By exploring the motion relationship of neighboring blocks and the coding cost characteristic, the prediction unit (PU) is first categorized into one of three classes, namely, motion-smooth PU, motion-medium PU and motion-complex PU. Then different search strategies are carefully designed for PUs of each class according to their respective motion and content characteristics. Furthermore, a fast search priority-based partial internal termination scheme is presented to rapidly skip impossible positions that speeds up cost computation during the ME process. Extensive experimental results demonstrate that the proposed algorithm achieves as much as 12.47% and 20.25% reductions in total encoder complexity when compared with TZS under low delay P and random access configuration, respectively, with negligible rate-distortion degradation; thus, it outperforms state-of-the-art fast ME algorithms in terms of both coding performance and complexity reduction.

Bayesian adaptive algorithm for fast coding unit decision in the High Efficiency Video Coding (HEVC) standard

The latest High Efficiency Video Coding standard (HEVC) provides a set of new coding tools to achieve a significantly higher coding efficiency than previous standards. In this standard, the pixels are first grouped into Coding Units (CU), then Prediction Units (PU), and finally Transform Units (TU). All these coding levels are organized into a quadtree-shaped arrangement that allows highly flexible data representation; however, they involve a very high computational complexity.In this paper, we propose an effective early CU depth decision algorithm to reduce the encoder complexity. Our proposal is based on a hierarchical approach, in which a hypothesis test is designed to make a decision at every CU depth, where the algorithm either produces an early termination or decides to evaluate the subsequent depth level. Moreover, the proposed method is able to adaptively estimate the parameters that define each hypothesis test, so that it adapts its behavior to the variable contents of the video sequences.The proposed method has been extensively tested, and the experimental results show that our proposal outperforms several state-of-the-art methods, achieving a significant reduction of the computational complexity (36.5% and 38.2% average reductions in coding time for two different encoder configurations) in exchange for very slight losses in coding performance (1.7% and 0.8% average bit rate increments).

Fast coding algorithm for HEVC based on video contents

High efficiency video coding (HEVC) achieves higher coding efficiency than previous standards but introduces a large computational complexity. Because HEVC adopted some new advanced tools and the most prominent one is the flexible hierarchical coding structures which include coding unit (CU), prediction unit (PU), transform unit (TU). All of those units must be test through rate-distortion optimisation. Since the CU is highly content dependent it is not efficient to test all the modes. In this study, the authors propose a fast coding algorithm for HEVC based on video contents. The authors statistically analysis the features of video contents from three aspects, the pixel gradient, the block mean value, and the block variance of CU. Then, jointly use these features with the CU depth levels and the prediction modes of spatiotemporal adjacent CUs to realise fast CU depth level decision and fast prediction mode decision. The experimental results show that the proposed algorithm can save 58.9 and 57.6% computational complexity on average with only average 1.8 and 1.9% bitrate losses under ‘random access, main’ and ‘low-delay, main’ conditions, respectively.