Bjontegaard Delta Bit Research Articles

Fast mode decision algorithm for HEVC intra coding based on texture partition and direction

High efficiency video coding (HEVC) is the newest video coding standard, which employs some advanced coding techniques as compared to the previous standard H.264. The flexible quad-tree partitioning of coding tree unit (CTU) and various candidate modes of prediction unit (PU) significantly promote the video compression efficiency; however, these techniques lead to a great amount of computational loads. In this paper, a fast mode decision algorithm for HEVC intra coding is proposed based on texture partition and direction. It consists of two sub-algorithms: the CTU depth range prediction (CDRP) and the intra-prediction mode selection (IPMS). The CDRP reduces the recursive partition number of coding unit (CU) based on the correlation between the CTU texture partition and the optimum CU partition, and it first calculates the texture partition flags of different-size CUs from bottom to top. Then, it employs these partition flags to predict the depth range of the current CTU and decide whether to terminate the CU partition in advance. In order to reduce the number of candidate PU modes for the Hadamard optimization, the IPMS first uses the three-step selection of the candidate modes. The first step selects the candidate modes based on the correlation between the texture directions and the optimum PU modes. The second step selects the candidate modes by using the best modes among the selected modes in the first step. The third step selects the candidate modes by using the spatial correlation of the optimum modes between the current PU and its adjacent PUs. Then, in order to reduce the number of candidate modes for the rate-distortion optimization, the IPMS utilizes the numerical relationship of the sorted Hadamard costs of above selected modes, the optimum modes of adjacent PUs and the statistical characteristics of the small-size PUs. Compared to the original algorithm in HEVC test model, the proposed overall algorithm can reduce 60% encoding time on average with only a 1.45% increase in Bjontegaard delta bit rate under the all-intra configuration. Compared to the most of state-of-the-art algorithms, the proposed overall algorithm has better computational performances and similar rate-distortion performances.

Parallel Content-Aware Adaptive Quantization-Oriented Lossy Frame Memory Recompression for HEVC

Since the development of ultrahigh-definition video, the huge bandwidth and power requirements of external memory have hindered the development of video encoder applications. Power constraints have become a particularly serious problem for portable video codec systems. With high-rate configurations [quantization parameter (QP) ≤ 22 in HEVC test model (HM) reference software], the compression performance of the existing lossless compression algorithms noticeably degrades, because the reference frames are becoming rich of textures. On the other hand, the mathematical analysis of this paper revealed that more quantization noises can be endured by the texture-rich area. Therefore, we develop an adaptive quantization-oriented parallel lossy frame memory recompression algorithm. The contributions of this paper include the following. First, a content-aware adaptive quantization method is devised to achieve a stable high compression ratio that does not deteriorate for highly quality texture-rich pictures. When QP $\in $ [12, 22], a data reduction ratio improvement of up to 14% is obtained compared with the best lossless algorithm. Furthermore, it can reduce the quality loss by 0.49–3.36 dB in terms of Bjontegaard delta peak signal-to-noise rate (BD-PSNR) compared with the fixed length quantization method. Second, to solve the low throughput problem caused by the pixel-grain prediction method, a parallel directional prediction scheme is developed. It can double or quadruple the throughput with a prediction accuracy loss of only 1.7% or 3.3%, respectively. Using the above-mentioned methods, bandwidth and memory requirements are reduced up to 70.6% and 41.0%, respectively, with a corresponding savings of 59.3% in the dynamic power consumption of the off-chip dynamic random access memory, while the BD-PSNR is −0.04 dB, or, equivalently, Bjontegaard delta bit rate (BD-BR) is 1.27%. Using TSMC 65-nm CMOS technology, the proposed frame memory compressor and decompressor can achieve the throughputs of up to 2.89 and 2.26 Gpixels/s, respectively. It is applicable to a Super Hi-Vision(8K)@68-frames/s real-time encoding with a Level D reference data reuse scheme.

Hash rearrangement scheme for HEVC screen content coding

This study presents a hash rearrangement scheme to improve coding efficiency of high-efficiency video coding for screen content (HEVC-SCC) by sharing hashes of the inter-search with intra block copy (IBC). Of the various methods introduced during the HEVC-SCC development, the IBC search technique can yield tremendous coding gains, but creates a massive computational burden on the encoder side. The authors propose an effective way to generate the IBC hash table to avoid redundant operations required for hash entry computation. Moreover, the authors propose a hash rearrangement scheme to apply the second hashes used in the inter-search to IBC and the corresponding IBC search method to reduce the computational burden and to improve the coding efficiency. The experimental results show that compared with the HEVC-SCC test model (SCM)-8.0, the proposed algorithm results 80% time reduction when considering IBC hash generation itself, and can save 9-30% of hash generation time even taking into account the proposed second hash generation. It can also reduce the hash-based IBC search time by 14.61%. Furthermore, the proposed algorithm can achieve Bjontegaard delta bit rate savings of -0.66, -0.45 and -0.66% on average for all intra, low-delay, and random access coding structures, respectively.

Light field image compression based on quality aware pseudo‐temporal sequence

Currently, a raw light field image can be captured by a light field camera and decomposed into multi-views with different perceptual quality. A novel light field compression approach for a light field image is proposed based on this observation, where the raw light field image is decomposed into multi-views as pseudo-sequencing frames and reordered according to their perceptual quality. The resulting pseudo-temporal sequence is then compressed by high-efficiency video coding (HEVC) encoder. Experimental results show that the proposed method can achieve a better coding performance (up to 28% Bjontegaard delta bit rate reduction (BDBR) savings) compared with state-of-the-art reordering techniques using HEVC.

Content-Adaptive Feature-Based CU Size Prediction for Fast Low-Delay Video Encoding in HEVC

Determining the best partitioning structure of a coding tree unit is one of the most time-consuming operations in High Efficiency Video Coding (HEVC) encoding. Specifically, it is the evaluation of the quadtree hierarchy using the rate-distortion (RD) optimization that has the most significant impact on the encoding time, especially in the cases of high definition (HD) and ultra HD videos. In order to expedite the encoding for low-delay applications, this paper proposes a coding unit (CU) size selection and encoding algorithm for inter prediction in the HEVC. To this end, it describes: 1) two CU classification models based on Inter $N\times N$ mode motion features and RD cost thresholds to predict the CU split decision; 2) an online training scheme for dynamic content adaptation; 3) a motion vector reuse mechanism to expedite the motion estimation process; and 4) finally introduces a computational complexity to coding efficiency tradeoff process to enable flexible control of the algorithm. The experimental results reveal that the proposed algorithm achieves a consistent average encoding time performance ranging from 55%–58% and 57%–61% with average Bjontegaard delta bit rate increases of 1.93%–2.26% and 2.14%–2.33% compared with the HEVC 16.0 reference software for the low delay P and low delay B configurations, respectively, across a wide range of content types and bit rates.

HEVC 부호화기를 위한 히스토리 기반의 고속 CU 깊이 결정 알고리즘

본 논문은 고효율 영상 부호화 코딩 부호화기에서 부호화 유닛의 깊이를 빠르게 결정할 수 있는 알고리즘을 제안한다. 제안하는 방법은 영상의 시간적 중복성을 최대한 이용한다. 첫째, 동일 위치에 존재하는 부호화 트리 유닛의 부호화 유닛 깊이 히스토리를 추적하고 그에 대한 평균과 표준편차를 구한다. 그리고 나서 율-왜곡 비용 연산을 수행할 부호화 유닛 깊이의 범위가 적응적으로 결정된다. 둘째, 만약 히스토리의 깊이가 모두 같은 특정한 깊이라면, 특정 깊이를 포함하는 주변 깊이에 대해서만 집중적으로 율-왜곡 연산을 실시한다. 제안하는 알고리즘은 기존의 HEVC 표준과 비교하였을 때 효과적으로 부호화 시간을 감소시키면서 약간의 Bjontegaard delta bitrate (BDBR) 손실을 가진다. 결과적으로, 제안하는 알고리즘은 HEVC test model (HM) 9.0 대비 42.15% (low delay B), 40.51% (low delay P), HM 12.0 대비 41.08% (low delay B)의 부호화 시간 감소율을 달성 하였다. 반면, 그에 따른 BDBR 손실은 HM 9.0 대비 1.22% (low delay B), 1.26% (low delay P), HM 12.0 대비 1.04% (low delay B)로 발생하였다.

Merge Mode for Deformable Block Motion Information Derivation

Motion estimation (ME) plays an important role in most video coding standards since it can improve the coding performance significantly. Block matching ME (BMME) based on translation motion model is employed in both H.264/Advanced Video Coding and High Efficiency Video Coding (HEVC), which assumes that all the pixels in a block undergo the same motion. However, BMME cannot represent the complex motion accurately, such as rotation, zoom, and deformation motion. To resolve the problem, a merge mode for deformable block motion information derivation is proposed, which considers bilinear interpolation model, six-parameter affine model, and four-parameter affine model simultaneously. The sum of absolute transformed differences criterion is used to select the best model for a block. In the proposed merge mode, the motion information of the current block is derived from the motion information of its neighbors. It does not need to perform ME and transmit motion parameters. The proposed merge mode is integrated into HEVC test model (HM) 14.0. The simulation results show that compared with HM14.0, the proposed merge mode can bring 1.3%, 1.7%, and 2.7% Bjontegaard delta (BD) bitrate savings on average for test sequences of HEVC, and 11.0%, 15.2%, and 17.4% BD bitrate savings for deformation sequences in random access main, low-delay B main, and low-delay P main test conditions, respectively, while the increased encoding and decoding complexities are moderate.

Adaptive Inter CU Depth Decision for HEVC Using Optimal Selection Model and Encoding Parameters

High efficiency video coding adopts a new hierarchical coding structure, including coding unit (CU), prediction unit (PU), and transform unit to achieve higher coding efficiency than its predecessor H.264/AVC high profile. However, its hierarchical unit partitioning strategy leads to huge computational complexity. In this paper, an adaptive inter CU depth decision algorithm is proposed, which exploits both temporal correlation of CU depth and available encoding parameters. An optimal selection model of CU depth is established to estimate the range of candidate CU depth by exploiting the temporal correlation of CU depth among current CU and temporally co-located CUs. To reduce the accumulated errors in the process of CU depth prediction, the maximum depth of the co-located CUs and the coded block flag (CBF) of the current CU are used. Moreover, PU size and CBF information are also used to decide the maximum depth for the current CU. Experimental results show that the proposed CU depth decision approach reduces 56.3% and 51.5% average encoding time, and the Bjontegaard delta bit rate increases only 1.3% and 1.1% for various test sequences under the random access and the low delay B conditions, respectively.

Binary and Multi-Class Learning Based Low Complexity Optimization for HEVC Encoding

High Efficiency Video Coding (HEVC) improves the compression efficiency at the cost of high computational complexity by using the quad-tree coding unit (CU) structure and variable prediction unit (PU) modes. To minimize the HEVC encoding complexity while maintaining its compression efficiency, a binary and multi-class support vector machine (SVM)-based fast HEVC encoding algorithm is presented in this paper. First, the processes of recursive CU decision and PU selection in HEVC are modeled as hierarchical binary classification and multi-class classification structures. Second, according to the two classification structures, the CU decision and PU selection are optimized by binary and multi-class SVM, i.e., the CU and PU can be predicted directly via classifiers without intensive rate distortion (RD) cost calculation. In particular, to achieve better prediction performance, a learning method is proposed to combine the off-line machine learning (ML) mode and on-line ML mode for classifiers based on a multiple reviewers system. Additionally, the optimal parameters determination scheme is adopted for flexible complexity allocation under a given RD constraint. Experimental results show that the proposed method can achieve 68.3%, 67.3%, and 65.6% time saving on average while the values of Bjontegaard delta peak signal-to-noise ratio are −0.093dB, −0.091dB, and −0.094dB and the values of Bjontegaard delta bit rate are 4.191%, 3.842%, and 3.665% under low delay $P$ main, low delay main, and random access configurations, respectively, when compared with the HEVC test model version HM 16.5. Meanwhile, the proposed method outperforms the state-of-the-art fast coding algorithms in terms of complexity reduction and RD performance.

Unimodal Stopping Model-Based Early SKIP Mode Decision for High-Efficiency Video Coding

High-efficiency video coding (HEVC) can greatly improve coding efficiency compared with the prior video coding standard H.264/AVC by adopting advanced hierarchical coding structures such as coding unit (CU), prediction unit (PU), and transform unit. For each CU, an exhaustive mode decision strategy is adopted to achieve the best rate distortion (RD) cost, which simultaneously results in enormous computational complexity. In this paper, an early SKIP mode decision algorithm is proposed for the HEVC encoder to speed up the process of mode decision. Each CU size is categorized into either rare used or frequent used by exploiting the correlation of CU depth, which is estimated from the temporally colocated CUs. For the rare-used CU size, the SKIP mode is directly selected as the optimal mode and the remaining mode decision process is early terminated. For the frequent-used CU size, a unimodal stopping model is designed for its early SKIP mode decision by exploiting both hierarchical mode structure and RD cost property. Experimental results show that the proposed early SKIP mode decision method achieves average 58.5% and 54.8% encoding time savings, while the Bjontegaard Delta bit rate only increases average 0.8% and 0.8% for various test sequences under the random access and the low delay B conditions, respectively.

Early DIRECT Mode Decision for MVC Using MB Mode Homogeneity and RD Cost Correlation

Multi-view video coding (MVC) adopts variable size mode decision to achieve high coding efficiency. However, its high computational complexity is a bottleneck of enabling MVC into practical real-time applications. In this paper, an early termination strategy is proposed for DIRECT mode decision of MVC by exploiting mode homogeneity and rate distortion (RD) cost correlation. By comparing the RD cost between DIRECT mode and Inter$16\times 16$ mode, an adaptive threshold is defined based on the MB's mode homogeneity and RD cost so as to early terminate the remaining inter and intra modes. Experimental results show that compared with the original JMVC model, the proposed approach can reduce the total encoding time from 65.08% to 91.45% (80.43% on average). Meanwhile, the Bjontegaard delta peak signal-to-noise ratio only decreases 0.031 dB and Bjontegaard delta bit rate increases 0.97% on average, which is a negligible loss of coding efficiency and superior to the performance of state-of-the-art methods.

Joint processing and fast encoding algorithm for multi-view depth video

The multi-view video plus depth format is the main representation of a three-dimensional (3D) scene. In the 3D extension of high-efficiency video coding (3D-HEVC), the main framework for depth video is similar to that of color video. However, because of the limitation of the mainstream capture technologies, depth video is inaccurate and inconsistent. In addition, the depth video coding method in the current 3D-HEVC software implementation is highly complex. In this paper, we introduce a joint processing and fast coding algorithm for depth video. The proposed algorithm utilizes the depth and color features to extract depth discontinuous regions, depth edge regions, and motion regions as masks for efficient processing and fast coding. The processing step includes spatial and temporal enhancement. The fast coding method mainly limits the traversal of the CU partition and mode decision. Experimental results demonstrate that the proposed algorithm reduces the coding time and the depth video coding bitrate; the proposed algorithm reduces overall coding time by 44.24 % and depth video coding time by 72.00 % on average. In addition, there is a 24.07 % depth video coding bitrate reduction with an average of 1.65 % Bjontegaard delta bitrate gains.

Fast prediction unit selection method for HEVC intra prediction based on salient regions

In order to reduce the computational complexity of the high efficiency video coding (HEVC) standard, a new algorithm for HEVC intra prediction, namely, fast prediction unit (PU) size selection method for HEVC based on salient regions is proposed in this paper. We first build a saliency map for each largest coding unit (LCU) to reduce its texture complexity. Secondly, the optimal PU size is determined via a scheme that implements an information entropy comparison among sub-blocks of saliency maps. Finally, we apply the partitioning result of saliency map on the original LCUs, obtaining the optimal partitioning result. Our algorithm can determine the PU size in advance to the angular prediction in intra coding, reducing computational complexity of HEVC. The experimental results show that our algorithm achieves a 37.9% reduction in encoding time, while producing a negligible loss in Bjontegaard delta bit rate (BDBR) of 0.62%.

SSIM-Based Game Theory Approach for Rate-Distortion Optimized Intra Frame CTU-Level Bit Allocation

A structural similarity (SSIM)-based game theory (GT) approach is proposed for rate-distortion (R-D) optimized CTU-level bit allocation in high efficiency video coding (HEVC). First, a SSIM-based bargaining game is formulated and the Nash bargaining solution (NBS) is proposed, in which a SSIM-based initial minimum utility is defined. Second, we propose a two-stage remaining bit refinement-based bit allocation scheme. The optimization scheme of the SSIM-based bargaining game sufficiently considers the different R-D characteristics of coding tree units (CTUs), in which the feasible utility set is proved to be convex based on the proposed SSIM-based utility and R-SSIM model. Compared with the other state-of-the-art CTU-level bit allocation methods, the R-D performance improvements on Bjontegaard delta bit-rate (BD-BR), Bjontegaard delta peak-signal-to-noise-ratio (BD-PSNR), and BD-SSIM metrics of the proposed method can averagely achieve significant gains, respectively. The achieved R-D performance gains have been very close to the coding performance limits from the FixedQP method. Moreover, the proposed SSIM-GT method also maintains good performances on quality smoothness, bit rate accuracy, and encoding complexity.

Efficient prediction of CU depth and PU mode for fast HEVC encoding using statistical analysis

High Efficiency Video Coding (HEVC) adopts complex quadtree-structured CU (coding unit) and PU (prediction unit) modes. Thus, the search process for optimal modes causes high computational complexity in HEVC. To solve this problem, a fast coding algorithm is proposed. Thirteen neighboring CTUs (Coding Tree Unit) are divided into three classes with the k-means method, and are selectively used as the reference CTUs to calculate the predicted CU depth levels for the current CTU. Additionally, for every CU depth, we skip some rarely used PU modes to reduce the complexity of the PU selection algorithm. Experimental results show that compared with the HEVC reference software, our algorithm can achieve 56.71% time saving with Low Delay Configuration profile and 59.76% with Random Access Configuration profile, whereas values of BDBR (Bjontegaard Delta Bit Rate) are only 1.0517% and 0.9918%, respectively. Compared with five state-of-the-art algorithms, the proposed algorithm has significant encoding time reduction with good bitrate performances.

Neyman-Pearson-Based Early Mode Decision for HEVC Encoding

The high efficiency video coding (HEVC) standard has highly improved the coding efficiency by adopting hierarchical structures of coding unit (CU), prediction unit (PU), and transform unit (TU). However, enormous computational complexity is introduced due to the recursive rate-distortion optimization (RDO) process on all CUs, PUs and TUs. In this paper, we propose a fast and efficient mode decision algorithm based on the Neyman-Pearson rule, which consists of early SKIP mode decision and fast CU size decision. First, the early mode decision is modeled as a binary classification problem of SKIP/non-SKIP or split/unsplit. The Neyman-Pearson-based rule is employed to balance the rate-distortion (RD) performance loss and the complexity reduction by minimizing the missed detection with a constrained incorrect decision rate. A nonparametric density estimation scheme is also developed to calculate the likelihood function of the statistical parameters. Furthermore, an online training scheme is employed to periodically update the probability density distributions for different quantization parameters (QPs) and CU depth levels. The experimental results show that the proposed overall algorithm can save 65% and 58% computational complexity on average with a 1.29% and 1.08% Bjontegaard Delta bitrate (BDBR) increase for various test sequences under random access and low delay P conditions, respectively. The proposed overall scheme also has the advantage that it can make the trade-off between the RD performance and time saving by setting different values for the incorrect decision rate.

Allowable depth distortion based fast mode decision and reference frame selection for 3D depth coding

To improve the coding efficiency and meanwhile reduce the complexity of the depth video encoder in 3D system, we present an efficient depth coding scheme based on the allowable depth distortion (ADD) in view synthesis. Firstly, we analyze the depth distortion in view synthesis and present a new ADD-based Rate Distortion (RD) optimization cost function to improve the mode/reference decision criteria. Then, we present an early mode and reference frame selection algorithm, which skips the unnecessary and complex mode and reference frame when the depth distortion is within the ADD interval. Meanwhile, the coding efficiency is improved by RD optimized mode decision and reference frame selection. The experimental results show that the proposed overall algorithm, which integrates the fast mode decision, reference frame selection and RD optimization, can reduce the depth coding complexity from 32.47 to 72.59, and 55.95 % on average over different test sequences. Meanwhile, the proposed algorithm achieves 1.34 and 1.58 dB average Bjontegaard Delta peak signal-to-noise ratio gain in terms of the view synthesis quality at low and high bit rate, respectively. In terms of the bit rate reduction, it reduces 54.66 and 49.60 % Bjontegaard Delta bit rate on average at low and high bit rate, respectively, while maintaining the same view synthesis image quality.

Adaptive mode decision for multiview video coding based on macroblock position constraint model

Multiview video coding (MVC) exploits mode decision, motion estimation and disparity estimation to achieve high compression ratio, which results in an extensive computational complexity. This paper presents an efficient mode decision approach for MVC using a macroblock (MB) position constraint model (MPCM). The proposed approach reduces the number of candidate modes by utilizing the mode correlation and rate distortion cost (RD cost) in the previously encoded frames/views. Specifically, the mode correlations both in the temporal-spatial domain and the inter-view are modeled with MPCM. Then, MPCM is exploited to select the optimal prediction direction for the current encoding MB. Finally, the inter mode is early determined in the optimal prediction direction. Experimental results show that the proposed method can save 86.03 % of encoding time compared with the exhaustive mode decision used in the reference software of joint multiview video coding, with only 0.077 dB loss in Bjontegaard delta peak signal-to-noise ratio (BDPSNR) and 2.29 % increment of the total Bjontegaard delta bit rate (BDBR), which is superior to the performances of state-of-the-art approaches.

Multiview video compression with 1-D transforms

Many alternative transforms have been developed recently for improved compression of images, intra prediction residuals or motion-compensated prediction residuals. In this paper, we propose alternative transforms for multiview video coding. We analyze the spatial characteristics of disparity-compensated prediction residuals, and the analysis results show that many regions have 1-D signal characteristics, similar to previous findings for motion-compensated prediction residuals. Signals with such characteristics can be transformed more efficiently with transforms adapted to these characteristics and we propose to use 1-D transforms in the compression of disparity-compensated prediction residuals in multiview video coding. To show the compression gains achievable from using these transforms, we modify the reference software (JMVC) of the multiview video coding amendment to H.264/AVC so that each residual block can be transformed either with a 1-D transform or with the conventional 2-D Discrete Cosine Transform. Experimental results show that coding gains ranging from about 1–15% of Bjontegaard-Delta bitrate savings can be achieved.

Fast Multiview Video Coding Using Adaptive Prediction Structure and Hierarchical Mode Decision

The multiview video coding (MVC) adopts hierarchical B picture prediction structure and offers many prediction modes to effectively remove the spatial, temporal, and inter-view redundancies inherited in multiview video (MVV), but at the price of extremely high computational complexity. To address this problem, a fast MVC method by jointly using adaptive prediction structure (APS) and hierarchical mode decision (HMD) is proposed in this paper. The complexity reduction is achieved by: 1) designing four APSs for different MVV contents based on the fact that the contribution of the inter-view prediction varies from sequence to sequence and 2) developing an HMD scheme based on the observation that the relationship between the rate distortion (RD) cost and size of prediction mode is a unimodal function. In particular, for the current group of picture of the input MVV, the prediction structure is adaptively selected based on its characteristic, which is measured by the ratio of the average RD cost of the base view frames to the sum of the average RD cost of the base view frames and that of anchor frames in nonbase views, and then an HMD scheme is further performed to skip the checking process of those unlikely modes. The experimental results have shown that compared with the exhaustive mode decision in the MVC, the proposed algorithm achieves a reduction of the computational complexity by 83.49% on average, whereas incurring only a 0.086 dB loss in Bjontegaard delta peak signal-to-noise ratio and 2.97% increment on the total Bjontegaard delta bit rate.