Early Mode Decision Research Articles

HEVC Intra Mode Selection Using Benford’s Law

In this paper, Benford’s law (BF) is analyzed and its feasibility is presented to solve the complexity issue of high-efficiency video coding (HEVC). Secondly, the intra modes selected for the current block are modeled using BF’s curve to identify the most important intra modes. Finally, the intra mode computation process is terminated when the probability of these intra modes does not follow BF’s curve. This termination results in an early mode decision algorithm that reduces the complexity of HEVC. Experimental results on HEVC’s dataset show that the proposed algorithm reduces the complexity of the HEVC by 32.77% and the increase in BD-BR (Bjontegaard delta bit rate) is only 1.62%.

Guided filter based intra mode accelerator for HEVC

In order to support fast communication of multimedia contents, the encoding complexity of the state of the art encoder i.e. High Efficiency Video Coding (HEVC) is reduced by proposing a fast intra mode selection process in this paper. The complexity is reduced by proposing a unique and novel model that incorporates the concept of Guided Filter (GF). In this article, the output of GF is used as weights for the intra modes; The GF is used in the same way as any other filter that takes into account some pixels and return a single value (e.g. average). Secondly, the ‘guidance’ parameter of GF is obtained using spatial/ temporal information about the block in order to efficiently handle the current problem. Thirdly, statistical evidences are presented to prove the feasibility of GF for early mode decision. Simulation results demonstrate that the proposed GF based model speeded up the HEVC by 30.50%, while Bjontegaard Delta Bit Rate (BD-BR) is only 1.21%. Moreover, the encoding time is improved by 2.28% while saving 1.68% BD-BR when compared with the cutting edge work in the intra field.

Multi-strategy depth intra mode decision algorithm in 3D-HEVC

The new generation of 3D video coding standard, denoted as 3D High Efficiency Video Coding (3D-HEVC), uses Multi-view Video Plus Depth (MVD) format to represent stereo videos. Due to the reason that the fringe information of depth maps has great influence on the video quality of synthesized views, Depth Modeling Modes (DMMs) are introduced to maintain the quality of depth maps, while causing computational complexity increase tremendously. Therefore, a multi-strategy depth intra mode decision algorithm is proposed in this paper. It incorporates early rough mode decision (RMD) termination strategy, candidate mode reduction strategy and fast DMM decision strategy. The experimental results indicate that the proposed algorithm can obtain an average encoding time reduction of 34.38% for depth map coding, with just 0.43% BDBR increase.

Fast HEVC to SCC Transcoder by Early CU Partitioning Termination and Decision Tree-Based Flexible Mode Decision for Intra-Frame Coding

Screen content coding (SCC) was developed to enhance High Efficiency Video Coding (HEVC) for encoding screen content videos. However, HEVC has dominated the market for many years, and it leaves many legacy screen content videos encoded by HEVC. Therefore, it is desired that the legacy screen content videos are migrated from HEVC to SCC to improve the coding efficiency. This paper presents a fast transcoding algorithm by analyzing various features from four categories. They are the features from the HEVC decoder, static features, dynamic features, and spatial features. First, the coding unit (CU) depth level collected from the HEVC decoder is utilized to early terminate the CU partition in SCC. Second, a flexible encoding structure is proposed to make early mode decisions with the help of various features. On the one hand, high decision accuracy is achieved because mode decision is considered from different aspects by utilizing features from more than one category. On the other hand, high computational complexity is reduced because the flexible structure considers the decision of each mode separately. The experimental results show that the proposed algorithm provides 51.24% and 54.65% re-encoding time reduction with 1.32% and 1.25% negligible Bjontegaard delta bitrate loss for YUV 4:2:0 and YUV 4:4:4 screen content sequences using all-intra configuration, respectively.

Fast 3D-HEVC inter mode decision algorithm based on the texture correlation of viewpoints

To reduce the computational complexity of 3D-HEVC, a fast inter mode decision algorithm based on texture correlation of adjacent viewpoints is proposed (FMD_inter). By studying the statistical probability of inter prediction mode distributions in 3D-HEVC coding scheme, the mode of dependent view is highly correlated with the base one, which enables the early mode decision. Moreover, by analyzing the relationship between video texture and the inter prediction mode, selectively skip checking symmetric motion partition (SMP) mode and asymmetric motion partition (AMP) mode according to the texture features of coding unit is reasonable. The proposed algorithm is able to achieve a reduction of computational complexity by 18.71% on average, while maintaining the coding efficiency. The subjective quality of the synthesized views also shows the effectiveness of the algorithm.

Fast Mode Decision Based on Grayscale Similarity and Inter-View Correlation for Depth Map Coding in 3D-HEVC

The 3D extension of High Efficiency Video Coding significantly improves the coding efficiency of 3D video at the expense of computational complexity. This paper presents a novel fast mode decision algorithm for depth map coding based on the grayscale similarity and inter-view correlation. First, depth map grayscale similarity is adopted to judge whether the reference frame could assist the coding of the current frame. When the difference in the average grayscale between the co-located coding unit (CU) and the current CU is smaller than the similarity threshold, the depth level of the current CU will be restricted by that of the coded reference CU. Second, the grayscale similarity and inter-view correlation are jointly used for dependent views to achieve early decision on the best prediction unit (PU) mode. The mode decision procedure will be determined early when the co-located CU, which has a grayscale similarity with the current CU, selects Merge or Inter 2N ×2N as the best prediction mode. Moreover, when the corresponding CU in the independent view selects Merge or Inter 2N × 2N as the best prediction mode, the current CU will skip other PU modes checking based on the strong inter-view correlation. Finally, different strategies are proposed for the P-frames and B-frames of dependent views in view of the characteristics of different prediction structures. For B frames, the PU mode information of the coded independent view is utilized as reference to skip the unnecessary mode decision processes. For P frames, the spatial-temporal correlation is considered in the process of early mode decision to determine whether to choose the Merge mode or Inter 2N × 2N as the best mode. Experimental results show that our proposed scheme achieves considerable time saving with negligible degradation of coding performance.

Fast mode decision and early termination based on perceptual visual quality for HEVC encoders

This paper presents an early termination and fast mode decision for the HEVC encoder with a perceptual visual quality. For early termination of residual quad-tree, if the predicted errors for a transform block are not visible in terms of perceptual quality, the block is coded as the all-zero block and the transforms for the block and smaller blocks are skipped in RDO process. In addition, when a coding mode in the RDO-based mode decision stage is satisfactory in terms of visual quality, the coding mode is selected and the remaining coding modes requiring more bits are skipped. The proposed fast encoding algorithm is designed by pruning many coding modes when the coding errors with a coding mode having a small number of bits are not perceived by the human visual system. In addition, the proposed perceptual quality-based fast encoding algorithms can work with the existing objective quality-based fast decision algorithms. We found that the proposed algorithm can achieve 1.77 times acceleration of HM-16.8 with minimal perceptual visual degradation and objective quality distortion.

Early DIRECT Mode Decision for MVC Using MB Mode Homogeneity and RD Cost Correlation

Multi-view video coding (MVC) adopts variable size mode decision to achieve high coding efficiency. However, its high computational complexity is a bottleneck of enabling MVC into practical real-time applications. In this paper, an early termination strategy is proposed for DIRECT mode decision of MVC by exploiting mode homogeneity and rate distortion (RD) cost correlation. By comparing the RD cost between DIRECT mode and Inter$16\times 16$ mode, an adaptive threshold is defined based on the MB's mode homogeneity and RD cost so as to early terminate the remaining inter and intra modes. Experimental results show that compared with the original JMVC model, the proposed approach can reduce the total encoding time from 65.08% to 91.45% (80.43% on average). Meanwhile, the Bjontegaard delta peak signal-to-noise ratio only decreases 0.031 dB and Bjontegaard delta bit rate increases 0.97% on average, which is a negligible loss of coding efficiency and superior to the performance of state-of-the-art methods.

Fast 3D-HEVC encoder algorithm for multiview video plus depth coding

The three-dimensional high efficiency video coding (3D-HEVC) is an extension of the HEVC standard for the compression of the multiview video plus depth (MVD) format. In 3D-HEVC, additional coding tools have been added to HEVC for improving the coding efficiency of the dependent video views and depth maps. Those tools achieve the highest possible coding efficiency, but also bring extremely high computational complexity which limits 3D-HEVC from real-time applications. In this paper, we propose a fast 3D-HEVC encoder algorithm based on the texture video and depth map correlation to reduce MVD coding computational complexity. Since the video texture and depth map represent the same scene at the same time instant (they have the same motion characteristics), it is not efficient to use all the prediction modes in 3D-HEVC mode decision procedure. The basic idea of the algorithm is to utilize the texture video and depth map property of coding information to predict the current CU prediction mode and early skip unnecessary variable-size mode decision. Experimental results demonstrate that the proposed fast 3D-HEVC encoder algorithm can save 75% computational complexity on average, while maintaining almost the same rate-distortion (RD) performance as the original 3D-HEVC encoder.

Adaptive early termination mode decision for 3D-HEVC using inter-view and spatio-temporal correlations

3D extension of high efficiency video coding (3D-HEVC) supports the coding of multi-view video and depth map. It adds new coding tools to the high efficiency video coding (HEVC) design, which improve the HEVC compression capabilities for 3D video. These coding tools achieve high coding efficiency, but it brings relatively high computational complexity which limits 3D-HEVC from practical applications. In this paper, we propose an adaptive early termination mode decision algorithm to reduce 3D-HEVC computational complexity. An experimental analysis is performed to study inter-view and spatio-temporal correlations in the coding information. Based on the analysis results, we can determine to skip some specific prediction mode at the early termination stage without a normal 3D-HEVC encoding process. The proposed algorithm uses the correlation between inter-view and spatio-temporal blocks. Experimental results demonstrate that the proposed algorithm can significantly reduce computational complexity of 3D-HEVC encoding while maintaining almost the same rate-distortion (RD) performance.

Neyman-Pearson-Based Early Mode Decision for HEVC Encoding

The high efficiency video coding (HEVC) standard has highly improved the coding efficiency by adopting hierarchical structures of coding unit (CU), prediction unit (PU), and transform unit (TU). However, enormous computational complexity is introduced due to the recursive rate-distortion optimization (RDO) process on all CUs, PUs and TUs. In this paper, we propose a fast and efficient mode decision algorithm based on the Neyman-Pearson rule, which consists of early SKIP mode decision and fast CU size decision. First, the early mode decision is modeled as a binary classification problem of SKIP/non-SKIP or split/unsplit. The Neyman-Pearson-based rule is employed to balance the rate-distortion (RD) performance loss and the complexity reduction by minimizing the missed detection with a constrained incorrect decision rate. A nonparametric density estimation scheme is also developed to calculate the likelihood function of the statistical parameters. Furthermore, an online training scheme is employed to periodically update the probability density distributions for different quantization parameters (QPs) and CU depth levels. The experimental results show that the proposed overall algorithm can save 65% and 58% computational complexity on average with a 1.29% and 1.08% Bjontegaard Delta bitrate (BDBR) increase for various test sequences under random access and low delay P conditions, respectively. The proposed overall scheme also has the advantage that it can make the trade-off between the RD performance and time saving by setting different values for the incorrect decision rate.

Early DIRECT mode decision based on all‐zero block and rate distortion cost for multiview video coding

The exhaustive variable-block-size mode decision can efficiently remove the redundancies among the multiview videos, while it also leads to significant increase of computational complexity in the multiview video coding (MVC) encoder, and the high encoding complexity becomes a bottleneck for the MVC encoder to achieve real-time multimedia applications. To address this bottleneck, many fast mode decision methods have been proposed. However, most of them are only suitable for optimising the encoding complexity of the odd views of the MVC encoder. In this study, based on the property of the all-zero block and rate distortion (RD) cost of the DIRECT mode as well as the correlations between the current macroblock (MB) and its spatial–temporal nearby MBs, an early DIRECT mode decision method is proposed for reducing the encoding complexity of the MVC. Experimental results show that the proposed method achieves 48.25 and 55.64% on average encoding time saving for the even and odd views, respectively, whereas the RD performance degradation is quite acceptable. In summary, the proposed method efficiently reduces the encoding complexity for the MVC encoder.

Early merge mode decision for texture coding in 3D-HEVC

As the upcoming 3D video coding standard, high efficiency video coding (HEVC) based 3D video coding (3D-HEVC) has been drafted. In 3D-HEVC, the computational complexity of mode decision process is significantly high due to exhaustive modes’ checks for coding units (CU) derived from recursive quad-tree partitioning. In this paper, we propose an early merge mode decision method for complexity reduction of dependent texture views. First, inter-view correlation and hierarchical depth correlation of coding modes are separately analyzed for B frame and P frame. Then, conditions to early determine merge mode coded CUs are derived based on the correlations. All of the early determined CUs only check merge modes in the mode decision process, which brings considerable complexity reduction. Experimental results demonstrate that the proposed method can achieve average 20.4% of encoding time saving for dependent texture views with negligible rate distortion performance loss.

Efficient Motion and Disparity Estimation Optimization for Low Complexity Multiview Video Coding

The use of variable block-size motion estimation (ME), disparity estimation (DE), and multiple reference frames selection aims to improve the coding efficiency of multiview video coding (MVC), however, this is at the cost of high computational complexity of these advanced coding techniques, which are not suitable for real-time video broadcasting applications. In this paper, we propose an efficient ME and DE algorithm for reducing the computational complexity of MVC. Firstly, according to the characteristics of the coded block pattern and rate distortion (RD) cost, an early DIRECT mode decision algorithm is proposed. Then, based on the characteristics of the initial search point in the ME/DE process and the observation that the best point is center-biased, an early ME/DE termination strategy is proposed. If the ME/DE early termination is not satisfied, the ME/DE search window will be reduced by applying the optimal theory. At last, two block matching search strategies are proposed to predict the best point for the ME/DE. Experimental results show that the proposed algorithm can achieve 50.05% to 77.61%, 64.83% on average encoding time saving. Meanwhile, the RD performance degradation is negligible. Especially, the proposed algorithm can be applied to not only the odd views but also the even views.

Fast Algorithm with Early Termination CU Split and Mode Decision

HEVC (High Efficiency Video Coding) is the next-generation video coding standard, which introduced many new technologies such as tree structure coding unit and a variety of intra prediction mode. Compared with H.264/AVC, HEVC has achieved higher coding efficiency but requires a very high computational complexity. So this paper proposes a fast algorithm through early termination of the CU (Coding Unit) splitting and mode decision. Based on the research of image features, we use the variance of the input image to early terminate CU splitting. In addition, based on the analysis of candidate modes which were obtained by the rough mode decision, the adjacent mode is used to early terminate the RDO process to reduce the computational complexity. The experimental results show that the proposed algorithm saves averagely about 40% of encoding time with neglecting degradation of coding performance.

Low-complexity depth map compression in HEVC-based 3D video coding

In this paper, a low-complexity algorithm is proposed to reduce the complexity of depth map compression in the high-efficiency video coding (HEVC)-based 3D video coding (3D-HEVC). Since the depth map and the corresponding texture video represent the same scene in a 3D video, there is a high correlation among the coding information from depth map and texture video. An experimental analysis is performed to study depth map and texture video correlation in the coding information such as the motion vector and prediction mode. Based on the correlation, we propose three efficient low-complexity approaches, including early termination mode decision, adaptive search range motion estimation (ME), and fast disparity estimation (DE). Experimental results show that the proposed algorithm can reduce about 66% computational complexity with negligible rate-distortion (RD) performance loss in comparison with the original 3D-HEVC encoder.

Fast encoder decision for texture coding in 3D-HEVC

As a 3D extension of the High Efficiency Video Coding (HEVC) standard, 3D-HEVC is developed to improve the coding efficiency of multi-view video. However, the improvement of the coding efficiency is obtained at the expense of a computational complexity increase. How to relieve the computational burden of the encoder is becoming a critical problem in applications. In this paper, a fast encoder decision algorithm to encode the dependent texture views is proposed, where two strategies to accelerate encoder decision by exploiting inter-view correlations are utilized. The first one is an early merge mode decision algorithm, and the second one is an early CU splitting termination algorithm. Experimental results show that the proposed algorithm can achieve 47.1% encoding time saving with overall 0.1% BD-rate reduction compared to HTM (3D-HEVC test model) version 7 under the common test condition (CTC). Both of the two strategies have been adopted into the 3D-HEVC reference software and enabled as a default encoding process under CTC.

Early MERGE Mode Decision Based on Motion Estimation and Hierarchical Depth Correlation for HEVC

The high efficiency video coding (HEVC) is the latest video coding standard, which adopts the quadtree structure based coding tree unit (CTU) to improve the coding efficiency. In the HEVC encoding process, the CTU is recursively split into the 8×8 size coding units (CUs) from the 64×64 size CU. Along with the increased number of the sizes of the CUs, the number of coding modes has been greatly increased, which results in high computational complexity in the HEVC encoder. In this paper, we propose an early MERGE mode decision algorithm to reduce the computational complexity of the HEVC encoder. Firstly, based on the all-zero block (AZB) and the motion estimation (ME) information of the INTER 2N×2N mode, an early MERGE mode decision is proposed for the root CUs (i.e., 64×64 size CUs). Then, an early MERGE mode decision is proposed for the children CUs (i.e., 32×32, 16×16, and 8×8 size CUs) by considering the mode selection correlation between the root CU and the children CUs. To maximize the computational complexity reduction, when the root CUs are encoded in the non-MERGE modes, the AZB and the ME information are also used for early termination of the children CUs. Experimental results demonstrate that compared to the state-of-the-art published method, the proposed algorithm can achieve about 35% encoding time on average saving while the rate distortion performance degradation is negligible.

Fast Intraframe Coding for High Efficiency Video Coding

The High Efficiency Video Coding (HEVC) is a new video coding standard that can provide much better compression efficiency than its predecessor H.264/AVC. However, it is computationally more intensive due to the use of flexible quadtree coding unit structure and more choices of prediction modes. In this paper, a fast intraframe coding scheme is proposed for HEVC. Firstly, a fast bottom-up pruning algorithm is designed to skip the mode decision process or reduce the candidate modes at larger block size coding unit. Then, a low complexity rough mode decision process is adopted to choose a small candidate set, followed by early DC and Planar mode decision and mode filtering to further reduce the number of candidate modes. The proposed method is evaluated by the HEVC reference software HM8.2. Averaging over 5 classes of HEVC test sequences, 41.39% encoding time saving is achieved with only 0.77% bitrate increase.

Global-local correlation-based early large-size mode decision for multiview video coding

Multiview video coding (MVC) is a recent extension of H.264/AVC, and it consumes huge encoding time to select the optimal macroblock (MB) mode, among different size candidate modes. As compared with the small-size mode (Inter16 × 8, Inter8 × 16, Inter8 × 8, Intra8 × 8, and Intra4 × 4), the large-size mode (Skip/Direct, Inter16 × 16, and Intra16 × 16) occupies most of the MB mode proportion with much less computational com- plexity. Thus, if the large-size mode could be early decided as the optimal MB mode, the complexity of mode decision could be effectively reduced. In this work, an early large-size mode decision algorithm is proposed based on the global correlation of rate-distortion (RD) costs between neighbor views and the local correlation of RD costs among candidate modes. Average RD costs of large-size and small-size MB modes in the neighbor view are employed as a global reference for the threshold of early decision. And RD costs of estimated modes are used to calculate the local adjustment for the threshold. Experimental results demonstrate that the proposed algorithm can significantly reduce the whole encoding time while maintaining an RD performance similar to that of the original MVC encoder. © The Authors. Published by SPIE under a Creative Commons Attribution 3.0 Unported License. Distribution or reproduction of this work in whole or in part requires full attribution of the original publication, including its DOI. (DOI: 10 .1117/1.JEI.23.1.013027)