Rate-distortion Cost Research Articles

Two-Stage Fast Inter CU Decision for HEVC Based on Bayesian Method and Conditional Random Fields

In the latest video coding standard high efficiency video coding (HEVC), a quadtree-based coding unit (CU) partitioning scheme is adopted to better adapt to the characteristics of the video contents. However, the flexible scheme significantly increases the coding complexity, because large amount of possible CU partitioning modes should be traversed. In this paper, we propose a two-stage fast inter CU decision method to reduce the coding complexity of the HEVC encoders. In Stage I, all the CUs are classified into three categories based on the Bayesian method after the prediction unit (PU) mode $merge\,\,2N\times 2N$ is checked. Early CU pruning and early CU skipping are then applied to two of the categories, respectively. For the remaining category which is difficult to differentiate by the rate-distortion cost of the PU mode $merge\,\,2N\times 2N$ , an early CU pruning scheme based on conditional random fields is performed in Stage II, which takes both the local characteristics of the current CU and the coding information of its neighboring CUs into consideration. Experimental results show that our method can reduce 54.93% and 45.84% of the coding complexity on average with only 1.19% and 1.03% Bjontegaard delta bitrate increment under the random access main and the low delay P configurations, respectively.

Fast inter-prediction algorithm based on motion vector information for high efficiency video coding

High Efficiency Video Coding (HEVC/H.265) is the latest international video coding standard, which achieves better compression ratio and supports higher resolution than Advanced Video Coding (H.264/AVC). However, HEVC/H.265 increases the computational burden. To reduce the coding complexity of the HEVC encoder, this paper proposes a fast inter-prediction algorithm to speed up coding time. We collect the average rate-distortion costs (RD-cost) of Skip modes and Merge modes to accelerate prediction unit (PU) mode decisions. In addition, we also acquire and analyze the motion vector range from Merge modes and Inter 2N × 2N modes to decide whether to execute Merge and advanced motion vector prediction (AMVP) of other PUs. The experimental results show that the proposed algorithm provides 48.54% time saving on average in random-access configuration and maintains good rate-distortion performance and video quality at the same time. The proposed algorithm also outperforms previous works.

Efficient Intra Mode Selection for Depth-Map Coding Utilizing Spatiotemporal, Inter-Component and Inter-View Correlations in 3D-HEVC.

3D-high efficiency video coding (HEVC) is developed for the compression of the multi-view video plus depth format, which is based on the latest generation of video coding standard, HEVC. It further adopts several new intra prediction modes, depth-modeling modes (DMMs) in intra candidate modes for a better representation of edges in depth maps, which introduces a drastic increase in the computational complexity. The procedure of depth intra mode decision together with DMMs and existing intra modes is a very time consuming part due to huge complexity of full rate distortion (RD) cost calculation. In this paper, a low complexity intra mode selection algorithm is proposed to reduce complexity of depth intra prediction in both intra-frames and inter-frames. An experimental analysis is first performed to study the inter-view correlation and the inter-component (texture video and its associated depth) correlation in intra coding information such as the intra mode and RD cost. All intra modes available in 3D-HEVC are classified into three activity classes assigned with different mode-weight factors, and the coding mode complexity of a coding unit (CU) is defined according to the intra mode information from available spatiotemporal, inter-view, and inter-component neighboring coded CUs. The coding mode complexity analysis is utilized to assign different candidate intra modes for different types of CUs. The optimal intra prediction mode and the RD cost value in current CU depth level are further used to skip unnecessary intra prediction sizes. Experimental results show that the proposed fast depth intra coding algorithm achieves 61% complexity reduction on intra prediction, while incurring a 0.2% Bjontegaard metric increase for coded and synthesized views compared to the test model of 3D-HEVC.

Reduction of intra-coding time for HEVC based on temporary direction map

The high-efficiency video coding (HEVC) standard uses 35 intra-prediction modes for 2N × 2N (N is an integer number ranging from six to two) luma blocks and five modes for chroma blocks. To find the luma block with the minimum rate–distortion, it must perform 11935 different rate–distortion cost calculations. Although this approach improves coding efficiency compared to the previous standards such as H.264/AVC, computational complexity is increased significantly. In this paper, an intra-prediction technique has been described to improve the performance of the HEVC standard by minimizing its computational complexity. The proposed algorithm consists of two stages. The first stage, called prediction unit size decision (PUSD) was introduced to decrease evaluation of prediction unit sizes by ~ 38%. The second stage called prediction mode fast decision (PMFD) was developed to minimize the number of modes in the rough mode decision (RMD) stage. The simulation results show that the time complexity is decreased by ~ 47%, while the BD rate is increased by 1.08%, and PSNR is decreased by 0.04 db. Accordingly, the proposed algorithms have a negligible effect on the video quality with great saving in the time complexity.

Convolutional Neural Network Based Synthesized View Quality Enhancement for 3D Video Coding.

The quality of synthesized view plays an important role in the three dimensional (3D) video system. In this paper, to further improve the coding efficiency, a convolutional neural network (CNN) based synthesized view quality enhancement method for 3D High Efficiency Video Coding (HEVC) is proposed. Firstly, the distortion elimination in synthesized view is formulated as an image restoration task with the aim to reconstruct the latent distortion free synthesized image. Secondly, the learned CNN models are incorporated into 3D HEVC codec to improve the view synthesis performance for both view synthesis optimization (VSO) and the final synthesized view, where the geometric and compression distortions are considered according to the specific characteristics of synthesized view. Thirdly, a new Lagrange multiplier in the rate-distortion (RD) cost function is derived to adapt the CNN based VSO process to embrace a better 3D video coding performance. Extensive experimental results show that the proposed scheme can efficiently eliminate the artifacts in the synthesized image, and reduce 25.9% and 11.7% bit rate in terms of peak-signal-to-noise ratio (PSNR) and structural similarity (SSIM) index, which significantly outperforms the state-of-theart methods.

Fast HEVC Intra Coding by Predicting the Rate-distortion Cost for a Low-complexity Encoder

High Efficiency Video Coding (HEVC) intra coding achieves significant improvements in coding efficiency compared with H.264/AVC intra coding by adopting 35 spatial intra prediction modes and a quadtree-based block partitioning structure. However, the encoding complexity is extremely high from performing rate-distortion optimization (RDO) on each mode and depth level. This paper proposes fast intra mode decision algorithms for real-time HEVC encoding. For a fast intra prediction mode decision, we estimate the lower bound on the rate-distortion (RD) cost of a prediction unit with non–most-probable modes (non-MPMs). If the RD cost of the MPM is lower than the lower bound, the MPM is selected as the best mode. For a fast intra coding unit (CU) size decision, we predict the RD cost of four split sub-CUs from the RD of the current CU using the relationship between the RDs of the current CU and its split sub-CUs. In the HEVC Test Model software, the proposed algorithm saves 48.56% of the encoding time, on average, while the RD loss is only 0.59%. In x265, the proposed algorithm improves the encoding speed to 71.95 fps, on average, from 49.46 fps. It achieves 1.8 times faster speed with a bit increase of only 0.44%, compared to x265.

High-throughput sample adaptive offset hardware architecture for high-efficiency video coding

A high-throughput hardware architecture for a sample adaptive offset (SAO) filter in the high-efficiency video coding video coding standard is presented. First, an implementation-friendly and simplified bitrate estimation method of rate-distortion cost calculation is proposed to reduce the computational complexity in the mode decision of SAO. Then, a high-throughput VLSI architecture for SAO is presented based on the proposed bitrate estimation method. Furthermore, multiparallel VLSI architecture for in-loop filters, which integrates both deblocking filter and SAO filter, is proposed. Six parallel strategies are applied in the proposed in-loop filters architecture to improve the system throughput and filtering speed. Experimental results show that the proposed in-loop filters architecture can achieve up to 48% higher throughput in comparison with prior work. The proposed architecture can reach a high-operating clock frequency of 297 MHz with TSMC 65-nm library and meet the real-time requirement of the in-loop filters for 8 K × 4 K video format at 132 fps.

Fast intra coding based on online learning for high efficiency video coding

The state-of-art high efficiency video coding (HEVC) standard attracts much attention as its excellent performances in video compression in recent years. However the inherent coding complexity of HEVC makes it difficult for real-time application. A fast algorithm for HEVC intra coding based on online learning is proposed in this paper. During learning coding, the partition threshold of coding unit (CU) is obtained by learning the random sample set which conforms to Poisson probability distribution. Meanwhile the threshold of rate distortion cost is also obtained by online learning. After that, the acquired thresholds are used for early skip of CU depth or early termination of CU partition in the stage of fast coding. Anchoring the HM16 test model, the proposed algorithm achieves an important improvement of average time-saving 46.02%. In addition, it is also demonstrated that the proposed algorithm gains higher time efficiency when video sequence is encoded with higher compression efficiency.

Fast Mode Decision for HD Scalable Video Coding via Statistical Content Analysis

Scalable video coding (SVC) can provide different bitrate, resolution, or quality formats of the same video through the one-time encoding process. However, SVC also increases encoding complexity significantly, making real-time implementations difficult especially when applied to high definition videos. In this work, we proposed to utilize a fast mode decision algorithm to speed up the SVC encoder. A simple model of the relationship of the video complexity is proposed, the rate-distortion (RD) cost and the statistical mode distribution of the enhancement layer (EL). The proposed mode decision algorithm evaluates the complexity of the video content, then sets the rate distortion RD cost thresholds for inter coding modes. The method also optimizes the process INTRA_BL for I slices. We demonstrate the method using four different resolutions (1080p, 720p, VGA and QVGA), four temporal layers, and two quality layers. We find that the proposed algorithm can reduce the computational complexity significantly while maintaining or even improving encoder efficiency.

Content-Adaptive Feature-Based CU Size Prediction for Fast Low-Delay Video Encoding in HEVC

Determining the best partitioning structure of a coding tree unit is one of the most time-consuming operations in High Efficiency Video Coding (HEVC) encoding. Specifically, it is the evaluation of the quadtree hierarchy using the rate-distortion (RD) optimization that has the most significant impact on the encoding time, especially in the cases of high definition (HD) and ultra HD videos. In order to expedite the encoding for low-delay applications, this paper proposes a coding unit (CU) size selection and encoding algorithm for inter prediction in the HEVC. To this end, it describes: 1) two CU classification models based on Inter $N\times N$ mode motion features and RD cost thresholds to predict the CU split decision; 2) an online training scheme for dynamic content adaptation; 3) a motion vector reuse mechanism to expedite the motion estimation process; and 4) finally introduces a computational complexity to coding efficiency tradeoff process to enable flexible control of the algorithm. The experimental results reveal that the proposed algorithm achieves a consistent average encoding time performance ranging from 55%–58% and 57%–61% with average Bjontegaard delta bit rate increases of 1.93%–2.26% and 2.14%–2.33% compared with the HEVC 16.0 reference software for the low delay P and low delay B configurations, respectively, across a wide range of content types and bit rates.

Block-level fast coding scheme for depth maps in three-dimensional high efficiency video coding

A block-level fast coding scheme for depth maps coding in 3D-high efficiency video coding (3D-HEVC) is presented. The proposed scheme uses the information of the correlated texture coding unit (CU) to accelerate the encoding depth map CU. The experimental analysis demonstrates that when the SKIP mode encodes the texture CU, the current depth map CU has a high probability of being encoded by SKIP or depth intraskip mode. Therefore, the evaluation of the remaining encoding modes can be skipped when any of these modes obtained a low rate-distortion cost. Experimental results obtained through 3D-HEVC test model 16.0 report that our scheme reduces the encoding time by 26.9% with encoding efficiency losses lower than 0.3%.

Probability-Based Depth Intra-Mode Skipping Strategy and Novel VSO Metric for DMM Decision in 3D-HEVC

Multiview video plus depth format has been adopted as the emerging 3D video representation recently. It includes a limited number of textures and depth maps to synthesize additional virtual views. Since the quality of depth maps influences the view synthesis process, their sharp edges should be well preserved to avoid mixing foreground with background. To address this issue, 3D-High Efficiency Video Coding (HEVC) introduces new coding tools, a partition-based intra mode [depth modeling mode (DMM)], a residual description technique [segmentwise depth coding (SDC)], and a more complex rate-distortion (RD) evaluation with view synthesis optimization (VSO), to provide more accurate predictions and achieve higher compression rate. However, these new techniques introduce a lot of possible candidates, and each of them requires complicated RD calculation in the process of intra-mode decision. They lead to unacceptable computational burden in a 3D-HEVC encoder. Therefore, in this paper, we raise two efficient techniques for depth intra-mode decision. First, by investigating the statistical characteristics of variance distributions in the two partitions of DMM, a simple but efficient criterion based on the squared Euclidean distance of variances (SEDV) is suggested to evaluate RD costs of the DMM candidates instead of the time-consuming VSO process. Second, a probability-based early depth intra-mode decision is proposed to select only the most promising mode and make the early determination of using SDC based on the low-complexity RD cost in rough mode decision. Experimental results show that the proposed algorithm with these two new techniques provides 33%–48% time reduction with little drop in the coding performance compared with the state-of-the-art algorithms.

VLSI Design of an ML-Based Power-Efficient Motion Estimation Controller for Intelligent Mobile Systems

In this paper, a machine learning (ML)-based power-efficient motion estimation (ME) controller algorithm and VLSI architecture incorporating coding bandwidth and rate-distortion (R-D) cost using convex optimization are proposed to effectuate a smart and bandwidth-efficient ME design for intelligent mobile systems. To be smart and adapt to time-altering coding bandwidth using intelligent power-management techniques in modern application processor systems, we first propose an ML-based bandwidth-on-demand ME controller algorithm based on the convex optimization method to resolve the lack of an awareness of coding bandwidth in prior ME designs. Then, a hardware-friendly and power-efficient VLSI architecture is developed to implement an intelligent, high-performance, and low-power ME controller design that can be combined with prior ME designs to satisfy the bandwidth-efficient ME design target under bandwidth constraints. The final implementation results show that the proposed smart ME controller architecture using our proposed bandwidth control scheme costs 0.816K gate counts, consumes 0.873 mW of power at a working frequency of 1.1 GHz with Taiwan Semiconductor Manufacture Company (TSMC) 90-nm CMOS technology, and achieves an average bandwidth reduction of 56.08% compared with previous non-bandwidth-on-demand ME designs for high-definition (HD) videos.

Fast Intra CTU Depth Decision for HEVC

Coding tree unit (CTU) partition technique is one of the most advanced techniques, which devotes to the excellent performance of High Efficiency Video Coding (HEVC). However, the enhancement of coding performance is at the expense of increased coding complexity. To reduce the complexity of HEVC intracoding, a fast CTU depth decision algorithm based on texture features and convolutional neural network (CNN) classification technique is proposed herein. First, the relationship between texture complexity and coding unit depth is explored. Based on this, CTUs are divided into simple CTUs and complex CTUs in line with their texture complexity, which are limited to different depth ranges. Then, the CNN for HEVC intradepth range (HIDR-CNN) decision-making is proposed, which is used for CTU classification and depth range restriction. Finally, the optimal CTU partition is achieved by recursive rate-distortion cost calculation in the depth range. Experimental results show that the proposed algorithm can yield average 27.54% encoding time reduction with 0.99% BDBR gain or 0.05 dB BDPSNR loss compared with HM 16.9. The proposed algorithm contributes to promote HEVC coding efficiency under real-time environments.

Fast CU size decision algorithm using machine learning for HEVC intra coding

High Efficiency Video Coding (HEVC) is a state-of-the-art video compression standard which improves coding efficiency significantly compared with the previous coding standard, H.264/AVC. In the HEVC standard, novel technologies consuming massive computational power are adopted, such as quad-tree-based coding unit (CU) partitioning. Although an HEVC encoder can efficiently compress various video sequences, the computational complexity of an exhaustive search has become a critical problem in HEVC encoder implementation. In this paper, we propose a fast algorithm for the CU partitioning process of the HEVC encoder using machine learning methods. A complexity measure based on the Sobel operator and rate-distortion costs are defined as features for our algorithm. A CU size can be determined early by employing Fisher’s linear discriminant analysis and the k-nearest neighbors classifier. The statistical data used for the proposed algorithm is updated by adaptive online learning phase. The experimental results show that the proposed algorithm can reduce encoding time by approximately 54.0% with a 0.68% Bjøntegaard-Delta bit-rate increase.

A Fast HEVC Encoding Method Using Depth Information of Collocated CUs and RD Cost Characteristics of PU Modes

The latest video coding standard, high efficiency video coding (HEVC), aims to achieve better coding efficiency than the H.264/AVC standard. To improve the coding performance of HEVC, tools and structures are included that also increase the computational complexity of encoding, especially for the mode decisions in the coding unit (CU), which is a structural element in the HEVC. For each CU size, the HEVC encoder performs mode decision and motion estimation using all prediction unit (PU) types and incurs massive computational burdens. In this paper, we investigate the depth correlation between the current and the collocated CU to avoid irrelevant CU procedure and truncate some PU predictions as a result. We also investigate the relationship of the rate-distortion (RD) costs after the Merge/SKIP prediction. By analyzing different CU sizes and quantization parameters, we build a mathematical model to represent the relationship for the RD costs after the Merge/SKIP prediction with an adaptive termination of the CU procedure. In addition, the relationship of the RD costs after the Merge/SKIP prediction and 2N $\boldsymbol {\times }2\text{N}$ mode is also explored. Search range reduction is used to further speed up our algorithm. Experimental results show that our algorithm can reduce the coding time by up to 80%, and provide average coding time savings of 46%, 45%, and 52%, for low delay B, low delay P, and random access configurations, respectively. Moreover, the proposed algorithm is able to maintain coding performance similar to HM 11.0. The proposed scheme outperforms previous work in terms of both the coding speed and the RD performance.

Probabilistic Decision Based Block Partitioning for Future Video Coding.

In the latest Joint Video Exploration Team development, the quadtree plus binary tree (QTBT) block partitioning structure has been proposed for future video coding. Compared to the traditional quadtree structure of High Efficiency Video coding (HEVC) standard, QTBT provides more flexible patterns for splitting the blocks, which results in dramatically increased combinations of block partitions and high computational complexity. In view of this, a confidence interval based early termination (CIET) scheme is proposed for QTBT to identify the unnecessary partition modes in the sense of rate-distortion (RD) optimization. In particular, a RD model is established to predict the RD cost of each partition pattern without the full encoding process. Subsequently, the mode decision problem is casted into a probabilistic framework to select the final partition based on the confidence interval decision strategy. Experimental results show that the proposed CIET algorithm can speed up QTBT block partitioning structure by reducing 54.7% encoding time with only 1.12% increase in terms of bit rate. Moreover, the proposed scheme performs consistently well for the high resolution sequences, of which the video coding efficiency is crucial in real applications.

Intra Prediction based on the Rate-distortion Cost in HEVC

High Efficiency Video Coding (HEVC) provides high coding efficiency, compared with H.264/AVC. In particular, intra coding efficiency is improved due to reference-sample filtering and unfiltering methods before intra prediction, where the reference-filtering methods are applied according to the block size and intra prediction mode. In this paper, a rate-distortion (RD) costbased intra prediction that computes the minimum RD cost between the filtered reference samples and the unfiltered reference samples is proposed. Intra coding efficiency is improved by using the probability update between the reference sample filtering in the current coding unit and the neighbor coding units, because the predefined reference-sample filtering in HEVC is not optimal in terms of intra coding efficiency. Experiment results show that the proposed method reduces the Bjøntegaard delta (BD) rates of the luma and chroma (Cb and Cr) by approximately 0.5% and 0.4%, respectively.

Binary and Multi-Class Learning Based Low Complexity Optimization for HEVC Encoding

High Efficiency Video Coding (HEVC) improves the compression efficiency at the cost of high computational complexity by using the quad-tree coding unit (CU) structure and variable prediction unit (PU) modes. To minimize the HEVC encoding complexity while maintaining its compression efficiency, a binary and multi-class support vector machine (SVM)-based fast HEVC encoding algorithm is presented in this paper. First, the processes of recursive CU decision and PU selection in HEVC are modeled as hierarchical binary classification and multi-class classification structures. Second, according to the two classification structures, the CU decision and PU selection are optimized by binary and multi-class SVM, i.e., the CU and PU can be predicted directly via classifiers without intensive rate distortion (RD) cost calculation. In particular, to achieve better prediction performance, a learning method is proposed to combine the off-line machine learning (ML) mode and on-line ML mode for classifiers based on a multiple reviewers system. Additionally, the optimal parameters determination scheme is adopted for flexible complexity allocation under a given RD constraint. Experimental results show that the proposed method can achieve 68.3%, 67.3%, and 65.6% time saving on average while the values of Bjontegaard delta peak signal-to-noise ratio are −0.093dB, −0.091dB, and −0.094dB and the values of Bjontegaard delta bit rate are 4.191%, 3.842%, and 3.665% under low delay $P$ main, low delay main, and random access configurations, respectively, when compared with the HEVC test model version HM 16.5. Meanwhile, the proposed method outperforms the state-of-the-art fast coding algorithms in terms of complexity reduction and RD performance.

JMJD6-mediated histone demethylation in preeclampsia is compromised due to reduced intracellular iron availability

In this paper, a novel dynamic voltage–frequency scaling-aware (DVFS-aware) bandwidth- efficient motion estimation (ME) scheme is presented for mobile application processor (AP) systems. Under volatile operating performance conditions due to the power management mechanism, we model the coding bandwidth (BW) and coding performance for the video processor as a convex function of the working frequency. In this paper, we present a bandwidth–rate–distortion (B–R–D) optimized framework that will guarantee the smallest possible rate–distortion cost among coding BW constraints applied in video coding design. By formulating the coding bandwidth-constrained ME problem as an optimization problem, known convex optimization theory can be applied to yield optimal resource-constrained compression. Using varied CIF (352×288)- and HP (1280×720)-sized video sequences with different motion activities over our proposed DVFS-aware video coding approach, the excellent results in terms of coding performance and coding bandwidth savings are obtained. With negligible quality loss, the proposed scheme under coding BW constraints achieves 45–65% coding BW usage reduction over HD-sized 30 frame/s video coding.