Rate-distortion Cost Function Research Articles

Convolutional Neural Network Based Synthesized View Quality Enhancement for 3D Video Coding.

The quality of synthesized view plays an important role in the three dimensional (3D) video system. In this paper, to further improve the coding efficiency, a convolutional neural network (CNN) based synthesized view quality enhancement method for 3D High Efficiency Video Coding (HEVC) is proposed. Firstly, the distortion elimination in synthesized view is formulated as an image restoration task with the aim to reconstruct the latent distortion free synthesized image. Secondly, the learned CNN models are incorporated into 3D HEVC codec to improve the view synthesis performance for both view synthesis optimization (VSO) and the final synthesized view, where the geometric and compression distortions are considered according to the specific characteristics of synthesized view. Thirdly, a new Lagrange multiplier in the rate-distortion (RD) cost function is derived to adapt the CNN based VSO process to embrace a better 3D video coding performance. Extensive experimental results show that the proposed scheme can efficiently eliminate the artifacts in the synthesized image, and reduce 25.9% and 11.7% bit rate in terms of peak-signal-to-noise ratio (PSNR) and structural similarity (SSIM) index, which significantly outperforms the state-of-theart methods.

High‐speed implementation of rate‐distortion optimised quantisation for H.265/HEVC

Rate-distortion optimised quantisation (RDOQ) is generally utilised in video coding for achieving higher coding efficiency. To determine the optimal quantised level for each transform coefficient, it requires considerable complexity in practice to calculate rate-distortion (RD) costs from multiple candidates of quantised levels. This study proposes a fast RDOQ algorithm with low computational complexity for the latest video coding standard, the high-efficiency video coding. Instead of calculating the RD costs from two candidates of quantised levels separately, their RD cost differences were derived. A bit-rate estimation method is then used to accurately calculate the length of coding bits in the RD cost function, avoiding the high-computation-cost process of context-adaptive binary arithmetic coding. Experimental results show that, with a negligible degradation of coding performance, the proposed algorithm is faster than RDOQ by 74.6% in average.

스케일러블 동영상 부호화 표준에서 움직임 예측 플래그를 위한 효율적인 부호화 방식

In the scalable video coding standard, inter-layer prediction based on the coding information of the base layer was adopted to increase the coding performance. This prediction tool results in new syntax elements called motion_prediction_flag (mPF) and residul_prediction_flag(rPF), which are carried to notify the motion vector predictor (MVP) and reference block required in the motion compensation of the decoder. In this paper, an efficient coding method for mPF is proposed to enhance coding efficiency of the salable video coding standard. Through an analysis on the transmission of mPF based on the relationship between the MVPs, we discover the conditions where mPF is unnecessary at the decoder and suggest a modified rate-distortion (RD) cost function to make RD optimization more effective. Simulation results show that the proposed method offers BD rate savings of approximately 1.4%, compared with the conventional SVC standard.

Early Termination for Residual Quadtree Decision-Making in HEVC

The progressive high efficiency video coding (HEVC) standard is based on a quadtree (QT) coding structure. The optimal residual quadtree (RQT) was selected for a given intra- and inter-prediction residual block by comparing the rate-distortion (R-D) cost function via all possible transform unit (TU) partitions recursively. However, zero-quantized blocks (ZQBs) are common after discrete cosine transform (DCT) and quantization (Q) due to the small values of prediction blocks. Therefore, when a large TU has negligible prediction residuals, the TU can be terminated early at the current depth of the RQT. This study proposes the use of ZQB detection techniques to accelerate RQT decision-making. In HEVC, RQT decision-making comprises a TU transform and TU split functions. The proposed method mathematically analyses DCT and Q processes, deriving two sufficient conditions to reduce the computational complexity of TU transform and TU split computations. Experimental results demonstrate that the proposed method is capable of reducing the number of computations associated with inter prediction RQT decisions by an average of 48.85% (low delay, high efficiency), 45.36% (low delay, low complexity), 50.96% (random access, high efficiency) and 48.33% (random access, low complexity), while retaining the encoding performance of the original HEVC encoder.

Rendering Distortion Estimation Model for 3D High Efficiency Depth Coding

A depth map represents three-dimensional (3D) scene geometry information and is used for depth image based rendering (DIBR) to synthesize arbitrary virtual views. Since the depth map is only used to synthesize virtual views and is not displayed directly, the depth map needs to be compressed in a certain way that can minimize distortions in the rendered views. In this paper, a modified distortion estimation model is proposed based on view rendering distortion instead of depth map distortion itself and can be applied to the high efficiency video coding (HEVC) rate distortion cost function process for rendering view quality optimization. Experimental results on various 3D video sequences show that the proposed algorithm provides about 31% BD-rate savings in comparison with HEVC simulcast and 1.3 dB BD-PSNR coding gain for the rendered view.

Intra mode decision and frame layer Lagrange multiplier adjustment scheme for H.264 based on structural similarity

H.264 takes rate distortion optimisation (RDO) technique to perform intra and inter mode decision and achieves higher coding efficiency, but the objective distortion metric such as mean square error (MSE) is employed in traditional RDO framework, which cannot acquire optimal subjective quality. In this paper, structural similarity (SSIM)-based subjective distortion is applied to RDO-based intra mode decision in H.264 I frame video coding, and a linear SSIM distortion model is firstly proposed and SSIM-based rate distortion cost function for intra mode decision is defined. Furthermore, a content adaptive frame layer Lagrange multiplier adjustment scheme is proposed to balance the tradeoff between rate and SSIM distortion better. Experimental results show that, the proposed method encodes image structural information more effectively and thus acquires better perceptual quality and subjective RDO performance compared with objective distortion-based RDO method. Under the same perceptual quality, our scheme achieves about 8·03% I frame bit rate reduction on average for various sequences over MSE-based RDO employed in JM reference software.

Distortion‐correcting algorithm of intra‐coding for flat region in depth map

An efficient distortion-correcting algorithm for the intra-coding of a depth map is presented by considering the property of the flat area in a depth map. The influence of the distortion of reference pixels on the rate-distortion (RD) cost function of the macro-block (MB) in a flat region of a depth map is analysed. Then, the RD performance of the flat region is improved by correcting the distortion of the first MB in the flat area on the basis of this analysis. Experiments show that the performance of the synthesis view of the proposed method can achieve average peak signal-to-noise ratio (PSNR) of 0.32 and 0.30 dB compared with the JMVC8.5 and the plane segmentation-based intra-prediction method, respectively.

Inter Frame Video Compression With Large Dictionaries of Tilings: Algorithms for Tiling Selection and Entropy Coding

We propose the use of large tree-structured dictionaries of tilings for video compression. Our first contribution is the construction of a rate-distortion cost function that admits fast search algorithms to select the optimal tiling for the motion compensation stage of a video coder. The computation of the cost is enabled through novel algorithms to approximate the bit rate and the distortion. Our second contribution is an efficient arithmetic coding algorithm to encode the selected tree-structured tiling. We illustrate the effectiveness of our approach by showing that a H.264/AVC-like video coder utilizing one of the proposed tiling selection methods results in up to 16% savings in bit rate for several standard video sequences as compared to H.264/AVC. This is accomplished with only a modest increase in the computation time at the encoder.

Enhanced SATD-based cost function for mode selection of H.264/AVC intra coding

In H.264/AVC, the concept of rate-distortion optimization (RDO) mode decision has proven to be an important coding tool. However, this mode decision method makes the encoding process extremely complex, especially in the computation of the rate-distortion cost function, which includes the computations of the sum of squared difference (SSD) between the original and reconstructed image blocks and context-based entropy coding of the block. In this paper, we propose an enhanced low-complexity cost function for H.264/AVC intra 4 × 4 and 8 × 8 mode selections. The enhanced cost function uses sum of absolute Hadamard-transformed differences (SATD) and mean absolute deviation of the residual block to estimate distortion part of the cost function. A threshold-based large coefficients count is also used for estimating the bit-rate part. The proposed method improves the rate-distortion performance of conventional fast cost functions while maintaining low complexity requirement. As a result, the encoding process can be significantly accelerated with the use of the proposed cost function. Simulation results confirm that the proposed method reduces about 83.74 % computation of intra encoding of JM 12.4 encoder with negligible rate-distortion performance degradation.

General Architecture for MPEG-2/H.263/H.264/AVC to H.264/AVC Intra Frame Transcoding

The latest international video-coding standard H.264/AVC significantly achieves better coding performance compared to prior video coding standards such as MPEG-2 and H.263, which have been widely used in today's digital video applications. To provide the interoperability between different coding standards, this paper proposes an efficient architecture for MPEG-2/H.263/H.264/AVC to H.264/AVC intra frame transcoding, using the original information such as discrete cosine transform (DCT) coefficients and coded mode type. Low-frequency components of DCT coefficients and a novel rate distortion cost function are used to select a set of candidate modes for rate distortion optimization (RDO) decision. For H.263 and H.264/AVC, a mode refinement scheme is utilized to eliminate unlikely modes before RDO mode decision, based on coded mode information. The experimental results, conducted on JM12.2 with fast C8MB mode decision, reveal that average 58%, 59% and 60% of computation (re-encoding) time can be saved for MPEG-2, H.263, H.264/AVC to H.264/AVC intra frame transcodings respectively, while preserving good coding performance when compared with complex cascaded pixel domain transcoding (CCPDT); or average 88% (a speed up factor of 8) when compared with CCPDT without considering fast C8MB. The proposed algorithm for H.264/AVC homogeneous transcoding is also compared to the simple cascaded pixel domain transcoding (with original mode reuse). The results of this comparison indicate that the proposed algorithm significantly outperforms the mode reuse algorithm in coding performance, with only slightly higher computation.

SATD-Based Intra Mode Decision for H.264/AVC Video Coding

In this letter, a rate estimation technique using transform coefficient variance is proposed in the rate-distortion cost function for intra mode decision. The new cost function achieves better coding performance than the cost function using the ¿-domain model. The new cost function brings out an average speed-up factor of 13 when compared with the rate-distortion optimization (RDO) mode decision, but with considerable degradation in rate-distortion performance. To improve the coding performance, we propose a sum of the absolute transformed differences (SATD)-based mode decision algorithm, which incorporates both SATD and the variance information into the RDO mode decision method. The results show that the SATD-based mode decision scheme can achieve a significant improvement in computation (with 21% of total encoding time, or a speed-up factor of 5) with negligible peak signal-to-noise ratio loss. The proposed algorithm is also compared with two other distinct algorithms , and the results indicate that the proposed algorithm outperforms these two edge detection algorithms in coding performance, as well as time saving.

Joint depth-motion dense estimation for multiview video coding

The multiview video coding (MVC) extension of H.264/MPEG-4 AVC [1] is one of the most promising visual encoders for three-dimensional television and free viewpoint video applications. In this paper, we propose a joint dense motion/disparity estimation algorithm, designed to replace the classical temporal/inter-view unit within MVC, which uses a block-based motion/disparity estimation. The motion vector fields and the disparity vector fields are therefore simultaneously derived using the stereo-motion consistency constraint in a set theoretic convex optimization framework. The obtained displacement vector fields are then jointly segmented by minimizing a rate-distortion cost function, in line with the multiple reference frame strategy used in H.264/MPEG-4 AVC. Experimental results demonstrate the benefits of the proposed method compared to the separated dense estimation scheme or the block-based estimation technique.

Transform-Domain Fast Sum of the Squared Difference Computation for H.264/AVC Rate-Distortion Optimization

In H.264/AVC, the rate-distortion optimization for mode decision plays a significant role to achieve its outstanding performance in terms of both compression efficiency and video quality. However, this mode decision process also introduces extremely high complexity in the encoding process especially the computation of the sum of squared differences (SSD) between the original and reconstructed image blocks. In this paper, fast SSD (FSSD) algorithms are proposed to reduce the complexity of the rate-distortion cost function implementation. The proposed FSSD algorithm is based on the theoretical equivalent of the SSDs in spatial and transform domains and determines the distortion in integer cosine transform domain using an iterative table-lookup quantization process. This approach could avoid the inverse quantization/transform and pixel reconstructions processes with nearly no rate-distortion performance degradation. In addition, the FSSD can also be used with efficient bit rate estimation algorithms to further reduce the cost function complexity. Experimental results show that the new FSSD can save up to 15% of total encoding time with less than 0.1% coding performance degradation and it can save up to 30% with ignorable performance degradation when combining with conventional bit rate estimation algorithm

Enhanced Intra-4$,times,$4 Mode Decision for H.264/AVC Coders

In this paper, we propose an enhanced rate-distortion cost function, which combines the sum of absolute integer-transformed differences (SAITD) and a rate predictor for H.264/AVC intra-4times4 mode decision. To reduce the computation of the SAITD, we further develop a fast computation algorithm, which successfully uses the property of linear transform and the fixed spatial relationship of predicted pixels in intra-modes. Simulation results show that the enhanced cost function with less computation achieves better coding performance than the cost function suggested in H.264 reference software JM 6.1d

Medical ultrasound image compression using joint optimization of thresholding quantization and best-basis selection of wavelet packets

This paper introduces an efficient image-coding algorithm using wavelet packets. The algorithm combines the top-down search approach with an operational rate-distortion (R-D) cost function to select the best wavelet packet basis at low-computational cost. The proposed method jointly optimizes the best-basis selection, coefficient ''thresholding'' and quantizer selection within the minimum description length (MDL) framework to develop a wavelet packet image coder named as JTQ-WP. We present results to verify the usefulness and versatility of this adaptive image coder both on medical US-images and natural images. The experimental results show that the joint optimization has a dramatic effect on the compression performance of medical ultrasound images. To further demonstrate the potential performance of the proposed method in comparison with the current state-of-the-art image coding algorithms, the results on Barbara image are also presented. The results show a coding gain of 0.91 dB over the benchmark wavelet-coding algorithm, SPIHT, on the Barbara image at a bit-rate of 0.25 bpp.

Low-complexity skip prediction for H.264 through Lagrangian cost estimation

A complexity reduction algorithm for an H.264 encoder is proposed. Computational savings are achieved by identifying, prior to motion estimation, macroblocks (MBs) that are likely to be skipped and hence saving further computational processing of these MBs. This early prediction is made by estimating a Lagrangian rate-distortion cost function which incorporates an adaptive model for the Lagrange multiplier parameter based on local sequence statistics. Simulation results demonstrate that the algorithm can achieve computational savings of 19%-67% (depending on the source sequence) with no significant loss of rate-distortion performance.

Hybrid multiple description video coding using optimal DCT coefficient splitting and SD/MD switching

This paper proposes a robust video transmission algorithm, which is based on the multiple description coding (MDC) scheme that optimally splits DCT coefficients, and also proposes a switching algorithm for the optimal operation of the encoder between the single description (SD) and multiple description (MD) mode under the given packet loss rate (PLR). In the proposed MDC algorithm, DCT coefficients are either duplicated to every stream or alternated to one of streams, which is decided by the rate-distortion (RD) optimization process. The optimization problem is formulated by modifying the DCT coefficient thresholding algorithm, which can be solved by dynamic programming. Although the MD coding is more resilient than the SD coding in the severe PLR condition, its excessive redundancy results in degraded RD performance in the low PLR. To increase the scalability of MDC, we propose the SD/MD switching method, which uses the SDC for the coding efficiency in the low PLR, and MDC for the error resilience in the high PLR. An RD optimization based strategy is proposed for optimal switching under the given channel condition. Since the decoder distortion is necessary to estimate RD cost function at the time of encoding, we adopt the recursive optimal per-pixel estimate (ROPE) technique. Experimental results show that the optimal combination of the optimal split of DCT coefficients and the SD/MD switching provides higher PSNR than the conventional error resilient methods based on the inter/intra switching in low PLR conditions as well as in high PLR.

Joint space-frequency segmentation using balanced wavelet packet trees for least-cost image representation

We examine the question of how to choose a space varying filterbank tree representation that minimizes some additive cost function for an image. The idea is that for a particular cost function, e.g., energy compaction or quantization distortion, some tree structures perform better than others. While the wavelet tree represents a good choice for many signals, it is generally outperformed by the best tree from the library of wavelet packet frequency-selective trees. The double-tree library of bases performs better still, by allowing different wavelet packet trees over all binary spatial segments of the image. We build on this foundation and present efficient new pruning algorithms for both one- and two-dimensional (1-D and 2-D) trees that will find the best basis from a library that is many times larger than the library of the single-tree or double-tree algorithms. The augmentation of the library of bases overcomes the constrained nature of the spatial variation in the double-tree bases, and is a significant enhancement in practice. Use of these algorithms to select the least-cost expansion for images with a rate-distortion cost function gives a very effective signal adaptive compression scheme. This scheme is universal in the sense that, without assuming a model for the signal or making use of training data, it performs very well over a large class of signal types. In experiments it achieves compression rates that are competitive with the best training-based schemes.

Quadtree Optimization for Image and Video Coding

The optimal design of quadtree-based codecs is addressed. Until now, work in this area has focused on the optimization of the quadtree structure for a given set of leaf quantizers while neglecting the design of the quantizers themselves. In cases where the leaf quantizers have been considered, codebooks have been optimized without regard to the ultimate quadtree segmentation. However, it is not sufficient to consider each problem independently, as separate optimization leads to an overall suboptimal solution. Rather, joint design of the quadtree structure and the leaf codebooks must be considered for overall optimality. The method we suggest is a “quadtree” constrained version of the entropy-constrained vector quantization design method. To this end, a centroid condition for the leaf codebooks is derived that represents a necessary optimality condition for variable-rate quadtree coding. This condition, when iterated with the optimal quadtree segmentation strategy of Sullivan and Baker results in a monotonically descending rate-distortion cost function, and consequently, an (at least locally) optimal quadtree solution.

Adaptive transforms for image coding using spatially varying wavelet packets

We introduce a novel, adaptive image representation using spatially varying wavelet packets (WPs), Our adaptive representation uses the fast double-tree algorithm introduced previously (Herley et al., 1993) to optimize an operational rate-distortion (R-D) cost function, as is appropriate for the lossy image compression framework. This involves jointly determining which filter bank tree (WP frequency decomposition) to use, and when to change the filter bank tree (spatial segmentation). For optimality, the spatial and frequency segmentations must be done jointly, not sequentially. Due to computational complexity constraints, we consider quadtree spatial segmentations and binary WP frequency decompositions (corresponding to two-channel filter banks) for application to image coding. We present results verifying the usefulness and versatility of this adaptive representation for image coding using both a first-order entropy rate-measure-based coder as well as a powerful space-frequency quantization-based (SPQ-based) wavelet coder introduced by Xiong et al. (1993).