Bitrate Savings Research Articles

Gradient-oriented directional predictor for HEVC planar and angular intra prediction modes to enhance lossless compression

Recent advancements in the capture and display technologies motivated the ITU-T Video Coding Experts Group and ISO/IEC Moving Picture Experts Group to jointly develop the High-Efficiency Video Coding (HEVC), a state-of-the-art video coding standard for efficient compression. The compression applications that essentially require lossless compression scenarios include medical imaging, video analytics, video surveillance, video streaming etc., where the content reconstruction should be flawless. In the proposed work, we present a gradient-oriented directional prediction (GDP) strategy at the pixel level to enhance the compression efficiency of the conventional block-based planar and angular intra prediction in the HEVC lossless mode. The detailed experimental analysis demonstrates that the proposed method outperforms the lossless mode of HEVC anchor in terms of bit-rate savings by 8.29%, 1.65%, 1.94% and 2.21% for Main-AI, LD, LDP and RA configurations respectively, without impairing the computational complexity. The experimental results also illustrates that the proposed predictor performs superior to the existing state-of-the-art techniques in the literature.

DC Coefficient Estimation of Intra-Predicted Residuals in HEVC

This paper presents a DC coefficient estimation algorithm for intra-predicted residual blocks in the High-Efficiency Video Coding (HEVC) standard. Discarding the DC coefficient directly in each transform block leads to substantial bit-saving, but at the same time produces strong discontinuities between neighboring blocks. To overcome this problem, we propose an estimation algorithm for the DC coefficient, which solves an optimal offset in a closed-form to recover the corresponding block edges. Then, we embed this algorithm into HEVC in its rate-distortion optimized quantization and sign bit hiding steps. Furthermore, a flag is signaled to decide whether the DC estimation strategy is used for each transform block. Test results under the common test condition show that our algorithm achieves 1.5% and 1.6% BD-rate reduction on average for luma and chroma, respectively, under all intra configuration. In the meantime, our simulation results show that both encoding time and decoding time increase only slightly (about 10%, without any special optimization on programming our proposed algorithm). When testing the proposed DC estimation algorithm on inter coding configurations, including low delay with P pictures, low delay with B pictures, and random access, we can also achieve 0.5%–1.1% bit-rate savings on average, while nearly no extra encoding and decoding time is needed.

Residual‐error prediction based on deep learning for lossless image compression

A novel residual-error prediction method based on deep learning with application in lossless image compression is introduced. The proposed method employs machine learning tools to minimise the residual error of the employed prediction tools. Experimental results demonstrate average bitrate savings of 32% over the state-of-the-art in lossless image compression. To the best of the authors’ knowledge, this Letter is the first to propose a deep-learning based method for residual-error prediction.

Forward Adaptive Laplacian Source Coding Based on Restricted Quantization

A novel solution for Laplacian source coding based on three-level quantization is proposed in this paper. The restricted three-level quantizer is designed by assuming the restricted Laplacian distribution of the input signal. Quantizer and Huffman encoder are jointly designed. Forward adaptive scheme was employed, where the adaptation to the signal variance (power) was performed on frame-by frame basis. We employ switched model that consists of two restricted quantizers having unequal support regions. The simulation results (measured as SQNR) of the proposed scheme with a switched restricted three-level quantizer are compared to the cases when it involves three-level unrestricted quantizer and the Lloyd-Max quantizers having N=2 and N=4 levels. It is shown that the proposed solution offers performance comparable to the one of N=4 levels Lloyd-Max’s baseline with large savings in bit rate, while outperforming two other baselines.DOI: http://dx.doi.org/10.5755/j01.itc.47.2.16670

Towards Real-Time Service from Remote Sensing: Compression of Earth Observatory Video Data via Long-Term Background Referencing

City surveillance enables many innovative applications of smart cities. However, the real-time utilization of remotely sensed surveillance data via unmanned aerial vehicles (UAVs) or video satellites is hindered by the considerable gap between the high data collection rate and the limited transmission bandwidth. High efficiency compression of the data is in high demand. Long-term background redundancy (LBR) (in contrast to local spatial/temporal redundancies in a single video clip) is a new form of redundancy common in Earth observatory video data (EOVD). LBR is induced by the repetition of static landscapes across multiple video clips and becomes significant as the number of video clips shot of the same area increases. Eliminating LBR improves EOVD coding efficiency considerably. First, this study proposes eliminating LBR by creating a long-term background referencing library (LBRL) containing high-definition geographically registered images of an entire area. Then, it analyzes the factors affecting the variations in the image representations of the background. Next, it proposes a method of generating references for encoding current video and develops the encoding and decoding framework for EOVD compression. Experimental results show that encoding UAV video clips with the proposed method saved an average of more than 54% bits using references generated under the same conditions. Bitrate savings reached 25–35% when applied to satellite video data with arbitrarily collected reference images. Applying the proposed coding method to EOVD will facilitate remote surveillance, which can foster the development of online smart city applications.

Plenoptic Image Coding using Macropixel-based Intra Prediction.

The plenoptic image in a super high resolution is composed of a number of macropixels recording both spatial and angular light radiance. Based on the analysis of spatial correlations of macropixel structure, this paper proposes a macropixel-based intra prediction method for plenoptic image coding. After applying an invertible image reshaping method to the plenoptic image, the macropixel structures are aligned with the coding unit grids of a block-based video coding standard. The reshaped and regularized image is compressed by the video encoder comprising the proposed macropixel-based intra prediction, which includes three modes: multi-block weighted prediction mode (MWP), co-located single-block prediction mode (CSP), and boundary matching based prediction mode (BMP). In the MWP mode and BMP mode, the predictions are generated by minimizing spatial Euclidean distance and boundary error among the reference samples, respectively, which can fully exploit spatial correlations among the pixels beneath the neighboring microlens. The proposed approach outperforms HEVC by an average of 47.0% bitrate reduction. Compared with other state-of-the-art methods, like pseudo-video based on tiling and arrangement method (PVTA), intra block copy (IBC) mode, and locally linear embedding (LLE) based prediction, it can also achieve 45.0%, 27.7% and 22.7% bitrate savings on average, respectively.

Adaptive Lagrangian Multiplier derivation model for depth map coding

In stereoscopic video coding, vast and varied coding modes are utilized to adapt various video content. In order to select the optimal coding parameter set for coding unit, rate–distortion optimization (RDO) criterion is employed as a critical technology. It has been found that the Lagrangian Multiplier of RDO criterion for depth map coding has an impact on coding performance of stereoscopic video. Therefore, the Lagrangian Multiplier should be determined properly. In this paper, we first provide an analysis of the factors that could affect the Lagrangian Multiplier. Then, an adaptive Lagrangian Multiplier derivation model for depth map coding in 3D-HEVC is established by involving these affect factors. At last, the optimal coding parameter set can be obtained by the adjusted RDO criterion through the derived Lagrangian Multiplier. Experimental results demonstrate that the proposed adaptive Lagrangian Multiplier derivation model can lead to a favorable coding performance of stereoscopic video. An average 3.13% bitrate saving can be achieved compared with the HTM 16.0 reference software. Moreover, compared to two other state-of-art methods, namely the Yang’s method and the Huo’s method, the proposed model can achieve 0.21% and 0.11% bitrate savings, respectively.

Source Distortion Temporal Propagation Analysis for Random-Access Hierarchical Video Coding Optimization

Due to the widely used inter prediction in the current video coding standards, encoding units in different frames is of temporal dependency in that the rate-distortion optimization (RDO) of one unit may affect the coding performance of the following units in the temporal domain. To achieve optimal coding solution for a given video sequence, temporal dependency among units needs to be considered in the RDO process, which is known as the temporally dependent RDO (TD-RDO). The hierarchical coding structure (HCS) employed in the High Efficiency Video Coding (HEVC) standard further complicates this problem by grouping frames into different layers of varying coding strategies, leading to a more complex temporal relationship. In our earlier work, we addressed TD-RDO for the low delay HCS (LD-HCS), where only uni-prediction is considered. This paper aims to address more complicated TD-RDO under random access HCS (RA-HCS), where both uni-prediction and bi-prediction are considered, making the temporal relationship even more intricate. The temporal dependency introduced in the RA-HCS is thoroughly examined and an RA-based TD-RDO scheme is formulated for each layer by modeling temporal propagation of distortion under different prediction types. Based on the formulation, the global Lagrange multiplier can be obtained analytically. Moreover, the effect of random access point pictures is considered in the RA-based TD-RDO scheme. The proposed method can be simply realized by updating the Lagrange multiplier as in the independent RDO formulation or combined with adjusting quantization parameter (QP) for better results in terms of BD-rate saving. Experimental results show that under RA-HCS, the proposed method, by adapting the Lagrange multiplier only, can achieve about 2.2% bitrate savings in average. With multi-QP optimization, an average BD-rate gain of 5.2% can be obtained.

HEVC Intra Frame Coding Based on Convolutional Neural Network

The HEVC standard offers high performance with a lower bitrate for intra frame coding, but still requires many bits. An alternative intra frame coding framework based on convolutional neural network (CNN) is proposed in this paper. Two CNN models, simplified CNN (S-CNN) and complicated CNN (C-CNN), were designed and trained to improve the coding performance of HEVC. In intra frame coding, the trained CNN predicts the residual for reconstructed blocks to enhance visual quality. Due to the high computational complexity of CNN in HEVC encoding, we further explore the tradeoff between computational complexity and coding performance. For S-CNN, an early termination mechanism is proposed to further reduce the HEVC encoding complexity. With regard to C-CNN, a GPU-based heterogeneous architecture is proposed to accelerate CNN processing. Experimental results show that the proposed method with S-CNN achieves bitrate savings of 3.1% with a 37% increase in time cost, and 2.8% in bitrate savings with only a 23% increase in time cost when applying the early termination algorithm. In the case of C-CNN, the execution time of CNN can reach increases in speed of 8.8–17.8 $\times$ and a bitrate savings of up to 5.1% on average.

Merge Mode for Deformable Block Motion Information Derivation

Motion estimation (ME) plays an important role in most video coding standards since it can improve the coding performance significantly. Block matching ME (BMME) based on translation motion model is employed in both H.264/Advanced Video Coding and High Efficiency Video Coding (HEVC), which assumes that all the pixels in a block undergo the same motion. However, BMME cannot represent the complex motion accurately, such as rotation, zoom, and deformation motion. To resolve the problem, a merge mode for deformable block motion information derivation is proposed, which considers bilinear interpolation model, six-parameter affine model, and four-parameter affine model simultaneously. The sum of absolute transformed differences criterion is used to select the best model for a block. In the proposed merge mode, the motion information of the current block is derived from the motion information of its neighbors. It does not need to perform ME and transmit motion parameters. The proposed merge mode is integrated into HEVC test model (HM) 14.0. The simulation results show that compared with HM14.0, the proposed merge mode can bring 1.3%, 1.7%, and 2.7% Bjontegaard delta (BD) bitrate savings on average for test sequences of HEVC, and 11.0%, 15.2%, and 17.4% BD bitrate savings for deformation sequences in random access main, low-delay B main, and low-delay P main test conditions, respectively, while the increased encoding and decoding complexities are moderate.

Light Field Image Coding Using High-Order Intrablock Prediction

This paper proposes a two-stage high-order intrablock prediction method for light field image coding. This method exploits the spatial redundancy in lenslet light field images by predicting each image block, through a geometric transformation applied to a region of the causal encoded area. Light field images comprise an array of microimages that are related by complex geometric transformations that cannot be efficiently compensated by state-of-the-art image coding techniques, which are usually based on low-order translational prediction models. The two-stage nature of the proposed method allows us to choose the order of the prediction model most suitable for each block, ranging from pure translations to projective or bilinear transformations, optimized according to an appropriate rate-distortion criterion. The proposed higher order intrablock prediction approach was integrated into a high efficiency video coding (HEVC) codec and evaluated for both unfocused and focused light field camera models, using different resolutions and microlens arrays. Experimental results show consistent bitrate savings, which can go up to 12.62%, when compared to a lower order intrablock prediction solution and 49.82% when compared to HEVC still picture coding.

High-Efficiency Image Coding via Near-Optimal Filtering

Wiener filtering, which has been widely used in the field of image restoration, is statistically optimal in the sense of mean square error. The adaptive loop filter in video coding inherits the design of Wiener filters, and has been proved to achieve significant improvement on compression performance by reducing coding artifacts and providing high-quality references for subsequent frames. To further improve the compression performance via filtering technique, we explore the factors that may hinder the potential performance of Wiener-based filters, and propose a near-optimal filter learning scheme for high-efficiency image coding. Based on the analyses, we observe that the foremost factor affecting the performance of Wiener-based filters is the divergence of statistical characteristics of training samples, instead of the filter taps or shapes. In view of this, we propose an iterative training method to derive the near-optimal Wiener filter parameters by simultaneously labeling sample categories at the pixel level. These parameters are compressed and transmitted to the decoder side to improve the quality of decoded images by reducing the coding artifacts. Experimental results show that the proposed scheme achieves significant bitrate savings compared with high-efficiency video coding in high-bitrate intra coding scenario.

Low-Complexity Enhancement Layer Compression for Scalable Lossless Video Coding Based on HEVC

Lossless compression is desired, especially for professional applications like medical imaging. However, scalability may be necessary to transmit a fast preview of pictures when the channel capacity is limited. Furthermore, there may be the need for random access to single frames of a losslessly reconstructed video. We present and evaluate a lossy-to-lossless scalable video coding system in which the scalability is achieved by using a lossy base layer (BL) in conjunction with lossless compression of the reconstruction error in the enhancement layer (EL). High Efficiency Video Coding (HEVC) is employed for encoding the BL. For compression of the EL, we propose a low-complexity scheme called sample-based weighted prediction for EL coding (SELC). Furthermore, EL compression using JPEG-LS and the scalable extension of HEVC (SHVC) are evaluated. The performance of scalable coding using each of these three methods is compared with lossless single-layer (SL) coding using the intra-main RExt configuration. Experimental results show that the scalable coding with SELC achieves average bitrate savings of 7.3% compared with the lossless SL coding, while average bitrate reductions of only 6.3% and 3.9% are obtained using JPEG-LS and SHVC, respectively. Concerning runtime, the additional encoder runtimes of SELC and JPEG-LS compared with the BL coding are negligible. In contrast to that, the runtime of the SHVC EL encoder is much higher and similar to the runtime of the BL encoder.

Performance Comparison of High-Efficiency Video Coding (HEVC) with Standard H.264/AVC Encoders

Background: The emergence of new video applications and beyond high-definition formats (e.g.,4k×2k or 8k×4k resolution) signifies the need to develop a new video coding standard that is more sophisticated than other existing standards, including H.264/AVC. The new standard must have higher coding efficiency than that of H.264/AVC. High efficiency video coding (HEVC) has been designed to address all existing applications of H.264/MPEG-4 AVC and focus on suitability for high-level aplications. Method: In this study, we compared the new HEVC standardization and the previous version (H.264/AVC). The two encoders and the different tools and algorithms used were also comprehensively investigated. Finally, a comparison between software and hardware implementation was conducted. This study provides a relevant new hardware architecture for HEVC (H.265) using the existing one for H.264/AVC. Result: Experimental results show that H.265/MPEG-HEVC provides significant average bitrate savings of 40.3% and 35.4% for interactive and entertainment applications, respectively, compared with H.264/MPEG-AVC. Keywords: Video compression, embedded systeM, HEVC, H.264/AVC, video coding standards, video CODECs Syntax overview.

Attention-Weighted Texture and Depth Bit-Allocation in General-Geometry Free-Viewpoint Television

In a free-viewpoint television network, each viewer chooses its point of view from which to watch a scene. We use the concept of total observed distortion, wherein we aim to minimize the distortion of the view observed by the viewers as opposed to the distortion of each camera, to develop an optimized bit-rate allocation for each camera. Our attention-weighted approach effectively gives more bits to the cameras that are more watched. The more concentrated the viewer distribution, the larger the bit-rate savings, for a given total observed distortion, compared with the uniform rate allocation. We analyze and model the distortion of a synthesized view as a function of the distortions (both in texture and/or depth) of the nearby cameras. Based on such models, we develop optimal rate-allocation methods for texture images, considering a uniform bit allocation for depth, and for both texture and depth simultaneously. Simulation results are shown, demonstrating not only the correctness of the optimized solution, but also measuring its improvement against uniform rate allocation for a few viewer distributions.

A new compressive sensing video coding framework based on Gaussian mixture model

In this paper, we specifically design an efficient compressive sensing video (CSV) coding framework for the CSV system, by considering the distribution characteristics of the CSV frame. To explore the spatial redundancy of the CSV, the CSV frame is first divided into blocks and each block is modeled by a Gaussian mixture model (GMM), and then it is compressed by a product vector quantization. We further explore the temporal redundancy of the CSV by encoding the adjacent CSV frames by the differential pulse code modulation technique and the arithmetic encoding technique. Experiment results show that the proposed CSV coding solution maintains low coding complexity, which is required by the CSV system. Meanwhile, it achieves significant BD-PSNR improvement by about 7.13–11.41dB (or equivalently 51.23–66.96% bitrate savings) compared with four existing video coding solutions, which also have low computational complexity and suit for the CSV system.

Utility-Driven Adaptive Preprocessing for Screen Content Video Compression

In this work, we propose a utility-driven preprocessing technique for high-efficiency screen content video (SCV) compression based on the temporal masking effect, which was found to be a fundamental attribute that plays an important role in human visual perception of video quality, but has not been fully exploited in the context of SCV coding. Specifically, we investigate the temporal masking effect from the perspective of perceived utility, which allows us to preserve the quality of the high utility content and substitute the low utility regions with the corresponding smooth version. To distinguish the regional utilities, a specifically designed block type identification algorithm for screen content is employed to measure the local properties. Subsequently, the Gaussian filter is applied to smooth out the high-frequency components in the detected low utility regions to save consumption bits. Validations based on subjective testings show that the proposed approach is capable of achieving significant bitrate savings with little sacrifice on the final utility compared with the conventional SCV coding scheme.

Just-Noticeable Difference-Based Perceptual Optimization for JPEG Compression

The Quantization table in JPEG, which specifies the quantization scale for each discrete cosine transform (DCT) coefficient, plays an important role in image codec optimization. However, the generic quantization table design that is based on the characteristics of human visual system (HVS) cannot adapt to the variations of image content. In this letter, we propose a just-noticeable difference (JND) based quantization table derivation method for JPEG by optimizing the rate-distortion costs for all the frequency bands. To achieve better perceptual quality, the DCT domain JND-based distortion metric is utilized to model the stair distortion perceived by HVS. The rate-distortion cost for each band is derived by estimating the rate with the first-order entropy of quantized coefficients. Subsequently, the optimal quantization table is obtained by minimizing the total rate-distortion costs of all the bands. Extensive experimental results show that the quantization table generated by the proposed method achieves significant bit-rate savings compared with JPEG recommended quantization table and specifically developed quantization tables in terms of both objective and subjective evaluations.

New adaptive filters as perceptual preprocessing for rate-quality performance optimization of video coding

In this paper, we introduce two perceptual filters as pre-processing techniques to reduce the bitrate of compressed high-definition (HD) video sequences at constant visual quality. The goal of these perceptual filters is to remove spurious noise and insignificant details from the original video prior to encoding. The proposed perceptual filters rely on two novel adaptive filters (called BilAWA and TBil) which combine the good properties of the bilateral and Adaptive Weighting Average (AWA) filters. The bilateral and AWA filters being initially dedicated to denoising, the behaviour of the proposed BilAWA and TBil adaptive filters is first analyzed in the context of noise removal on HD test images. The first set of experimental results demonstrates their effectiveness in terms of noise removal while preserving image sharpness. A just noticeable distortion (JND) model is then introduced in the novel BilAWA and TBil filters to adaptively control the strength of the filtering process, taking into account the human visual sensitivity to signal distortion. Visual details which cannot be perceived are smoothed, hence saving bitrate without compromising perceived quality. A thorough experimental analysis of the perceptual JND-guided filters is conducted when using these filters as a pre-processing step prior to MPEG-4/AVC encoding. Psychovisual evaluation tests show that the proposed BilAWA pre-processing filter leads to an average bitrate saving of about 19.3% (up to 28.7%) for the same perceived visual quality. The proposed new pre-filtering approach has been also tested with the new state-of-the-art HEVC standard and has given similar efficiency in terms of bitrate savings for constant visual quality.

Pseudo-Sequence-Based 2-D Hierarchical Coding Structure for Light-Field Image Compression

In this paper, we present a novel pseudo sequence based 2-D hierarchical reference structure for light-field image compression. In the proposed scheme, we first decompose the light-field image into multiple views and organize them into a 2-D coding structure according to the spatial coordinates of the corresponding microlens. Then we mainly develop three technologies to optimize the 2-D coding structure. First, we divide all the views into four quadrants, and all the views are encoded one quadrant after another to reduce the reference buffer size as much as possible. Inside each quadrant, all the views are encoded hierarchically to fully exploit the correlations between different views. Second, we propose to use the distance between the current view and its reference views as the criteria for selecting better reference frames for each inter view. Third, we propose to use the spatial relative positions between different views to achieve more accurate motion vector scaling. The whole scheme is implemented in the reference software of High Efficiency Video Coding. The experimental results demonstrate that the proposed novel pseudo-sequence based 2-D hierarchical structure can achieve maximum 14.2% bit-rate savings compared with the state-of-the-art light-field image compression method.