Just Noticeable Distortion Thresholds Research Articles

Learning-Based Just-Noticeable-Quantization- Distortion Modeling for Perceptual Video Coding.

Conventional predictive video coding-based approaches are reaching the limit of their potential coding efficiency improvements, because of severely increasing computation complexity. As an alternative approach, perceptual video coding (PVC) has attempted to achieve high coding efficiency by eliminating perceptual redundancy, using just-noticeable-distortion (JND) directed PVC. The previous JNDs were modeled by adding white Gaussian noise or specific signal patterns into the original images, which were not appropriate in finding JND thresholds due to distortion with energy reduction. In this paper, we present a novel discrete cosine transform-based energy-reduced JND model, called ERJND, that is more suitable for JND-based PVC schemes. Then, the proposed ERJND model is extended to two learning-based just-noticeable-quantization-distortion (JNQD) models as preprocessing that can be applied for perceptual video coding. The two JNQD models can automatically adjust JND levels based on given quantization step sizes. One of the two JNQD models, called LR-JNQD, is based on linear regression and determines the model parameter for JNQD based on extracted handcraft features. The other JNQD model is based on a convolution neural network (CNN), called CNN-JNQD. To our best knowledge, our paper is the first approach to automatically adjust JND levels according to quantization step sizes for preprocessing the input to video encoders. In experiments, both the LR-JNQD and CNN-JNQD models were applied to high efficiency video coding (HEVC) and yielded maximum (average) bitrate reductions of 38.51% (10.38%) and 67.88% (24.91%), respectively, with little subjective video quality degradation, compared with the input without preprocessing applied.

Saliency-Guided Just Noticeable Distortion Estimation Using the Normalized Laplacian Pyramid

The human visual system (HVS), like any other physical system, has limitations. For instance, it is known that the HVS can only sense the content changes that are larger than the so-called just noticeable distortion (JND) threshold. Also, to reduce the computational load on the brain, the visual attention mechanism is deployed such that regions with higher visual saliency are processed with higher priority than other less-salient regions. It is also known that visual saliency has a modulatory effect on JND thresholds. In this letter, we present a novel pixel-wise JND estimation method that considers the interplay between visual saliency and JND thresholds. In the proposed method, the largest JND thresholds of a given image are found such that the perceptual distance between the image and its JND noise-contaminated version is minimized in a perceptual space defined by the coefficients of the image in a normalized Laplacian pyramid. Experimental results indicate that the proposed method outperforms four of the latest JND models for static images.

A fast and HEVC-compatible perceptual video coding scheme using a transform-domain Multi-Channel JND model

Coding optimization methods incorporating the just noticeable distortion (JND) model, called perceptual video coding (PVC), have drawn much attention in recent years for better video coding performance. To further remove perceptual redundancy in every channel and improve the coding performance, this paper proposes a fast PVC scheme in the latest High Efficiency Video Coding (HEVC) framework based on our proposed variable block-size transform-domain multi-channel JND model. Firstly, through extensive experiments, we find out for the first time that the contrast masking (CM) effects for chroma channels show a lowpass property in frequency, which differs from the luma channel that has a bypass property. Based on this observation, CM effects in chroma blue (Cb) and chroma red (Cr) channels are modeled as a continuous function for variable-sized blocks, respectively. Secondly, since different characteristics of the human visual system (HVS) exhibit quite distinct effects in luma and chroma channels and effects in chroma channels were not well explored, we develop a new JND model through comprehensive consideration for both luma and chroma channels of five typical HVS effects, with especial focus on parameterized modeling of chroma channels in each effect. Finally, to incorporate the proposed JND model into the latest HEVC coding framework, a multi-channel coefficients suppression method based on JND thresholds and quantization parameter (QP) ranges is proposed in the transform and quantization process, which can decrease the computational complexity. Extensive experimental results show that the proposed PVC scheme implemented in HEVC reference software (HM15.0) can yields significant bit saving of up to 25.91% and on average 9.42% with similar subjective quality, compared to HM15.0, and consistently outperforms two PVC schemes with much reduced bitrate and complexity overhead.

A saliency-modulated just-noticeable-distortion model with non-linear saliency modulation functions

It is well-known that the human visual system (HVS) cannot sense small variations of visual signals below the so-called just-noticeable distortion (JND) thresholds due to their underlying spatial/temporal masking properties. It is also known that the visual attention mechanism of the human brain can enhance or reduce visual sensitivity. In other words, the visual attention has modulatory effects on JND thresholds. The current knowledge also states that the visual attention is mainly driven by visual saliency in an automatic and involuntary manner. In this paper we present a saliency-modulated JND (SJND) model for static images in the discrete cosine transform (DCT) domain. In the proposed model, the JND thresholds of each block in a given image are elevated by two non-linear modulation functions using the visual saliency of the block. The parameters of the saliency modulation functions are obtained through an optimization framework, which utilizes a state-of-the-art saliency-based objective image quality assessment method. To evaluate the proposed SJND model, two subjective experiments were conducted. The obtained experimental results demonstrated that the proposed method achieves a high accuracy in JND estimation, and also it provides a high distortion-hiding capacity.

Fast synthesized and predicted just noticeable distortion maps for perceptual multiview video coding

The just noticeable distortion (JND) map is a useful tool for perceptual video coding. However, direct calculation of the JND map incurs high complexity, and the problem is aggravated in multiview video coding. In this paper, two fast methods are proposed to generate the JND maps of multiview videos. In the first method, the JND maps of some anchor views are used to synthesize the JND maps of other views via the depth image based rendering (DIBR), which can be much faster than direct JND computation. In the second method, the motion and disparity vectors obtained during the video coding are employed to predict the JND maps. If the prediction is not satisfactory, the JND block will be refreshed by calculating the JND directly. This method does not need any camera parameters and depth maps. The performances of the two fast JND map generation methods are evaluated in a perceptual MVC framework, where the residuals after spatial, temporal, or inter-view prediction are tuned according to the JND thresholds to save the bits without affecting the perceptual quality. Experimental results show that the JND prediction method has better accuracy and lower complexity. In addition, both fast JND methods lead to negligible degradation of the coding performance, compared to the direct JND method.