Proposed Just Noticeable Distortion Model Research Articles

Surprise-based JND estimation for perceptual quantization in H.265/HEVC codecs

Just noticeable distortion (JND), reflecting the perceptual redundancy directly, has been widely used in image and video compression. However, the human visual system (HVS) is extremely complex and the visual signal processing has not been fully understood, which result in existing JND models are not accurate enough and the bitrate saving of JND-based perceptual compression schemes is limited. This paper presents a novel pixel-based JND model for videos and a JND-based perceptual quantization scheme for HEVC codecs. In particular, positive and negative perception effects of the inter-frame difference and the motion information are analyzed and measured with an information-theoretic approach. Then, a surprise-based JND model is developed for perceptual video coding (PVC). In our PVC scheme, the frame-level perceptual quantization parameter (QP) is derived on the premise that the coding distortion is infinitely close to the estimated JND threshold. On the basis of the frame-level perceptual QP, we determine the perceptual QP for each coding unit through a perceptual adjustment function to achieve better perceptual quality. Experimental results indicate that the proposed JND model outperforms existing models significantly, the proposed perceptual quantization scheme improves video compression efficiency with better perceptual quality and lower coding complexity.

Hierarchical Predictive Coding-Based JND Estimation for Image Compression.

The human visual system (HVS) is a hierarchical system, in which visual signals are processed hierarchically. In this paper, the HVS is modeled as a three-level communication system and visual perception is divided into three stages according to the hierarchical predictive coding theory. Then, a novel just noticeable distortion (JND) estimation scheme is proposed. In visual perception, the input signals are predicted constantly and spontaneously in each hierarchy, and neural response is evoked by the central residue and inhibited by surrounding residues. These two types' residues are regarded as the positive and negative visual incentives which cause positive and negative perception effects, respectively. In neuroscience, the effect of incentive on observer is measured by the surprise of this incentive. Thus, we propose a surprise-based measurement method to measure both perception effects. Specifically, considering the biased competition of visual attention, we define the product of the residue self-information (i.e., surprise) and the competition biases as the perceptual surprise to measure the positive perception effect. As for the negative perception effect, it is measured by the average surprise (i.e., the local Shannon entropy). The JND threshold of each stage is estimated individually by considering both perception effects. The total JND threshold is finally obtained by non-linear superposition of three stage thresholds. Furthermore, the proposed JND estimation scheme is incorporated into the codec of Versatile Video Coding for image compression. Experimental results show that the proposed JND model outperforms the relevant existing ones, and over 16% of bit rate can be reduced without jeopardizing the perceptual quality.

Surprise-Based JND Estimation for Images

In visual perception, the neural response is evoked by the central stimulus and inhibited by the surrounding stimuli. These two stimuli can be regarded as the positive and negative incentives for perception. In this letter, the aftereffect of two incentives is incorporated into the just noticeable distortion (JND) estimation to avoid the threshold overestimation issue. In neuroscience, how an incentive affects an observer is measured by the surprise of this incentive which is estimated by the residual self-information according to the effective coding theory. Considering the competition biases, we define the product of the residual self-information and the biases, named as perceptual surprise, to measure the positive effect. As for the negative effect, we measure it with the average surprise (i.e., local Shannon entropy). Besides, visual signals are processed hierarchically in the HVS. To simplify the JND modeling process, the HVS is divided into two levels and the perception is simply divided into two stages. In each stage, the threshold is estimated individually and the total JND threshold is obtained by non-linear superposition of two stage thresholds. Experimental results show that the proposed JND model outperforms the existing pixel-based JND models and effectively avoid the threshold overestimation issue.

Perceptual Video Coding Scheme Using Just Noticeable Distortion Model Based on Entropy Filter

Because perceptual video coding (PVC) can reduce bitrates with negligible visual quality loss in video compression, a PVC scheme based on just noticeable distortion (JND) model is proposed for ultra-high definition video. Firstly, the proposed JND model is designed, considering the spatial JND characteristics such as contrast sensitivity, luminance adaptation and saliency weight factor. Secondly, in order to perform precise JND suppression, the Gauss differential entropy (GDE) filter is designed to divide the image into smooth and complex texture region. Thirdly, through incorporating the proposed JND model into the encoding process, the transform coefficients are suppressed in harmonization with the transform/quantization process of high efficiency video coding (HEVC). In order to achieve the JND suppression effectively, a distortion compensation factor and distortion compensation control factor are incorporated to control the extent of distortion in the rate distortion optimization process. The experimental results show that the proposed PVC scheme can achieve a remarkable bitrate reduction of 32.98% for low delay (LD) configuration and 28.61% for random access (RA) configuration with a negligible subjective quality loss. Meanwhile, the proposed method only causes about average 12.94% and 22.45% encoding time increase under LD and RA configuration compared with an HEVC reference software, respectively.

Pattern complexity-based JND estimation for quantization watermarking

The perceptual just noticeable distortion (JND), which plays an important role in perceptual image watermarking and can refers to the optimal tradeoff between imperceptibility and robustness. Major challenges of the perceptual quantization watermarking approach are two-fold: (1) Most DCT-based JND models cannot accurately estimate the contrast masking effect due to the complicated interaction among visual contents. Research on cognitive science shows that the HVS is adaptive to extract the visual pattern for image understanding, and we can formulate the pattern complexity as another factor to determine the total watermarking strength. (2) Moreover, the calculated JND values will change as watermark embedding can affect the pixels of the image, i.e. the operation in cross-domain reduce the watermarking robustness. Therefore, in this regard, the maximum directional energy is calculated by three AC DCT coefficients, which can measure the different direction energy and keep the pattern complexity measurement insensitive to the changes caused by watermarking procedure. The luminance contrast is also calculated in the DCT domain. So a pattern complexity-based perceptual JND estimation model is designed by takeing DCT-based pattern complexity and luminance contrast into account. Furthermore, a new logarithmic quantization watermarking scheme is presented based on the proposed model to verify the feasibility and effectiveness of our proposed JND model. Experimental results show that the new built JND model can effectively enhance the robustness of the quantization watermarking scheme.

A Novel STDM Watermarking Using Visual Saliency-Based JND Model

The just noticeable distortion (JND) model plays an important role in measuring the visual visibility for spread transform dither modulation (STDM) watermarking. However, the existing JND model characterizes the suprathreshold distortions with an equal saliency level. Visual saliency (VS) has been widely studied by psychologists and computer scientists during the last decade, where the distortions are more likely to be noticeable to any viewer. With this consideration, we proposed a novel STDM watermarking method for a monochrome image by exploiting a visual saliency-based JND model. In our proposed JND model, a simple VS model is employed as a feature to reflect the importance of a local region and compute the final JND map. Extensive experiments performed on the classic image databases demonstrate that the proposed watermarking scheme works better in terms of the robustness than other related methods.

A fast and HEVC-compatible perceptual video coding scheme using a transform-domain Multi-Channel JND model

Coding optimization methods incorporating the just noticeable distortion (JND) model, called perceptual video coding (PVC), have drawn much attention in recent years for better video coding performance. To further remove perceptual redundancy in every channel and improve the coding performance, this paper proposes a fast PVC scheme in the latest High Efficiency Video Coding (HEVC) framework based on our proposed variable block-size transform-domain multi-channel JND model. Firstly, through extensive experiments, we find out for the first time that the contrast masking (CM) effects for chroma channels show a lowpass property in frequency, which differs from the luma channel that has a bypass property. Based on this observation, CM effects in chroma blue (Cb) and chroma red (Cr) channels are modeled as a continuous function for variable-sized blocks, respectively. Secondly, since different characteristics of the human visual system (HVS) exhibit quite distinct effects in luma and chroma channels and effects in chroma channels were not well explored, we develop a new JND model through comprehensive consideration for both luma and chroma channels of five typical HVS effects, with especial focus on parameterized modeling of chroma channels in each effect. Finally, to incorporate the proposed JND model into the latest HEVC coding framework, a multi-channel coefficients suppression method based on JND thresholds and quantization parameter (QP) ranges is proposed in the transform and quantization process, which can decrease the computational complexity. Extensive experimental results show that the proposed PVC scheme implemented in HEVC reference software (HM15.0) can yields significant bit saving of up to 25.91% and on average 9.42% with similar subjective quality, compared to HM15.0, and consistently outperforms two PVC schemes with much reduced bitrate and complexity overhead.

Just Noticeable Distortion Profile for Flat-Shaded 3D Mesh Surfaces.

It is common that a 3D mesh undergoes some lossy operations (e.g., compression, watermarking and transmission through noisy channels), which can introduce geometric distortions as a change in vertex position. In most cases the end users of 3D meshes are human beings; therefore, it is important to evaluate the visibility of introduced vertex displacement. In this paper we present a model for computing a Just Noticeable Distortion (JND) profile for flat-shaded 3D meshes. The proposed model is based on an experimental study of the properties of the human visual system while observing a flat-shaded 3D mesh surface, in particular the contrast sensitivity function and contrast masking. We first define appropriate local perceptual properties on 3D meshes. We then detail the results of a series of psychophysical experiments where we have measured the threshold needed for a human observer to detect the change in vertex position. These results allow us to compute the JND profile for flat-shaded 3D meshes. The proposed JND model has been evaluated via a subjective experiment, and applied to guide 3D mesh simplification as well as to determine the optimal vertex coordinates quantization level for a 3D model.

Haar‐Wavelet‐Based Just Noticeable Distortion Model for Transparent Watermark

Watermark transparency is required mainly for copyright protection. Based on the characteristics of human visual system, the just noticeable distortion (JND) can be used to verify the transparency requirement. More specifically, any watermarks whose intensities are less than the JND values of an image can be added without degrading the visual quality. It takes extensive experimentations for an appropriate JND model. Motivated by the texture masking effect and the spatial masking effect, which are key factors of JND, Chou and Li (1995) proposed the well‐known full‐band JND model for the transparent watermark applications. In this paper, we propose a novel JND model based on discrete wavelet transform. Experimental results show that the performance of the proposed JND model is comparable to that of the full‐band JND model. However, it has the advantage of saving a lot of computation time; the speed is about 6 times faster than that of the full‐band JND model.