Just Noticeable Distortion Profile Research Articles

Just Noticeable Distortion Profile Inference: A Patch-Level Structural Visibility Learning Approach.

In this paper, we propose an effective approach to infer the just noticeable distortion (JND) profile based on patch-level structural visibility learning. Instead of pixel-level JND profile estimation, the image patch, which is regarded as the basic processing unit to better correlate with the human perception, can be further decomposed into three conceptually independent components for visibility estimation. In particular, to incorporate the structural degradation into the patch-level JND model, a deep learning-based structural degradation estimation model is trained to approximate the masking of structural visibility. In order to facilitate the learning process, a JND dataset is further established, including 202 pristine images and 7878 distorted images generated by advanced compression algorithms based on the upcoming Versatile Video Coding (VVC) standard. Extensive experimental results further show the superiority of the proposed approach over the state-of-the-art. Our dataset is available at: https://github.com/ShenXuelin-CityU/PWJNDInfer.

A novel attention-guided JND Model for improving robust image watermarking

Just noticeable distortion (JND) and visual attention (VA), which are two widely used mathematical models of human visual system (HVS) that aim to simulate the human brain mechanism, are sufficiently explored and applied to many researches including digital watermarking. The activity of human brain, however, is extremely complex and it can be more limited due to complicated fusion of spatial saliency for image domain. In this paper, we propose a novel VA guided JND model in which we fuse the final attention map from the low-level features by using two laws of Gestalt principle. Firstly, we demonstrate a classic JND model in DCT domain, which consists of spatial contrast sensitivity function (CSF), luminance adaptation (LA) and contrast masking (CM). The foveation effect and orientation feature are considered to obtain the CSF and CM factor. The foveation effect is affected by spatial attention, and the orientation features are modeled for CM effect together with traditional block texture strength through three direction-based AC coefficients in DCT domain. The attention features are integrated with a novel Gestalt principle-based weighting mechanism for the final block-based VA model, which is then used to modulate JND profiles with two non-linear functions. Finally, the proposed VA-guided JND model is incorporated into a logarithmic spread transform dither modulation (L-STDM) watermarking scheme. Experimental results show that the newly proposed algorithm can achieve good performance in term of robustness and get better visual quality.

A spatiotemporal saliency-modulated JND profile applied to video watermarking

The just noticeable distortion (JND) has been considered a suitable solution for controlling the watermark strength and generating robust watermarking schemes with distortions that are below the sensitivity threshold. However, JND assumes the same attention level for all image regions, which does not reflect the behavior of an observer. Recently, several models have utilized the modulatory effect of visual attention over JND to improve the efficiency of watermarking schemes. However, most of them have focused on still images. In this paper, we propose a saliency-modulated JND profile for improving video watermarking schemes. Our method aims to adapt the watermark strength to obtain the most robust possible scheme with an imperceptible watermark. Moreover, it has the advantage of fully exploiting the spatiotemporal properties of video to minimize its perceptual redundancies and achieve low computational complexity. Experimental results show the effectiveness of our proposed method and its contributions to video watermarking process.

Just Noticeable Distortion Profile for Flat-Shaded 3D Mesh Surfaces.

It is common that a 3D mesh undergoes some lossy operations (e.g., compression, watermarking and transmission through noisy channels), which can introduce geometric distortions as a change in vertex position. In most cases the end users of 3D meshes are human beings; therefore, it is important to evaluate the visibility of introduced vertex displacement. In this paper we present a model for computing a Just Noticeable Distortion (JND) profile for flat-shaded 3D meshes. The proposed model is based on an experimental study of the properties of the human visual system while observing a flat-shaded 3D mesh surface, in particular the contrast sensitivity function and contrast masking. We first define appropriate local perceptual properties on 3D meshes. We then detail the results of a series of psychophysical experiments where we have measured the threshold needed for a human observer to detect the change in vertex position. These results allow us to compute the JND profile for flat-shaded 3D meshes. The proposed JND model has been evaluated via a subjective experiment, and applied to guide 3D mesh simplification as well as to determine the optimal vertex coordinates quantization level for a 3D model.

JND-Based Watermark Embedding and GA-Based Watermark Extraction with Fuzzy Inference System for Image Verification

This paper presents an adaptive image-watermarking technique based on just-noticeable distortion (JND) profile and fuzzy inference system (FIS) optimized with genetic algorithm (GA). Here it is referred to as the AIWJFG technique. During watermark embedding, it embeds a watermark into an image by referring the JND profile of the image so as to make the watermark more imperceptible. It employs image features and local statistics in the construction of an FIS, and then exploits the FIS to extract watermarks without original images. In addition, the FIS can be further optimized by a GA to improve its watermark-extraction performance remarkably. Experimental results demonstrate that the AIWJFG technique not only makes the embedded watermarks further imperceptible but also possesses adaptive and robust capabilities to resist on image-manipulation attacks being considered in the paper.

Visual saliency’s modulatory effect on just noticeable distortion profile and its application in image watermarking

Perceptual watermarking should take full advantage of the results from human visual system (HVS) studies. Just noticeable distortion (JND) gives us a way to model the HVS accurately. In this paper, another very important aspect affecting human perception, visual saliency, is introduced to modulate JND model. Based on the visual saliency’s modulatory effect on JND model which incorporates visual attention’s influence on visual sensitivity, the saliency modulated JND profile guided image watermarking scheme is proposed. The saliency modulated JND profile guided watermarking scheme, where the visual sensitivity model combined with visual saliency’s modulatory effect is fully used to determine image-dependent upper bounds on watermark insertion, allows us to provide the maximum strength transparent watermark. Experimental results confirm the improved performance of our saliency modulated JND profile guided watermarking scheme in terms of transparency and robustness. Our watermarking scheme is capable of shaping lower injected-watermark energy onto more sensitive regions and higher energy onto the less perceptually significant regions in the image, which yields better visual quality of the watermarked image. At the same time, the proposed saliency modulated JND profile guided image watermarking scheme is more robust compared to unmodulated JND profile guided image watermarking scheme.

An adaptive steganographic method based on the measurement of just noticeable distortion profile

This paper presents an adaptive steganographic method based on just noticeable distortion (JND) profile measurement. According to the input requirements, our method can produce a higher quality or higher embedding capacity stego-image. In the embedding secret data bits into a target pixel process, four different impact factors are utilized to compute how much information can be embedded and what the stego-pixel value will be. These are difference values that represent the correlation between neighboring pixels, the JND value of the target pixel, a predefined embedding capacity control factor, and the contents of various lengths secret data bits. The proposed method embeds more secret data bits within complex areas and less data bits in smooth areas. The difference between the target pixel and the stego-pixel values is controlled, as far as possible, to less than or equal to the JND value of the target pixel. Thus, the stego-image maintains good imperceptible property. In the extraction phase, the embedded secret data can be extracted from the stego-image without referencing the original image and the JND profile. The experimental results show that our method improves stego-image quality and conspicuously increases the embedding capacity at the same time.

A combined just noticeable distortion model–guided image watermarking

Perceptual watermarking should take full advantage of the results from human visual system (HVS) studies. Just noticeable distortion (JND), which refers to the maximum distortion that the HVS does not perceive, gives us a way to model the HVS accurately. In this paper, another very important aspect affecting human perception, visual saliency, is introduced to modulate JND profile. Based upon visual saliency's modulatory effect on JND profile which incorporates visual attention's influence on visual sensitivity, the saliency modulated JND model guided image watermarking scheme is proposed. Experimental results confirm the improved performance of our saliency modulated JND model. Our saliency modulated JND model is capable of shaping lower injected-watermark energy onto more sensitive regions and higher energy onto the less perceptually significant regions in the image, which yields better visual quality of the watermarked image. At the same time, the proposed saliency modulated JND model guided image watermarking scheme is more robust than unmodulated JND model guided image watermarking scheme while retaining the watermark transparency.

Perceptually tuned subband coder for JPEG

This paper presents a perceptually tuned subband image coder, which selectively operates on the subbands of an image to achieve good compression ratio. The proposed encoder divides an image into 8 × 8 sub-images and subsequently decomposes each sub-image into four subbands. A just noticeable distortion (JND) model is employed to remove all the perceptually insignificant subbands from the sub-images. The remaining subbands are processed by the subband discrete cosine transform (SBDCT). This reduces the computational cost of the SBDCT without introducing perceptible distortion. To further reduce the perceptual redundancy in the image, an image-dependent quantization matrix is computed from the JND profile of the image. Our simulation results indicate that our proposed encoder outperforms the JPEG baseline coder at bit rates from 0.25 to 0.9 bpp.

Spatio-Temporal Just Noticeable Distortion Profile for Grey Scale Image/Video in DCT Domain

In image and video processing field, an effective compression algorithm should remove not only the statistical redundancy information but also the perceptually insignificant component from the pictures. Just-noticeable distortion (JND) profile is an efficient model to represent those perceptual redundancies. Human eyes are usually not sensitive to the distortion below the JND threshold. In this paper, a DCT based JND model for monochrome pictures is proposed. This model incorporates the spatial contrast sensitivity function (CSF), the luminance adaptation effect, and the contrast masking effect based on block classification. Gamma correction is also considered to compensate the original luminance adaptation effect which gives more accurate results. In order to extend the proposed JND profile to video images, the temporal modulation factor is included by incorporating the temporal CSF and the eye movement compensation. Moreover, a psychophysical experiment was designed to parameterize the proposed model. Experimental results show that the proposed model is consistent with the human visual system (HVS). Compared with the other JND profiles, the proposed model can tolerate more distortion and has much better perceptual quality. This model can be easily applied in many related areas, such as compression, watermarking, error protection, perceptual distortion metric, and so on.

Just Noticeable Distortion Model and Its Application in Color Image Watermarking

In this paper, a perceptually adaptive watermarking scheme for color images is proposed in order to achieve robustness and transparency. A new just noticeable distortion (JND) estimator for color images is first designed in the wavelet domain. The key issue of the JND model is to effectively integrate visual masking effects. The estimator is an extension to the perceptual model that is used in image coding for grayscale images. Except for the visual masking effects given coefficient by coefficient by taking into account the luminance content and the texture of grayscale images, the crossed masking effect given by the interaction between luminance and chrominance components and the effect given by the variance within the local region of the target coefficient are investigated such that the visibility threshold for the human visual system (HVS) can be evaluated. In a locally adaptive fashion based on the wavelet decomposition, the estimator applies to all subbands of luminance and chrominance components of color images and is used to measure the visibility of wavelet quantization errors. The subband JND profiles are then incorporated into the proposed color image watermarking scheme. Performance in terms of robustness and transparency of the watermarking scheme is obtained by means of the proposed approach to embed the maximum strength watermark while maintaining the perceptually lossless quality of the watermarked color image. Simulation results show that the proposed scheme with inserting watermarks into luminance and chrominance components is more robust than the existing scheme while retaining the watermark transparency.

Estimating Just-Noticeable Distortion for Video

Just-noticeable distortion (JND), which refers to the maximum distortion that the human visual system (HVS) cannot perceive, plays an important role in perceptual image and video processing. In comparison with JND estimation for images, estimation of the JND profile for video needs to take into account the temporal HVS properties in addition to the spatial properties. In this paper, we develop a spatio-temporal model estimating JND in the discrete cosine transform domain. The proposed model incorporates the spatio-temporal contrast sensitivity function, the influence of eye movements, luminance adaptation, and contrast masking to be more consistent with human perception. It is capable of yielding JNDs for both still images and video with significant motion. The experiments conducted in this study have demonstrated that the JND values estimated for video sequences with moving objects by the model are in line with the HVS perception. The accurate JND estimation of the video towards the actual visibility bounds can be translated into resource savings (e.g., for bandwidth/storage or computation) and performance improvement in video coding and other visual processing tasks (such as perceptual quality evaluation, visual signal restoration/enhancement, watermarking, authentication, and error protection)

Just noticeable distortion model and its applications in video coding

We explore a new perceptually-adaptive video coding (PVC) scheme for hybrid video compression, in order to achieve better perceptual coding quality and operational efficiency. A new just noticeable distortion (JND) estimator for color video is first devised in the image domain. How to efficiently integrate masking effects together is a key issue of JND modelling. We integrate spatial masking factors with the nonlinear additivity model for masking (NAMM). The JND estimator applies to all color components and accounts for the compound impact of luminance masking, texture masking and temporal masking. Extensive subjective viewing confirms that it is capable of determining a more accurate visibility threshold that is close to the actual JND bound in human eyes. Secondly, the image-domain JND profile is incorporated into hybrid video encoding via the JND-adaptive motion estimation and residue filtering process. The scheme works with any prevalent video coding standards and various motion estimation strategies. To demonstrate the effectiveness of the proposed scheme, it has been implemented in the MPEG-2 TM5 coder and demonstrated to achieve average improvement of over 18% in motion estimation efficiency, 0.6 dB in average peak signal-to perceptual-noise ratio (PSPNR) and most remarkably, 0.17 dB in the objective coding quality measure (PSNR) on average. Theoretical explanation is presented for the improvement on the objective coding quality measure. With the JND-based motion estimation and residue filtering process, hybrid video encoding can be more efficient and the use of bits is optimized for visual quality.

A perceptually tuned subband image coder based on the measure of just-noticeable-distortion profile

To represent an image of high perceptual quality with the lowest possible bit rate, an effective image compression algorithm should not only remove the redundancy due to statistical correlation but also the perceptually insignificant components from image signals. In this paper, a perceptually tuned subband image coding scheme is presented, where a just-noticeable distortion (JND) or minimally noticeable distortion (MND) profile is employed to quantify the perceptual redundancy. The JND profile provides each signal being coded with a visibility threshold of distortion, below which reconstruction errors are rendered imperceptible. Based on a perceptual model that incorporates the threshold sensitivities due to background luminance and texture masking effect, the JND profile is estimated from analyzing local properties of image signals. According to the sensitivity of human visual perception to spatial frequencies, the full-band JND/MND profile is decomposed into component JND/MND profiles of different frequency subbands. With these component profiles, perceptually insignificant signals in each subband can be screened out, and significant signals can be properly encoded to meet the visibility threshold. A new quantitative fidelity measure, termed as peak signal-to-perceptible-noise ratio (PSPNR), is proposed to assess the quality of the compressed image by taking the perceptible part of the distortion into account. Simulation results show that near-transparent image coding can be achieved at less than 0.4 b/pixel. As compared to the ISO-JPEG standard, the proposed algorithm can remove more perceptual redundancy from the original image, and the visual quality of the reconstructed image is much more acceptable at low bit rates.