Just Noticeable Distortion Research Articles

A perceptually temporal adaptive quantization algorithm for HEVC

Adaptive quantization (AQ) proves to be an effective coding tool to improve the performance of video coding. This paper presents a perceptually temporal AQ method to improve the subjective coding performance for High Efficiency Video Coding (HEVC). We first put forward a perceptual quality oriented motion estimation algorithm, which is conducted with a spatial-temporal just noticeable distortion (JND) model. Then one perceptual feature in temporal domain is proposed to develop our AQ method, which can generate different quantization parameter (QP) offsets for each coding unit (CU). The proposed method fully utilizes the temporal and perceptual characteristics of each CU, which can produce more visual-friendly QP offsets distribution. Experiments are conducted on HM16.0 (HEVC reference software), and with SSIM (Structure Similarity Index Metric) as the distortion metric, more than 8.08% and 7.95% rate savings can be obtained for Low-Delay-P (LDP) and Low-Delay-B (LDB) configurations on average, respectively. The subjective quality evaluation demonstrates that the proposed AQ method can achieve comparable visual quality as the HM16.0 while the proposed method can yield remarkable bitrate reductions.

Correction to: A fuzzy rate controller for variable bit rate video using foveated just-noticeable distortion model

In the original publication, the affiliations of the two authors were incorrectly written as “University of Sistan and Bluchestan” when it should be “University of Sistan and Baluchestan.” The correct affiliations are presented in this erratum.

A Novel STDM Watermarking Using Visual Saliency-Based JND Model

The just noticeable distortion (JND) model plays an important role in measuring the visual visibility for spread transform dither modulation (STDM) watermarking. However, the existing JND model characterizes the suprathreshold distortions with an equal saliency level. Visual saliency (VS) has been widely studied by psychologists and computer scientists during the last decade, where the distortions are more likely to be noticeable to any viewer. With this consideration, we proposed a novel STDM watermarking method for a monochrome image by exploiting a visual saliency-based JND model. In our proposed JND model, a simple VS model is employed as a feature to reflect the importance of a local region and compute the final JND map. Extensive experiments performed on the classic image databases demonstrate that the proposed watermarking scheme works better in terms of the robustness than other related methods.

Saliency-Guided Just Noticeable Distortion Estimation Using the Normalized Laplacian Pyramid

The human visual system (HVS), like any other physical system, has limitations. For instance, it is known that the HVS can only sense the content changes that are larger than the so-called just noticeable distortion (JND) threshold. Also, to reduce the computational load on the brain, the visual attention mechanism is deployed such that regions with higher visual saliency are processed with higher priority than other less-salient regions. It is also known that visual saliency has a modulatory effect on JND thresholds. In this letter, we present a novel pixel-wise JND estimation method that considers the interplay between visual saliency and JND thresholds. In the proposed method, the largest JND thresholds of a given image are found such that the perceptual distance between the image and its JND noise-contaminated version is minimized in a perceptual space defined by the coefficients of the image in a normalized Laplacian pyramid. Experimental results indicate that the proposed method outperforms four of the latest JND models for static images.

Medical image watermarking using bit threshold map based on just noticeable distortion in discrete cosine transform

Medical images stored in a hospital system are generally confidential and integrated and require strict security. However, medical information stored on digital medical imaging systems, as well as picture archiving and communication systems (PACS), are vulnerable to attack when the data are transferred over wireless or wired communication networks. To solve this problem, a watermarking algorithm for medical images is proposed using a bit threshold map based on just noticeable distortion (JND) in the discrete cosine transform (DCT) method. The low-frequency component comprises a considerable amount of the signal energy for most images. As a result, it has a crucial effect on the image quality. Therefore, in this paper, the proposed algorithm embeds watermarks based on the low-frequency components of the image, such as the DC coefficient of the DCT. When watermarks are embedded in a low frequency area, the subjective image quality is often degraded. To compensate for the degradation of the imperceptibility of the watermarking system, which results from embedding watermarks in the low-frequency component, this research considers the human visual system. In addition, the embedding strength of the JND value is used to improve the watermarking imperceptibility. We applied the proposed watermarking algorithm to a variety of medical images using a computer simulation. The algorithm's performance was verified using a variety of attacks for eliminating watermarks. The simulation results show that the proposed algorithm robustly provides protection against a variety of possible attacks.

Estimating just‐noticeable distortion for images/videos in pixel domain

Existing pixel-based just noticeable distortion (JND) models only take into account luminance adaptation and texture masking (TM). Similarly, existing discrete cosine transform (DCT) based models do not take into account foveal vision effects and do not estimate TM efficiently. As human visual system (HVS) is not sensitive to distortion below the JND threshold, estimation of the perceptual visibility threshold is widely used in digital and video processing applications. The authors propose a comprehensive and efficient pixel-based JND model incorporating all major factors which contribute to the JND estimation. The evaluation of contrast masking (CM) is done by distinguishing the edge and TM with respect to the entropy masking properties of the HVS. Similarly, the foveal vision effects are also taken into account for the comprehensive estimation of contrast sensitivity function (CSF). Hence, the proposed pixel-based JND model incorporates the spatio-temporal CSF, foveal vision effects, influence of eye-movement, luminance adaptation and CM to be more consistent with human perception. The incorporation of these important factors makes the proposed model the most comprehensive and efficient in the current literature. Psychophysical experiments were performed to test the proposed model. The results show the proposed model comprehensively outperforms other existing models proving its efficiency.

A fast and HEVC-compatible perceptual video coding scheme using a transform-domain Multi-Channel JND model

Coding optimization methods incorporating the just noticeable distortion (JND) model, called perceptual video coding (PVC), have drawn much attention in recent years for better video coding performance. To further remove perceptual redundancy in every channel and improve the coding performance, this paper proposes a fast PVC scheme in the latest High Efficiency Video Coding (HEVC) framework based on our proposed variable block-size transform-domain multi-channel JND model. Firstly, through extensive experiments, we find out for the first time that the contrast masking (CM) effects for chroma channels show a lowpass property in frequency, which differs from the luma channel that has a bypass property. Based on this observation, CM effects in chroma blue (Cb) and chroma red (Cr) channels are modeled as a continuous function for variable-sized blocks, respectively. Secondly, since different characteristics of the human visual system (HVS) exhibit quite distinct effects in luma and chroma channels and effects in chroma channels were not well explored, we develop a new JND model through comprehensive consideration for both luma and chroma channels of five typical HVS effects, with especial focus on parameterized modeling of chroma channels in each effect. Finally, to incorporate the proposed JND model into the latest HEVC coding framework, a multi-channel coefficients suppression method based on JND thresholds and quantization parameter (QP) ranges is proposed in the transform and quantization process, which can decrease the computational complexity. Extensive experimental results show that the proposed PVC scheme implemented in HEVC reference software (HM15.0) can yields significant bit saving of up to 25.91% and on average 9.42% with similar subjective quality, compared to HM15.0, and consistently outperforms two PVC schemes with much reduced bitrate and complexity overhead.

Perceptual stereoscopic video coding using disparity just-noticeable-distortion model

In this paper, we propose perceptual stereoscopic video coding using a disparity just-noticeable-distortion (JND) model. We obtain the disparity JND model in stereo videos by disparity masking effects of the human visual system (HVS). The disparity JND model represents the maximum distortion of stereo perception that HVS cannot perceive. Based on the disparity JND model, we adjust prediction residuals to remove the perceptual redundancy of stereo videos. Thus, we achieve significant bit-rate saving while maintaining visual quality. Experimental results demonstrate that the proposed method significantly improves coding efficiency without loss of stereoscopic perceptual quality.

Statistical Evaluation of No-Reference Image Quality Assessment Metrics for Remote Sensing Images

Image quality assessment plays an important role in image processing applications. In many image applications, e.g., image denoising, deblurring, and fusion, a reference image is rarely available for comparison with the enhanced image. Thus, the quality of enhanced images must be evaluated blindly without references. In recent years, many no-reference image quality metrics (IQMs) have been proposed for assessing digital image quality. In this paper, we first review 21 commonly employed no-reference IQMs. Second, we apply these measures to Quickbird images with three different types of general content (urban, rural, and harbor) subjected to three types of degradation (average filtering, Gaussian white noise, and linear motion degradation), each with 40 degradation levels. We evaluate the robustness of the IQMs based on the criteria of prediction accuracy, prediction monotonicity, and prediction consistency. Then, we perform factor analysis on those IQMs deemed robust, and cluster them into several components. We then select the IQM with the highest loading coefficient as the representative IQM for that component. Experimental results suggest that different measures perform differently for images with different contents and subjected to different types of degradation. Generally, the degradation method has a stronger effect than the image content on the evaluation results of an IQM. The same IQM can provide opposite dependences on the level of degradation for different degradation types, and an IQM that performed well with one type of degradation may not perform well with another type. The training-based measures are not appropriate for remote sensing images because the results are highly dependent on the samples employed for training. Only seven of the 21 IQMs were found to fulfill the requirements of robustness. Edge intensity (EI) and just noticeable distortion (JND) are suggested for evaluating the quality of images subjected to average filter degradation. EI, blind image quality assessment through anisotropy (BIQAA), and mean metric (MM) are suggested for evaluating the quality of images subjected to Gaussian white noise degradation. Laplacian derivative (LD), JND, and standard deviation (SD) are suggested for evaluating the quality of images subjected to linear motion. Finally, EI is suggested for evaluating the quality of an image subjected to an unknown type of degradation.

A New Perceptual Mapping Model Using Lifting Wavelet Transform

Perceptual mappingapproaches have been widely used in visual information processing in multimedia and internet of things (IOT) applications. Accumulative Lifting Difference (ALD) is proposed in this paper as texture mapping model based on low-complexity lifting wavelet transform, and combined with luminance masking for creating an efficient perceptual mapping model to estimate Just Noticeable Distortion (JND) in digital images. In addition to low complexity operations, experiments results show that the proposed modelcan tolerate much more JND noise than models proposed before

New adaptive filters as perceptual preprocessing for rate-quality performance optimization of video coding

In this paper, we introduce two perceptual filters as pre-processing techniques to reduce the bitrate of compressed high-definition (HD) video sequences at constant visual quality. The goal of these perceptual filters is to remove spurious noise and insignificant details from the original video prior to encoding. The proposed perceptual filters rely on two novel adaptive filters (called BilAWA and TBil) which combine the good properties of the bilateral and Adaptive Weighting Average (AWA) filters. The bilateral and AWA filters being initially dedicated to denoising, the behaviour of the proposed BilAWA and TBil adaptive filters is first analyzed in the context of noise removal on HD test images. The first set of experimental results demonstrates their effectiveness in terms of noise removal while preserving image sharpness. A just noticeable distortion (JND) model is then introduced in the novel BilAWA and TBil filters to adaptively control the strength of the filtering process, taking into account the human visual sensitivity to signal distortion. Visual details which cannot be perceived are smoothed, hence saving bitrate without compromising perceived quality. A thorough experimental analysis of the perceptual JND-guided filters is conducted when using these filters as a pre-processing step prior to MPEG-4/AVC encoding. Psychovisual evaluation tests show that the proposed BilAWA pre-processing filter leads to an average bitrate saving of about 19.3% (up to 28.7%) for the same perceived visual quality. The proposed new pre-filtering approach has been also tested with the new state-of-the-art HEVC standard and has given similar efficiency in terms of bitrate savings for constant visual quality.

Perception oriented transmission estimation for high quality image dehazing

Single image dehazing has captured much attention due to increasing applications. However, state-of-the-art image dehazing algorithms often suffer from undesirable quantization artifacts and noises in heavily hazy regions or sky patches of hazy image where dense scattering often occurs, so that dehazed results may have poor image quality or even lose the original spectral or structural information. To address this problem, we propose a perception oriented transmission estimation method for high quality image dehazing. As the key contribution, a novel transmission model is firstly developed by posing image dehazing as a local contrast optimization problem. This transmission model can flexibly adjust haze removal to accommodate the expected local contrast gain. Specially, this model can lead to a solution which is similar to the one using the dark channel prior, but it is not confined to the dark channel prior assumption. Then, in order to remove haze and simultaneously suppress quantization artifacts and noises, two specific steps are introduced. First, we develop a scattering-aware method via a Bayesian framework to estimate the scattering probability of each pixel in a hazy image. Second, a perceptually adaptive parameter selection scheme is proposed to determine the expected contrast gain for the transmission estimation by taking advantage of the just-noticeable-distortion (JND) model. Experimental results demonstrate that the proposed algorithm can effectively remove haze and suppress undesirable degradation on dehazed images, both quantitatively and qualitatively, when compared with the state-of-the-art algorithms under dense scattering conditions.

Visual Contrast Sensitivity and Discrimination for 3D Meshes and their Applications

AbstractIn this paper, we first introduce an algorithm for estimating the visual contrast on a 3D mesh. We then perform a series of psychophysical experiments to study the effects of contrast sensitivity and contrast discrimination of the human visual system for the task of differentiating between two contrasts on a 3D mesh. The results of these experiments allow us to propose a perceptual model that is able to predict whether a change in local contrast on 3D mesh, induced by a local geometric distortion, is visible or not. Finally, we illustrate the utility of the proposed perceptual model in a number of applications: we compute the Just Noticeable Distortion (JND) profile for smooth‐shaded 3D meshes and use the model to guide mesh processing algorithms.

A saliency-modulated just-noticeable-distortion model with non-linear saliency modulation functions

It is well-known that the human visual system (HVS) cannot sense small variations of visual signals below the so-called just-noticeable distortion (JND) thresholds due to their underlying spatial/temporal masking properties. It is also known that the visual attention mechanism of the human brain can enhance or reduce visual sensitivity. In other words, the visual attention has modulatory effects on JND thresholds. The current knowledge also states that the visual attention is mainly driven by visual saliency in an automatic and involuntary manner. In this paper we present a saliency-modulated JND (SJND) model for static images in the discrete cosine transform (DCT) domain. In the proposed model, the JND thresholds of each block in a given image are elevated by two non-linear modulation functions using the visual saliency of the block. The parameters of the saliency modulation functions are obtained through an optimization framework, which utilizes a state-of-the-art saliency-based objective image quality assessment method. To evaluate the proposed SJND model, two subjective experiments were conducted. The obtained experimental results demonstrated that the proposed method achieves a high accuracy in JND estimation, and also it provides a high distortion-hiding capacity.

Robust blind image watermarking by modulating the mean of partly sign-altered DCT coefficients guided by human visual perception

A novel discrete cosine transform (DCT)-based blind watermarking scheme is proposed to embed binary information into images with substantial improvement in robustness against commonly encountered attacks. This scheme modulates the partly sign-altered (PSA) mean of selected DCT coefficients derived from an 8×8 image block. To enhance the imperceptibility of the watermark, the variation margin of each coefficient is regulated either by referring to a JPEG quantization table or the just noticeable distortion (JND) of the human visual system. In particular, a companding function is employed to emulate the contrast masking effect of human eyes. Compared with other DCT-based methods executed at a peak signal-to-noise ratio of approximately 40dB, the incorporation of the JPEG quantization table into the PSA mean modulation achieves significant improvement in terms of bit error rates for attacks like JPEG & JPEG 2000 compression and noise corruption. The exploitation of JND also offers advantages in cases of median filtering, Gaussian lowpass filtering, and scaling correction.

Blind prediction-based wavelet watermarking

In recent years, wavelet transform has been widely applied in multimedia signal processing and digital watermarking is involved for ensuring security. This study presents a blind wavelet-based watermarking incorporated with the Human Visual System (HVS), which embeds watermarks into detail-subband coefficients. Since imperceptibility is the most significant issue in watermarking, the approximation band is maintained constant, while the detail subbands are modified to carry information. The perceptual embedded weights for all subbands are determined according to the Just Noticeable Distortion (JND) criterion. The strength of the modification is investigated to provide a balanced result between robustness and image quality. In the decoder, the Least-Mean-Square (LMS) is employed to predict the original detail-subband coefficients and then extract the embedded watermarks. As documented in the experimental results, the proposed method provides good robustness and excellent image quality.

A fuzzy rate controller for variable bit rate video using foveated just-noticeable distortion model

In this paper, a rate control algorithm for variable bit rate video applications with buffer constraint is proposed. The proposed algorithm utilizes a fuzzy rate controller and a perceptual quality...

Just Noticeable Distortion Profile for Flat-Shaded 3D Mesh Surfaces.

It is common that a 3D mesh undergoes some lossy operations (e.g., compression, watermarking and transmission through noisy channels), which can introduce geometric distortions as a change in vertex position. In most cases the end users of 3D meshes are human beings; therefore, it is important to evaluate the visibility of introduced vertex displacement. In this paper we present a model for computing a Just Noticeable Distortion (JND) profile for flat-shaded 3D meshes. The proposed model is based on an experimental study of the properties of the human visual system while observing a flat-shaded 3D mesh surface, in particular the contrast sensitivity function and contrast masking. We first define appropriate local perceptual properties on 3D meshes. We then detail the results of a series of psychophysical experiments where we have measured the threshold needed for a human observer to detect the change in vertex position. These results allow us to compute the JND profile for flat-shaded 3D meshes. The proposed JND model has been evaluated via a subjective experiment, and applied to guide 3D mesh simplification as well as to determine the optimal vertex coordinates quantization level for a 3D model.

Improved logarithmic spread transform dither modulation using a robust perceptual model

In the quantization-based watermarking framework, the perceptual just noticeable distortion (JND) model has been widely used to determine the quantization step size, as it can be used for the bette...

Design and Analysis of Efficient Blind Reversible Watermarking Technique

This paper proposes a blind reversible watermarking techniquewhich incorporates the histogram shifting in wavelet domain. The integer wavelet transform is used to transform the image into wavelet domain. The watermark data is embedded into the sub-band coefficients identified by just noticeable distortion (JND)of the transformed host image. To ensure the prohibition of pixel overflow and underflow in spatial domain property inspired pixel adjustment (PIPA) is also employed, finally wavelet-domain statistical quantity histogram shifting and clustering are used for improving robustness and reducing run-time complexity of the extracting process. The experimental results demonstrate predominance of the proposed approach for different host and watermark images. It also demonstrate the efficient recovery of both watermark and host image.