Just-noticeable Difference Profile Research Articles

Content adaptive JND profile by leveraging HVS inspired channel modeling and perception oriented energy allocation optimization

The existing just noticeable difference (JND) models consider the effects of various covariates, however, they rarely account for the fusion relationship between the covariates, i.e., they lack a holistic understanding of the mechanisms of visual perception and disregarding the significant impact of energy consumption on visual perception. In fact, visual perception is no exception to the rule that nerve activities and energy supply are inextricably linked. Based on this insight, this paper proposes a novel JND estimation model employing content-adaptive energy allocation. Primarily, the information theory is applied to the visual perception system by conceptualizing human visual system (HVS) as an information communication framework. Then, leveraging the relationship between energy consumption and information perception, this paper quantitatively measures the HVS energy consumption as uniform metric to describe the complicated and heterogeneous HVS perception process, and then construct JND model by fusing low-level and Semantic-level features. Numerous simulation results verify that the proposed JND model is significantly competitive with other frontier models and highly compatible with HVS.

Spatiotemporal just noticeable difference modeling with heterogeneous temporal visual features

Developing accurate Just-noticeable difference (JND) models are challenged by complicated HVS characteristics and nonstationary features of video sequence. Great efforts have been devoted to JND modeling, and inspiring performance improvements are witnessed in the literature, especially spatial JND models. However, there are not only urgent requirement but also technical potentiality for improving temporal JND models fully accounting for the temporal perception characteristics. In terms of temporal JND modeling, there are two challenges, one is how to extract perceptual feature parameters of source video, and the other is how to quantitatively characterize the interaction relationship between feature parameters and HVS characteristics? Firstly, this work extracts content-aware temporal feature parameters having predominate impacts on vision perception, including motion (foreground/background), pixel-correspondence duration and inter-frame residue fluctuation intensity along temporal trajectory, and investigates the HVS responses to these four heterogeneous feature parameters. Secondly, this work proposes respective probability density functions (PDF) in the perception sense to quantitatively depict the attention and suppression perception responses of feature parameters, accounting for the temporal perception characteristics. Using these PDF models, we fuse the heterogeneous feature parameters from the viewpoint of uniform dimension,i.e. self-information measured visual attention and information entropy measured masking uncertainty, achieving heterogeneous parameter homogenization. Thirdly, with self-information and entropy results, this work then proposes a temporal weight model, by striking the balance between visual attention and masking suppression, to adjust the spatial JND threshold, and then develops the improved spatiotemporal JND model. Intensive simulation results verity the effectiveness of the proposed spatiotemporal JND profile, with competitive model accuracy compared with the-state-of-the-art candidate models.

Robust Image Watermarking Based on Two-Layer Visual Saliency-Induced JND Profile

Visual saliency (VS) is an important mechanism for defining which areas of an image will attract more attention of the HVS. Thus, VS can be employed to weight the just noticeable difference (JND) with different attention levels. Some VS-based JND profiles have been proposed in the DCT domain, which used the bottom-up features, such as luminance and texture only. Recently, the research about saliency detection has shown that a better saliency model considering both bottom-up features and top-down features will lead to a significant improvement for the overall saliency detection performance. In this paper, we propose a novel two-layer VS-induced JND profile, which is composed of the bottom-up features and the top-down feature extracted from DCT blocks in the transformed domain. In this model, the luminance and texture features are adapted to calculate the bottom-up features maps, while the top-down feature of focus is used to guide the generation of final salient regions since the camerapersons are used to facilitate the attention regions in focus. The proposed two-layer saliency-induced JND model is further applied to modulate the quantization step in the watermarking framework, which can make full use of its individual merits to achieve a better tradeoff between fidelity and robustness. The experimental results show that the proposed scheme has superior performance than the previous watermarking schemes.

Visual Attention Guided Pixel-Wise Just Noticeable Difference Model

The just noticeable difference (JND) models in pixel domain are generally composed of luminance adaptation (LA) and contrast masking (CM), which takes edge masking (EM) and texture masking (TM) into consideration. However, in existing pixel-wise JND models, CM is not evaluated appropriately since they overestimate the masking effect of regular oriented texture regions and neglect the visual attention characteristic of human eyes for the real image. In this work, a novel JND model in pixel domain is proposed, where orderly texture masking (OTM) for regular texture areas (also called orderly texture regions) and disorderly texture masking (DTM) for complex texture areas (also called disorderly texture regions) are presented based on the orientation complexity. Meanwhile, the visual saliency is set as the weighting factor and is incorporated into CM evaluation to enhance JND thresholds. Experimental results indicate that compared with existing relevant JND profiles, the proposed JND model tolerates more distortion in the same perceptual quality, and brings better visual perception in the same level of the injected JND-noise energy.

A DCT-Based Total JND Profile for Spatiotemporal and Foveated Masking Effects

In image and video processing fields, Discrete Cosine Transform (DCT)-based just-noticeable difference (JND) profiles have effectively been utilized to remove perceptual redundancies in pictures for compression. In this paper, we solve two problems that are often intrinsic to the conventional DCT-based JND profiles: 1) no foveated masking (FM) JND model has been incorporated in modeling the DCT-based JND profiles and 2) the conventional temporal masking (TM) JND models assume that all moving objects in frames can be well tracked by the eyes and that they are projected on the fovea regions of the eyes, which is not a realistic assumption and may result in poor estimation of JND values for untracked moving objects (or image regions). To solve these two problems, we first propose a generalized JND model for joint effects between TM and FM effects. With this model, called the temporal–foveated masking (TFM) JND model, JND thresholds for any tracked/untracked and moving/still image regions can be elaborately estimated. Finally, the TFM-JND model is incorporated into a total DCT-based JND profile with a spatial contrast sensitivity function, luminance masking, and contrast masking JND models. In addition, we propose a JND adjustment method for our total JND profile to avoid overestimation of JND values for image blocks of fixed sizes with various image characteristics. To validate the effectiveness of the total JND profile, an experiment involving a subjective distortion-visibility assessment has been conducted. The experiment results show that the proposed total DCT-based JND profile yields significant performance improvement with much higher capability of distortion concealment (average 5.6-dB lower PSNR) compared with state-of-the-art JND profiles. The MATLAB source code of the proposed total DCT-based JND profile is publicly available online at https://sites.google.com/site/sunghobaecv/jnd .

Just Noticeable Difference Estimation for Screen Content Images.

We propose a novel just noticeable difference (JND) model for a screen content image (SCI). The distinct properties of the SCI result in different behaviors of the human visual system when viewing the textual content, which motivate us to employ a local parametric edge model with an adaptive representation of the edge profile in JND modeling. In particular, we decompose each edge profile into its luminance, contrast, and structure, and then evaluate the visibility threshold in different ways. The edge luminance adaptation, contrast masking, and structural distortion sensitivity are studied in subjective experiments, and the final JND model is established based on the edge profile reconstruction with tolerable variations. Extensive experiments are conducted to verify the proposed JND model, which confirm that it is accurate in predicting the JND profile, and outperforms the state-of-the-art schemes in terms of the distortion masking ability. Furthermore, we explore the applicability of the proposed JND model in the scenario of perceptually lossless SCI compression, and experimental results show that the proposed scheme can outperform the conventional JND guided compression schemes by providing better visual quality at the same coding bits.

Optimizing the Perceptual Quality of Real-Time Multimedia Applications

The perceptual quality of a real-time multimedia system depends on a combination of performance metrics that are affected by run-time control inputs. As perceptual quality is subjective in nature and lacks a well-defined model with respect to control inputs, designing a multimedia system to achieve high perceptual quality requires a black-box approach to tune control inputs according to preferences from subjective tests. This article presents a systematic method, called OptPQ, for finding operating points that achieve good perceptual quality using an offline measured just-noticeable difference (JND) profile to capture human awareness of the adjustment of control inputs. It also presents an online method for combining multiple independent JND profiles to find the best control inputs.

A novel generalized DCT-based JND profile based on an elaborate CM-JND model for variable block-sized transforms in monochrome images.

In this paper, we propose a new DCT-based just noticeable difference (JND) profile incorporating the spatial contrast sensitivity function, the luminance adaptation effect, and the contrast masking (CM) effect. The proposed JND profile overcomes two limitations of conventional JND profiles: 1) the CM JND models in the conventional JND profiles employed simple texture complexity metrics, which are not often highly correlated with perceived complexity, especially for unstructured patterns. So, we proposed a new texture complexity metric that considers not only contrast intensity, but also structureness of image patterns, called the structural contrast index. We also newly found out that, as the structural contrast index of a background texture pattern increases, the modulation factors for CM-JND show a bandpass property in frequency. Based on this observation, a new CM-JND is modeled as a function of DCT frequency and the proposed structural contrast index, showing significantly high correlations with measured CM-JND values and 2) while the conventional DCT-based JND profiles are only applicable for specific transform block sizes, our proposed DCT-based JND profile is first designed to be applicable to any size of transform by deriving a new summation effect function, which can also be appropriately applied for quad-tree transform of high efficiency video coding. For the overall performance, the proposed DCT-based JND profile shows more tolerance for distortions with better perceptual quality than other JND profiles under comparison.

Timbre discrimination in cochlear implant users and normal hearing subjects using cross-faded synthetic tones

The identification and discrimination of timbre are essential features of music perception in cochlear implant users. As timbre differences appear as multidimensional cues, the spectral shape, the spectral fluctuation, and the rise time are the most dominating parameters of timbre in normal hearing listeners.Recently, a psychoacoustical test was developed to determine the timbre discrimination abilities using only the spectral shape difference as a cue. Therefore, a synthetically generated tone continuum was used in an adaptive alternative forced choice paradigm. The spectral shape was modified by cross-fading the tones adaptively, depending on the listeners’ response which allows very precise determinations of the just noticeable difference (JND). With this behavioral test, the spectral shape JND for complex tones with different fundamental frequencies was measured in cochlear implant users and compared to normal hearing listeners.The results confirm the applicability of the test to measure timbre discrimination in cochlear implant users. The resulting individual spectral shape JND profiles reveal a maximum with a fundamental frequency of 525Hz, whereas the JND profiles were rather flat in the normal hearing individuals.

Wavelet-based image watermarking with visibility range estimation based on HVS and neural networks

This work proposes a wavelet-based image watermarking (WIW) technique, based on the human visible system (HVS) model and neural networks, for image copyright protection. A characteristic of the HVS, which is called the just noticeable difference (JND) profile, is employed in the watermark embedding to enhance the imperceptibility of the technique. First, we derive the allowable visibility ranges of the JND thresholds for all coefficients of a wavelet-transformed image. The WIW technique exploits the ranges to compute the adaptive strengths to be superimposed in the wavelet coefficients while embedding watermarks. An artificial neural network (ANN) is then used to memorize the relationships between the original wavelet coefficients and its watermark version. Consequently, the trained ANN is utilized for estimating the watermark without the original image. Many existing schemes require the original image to be involved in the calculation of the JND profile of the image. Finally, computer simulations demonstrate that both transparency and robustness of the WIW technique are superior to that of other proposed methods.