Just Noticeable Difference Model Research Articles

Route recommendation method for frequent passengers in subway based on passenger preference ranking

To provide personalized and precise guidance services to subway passengers, a route recommendation method for frequent subway riders based on passenger preference ranking is proposed. Firstly, a data-driven analysis method of travel preference is proposed to identify the travel preference of frequent passengers in different origin-destinations (ODs) and time periods. Secondly, a route recommendation model of dictionary order preference based on just noticeable difference (JND) is proposed to solve the problem of different passenger perception in route attributes. Thirdly, the model of depth deterministic strategy gradient (DDPG) is proposed to consider dynamic parameters of JND. A case study of Guangzhou Metro is illustrated to verify the accuracy of the route recommendation model. The results show that the individual passenger recommendation in the proposed method is more accurate than the traditional logit model and the JND model without parameter optimization by 30% and 20%. The group recommendation in the proposed method is more accurate than the comparative models by 37% and 11%, respectively. Specifically, the accuracy of this method in group recommendation fluctuates less, and its accuracy variance is much smaller than the traditional logit model. Therefore, the method proposed in this study performs better in reflecting personalized need of passenger travel.

Effective method for low-light image enhancement based on the JND and OCTM models.

Low-light images always suffer from dim overall brightness, low contrast, and low dynamic ranges, thus result in image degradation. In this paper, we propose an effective method for low-light image enhancement based on the just-noticeable-difference (JND) and the optimal contrast-tone mapping (OCTM) models. First, the guided filter decomposes the original images into base and detail images. After this filtering, detail images are processed based on the visual masking model to enhance details effectively. At the same time, the brightness of base images is adjusted based on the JND and OCTM models. Finally, we propose a new method to generate a sequence of artificial images to adjust the brightness of the output, which has a better performance in image detail preservation compared with other single-input algorithms. Experiments have demonstrated that the proposed method not only achieves low-light image enhancement, but also outperforms state-of-the-art methods qualitatively and quantitatively.

Just Noticeable Difference Model for Images with Color Sensitivity.

The just noticeable difference (JND) model reflects the visibility limitations of the human visual system (HVS), which plays an important role in perceptual image/video processing and is commonly applied to perceptual redundancy removal. However, existing JND models are usually constructed by treating the color components of three channels equally, and their estimation of the masking effect is inadequate. In this paper, we introduce visual saliency and color sensitivity modulation to improve the JND model. Firstly, we comprehensively combined contrast masking, pattern masking, and edge protection to estimate the masking effect. Then, the visual saliency of HVS was taken into account to adaptively modulate the masking effect. Finally, we built color sensitivity modulation according to the perceptual sensitivities of HVS, to adjust the sub-JND thresholds of Y, Cb, and Cr components. Thus, the color-sensitivity-based JND model (CSJND) was constructed. Extensive experiments and subjective tests were conducted to verify the effectiveness of the CSJND model. We found that consistency between the CSJND model and HVS was better than existing state-of-the-art JND models.

Towards perceptual image watermarking with robust texture measurement

Improving the perceptual quality of images is challenging for the robust watermarking method. To address this problem, most existing algorithms proposed to guide the watermark embedding based on just noticeable difference (JND) model, which mainly consists of corresponding contrast masking factor. However, existing metrics are mostly based on the intra-block texture energy, which cannot exactly describe the texture information and meet the needs of watermarking robustness. In this paper, we present a new texture metric, which incorporates the visual saliency based inter-block texture regularity with the robust discrete cosine transform (DCT) coefficients. The JND model is constructed as a modulation factor by using the proposed robust texture metric, which embeds the intra-block energy and inter-block regularity measurement to capture local and global features. Based on the proposed model, we introduce an adaptive quantization step for the watermarking framework. Both quantitative and qualitative results show that our scheme achieves better performance in comparison with several state-of-the-art schemes.

Pixel-Domain Just Noticeable Difference Modeling with Heterogeneous Color Features.

With the rapidly emerging user-generated images, perception compression for color image is an inevitable mission. Whilst in existing just noticeable difference (JND) models, color-oriented features are not fully taken into account for coinciding with HVS perception characteristics, such as sensitivity, attention, and masking. To fully imitate the color perception process, we extract color-related feature parameters as local features, including color edge intensity and color complexity, as well as region-wise features, including color area proportion, color distribution position and color distribution dispersion, and inherent feature irrelevant to color content called color perception difference. Then, the potential interaction among them is analyzed and modeled as color contrast intensity. To utilize them, color uncertainty and color saliency are envisaged to emanate from feature integration in the information communication framework. Finally, color and uncertainty saliency models are applied to improve the conventional JND model, taking the masking and attention effect into consideration. Subjective and objective experiments validate the effectiveness of the proposed model, delivering superior noise concealment capacity compared with start-of-the-art works.

Cross-scale feature fusion-based JND estimation for robust image watermarking in quaternion DWT domain

Owing to the rapid development of just-noticeable difference (JND) and the high robustness of discrete wavelet transform (DWT) based watermarking, robust watermarking methods based on JND models have been extensively adopted in the area of image copyright protection. In general, in existing JND model-based methods, transform domain-based features play an important role during the quantization strength measurement. However, most existing methods only use the subband-specific features from the final multi-scale wavelet transform domain, ignoring the feature relevance among subband-specific features with different scales among RGB channels. To address this issue, in this study, we put forward a JND estimation based on cross-scale feature fusion (CFJnd) mechanism in quaternionic DWT domain for robust image watermarking. For the proposed CFJnd, we first design different quaternionic wavelet scales for the JND modeling, which can be used to measure the minimum distortion level of the embedding-specific subband relevance. Furthermore, the cross-scale features among different subbands are adopted to jointly generate the pattern, texture and color measurement, making the final perceptual masking more representative. In addition, we further develop a CFJnd-based robust image watermarking, and the watermark is embedded by quantifying the candidate quaternionic DWT coefficients of the embedding-specific subband on a level which is adaptively defined according to the proposed CFJnd. Experimental results based on the publicly available image database show that the proposed method can produce reliable and promising results, compared to other state-of-the-art image watermarking methods.

Robust watermarking based on blur‐guided JND model for macrophotography images

Macrophotography Images (MPIs) have recently emerged as an active topic due to the development of mobile phone camera technology. A large number of MPIs have been rapidly increasing in many rich visual services, such as smartphones or high-definition monitors. MPIs are often composed of sharp macroimage and blur background, which exhibit different perceptual properties that often lead to different just noticeable difference (JND) estimation. Inspired by this, we formulate the blur concealment (BC) as another factor to determine the total masking effect: the interaction is relatively straightforward with a limited masking effect in the sharp regions, and is complicated with a strong masking effect in the blur parts. Furthermore, texture and orientation adaption and color information weighting are separately incorporated into the contrast masking and color masking. Finally, considering both BC and masking effects, a novel robust watermarking framework based on the proposed blur-guided JND model for MPIs, targeting at further improving the MPIs copyright protection performance. Extensive experiments on MP2020 and Blur Detection data sets show that the applicability of the proposed JND model in the scenario of perceptually MPIs watermarking, and our proposed scheme can outperform the state-of-the-art watermarking schemes by providing better robustness performance at the uniform visual quality.

Feedforward and Feedback Modulations Based Foveated JND Estimation for Images

The just noticeable difference (JND) reveals the key characteristic of visual perception, which has been widely used in many perception-based image and video applications. Nevertheless, the modulatory mechanism of the human visual system (HVS) has not been fully exploited in JND threshold estimation, which results in the existing JND models not being accurate enough. In this article, by analyzing the feedforward and feedback modulatory behaviors in the HVS, an enhanced foveated JND (FJND) estimation model is proposed considering modulatory effects and masking effects in visual perception. The contributions of this article are mainly twofold. On the one hand, by analyzing the modulatory behaviors in the HVS, the modulatory mechanism is incorporated into JND estimation and a hierarchical modulation-based JND estimation framework is proposed for the first time. On the other hand, according to the response characteristics of visual neurons, modulatory effects on visual sensitivity are formulated as several modulatory factors to modulate the estimated JND threshold properly. Compared with existing models, the proposed model is developed in view of not only the masking effects but also the modulatory effects, which makes our model more consistent with the HVS. For different complex input images, experimental results show that the proposed FJND model tolerates more distortion at the same perceptual quality in comparison with other existing models.

Just Noticeable Difference for Deep Machine Vision

As an important perceptual characteristic of the Human Visual System (HVS), the Just Noticeable Difference (JND) has been studied for decades with image and video processing (e.g., perceptual visual signal compression). However, there is little exploration on the existence of JND for the Deep Machine Vision (DMV), although the DMV has made great strides in many machine vision tasks. In this paper, we take an initial attempt, and demonstrate that the DMV has the JND, termed as the DMV-JND. We then propose a JND model for the image classification task in the DMV. It has been discovered that the DMV can tolerate distorted images with average PSNR of only 9.56dB (the lower the better), by generating JND via unsupervised learning with the proposed DMV-JND-NET. In particular, a semantic-guided redundancy assessment strategy is designed to restrain the magnitude and spatial distribution of the DMV-JND. Experimental results on image classification demonstrate that we successfully find the JND for deep machine vision. Our DMV-JND facilitates a possible direction for DMV-oriented image and video compression, watermarking, quality assessment, deep neural network security, and so on.

Primena modela predikcije praga uočljivih razlika u proceni kvaliteta slika sa JPEG kompresijom

Introduction/purpose: The paper presents interesting research related to the performance analysis of the picture-wise just noticeable difference (JND) prediction model and its application in the quality assessment of images with JPEG compression. Methods: The performance analysis of the JND model was conducted in an indirect way by using the publicly available results of subject-rated image datasets with the separation of images into two classes (above and below the threshold of visible differences). In the performance analysis of the JND prediction model and image quality assessment, five image datasets were used, four of which come from the visible wavelength range, and one dataset is intended for remote sensing and surveillance with images from the infrared part of the electromagnetic spectrum. Results: The pap 86 er shows that using a picture-wise JND model, subjective image quality assessment scores can be estimated with better accuracy, leading to significant performance improvements of the traditional peak signal-to-noise ratio (PSNR). The gain achieved by introducing the picture-wise JND model in the objective assessment depends on the chosen dataset and the results of the initial simple to compute PSNR measure, and it was obtained on all five datasets. The mean linear correlation coefficient (for five datasets) between subjective and PSNR objective quality estimates increased from 74% (traditional PSNR) to 90% (picture-wise JND PSNR). Conclusion: Further improvement of the JND-based objective measure can be obtained by improving the picture-wise model of JND prediction.

Toward Top-Down Just Noticeable Difference Estimation of Natural Images.

Just noticeable difference (JND) of natural images refers to the maximum pixel intensity change magnitude that typical human visual system (HVS) cannot perceive. Existing efforts on JND estimation mainly dedicate to modeling the diverse masking effects in either/both spatial or/and frequency domains, and then fusing them into an overall JND estimate. In this work, we turn to a dramatically different way to address this problem with a top-down design philosophy. Instead of explicitly formulating and fusing different masking effects in a bottom-up way, the proposed JND estimation model dedicates to first predicting a critical perceptual lossless (CPL) counterpart of the original image and then calculating the difference map between the original image and the predicted CPL image as the JND map. We conduct subjective experiments to determine the critical points of 500 images and find that the distribution of cumulative normalized KLT coefficient energy values over all 500 images at these critical points can be well characterized by a Weibull distribution. Given a testing image, its corresponding critical point is determined by a simple weighted average scheme where the weights are determined by a fitted Weibull distribution function. The performance of the proposed JND model is evaluated explicitly with direct JND prediction and implicitly with two applications including JND-guided noise injection and JND-guided image compression. Experimental results have demonstrated that our proposed JND model can achieve better performance than several latest JND models. In addition, we also compare the proposed JND model with existing visual difference predicator (VDP) metrics in terms of the capability in distortion detection and discrimination. The results indicate that our JND model also has a good performance in this task. The code of this work are available at https://github.com/Zhentao-Liu/KLT-JND.

Spatiotemporal just noticeable difference modeling with heterogeneous temporal visual features

Developing accurate Just-noticeable difference (JND) models are challenged by complicated HVS characteristics and nonstationary features of video sequence. Great efforts have been devoted to JND modeling, and inspiring performance improvements are witnessed in the literature, especially spatial JND models. However, there are not only urgent requirement but also technical potentiality for improving temporal JND models fully accounting for the temporal perception characteristics. In terms of temporal JND modeling, there are two challenges, one is how to extract perceptual feature parameters of source video, and the other is how to quantitatively characterize the interaction relationship between feature parameters and HVS characteristics? Firstly, this work extracts content-aware temporal feature parameters having predominate impacts on vision perception, including motion (foreground/background), pixel-correspondence duration and inter-frame residue fluctuation intensity along temporal trajectory, and investigates the HVS responses to these four heterogeneous feature parameters. Secondly, this work proposes respective probability density functions (PDF) in the perception sense to quantitatively depict the attention and suppression perception responses of feature parameters, accounting for the temporal perception characteristics. Using these PDF models, we fuse the heterogeneous feature parameters from the viewpoint of uniform dimension,i.e. self-information measured visual attention and information entropy measured masking uncertainty, achieving heterogeneous parameter homogenization. Thirdly, with self-information and entropy results, this work then proposes a temporal weight model, by striking the balance between visual attention and masking suppression, to adjust the spatial JND threshold, and then develops the improved spatiotemporal JND model. Intensive simulation results verity the effectiveness of the proposed spatiotemporal JND profile, with competitive model accuracy compared with the-state-of-the-art candidate models.

Deep Learning Based Just Noticeable Difference and Perceptual Quality Prediction Models for Compressed Video

Human visual system has a limitation of sensitivity in detecting small distortion in an image/video and the minimum perceptual threshold is so called Just Noticeable Difference (JND). JND modelling is challenging since it highly depends on visual contents and perceptual factors are not fully understood. In this paper, we propose deep learning based JND and perceptual quality prediction models, which are able to predict the Satisfied User Ratio (SUR) and Video Wise JND (VWJND) of compressed videos with different resolutions and coding parameters. Firstly, the SUR prediction is modeled as a regression problem that fits deep learning tools. Then, Video Wise Spatial SUR method (VW-SSUR) is proposed to predict the SUR value for compressed video, which mainly considers the spatial distortion. Thirdly, we further propose Video Wise Spatial-Temporal SUR (VW-STSUR) method to improve the SUR prediction accuracy by considering the spatial and temporal information. Two fusion schemes that fuse the spatial and temporal information in quality score level and in feature level, respectively, are investigated. Finally, key factors including key frame and patch selections, cross resolution prediction and complexity are analyzed. Experimental results demonstrate the proposed VW-SSUR method outperforms in both SUR and VWJND prediction as compared with the state-of-the-art schemes. Moreover, the proposed VW-STSUR further improves the accuracy as compared with the VW-SSUR and the conventional JND models, where the mean SUR prediction error is 0.049, and mean VWJND prediction error is 1.69 in quantization parameter and 0.84 dB in peak signal-to-noise ratio.

A Novel Improved Reversible Visible Image Watermarking Algorithm Based on Grad-CAM and JND

With the rapid access convenience of content brought by 5G technology, the integrity protection of content becomes more important. The reversible visible watermarking algorithm has attracted more attention due to its effective content protection. In this paper, a novel improved reversible visible image watermarking scheme based on gradient-weighted class activation mapping (Grad-CAM) and the just noticeable difference (JND) model has been presented. The proposed region of interest (ROI) selection strategy is used to locate the main protected body of images for watermark embedding. Divide the watermark and ROI into nonoverlapping blocks in the same way and then embed the classified two types of watermark blocks into corresponding ROI blocks with the JND model. The optimal bit positions for watermark embedding can be selected adaptively with JND threshold and achieve the tradeoff between the watermark visibility and watermarked image quality. For lossless image recovery and watermark extraction, the recovery information is reversibly hidden into watermarked image. In the experiments, the same process of grayscale images is used to each channel separately for color images watermarking. Besides, there are six aspects in this paper to estimate the proposed scheme; with the comparison to other reversible visible watermarking schemes, experimental results demonstrate the effectiveness of our proposed scheme.

Adaptive CU split prediction and fast mode decision for 3D-HEVC texture coding based on just noticeable difference model

3D-high efficiency video coding (3D-HEVC) is the latest extension of HEVC standard. Through adopting novel coding tools, 3D-HEVC could improve the compression efficiency of multi-view texture video. However, with the great bitrate savings, 3D-HEVC brings the extreme complexity. To reduce the encoding time of texture video, a fast 3D-HEVC method based on just noticeable difference (JND) model is proposed in this paper. The JND model is introduced to analyze the coding tree block (CTB) characteristics of the texture image. Moreover, combining with adaptive coding unit (CU) split prediction and fast mode decision, the proposed scheme could skip some CTBs of texture video at the early stage without a normal coding procedure. Simulation results demonstrate that the proposed scheme can reduce 51.5% encoding runtime on average with negligible loss of subjective quality.

JND-Guided Perceptually Color Image Watermarking in Spatial Domain

Watermarking is an effective solution for copyright protection and forensics tracking via hiding information into the image signal. Recently, spatial-embedding watermarking methods from different transform domain have been proposed rapidly and effectively protect the copyright of the color image. Compared to the individual frequency domain method, it has both advantages of spatial domain and frequency domain. Here, we proposed a novel spatial-embedding watermarking method based on an attended just noticeable difference (JND) model with color complexity, to achieve a good tradeoff between robustness and perceptual quality. In particular, at the spatial embedding level, the direct current (DC) coefficients are selected for embedding and the perceptual JND model is used to guide the amount of pixel modification to improve visual quality. Different from the previous JND model, we proposed an attended JND model that considers color complexity, which is more consistent of the human visual perception model. Compared with the other JND models, the proposed JND model is more suitable for the watermarking framework. Experimental results show that the proposed watermarking scheme has better performance than other existing watermarking schemes. This greatly benefits the practical implementations of the spatial-embedding watermarking methods.

Asymmetric Foveated Just-Noticeable-Difference Model for Images With Visual Field Inhomogeneities

Just noticeable difference (JND) models reveal visibility limitation of human visual system (HVS) and exploit perceptual redundancy in the visual signal. Conventional just noticeable difference (JND) model supposes the visual acuity is homogeneous with or without considerations on the isoeccentric locations. Recent research on vision science indicates that the effect of eccentricity on visual sensitivity is not homogeneous across the visual field (VF). Relevant studies have found visual acuity at the isoeccentric locations is better along the horizontal meridian than along the vertical meridian, i.e., horizontal-vertical anisotropy (HVA). Along the vertical meridian, better performance has been reported for the inferior meridian compared to the superior meridian, i.e., vertical-meridian asymmetry (VMA). These properties of HVS should be taken into account to yield a better JND estimation, for better exploiting the perceptual redundancy. In this paper, we use two hemi-ellipses to represent the iso-acuity contour, which can better express the HVA and VMA effects. Moreover, we design psychophysical experiments under different test conditions, where the results show the hemi-ellipse model performs better when compared to the circular model. Finally, we build an asymmetric foveated JND model using two hemi-ellipses for approximation. When comparing with traditional JND models, experimental results demonstrate the effectiveness of the proposed model.

Deep Learning based Picture-Wise Just Noticeable Distortion Prediction Model for Image Compression.

Picture Wise Just Noticeable Difference (PW-JND), which accounts for the minimum difference of a picture that human visual system can perceive, can be widely used in perception-oriented image and video processing. However, the conventional Just Noticeable Difference (JND) models calculate the JND threshold for each pixel or sub-band separately, which may not reflect the total masking effect of a picture accurately. In this paper, we propose a deep learning based PW-JND prediction model for image compression. Firstly, we formulate the task of predicting PW-JND as a multi-class classification problem, and propose a framework to transform the multi-class classification problem to a binary classification problem solved by just one binary classifier. Secondly, we construct a deep learning based binary classifier named perceptually lossy/lossless predictor which can predict whether an image is perceptually lossy to another or not. Finally, we propose a sliding window based search strategy to predict PW-JND based on the prediction results of the perceptually lossy/lossless predictor. Experimental results show that the mean accuracy of the perceptually lossy/lossless predictor reaches 92%, and the absolute prediction error of the proposed PW-JND model is 0.79 dB on average, which shows the superiority of the proposed PW-JND model to the conventional JND models.

A JND-Based Pixel-Domain Algorithm and Hardware Architecture for Perceptual Image Coding.

This paper presents a hardware efficient pixel-domain just-noticeable difference (JND) model and its hardware architecture implemented on an FPGA. This JND model architecture is further proposed to be part of a low complexity pixel-domain perceptual image coding architecture, which is based on downsampling and predictive coding. The downsampling is performed adaptively on the input image based on regions-of-interest (ROIs) identified by measuring the downsampling distortions against the visibility thresholds given by the JND model. The coding error at any pixel location can be guaranteed to be within the corresponding JND threshold in order to obtain excellent visual quality. Experimental results show the improved accuracy of the proposed JND model in estimating visual redundancies compared with classic JND models published earlier. Compression experiments demonstrate improved rate-distortion performance and visual quality over JPEG-LS as well as reduced compressed bit rates compared with other standard codecs such as JPEG 2000 at the same peak signal-to-perceptible-noise ratio (PSPNR). FPGA synthesis results targeting a mid-range device show very moderate hardware resource requirements and over 100 Megapixel/s throughput of both the JND model and the perceptual encoder.

Survey of visual just noticeable difference estimation

The concept of just noticeable difference (JND), which accounts for the visibility threshold (visual redundancy) of the human visual system, is useful in perception-oriented signal processing systems. In this work, we present a comprehensive review of JND estimation technology. First, the visual mechanism and its corresponding computational modules are illustrated. These include luminance adaptation, contrast masking, pattern masking, and the contrast sensitivity function. Next, the existing pixel domain and subband domain JND models are presented and analyzed. Finally, the challenges associated with JND estimation are discussed.