Contrast Masking Research Articles

A Hybrid Contrast and Texture Masking Model to Boost High Efficiency Video Coding Perceptual Rate-Distortion Performance

A Hybrid Contrast and Texture Masking Model to Boost High Efficiency Video Coding Perceptual Rate-Distortion Performance

ColorVideoVDP: A visual difference predictor for image, video and display distortions

ColorVideoVDP is a video and image quality metric that models spatial and temporal aspects of vision for both luminance and color. The metric is built on novel psychophysical models of chromatic spatiotemporal contrast sensitivity and cross-channel contrast masking. It accounts for the viewing conditions, geometric, and photometric characteristics of the display. It was trained to predict common video-streaming distortions (e.g., video compression, rescaling, and transmission errors) and also 8 new distortion types related to AR/VR displays (e.g., light source and waveguide non-uniformities). To address the latter application, we collected our novel XR-Display-Artifact-Video quality dataset (XR-DAVID), comprised of 336 distorted videos. Extensive testing on XR-DAVID, as well as several datasets from the literature, indicate a significant gain in prediction performance compared to existing metrics. ColorVideoVDP opens the doors to many novel applications that require the joint automated spatiotemporal assessment of luminance and color distortions, including video streaming, display specification, and design, visual comparison of results, and perceptually-guided quality optimization. The code for the metric can be found at https://github.com/gfxdisp/ColorVideoVDP.

A Ray Tracing Model for Electron Optical Imaging in Electron Beam Powder Bed Fusion

The recent success of the process monitoring method Electron Optical Imaging, applied in the additive manufacturing process Electron Beam Powder Bed Fusion, necessitates a clear understanding of the underlying image formation process. Newly developed multi-detector systems enable the reconstruction of the build surface topography in-situ but add complexity to the method. This work presents a physically based raytracing model, which rationalises the effect of detector positioning on image contrast development and masking. The model correctly describes the effect of multiple scattering events on vacuum chamber walls or heat shields and represents, therefore, a predictive tool for designing future detector systems. Most importantly, this work provides a validated method to compute build surface height gradients directly from experimentally recorded electron-optical images of a multi-detector system without any calibration steps. The computed surface height gradients can be used subsequently as input of normal integration algorithms aiming at the in-situ reconstruction of the build surface topography.

Just Noticeable Difference Model for Images with Color Sensitivity.

The just noticeable difference (JND) model reflects the visibility limitations of the human visual system (HVS), which plays an important role in perceptual image/video processing and is commonly applied to perceptual redundancy removal. However, existing JND models are usually constructed by treating the color components of three channels equally, and their estimation of the masking effect is inadequate. In this paper, we introduce visual saliency and color sensitivity modulation to improve the JND model. Firstly, we comprehensively combined contrast masking, pattern masking, and edge protection to estimate the masking effect. Then, the visual saliency of HVS was taken into account to adaptively modulate the masking effect. Finally, we built color sensitivity modulation according to the perceptual sensitivities of HVS, to adjust the sub-JND thresholds of Y, Cb, and Cr components. Thus, the color-sensitivity-based JND model (CSJND) was constructed. Extensive experiments and subjective tests were conducted to verify the effectiveness of the CSJND model. We found that consistency between the CSJND model and HVS was better than existing state-of-the-art JND models.

Towards perceptual image watermarking with robust texture measurement

Improving the perceptual quality of images is challenging for the robust watermarking method. To address this problem, most existing algorithms proposed to guide the watermark embedding based on just noticeable difference (JND) model, which mainly consists of corresponding contrast masking factor. However, existing metrics are mostly based on the intra-block texture energy, which cannot exactly describe the texture information and meet the needs of watermarking robustness. In this paper, we present a new texture metric, which incorporates the visual saliency based inter-block texture regularity with the robust discrete cosine transform (DCT) coefficients. The JND model is constructed as a modulation factor by using the proposed robust texture metric, which embeds the intra-block energy and inter-block regularity measurement to capture local and global features. Based on the proposed model, we introduce an adaptive quantization step for the watermarking framework. Both quantitative and qualitative results show that our scheme achieves better performance in comparison with several state-of-the-art schemes.

Image quality assessment based on the perceived structural similarity index of an image.

Image quality assessment (IQA) has a very important role and wide applications in image acquisition, storage, transmission and processing. In designing IQA models, human visual system (HVS) characteristics introduced play an important role in improving their performances. In this paper, combining image distortion characteristics with HVS characteristics, based on the structure similarity index (SSIM) model, a novel IQA model based on the perceived structure similarity index (PSIM) of image is proposed. In the method, first, a perception model for HVS perceiving real images is proposed, combining the contrast sensitivity, frequency sensitivity, luminance nonlinearity and masking characteristics of HVS; then, in order to simulate HVS perceiving real image, the real images are processed with the proposed perception model, to eliminate their visual redundancy, thus, the perceived images are obtained; finally, based on the idea and modeling method of SSIM, combining with the features of perceived image, a novel IQA model, namely PSIM, is proposed. Further, in order to illustrate the performance of PSIM, 5335 distorted images with 41 distortion types in four image databases (TID2013, CSIQ, LIVE and CID) are used to simulate from three aspects: overall IQA of each database, IQA for each distortion type of images, and IQA for special distortion types of images. Further, according to the comprehensive benefit of precision, generalization performance and complexity, their IQA results are compared with those of 12 existing IQA models. The experimental results show that the accuracy (PLCC) of PSIM is 9.91% higher than that of SSIM in four databases, on average; and its performance is better than that of 12 existing IQA models. Synthesizing experimental results and theoretical analysis, it is showed that the proposed PSIM model is an effective and excellent IQA model.

A contrast masking investigation of color induction

A contrast masking investigation of color induction

Cerebral achromatopsia

AbstractChromatic sensitivity was assessed in patients with pregeniculate and postgeniculate lesions. We examine changes in central vision along at foveal fixation and in each of the four paracentral quadrants of the visual field using the Colour Assessment and Diagnosis (CAD) test, a dynamic luminance contrast masking technique that isolates the use of Red / Green (RG) and Yellow / Blue (YB) colour signals. Because the extraction of colour signals in early visual processing involves opponent mechanisms, subjects with congenital RG loss of sensitivity exhibit symmetric increase in thresholds towards the long wavelength (“red”) and middle wavelength (“green”) regions of the spectrum locus. This is also frequently the case with acquired loss of chromatic sensitivity as a result of retinal or optic nerve disease. The findings from this study represent an exception to this rule. The patients investigated showed a marked asymmetry in colour thresholds. The study reveals how discrete, localized damage to visual pathways can cause selective loss of visual function and how the latter varies with retinal topography.The 36 pregeniculate patients investigated exhibited impairment in both RG and YB. Loss of colour vision was symmetrical for both RG and YB discrimination. Some pregeniculate patients exhibited substantial loss of colour vision in areas where perimetric loss was largely absent. All quadrants revealed similar symmetric loss of colour sensitivity.In 23 patients with postgeniculate lesions, patients with pre‐striate damage exhibited symmetric loss of chromatic sensitivity which affected either one or both chromatic mechanisms. Striate or extra‐striate lesions tended to exhibit general loss of colour vision within the area identified by visual field testing. More specific chromatic loss was associated with less impaired areas that were often normal on perimetric testing. Some patients with striate or extra‐striate lesions presented with asymmetric chromatic loss for one or more colour categories. When striate or extra‐striate lesions were also accompanied by underlying pre‐striate damage, chromatic sensitivity loss was always symmetric.The findings from this study show that the loss of chromatic sensitivity in cerebral achromatopsia depends strongly on the exact location and extent of the lesion with significant variation over the visual field. The same subject can exhibit loss of chromatic sensitivity that is either RG or YB specific, but also, spatially localized, non‐opponent loss of sensitivity for primary colours.

Robust watermarking based on blur‐guided JND model for macrophotography images

Macrophotography Images (MPIs) have recently emerged as an active topic due to the development of mobile phone camera technology. A large number of MPIs have been rapidly increasing in many rich visual services, such as smartphones or high-definition monitors. MPIs are often composed of sharp macroimage and blur background, which exhibit different perceptual properties that often lead to different just noticeable difference (JND) estimation. Inspired by this, we formulate the blur concealment (BC) as another factor to determine the total masking effect: the interaction is relatively straightforward with a limited masking effect in the sharp regions, and is complicated with a strong masking effect in the blur parts. Furthermore, texture and orientation adaption and color information weighting are separately incorporated into the contrast masking and color masking. Finally, considering both BC and masking effects, a novel robust watermarking framework based on the proposed blur-guided JND model for MPIs, targeting at further improving the MPIs copyright protection performance. Extensive experiments on MP2020 and Blur Detection data sets show that the applicability of the proposed JND model in the scenario of perceptually MPIs watermarking, and our proposed scheme can outperform the state-of-the-art watermarking schemes by providing better robustness performance at the uniform visual quality.

QuatJND: A Robust Quaternion JND Model for Color Image Watermarking

Robust quantization watermarking with perceptual JND model has made a great success for image copyright protection. Generally, either restores each color channel separately or processes the vector representation from three color channels with the traditional monochromatic model. And it cannot make full use of the high correlation among RGB channels. In this paper, we proposed a robust quaternion JND Model for color image watermarking (QuatJND). In contrast to the existing perceptual JND models, the advantage of QuatJND is that it can integrate quaternion representation domain and colorfulness simultaneously, and QuatJND incorporates the pattern guided contrast masking effect in quaternion domain. On the other hand, in order to efficiently utilize the color information, we further develop a robust quantization watermarking framework using the color properties of the quaternion DCT coefficients in QuatJND. And the quantization steps of each quaternion DCT block in the scheme are optimal. Experimental results show that our method has a good performance in term of robustness with better visual quality.

AIVMAF: Automatic Image Quality Estimation Based on Improved VMAF and YOLOv4

The current most widely used way of image quality estimation relies heavily on the subjective assessment, while majority of past objective estimation methods are not satisfactory on accuracy. To solve them and realize unsupervised image quality estimation with high precision, this paper creates a linear way with “Proportional Partition” controlled by horizontal and vertical rates of extracted pixel to get best representations of the image with patching, balance the uneven distribution of image quality in each source image, and offer dynamic compatibility to images with high resolution. Besides, it estimates the image quality automatically with a model trained by current best artificial intelligence (AI) algorithm for target detection YOLOv4 with 1000 images random selected from ImageNet2013 database. The proposal also uses the spirit of joint indices from the current widely used method named Video Multimethod Assessment Fusion (VMAF). But we replace its Visual Information Fidelity (VIF) with Visual Saliency-induced Index (VSI) and add VSI to our target function because of VIF’s dependence on subjective assessment, and also for VSI’s better performance surpassing most recent IQA estimators as TOP3 best model in recent world. Besides, contrast masking is also included by objective function for the KL-divergence to simulate the human visual perception better. A creative “Batch Learning” way is found to address patches for less calculation and faster speed. All source images are pretreated with colour space transformation and normalization to improve descriptiveness of images and reduce the redundant points, and a threshold is devised to formulate suppression mechanisms. The proposed solution is tested to be a good image quality assessor in many aspects such as correctness, consistency, linearity, monotonicity and speed, and performs well on even HD images.

Learning a self‐supervised tone mapping operator via feature contrast masking loss

AbstractHigh Dynamic Range (HDR) content is becoming ubiquitous due to the rapid development of capture technologies. Nevertheless, the dynamic range of common display devices is still limited, therefore tone mapping (TM) remains a key challenge for image visualization. Recent work has demonstrated that neural networks can achieve remarkable performance in this task when compared to traditional methods, however, the quality of the results of these learning‐based methods is limited by the training data. Most existing works use as training set a curated selection of best‐performing results from existing traditional tone mapping operators (often guided by a quality metric), therefore, the quality of newly generated results is fundamentally limited by the performance of such operators. This quality might be even further limited by the pool of HDR content that is used for training. In this work we propose a learning‐based self‐supervised tone mapping operator that is trained at test time specifically for each HDR image and does not need any data labeling. The key novelty of our approach is a carefully designed loss function built upon fundamental knowledge on contrast perception that allows for directly comparing the content in the HDR and tone mapped images. We achieve this goal by reformulating classic VGG feature maps into feature contrast maps that normalize local feature differences by their average magnitude in a local neighborhood, allowing our loss to account for contrast masking effects. We perform extensive ablation studies and exploration of parameters and demonstrate that our solution outperforms existing approaches with a single set of fixed parameters, as confirmed by both objective and subjective metrics.

Glossiness Index of Objects in Halftone Color Images Based on Structure and Appearance Distortion.

This paper proposes an objective glossiness index for objects in halftone color images. In the proposed index, we consider the characteristics of the human visual system (HVS) and associate the image’s structure distortion and statistical information. According to the difference in the number of strategies adopted by the HVS in judging the difference between images, it is divided into single and multi-strategy modeling. In this study, we advocate multiple strategies to determine glossy or non-glossy quality. We assumed that HVS used different visual mechanisms to evaluate glossy and non-glossy objects. For non-glossy images, the image structure dominated, so the HVS tried to use structural information to judge distortion (a strategy based on structural distortion detection). For glossy images, the glossy appearance dominated; thus, the HVS tried to search for the glossiness difference (an appearance-based strategy). Herein, we present an index for glossiness assessment that attempts to explicitly model the structural dissimilarity and appearance distortion. We used the contrast sensitivity function to account for the mechanism of halftone images when viewed by the human eye. We estimated the structure distortion for the first strategy by using local luminance and contrast masking; meanwhile, local statistics changing in the spatial frequency components for skewness and standard deviation were used to estimate the appearance distortion for the second strategy. Experimental results showed that these two mixed-distortion measurement strategies performed well in consistency with the subjective ratings of glossiness in halftone color images.

Rich Structural Index for Stereoscopic Image Quality Assessment.

The human visual system (HVS), affected by viewing distance when perceiving the stereo image information, is of great significance to study of stereoscopic image quality assessment. Many methods of stereoscopic image quality assessment do not have comprehensive consideration for human visual perception characteristics. In accordance with this, we propose a Rich Structural Index (RSI) for Stereoscopic Image objective Quality Assessment (SIQA) method based on multi-scale perception characteristics. To begin with, we put the stereo pair into the image pyramid based on Contrast Sensitivity Function (CSF) to obtain sensitive images of different resolution. Then, we obtain local Luminance and Structural Index (LSI) in a locally adaptive manner on gradient maps which consider the luminance masking and contrast masking. At the same time we use Singular Value Decomposition (SVD) to obtain the Sharpness and Intrinsic Structural Index (SISI) to effectively capture the changes introduced in the image (due to distortion). Meanwhile, considering the disparity edge structures, we use gradient cross-mapping algorithm to obtain Depth Texture Structural Index (DTSI). After that, we apply the standard deviation method for the above results to obtain contrast index of reference and distortion components. Finally, for the loss caused by the randomness of the parameters, we use Support Vector Machine Regression based on Genetic Algorithm (GA-SVR) training to obtain the final quality score. We conducted a comprehensive evaluation with state-of-the-art methods on four open databases. The experimental results show that the proposed method has stable performance and strong competitive advantage.

Perceptual Functioning.

Perceptual disorders are not part of the diagnosis criteria for schizophrenia. Yet, a considerable amount of work has been conducted, especially on visual perception abnormalities, and there is little doubt that visual perception is altered in patients. There are several reasons why such perturbations are of interest in this pathology. They are observed during the prodromal phase of psychosis, they are related to the pathophysiology (clinical disorganization, disorders of the sense of self), and they are associated with neuronal connectivity disorders. Perturbations occur at different levels of processing and likely affect how patients interact and adapt to their surroundings. The literature has become very large, and here we try to summarize different models that have guided the exploration of perception in patients. We also illustrate several lines of research by showing how perception has been investigated and by discussing the interpretation of the results. In addition to discussing domains such as contrast sensitivity, masking, and visual grouping, we develop more recent fields like processing at the level of the retina, and the timing of perception.

FovVideoVDP

FovVideoVDP is a video difference metric that models the spatial, temporal, and peripheral aspects of perception. While many other metrics are available, our work provides the first practical treatment of these three central aspects of vision simultaneously. The complex interplay between spatial and temporal sensitivity across retinal locations is especially important for displays that cover a large field-of-view, such as Virtual and Augmented Reality displays, and associated methods, such as foveated rendering. Our metric is derived from psychophysical studies of the early visual system, which model spatio-temporal contrast sensitivity, cortical magnification and contrast masking. It accounts for physical specification of the display (luminance, size, resolution) and viewing distance. To validate the metric, we collected a novel foveated rendering dataset which captures quality degradation due to sampling and reconstruction. To demonstrate our algorithm's generality, we test it on 3 independent foveated video datasets, and on a large image quality dataset, achieving the best performance across all datasets when compared to the state-of-the-art.

On visual masking estimation for adaptive quantization using steerable filters

A fast and accurate assessment of visual masking effects is desirable while encoding in order to utilize such effects to improve the quality of compressed videos through an adaptive quantization (AQ) scheme. Here, we propose a method of estimating the contrast masking threshold on natural scene patches, using texture cues imparted by steerable filter responses. We then employ the estimated thresholds to perform AQ for AV1 encoding. Our experimental results establish that the proposed method is able to outperform existing visual masking models in terms of estimation performance while being relatively computationally inexpensive than these models, and is also able to improve the variance based AQ algorithm that is currently deployed in the SVT-AV1 codec. Using the multi-scale structural similarity index measure (MS-SSIM) as the quality model, our approach achieves an average BD-rate of -1.82% using the uniform quantization scheme as anchor as compared to 5.83% obtained with the variance based method. We note that the proposed approach produces less visible compression artifacts than the variance based AQ approach at lower bitrates, while maintaining similar encoding complexity.

Blind Photograph Watermarking with Robust Defocus-Based JND Model

Just noticeable distortion (JND) is widely employed to describe the perception redundancy in the quantization-based watermarking framework. However, the existing JND models are generally constructed to treat every region of the photograph with an equal focus level, whereas the defocus effect has never been considered. In this paper, the defocus feature, which can portray the aesthetic emphasis in the photograph, is provided to improve the perceptual JND model. Firstly, two indicators which consider the block energy in the defocus measurement (DM) are proposed. Then, the defocus feature map (DFM) is obtained by integrating the influence of the circumambient blocks, and it is applied to the proposed JND contrast masking (CM) processing. In this way, a new blind photograph watermarking method, with emphasis on defocus-JND estimation combined with the proposed CM, is presented. Simulations show that the proposed JND is more suitable for watermarking framework than some exiting JND models, and the proposed watermarking scheme with the improved defocus-based JND model has superior robustness compared with some watermarking schemes.

An improved DCT-based JND estimation model considering multiple masking effects

The just noticeable distortion (JND) in the contour and orderly regions is easy to be overestimated and that in the disorderly areas is usually underestimated. In order to estimate the JND threshold more accurately, this paper proposes an improved DCT-based JND estimation model considering multiple masking effects properly. The contributions of this paper are characterized by twofold. On the one hand, a mean absolute difference based (MAD-based) block classification method is developed at first to classify image blocks into plain, contour and texture types accurately and quickly. And the JND model for contrast masking effect (CM-JND) is constructed as a modulation factor based on the MAD of each block. On the other hand, we propose a distance-based disorder evaluation metric to measure the disorder intensity in block level. Then, the JND model for the disorderly concealment effect (DC-JND) is proposed based on our psychological experiment. Finally, the total JND estimation threshold is modeled by fusing the spatial contrast sensitivity function, the luminance adaptation effect, the CM and DC effects. Experimental results show that the proposed DCT-based JND estimation model outperforms existing models in performance and complexity. Specifically, the proposed model shows more tolerance for distortions, lower computational complexity with better perceptual quality than other JND models.

A novel attention-guided JND Model for improving robust image watermarking

Just noticeable distortion (JND) and visual attention (VA), which are two widely used mathematical models of human visual system (HVS) that aim to simulate the human brain mechanism, are sufficiently explored and applied to many researches including digital watermarking. The activity of human brain, however, is extremely complex and it can be more limited due to complicated fusion of spatial saliency for image domain. In this paper, we propose a novel VA guided JND model in which we fuse the final attention map from the low-level features by using two laws of Gestalt principle. Firstly, we demonstrate a classic JND model in DCT domain, which consists of spatial contrast sensitivity function (CSF), luminance adaptation (LA) and contrast masking (CM). The foveation effect and orientation feature are considered to obtain the CSF and CM factor. The foveation effect is affected by spatial attention, and the orientation features are modeled for CM effect together with traditional block texture strength through three direction-based AC coefficients in DCT domain. The attention features are integrated with a novel Gestalt principle-based weighting mechanism for the final block-based VA model, which is then used to modulate JND profiles with two non-linear functions. Finally, the proposed VA-guided JND model is incorporated into a logarithmic spread transform dither modulation (L-STDM) watermarking scheme. Experimental results show that the newly proposed algorithm can achieve good performance in term of robustness and get better visual quality.