Human Visual Sensitivity Research Articles

Riemannian color difference metric for spatial sinusoidal color variations

Several studies report on the sensitivity of human vision to static spatial sinusoidal achromatic and chromatic contrast variations. However, a Riemannian color difference metric, which includes the spatial and colorimetric properties of sinusoidal gratings, is lacking. Such a metric is important for various applications. Here we report on the development of a new Riemannian metric, for the prediction of detection ellipsoids in color space, for spatial sinusoidal gratings as a function of the grating’s size, spatial frequency, luminance and chromaticity. The metric is based on measurements and models of achromatic and isoluminous chromatic contrast sensitivity functions available in literature, and the Riemannian metric for split fields which we reported earlier. We find adequate agreement with various data sets of experimental achromatic and isoluminous chromatic contrast sensitivity functions and with experimentally determined threshold ellipses of isoluminous chromatic Gabor gratings.

Two adaptive enhancement algorithms for high gray-scale RAW infrared images based on multi-scale fusion and chromatographic remapping

As an important part of unmanned platform, the images taken by vehicle-mounted infrared camera are less affected by night, strong light and bad weather, which are widely used in the field of path planning and decision control in complex scenes. However, infrared images also have some shortcomings, such as high dynamic range, low contrast, high noise and blurred details, which make it difficult to enhance images and detect targets. In order to solve the above problems, two high grayscale infrared image enhancement algorithms were proposed in this paper. The MDDC_HDRII algorithm first proposed the idea of dehazing to solve the overexposure problem of the image, which adopted the method of multi-scale detail enhancement to improve the detail information of the image. PAEA_HDRII algorithm took the sensitivity of human vision as a reference factor, which made the image mostly in blue and red channels and enriched the detailed information of the image.Firstly, a high grayscale infrared image enhancement algorithm based on multi-scale fusion was proposed by combining the idea of debate and exposure fusion. Secondly, to solve the problems of poor visual effect and limited self-adaptation ability in the current pseudo-color ehancement algorithm, a pseudo-color enhancement algorithm for high gray-scale infrared images based on chromatographic remapping was proposed. Thirdly, qualitative and quantitative experiments were carried out by using the published FLIR data set. The results show that the two methods proposed in this paper can improve the overall quality of infrared images more effectively and highlight the details of the image. The objective image quality evaluation indexes are superior to most of the existing mainstream traditional algorithms. Then, two algorithms were used to enhance SAR images, and the experimental results show that this algorithm has good universality. Finally, the images enhanced by the two algorithms in this paper was sent to the YOLOv4 network for target detection, which improved the accuracy of target detection. The MDDC_HDRII algorithm first proposed the idea of dehazing to solve the overexposure problem of the image, which adopted the method of multi-scale detail enhancement to improve the detail information of the image. PAEA_HDRII algorithm took the sensitivity of human vision as a reference factor, which made the image mostly in blue and red channels and enriched the detailed information of the image.The algorithm can be applied in the image enhancement and target detection of vehicle-borne infrared images, SAR images, medical images and industrial film images, and provide reliable technical support for applications in related fields.

Learning GAN-Based Foveated Reconstruction to Recover Perceptually Important Image Features

A foveated image can be entirely reconstructed from a sparse set of samples distributed according to the retinal sensitivity of the human visual system, which rapidly decreases with increasing eccentricity. The use of generative adversarial networks (GANs) has recently been shown to be a promising solution for such a task, as they can successfully hallucinate missing image information. As in the case of other supervised learning approaches, the definition of the loss function and the training strategy heavily influence the quality of the output. In this work,we consider the problem of efficiently guiding the training of foveated reconstruction techniques such that they are more aware of the capabilities and limitations of the human visual system, and thus can reconstruct visually important image features. Our primary goal is to make the training procedure less sensitive to distortions that humans cannot detect and focus on penalizing perceptually important artifacts. Given the nature of GAN-based solutions, we focus on the sensitivity of human vision to hallucination in case of input samples with different densities. We propose psychophysical experiments, a dataset, and a procedure for training foveated image reconstruction. The proposed strategy renders the generator network flexible by penalizing only perceptually important deviations in the output. As a result, the method emphasized the recovery of perceptually important image features. We evaluated our strategy and compared it with alternative solutions by using a newly trained objective metric, a recent foveated video quality metric, and user experiments. Our evaluations revealed significant improvements in the perceived image reconstruction quality compared with the standard GAN-based training approach.

Multi-resolutional human visual perception optimized pathology image progressive coding based on JPEG2000

As one of the most studied areas, telemedicine allows doctors and pathologists to share their intelligence all over the world. To achieve the highest efficiency of remote pathology image browsing, especially for the whole-slide pathology image (WSI) big data, we develop a method to conduct the image coding based on the human visual quality for multiple displaying resolutions. In this paper, we establish a multi-resolutional human visual sensitivity model of Wavelet transform coefficients, based on the JPEG2000 standard and the human visual quality characters. With the analysis of the visual masking effect, we further propose a visibility threshold (VT) model for the human visual sensitivity optimized image coding of the WSIs, which considers multi-resolution and different visual qualities jointly. The proposed image coding methods with multiple quality or resolution layers can also be implemented for progressive image transmission and decoding. Compared with the conventional MSE optimized image coding algorithm, the decoded pathology image quality obtained from certain amounts of coded data using the proposed algorithm was proved to be improved from the aspect of visual perception through both objective and subjective evaluations. The perception-optimized progressive pathology image transmission and decoding were also verified with the combination of the proposed VT model and the JPIP protocol. With the proposed method, a remote medical server can adaptively transmit the image data according to the client’s subjective requirements for the remote browsing of pathology image big data, leading to improved efficiency of diagnosis in telemedicine.

A Multiscale Approach to Deep Blind Image Quality Assessment.

Faithful measurement of perceptual quality is of significant importance to various multimedia applications. By fully utilizing reference images, full-reference image quality assessment (FR-IQA) methods usually achieves better prediction performance. On the other hand, no-reference image quality assessment (NR-IQA), also known as blind image quality assessment (BIQA), which does not consider the reference image, makes it a challenging but important task. Previous NR-IQA methods have focused on spatial measures at the expense of information in the available frequency bands. In this paper, we present a multiscale deep blind image quality assessment method (BIQA, M.D.) with spatial optimal-scale filtering analysis. Motivated by the multi-channel behavior of the human visual system and contrast sensitivity function, we decompose an image into a number of spatial frequency bands by multiscale filtering and extract features for mapping an image to its subjective quality score by applying convolutional neural network. Experimental results show that BIQA, M.D. compares well with existing NR-IQA methods and generalizes well across datasets.

Human visual sensitivity based optimal local backlight dimming methodologies under different viewing conditions

An adaptive local backlight dimming algorithm is proposed considering human visual sensitivities under various viewing conditions. Local dimming amount is maximized based on contrast sensitivity function and local image characteristics to minimize perceived luminance and contrast losses maintaining image qualities. Average luminance, contrast and spatial frequency are evaluated as local image characteristics. As dimming based on local characteristics has blocking artifacts, a compensation method using edge luminance difference is proposed considering human responses. The proposed algorithms reduce backlight power consumption by 39.69% on average, with an average SSIM of 0.995. It can be increased by 5.66% with varying viewing conditions, resulting in a 45.35% power consumption reduction on average. The outperformances of the proposed algorithm in power consumption reduction and maintaining image quality are verified with various image quality metrics compared to recent dimming algorithms.

Educational example of masking textual information in a photographic signal

The paper presents a steganographic method of masking textual information in photo files. Concealment is implemented in Python. The introduction of individual letters of the text into the image is carried out by the method of "least significant bit". The program can be used for both educational and practical purposes. The Pillow graphics library was used to implement the program. This is not a specialized library for steganographic needs. The use of this library makes it possible to visualize the mechanism of hiding information in the lessons, while the conciseness of the program code gives the possibility to demonstrate it in the classroom setting. It is also important for educational purposes, that working within the Pillow library allows you to see the state of an empty and filled container at the level of individual bits. To assess the practical value of the program, it was tested with texts of different lengths and with containers (photographs) of various kinds. The experiment showed the correct reproduction of texts. Careful visual examination of the empty and correspondingly filled containers (photographs) revealed no differences or suspicions of text bookmarks. Of course, if the party who intercepted the masked message has guesses about how the text is hidden, then this text is easily revealed. Therefore, it is obvious, that the use of the program for practical purposes requires additional manipulations in the code, in particular related to the order of implementation of the text and the choice of location.
 It is also desirable to additionally encrypt the text with at least a simple method. Such encryption is possible with the usage of a separate program. Analysis of photographs and manipulation with them at the level of individual bits also has educational value in terms of disclosing a method of capturing the corresponding physical signal. The latter gives an explanation of the methods of encoding static images, noise level, the magnitude of the useful physical signal, and the limits of sensitivity of human vision

Noise-based enhancement for foveated rendering

Human visual sensitivity to spatial details declines towards the periphery. Novel image synthesis techniques, so-called foveated rendering, exploit this observation and reduce the spatial resolution of synthesized images for the periphery, avoiding the synthesis of high-spatial-frequency details that are costly to generate but not perceived by a viewer. However, contemporary techniques do not make a clear distinction between the range of spatial frequencies that must be reproduced and those that can be omitted. For a given eccentricity, there is a range of frequencies that are detectable but not resolvable. While the accurate reproduction of these frequencies is not required, an observer can detect their absence if completely omitted. We use this observation to improve the performance of existing foveated rendering techniques. We demonstrate that this specific range of frequencies can be efficiently replaced with procedural noise whose parameters are carefully tuned to image content and human perception. Consequently, these frequencies do not have to be synthesized during rendering, allowing more aggressive foveation, and they can be replaced by noise generated in a less expensive post-processing step, leading to improved performance of the rendering system. Our main contribution is a perceptually-inspired technique for deriving the parameters of the noise required for the enhancement and its calibration. The method operates on rendering output and runs at rates exceeding 200 FPS at 4K resolution, making it suitable for integration with real-time foveated rendering systems for VR and AR devices. We validate our results and compare them to the existing contrast enhancement technique in user experiments.

Melanopsin photoreception differentially modulates rod-mediated and cone-mediated human temporal vision

SummaryTo evaluate the nature of interactions between visual pathways transmitting the slower melanopsin and faster rod and cone signals, we implement a temporal phase summation paradigm in human observers using photoreceptor-directed stimuli. We show that melanopsin stimulation interacts with and alters both rod-mediated and cone-mediated vision regardless of whether it is perceptually visible or not. Melanopsin-rod interactions result in either inhibitory or facilitatory summation depending on the temporal frequency and photoreceptor pathway contrast sensitivity. Moreover, by isolating rod vision, we reveal a bipartite intensity response property of the rod pathway in photopic lighting that extends its operational range at lower frequencies to beyond its classic saturation limits but at the expense of attenuating sensitivity at higher frequencies. In comparison, melanopsin-cone interactions always lead to facilitation. These interactions can be described by linear or probability summations and potentially involve multiple intraretinal and visual cortical pathways to set human visual contrast sensitivity.

Depth-aware salient object segmentation

Depth-aware salient object segmentation

Systematic Differences Between Perceptually Relevant Image Statistics of Brain MRI and Natural Images.

It is well-known that the human visual system is adapted to the statistical structure of natural scenes. Yet there are important classes of images – for example, medical images – that are not natural scenes, and therefore, that are expected to have statistical properties that deviate from the class of images that shaped the evolution and development of human vision. Here, focusing on structural brain MRI images, we quantify and characterize these deviations in terms of a set of local image statistics to which human visual sensitivity has been well-characterized, and that has previously been used for natural image analysis. We analyzed MRI images in multiple databases including T1-weighted and FLAIR sequence types, and simulated MRI images based on a published image simulation procedure for T1 images, which we also modified to generate FLAIR images. We first computed the power spectra of MRI images; spectral slopes were in the range −2.6 to −3.1 for T1 sequences, and −2.2 to −2.7 for FLAIR sequences. Analysis of local image statistics was then carried out on whitened images. For all of the databases as well as for the simulated images, we found that the three-point correlations contributed substantially to the differences between the “texture” of randomly selected ROIs. The informative nature of three-point correlations for brain MRI was greater than for natural images, and also disproportionate to human visual sensitivity. As this finding was consistent across databases, it is likely to result from brain geometry at the scale of brain MRI resolution, rather than characteristics of specific imaging and reconstruction methods.

Chromatic exclusivity hypothesis and the physical basis of floral color

This paper presents the results of floral spectral studies on 1275 flowers from India, Brazil, Israel, Germany, and Norway. Floral spectral reflectance from 400 to 700 nm (nm) was used to quantitatively represent ‘human-perceived’ color of flowers in Red, Green, Blue color space. Floral spectral reflectance from 350 to 600 nm was used to discern and objectively represent ‘insect pollinator-perceived’ flower colors in color hexagon. We leverage the advantage offered by ‘quantified human perception’ provided by ‘human-perceived’ floral colors to represent the distribution of floral hues and uncover the relationship between the composition of incoming solar radiation and predominant ‘human-perceived’ floral colors at the tropics and the higher latitudes. Further, the observed species-level mutual exclusivity of ‘insect pollinator-perceived’ floral colors is stated as chromatic exclusivity hypothesis. We compare ‘human-perceived’ and ‘insect pollinator-perceived’ floral colors at Trivandrum (India) and provide a physical explanation for short and long ‘wavelength triads’ of insect pollinator and human visual sensitivity respectively.

Optical Detection of Green Emission for Non-Uniformity Film in Flat Panel Displays

Among colours, the green colour has the most sensitivity in human vision so that green colour defects on displays can be effortlessly perceived by a photopic eye with the most intensity in the wavelength 555 nm of the spectrum. With the market moving forward to high resolution, displays can have resolutions of 10 million pixels. Therefore, the method of detecting the appearance of the panel using ultra-high resolutions in TFT-LCD is important. The machine vision associated with transmission chromaticity spectrometer that quantises the defects are explored, such as blackening and whitening. The result shows the significant phenomena to recognize the non-uniformity of film-related chromatic variation. In contrast, the quantitative assessment illustrates that the just noticeable difference (JND) of chromaticity CIE xyY at 0.001 is the measuring sensitivity for the chromatic variables (x, y), whereas JND is a perceptible threshold for a colour difference metric. Moreover, an optical device associated with a 198Hg discharge lamp calibrates the spectrometer accuracy.

Investigating Human Visual Sensitivity to Binocular Motion-in-Depth for Anti- and De-Correlated Random-Dot Stimuli.

Motion-in-depth can be detected by using two different types of binocular cues: change of disparity (CD) and inter-ocular velocity differences (IOVD). To investigate the underlying detection mechanisms, stimuli can be constructed that isolate these cues or contain both (FULL cue). Two different methods to isolate the IOVD cue can be employed: anti-correlated (aIOVD) and de-correlated (dIOVD) motion signals. While both types of stimuli have been used in studies investigating the perception of motion-in-depth, for the first time, we explore whether both stimuli isolate the same mechanism and how they differ in their relative efficacy. Here, we set out to directly compare aIOVD and dIOVD sensitivity by measuring motion coherence thresholds. In accordance with previous results by Czuba et al. (2010), we found that motion coherence thresholds were similar for aIOVD and FULL cue stimuli for most participants. Thresholds for dIOVD stimuli, however, differed consistently from thresholds for the two other cues, suggesting that aIOVD and dIOVD stimuli could be driving different visual mechanisms.

SPECTRAL SENSITIVITY OF HUMAN VISION TO THE LIGHT PULSES

. In the paper we represent experimental results of the research of human vision sensitivity to pulsed radiation of discrete light emitting diodes (LEDs) of different colors when their dissipated electrical power was the same. We used color components of RGB LED matrix and determined the values of the dissipated power at which resultant LED radiation was perceived as white light. These power values were used to investigate sensitivity of human eye to radiation of each color of RGB matrix. Spectral characteristic of RGB LED under investigation was checked for presence of additional spectral components. The results we obtained give the possibility to develop information systems with concealment of the data transfer process using white light which color components are informative.

Optimum Space-Frequency Partition in Subband Image Coding with Human Visual Sensitivity and Region-of-Interest

Optimum Space-Frequency Partition in Subband Image Coding with Human Visual Sensitivity and Region-of-Interest

Rhythmic modulation of human visual sensitivity synchronized with planning of saccades

Rhythmic modulation of human visual sensitivity synchronized with planning of saccades

Rapid Processing of a Global Feature in the ON Visual Pathways of Behaving Monkeys

Visual objects are recognized by their features. Whereas, some features are based on simple components (i.e., local features, such as orientation of line segments), some features are based on the whole object (i.e., global features, such as an object having a hole in it). Over the past five decades, behavioral, physiological, anatomical, and computational studies have established a general model of vision, which starts from extracting local features in the lower visual pathways followed by a feature integration process that extracts global features in the higher visual pathways. This local-to-global model is successful in providing a unified account for a vast sets of perception experiments, but it fails to account for a set of experiments showing human visual systems' superior sensitivity to global features. Understanding the neural mechanisms underlying the “global-first” process will offer critical insights into new models of vision. The goal of the present study was to establish a non-human primate model of rapid processing of global features for elucidating the neural mechanisms underlying differential processing of global and local features. Monkeys were trained to make a saccade to a target in the black background, which was different from the distractors (white circle) in color (e.g., red circle target), local features (e.g., white square target), a global feature (e.g., white ring with a hole target) or their combinations (e.g., red square target). Contrary to the predictions of the prevailing local-to-global model, we found that (1) detecting a distinction or a change in the global feature was faster than detecting a distinction or a change in color or local features; (2) detecting a distinction in color was facilitated by a distinction in the global feature, but not in the local features; and (3) detecting the hole was interfered by the local features of the hole (e.g., white ring with a squared hole). These results suggest that monkey ON visual systems have a subsystem that is more sensitive to distinctions in the global feature than local features. They also provide the behavioral constraints for identifying the underlying neural substrates.

A novel adaptive data hiding based on improved EMD and interpolation

Hiding data on grayscale images has been drawing much attention in recent years. High capacity and better image quality are the two foremost research contents of data hiding. The traditional EMD which concealed an equal amount of data into each pixel and cause equal degree of distortion. Meanwhile due to being only an approximation, an interpolated image will always lose some quality each time interpolation is performed. In view of the problems existing in the two above method, we propose a novel data hiding scheme that employs human vision sensitivity to hide a large amount of secret bits into a original image with a high imperceptibility and low computational complexity. Experimental results reveal that the proposed method performs better than several state-of-the-art methods. It can resist the RS steganalysis attack and it is statistically invisible for the attack of PVD histogram attack.

What has been missed for predicting human attention in viewing driving clips?

Recent research progress on the topic of human visual attention allocation in scene perception and its simulation is based mainly on studies with static images. However, natural vision requires us to extract visual information that constantly changes due to egocentric movements or dynamics of the world. It is unclear to what extent spatio-temporal regularity, an inherent regularity in dynamic vision, affects human gaze distribution and saliency computation in visual attention models. In this free-viewing eye-tracking study we manipulated the spatio-temporal regularity of traffic videos by presenting them in normal video sequence, reversed video sequence, normal frame sequence, and randomised frame sequence. The recorded human gaze allocation was then used as the ‘ground truth’ to examine the predictive ability of a number of state-of-the-art visual attention models. The analysis revealed high inter-observer agreement across individual human observers, but all the tested attention models performed significantly worse than humans. The inferior predictability of the models was evident from indistinguishable gaze prediction irrespective of stimuli presentation sequence, and weak central fixation bias. Our findings suggest that a realistic visual attention model for the processing of dynamic scenes should incorporate human visual sensitivity with spatio-temporal regularity and central fixation bias.