Human Contrast Sensitivity Function Research Articles

Contrast sensitivity function in deep networks

The contrast sensitivity function (CSF) is a fundamental signature of the visual system that has been measured extensively in several species. It is defined by the visibility threshold for sinusoidal gratings at all spatial frequencies. Here, we investigated the CSF in deep neural networks using the same 2AFC contrast detection paradigm as in human psychophysics. We examined 240 networks pretrained on several tasks. To obtain their corresponding CSFs, we trained a linear classifier on top of the extracted features from frozen pretrained networks. The linear classifier is exclusively trained on a contrast discrimination task with natural images. It has to find which of the two input images has higher contrast. The network’s CSF is measured by detecting which one of two images contains a sinusoidal grating of varying orientation and spatial frequency. Our results demonstrate characteristics of the human CSF are manifested in deep networks both in the luminance channel (a band-limited inverted U-shaped function) and in the chromatic channels (two low-pass functions of similar properties). The exact shape of the networks’ CSF appears to be task-dependent. The human CSF is better captured by networks trained on low-level visual tasks such as image-denoising or autoencoding. However, human-like CSF also emerges in mid- and high-level tasks such as edge detection and object recognition. Our analysis shows that human-like CSF appears in all architectures but at different depths of processing, some at early layers, while others in intermediate and final layers. Overall, these results suggest that (i) deep networks model the human CSF faithfully, making them suitable candidates for applications of image quality and compression, (ii) efficient/purposeful processing of the natural world drives the CSF shape, and (iii) visual representation from all levels of visual hierarchy contribute to the tuning curve of the CSF, in turn implying a function which we intuitively think of as modulated by low-level visual features may arise as a consequence of pooling from a larger set of neurons at all levels of the visual system.

Contrast sensitivity is formed by visual experience and task demands

The contrast sensitivity function (CSF) is a fundamental signature of the visual system and has been measured extensively in numerous species. It is defined by measurements of the visibility threshold for sinusoidal gratings at all spatial frequencies. Here, we measured the CSF in artificial visual systems, namely deep neural networks (DNNs), using the same contrast discrimination paradigm as used in human psychophysical experiments. During training, networks are exclusively exposed to natural images, and the task is to identify which one of two input images has higher contrast. The contrast discriminator networks learn a linear classifier over frozen features of pretrained DNNs. At the testing stage, we measured the network's CSF by presenting sinusoidal gratings of different orientations and spatial frequencies. Our results demonstrate that the pretrained DNNs show the band-limited, inverted U-shaped CSF, that is characteristic of the human CSF. The exact shape of the DNNs' CSF appears to be task-dependent. The human CSF is better captured by scene segmentation DNNs than image classification ones (tested under identical settings such as the architecture and computational complexity). When the network was trained to discriminate the contrast in natural images from scratch (i.e., with no previous training task), a relatively flat CSF emerged, dissimilar to the human CSF. The environment of pretrained DNNs proved influential as well. Those with a diet of natural images, similar to human experience (e.g. objects or faces), obtain more human-like CSF. Pretrained DNNs with aerial pictures, such as a bird flying in the sky would see, obtain a shifted CSF towards higher spatial frequencies, similar to that of an eagle. In conclusion, the CSFs derived from the DNNs are the result of efficient processing of the natural world around us. Different visual environments have succinct effects on the shape of the CSFs.

A computational observer model of spatial contrast sensitivity: Effects of photocurrent encoding, fixational eye movements, and inference engine.

We have recently shown that the relative spatial contrast sensitivity function (CSF) of a computational observer operating on the cone mosaic photopigment excitations of a stationary retina has the same shape as human subjects. Absolute human sensitivity, however, is 5- to 10-fold lower than the computational observer. Here we model how additional known features of early vision affect the CSF: fixational eye movements and the conversion of cone photopigment excitations to cone photocurrents (phototransduction). For a computational observer that uses a linear classifier applied to the responses of a stimulus-matched linear filter, fixational eye movements substantially change the shape of the CSF by reducing sensitivity above 10 c/deg. For a translation-invariant computational observer that operates on the squared response of a quadrature-pair of linear filters, the CSF shape is little changed by eye movements, but there is a two fold reduction in sensitivity. Phototransduction dynamics introduce an additional two fold sensitivity decrease. Hence, the combined effects of fixational eye movements and phototransduction bring the absolute CSF of the translation-invariant computational observer to within a factor of 1 to 2 of the human CSF. We note that the human CSF depends on processing of the retinal representation by many thalamo-cortical neurons, which are individually quite noisy. Our modeling suggests that the net effect of post-retinal noise on contrast-detection performance, when considered at the neural population and behavioral level, is quite small: The inference mechanisms that determine the CSF, presumably in cortex, make efficient use of the information carried by the cone photocurrents of the fixating eye.

The spatial frequency spectrum of fearful faces modulates early and mid-latency ERPs but not the N170.

Prioritized processing of fearful compared to neutral faces is reflected in behavioral advantages such as lower detection thresholds, but also in enhanced early and late event-related potentials (ERPs). Behavioral advantages have recently been associated with the spatial frequency spectrum of fearful faces, better fitting the human contrast sensitivity function than the spectrum of neutral faces. However, it is unclear whether and to which extent early and late ERP differences are due to low-level spatial frequency spectrum information or high-level representations of the facial expression. In this pre-registered EEG study (N=38), the effects of fearful-specific spatial frequencies on event-related ERPs were investigated by presenting faces with fearful and neutral expressions whose spatial frequency spectra were manipulated so as to contain either the average power spectra of neutral, fearful, or both expressions combined. We found an enlarged N170 to fearful versus neutral faces, not interacting with spatial frequency. Interactions of emotional expression and spatial frequencies were observed for the P1 and Early Posterior Negativity (EPN). For both components, larger emotion differences were observed when the spectrum contained neutral as opposed to fearful frequencies. Importantly, for the EPN, fearful and neutral expressions did not differ anymore when inserting fearful frequencies into neutral expressions, whereas typical emotion differences were found when faces contained average or neutral frequencies. Our findings show that N170 emotional modulations are unaffected by expression-specific spatial frequencies. However, expression-specific spatial frequencies alter early and mid-latency ERPs. Most notably, the EPN to neutral expressions is boosted by adding fearful spectra-but not vice versa.

Contrast sensitivity reveals an oculomotor strategy for temporally encoding space

The contrast sensitivity function (CSF), how sensitivity varies with the frequency of the stimulus, is a fundamental assessment of visual performance. The CSF is generally assumed to be determined by low-level sensory processes. However, the spatial sensitivities of neurons in the early visual pathways, as measured in experiments with immobilized eyes, diverge from psychophysical CSF measurements in primates. Under natural viewing conditions, as in typical psychophysical measurements, humans continually move their eyes even when looking at a fixed point. Here, we show that the resulting transformation of the spatial scene into temporal modulations on the retina constitutes a processing stage that reconciles human CSF and the response characteristics of retinal ganglion cells under a broad range of conditions. Our findings suggest a fundamental integration between perception and action: eye movements work synergistically with the spatio-temporal sensitivities of retinal neurons to encode spatial information.

A hierarchical Bayesian approach to adaptive vision testing: A case study with the contrast sensitivity function.

Measurement efficiency is of concern when a large number of observations are required to obtain reliable estimates for parametric models of vision. The standard entropy-based Bayesian adaptive testing procedures addressed the issue by selecting the most informative stimulus in sequential experimental trials. Noninformative, diffuse priors were commonly used in those tests. Hierarchical adaptive design optimization (HADO; Kim, Pitt, Lu, Steyvers, & Myung, 2014) further improves the efficiency of the standard Bayesian adaptive testing procedures by constructing an informative prior using data from observers who have already participated in the experiment. The present study represents an empirical validation of HADO in estimating the human contrast sensitivity function. The results show that HADO significantly improves the accuracy and precision of parameter estimates, and therefore requires many fewer observations to obtain reliable inference about contrast sensitivity, compared to the method of quick contrast sensitivity function (Lesmes, Lu, Baek, & Albright, 2010), which uses the standard Bayesian procedure. The improvement with HADO was maintained even when the prior was constructed from heterogeneous populations or a relatively small number of observers. These results of this case study support the conclusion that HADO can be used in Bayesian adaptive testing by replacing noninformative, diffuse priors with statistically justified informative priors without introducing unwanted bias.

Contrast sensitivity function in stereoscopic viewing of Gabor patches on a medical polarized three-dimensional stereoscopic display

While three-dimensional (3-D) imaging systems are entering hospitals, no study to date has explored the luminance calibration needs of 3-D stereoscopic diagnostic displays and if they differ from two-dimensional (2-D) displays. Since medical display calibration incorporates the human contrast sensitivity function (CSF), we first assessed the 2-D CSF for benchmarking and then examined the impact of two image parameters on the 3-D stereoscopic CSF: (1) five depth plane (DP) positions (between DP: −171 and DP: 2853 mm), and (2) three 3-D inclinations (0 deg, 45 deg, and 60 deg around the horizontal axis of a DP). Stimuli were stereoscopic images of a vertically oriented 2-D Gabor patch at one of seven frequencies ranging from 0.4 to 10 cycles/deg. CSFs were measured for seven to nine human observers with a staircase procedure. The results indicate that the 2-D CSF model remains valid for a 3-D stereoscopic display regardless of the amount of disparity between the stereo images. We also found that the 3-D CSF at DP≠0 does not differ from the 3-D CSF at DP=0 for DPs and disparities which allow effortless binocular fusion. Therefore, the existing 2-D medical luminance calibration algorithm remains an appropriate tool for calibrating polarized stereoscopic medical displays.

Stochastic resonance in visual sensitivity.

It is well known from psychophysical studies that stochastic resonance, in its simplest threshold paradigm, can be used as a tool to measure the detection sensitivity to fine details in noise contaminated stimuli. In the present manuscript, we report simulation studies conducted in the similar threshold paradigm of stochastic resonance. We have estimated the contrast sensitivity in detecting noisy sine-wave stimuli, with varying area and spatial frequency, as a function of noise strength. In all the cases, the measured sensitivity attained a peak at intermediate noise strength, which indicate the occurrence of stochastic resonance. The peak sensitivity exhibited a strong dependence on area and spatial frequency of the stimulus. We show that the peak contrast sensitivity varies with spatial frequency in a nonmonotonic fashion and the qualitative nature of the sensitivity variation is in good agreement with human contrast sensitivity function. We also demonstrate that the peak sensitivity first increases and then saturates with increasing area, and this result is in line with the results of psychophysical experiments. Additionally, we also show that critical area, denoting the saturation of contrast sensitivity, decreases with spatial frequency and the associated maximum contrast sensitivity varies with spatial frequency in a manner that is consistent with the results of psychophysical experiments. In all the studies, the sensitivities were elevated via a nonlinear filtering operation called stochastic resonance. Because of this nonlinear effect, it was not guaranteed that the sensitivities, estimated at each frequency, would be in agreement with the corresponding results of psychophysical experiments; on the contrary, close agreements were observed between our results and the findings of psychophysical investigations. These observations indicate the utility of stochastic resonance in human vision and suggest that this paradigm can be useful in psychophysical studies.

Developing a clinically viable protocol for measuring and analysing the human contrast sensitivity function on a standard computerised chart

AbstractPurpose Contrast sensitivity testing is an important part of visual assessment and can identify visual deficits that go undetected by measuring visual acuity (VA) alone. Despite this, contrast sensitivity testing is rarely performed clinically due to the expensive processes required and limited test availability, with few spatial frequencies used. Further, gratings can be difficult for subjects to learn and results are often not standardized. We developed and tested a convenient, inexpensive protocol using Snellen optotypes to measure the CSF using a widely available computerised VA chart. Effects of blur and neutral density (ND) light attenuation were analysed to evaluate the technique under different conditions.Methods 34 eyes of 34 subjects were tested monocularly under photopic conditions. Subjects had corrected VA of 6/4.8 (or better) and normal ocular health. A computerised chart (Luneau) was used to measure contrast sensitivity thresholds across 19 spatial frequencies between 0.9 and 60.0 cpd using an ascending Method of Limits. This was repeated with +1.50DS blur and a 2.9 ND filter. CSFs were analysed and modeled using R and Prism to extract useful parameters: the area under the log10 CSF (AULCSF), CSF peak, high‐ and low‐spatial frequency limits.Results A quadratic function was found to best fit the pooled log CSF data, minimising cross‐validation error. The AULCSF was calculated between log spatial frequencies of 0.6 and 1.4 using integration of the fitted quadratic function. Mean AULCSF was 1.82±0.133.Conclusion Our method can measure, display and calculate the normal CSF. This can be used to compare and assess the CSFs of patients with ocular diseases.

Modified human contrast sensitivity functionbased phase mask for susceptibility-weighted imaging.

The aim of the work is to increase the visual information in magnetic resonance imaging based susceptibility weighted images. Our approach is to amplify spatial frequency information of the phase mask used to increase susceptibility weighting using a modified version of the human contrast sensitivity function. Thereby, we propose a modified version of the human contrast sensitivity function for use in phase mask creation. Comparison with conventional susceptibility-weighted imaging was undertaken on a qualitative basis and quantitatively with a number of established image quality metrics on ex vivo mouse brain magnetic resonance images obtained at 16.4 T at various echo times. Four experts also compared the quality of in vivo 1.5 and 3 T human brain magnetic resonance images generated with traditional susceptibility weighted imaging and with the new method. We found that parameters of the modified human contrast sensitivity function can be chosen to improve delineation of structural detail of mouse and human brains. Information contained in susceptibility-weighted images generated using the modified human contrast sensitivity function based phase mask corresponds to that in the conventional method, however the visual range over which it is depicted has improved visual perception. Hence, qualitative evaluation of information contained in susceptibility-weighted images can be improved by amplifying spatial frequencies where human contrast sensitivity is reduced.

Vision vs. Hearing: Direct Comparison of the Human Contrast Sensitivity and Audibility Functions.

Vision vs. Hearing: Direct Comparison of the Human Contrast Sensitivity and Audibility Functions.

Contrast-sensitivity-function-based clutter metric

Background clutter can obscure or mimic the desired target and confuse the operator. Thus, the similarity between the background and the target is a typical feature of the background clutter. A clutter metric based on human contrast sensitivity function is proposed. It measures the similarity between the background and the target in the frequency domain, after the differences that cannot be seen by the visual system have been "thrown away" and the visible differences have been weighted according to the sensitivity of the visual system. The correlation between the experimental probability of detection and that predicted by the clutter metric are presented. Experiment results show that the contrast-sensitivity-function-based clutter metric can perform other clutter metrics in open literature in quantifying background clutter.

Spatial-frequency cutoff requirements for pattern recognition in central and peripheral vision

It is well known that object recognition requires spatial frequencies exceeding some critical cutoff value. People with central scotomas who rely on peripheral vision have substantial difficulty with reading and face recognition. Deficiencies of pattern recognition in peripheral vision, might result in higher cutoff requirements, and may contribute to the functional problems of people with central-field loss. Here we asked about differences in spatial-cutoff requirements in central and peripheral vision for letter and face recognition. The stimuli were the 26 letters of the English alphabet and 26 celebrity faces. Each image was blurred using a low-pass filter in the spatial frequency domain. Critical cutoffs (defined as the minimum low-pass filter cutoff yielding 80% accuracy) were obtained by measuring recognition accuracy as a function of cutoff frequency (in cycles per object). Our data showed that critical cutoffs increased from central to peripheral vision by 20% for letter recognition and by 50% for face recognition. We asked whether these differences could be accounted for by central/peripheral differences in the contrast sensitivity function (CSF). We addressed this question by implementing an ideal-observer model which incorporates empirical CSF measurements and tested the model on letter and face recognition. The success of the model indicates that central/peripheral differences in the cutoff requirements for letter and face recognition can be accounted for by the information content of the stimulus limited by the shape of the human CSF, combined with a source of internal noise and followed by an optimal decision rule.

Effects of Multiplicative Power Law Neural Noise in Visual Information Processing

The human visual system is intrinsically noisy. The benefits of internal noise as part of visual code are controversial. Here the information-theoretic properties of multiplicative (i.e. signal-dependent) neural noise are investigated. A quasi-linear communication channel model is presented. The model shows that multiplicative power law neural noise promotes the minimum information transfer after efficient coding. It is demonstrated that Weber's law and the human contrast sensitivity function arise on the basis of minimum transfer of information and power law neural noise. The implications of minimum information transfer in self-organized neural networks and weakly coupled neurons are discussed.

Derivation of a color space for image color difference measurement

AbstractIn digital image reproduction, it is often desirable to compute image difference of reproductions and the original images. The traditional CIE color difference formula, designed for simple color patches in controlled viewing conditions, is not adequate for computing image difference for spatially complex image stimuli. Zhang and Wandell [Proceedings of the SID Symposium, 1996; p 731–734] introduced the S‐CIELAB model to account for complex color stimuli using spatial filtering as a preprocessing stage. Building on S‐CIELAB, iCAM was designed to serve as both a color appearance model and also an image difference metric for complex color stimuli [IS&T/SID 10th Color Imaging Conference, 2002; p 33–38]. These image difference models follow a similar image processing path to approximate the behavior of human observers. Generally, image pairs are first converted into device‐independent coordinates such as CIE XYZ tristimulus values or approximate human cone responses (LMS), and then further transformed into opponent‐color channels approximating white‐black, red‐green, and yellow‐blue color perceptions. Once in the opponent space, the images are filtered with approximations of human contrast sensitivity functions (CSFs) to remove information that is invisible to the human visual system. The images are then transformed back to a color difference space such as CIELAB, and pixel‐by‐pixel color differences are calculated. The shape and effectiveness of the CSF spatial filters used in this type of modeling is highly dependent on the choice of opponent color space. For image difference calculations, the ideal opponent color space would be both linear and orthogonal such that the linear filtering is correct and any spatial processing on one channel does not affect the others. This article presents a review of historical opponent color spaces and an experimental derivation of a new color space and corresponding spatial filters specifically designed for image color difference calculations. © 2010 Wiley Periodicals, Inc. Col Res Appl, 2010

Binocular Interaction in Post-LASIK Subjects with Unsatisfactory Outcome

Purpose: A shift from binocular summation to binocular inhibition has been observed when there is a significant decrease in contrast sensitivity or reduced retinal illuminance in one eye compared to the other as in cases of unilateral cataract and amblyopia. This study aims to determine how binocular function in post-LASIK subjects with unsatisfactory outcomes is influenced by differences between the two eyes in visual acuity and contrast sensitivity.Materials and Methods: A retrospective analysis of monocular and binocular visual acuity (VA) and contrast sensitivity (CS) results was performed for 23 LASIK subjects who previously underwent a Functional Vision Test battery at Vision Sciences Research Corp. (San Ramon, CA). Photopic VA was measured with ETDRS chart and sine-wave grating CS was measured with FACT chart.Results: The difference between binocular and better monocular VA and CS was calculated for each subject and referred to as binocular difference. The interocular difference was calculated as better eye-worse eye for VA and CS. Significant negative correlations (P < 0.01) were observed between binocular difference and interocular difference for VA (r = -0.84) and CS at each spatial frequency of 1.5 (r = -0.90), 3 (r = -0.85), 6 (r = -0.79), 12 (r = -0.68), and 18c/d (r = -0.72). The greater the difference between the eyes, the lesser the binocular summation. Linear regression predicted a 1-line decrease in binocular summation for a 4-line increase in interocular acuity difference and a 35% decrease in binocular summation for a 50% increase in interocular difference of contrast sensitivity at 3c/d (ANOVA, P < 0.01).Conclusions: Binocular inhibition in post-LASIK subjects increases as the visual sensitivity difference between the two eyes increases. Since the peak of human contrast sensitivity function is at the middle spatial frequencies, the decrease in binocular CS at 3c/d due to reduced CS in one eye is functionally significant.

Efficient integration across spatial frequencies for letter identification in foveal and peripheral vision

Objects in natural scenes are spatially broadband; in contrast, feature detectors in the early stages of visual processing are narrowly tuned in spatial frequency. Earlier studies of feature integration using gratings suggested that integration across spatial frequencies is suboptimal. Here we re-examined this conclusion using a letter identification task at the fovea and at 10 deg in the lower visual field. We found that integration across narrow-band (1-octave) spatial frequency components of letter stimuli is optimal in the fovea. Surprisingly, this optimality is preserved in the periphery, even though feature integration is known to be deficient in the periphery from studies of other form-vision tasks such as crowding. A model that is otherwise a white-noise ideal observer except for a limited spatial resolution defined by the human contrast sensitivity function and using internal templates slightly wider in bandwidth than the stimuli is able to account for the human data. Our findings suggest that deficiency in feature integration found in peripheral vision is not across spatial frequencies.

Developing a new method to assess poor visibility level on roads by digital images

This study proposes a new index for estimating visibility under poor-visibility conditions by using digital images from a road monitoring camera. The method allows representation of the amount of visual information based on human spatial frequency characteristics. A laboratory experiment was performed in 2002 under artificial fog. We examined the weighted intensity of power spectra (WIPS) to determine its applicability as a visibility assessment value. The magnitude of WIPS represented the difference in spatial frequencies within the image based on human contrast sensitivity function. WIPS was calculated from the following image-processing procedure. As an initial step, a box measuring 128 × 128 pixels was cut from the original image. The spatial frequency of the cutout image was calculated using two-dimensional Fourier transform, and the power spectrum of the cutout image was calculated. We totaled the WIPS at five spatial frequencies (cycles per degree, cpd): 1.5 cpd, 3 cpd, 6 cpd, 12 cpd and 18 cpd. WIPS were compared with the subjective visibility assessment values (SVAV) given by test subjects. There was a clear relationship between WIPS and total SVAV: WIPS increased as total SVAV increased. These results suggest that WIPS might be an appropriate index for estimating visibility using digital images.

Contrast sensitivity with presbyopic contact lenses

The purpose of the work was to assess the contrast sensitivity function of individuals wearing gas permeable (GP) multifocal contact lenses, soft bifocal contact lenses, and GP monovision lenses. Twenty-six females and six males between the ages of 42 and 65 participated in this study. The study included subjects wearing monovision (N = 8), the Acuvue Bifocal (Johnson & Johnson) (N = 8), Essential GP Multifocal (Blanchard) lenses (N = 8) and progressive addition spectacle lenses (PAL) (N = 8), with PAL wearers forming the control group. Measurements of binocular contrast sensitivity were obtained using the VISTECH 6500 system. Thresholds for each spatial frequency were fit to the equation CS(k) = ak exp(−bk) [1 + c exp(bk)]−1/2, to describe the human contrast sensitivity function. The area under the contrast sensitivity function (CSF) curve was calculated for all four groups and compared. An index of performance was obtained, which was defined as the ratio of CSF with the contact lens correction to the CSF with spectacles. Of the contact lens wearing groups, GP multifocal contact lens wearers had the highest contrast sensitivity at all the spatial frequencies. Soft bifocal contact lens wearers exhibited higher contrast sensitivity than monovision wearers at all spatial frequencies. Subjects wearing GP multifocals had the largest area under the CSF; followed by those wearing soft bifocals, with monovision wearers having the smallest area. GP multifocals have the best visual function at 0.98, soft bifocals have an index of 0.65 and monovision has an index of 0.59. This study quantifies the visual performance of the three lens systems by measuring the area under the CSF curve. In addition, it provides indices of visual function with the contact lenses that will be helpful for analyses and comparisons in future studies.

Method of Processing Closed-Circuit Television Digital Images for Poor Visibility Identification

This study proposes a method for identifying poor visibility under adverse weather conditions by processing closed-circuit television (CCTV) digital images. The weighted intensity of the power spectrum (WIPS) was examined to determine its applicability as a value for the identification of poor visibility. The magnitude of WIPS represents the difference in spatial frequencies within the image on the basis of the human contrast sensitivity function. WIPS was calculated by the following image-processing procedure: the spatial frequency of the cutout image was calculated with a two-dimensional Fourier transform, and the power spectrum of the cutout image was calculated; WIPS was totaled at spatial frequencies that ranged from 1.5 to 18 cycles per degree. Two kinds of experiments were performed to determine whether WIPS represented the subjective visibility assessment values (SVAVs) given by the test subjects. Clear linear relationships between WIPS and SVAVs were found in both experiments. In addition, the tw...