Model Of Spatial Vision Research Articles

Plaid masking explained with input-dependent dendritic nonlinearities

A serious obstacle for understanding early spatial vision comes from the failure of the so-called standard model (SM) to predict the perception of plaid masking. But the SM originated from a major oversimplification of single neuron computations, ignoring fundamental properties of dendrites. Here we show that a spatial vision model including computations mimicking the input-dependent nature of dendritic nonlinearities, i.e. including nonlinear neural summation, has the potential to explain plaid masking data.

Standard models of spatial vision mispredict edge sensitivity at low spatial frequencies

One well-established characteristic of early visual processing is the contrast sensitivity function (CSF) which describes how sensitivity varies with the spatial frequency (SF) content of the visual input. The CSF prompted the development of a now standard model of spatial vision. It represents the visual input by activity in orientation- and SF selective channels which are nonlinearly recombined to predict a perceptual decision. The standard spatial vision model has been extensively tested with sinusoidal gratings at low contrast because their narrow SF spectra isolate the underlying SF selective mechanisms. It is less studied how well these mechanisms account for sensitivity to more behaviourally relevant stimuli such as sharp edges at high contrast (i.e. object boundaries) which abound in the natural environment and have broader SF spectra. Here, we probe sensitivity to edges (2-AFC, edge localization) in the presence of broadband and narrowband noises. We use Cornsweet luminance profiles with peak frequencies at 0.5, 3 and 9 cpd as edge stimuli. To test how well mechanisms underlying sinusoidal contrast sensitivity can account for edge sensitivity, we implement a single- and a multi-scale model building upon standard spatial vision model components. Both models account for most of the data but also systematically deviate in their predictions, particularly in the presence of pink noise and for the lowest SF edge. These deviations might indicate a transition from contrast- to luminance-based detection at low SFs. Alternatively, they might point to a missing component in current spatial vision models.

Fixational eye movements enable robust edge detection.

Human vision relies on mechanisms that respond to luminance edges in space and time. Most edge models use orientation-selective mechanisms on multiple spatial scales and operate on static inputs assuming that edge processing occurs within a single fixational instance. Recent studies, however, demonstrate functionally relevant temporal modulations of the sensory input due to fixational eye movements. Here we propose a spatiotemporal model of human edge detection that combines elements of spatial and active vision. The model augments a spatial vision model by temporal filtering and shifts the input images over time, mimicking an active sampling scheme via fixational eye movements. The first model test was White's illusion, a lightness effect that has been shown to depend on edges. The model reproduced the spatial-frequency-specific interference with the edges by superimposing narrowband noise (1–5 cpd), similar to the psychophysical interference observed in White's effect. Second, we compare the model's edge detection performance in natural images in the presence and absence of Gaussian white noise with human-labeled contours for the same (noise-free) images. Notably, the model detects edges robustly against noise in both test cases without relying on orientation-selective processes. Eliminating model components, we demonstrate the relevance of multiscale spatiotemporal filtering and scale-specific normalization for edge detection. The proposed model facilitates efficient edge detection in (artificial) vision systems and challenges the notion that orientation-selective mechanisms are required for edge detection.

Perceptually guided contrast enhancement based on viewing distance.

We propose an image-space contrast enhancement method for color-encoded visualization. The contrast of an image is enhanced through a perceptually guided approach that interfaces with the user with a single and intuitive parameter of the virtual viewing distance. To this end, we analyze a multiscale contrast model of the input image and test the visibility of bandpass images of all scales at a virtual viewing distance. By adapting weights of bandpass images with a threshold model of spatial vision, this image-based method enhances contrast to compensate for contrast loss caused by viewing the image at a certain distance. Relevant features in the color image can be further emphasized by the user using overcompensation. The weights can be assigned with a simple band-based approach, or with an efficient pixel-based approach that reduces ringing artifacts. The method is efficient and can be integrated into any visualization tool as it is a generic image-based post-processing technique. Using highly diverse datasets, we show the usefulness of perception compensation across a wide range of typical visualizations.

Disentangling bottom-up versus top-down and low-level versus high-level influences on eye movements over time.

Bottom-up and top-down as well as low-level and high-level factors influence where we fixate when viewing natural scenes. However, the importance of each of these factors and how they interact remains a matter of debate. Here, we disentangle these factors by analyzing their influence over time. For this purpose, we develop a saliency model that is based on the internal representation of a recent early spatial vision model to measure the low-level, bottom-up factor. To measure the influence of high-level, bottom-up features, we use a recent deep neural network-based saliency model. To account for top-down influences, we evaluate the models on two large data sets with different tasks: first, a memorization task and, second, a search task. Our results lend support to a separation of visual scene exploration into three phases: the first saccade, an initial guided exploration characterized by a gradual broadening of the fixation density, and a steady state that is reached after roughly 10 fixations. Saccade-target selection during the initial exploration and in the steady state is related to similar areas of interest, which are better predicted when including high-level features. In the search data set, fixation locations are determined predominantly by top-down processes. In contrast, the first fixation follows a different fixation density and contains a strong central fixation bias. Nonetheless, first fixations are guided strongly by image properties, and as early as 200 ms after image onset, fixations are better predicted by high-level information. We conclude that any low-level, bottom-up factors are mainly limited to the generation of the first saccade. All saccades are better explained when high-level features are considered, and later, this high-level, bottom-up control can be overruled by top-down influences.

An image-computable psychophysical spatial vision model.

A large part of classical visual psychophysics was concerned with the fundamental question of how pattern information is initially encoded in the human visual system. From these studies a relatively standard model of early spatial vision emerged, based on spatial frequency and orientation-specific channels followed by an accelerating nonlinearity and divisive normalization: contrast gain-control. Here we implement such a model in an image-computable way, allowing it to take arbitrary luminance images as input. Testing our implementation on classical psychophysical data, we find that it explains contrast detection data including the ModelFest data, contrast discrimination data, and oblique masking data, using a single set of parameters. Leveraging the advantage of an image-computable model, we test our model against a recent dataset using natural images as masks. We find that the model explains these data reasonably well, too. To explain data obtained at different presentation durations, our model requires different parameters to achieve an acceptable fit. In addition, we show that contrast gain-control with the fitted parameters results in a very sparse encoding of luminance information, in line with notions from efficient coding. Translating the standard early spatial vision model to be image-computable resulted in two further insights: First, the nonlinear processing requires a denser sampling of spatial frequency and orientation than optimal coding suggests. Second, the normalization needs to be fairly local in space to fit the data obtained with natural image masks. Finally, our image-computable model can serve as tool in future quantitative analyses: It allows optimized stimuli to be used to test the model and variants of it, with potential applications as an image-quality metric. In addition, it may serve as a building block for models of higher level processing.

Retinex at 50: color theory and spatial algorithms, a review

Retinex Imaging shares two distinct elements: first, a model of human color vision; second, a spatial-imaging algorithm for making better reproductions. Edwin Land’s 1964 Retinex Color Theory began as a model of human color vision of real complex scenes. He designed many experiments, such as Color Mondrians, to understand why retinal cone quanta catch fails to predict color constancy. Land’s Retinex model used three spatial channels (L, M, S) that calculated three independent sets of monochromatic lightnesses. Land and McCann’s lightness model used spatial comparisons followed by spatial integration across the scene. The parameters of their model were derived from extensive observer data. This work was the beginning of the second Retinex element, namely, using models of spatial vision to guide image reproduction algorithms. Today, there are many different Retinex algorithms. This special section, “Retinex at 50,” describes a wide variety of them, along with their different goals, and ground truths used to measure their success. This paper reviews (and provides links to) the original Retinex experiments and image-processing implementations. Observer matches (measuring appearances) have extended our understanding of how human spatial vision works. This paper describes a collection very challenging datasets, accumulated by Land and McCann, for testing algorithms that predict appearance.

Noise Provides New Insights on Contrast Sensitivity Function

Sensitivity to luminance difference, or contrast sensitivity, is critical for animals to survive in and interact with the external world. The contrast sensitivity function (CSF), which measures visual sensitivity to spatial patterns over a wide range of spatial frequencies, provides a comprehensive characterization of the visual system. Despite its popularity and significance in both basic research and clinical practice, it hasn’t been clear what determines the CSF and how the factors underlying the CSF change in different conditions. In the current study, we applied the external noise method and perceptual template model to a wide range of external noise and spatial frequency (SF) conditions, and evaluated how the various sources of observer inefficiency changed with SF and determined the limiting factors underlying the CSF. We found that only internal additive noise and template gain changed significantly with SF, while the transducer non-linearity and coefficient for multiplicative noise were constant. The 12-parameter model provided a very good account of all the data in the 200 tested conditions (86.5%, 86.2%, 89.5%, and 96.4% for the four subjects, respectively). Our results suggest a re-consideration of the popular spatial vision model that employs the CSF as the front-end filter and constant internal additive noise across spatial frequencies. The study will also be of interest to scientists and clinicians engaged in characterizing spatial vision deficits and/or developing rehabilitation methods to restore spatial vision in clinical populations.

Mach bands and multiscale models of spatial vision: The role of first, second, and third derivative operators in encoding bars and edges

Ernst Mach observed that light or dark bands could be seen at abrupt changes of luminance gradient in the absence of peaks or troughs in luminance. Many models of feature detection share the idea that bars, lines, and Mach bands are found at peaks and troughs in the output of even-symmetric spatial filters. Our experiments assessed the appearance of Mach bands (position and width) and the probability of seeing them on a novel set of generalized Gaussian edges. Mach band probability was mainly determined by the shape of the luminance profile and increased with the sharpness of its corners, controlled by a single parameter (n). Doubling or halving the size of the images had no significant effect. Variations in contrast (20%-80%) and duration (50-300 ms) had relatively minor effects. These results rule out the idea that Mach bands depend simply on the amplitude of the second derivative, but a multiscale model, based on Gaussian-smoothed first- and second-derivative filtering, can account accurately for the probability and perceived spatial layout of the bands. A key idea is that Mach band visibility depends on the ratio of second- to first-derivative responses at peaks in the second-derivative scale-space map. This ratio is approximately scale-invariant and increases with the sharpness of the corners of the luminance ramp, as observed. The edges of Mach bands pose a surprisingly difficult challenge for models of edge detection, but a nonlinear third-derivative operation is shown to predict the locations of Mach band edges strikingly well. Mach bands thus shed new light on the role of multiscale filtering systems in feature coding.

Simulating Images Seen by Patients with Inhomogeneous Sensitivity Losses

We aim to simulate how colored images are perceived by subjects with local achromatic and chromatic contrast sensitivity losses in the visual field (VF). The spatiochromatic corresponding pair algorithm, introduced in a previous article (J Opt Soc Am (A) 2004;21:176-186), has been implemented with a linear model of the visual system. Spatial information is processed separately by the chromatic and achromatic mechanisms by means of a multiscale model, with sensors selective to frequency, orientation, and spatial position, whose mechanism-dependent relative weights change with the spatial location of the image. These weights have been obtained from perimetric data from a patient with Leber's Hereditary Optic Neuropathy and an age-matched sample of normal subjects, using achromatic, red-green, and blue-yellow gratings of different spatial frequencies. Distortion contours for each mechanism have been derived from the images simulating the perception of these subjects at different locations in the VF. The images simulating the perception of normal subjects at different locations of the VF show a fast decrease in image quality with eccentricity. The same analysis carried out with the Leber's Hereditary Optic Neuropathy patient reveals worse overall image quality throughout the VF, plus a color vision defect resembling red-green dichromacy at fovea and trichromatic anomaly in the rest of the VF. In the present article, we show that implementing the algorithm with a spatial vision model that considers the changes in contrast sensitivity with spatial location of the stimulus may reveal the local effects that are suffered, in general, by pathological subjects, and that are ignored by simpler spatial vision models.

Mach bands and models of spatial vision: the role of 1st, 2nd and 3rd derivative operators in encoding edges and bars

Mach bands and models of spatial vision: the role of 1st, 2nd and 3rd derivative operators in encoding edges and bars

Orientation masking and cross-orientation suppression (XOS): Implications for estimates of filter bandwidth

Most contemporary models of spatial vision include a cross-oriented route to suppression (masking from a broadly tuned inhibitory pool), which is most potent at low spatial and high temporal frequencies (T. S. Meese & D. J. Holmes, 2007). The influence of this pathway can elevate orientation-masking functions without exciting the target mechanism, and because early psychophysical estimates of filter bandwidth did not accommodate this, it is likely that they have been overestimated for this corner of stimulus space. Here we show that a transient 40% contrast mask causes substantial binocular threshold elevation for a transient vertical target, and this declines from a mask orientation of 0° to about 40° (indicating tuning), and then more gently to 90°, where it remains at a factor of ∼4. We also confirm that cross-orientation masking is diminished or abolished at high spatial frequencies and for sustained temporal modulation. We fitted a simple model of pedestal masking and cross-orientation suppression (XOS) to our data and those of G. C. Phillips and H. R. Wilson (1984) and found the dependency of orientation bandwidth on spatial frequency to be much less than previously supposed. An extension of our linear spatial pooling model of contrast gain control and dilution masking (T. S. Meese & R. J. Summers, 2007) is also shown to be consistent with our results using filter bandwidths of ±20°. Both models include tightly and broadly tuned components of divisive suppression. More generally, because XOS and/or dilution masking can affect the shape of orientation-masking curves, we caution that variations in bandwidth estimates might reflect variations in processes that have nothing to do with filter bandwidth.

Effects of arbitrary structural choices on the parameters of early spatial vision models

Effects of arbitrary structural choices on the parameters of early spatial vision models

Images Perceived After Chromatic or Achromatic Contrast Sensitivity Losses

We simulate how subjects with losses in chromatic and achromatic contrast sensitivity perceive colored images by using the spatiochromatic corresponding pair algorithm. This is a generalized version of the algorithm by Capilla et al. (J Opt Soc Am (A) 2004;21:176-186) for simulating color perception of color deviant subjects, which incorporates a simple spatial vision model, consisting of a linear filtering stage, with a band-pass achromatic filter and two low-pass chromatic ones, for the red-green and blue-yellow mechanisms. These filters, except for the global scaling, are the subject's contrast sensitivity functions measured along the cardinal directions of the color space. In its present form, the algorithm would serve to simulate alterations both in the spectral sensitivities and in the contrast sensitivities of the visual mechanisms. After a preliminary theoretical study on the effect of frequency selective and overall reductions in the contrast sensitivity function of a single mechanism, we present cases of real subjects with glaucoma and diabetes, suffering alterations of different magnitude in the three mechanisms. The simulations allow us to learn about the different types of distortions that can be experienced by a subject with impaired contrast sensitivities (blur, haloes, color shifts, local or global contrast, brightness and colorfulness reductions, etc.) and highlight the difficulties arising when trying to predict the quality of the final image from the losses in the individual mechanisms.

Modelling contrast discrimination data suggest both the pedestal effect and stochastic resonance to be caused by the same mechanism

Computational models of spatial vision typically make use of a (rectified) linear filter, a nonlinearity and dominant late noise to account for human contrast discrimination data. Linear-nonlinear cascade models predict an improvement in observers' contrast detection performance when low, subthreshold levels of external noise are added (i.e., stochastic resonance). Here, we address the issue whether a single contrast gain-control model of early spatial vision can account for both the pedestal effect, i.e., the improved detectability of a grating in the presence of a low-contrast masking grating, and stochastic resonance. We measured contrast discrimination performance without noise and in both weak and moderate levels of noise. Making use of a full quantitative description of our data with few parameters combined with comprehensive model selection assessments, we show the pedestal effect to be more reduced in the presence of weak noise than in moderate noise. This reduction rules out independent, additive sources of performance improvement and, together with a simulation study, supports the parsimonious explanation that a single mechanism underlies the pedestal effect and stochastic resonance in contrast perception.

On the calculation of optical performance factors from vertebrate spatial contrast sensitivity

A novel technique for calculating the visual optical modulation transfer function (OMTF) is described. The technique involves application of the Rovamo–Barten model of spatial vision to measured contrast sensitivity data. [For details of the basic model see; Rovamo, J., Mustonen, J., & Nasanen, R. (1994). Modelling contrast sensitivity as a function of retinal illuminance and grating area. Vision Research, 34, 1301–1314 and Barten, P. J. G. (1999). Contrast sensitivity of the human eye and its effects on image quality. Washington: SPIE Optical Engineering Press.] In order to obtain OMTF, the model was simplified for use in the high spatial frequency range and also modified to include a transfer function term relating to attenuation by the retinal receptor sampling process. Calculations of OMTF were initially obtained from published contrast sensitivity for the human, cat, rat and chicken. The results were found to correlate well with OMTF values directly obtained through a double-pass optical measuring technique applied to all four species. It was assumed, following this initial test, that the modified Rovamo–Barten model could be used to extract OMTF from vertebrate contrast sensitivity data in general. Using published behavioural contrast sensitivity, further OMTF values were calculated from the model for the pigeon, goldfish, owl monkey, and tree shrew. The results obtained were used to provide a direct inter-species comparison of optical performance for a matched stimulus luminance. This study also confirms that, in many cases, vertebrate optical and receptor sampling processes are well matched in their attenuation properties.

A cortical pooling model of spatial summation for perimetric stimuli

Contemporary models of perimetric sensitivity assume probability summation of retinal ganglion cell sensitivities, ignoring cortical processing. To assess the role of cortical processing in perimetric spatial summation, we used a common form of multiple-mechanism spatial vision model in which the stimulus is sampled by receptive fields analogous to those of simple cells in primary visual cortex. Psychophysical threshold was computed by probability summation across the receptive fields. When the receptive fields were nonoriented (like ganglion cells), the spatial summation function had a large nonmonotonic transitional region that was inconsistent with perimetric spatial summation data. When the receptive fields were orientation tuned (like cortical cells), the model was able to give good fits to perimetric spatial summation data. The predictions of the model were evaluated with a masking study, in which noise masks either enlarged the critical area or changed the shape of the spatial summation functions. We conclude that cortical pooling by multiple spatial mechanisms can account for perimetric spatial summation, whereas probability summation across ganglion cells cannot.

Scale invariance is driven by stimulus density

Scale invariance refers to aspects of visual perception that remain constant with changes in viewing distance. Previously, Dakin and Herbert [Proc. Roy. Soc. B. 265 (1397) (1998) 659] reported that the spatial integration region (IR) for mirror symmetry in bandpass noise is scale invariant because its dimensions scale with the inverse of peak spatial frequency. In bandpass noise, however, peak spatial frequency covaries with stimulus numerosity (i.e. the total number of information samples) and density (i.e. the total number of information samples per unit area). In this study, we report four experiments that decoupled properties of the retinal image affected by viewing distance––spatial frequency, numerosity, size, and density––and measured their effect on IR size. Stimuli consisted of bandpass microelements with vertically mirror-symmetric but otherwise random positions, and we measured observer resistance to random jitter imposed on microelement position. Results show that jitter resistance and IR size vary with the inverse of stimulus density but are unaffected by changes in stimulus spatial frequency, numerosity, or size. We found the IR has a 2:1 height-to-width aspect ratio and integrates information from ≈18 microelements regardless of their spatial separation. Our results reveal that stimulus density plays a central role in the visual system's implementation of scale invariance. Using an ideal-observer, we demonstrate that scale invariance reflects genuine neural scale selection rather than a physical limitation on the stimulus' information content. Our findings that jitter resistance and IR size vary with the inverse of density challenge current models of spatial vision but can be reconciled with a model that compares the output of bandpass non-Fourier mechanisms to select spatial scales that match stimulus density.

The relatively small decline in orientation acuity as stimulus size decreases

Orientation acuity was measured with circular patches of sinusoidal gratings of various sizes. Threshold estimates were lowest (acuity highest) for the largest size patch, and increased as the stimulus size was reduced, consistent with the results of many researchers using line stimuli. These results are compared with the predictions of a simple and widely accepted model of spatial vision whereby the output of independent feed-forward filters are combined to produce threshold estimates. Specifically, the rectified output of a number of independent filters (i.e. Gabors) spanning the stimulus space (i.e. orientation) are combined via Bayesian decision theory. This model cannot account quantitatively for the relatively low thresholds estimated for the small sized stimuli when compared to the thresholds measured with larger patches. Application of a comparable analysis, with preliminary measurements of neuronal responses from primary visual cortex replacing the response rectified Gabor filter's responses, provides a more reasonable account of behavioral acuity. This indicates a fundamental inadequacy of the feed-forward filter model in accounting for V1 neurons' role in perception.

Energy model for contrast detection: spatial-frequency and orientation selectivity in grating summation

Models of spatial vision usually assume a ‘front-end’ of spatial-frequency and orientation selective channels. Subthreshold-summation studies have provided some of the strongest support for this notion. We applied a single-channel energy model and a multiple-channels probability-summation model to explore subthreshold-summation phenomena. We measured the contrast thresholds for detection of two superimposed Gabor patches as a function of the spatial-frequency and orientation difference between the components. The stimuli were centred 7.5 deg above the fixation point and were windowed by a Gaussian function with one of two different spatial spreads. We have shown that the spatial-frequency and orientation selectivity in subthreshold summation of Gabor patches is determined by the similarity (cross-correlation) between the stimulus components. A single-channel energy model as well as a multiple-channels probability-summation model could explain the summation data.