Contrast Gain Control Mechanism Research Articles

A gain-control disparity energy model for perceived depth from disparity

Luminance contrast is one of the key factors in the visibility of objects in the world around us. Previous work has shown that the perceived depth from binocular disparity depends profoundly on the luminance contrast of the image. This dependence cannot be explained by existing disparity models, such as the well-established disparity energy model, because they predict no effect of luminance contrast on depth perception. Here, we develop a model for disparity processing that incorporates contrast normalization of the neural response into the disparity energy model to account for the contrast dependence of perceived depth from disparity. Our model contains an array of disparity channels, each with a different disparity selectivity. The binocular images are first processed by the left- and right-eye receptive fields of each channel. The outputs of the two receptive fields are combined linearly as the excitatory disparity sensitivity and then fed into a nonlinear contrast gain control mechanism. The perceived depth is determined by the weighted average of all the disparity channels that respond to the binocular images. This model provides the first analytic account of how luminance contrast affects perceived depth from disparity.

Nonlinear Lateral Interactions in V1 Population Responses Explained by a Contrast Gain Control Model.

How do cortical responses to local image elements combine to form a spatial pattern of population activity in primate V1? Here, we used voltage-sensitive dye imaging, which measures summed membrane potential activity, to examine the rules that govern lateral interactions between the representations of two small local-oriented elements in macaque (Macaca mulatta) V1. We find strong subadditive and mostly orientation-independent interactions for nearby elements [2-4 mm interelement cortical distance (IED)] that gradually become linear at larger separations (>6 mm IED). These results are consistent with a population gain control model describing nonlinear V1 population responses to single oriented elements. However, because of the membrane potential-to-spiking accelerating nonlinearity, the model predicts supra-additive lateral interactions of spiking responses for intermediate separations at a range of locations between the two elements, consistent with some prior facilitatory effects observed in electrophysiology and psychophysics. Overall, our results suggest that population-level lateral interactions in V1 are primarily explained by a simple orientation-independent contrast gain control mechanism.SIGNIFICANCE STATEMENT Interactions between representations of simple visual elements such as oriented edges in primary visual cortex (V1) are thought to contribute to our ability to easily integrate contours and segment surfaces, but the mechanisms that govern these interactions are primarily unknown. Our study provides novel evidence that lateral interactions at the population level are governed by a simple contrast gain-control mechanism, and we show how this divisive gain-control mechanism can give rise to apparently facilitatory spiking responses.

Binocular contrast-gain control for natural scenes: Image structure and phase alignment

In the context of natural scenes, we applied the pattern-masking paradigm to investigate how image structure and phase alignment affect contrast-gain control in binocular vision. We measured the discrimination thresholds of bandpass-filtered natural-scene images (targets) under various types of pedestals. Our first experiment had four pedestal types: bandpass-filtered pedestals, unfiltered pedestals, notch-filtered pedestals (which enabled removal of the spatial frequency), and misaligned pedestals (which involved rotation of unfiltered pedestals). Our second experiment featured six types of pedestals: bandpass-filtered, unfiltered, and notch-filtered pedestals, and the corresponding phase-scrambled pedestals. The thresholds were compared for monocular, binocular, and dichoptic viewing configurations. The bandpass-filtered pedestal and unfiltered pedestals showed classic dipper shapes; the dipper shapes of the notch-filtered, misaligned, and phase-scrambled pedestals were weak. We adopted a two-stage binocular contrast-gain control model to describe our results. We deduced that the phase-alignment information influenced the contrast-gain control mechanism before the binocular summation stage and that the phase-alignment information and structural misalignment information caused relatively strong divisive inhibition in the monocular and interocular suppression stages. When the pedestals were phase-scrambled, the elimination of the interocular suppression processing was the most convincing explanation of the results. Thus, our results indicated that both phase-alignment information and similar image structures cause strong interocular suppression.

Convis: A Toolbox to Fit and Simulate Filter-Based Models of Early Visual Processing.

We developed Convis, a Python simulation toolbox for large scale neural populations which offers arbitrary receptive fields by 3D convolutions executed on a graphics card. The resulting software proves to be flexible and easily extensible in Python, while building on the PyTorch library (The Pytorch Project, 2017), which was previously used successfully in deep learning applications, for just-in-time optimization and compilation of the model onto CPU or GPU architectures. An alternative implementation based on Theano (Theano Development Team, 2016) is also available, although not fully supported. Through automatic differentiation, any parameter of a specified model can be optimized to approach a desired output which is a significant improvement over e.g., Monte Carlo or particle optimizations without gradients. We show that a number of models including even complex non-linearities such as contrast gain control and spiking mechanisms can be implemented easily. We show in this paper that we can in particular recreate the simulation results of a popular retina simulation software VirtualRetina (Wohrer and Kornprobst, 2009), with the added benefit of providing (1) arbitrary linear filters instead of the product of Gaussian and exponential filters and (2) optimization routines utilizing the gradients of the model. We demonstrate the utility of 3d convolution filters with a simple direction selective filter. Also we show that it is possible to optimize the input for a certain goal, rather than the parameters, which can aid the design of experiments as well as closed-loop online stimulus generation. Yet, Convis is more than a retina simulator. For instance it can also predict the response of V1 orientation selective cells. Convis is open source under the GPL-3.0 license and available from https://github.com/jahuth/convis/ with documentation at https://jahuth.github.io/convis/.

The role of chromatic variance in modulating color appearance.

Chromatictarget patches embedded in a chromatically variegated surround appear less saturated than when they are embedded in an achromatic uniform surround (Brown & MacLeod, 1997), which can be construed as either a form of gamut expansion for targets on uniform surrounds or as a form of gamut compression for targets on variegated surrounds.Ekroll, Faul, and Niederée (2004) suggested that the difference in perceived chromaticity on the two surrounds is caused by a layered scene decomposition, wherein the increased saturation of targets on homogenous surrounds is attributed to a decomposition of a target patch into a chromatically saturated transparent layer overlying an achromatic background.Here, we report asymmetric matching data that show the perceived chromaticity difference observed on the two surrounds depends on the particular direction of chromatic variation applied to the variegated surround. If the chromatic variegated surround has the same or a similar hue to that of the target and the saturation variation of the surround is large compared to the saturation of the target the gamut expansion effect is also large. However, if the variegated surround has a different hue than the hue of the target, the perceived chromaticity difference is small and largely does not depend on the variation in saturation of the surround. These results suggest that a layered scene representation cannot fully explain the gamut expansion effect and suggest that a chromatically tuned contrast gain control mechanism may contribute to the difference in perceived color of targets on achromatic homogeneous surrounds and chromatically variegated surrounds.

Alert Response to Motion Onset in the Retina

Previous studies have shown that motion onset is very effective at capturing attention and is more salient than smooth motion. Here, we find that this salience ranking is present already in the firing rate of retinal ganglion cells. By stimulating the retina with a bar that appears, stays still, and then starts moving, we demonstrate that a subset of salamander retinal ganglion cells, fast OFF cells, responds significantly more strongly to motion onset than to smooth motion. We refer to this phenomenon as an alert response to motion onset. We develop a computational model that predicts the time-varying firing rate of ganglion cells responding to the appearance, onset, and smooth motion of a bar. This model, termed the adaptive cascade model, consists of a ganglion cell that receives input from a layer of bipolar cells, represented by individual rectified subunits. Additionally, both the bipolar and ganglion cells have separate contrast gain control mechanisms. This model captured the responses to our different motion stimuli over a wide range of contrasts, speeds, and locations. The alert response to motion onset, together with its computational model, introduces a new mechanism of sophisticated motion processing that occurs early in the visual system.

Multifocal visual evoked potential responses to pattern-reversal, pattern-onset, pattern-offset, and sparse pulse stimuli

The purpose of the present study was to compare standard multifocal visual evoked potential (mfVEP) pattern-reversal responses with those produced by pattern-onset, pattern-offset, and pulsed pattern stimuli. mfVEP recordings were obtained from five normal subjects using VERIS and a 4-electrode array. The standard reversal stimulus had 2(15) m-sequence steps (7.5-min duration). Pattern-onset and -offset responses were evaluated using sequences that all had 32 frames per m-step and 2(10) total steps (7.5 min); but the duration of the contrast step varied so that it was 1, 2, 4, 8, 12, or 16 of the 32 frames. The same series was also inverted so that adapting contrast was high and the stimulus step began with a contrast decrement. The effect of temporal sparseness was studied with positive contrast pulses (two-frame duration) within 16, 20, 24, 28, or 32 frames per m-step (all had 2(11) total steps). Four stimulus locations were isolated to study the effect of spatial sparseness. Standard mfVEP reversal responses were virtually identical to onset responses throughout the field, but ~3.5 times smaller. Responses to pattern onset were about twice as large as those for offset, especially in the lower hemifield, irrespective of adapting contrast level. Offset responses exhibited a different waveform compared with reversal, onset, or brief pulse responses. Though temporally sparse pattern-pulse responses were ~3.5 times larger than standard reversal responses, there was no improvement in signal-to-noise ratio (SNR). However, spatial isolation increased SNR by 22% for reversal responses and by 62% for temporally sparse pulses. Temporally sparse pattern-pulse stimuli do not improve mfVEP SNR unless they are also spatially sparse, suggesting that lateral inhibitory mechanisms have a greater impact than temporal contrast gain control mechanisms. The mfVEP response depends on the polarity of a contrast step, irrespective of the state of contrast adaptation.

Motion transparency from opposing luminance modulated and contrast modulated gratings

Two luminance gratings of identical orientation and opposite directions of motion are seen as moving across one another (i.e. moving transparently) only if they differ in spatial frequency (SF) by a factor of four or more. Identical SF gratings produce counter-phase flicker. This suggests that opposite motions cancel each other at the level of motion detection. Here we show that motion transparency is perceived with two gratings of the same SF and orientation moving in opposite directions, when one grating is a first-order, luminance modulated (LM) stimulus and the other is a second-order, contrast modulated (CM) stimulus. Participants were presented with various combinations of LM and CM gratings. In experiment 1, the test stimulus contained the summation of oppositely moving LM and CM gratings. In order to assess the simultaneous perception of both motions, we used a paradigm where observers were required to discriminate the direction of motion of each component from counter-phase flicker. Results show that observers can accurately discriminate both LM and CM directions of motion in a transparent configuration. We next measured the effect of varying the contrast/modulation depth of LM and CM gratings on the perception of transparency. The perception of motion transparency depends upon the relative contrast/modulation depth of the component gratings: raising the contrast of the LM component necessitates a greater modulation depth for the CM component if motion transparency is to be perceived. Our results are consistent with a motion system comprised of two separate, but not wholly independent, pathways for the encoding of LM and CM signals. We hypothesise that the observed contrast dependence is the result of contrast gain control mechanisms that receive inputs from separate motion systems.

Systematic misestimation in a vernier task arising from contrast mismatch

Luminance signals mediated by the magnocellular (MC) pathway play an important role in vernier tasks. MC ganglion cells show a phase advance in their responses to sinusoidal stimuli with increasing contrast due to contrast gain control mechanisms. If the phase information in MC ganglion cell responses were utilized by central mechanisms in vernier tasks, one might expect systematic errors caused by the phase advance. This systematic error may contribute to the contrast paradox phenomenon, where vernier performance deteriorates, rather than improves, when only one of the target pair increases in contrast. Vernier psychometric functions for a pair of gratings of mismatched contrast were measured to seek such misestimation. In associated electrophysiological experiments, MC and parvocellular (PC) ganglion cells' responses to similar stimuli were measured to provide a physiological reference. The psychophysical experiments show that a high-contrast grating is perceived as phase advanced in the drift direction compared to a low-contrast grating, especially at a high drift rate (8 Hz). The size of the phase advance was comparable to that seen in MC cells under similar stimulus conditions. These results are consistent with the MC pathway supporting vernier performance with achromatic gratings. The shifts in vernier psychometric functions were negligible for pairs of chromatic gratings under the conditions tested here, consistent with the lack of phase advance both in responses of PC ganglion cells and in frequency-doubled chromatic responses of MC ganglion cells.

Binocular contrast interactions in two-frame motion discrimination

Previous studies have shown that two-frame motion detection thresholds are elevated if one frame's contrast is raised, despite the increase in average contrast--the "contrast paradox". In this study, we investigated if such contrast interactions occurred at a monocular or binocular site of visual processing. Two-frame motion direction discrimination thresholds were measured for motion frames that were presented binocularly, dichoptically or interocularly. Thresholds for each presentation condition were measured for motion frames that comprised either matched or unmatched contrasts. The results showed that contrast mechanisms producing the contrast paradox combine contrast signals from both eyes prior to motion computation. Furthermore, the results are consistent with the existence of monocular and binocular contrast gain control mechanisms that coexist either as combined or independent systems.

Stevens's brightness law, contrast gain control, and edge integration in achromatic color perception: a unified model

The brightness of an isolated test patch is related to its luminance by a power law having an exponent of about 1/3, a result known as Stevens's brightness law. The brightness law exponent characterizes the rate at which brightness grows with luminance and can thus be thought of as an "exponential" gain factor. We studied changes in this gain factor for incremental and decremental test squares as a function of the size of a surrounding frame of homogeneous luminance. For incremental targets, the gain decreased as an approximately linear function of the frame width. For decremental targets, the gain increased as an approximately linear function of the frame width. We modeled the brightness of the frame-embedded target with a quantitative theory based on the assumption that the target brightness is determined by the sum of achromatic color induction signals originating from the inner and outer edges of the surround, a theory that has previously been used to account for the results of several other brightness matching experiments. To account for the frame-width-dependent gain changes observed in the present study, we elaborate this edge integration theory by proposing the existence of a cortical contrast gain control mechanism by which the gains applied to neural edge detectors are influenced by the responses of other edge detectors responding to the nearby edges.

Modeling spatial integration in the ocular following response using a probabilistic framework

The machinery behind the visual perception of motion and the subsequent sensori-motor transformation, such as in ocular following response (OFR), is confronted to uncertainties which are efficiently resolved in the primate’s visual system. We may understand this response as an ideal observer in a probabilistic framework by using Bayesian theory [Weiss, Y., Simoncelli, E.P., Adelson, E.H., 2002. Motion illusions as optimal percepts. Nature Neuroscience, 5(6), 598–604, doi:10.1038/nn858] which we previously proved to be successfully adapted to model the OFR for different levels of noise with full field gratings. More recent experiments of OFR have used disk gratings and bipartite stimuli which are optimized to study the dynamics of center–surround integration. We quantified two main characteristics of the spatial integration of motion: (i) a finite optimal stimulus size for driving OFR, surrounded by an antagonistic modulation and (ii) a direction selective suppressive effect of the surround on the contrast gain control of the central stimuli [Barthélemy, F.V., Vanzetta, I., Masson, G.S., 2006. Behavioral receptive field for ocular following in humans: dynamics of spatial summation and center–surround interactions. Journal of Neurophysiology, (95), 3712–3726, doi:10.1152/jn.00112.2006]. Herein, we extended the ideal observer model to simulate the spatial integration of the different local motion cues within a probabilistic representation. We present analytical results which show that the hypothesis of independence of local measures can describe the spatial integration of the motion signal. Within this framework, we successfully accounted for the contrast gain control mechanisms observed in the behavioral data for center–surround stimuli. However, another inhibitory mechanism had to be added to account for suppressive effects of the surround.

The perceived contrast of texture patches embedded in natural images

The visibility of an isolated simple stimulus is known to depend on its contrast. However, when such a stimulus is surrounded by other geometrically-simple stimuli, its perceived contrast can change markedly. Here, we examined whether such effects contribute to our perception of contrasts when we view real world scenes. We show that the perceived contrast of a luminance texture patch is suppressed when it is surrounded by images of real world scenes. We also show that the amount of this suppression depends on the spatial statistics of the surrounding images. We manipulated the second-order statistics of the images and found minimal suppression of perceived contrast at “un-natural” image statistics and maximal suppression at the characteristic statistics of natural images. This suggests that contrast gain control mechanisms in our visual system are optimally engaged when we view real world images.

The Statistical Computation Underlying Contrast Gain Control

In the early visual system, a contrast gain control mechanism sets the gain of responses based on the locally prevalent contrast. The measure of contrast used by this adaptation mechanism is commonly assumed to be the standard deviation of light intensities relative to the mean (root-mean-square contrast). A number of alternatives, however, are possible. For example, the measure of contrast might depend on the absolute deviations relative to the mean, or on the prevalence of the darkest or lightest intensities. We investigated the statistical computation underlying this measure of contrast in the cat's lateral geniculate nucleus, which relays signals from retina to cortex. Borrowing a method from psychophysics, we recorded responses to white noise stimuli whose distribution of intensities was precisely varied. We varied the standard deviation, skewness, and kurtosis of the distribution of intensities while keeping the mean luminance constant. We found that gain strongly depends on the standard deviation of the distribution. At constant standard deviation, moreover, gain is invariant to changes in skewness or kurtosis. These findings held for both ON and OFF cells, indicating that the measure of contrast is independent of the range of stimulus intensities signaled by the cells. These results confirm the long-held assumption that contrast gain control computes root-mean-square contrast. They also show that contrast gain control senses the full distribution of intensities and leaves unvaried the relative responses of the different cell types. The advantages to visual processing of this remarkably specific computation are not entirely known.

Local luminance and contrast in natural images

Within natural images there is substantial spatial variation in both local contrast and local luminance. Understanding the statistics of these variations is important for understanding the dynamics of receptive field stimulation that occur under natural viewing conditions and for understanding the requirements for effective luminance and contrast gain control. Local luminance and contrast were measured in a large set of calibrated 12-bit gray-scale natural images, for a number of analysis patch sizes. For each image and patch size we measured the range of contrast, the range of luminance, the correlation in contrast and luminance as a function of the distance between patches, and the correlation between contrast and luminance within patches. The same analyses were also performed on hand segmented regions containing only “sky”, “ground”, “foliage”, or “backlit foliage”. Within the typical image, the 95% range (2.5–97.5 percentile) for both local luminance and local contrast is somewhat greater than a factor of 10. The correlation in contrast and the correlation in luminance diminish rapidly with distance, and the typical correlation between luminance and contrast within patches is small (e.g., −0.2 compared to −0.8 for 1/ f noise). We show that eye movements are frequently large enough that there will be little correlation in the contrast or luminance on a receptive field from one fixation to the next, and thus rapid contrast and luminance gain control are essential. The low correlation between local luminance and contrast implies that efficient contrast gain control mechanisms can operate largely independently of luminance gain control mechanisms.

Contrast conservation in human vision

Visual experience, which is defined by brief saccadic sampling of complex scenes at high contrast, has typically been studied with static gratings at threshold contrast. To investigate how suprathreshold visual processing is related to threshold vision, we tested the temporal integration of contrast in the presence of large, sudden changes in the stimuli such occur during saccades under natural conditions. We observed completely different effects under threshold and suprathreshold viewing conditions. The threshold contrast of successively presented gratings that were either perpendicularly oriented or of inverted phase showed probability summation, implying no detectable interaction between independent visual detectors. However, at suprathreshold levels we found complete algebraic summation of contrast for stimuli longer than 53 ms. The same results were obtained during sudden changes between random noise patterns and between natural scenes. These results cannot be explained by traditional contrast gain-control mechanisms or the effect of contrast constancy. Rather, at suprathreshold levels, the visual system seems to conserve the contrast information from recently viewed images, perhaps for the efficient assessment of the contrast of the visual scene while the eye saccades from place to place.

Visual physiology of the lateral geniculate nucleus in two species of new world monkey: Saimiri sciureus and Aotus trivirgatis.

1. Visual responses were recorded from neurones in the magnocellular and parvocellular layers of the lateral geniculate nucleus (LGN) of the thalamus in two species of New World monkeys - the diurnal squirrel monkey (Saimiri sciureus) and the nocturnal owl monkey (Aotus trivirgatis). Recording sites were reconstructed in postmortem tissue and comparisons were made between the response properties of magnocellular and parvocellular neurones. 2. Receptive fields were characterized with both white noise and drifting gratings. We found that most of the differences between magnocellular and parvocellular neurones that have been described in the macaque monkey hold for the squirrel monkey and owl monkey. In squirrel monkey and owl monkey, receptive fields of magnocellular neurones were larger than those of parvocellular neurones at similar eccentricities. Although visual responses in the owl monkey were significantly slower than in the squirrel monkey, in both species magnocellular neurones differed from parvocellular neurones in that their responses (1) had higher contrast gains, (2) tended to peak at higher temporal frequencies (but with considerable overlap), (3) had shorter response latencies, and (4) were more transient. 3. The strength of a neurone's receptive-field surround was assessed by comparing neuronal responses to gratings of optimal spatial frequency with responses to gratings of low spatial frequency. Using this approach, receptive-field surrounds were found to be equally strong on average for magnocellular and parvocellular neurones. 4. Spatial summation, as measured by a null test, was linear for all magnocellular and parvocellular cells tested; that is, Y cells were not observed in either species. Finally, most magnocellular neurones showed a contrast gain control mechanism, although this was not seen for parvocellular neurones.

Visual signal detection in structured backgrounds. II. Effects of contrast gain control, background variations, and white noise.

Studies of visual detection of a signal superimposed on one of two identical backgrounds show performance degradation when the background has high contrast and is similar in spatial frequency and/or orientation to the signal. To account for this finding, models include a contrast gain control mechanism that pools activity across spatial frequency, orientation and space to inhibit (divisively) the response of the receptor sensitive to the signal. In tasks in which the observer has to detect a known signal added to one of M different backgrounds grounds due to added visual noise, the main sources of degradation are the stochastic noise in the image and the suboptimal visual processing. We investigate how these two sources of degradation (contrast gain control and variations in the background) interact in a task in which the signal is embedded in one of M locations in a complex spatially varying background (structured background). We use backgrounds extracted from patient digital medical images. To isolate effects of the fixed deterministic background (the contrast gain control) from the effects of the background variations, we conduct detection experiments with three different background conditions: (1) uniform background, (2) a repeated sample of structured background, and (3) different samples of structured background. Results show that human visual detection degrades from the uniform background condition to the repeated background condition and degrades even further in the different backgrounds condition. These results suggest that both the contrast gain control mechanism and the background random variations degrade human performance in detection of a signal in a complex, spatially varying background. A filter model and added white noise are used to generate estimates of sampling efficiencies, an equivalent internal noise, an equivalent contrast-gain-control-induced noise, and an equivalent noise due to the variations in the structured background.

Contrast gain control and fine spatial discriminations.

The effect of contrast gain control mechanisms on discrimination between highly similar simple and complex stimuli is examined, with a focus on how discrimination accuracy changes as a function of the contrast of stimulus components. Two models of contrast gain control are evaluated. In both, the response of each pathway is attenuated by a factor determined by the total activity in a large pool of pathways. One model bases attenuation on the sum of linear filter responses within this pool; the other, based on Heeger's contrast energy-driven model [J. Neurophysiol. 70, 1985 (1993)], uses squared filter response. Predictions generated from the models are compared with data from experiments reported here and from the literature. Predictions are made for simple grafting stimuli of different sizes and for stimuli to which a second grafting component is added either as a second cue or as a mask. With one exception, predictions of the models agree closely with each other and with the data. The exception is a masking study that differentiates the models and supports the filter-driven model over the energy-driven model.

Temporal impulse response functions for luminance and colour during saccades.

Previous work has shown that during saccadic eye movements, contrast sensitivity for low spatial frequency patterns modulated in luminance is selectively reduced by up to one logarithmic unit, while high spatial frequency patterns, and equiluminant patterns of all spatial frequencies are not suppressed at all [Burr et al. (1994). Nature, 371, 511-513]. Here we study the temporal characteristics for sensitivity to luminance and chromatic patterns during saccades, using the two-pulse summation technique. Sensitivity was measured for detecting two successive pulses as a function of stimulus-onset asynchrony, during normal viewing and during saccades. Impulse response functions were estimated from the summation data, for all conditions. For equiluminance, the functions were monophasic during normal viewing and saccades. For luminance modulation, the impulse response functions were di-phasic in both normal viewing and saccades. However, during saccades the impulse responses were faster in normal viewing. This result is consistent with the suggestion that saccadic suppression is mediated by contrast gain control mechanisms, known to occur in M-cells but not P-cells.