Primary Visual Cortex Contributes to Color Constancy by Predicting Rather than Discounting the Illuminant.

The double-opponent (DO) color-sensitive cells in the primary visual cortex (V1) of the human visual system (HVS) have long been recognized as the physiological basis of color constancy. In this work we propose a new color constancy model by imitating the functional properties of the HVS from the single-opponent (SO) cells in the retina to the DO cells in V1 and the possible neurons in the higher visual cortexes. The idea behind the proposed double-opponency based color constancy (DOCC) model originates from the substantial observation that the color distribution of the responses of DO cells to the color-biased images coincides well with the vector denoting the light source color. Then the illuminant color is easily estimated by pooling the responses of DO cells in separate channels in LMS space with the pooling mechanism of sum or max. Extensive evaluations on three commonly used datasets, including the test with the dataset dependent optimal parameters, as well as the intra- and inter-dataset cross validation, show that our physiologically inspired DOCC model can produce quite competitive results in comparison to the state-of-the-art approaches, but with a relative simple implementation and without requiring fine-tuning of the method for each different dataset.

Event Abstract Back to Event Color Constancy of V1 Double Opponent Cells to Natural Images The human visual system perceives colors of objects as largely independent of the lighting conditions even though the spectral composition of the incident light, and thus of the light reflected off objects and reaching the eye, can be very different under different types of lighting (such as midday sun, sunset, fluorescent or incandescent light). The ability to maintain constant perception of object color is called color constancy, and its neural basis remains unknown. Neurons in the retina and lateral geniculate nucleus (LGN) do not show response properties consistent with color constancy, indicating that this computation is performed at a subsequent stage of neural processing. The critical computation underlying color constancy is a comparison of the relative cone activations across visual space. Recent studies have confirmed the existence of double-opponent cells in V1 that, in principle, have the appropriate receptive field structure to contribute to color constancy: they are sensitive not to absolute cone responses but rather to differences in cone responses both across cone types (cone opponency) and across space (spatial opponency). To test the hypothesis that double-opponent cells contribute to color constancy we compare the color constancy of experimentally characterized V1 neurons to that of model LGN neurons and model V1 neurons constructed on the basis of physiological measurements. We use experimentally measured receptive fields of V1 double-opponent cells in alert macaque monkeys (Conway, 2001; Conway and Livingstone, 2006). To model the responses of a neuron population tiling a portion of the visual field, each measured receptive field was spatially convolved with natural images both before and after a simulated change in lighting conditions (modeled by a von Kries transformation). Natural images were taken from Olmos and Kingdom (2004). We show that, due to their double-opponent structure, both the physiologically characterized and model V1 cells show stronger color constancy in response to natural images than the model LGN neurons. We quantify the improvement across a range of different recorded cells and natural images. We further quantify the effects of receptive-field shape and cone-type balance upon the color constancy of V1 double-opponent cells by comparing the recorded V1 cells to model V1 cells with different surround shapes and balances of L, M, and S cone contributions. Finally, we consider the effect of contrast normalization by the responses of neighboring neurons and generate experimental predictions for the influence of contrast normalization on color constancy. Acknowledgments: This work was supported by the Whitehall Foundation (BC); and the Sloan Foundation, UC Davis, and a UC Davis Ophthalmology Research to Prevent Blindness grant (MG,DF).

We investigate the processing and representation of static visual patterns in the early visual system of mammals (especially cats and primates). We demonstrate that neurophysiological and anatomical findings can motivate theoretical considerations about the neural processing and vice versa. We explore “How?” and “Why?” questions in a close connection to each other. Methodologically this means using biologically detailed “bottom-up” computational models and abstract “top-down” models in parallel or in combination. Specifically, we focus on the contrastand orientation-processing in the primary visual cortex (V1) with a strong emphasis on the dynamics of the neural activity and synapses. We consider neural dynamics on three different time scales: (i) the fast time evolution of the cortical activity with a time constant of 16 20 msec; (ii) the intermediate modulation of the recurrent cortical competition strength with a time constant in the order of 100 200 msec (the approximate length of a fixation period); (iii) contrast adaptation by the slow modulation of the dynamic nature of the synaptic transmission with a time constant of 5 10 sec. Firstly, we explore how orientation selectivity could be generated in the primary visual cortex (V1) (chapters 2, 3). Orientation selectivity is a remarkable and well-explored feature of the simple cells in V1. However, there is still considerable debate about the neurophysiological and anatomical origin of the highly feature selective response of these cells. The major question concerns the extent to which the simple cell properties are determined by the structure of their feed-forward connectivity versus the recurrent projections. In contrast to previous models, in which the initial orientation bias is generated by convergent geniculate (feed-forward) input to the simple cells, and subsequently sharpened by the lateral circuits, our approach is based on anisotropic intracortical excitatory connections. We study the hypothesis that these recurrent projections provide both the initial orientation bias and its subsequent amplification and therefore orientation selectivity is generated purely intracortically. Our computational study shows that indeed the “intracortical hypothesis” is a plausible alternative to the other existing hypotheses. The model predicts that the dynamics of the orientation tuning could be indicative of the underlying neural mechanism. Therefore we investigate recurrent dynamics in a cortical orientation hypercolumn in a more biologically detailed statistical neural field model (chapter 3). Secondly, we study why the recurrent cortical re-processing of the feed-forward input is important for the representation of the image projected on the retina (chapter 4). We propose that the recurrent lateral connections implement competition between orientation selective simple cells with overlapping receptive fields. Then, we introduce the concept of “dynamic coding”, and investigate the short term dynamics of the recurrent competition in the primary visual cortex in terms of information processing. We find that information transfer is optimal in any increasing time window after stimulus onset if the recurrent cortical amplification decreases. In the model, the initially strong cortical competition decreases, and the role of the geniculate origin feed-forward projections becomes more important. These geniculo-cortical projections carry a topographic representation of the image projected to the retina. Motivated by information theory, our results offer a compromise between the “feed-forward” and the “recurrent” hypotheses for orientation selectivity. We suggest that both are valid, however, in different phases of the cortical processing during a fixation period. In the initial phase of processing, the recurrent competition is strong, and the salient orientation is signaled in a winner-take-all fashion. In the second phase, cortical competition becomes weaker, allowing the detection of multiple orientations. A detailed computational model provides experimentally testable

There are two distinct classes of cells in the primary visual cortex (V1): simple cells and complex cells. One defining feature of complex cells is their spatial phase invariance; they respond strongly to oriented grating stimuli with a preferred orientation but with a wide range of spatial phases. A classical model of complete spatial phase invariance in complex cells is the energy model, in which the responses are the sum of the squared outputs of two linear spatially phase-shifted filters. However, recent experimental studies have shown that complex cells have a diverse range of spatial phase invariance and only a subset can be characterized by the energy model. While several models have been proposed to explain how complex cells could learn to be selective to orientation but invariant to spatial phase, most existing models overlook many biologically important details. We propose a biologically plausible model for complex cells that learns to pool inputs from simple cells based on the presentation of natural scene stimuli. The model is a three-layer network with rate-based neurons that describes the activities of LGN cells (layer 1), V1 simple cells (layer 2), and V1 complex cells (layer 3). The first two layers implement a recently proposed simple cell model that is biologically plausible and accounts for many experimental phenomena. The neural dynamics of the complex cells is modeled as the integration of simple cells inputs along with response normalization. Connections between LGN and simple cells are learned using Hebbian and anti-Hebbian plasticity. Connections between simple and complex cells are learned using a modified version of the Bienenstock, Cooper, and Munro (BCM) rule. Our results demonstrate that the learning rule can describe a diversity of complex cells, similar to those observed experimentally.

There are two distinct classes of cells in the primary visual cortex (V1): simple cells and complex cells. One defining feature of complex cells is their spatial phase invariance; they respond strongly to oriented grating stimuli with a preferred orientation but with a wide range of spatial phases. A classical model of complete spatial phase invariance in complex cells is the energy model, in which the responses are the sum of the squared outputs of two linear spatially phase-shifted filters. However, recent experimental studies have shown that complex cells have a diverse range of spatial phase invariance and only a subset can be characterized by the energy model. While several models have been proposed to explain how complex cells could learn to be selective to orientation but invariant to spatial phase, most existing models overlook many biologically important details. We propose a biologically plausible model for complex cells that learns to pool inputs from simple cells based on the presentation of natural scene stimuli. The model is a three-layer network with rate-based neurons that describes the activities of LGN cells (layer 1), V1 simple cells (layer 2), and V1 complex cells (layer 3). The first two layers implement a recently proposed simple cell model that is biologically plausible and accounts for many experimental phenomena. The neural dynamics of the complex cells is modeled as the integration of simple cells inputs along with response normalization. Connections between LGN and simple cells are learned using Hebbian and anti-Hebbian plasticity. Connections between simple and complex cells are learned using a modified version of the Bienenstock, Cooper, and Munro (BCM) rule. Our results demonstrate that the learning rule can describe a diversity of complex cells, similar to those observed experimentally.

We recorded the responses of direction-selective simple and complex cells in the primary visual cortex (V1) of anesthetized, paralyzed macaque monkeys. When studied with sine-wave gratings, almost all simple cells in V1 had responses that were separable for spatial and temporal frequency: the preferred temporal frequency did not change and preferred speed decreased as a function of the spatial frequency of the grating. As in previous recordings from the middle temporal visual area (MT), approximately one-quarter of V1 complex cells had separable responses to spatial and temporal frequency, and one-quarter were "speed tuned" in the sense that preferred speed did not change as a function of spatial frequency. Half fell between these two extremes. Reducing the contrast of the gratings caused the population of V1 complex cells to become more separable in their tuning for spatial and temporal frequency. Contrast dependence is explained by the contrast gain of the neurons, which was relatively higher for gratings that were either both of high or both of low temporal and spatial frequency. For stimuli that comprised two spatially superimposed sine-wave gratings, the preferred speeds and tuning bandwidths of V1 neurons could be predicted from the sum of the responses to the component gratings presented alone, unlike neurons in MT that showed nonlinear interactions. We conclude that spatiotemporal modulation of contrast gain creates speed tuning from separable inputs in V1 complex cells. Speed tuning in MT could be primarily inherited from V1, but processing that occurs after V1 and possibly within MT computes selective combinations of speed-tuned signals of special relevance for downstream perceptual and motor mechanisms.

Mechanisms of Neuronal Computation in Mammalian Visual Cortex

Are cortical models really bound by the "binding problem"?

Rules by which V1 neurons combine signals originating in the cone photoreceptors are poorly understood. We measured cone inputs to V1 neurons in awake, fixating monkeys with white-noise analysis techniques that reveal properties of light responses not revealed by purely linear models used in previous studies. Simple cells were studied by spike-triggered averaging that is robust to static nonlinearities in spike generation. This analysis revealed, among heterogeneously tuned neurons, two relatively discrete categories: one with opponent L- and M-cone weights and another with nonopponent cone weights. Complex cells were studied by spike-triggered covariance, which identifies features in the stimulus sequence that trigger spikes in neurons with receptive fields containing multiple linear subunits that combine nonlinearly. All complex cells responded to nonopponent stimulus modulations. Although some complex cells responded to cone-opponent stimulus modulations too, none exhibited the pure opponent sensitivity observed in many simple cells. These results extend the findings on distinctions between simple and complex cell chromatic tuning observed in previous studies in anesthetized monkeys.

Event Abstract Back to Event Hierarchical novelty-familiarity representation in the visual cortex Olshausen and Field (1996, 1997) and Rao and Ballard (1999) have shown that the statistics of natural images can explain some receptive field (RF) properties of simple cells in V1. However, only small patches of images were used, with many cells reading out multiple statistics from each RF. Furthermore, no attempt was made to evaluate how well the read-outs thus obtained encoded the stimuli. Predictive coding used by Rao and Ballard is an instance of a wider class of generative Bayesian models that has been suggested to serve as an organizing principle for the entire cortical hierarchy (Lee and Mumford, 2003; Friston, 2005). Thus, we investigate here how good predictive coding actually is at coding whole images using a natural topographic connectivity, where no two cells have exactly the same RF, but adjacent cells' RFs have strong overlaps. In predictive coding, feedback from higher areas carries expectations of lower-level activity (familiarity signal), whereas the feedforward (novelty) signals carry discrepancies between the expectations and the stimuli. Visual recognition becomes an iterative process relaxing to a solution that matches experience with sensory input. We use two interconnected populations of neurons, one coding for the familiarity, and the other, for novelty. These form one of multiple levels in a hierarchical processing structure. In contrast to Rao and Ballard, in our model top-down effects are confined to a single level. We suggest that this limited feedback is preferable for biologically compatible fast recognition. We use 1000 natural images (Van Hateren and van der Schaaf, 1998) for unsupervised learning of the synaptic efficacies of two visual processing levels. The coding performance is then evaluated on a set of 200 different images from the same database by reconstructing the stimulus based on the internal code in our generative model and directly comparing the prediction to the actual image. Despite a compression factor of 4 for each level, the image reconstruction quality on the test images is quite good and strongly exceeds that of local averaging, implying that the learning results in the extraction of features characteristic of the set of natural images as a whole. With our model, we have found that the extra-classical RF effect of endstopping can arise due to the topographic connectivity from interactions within the first processing level without the need for feedback from the next level, as in Rao and Ballard. Furthermore, the effective RF's after learning resemble those of simple cells in V1 and learning with the topographic map leads to a global organization of the RF's. Finally, the proposed architecture allows for the simultaneous, but separate, representation of familiarity and novelty in the visual cortex. The novelty signal could produce read-out to higher processing centers, activating them if a localized novel signal, such as a predator in a serene environment, suddenly appears in a familiar image. Thus, our implementation of predictive coding could be an effective way for the visual system to combine fast hierarchical visual representation with the interaction with higher information-processing areas in the brain. Conference: Computational and systems neuroscience 2009, Salt Lake City, UT, United States, 26 Feb - 3 Mar, 2009. Presentation Type: Poster Presentation Topic: Poster Presentations Citation: (2009). Hierarchical novelty-familiarity representation in the visual cortex. Front. Syst. Neurosci. Conference Abstract: Computational and systems neuroscience 2009. doi: 10.3389/conf.neuro.06.2009.03.096 Copyright: The abstracts in this collection have not been subject to any Frontiers peer review or checks, and are not endorsed by Frontiers. They are made available through the Frontiers publishing platform as a service to conference organizers and presenters. The copyright in the individual abstracts is owned by the author of each abstract or his/her employer unless otherwise stated. Each abstract, as well as the collection of abstracts, are published under a Creative Commons CC-BY 4.0 (attribution) licence (https://creativecommons.org/licenses/by/4.0/) and may thus be reproduced, translated, adapted and be the subject of derivative works provided the authors and Frontiers are attributed. For Frontiers’ terms and conditions please see https://www.frontiersin.org/legal/terms-and-conditions. Received: 02 Feb 2009; Published Online: 02 Feb 2009. Login Required This action requires you to be registered with Frontiers and logged in. To register or login click here. Abstract Info Abstract The Authors in Frontiers Google Google Scholar PubMed Related Article in Frontiers Google Scholar PubMed Abstract Close Back to top Javascript is disabled. Please enable Javascript in your browser settings in order to see all the content on this page.

This work describes a novel model for contrast detection inspired by regular spatial structure of simple cells in the visual cortex of mammals. The model employs major cortical connections in the laminar circuitry of extrastriate visual pathways and the area V1 of the visual cortex. The model employs two new features, namely, the iterative processing of visual inputs and amplification of tuned responses of spatially close simple cells in V1. Results show that after several iterations the processing converges to a stable solution while making edge detection being largely contrast independent. The model suppresses spurious noise in the vicinity of contrastive luminance changes while enhancing isolated low-intensity changes. We demonstrate the capabilities of the model by processing synthetic as well as natural images and compare our results to Canny edge detector.

Standing Waves and Traveling Waves Distinguish Two Circuits in Visual Cortex

Frontiers Events is a rapidly growing calendar management system dedicated to the scheduling of academic events. This includes announcements and invitations, participant listings and search functionality, abstract handling and publication, related events and post-event exchanges. Whether an organizer or participant, make your event a Frontiers Event!

Figure and Ground in the Visual Cortex: V2 Combines Stereoscopic Cues with Gestalt Rules

Color constancy was investigated in behavioral training experiments on colors ranging from blue to yellow, located in the color space close to Planck's locus representing the main changes in natural skylight. Two individual goldfish were trained to peck at a test field of medium hue out of a series of 13-15 yellowish and bluish test fields presented simultaneously on a black background. During training the tank in which the fish were swimming freely was illuminated with white light. Correct choices were rewarded with food. During the tests differently saturated yellow or blue illumination was used. The degree of color constancy was inferred from the choice behavior under these illuminations. Perfect color constancy was found up to a certain degree of saturation of the colored light. Beyond this level test fields other than the training test field were chosen, indicating imperfect color constancy. Color constancy was quantified by applying color metrics on the basis of the goldfish cone sensitivity functions.