Complex Cell Receptive Fields Research Articles

Full-reference quality assessment of stereoscopic images by learning binocular visual properties.

Stereoscopic imaging technology has been growingly prevalent driven by both the entertainment industry and scientific applications in today's world. But objective quality assessment of stereoscopic images is a challenging task. In this paper, we propose a novel stereoscopic image quality assessment (SIQA) method by jointly considering monocular perception and binocular interaction. As the most significant contribution of this study, binocular perceptual properties of simple and complex cells are considered for full-reference (FR) SIQA. Specifically, the proposed scheme first simulates the receptive fields of simple cells (one class of V1 neurons) using a push-pull combination of receptive fields response, which is used to represent a monocular cue. Further, the receptive fields of complex cells (the other class of V1 neurons) are simulated by using binocular energy response and binocular rivalry response, which are used to represent a binocular cue. Subsequently, various quality-aware features are extracted from the response of area V1 by calculating the self-weighted histogram of the local binary pattern on four types of feature maps of similarity measurement that will change in the presence of distortions. Finally, kernel ridge regression is used to simulate a nonlinear relationship between the quality-aware features and objective quality scores. The performance of our method is evaluated over popular stereoscopic image databases and shown to be competitive with the state-of-the-art FR SIQA algorithms.

Spatial phase sensitivity of complex cells in primary visual cortex depends on stimulus contrast

Neurons in primary visual cortex are classified as simple, which are phase sensitive, or complex, which are significantly less phase sensitive. Previously, we have used drifting gratings to show that the phase sensitivity of complex cells increases at low contrast and after contrast adaptation while that of simple cells remains the same at all contrasts (Cloherty SL, Ibbotson MR. J Neurophysiol 113: 434-444, 2015; Crowder NA, van Kleef J, Dreher B, Ibbotson MR. J Neurophysiol 98: 1155-1166, 2007; van Kleef JP, Cloherty SL, Ibbotson MR. J Physiol 588: 3457-3470, 2010). However, drifting gratings confound the influence of spatial and temporal summation, so here we have stimulated complex cells with gratings that are spatially stationary but continuously reverse the polarity of the contrast over time (contrast-reversing gratings). By varying the spatial phase and contrast of the gratings we aimed to establish whether the contrast-dependent phase sensitivity of complex cells results from changes in spatial or temporal processing or both. We found that most of the increase in phase sensitivity at low contrasts could be attributed to changes in the spatial phase sensitivities of complex cells. However, at low contrasts the complex cells did not develop the spatiotemporal response characteristics of simple cells, in which paired response peaks occur 180° out of phase in time and space. Complex cells that increased their spatial phase sensitivity at low contrasts were significantly overrepresented in the supragranular layers of cortex. We conclude that complex cells in supragranular layers of cat cortex have dynamic spatial summation properties and that the mechanisms underlying complex cell receptive fields differ between cortical layers.

Simulating binocular vision for no-reference 3D visual quality measurement.

Perceptual quality measurement of three-dimensional (3D) visual signals has become a fundamental challenge in 3D imaging fields. This paper proposes a novel no-reference (NR) 3D visual quality measurement (VQM) metric that uses simulations of the primary visual cortex (V1) of binocular vision. As the major technical contribution of this study, perceptual properties of simple and complex cells are considered for NR 3D-VQM. More specifically, the metric simulates the receptive fields of simple cells (one class of V1 neurons) using Gaussian derivative functions, and the receptive fields of complex cells (the other class of V1 neurons) using disparity energy responses and binocular rivalry responses. Subsequently, various quality-aware features are extracted from the primary visual cortex; these will change in the presence of distortions. Finally, those features are mapped to the subjective quality score of the distorted 3D visual signal by using support vector regression (SVR). Experiments on two publicly available 3D databases confirm the effectiveness of our proposed metric, compared to the relevant full-reference (FR) and NR metrics.

Perceptual quality measurement of 3D images based on binocular vision

Three-dimensional (3D) technology has become immensely popular in recent years and widely adopted in various applications. Hence, perceptual quality measurement of symmetrically and asymmetrically distorted 3D images has become an important, fundamental, and challenging issue in 3D imaging research. In this paper, we propose a binocular-vision-based 3D image-quality measurement (IQM) metric. Consideration of the 3D perceptual properties of the primary visual cortex (V1) and the higher visual areas (V2) for 3D-IQM is the major technical contribution to this research. To be more specific, first, the metric simulates the receptive fields of complex cells (V1) using binocular energy response and binocular rivalry response and the higher visual areas (V2) using local binary patterns features. Then, three similarity scores of 3D perceptual properties between the reference and distorted 3D images are measured. Finally, by using support vector regression, three similarity scores are integrated into an overall 3D quality score. Experimental results for two public benchmark databases demonstrate that, in comparison with most current 2D and 3D metrics, the proposed metric achieves significantly higher consistency in alignment with subjective fidelity ratings.

Second order receptive field properties of simple and complex cells support a new standard model of thalamocortical circuitry in V1.

[Hubel and Wiesel's (1962)][1] hierarchical model of simple and complex cell receptive fields (RFs) has shaped our understanding of the structure and function of the primary visual cortex (V1). They proposed that the characteristic elongated ON or OFF subregions of simple-cell RFs are constructed

Spatiotemporal receptive fields of cells in V1 are optimally shaped for stimulus velocity estimation

In recent literature, particularly interesting stimulus velocity-selective behaviors were found in the response properties of neurons belonging to the primary visual cortex (V1). In this work, 93 simple and complex cell receptive fields were obtained from the recordings of different experiments made on cats (DeAngelis, Blanche, Touryan) with reverse correlation and the spike-triggered covariance methods and then fitted with a three-dimensional Gabor model, so that cells are seen as minimizers of the Heisenberg uncertainty principle over both space and time. Analysis of the model parameters' cortical distribution suggests that V1 is spatiotemporally organized to maximize the resolution on the stimulus velocity measure.

A reaction-diffusion model to capture disparity selectivity in primary visual cortex.

Decades of experimental studies are available on disparity selective cells in visual cortex of macaque and cat. Recently, local disparity map for iso-orientation sites for near-vertical edge preference is reported in area 18 of cat visual cortex. No experiment is yet reported on complete disparity map in V1. Disparity map for layer IV in V1 can provide insight into how disparity selective complex cell receptive field is organized from simple cell subunits. Though substantial amounts of experimental data on disparity selective cells is available, no model on receptive field development of such cells or disparity map development exists in literature. We model disparity selectivity in layer IV of cat V1 using a reaction-diffusion two-eye paradigm. In this model, the wiring between LGN and cortical layer IV is determined by resource an LGN cell has for supporting connections to cortical cells and competition for target space in layer IV. While competing for target space, the same type of LGN cells, irrespective of whether it belongs to left-eye-specific or right-eye-specific LGN layer, cooperate with each other while trying to push off the other type. Our model captures realistic 2D disparity selective simple cell receptive fields, their response properties and disparity map along with orientation and ocular dominance maps. There is lack of correlation between ocular dominance and disparity selectivity at the cell population level. At the map level, disparity selectivity topography is not random but weakly clustered for similar preferred disparities. This is similar to the experimental result reported for macaque. The details of weakly clustered disparity selectivity map in V1 indicate two types of complex cell receptive field organization.

Complex Cell Descriptor Learning for Robust Object Recognition

An unsupervised algorithm is proposed for learning overcomplete topographic representations of nature image. Our method is based on Independent Component Analysis (ICA) model due to its superiority on feature extraction, and overcomes the weakness of traditional method in fast overcomplete learning. Besides, the learnt topographic representation, resembling receptive fields of complex cells, can be used as descriptors to extract invariant features. Recognition experiments on Caltech-101 dataset confirm that these complex cell descriptors are not only efficient in feature extraction but achieve comparable performances to traditional descriptors.

Complex cell receptive fields: evidence for a hierarchical mechanism

Simple cells in the primary visual cortex have segregated ON and OFF subregions in their receptive fields, while complex cells have overlapping ON and OFF subregions. These two cell types form the extremes at each end of a continuum of receptive field types. Hubel and Wiesel in 1962 suggested a hierarchical scheme of processing whereby spatially offset simple cells drive complex cells. Simple and complex cells are often classified by their responses to moving sine wave gratings: simple cells have oscillatory responses while complex cells produce unmodulated responses. Here, using moving gratings as stimuli, we show that a significant number of cells that display low levels of response modulation at high contrasts demonstrate high levels of response modulation at low contrasts. Most often a drifting low contrast grating generates a large phasic response at the fundamental frequency of the grating (F(1)) and a smaller but significant phasic response that is approximately 180 deg out-of-phase with the F(1) component. We present several models capable of capturing the effects of stimulus contrast on complex cell responses. The model that best reproduces our experimental results is a variation of the classical hierarchical model. In our model several spatially offset simple cells provide input to a complex cell, with each simple cell exhibiting a different contrast response function. At low contrasts only one of these simple cells is sufficiently excited to reveal its receptive field properties. As contrast is increased additional spatially offset simple cells with higher contrast thresholds add their responses to the overall spiking activity.

Statistical models of natural images and cortical visual representation.

A fundamental question in visual neuroscience is: Why are the response properties of visual neurons as they are? A modern approach to this problem emphasizes the importance of adaptation to ecologically valid input, and it proceeds by modeling statistical regularities in ecologically valid visual input (natural images). A seminal model was linear sparse coding, which is equivalent to independent component analysis (ICA), and provided a very good description of the receptive fields of simple cells. Further models based on modeling residual dependencies of the ''independent" components have later been introduced. These models lead to emergence of further properties of visual neurons: the complex cell receptive fields, the spatial organization of the cells, and some surround suppression and Gestalt effects. So far, these models have concentrated on the response properties of neurons, but they hold great potential to model various forms of inference and learning.

Spatiotemporal structure of complex cell receptive fields and influence of GABAergic inhibition

Spatiotemporal receptive field (RF) profiles were mapped with reverse correlation technique for complex cells in the striate cortex of cat. The RFs were constituted with ON and OFF subfields that overlapped much extensively in space and also largely in time. The subfields had spatial width of 1.0-4.3 degrees and temporal duration of 33-139 ms, whereas late responses were absent in most cases. When microiontophoresis of gamma-aminobutyric acid type A (GABAA) antagonist bicuculline was performed on the cells, little change occurred in width and onset time of the subfields, but the duration was prolonged in a subset of cells. These results suggest that intracortical inhibition may contribute to improve the accuracy of visual signals encoded in neuronal activities, but is unlikely critical for determining the primary structure of complex cell RF.

Internal Spatial Organization of Receptive Fields of Complex Cells in the Early Visual Cortex

The receptive fields of complex cells in the early visual cortex are economically modeled by combining outputs of a quadrature pair of linear filters. For actual complex cells, such a minimal model may be insufficient because many more simple cells are thought to make up a complex cell receptive field. To examine the minimalist model physiologically, we analyzed spatial relationships between the internal structure (subunits) and the overall receptive fields of individual complex cells by a two-stimulus interaction technique. The receptive fields of complex cells are more circular and only slightly larger than their subunits in size. In addition, complex cell subunits occupy spatial extents similar to those of simple cell receptive fields. Therefore in these respects, the minimalist schema is a fair approximation to actual complex cells. However, there are violations against the minimal model. Simple cell receptive fields have significantly fewer subregions than complex cell subunits and, in general, simple cell receptive fields are elongated more horizontally than vertically. This bias is absent in complex cell subunits and receptive fields. Thus simple cells cannot be equated to individual complex cell subunits and spatial pooling of simple cells may occur anisotropically to constitute a complex cell subunit. Moreover, when linear filters for complex cell subunits are examined separately for bright and dark responses, there are significant imbalances and position displacements between them. This suggests that actual complex cell receptive fields are constructed by a richer combination of linear filters than proposed by the minimalist model.

Standing Waves and Traveling Waves Distinguish Two Circuits in Visual Cortex

The visual cortex represents stimuli through the activity of neuronal populations. We measured the evolution of this activity in space and time by imaging voltage-sensitive dyes in cat area V1. Contrast-reversing stimuli elicit responses that oscillate at twice the stimulus frequency, indicating that signals originate mostly in complex cells. These responses stand clear of the noise, whose amplitude decreases as 1/frequency, and yield high-resolution maps of orientation preference and retinotopy. We first show how these maps are combined to yield the responses to focal, oriented stimuli. We then study the evolution of the oscillating activity in space and time. In the orientation domain, it is a standing wave. In the spatial domain, it is a traveling wave propagating at 0.2-0.5 m/s. These different dynamics indicate a fundamental distinction in the circuits underlying selectivity for position and orientation, two key stimulus attributes.

Complex cell pooling and the statistics of natural images

In previous work, we presented a statistical model of natural images that produced outputs similar to receptive fields of complex cells in primary visual cortex. However, a weakness of that model was that the structure of the pooling was assumed a priori and not learned from the statistical properties of natural images. Here, we present an extended model in which the pooling nonlinearity and the size of the subspaces are optimized rather than fixed, so we make much fewer assumptions about the pooling. Results on natural images indicate that the best probabilistic representation is formed when the size of the subspaces is relatively large, and that the likelihood is considerably higher than for a simple linear model with no pooling. Further, we show that the optimal nonlinearity for the pooling is squaring. We also highlight the importance of contrast gain control for the performance of the model. Our model is novel in that it is the first to analyze optimal subspace size and how this size is influenced by contrast normalization.

Critical contribution of slow-decay synaptic potential to complex-cell receptive field formation

Critical contribution of slow-decay synaptic potential to complex-cell receptive field formation

Sparse Approximation of Images Inspired from the Functional Architecture of the Primary Visual Areas

Several drawbacks of critically sampled wavelets can be solved by overcomplete multiresolution transforms and sparse approximation algorithms. Facing the difficulty to optimize such nonorthogonal and nonlinear transforms, we implement a sparse approximation scheme inspired from the functional architecture of the primary visual cortex. The scheme models simple and complex cell receptive fields through log-Gabor wavelets. The model also incorporates inhibition and facilitation interactions between neighboring cells. Functionally these interactions allow to extract edges and ridges, providing an edge-based approximation of the visual information. The edge coefficients are shown sufficient for closely reconstructing the images, while contour representations by means of chains of edges reduce the information redundancy for approaching image compression. Additionally, the ability to segregate the edges from the noise is employed for image restoration.

What simple and complex cells compute

Visual neuroscientists seek to answer two related questions. First, what does the visual system do? Second, how does it do it? While an answer to the second question is a description based on anatomy and biophysics, an answer to the first question is a description of computations performed on images. Important steps in providing this description were made with the publication of two classical studies on primary visual cortex by Movshon et al. (1978a,b). These studies have not aged in their readability, and do not require an interpretive key to be enjoyed by the contemporary reader. As an introduction, however, this Perspectives article briefly reviews the state of the art at the time of their appearance, their main findings, and their influence on three decades of subsequent investigations.

Spatial Structure of Complex Cell Receptive Fields Measured with Natural Images

Neuronal receptive fields (RFs) play crucial roles in visual processing. While the linear RFs of early neurons have been well studied, RFs of cortical complex cells are nonlinear and therefore difficult to characterize, especially in the context of natural stimuli. In this study, we used a nonlinear technique to compute the RFs of complex cells from their responses to natural images. We found that each RF is well described by a small number of subunits, which are oriented, localized, and bandpass. These subunits contribute to neuronal responses in a contrast-dependent, polarity-invariant manner, and they can largely predict the orientation and spatial frequency tuning of the cell. Although the RF structures measured with natural images were similar to those measured with random stimuli, natural images were more effective for driving complex cells, thus facilitating rapid identification of the subunits. The subunit RF model provides a useful basis for understanding cortical processing of natural stimuli.

Learning the invariance properties of complex cells from their responses to natural stimuli.

Neurons in primary visual cortex are typically classified as either simple or complex. Whereas simple cells respond strongly to grating and bar stimuli displayed at a certain phase and visual field location, complex cell responses are insensitive to small translations of the stimulus within the receptive field [Hubel & Wiesel (1962) J. Physiol. (Lond.), 160, 106-154; Kjaer et al. (1997) J. Neurophysiol., 78, 3187-3197]. This constancy in the response to variations of the stimuli is commonly called invariance. Hubel and Wiesel's classical model of the primary visual cortex proposes a connectivity scheme which successfully describes simple and complex cell response properties. However, the question as to how this connectivity arises during normal development is left open. Based on their work and inspired by recent physiological findings we suggest a network model capable of learning from natural stimuli and developing receptive field properties which match those of cortical simple and complex cells. Stimuli are drawn from videos obtained by a camera mounted to a cat's head, so they should approximate the natural input to the cat's visual system. The network uses a competitive scheme to learn simple and complex cell response properties. Employing delayed signals to learn connections between simple and complex cells enables the model to utilize temporal properties of the input. We show that the temporal structure of the input gives rise to the emergence and refinement of complex cell receptive fields, whereas removing temporal continuity prevents this processes. This model lends a physiologically based explanation of the development of complex cell invariance response properties.

A two-layer sparse coding model learns simple and complex cell receptive fields and topography from natural images

The classical receptive fields of simple cells in the visual cortex have been shown to emerge from the statistical properties of natural images by forcing the cell responses to be maximally sparse, i.e. significantly activated only rarely. Here, we show that this single principle of sparseness can also lead to emergence of topography (columnar organization) and complex cell properties as well. These are obtained by maximizing the sparsenesses of locally pooled energies, which correspond to complex cell outputs. Thus, we obtain a highly parsimonious model of how these properties of the visual cortex are adapted to the characteristics of the natural input.