Receptive Fields Of Simple Cells Research Articles

Biological direct-shortcut deep residual learning for sparse visual processing

We show, based on the following three grounds, that the primary visual cortex (V1) is a biological direct-shortcut deep residual learning neural network (ResNet) for sparse visual processing: (1) We first highlight that Gabor-like sets of basis functions, which are similar to the receptive fields of simple cells in the primary visual cortex (V1), are excellent candidates for sparse representation of natural images; i.e., images from the natural world, affirming the brain to be optimized for this. (2) We then prove that the intra-layer synaptic weight matrices of this region can be reasonably first-order approximated by identity mappings, and are thus sparse themselves. (3) Finally, we point out that intra-layer weight matrices having identity mapping as their initial approximation, irrespective of this approximation being also a reasonable first-order one or not, resemble the building blocks of direct-shortcut digital ResNets, which completes the grounds. This biological ResNet interconnects the sparsity of the final representation of the image to that of its intra-layer weights. Further exploration of this ResNet, and understanding the joint effects of its architecture and learning rules, e.g. on its inductive bias, could lead to major advancements in the area of bio-inspired digital ResNets. One immediate line of research in this context, for instance, is to study the impact of forcing the direct-shortcuts to be good first-order approximations of each building block. For this, along with the ℓ1\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$${{\\ell}}_{1}$$\\end{document}-minimization posed on the basis function coefficients the ResNet finally provides at its output, another parallel one could e.g. also be posed on the weights of its residual layers.

Photosynthetic protein-based simple cell receptive fields for detection of Café Wall illusions

The differential response of the photosynthetic protein bacteriorhodopsin (bR) is highlighted as a means to emulate excitation and inhibition in neural response. We present a neuromorphic device that mimics a simple cell receptive field fabricated by patterning bR onto two transparent electrodes and sealing them with an electrolyte solution. The artificial receptive field is characterised as a bandpass filter that simultaneously emulates excitation and inhibition with a single element, requiring no bias power supply or external operating circuitry. The filter was employed to detect the Café Wall illusion, a type of orientation illusion, and demonstrated its ability to replicate the human eye’s perception using only the properties of the protein.

Efficient processing of natural scenes in visual cortex.

Neural circuits in the periphery of the visual, auditory, and olfactory systems are believed to use limited resources efficiently to represent sensory information by adapting to the statistical structure of the natural environment. This "efficient coding" principle has been used to explain many aspects of early visual circuits including the distribution of photoreceptors, the mosaic geometry and center-surround structure of retinal receptive fields, the excess OFF pathways relative to ON pathways, saccade statistics, and the structure of simple cell receptive fields in V1. We know less about the extent to which such adaptations may occur in deeper areas of cortex beyond V1. We thus review recent developments showing that the perception of visual textures, which depends on processing in V2 and beyond in mammals, is adapted in rats and humans to the multi-point statistics of luminance in natural scenes. These results suggest that central circuits in the visual brain are adapted for seeing key aspects of natural scenes. We conclude by discussing how adaptation to natural temporal statistics may aid in learning and representing visual objects, and propose two challenges for the future: (1) explaining the distribution of shape sensitivity in the ventral visual stream from the statistics of object shape in natural images, and (2) explaining cell types of the vertebrate retina in terms of feature detectors that are adapted to the spatio-temporal structures of natural stimuli. We also discuss how new methods based on machine learning may complement the normative, principles-based approach to theoretical neuroscience.

Cortical mechanisms of visual brightness.

SUMMARYThe primary visual cortex signals the onset of light and dark stimuli with ON and OFF cortical pathways. Here, we demonstrate that both pathways generate similar response increments to large homogeneous surfaces and their response average increases with surface brightness. We show that, in cat visual cortex, response dominance from ON or OFF pathways is bimodally distributed when stimuli are smaller than one receptive field center but unimodally distributed when they are larger. Moreover, whereas small bright stimuli drive opposite responses from ON and OFF pathways (increased versus suppressed activity), large bright surfaces drive similar response increments. We show that this size-brightness relation emerges because strong illumination increases the size of light surfaces in nature and both ON and OFF cortical neurons receive input from ON thalamic pathways. We conclude that visual scenes are perceived as brighter when the average response increments from ON and OFF cortical pathways become stronger.

Reconstructing Group Wavelet Transform From Feature Maps With a Reproducing Kernel Iteration.

In this article, we consider the problem of reconstructing an image that is downsampled in the space of its SE(2) wavelet transform, which is motivated by classical models of simple cell receptive fields and feature preference maps in the primary visual cortex. We prove that, whenever the problem is solvable, the reconstruction can be obtained by an elementary project and replace iterative scheme based on the reproducing kernel arising from the group structure, and show numerical results on real images.

A Surrogate of Slow-Wave Sleep is Necessary for Stable Unsupervised Learning in a Model of Cortical Development

Sparse coding has been used to model the acquisition of V1 simple cell receptive fields. The Locally Competitive Algorithm (LCA) provides a biologically plausible implementation of sparse coding based on lateral inhibition. LCA can be reformulated to support dictionary learning via an online local Hebbian rule that reduces predictive coding error. Although originally formulated in terms of leaky integrator rate-coded neurons, LCA based on lateral inhibition between leaky integrate-and-fire (LIF) neurons has been implemented on spiking neuromorphic processors but such implementations preclude local online learning.

Synaptic and intrinsic mechanisms underlying development of cortical direction selectivity.

Modifications of synaptic inputs and cell-intrinsic properties both contribute to neuronal plasticity and development. To better understand these mechanisms, we undertook an intracellular analysis of the development of direction selectivity in the ferret visual cortex, which occurs rapidly over a few days after eye opening. We found strong evidence of developmental changes in linear spatiotemporal receptive fields of simple cells, implying alterations in circuit inputs. Further, this receptive field plasticity was accompanied by increases in near-spike-threshold excitability and input-output gain that resulted in dramatically increased spiking responses in the experienced state. Increases in subthreshold membrane responses induced by the receptive field plasticity and the increased input-output spiking gain were both necessary to explain the elevated firing rates in experienced ferrets. These results demonstrate that cortical direction selectivity develops through a combination of plasticity in inputs and in cell-intrinsic properties.

Binocular Matching Model Based on Hierarchical V1 and V2 Receptive Fields With Color, Orientation, and Region Feature Information.

Binocular matching models serve as the core component in most stereo visual aid systems developed for people with visual impairments. However, purely computational models lack a neuro-biological basis for explaining the phenomena observed in neuro-biology, and therefore offer no support for the development of bioengineering applications, and are overly complex for hardware implementation. In contrast, existing neurobiological models suffer from low matching calculation accuracy. Therefore, the present work proposes a novel binocular matching model based on the receptive field of simple cells rather than on image pixels, and thereby incorporates neurobiological structure, reduces hardware complexity, has enough accuracy and can be used in visual aid system. The proposed model is employed to calculate and optimize the binocular disparity via a cost function. Specifically, we simulate the functions and structures of V1 and V2 neurons according to the discoveries of modern neurobiology. Accordingly, the receptive fields of V1 layer neurons are aggregated to obtain the receptive fields of the V2 layer, and the disparity is obtained in the V2 layer. The accuracy of the proposed model is verified by comparisons of the disparity results obtained using the proposed model with those obtained using other neurobiological model, and thereby demonstrates that the model can guide the design of visual aid systems.

Adaptation to Binocular Anticorrelation Results in Increased Neural Excitability.

Throughout the brain, information from individual sources converges onto higher order neurons. For example, information from the two eyes first converges in binocular neurons in area V1. Some neurons are tuned to similarities between sources of information, which makes intuitive sense in a system striving to match multiple sensory signals to a single external cause-that is, establish causal inference. However, there are also neurons that are tuned to dissimilar information. In particular, some binocular neurons respond maximally to a dark feature in one eye and a light feature in the other. Despite compelling neurophysiological and behavioral evidence supporting the existence of these neurons [Katyal, S., Vergeer, M., He, S., He, B., & Engel, S. A. Conflict-sensitive neurons gate interocular suppression in human visual cortex. Scientific Reports, 8, 1239, 2018; Kingdom, F. A. A., Jennings, B. J., & Georgeson, M. A. Adaptation to interocular difference. Journal of Vision, 18, 9, 2018; Janssen, P., Vogels, R., Liu, Y., & Orban, G. A. At least at the level of inferior temporal cortex, the stereo correspondence problem is solved. Neuron, 37, 693-701, 2003; Tsao, D. Y., Conway, B. R., & Livingstone, M. S. Receptive fields of disparity-tuned simple cells in macaque V1. Neuron, 38, 103-114, 2003; Cumming, B. G., & Parker, A. J. Responses of primary visual cortical neurons to binocular disparity without depth perception. Nature, 389, 280-283, 1997], their function has remained opaque. To determine how neural mechanisms tuned to dissimilarities support perception, here we use electroencephalography to measure human observers' steady-state visually evoked potentials in response to change in depth after prolonged viewing of anticorrelated and correlated random-dot stereograms (RDS). We find that adaptation to anticorrelated RDS results in larger steady-state visually evoked potentials, whereas adaptation to correlated RDS has no effect. These results are consistent with recent theoretical work suggesting "what not" neurons play a suppressive role in supporting stereopsis [Goncalves, N. R., & Welchman, A. E. "What not" detectors help the brain see in depth. Current Biology, 27, 1403-1412, 2017]; that is, selective adaptation of neurons tuned to binocular mismatches reduces suppression resulting in increased neural excitability.

Automatic skin lesion segmentation by coupling deep fully convolutional networks and shallow network with textons.

Segmentation of skin lesions is an important step in computer-aided diagnosis of melanoma; it is also a very challenging task due to fuzzy lesion boundaries and heterogeneous lesion textures. We present a fully automatic method for skin lesion segmentation based on deep fully convolutional networks (FCNs). We investigate a shallow encoding network to model clinically valuable prior knowledge, in which spatial filters simulating simple cell receptive fields function in the primary visual cortex (V1) is considered. An effective fusing strategy using skip connections and convolution operators is then leveraged to couple prior knowledge encoded via shallow network with hierarchical data-driven features learned from the FCNs for detailed segmentation of the skin lesions. To our best knowledge, this is the first time the domain-specific hand craft features have been built into a deep network trained in an end-to-end manner for skin lesion segmentation. The method has been evaluated on both ISBI 2016 and ISBI 2017 skin lesion challenge datasets. We provide comparative evidence to demonstrate that our newly designed network can gain accuracy for lesion segmentation by coupling the prior knowledge encoded by the shallow network with the deep FCNs. Our method is robust without the need for data augmentation or comprehensive parameter tuning, and the experimental results show great promise of the method with effective model generalization compared to other state-of-the-art-methods.

Toward a Biologically Plausible Model of LGN-V1 Pathways Based on Efficient Coding.

Increasing evidence supports the hypothesis that the visual system employs a sparse code to represent visual stimuli, where information is encoded in an efficient way by a small population of cells that respond to sensory input at a given time. This includes simple cells in primary visual cortex (V1), which are defined by their linear spatial integration of visual stimuli. Various models of sparse coding have been proposed to explain physiological phenomena observed in simple cells. However, these models have usually made the simplifying assumption that inputs to simple cells already incorporate linear spatial summation. This overlooks the fact that these inputs are known to have strong non-linearities such the separation of ON and OFF pathways, or separation of excitatory and inhibitory neurons. Consequently these models ignore a range of important experimental phenomena that are related to the emergence of linear spatial summation from non-linear inputs, such as segregation of ON and OFF sub-regions of simple cell receptive fields, the push-pull effect of excitation and inhibition, and phase-reversed cortico-thalamic feedback. Here, we demonstrate that a two-layer model of the visual pathway from the lateral geniculate nucleus to V1 that incorporates these biological constraints on the neural circuits and is based on sparse coding can account for the emergence of these experimental phenomena, diverse shapes of receptive fields and contrast invariance of orientation tuning of simple cells when the model is trained on natural images. The model suggests that sparse coding can be implemented by the V1 simple cells using neural circuits with a simple biologically plausible architecture.

Biologically inspired computational modeling of motion based on middle temporal area

Abstract This paper describes a bio-inspired algorithm for motion computation based on V1 (Primary Visual Cortex) andMT (Middle Temporal Area) cells. The behavior of neurons in V1 and MT areas contain significant information to understand the perception of motion. From a computational perspective, the neurons are treated as two dimensional filters to represent the receptive fields of simple cells that compose the complex cells. A modified elaborated Reichardt detector, adding an output exponent before the last stage followed by a re-entry stage of modulating feedback from MT, (reciprocal connections of V1 and MT) in a hierarchical framework, is proposed. The endstopped units, where the receptive fields of cells are surrounded by suppressive regions, are modeled as a divisive operation. MT cells play an important role for integrating and interpreting inputs from earlier-level (V1).We fit a normalization and a pooling to find the most active neurons for motion detection. All steps employed are physiologically inspired processing schemes and need some degree of simplification and abstraction. The results suggest that our proposed algorithm can achieve better performance than recent state-of-the-art bio-inspired approaches for real world images.

Inhibition in Simple Cell Receptive Fields Is Broad and OFF-Subregion Biased.

Inhibition in thalamorecipient layer 4 simple cells of primary visual cortex is believed to play important roles in establishing visual response properties and integrating visual inputs across their receptive fields (RFs). Simple cell RFs are characterized by nonoverlapping, spatially restricted subregions in which visual stimuli can either increase or decrease the firing rate of the cell, depending on contrast. Inhibition is believed to be triggered exclusively from visual stimulation of individual RF subregions. However, this view is at odds with the known anatomy of layer 4 interneurons in visual cortex and differs from recent findings in mouse visual cortex. Here we show with in vivo intracellular recordings in cats that while excitation is restricted to RF subregions, inhibition spans the width of simple cell RFs. Consequently, excitatory stimuli within a subregion concomitantly drive excitation and inhibition. Furthermore, we found that the distribution of inhibition across the RF is stronger toward OFF subregions. This inhibitory OFF-subregion bias has a functional consequence on spatial integration of inputs across the RF. A model based on the known anatomy of layer 4 demonstrates that the known proportion and connectivity of inhibitory neurons in layer 4 of primary visual cortex is sufficient to explain broad inhibition with an OFF-subregion bias while generating a variety of phase relations, including antiphase, between excitation and inhibition in response to drifting gratings.SIGNIFICANCE STATEMENT The wiring of excitatory and inhibitory neurons in cortical circuits is key to determining the response properties in sensory cortex. In the visual cortex, the first cells that receive visual input are simple cells in layer 4. The underlying circuitry responsible for the response properties of simple cells is not yet known. In this study, we challenge a long-held view concerning the pattern of inhibitory input and provide results that agree with current known anatomy. We show here that inhibition is evoked broadly across the receptive fields of simple cells, and we identify a surprising bias in inhibition within the receptive field. Our findings represent a step toward a unified view of inhibition across different species and sensory systems.

Full-reference quality assessment of stereoscopic images by learning binocular visual properties.

Stereoscopic imaging technology has been growingly prevalent driven by both the entertainment industry and scientific applications in today's world. But objective quality assessment of stereoscopic images is a challenging task. In this paper, we propose a novel stereoscopic image quality assessment (SIQA) method by jointly considering monocular perception and binocular interaction. As the most significant contribution of this study, binocular perceptual properties of simple and complex cells are considered for full-reference (FR) SIQA. Specifically, the proposed scheme first simulates the receptive fields of simple cells (one class of V1 neurons) using a push-pull combination of receptive fields response, which is used to represent a monocular cue. Further, the receptive fields of complex cells (the other class of V1 neurons) are simulated by using binocular energy response and binocular rivalry response, which are used to represent a binocular cue. Subsequently, various quality-aware features are extracted from the response of area V1 by calculating the self-weighted histogram of the local binary pattern on four types of feature maps of similarity measurement that will change in the presence of distortions. Finally, kernel ridge regression is used to simulate a nonlinear relationship between the quality-aware features and objective quality scores. The performance of our method is evaluated over popular stereoscopic image databases and shown to be competitive with the state-of-the-art FR SIQA algorithms.

Generalized Gabor filters for palmprint recognition

Orientation-based coding approaches have recently been widely employed for face and palmprint recognition where generally, one starts with a set of Gabor filters to extract orientation information and then proceeds to code dominant orientations as features for each point of the palmprint. However, as the Gabor filter is developed to model two-dimensional receptive fields of simple cells in straits cortex, it might not be our best choice when dealing with curved and complex structures inherent in the palmprint texture. Motivated by this intuition, this paper shows that Gabor filters are a subset of a bigger family of filters which we refer to as generalized Gabor filter (GGF). Depending on the values of its parameters, a GGF takes a rather diverse shapes and orientations, which results in a potentially finer feature extraction capability. We show this improved capability by employing GGFs in the palmprint verification process. In applying our method, two different sub-banks of GGFs are defined for the orientation-based feature extraction of palmprints, and when compared with Gabor filters, it will be shown that GGFs have the upper hand in capturing orientation features. Furthermore, compared with the competitive code—one of the well-known orientation-based coding methods—the number of employed orientations is reduced to half. This would automatically compensate for a double usage of the filter banks, which otherwise could increase the time complexity of using GGFs. These ideas are further elaborated using a set of experiments on PolyU II and PolyU 2D/3D palmprint databases. The results show the preeminence of using GGFs both in terms of accuracy and efficiency.

The Two-Dimensional Gabor Function Adapted to Natural Image Statistics: A Model of Simple-Cell Receptive Fields and Sparse Structure in Images.

The two-dimensional Gabor function is adapted to natural image statistics, leading to a tractable probabilistic generative model that can be used to model simple cell receptive field profiles, or generate basis functions for sparse coding applications. Learning is found to be most pronounced in three Gabor function parameters representing the size and spatial frequency of the two-dimensional Gabor function and characterized by a nonuniform probability distribution with heavy tails. All three parameters are found to be strongly correlated, resulting in a basis of multiscale Gabor functions with similar aspect ratios and size-dependent spatial frequencies. A key finding is that the distribution of receptive-field sizes is scale invariant over a wide range of values, so there is no characteristic receptive field size selected by natural image statistics. The Gabor function aspect ratio is found to be approximately conserved by the learning rules and is therefore not well determined by natural image statistics. This allows for three distinct solutions: a basis of Gabor functions with sharp orientation resolution at the expense of spatial-frequency resolution, a basis of Gabor functions with sharp spatial-frequency resolution at the expense of orientation resolution, or a basis with unit aspect ratio. Arbitrary mixtures of all three cases are also possible. Two parameters controlling the shape of the marginal distributions in a probabilistic generative model fully account for all three solutions. The best-performing probabilistic generative model for sparse coding applications is found to be a gaussian copula with Pareto marginal probability density functions.

A stereoscopic image quality assessment model based on independent component analysis and binocular fusion property

Quality assessment for stereoscopic image is an important research issue in three-dimensional researches. In this paper, an independent component analysis(ICA) and binocular combination-based full reference image quality assessment (FR-IQA) method is proposed for color stereoscopic image. Specifically, image features that reflect the responds of simple cells in the cortex are first extracted by an ICA-based feature detector, which models the functions of the receptive fields of simple cells in the primary visual cortex. Both image feature similarity (IFS) and local luminance consistency (LLC) of the right and left images are calculated to measure the structure and brightness distortions respectively. Image patches that the mean pixel values have been removed are used for IFS calculation, while the computation of LLC is based on the removed mean pixel value. To simulate the binocular fusion properties of the complex cells in the cortex, feature energy of the extracted image features is utilized to calculate the weighting factors of binocular combination, following which a single feature similarity index is obtained by fusing the right and left images IFS. Furthermore, global relative luminance information of the selected image patches is used to integrate the right and left images LLC into a single luminance consistency index. Finally, image quality is obtained by combining above two indexes. Experimental results demonstrate that the proposed algorithm achieves high consistency with subjective assessment on 3D image quality assessment databases.

Boltzmann-Machine Learning of Prior Distributions of Binarized Natural Images

Prior distributions of binarized natural images are learned by using a Boltzmann machine. According the results of this study, there emerges a structure with two sublattices in the interactions, and the nearest-neighbor and next-nearest-neighbor interactions correspondingly take two discriminative values, which reflects the individual characteristics of the three sets of pictures that we process. Meanwhile, in a longer spatial scale, a longer-range, although still rapidly decaying, ferromagnetic interaction commonly appears in all cases. The characteristic length scale of the interactions is universally up to approximately four lattice spacings $\xi \approx 4$. These results are derived by using the mean-field method, which effectively reduces the computational time required in a Boltzmann machine. An improved mean-field method called the Bethe approximation also gives the same results, as well as the Monte Carlo method does for small size images. These reinforce the validity of our analysis and findings. Relations to criticality, frustration, and simple-cell receptive fields are also discussed.

Performance analysis of Gabor wavelet for extracting most informative and efficient features

Gabor wavelet can extract most informative and efficient texture features for different computer vision and multimedia applications. Features extracted by Gabor wavelet have similar information as visualized by the receptive field of simple cells in the visual cortex of the mammalian brains. This motivates researchers to use Gabor wavelet for feature extraction. Gabor wavelet features are used for many multimedia applications such as stereo matching, face and facial expression recognition (FER), texture representation for segmentation. This motivates us to analyze Gabor features to evaluate their effectiveness in representing an image. In this paper, three major characteristics of Gabor features are established viz., (i) Real coefficients of Gabor wavelet alone is sufficient enough to represent an image; (ii) Local Gabor wavelet features with overlapping regions represent an image more accurately as compared to the global Gabor features and the local features extracted for the non-overlapping regions; and (iii) Real coefficients of overlapping regions are more robust to radiometric changes as compared to the features extracted from both global and local (non-overlapping regions) by using real, imaginary and magnitude information of a Gabor wavelet. The efficacy and effectiveness of these findings are evaluated by reconstructing the original image using the extracted features, and subsequently the reconstructed image is compared with the original image. Experimental results show that the local Gabor wavelet features extracted from overlapping regions represent an image more efficiently than the global and non-overlapping region-based features. Experimental results also show that the real coefficients alone is sufficient enough to represent an image more accurately as compared to the imaginary and magnitude informations.

Synaptic Mechanisms Generating Orientation Selectivity in the ON Pathway of the Rabbit Retina.

A core goal for visual neuroscientists is to understand how neural circuits at each stage of the visual system extract and encode features from the visual scene. This study documents a novel type of orientation-selective ganglion cell in the retina and shows that the receptive field structure is remarkably similar to that of simple cells in primary visual cortex. However, the data indicate that, unlike in the cortex, orientation selectivity in the retina depends on the activity of inhibitory interneurons. The results further reveal the physiological basis for feature detection in the visual system, elucidate the synaptic mechanisms that generate orientation selectivity at an early stage of visual processing, and illustrate a novel role for NMDA receptors in retinal processing.