Cells In Visual Cortex Research Articles

Oscillations in an artificial neural network convert competing inputs into a temporal code.

The field of computer vision has long drawn inspiration from neuroscientific studies of the human and non-human primate visual system. The development of convolutional neural networks (CNNs), for example, was informed by the properties of simple and complex cells in early visual cortex. However, the computational relevance of oscillatory dynamics experimentally observed in the visual system are typically not considered in artificial neural networks (ANNs). Computational models of neocortical dynamics, on the other hand, rarely take inspiration from computer vision. Here, we combine methods from computational neuroscience and machine learning to implement multiplexing in a simple ANN using oscillatory dynamics. We first trained the network to classify individually presented letters. Post-training, we added temporal dynamics to the hidden layer, introducing refraction in the hidden units as well as pulsed inhibition mimicking neuronal alpha oscillations. Without these dynamics, the trained network correctly classified individual letters but produced a mixed output when presented with two letters simultaneously, indicating a bottleneck problem. When introducing refraction and oscillatory inhibition, the output nodes corresponding to the two stimuli activate sequentially, ordered along the phase of the inhibitory oscillations. Our model implements the idea that inhibitory oscillations segregate competing inputs in time. The results of our simulations pave the way for applications in deeper network architectures and more complicated machine learning problems.

A single-cell transcriptomic atlas of sensory-dependent gene expression in developing mouse visual cortex.

Sensory experience drives the refinement and maturation of neural circuits during postnatal brain development through molecular mechanisms that remain to be fully elucidated. One likely mechanism involves the sensory-dependent expression of genes that encode direct mediators of circuit remodeling within developing cells. However, while studies in adult systems have begun to uncover crucial roles for sensory-induced genes in modifying circuit connectivity, the gene programs induced by brain cells in response to sensory experience during development remain to be fully characterized. Here, we present a single-nucleus RNA-sequencing dataset describing the transcriptional responses of cells in mouse visual cortex to sensory deprivation or sensory stimulation during a developmental window when visual input is necessary for circuit refinement. We sequenced 118,529 individual nuclei across sixteen neuronal and non-neuronal cortical cell types isolated from control, sensory deprived, and sensory stimulated mice, identifying 1,268 unique sensory-induced genes within the developing brain. To demonstrate the utility of this resource, we compared the architecture and ontology of sensory-induced gene programs between cell types, annotated transcriptional induction and repression events based upon RNA velocity, and discovered Neurexin and Neuregulin signaling networks that underlie cell-cell interactions via CellChat . We find that excitatory neurons, especially layer 2/3 pyramidal neurons, are highly sensitive to sensory stimulation, and that the sensory-induced genes in these cells are poised to strengthen synapse-to-nucleus crosstalk by heightening protein serine/threonine kinase activity. Altogether, we expect this dataset to significantly broaden our understanding of the molecular mechanisms through which sensory experience shapes neural circuit wiring in the developing brain.

Biological direct-shortcut deep residual learning for sparse visual processing

We show, based on the following three grounds, that the primary visual cortex (V1) is a biological direct-shortcut deep residual learning neural network (ResNet) for sparse visual processing: (1) We first highlight that Gabor-like sets of basis functions, which are similar to the receptive fields of simple cells in the primary visual cortex (V1), are excellent candidates for sparse representation of natural images; i.e., images from the natural world, affirming the brain to be optimized for this. (2) We then prove that the intra-layer synaptic weight matrices of this region can be reasonably first-order approximated by identity mappings, and are thus sparse themselves. (3) Finally, we point out that intra-layer weight matrices having identity mapping as their initial approximation, irrespective of this approximation being also a reasonable first-order one or not, resemble the building blocks of direct-shortcut digital ResNets, which completes the grounds. This biological ResNet interconnects the sparsity of the final representation of the image to that of its intra-layer weights. Further exploration of this ResNet, and understanding the joint effects of its architecture and learning rules, e.g. on its inductive bias, could lead to major advancements in the area of bio-inspired digital ResNets. One immediate line of research in this context, for instance, is to study the impact of forcing the direct-shortcuts to be good first-order approximations of each building block. For this, along with the ℓ1\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$${{\\ell}}_{1}$$\\end{document}-minimization posed on the basis function coefficients the ResNet finally provides at its output, another parallel one could e.g. also be posed on the weights of its residual layers.

Estimating receptive fields of simple and complex cells in early visual cortex: A convolutional neural network model with parameterized rectification.

Neurons in the primary visual cortex respond selectively to simple features of visual stimuli, such as orientation and spatial frequency. Simple cells, which have phase-sensitive responses, can be modeled by a single receptive field filter in a linear-nonlinear model. However, it is challenging to analyze phase-invariant complex cells, which require more elaborate models having a combination of nonlinear subunits. Estimating parameters of these models is made additionally more difficult by cortical neurons' trial-to-trial response variability. We develop a simple convolutional neural network method to estimate receptive field models for both simple and complex visual cortex cells from their responses to natural images. The model consists of a spatiotemporal filter, a parameterized rectifier unit (PReLU), and a two-dimensional Gaussian "map" of the receptive field envelope. A single model parameter determines the simple vs. complex nature of the receptive field, capturing complex cell responses as a summation of homogeneous subunits, and collapsing to a linear-nonlinear model for simple type cells. The convolutional method predicts simple and complex cell responses to natural image stimuli as well as grating tuning curves. The fitted models yield a continuum of values for the PReLU parameter across the sampled neurons, showing that the simple/complex nature of cells can vary in a continuous manner. We demonstrate that complex-like cells respond less reliably than simple-like cells. However, compensation for this unreliability with noise ceiling analysis reveals predictive performance for complex cells proportionately closer to that for simple cells. Most spatial receptive field structures are well fit by Gabor functions, whose parameters confirm well-known properties of cat A17/18 receptive fields.

Dynamic predictive coding: A model of hierarchical sequence learning and prediction in the neocortex.

We introduce dynamic predictive coding, a hierarchical model of spatiotemporal prediction and sequence learning in the neocortex. The model assumes that higher cortical levels modulate the temporal dynamics of lower levels, correcting their predictions of dynamics using prediction errors. As a result, lower levels form representations that encode sequences at shorter timescales (e.g., a single step) while higher levels form representations that encode sequences at longer timescales (e.g., an entire sequence). We tested this model using a two-level neural network, where the top-down modulation creates low-dimensional combinations of a set of learned temporal dynamics to explain input sequences. When trained on natural videos, the lower-level model neurons developed space-time receptive fields similar to those of simple cells in the primary visual cortex while the higher-level responses spanned longer timescales, mimicking temporal response hierarchies in the cortex. Additionally, the network's hierarchical sequence representation exhibited both predictive and postdictive effects resembling those observed in visual motion processing in humans (e.g., in the flash-lag illusion). When coupled with an associative memory emulating the role of the hippocampus, the model allowed episodic memories to be stored and retrieved, supporting cue-triggered recall of an input sequence similar to activity recall in the visual cortex. When extended to three hierarchical levels, the model learned progressively more abstract temporal representations along the hierarchy. Taken together, our results suggest that cortical processing and learning of sequences can be interpreted as dynamic predictive coding based on a hierarchical spatiotemporal generative model of the visual world.

A spatiotemporal energy model based on spiking neurons for human motion perception.

Inspired by the motion processing pathway, this paper proposes a bio-inspired feedforward spiking network model based on Hodgkin-Huxley neurons for human motion perception. The proposed network mimics the mechanisms of direction selectivity found in simple and complex cells of the primary visual cortex. Simple cells' receptive fields are modeled using Gabor energy filters, while complex cells' receptive fields are constructed by integrating the responses of simple cells in an energy model. To generate the motion map, the spiking output of the network integrates motion information encoded by the responses of complex cells with various preferred directions. Simulation results demonstrate that the spiking neuron-based network effectively replicates the directional selectivity operation of the visual cortex when presented with a sequence of time-varying images. We evaluate the proposed model against state-of-the-art spiking neuron-based motion detection models using publicly available datasets. The results highlight the model's capability to extract motion energy from diverse video sequences, akin to human visual motion perception models. Additionally, we showcase the application of the proposed model in motion segmentation tasks and compare its performance with state-of-the-art motion-based segmentation models using challenging video segmentation benchmarks. The results indicate competitive performance. The motion maps generated by the proposed model can be utilized for action recognition in input videos.

Visual Neuroprosthesis - Stimulation of Visual Cortical Centers in The Brain. Design of Non-Invasive Transcranial Stimulation of Functional Neurons.

The purpose of the article is to present the history and current status of visual cortical neuroprostheses, and to present a new method ofstimulating intact visual cortex cells. This paper contains an overview of the history and current status of visual cortex stimulation in severe visual impairment, but also highlights its shortcomings. These include mainly the stimulation of currently damaged cortical cells over a small area and, from a morphological point of view, possible damage to the stimulated neurons by the electrodes and their encapsulation by gliotic tissue. The paper also presents a proposal for a new technology of image processing and its transformation into a form of non-invasive transcranial stimulation of undamaged parts of the brain, which is protected by a national and international patent. The paper presents a comprehensive review of the current options for compensating for lost vision at the level of the cerebral cortex and a proposal for a new non-invasive method of stimulating the functional neurons of the visual cortex.

Functional specificity of recurrent inhibition in visual cortex

In the neocortex, neural activity is shaped by the interaction of excitatory and inhibitory neurons, defined by the organization of their synaptic connections. Although connections among excitatory pyramidal neurons are sparse and functionally tuned, inhibitory connectivity is thought to be dense and largely unstructured. By measuring invivo visual responses and synaptic connectivity of parvalbumin-expressing (PV+) inhibitory cells in mouse primary visual cortex, we show that the synaptic weights of their connections to nearby pyramidal neurons are specifically tuned according to the similarity of the cells' responses. Individual PV+ cells strongly inhibit those pyramidal cells that provide them with strong excitation and share their visual selectivity. This structured organization of inhibitory synaptic weights provides a circuit mechanism for tuned inhibition onto pyramidal cells despite dense connectivity, stabilizing activity within feature-specific excitatory ensembles while supporting competition between them.

Truly pattern: Nonlinear integration of motion signals is required to account for the responses of pattern cells in rat visual cortex.

A key feature of advanced motion processing in the primate dorsal stream is the existence of pattern cells-specialized cortical neurons that integrate local motion signals into pattern-invariant representations of global direction. Pattern cells have also been reported in rodent visual cortex, but it is unknown whether the tuning of these neurons results from truly integrative, nonlinear mechanisms or trivially arises from linear receptive fields (RFs) with a peculiar geometry. Here, we show that pattern cells in rat primary (V1) and lateromedial (LM) visual cortex process motion direction in a way that cannot be explained by the linear spatiotemporal structure of their RFs. Instead, their tuning properties are consistent with and well explained by those of units in a state-of-the-art neural network model of the dorsal stream. This suggests that similar cortical processes underlay motion representation in primates and rodents. The latter could thus serve as powerful model systems to unravel the underlying circuit-level mechanisms.

Deep matched filtering for retinal vessel segmentation

The structure of the retinal vascular tree can reflect many indicators of ophthalmic health status. Retinal vessel segmentation, an important basis for the quantitative analysis of vascular structure, has been developed over decades. However, in the areas of imaging quality and contrast, there are still vascular ruptures and missed detection by existing methods. Moreover, the problem of false positives caused by existing lesions has not been properly addressed. Inspired by the primary visual cortex of the biological visual system, this paper proposes Anisotropic Perceptive Convolution (APC) and Anisotropic Enhancement Module (AEM) to model visual cortex cells and their orientation selection mechanism, respectively, as well as a novel network named W-shaped Deep Matched Filtering (WS-DMF) having a W-shaped overall framework. In the feature extraction backbone, a DMF based on a multilayer aggregation of APC is designed to enhance vascular features and inhibit the expression of pathological features. In addition, AEMs are embedded into DMF to enhance the orientation and position information of high-dimensional features. Furthermore, in order to enhance the ability of APC for perceiving linear textures (such as blood vessels), an Orientation Anisotropic Loss (OAL) is introduced. Based on experiments on several widely used datasets and several professional vessel segmentation metrics, the proposed method exhibits strong abilities of vessel extraction and lesion inhibition, surpassing many state-of-the-art algorithms. The results indicate the potential value of WS-DMF in clinical practice. The source code is available at https://www.github.com/tyb311/WS-DMF.

Reciprocal Connections between Parvalbumin-Expressing Cells and Adjacent Pyramidal Cells Are Regulated by Clustered Protocadherin γ.

Functional neural circuits in the cerebral cortex are established through specific neural connections between excitatory and various inhibitory cell types. However, the molecular mechanisms underlying synaptic partner recognition remain unclear. In this study, we examined the impact of clustered protocadherin-γ (cPcdhγ) gene deletion in parvalbumin-positive (PV+) cells on intralaminar and translaminar neural circuits formed between PV+ and pyramidal (Pyr) cells in the primary visual cortex (V1) of male and female mice. First, we used whole-cell recordings and laser-scan photostimulation with caged glutamate to map excitatory inputs from layer 2/3 to layer 6. We found that cPcdhγ-deficient PV+ cells in layer 2/3 received normal translaminar inputs from Pyr cells through layers 2/3-6. Second, to further elucidate the effect on PV+-Pyr microcircuits within intralaminar layer 2/3, we conducted multiple whole-cell recordings. While the overall connection probability of PV+-Pyr cells remained largely unchanged, the connectivity of PV+-Pyr was significantly different between control and PV+-specific cPcdhγ-conditional knock-out (PV-cKO) mice. In control mice, the number of reciprocally connected PV+ cells was significantly higher than PV+ cells connected one way to Pyr cells, a difference that was not significant in PV-cKO mice. Interestingly, the proportion of highly reciprocally connected PV+ cells to Pyr cells with large unitary IPSC (uIPSC) amplitudes was reduced in PV-cKO mice. Conversely, the proportion of middle reciprocally connected PV+ cells to Pyr cells with large uIPSC amplitudes increased compared with control mice. This study demonstrated that cPcdhγ in PV+ cells modulates their reciprocity with Pyr cells in the cortex.

MANGEM: A web app for multimodal analysis of neuronal gene expression, electrophysiology, and morphology

Single-cell techniques like Patch-seq have enabled the acquisition of multimodal data from individual neuronal cells, offering systematic insights into neuronal functions. However, these data can be heterogeneous and noisy. To address this, machine learning methods have been used to align cells from different modalities onto a low-dimensional latent space, revealing multimodal cell clusters. The use of those methods can be challenging without computational expertise or suitable computing infrastructure for computationally expensive methods. To address this, we developed a cloud-based web application, MANGEM (multimodal analysis of neuronal gene expression, electrophysiology, and morphology). MANGEM provides a step-by-step accessible and user-friendly interface to machine learning alignment methods of neuronal multimodal data. It can run asynchronously for large-scale data alignment, provide users with various downstream analyses of aligned cells, and visualize the analytic results. We demonstrated the usage of MANGEM by aligning multimodal data of neuronal cells in the mouse visual cortex.

Mapping thalamic innervation to individual L2/3 pyramidal neurons and modeling their 'readout' of visual input.

The thalamus is the main gateway for sensory information from the periphery to the mammalian cerebral cortex. A major conundrum has been the discrepancy between the thalamus's central role as the primary feedforward projection system into the neocortex and the sparseness of thalamocortical synapses. Here we use new methods, combining genetic tools and scalable tissue expansion microscopy for whole-cell synaptic mapping, revealing the number, density and size of thalamic versus cortical excitatory synapses onto individual layer 2/3 (L2/3) pyramidal cells (PCs) of the mouse primary visual cortex. We find that thalamic inputs are not only sparse, but remarkably heterogeneous in number and density across individual dendrites and neurons. Most surprising, despite their sparseness, thalamic synapses onto L2/3 PCs are smaller than their cortical counterparts. Incorporating these findings into fine-scale, anatomically faithful biophysical models of L2/3 PCs reveals how individual neurons with sparse and weak thalamocortical synapses, embedded in small heterogeneous neuronal ensembles, may reliably 'read out' visually driven thalamic input.

Binary and analog variation of synapses between cortical pyramidal neurons.

Learning from experience depends at least in part on changes in neuronal connections. We present the largest map of connectivity to date between cortical neurons of a defined type (layer 2/3 [L2/3] pyramidal cells in mouse primary visual cortex), which was enabled by automated analysis of serial section electron microscopy images with improved handling of image defects (250 × 140 × 90 μm3 volume). We used the map to identify constraints on the learning algorithms employed by the cortex. Previous cortical studies modeled a continuum of synapse sizes by a log-normal distribution. A continuum is consistent with most neural network models of learning, in which synaptic strength is a continuously graded analog variable. Here, we show that synapse size, when restricted to synapses between L2/3 pyramidal cells, is well modeled by the sum of a binary variable and an analog variable drawn from a log-normal distribution. Two synapses sharing the same presynaptic and postsynaptic cells are known to be correlated in size. We show that the binary variables of the two synapses are highly correlated, while the analog variables are not. Binary variation could be the outcome of a Hebbian or other synaptic plasticity rule depending on activity signals that are relatively uniform across neuronal arbors, while analog variation may be dominated by other influences such as spontaneous dynamical fluctuations. We discuss the implications for the longstanding hypothesis that activity-dependent plasticity switches synapses between bistable states.

Identification of visual cortex cell types and species differences using single-cell RNA sequencing

The primate neocortex exerts high cognitive ability and strong information processing capacity. Here, we establish a single-cell RNA sequencing dataset of 133,454 macaque visual cortical cells. It covers major cortical cell classes including 25 excitatory neuron types, 37 inhibitory neuron types and all glial cell types. We identified layer-specific markers including HPCAL1 and NXPH4, and also identified two cell types, an NPY-expressing excitatory neuron type that expresses the dopamine receptor D3 gene; and a primate specific activity-dependent OSTN + sensory neuron type. Comparisons of our dataset with humans and mice show that the gene expression profiles differ between species in relation to genes that are implicated in the synaptic plasticity and neuromodulation of excitatory neurons. The comparisons also revealed that glutamatergic neurons may be more diverse across species than GABAergic neurons and non-neuronal cells. These findings pave the way for understanding how the primary cortex fulfills the high-cognitive functions.

Object Boundary Detection in Natural Images May Depend on "Incitatory" Cell-Cell Interactions.

Detecting object boundaries is crucial for recognition, but how the process unfolds in visual cortex remains unknown. To study the problem faced by a hypothetical boundary cell, and to predict how cortical circuitry could produce a boundary cell from a population of conventional "simple cells," we labeled 30,000 natural image patches and used Bayes' rule to help determine how a simple cell should influence a nearby boundary cell depending on its relative offset in receptive field position and orientation. We identified the following three basic types of cell-cell interactions: rising and falling interactions with a range of slopes and saturation rates, and nonmonotonic (bump-shaped) interactions with varying modes and amplitudes. Using simple models, we show that a ubiquitous cortical circuit motif consisting of direct excitation and indirect inhibition-a compound effect we call "incitation"-can produce the entire spectrum of simple cell-boundary cell interactions found in our dataset. Moreover, we show that the synaptic weights that parameterize an incitation circuit can be learned by a single-layer "delta" rule. We conclude that incitatory interconnections are a generally useful computing mechanism that the cortex may exploit to help solve difficult natural classification problems.SIGNIFICANCE STATEMENT Simple cells in primary visual cortex (V1) respond to oriented edges and have long been supposed to detect object boundaries, yet the prevailing model of a simple cell-a divisively normalized linear filter-is a surprisingly poor natural boundary detector. To understand why, we analyzed image statistics on and off object boundaries, allowing us to characterize the neural-style computations needed to perform well at this difficult natural classification task. We show that a simple circuit motif known to exist in V1 is capable of extracting high-quality boundary probability signals from local populations of simple cells. Our findings suggest a new, more general way of conceptualizing cell-cell interconnections in the cortex.

Brevican, Neurocan, Tenascin-C, and Tenascin-R Act as Important Regulators of the Interplay Between Perineuronal Nets, Synaptic Integrity, Inhibitory Interneurons, and Otx2.

Fast-spiking parvalbumin interneurons are critical for the function of mature cortical inhibitory circuits. Most of these neurons are enwrapped by a specialized extracellular matrix (ECM) structure called perineuronal net (PNN), which can regulate their synaptic input. In this study, we investigated the relationship between PNNs, parvalbumin interneurons, and synaptic distribution on these cells in the adult primary visual cortex (V1) of quadruple knockout mice deficient for the ECM molecules brevican, neurocan, tenascin-C, and tenascin-R. We used super-resolution structured illumination microscopy (SIM) to analyze PNN structure and associated synapses. In addition, we examined parvalbumin and calretinin interneuron populations. We observed a reduction in the number of PNN-enwrapped cells and clear disorganization of the PNN structure in the quadruple knockout V1. This was accompanied by an imbalance of inhibitory and excitatory synapses with a reduction of inhibitory and an increase of excitatory synaptic elements along the PNNs. Furthermore, the number of parvalbumin interneurons was reduced in the quadruple knockout, while calretinin interneurons, which do not wear PNNs, did not display differences in number. Interestingly, we found the transcription factor Otx2 homeoprotein positive cell population also reduced. Otx2 is crucial for parvalbumin interneuron and PNN maturation, and a positive feedback loop between these parameters has been described. Collectively, these data indicate an important role of brevican, neurocan, tenascin-C, and tenascin-R in regulating the interplay between PNNs, inhibitory interneurons, synaptic distribution, and Otx2 in the V1.

How Stimulus Statistics Affect the Receptive Fields of Cells in Primary Visual Cortex.

We studied the changes that neuronal receptive field (RF) models undergo when the statistics of the stimulus are changed from those of white Gaussian noise (WGN) to those of natural scenes (NSs), by fitting the models to multielectrode data recorded from primary visual cortex (V1) of female cats. This allowed the estimation of both a cascade of linear filters on the stimulus, as well as the static nonlinearities that map the output of the filters to the neuronal spike rates. We found that cells respond differently to these two classes of stimuli, with mostly higher spike rates and shorter response latencies to NSs than to WGN. The most striking finding was that NSs resulted in RFs that had additional uncovered filters compared with WGN. This finding was not an artifact of the higher spike rates observed for NSs relative to WGN, but rather was related to a change in coding. Our results reveal a greater extent of nonlinear processing in V1 neurons when stimulated using NSs compared with WGN. Our findings indicate the existence of nonlinear mechanisms that endow V1 neurons with context-dependent transmission of visual information.SIGNIFICANCE STATEMENT This study addresses a fundamental question about the concept of the receptive field (RF): does the encoding of information depend on the context or statistical regularities of the stimulus type? We applied state-of-the-art RF modeling techniques to data collected from multielectrode recordings from cat visual cortex in response to two statistically distinct stimulus types: white Gaussian noise and natural scenes. We find significant differences between the RFs that emerge from our data-driven modeling. Natural scenes result in far more complex RFs that combine multiple features in the visual input. Our findings reveal that different regimes or modes of operation are at work in visual cortical processing depending on the information present in the visual input. The complexity of V1 neural coding appears to be dependent on the complexity of the stimulus. We believe this new finding will have interesting implications for our understanding of the efficient transmission of information in sensory systems, which is an integral assumption of many computational theories (e.g., efficient and predictive coding of sensory processing in the brain).

Differentiation of Hebbian and homeostatic plasticity mechanisms within layer 5 visual cortex neurons.

Cortical layer 5 contains two major types of projection neuron known as IB (intrinsic bursting) cells that project sub-cortically and RS (regular spiking) cells that project between cortical areas. This study describes the plasticity properties of RS and IB cells in the mouse visual cortex during the critical period for ocular dominance plasticity. We find that RS neurons exhibit synaptic depression in response to both dark exposure (DE) and monocular deprivation (MD), and their homeostatic recovery from depression is dependent on TNF-α. In contrast, IB cells demonstrate opposite responses to DE and MD, potentiating to DE and depressing to MD. IB cells' potentiation depends on CaMKII-autophosphorylation and not TNF-α. IB cells show mature synaptic properties at the start of the critical period while RS cells mature during the critical period. Together with observations in somatosensory cortex, these results suggest that differences in RS and IB plasticity mechanisms are a general cortical property.

Community Newsletter: Race reporting, benchmarking, chandelier cells

This week’s Community Newsletter takes up tweets about how often autism intervention research fails to report participants’ race and ethnicity, benchmarks for effect sizes and mapping chandelier cells in the mouse visual cortex.