Readout Neurons Research Articles

Structural influences on synaptic plasticity: The role of presynaptic connectivity in the emergence of E/I co-tuning.

Cortical neurons are versatile and efficient coding units that develop strong preferences for specific stimulus characteristics. The sharpness of tuning and coding efficiency is hypothesized to be controlled by delicately balanced excitation and inhibition. These observations suggest a need for detailed co-tuning of excitatory and inhibitory populations. Theoretical studies have demonstrated that a combination of plasticity rules can lead to the emergence of excitation/inhibition (E/I) co-tuning in neurons driven by independent, low-noise signals. However, cortical signals are typically noisy and originate from highly recurrent networks, generating correlations in the inputs. This raises questions about the ability of plasticity mechanisms to self-organize co-tuned connectivity in neurons receiving noisy, correlated inputs. Here, we study the emergence of input selectivity and weight co-tuning in a neuron receiving input from a recurrent network via plastic feedforward connections. We demonstrate that while strong noise levels destroy the emergence of co-tuning in the readout neuron, introducing specific structures in the non-plastic pre-synaptic connectivity can re-establish it by generating a favourable correlation structure in the population activity. We further show that structured recurrent connectivity can impact the statistics in fully plastic recurrent networks, driving the formation of co-tuning in neurons that do not receive direct input from other areas. Our findings indicate that the network dynamics created by simple, biologically plausible structural connectivity patterns can enhance the ability of synaptic plasticity to learn input-output relationships in higher brain areas.

Neural representational geometry underlies few-shot concept learning

Understanding the neural basis of the remarkable human cognitive capacity to learn novel concepts from just one or a few sensory experiences constitutes a fundamental problem. We propose a simple, biologically plausible, mathematically tractable, and computationally powerful neural mechanism for few-shot learning of naturalistic concepts. We posit that the concepts that can be learned from few examples are defined by tightly circumscribed manifolds in the neural firing-rate space of higher-order sensory areas. We further posit that a single plastic downstream readout neuron learns to discriminate new concepts based on few examples using a simple plasticity rule. We demonstrate the computational power of our proposal by showing that it can achieve high few-shot learning accuracy on natural visual concepts using both macaque inferotemporal cortex representations and deep neural network (DNN) models of these representations and can even learn novel visual concepts specified only through linguistic descriptors. Moreover, we develop a mathematical theory of few-shot learning that links neurophysiology to predictions about behavioral outcomes by delineating several fundamental and measurable geometric properties of neural representations that can accurately predict the few-shot learning performance of naturalistic concepts across all our numerical simulations. This theory reveals, for instance, that high-dimensional manifolds enhance the ability to learn new concepts from few examples. Intriguingly, we observe striking mismatches between the geometry of manifolds in the primate visual pathway and in trained DNNs. We discuss testable predictions of our theory for psychophysics and neurophysiological experiments.

Learning and replaying spatiotemporal sequences: A replication study.

Learning and replaying spatiotemporal sequences are fundamental computations performed by the brain and specifically the neocortex. These features are critical for a wide variety of cognitive functions, including sensory perception and the execution of motor and language skills. Although several computational models demonstrate this capability, many are either hard to reconcile with biological findings or have limited functionality. To address this gap, a recent study proposed a biologically plausible model based on a spiking recurrent neural network supplemented with read-out neurons. After learning, the recurrent network develops precise switching dynamics by successively activating and deactivating small groups of neurons. The read-out neurons are trained to respond to particular groups and can thereby reproduce the learned sequence. For the model to serve as the basis for further research, it is important to determine its replicability. In this Brief Report, we give a detailed description of the model and identify missing details, inconsistencies or errors in or between the original paper and its reference implementation. We re-implement the full model in the neural simulator NEST in conjunction with the NESTML modeling language and confirm the main findings of the original work.

Breast Cancer Recognition Using Saliency-Based Spiking Neural Network

The spiking neural networks (SNNs) use event-driven signals to encode physical information for neural computation. SNN takes the spiking neuron as the basic unit. It modulates the process of nerve cells from receiving stimuli to firing spikes. Therefore, SNN is more biologically plausible. Although the SNN has more characteristics of biological neurons, SNN is rarely used for medical image recognition due to its poor performance. In this paper, a reservoir spiking neural network is used for breast cancer image recognition. Due to the difficulties of extracting the lesion features in medical images, a salient feature extraction method is used in image recognition. The salient feature extraction network is composed of spiking convolution layers, which can effectively extract the features of lesions. Two temporal encoding manners, namely, linear time encoding and entropy-based time encoding methods, are used to encode the input patterns. Readout neurons use the ReSuMe algorithm for training, and the Fruit Fly Optimization Algorithm (FOA) is employed to optimize the network architecture to further improve the reservoir SNN performance. Three modality datasets are used to verify the effectiveness of the proposed method. The results show an accuracy of 97.44% for the BreastMNIST database. The classification accuracy is 98.27% on the mini-MIAS database. And the overall accuracy is 95.83% for the BreaKHis database by using the saliency feature extraction, entropy-based time encoding, and network optimization.

Built to Last: Functional and Structural Mechanisms in the Moth Olfactory Network Mitigate Effects of Neural Injury.

Most organisms suffer neuronal damage throughout their lives, which can impair performance of core behaviors. Their neural circuits need to maintain function despite injury, which in particular requires preserving key system outputs. In this work, we explore whether and how certain structural and functional neuronal network motifs act as injury mitigation mechanisms. Specifically, we examine how (i) Hebbian learning, (ii) high levels of noise, and (iii) parallel inhibitory and excitatory connections contribute to the robustness of the olfactory system in the Manduca sexta moth. We simulate injuries on a detailed computational model of the moth olfactory network calibrated to data. The injuries are modeled on focal axonal swellings, a ubiquitous form of axonal pathology observed in traumatic brain injuries and other brain disorders. Axonal swellings effectively compromise spike train propagation along the axon, reducing the effective neural firing rate delivered to downstream neurons. All three of the network motifs examined significantly mitigate the effects of injury on readout neurons, either by reducing injury’s impact on readout neuron responses or by restoring these responses to pre-injury levels. These motifs may thus be partially explained by their value as adaptive mechanisms to minimize the functional effects of neural injury. More generally, robustness to injury is a vital design principle to consider when analyzing neural systems.

A Neural Network Model With Gap Junction for Topological Detection.

Visual information processing in the brain goes from global to local. A large volume of experimental studies has suggested that among global features, the brain perceives the topological information of an image first. Here, we propose a neural network model to elucidate the underlying computational mechanism. The model consists of two parts. The first part is a neural network in which neurons are coupled through gap junctions, mimicking the neural circuit formed by alpha ganglion cells in the retina. Gap junction plays a key role in the model, which, on one hand, facilitates the synchronized firing of a neuron group covering a connected region of an image, and on the other hand, staggers the firing moments of different neuron groups covering disconnected regions of the image. These two properties endow the network with the capacity of detecting the connectivity and closure of images. The second part of the model is a read-out neuron, which reads out the topological information that has been converted into the number of synchronized firings in the retina network. Our model provides a simple yet effective mechanism for the neural system to detect the topological information of images in ultra-speed.

Learning spatiotemporal signals using a recurrent spiking network that discretizes time.

Learning to produce spatiotemporal sequences is a common task that the brain has to solve. The same neurons may be used to produce different sequential behaviours. The way the brain learns and encodes such tasks remains unknown as current computational models do not typically use realistic biologically-plausible learning. Here, we propose a model where a spiking recurrent network of excitatory and inhibitory spiking neurons drives a read-out layer: the dynamics of the driver recurrent network is trained to encode time which is then mapped through the read-out neurons to encode another dimension, such as space or a phase. Different spatiotemporal patterns can be learned and encoded through the synaptic weights to the read-out neurons that follow common Hebbian learning rules. We demonstrate that the model is able to learn spatiotemporal dynamics on time scales that are behaviourally relevant and we show that the learned sequences are robustly replayed during a regime of spontaneous activity.

Reading-out task variables as a low-dimensional reconstruction of neural spike trains in single trials.

We propose a new model of the read-out of spike trains that exploits the multivariate structure of responses of neural ensembles. Assuming the point of view of a read-out neuron that receives synaptic inputs from a population of projecting neurons, synaptic inputs are weighted with a heterogeneous set of weights. We propose that synaptic weights reflect the role of each neuron within the population for the computational task that the network has to solve. In our case, the computational task is discrimination of binary classes of stimuli, and weights are such as to maximize the discrimination capacity of the network. We compute synaptic weights as the feature weights of an optimal linear classifier. Once weights have been learned, they weight spike trains and allow to compute the post-synaptic current that modulates the spiking probability of the read-out unit in real time. We apply the model on parallel spike trains from V1 and V4 areas in the behaving monkey macaca mulatta, while the animal is engaged in a visual discrimination task with binary classes of stimuli. The read-out of spike trains with our model allows to discriminate the two classes of stimuli, while population PSTH entirely fails to do so. Splitting neurons in two subpopulations according to the sign of the weight, we show that population signals of the two functional subnetworks are negatively correlated. Disentangling the superficial, the middle and the deep layer of the cortex, we show that in both V1 and V4, superficial layers are the most important in discriminating binary classes of stimuli.

Neural Classifiers with Limited Connectivity and Recurrent Readouts.

For many neural network models in which neurons are trained to classify inputs like perceptrons, the number of inputs that can be classified is limited by the connectivity of each neuron, even when the total number of neurons is very large. This poses the problem of how the biological brain can take advantage of its huge number of neurons given that the connectivity is sparse. One solution is to combine multiple perceptrons together, as in committee machines. The number of classifiable random patterns would then grow linearly with the number of perceptrons, even when each perceptron has limited connectivity. However, the problem is moved to the downstream readout neurons, which would need a number of connections as large as the number of perceptrons. Here we propose a different approach in which the readout is implemented by connecting multiple perceptrons in a recurrent attractor neural network. We prove analytically that the number of classifiable random patterns can grow unboundedly with the number of perceptrons, even when the connectivity of each perceptron remains finite. Most importantly, both the recurrent connectivity and the connectivity of downstream readouts also remain finite. Our study shows that feedforward neural classifiers with numerous long-range afferent connections can be replaced by recurrent networks with sparse long-range connectivity without sacrificing the classification performance. Our strategy could be used to design more general scalable network architectures with limited connectivity, which resemble more closely the brain neural circuits that are dominated by recurrent connectivity.SIGNIFICANCE STATEMENT The mammalian brain has a huge number of neurons, but the connectivity is rather sparse. This observation seems to contrast with the theoretical studies showing that for many neural network models the performance scales with the number of connections per neuron and not with the total number of neurons. To solve this dilemma, we propose a model in which a recurrent network reads out multiple neural classifiers. Its performance scales with the total number of neurons even when each neuron of the network has limited connectivity. Our study reveals an important role of recurrent connections in neural systems like the hippocampus, in which the computational limitations due to sparse long-range feedforward connectivity might be compensated by local recurrent connections.

The Temporal Tuning of the Drosophila Motion Detectors Is Determined by the Dynamics of Their Input Elements

Detecting the direction of motion contained in the visual scene is crucial for many behaviors. However, because single photoreceptors only signal local luminance changes, motion detection requires a comparison of signals from neighboring photoreceptors across time in downstream neuronal circuits. For signals to coincide on readout neurons that thus become motion and direction selective, different input lines need to be delayed with respect to each other. Classical models of motion detection rely on non-linear interactions between two inputs after different temporal filtering. However, recent studies have suggested the requirement for at least three, not only two, input signals. Here, we comprehensively characterize the spatiotemporal response properties of all columnar input elements to the elementary motion detectors in the fruit fly, T4 and T5 cells, via two-photon calcium imaging. Between these input neurons, we find large differences in temporal dynamics. Based on this, computer simulations show that only a small subset of possible arrangements of these input elements maps onto a recently proposed algorithmic three-input model in a way that generates a highly direction-selective motion detector, suggesting plausible network architectures. Moreover, modulating the motion detection system by octopamine-receptor activation, we find the temporal tuning of T4 and T5 cells to be shifted toward higher frequencies, and this shift can be fully explained by the concomitant speeding of the input elements.

Learning Polychronous Neuronal Groups Using Joint Weight-Delay Spike-Timing-Dependent Plasticity

Polychronous neuronal group (PNG), a type of cell assembly, is one of the putative mechanisms for neural information representation. According to the reader-centric definition, some readout neurons can become selective to the information represented by polychronous neuronal groups under ongoing activity. Here, in computational models, we show that the frequently activated polychronous neuronal groups can be learned by readout neurons with joint weight-delay spike-timing-dependent plasticity. The identity of neurons in the group and their expected spike timing at millisecond scale can be recovered from the incoming weights and delays of the readout neurons. The detection performance can be further improved by two layers of readout neurons. In this way, the detection of polychronous neuronal groups becomes an intrinsic part of the network, and the readout neurons become differentiated members in the group to indicate whether subsets of the group have been activated according to their spike timing. The readout spikes representing this information can be used to analyze how PNGs interact with each other or propagate to downstream networks for higher-level processing.

A model of topological mapping of space in bat hippocampus

The mammalian hippocampus plays a key role in spatial learning and memory, but the exact nature of the hippocampal representation of space is still being explored. Recently, there has been a fair amount of success in modeling hippocampal spatial maps in rats, assuming a topological perspective on spatial information processing. In this article, we use the topological approach to study the formation of a 3D spatial map in bats, which produces several insights into neurophysiological mechanisms of the hippocampal spatial leaning. First, we demonstrate that, in order to produce accurate maps of the environment, place cell should be organized into functional groups, which can be interpreted as cell assemblies. Second, the model suggests that the readout neurons in these cell assemblies should function as integrators of synaptic inputs, rather than detectors of place cells' coactivity, which allows estimating the integration time window. Lastly, the model suggests that, in contrast with relatively slow moving rats, suppressing θ-precession in bats improves the place cells capacity to encode spatial maps, which is consistent with the experimental observations. © 2016 Wiley Periodicals, Inc.

A Topological Model of the Hippocampal Cell Assembly Network.

It is widely accepted that the hippocampal place cells' spiking activity produces a cognitive map of space. However, many details of this representation's physiological mechanism remain unknown. For example, it is believed that the place cells exhibiting frequent coactivity form functionally interconnected groups—place cell assemblies—that drive readout neurons in the downstream networks. However, the sheer number of coactive combinations is extremely large, which implies that only a small fraction of them actually gives rise to cell assemblies. The physiological processes responsible for selecting the winning combinations are highly complex and are usually modeled via detailed synaptic and structural plasticity mechanisms. Here we propose an alternative approach that allows modeling the cell assembly network directly, based on a small number of phenomenological selection rules. We then demonstrate that the selected population of place cell assemblies correctly encodes the topology of the environment in biologically plausible time, and may serve as a schematic model of the hippocampal network.

Reservoir of neurons with adaptive time constants: a hybrid model for robust motor-sensory temporal processing

The ability to precisely quantify time on the scale ofhundreds of milliseconds is critical towards the proces-sing of complex sensory and motor patterns. However,the natures of neural mechanisms for temporal proces-sing (at this scale) in the brain are mostly unknown.Based on experimental data (psychophysics, cell cul-tures, electrophysiology) and theoretical studies, it is lar-gely debated whetherdedicated circuits or intrinsicmechanisms of neural circuits underlie the timing pro-cess [1]. One specific type of timing model, namelystate-dependent networks (SDN) [2], shows that time isencoded in the temporal patterns of activity of neuralpopulations and emerges from the internal dynamics ofrecurrent networks. This can be achieved without theneed of dedicated timing units. However, such intrinsicmodels in their present form have difficulty accountingfor crossmodal transfer [1]. In contrast, recent experi-mental evidence indicates that medial premotor corticalneurons of behaving monkeys show specific intervaltuning across modalities (auditory and visual) [3]. In thiswork we propose a hybrid model, making the hypothesisthat dedicated interval tuning mechanisms of individualneurons augment the intrinsic dynamics of large recur-rent networks (dynamic reservoir). Using a networkmodel of rate-coded neurons starting with random initi-alization of synaptic connections, we propose a learningrule based on local active information storage (LAIS) [4]to adapt neuronal time constants with respect to theinput stimuli to the network. Measured at each spatio-temporal location of the reservoir, LAIS gives a prob-abilistic measure of the amount of information in theprevious state of the neuron that is relevant in predict-ing the next state. Interestingly high LAIS regions in thenetwork correlate to significant events in time (intervals)of the driving stimulus. Furthermore, we combine thiswith mutual information driven intrinsic plasticityscheme in order to stabilize chaotic activity in the net-work. Incoming input drives the network which, in turn,is connected to readout neurons (Figure 1.A) that dis-play the learned behavior for temporally dependent sen-sory motor tasks. Reservoir-to-output connections canbe adapted using both supervised and reward modulatedlearning rules. Using single and multiple interval discri-mination tasks, we show that our network reproduces(across modalities) a linear increase in temporal variabil-ity with increase in interval duration. This correlation isalso observed in experimental data [3]. Furthermore wedemonstrate that our dedicated timing mechanism com-plements the inherent transient dynamics of the net-work by successfully learning complex time dependentmotor behaviors; like handwriting generation (Figure 1.Band C), locomotion pattern transformation and temporalmemory tasks. In essence, our hybrid model demon-stratesthattimecanbeencodedbyacombinationofdedicated and intrinsic mechanisms with the possibilityto ‘learn’ the temporal structure of incoming stimuli [5].

The Effects of Theta Precession on Spatial Learning and Simplicial Complex Dynamics in a Topological Model of the Hippocampal Spatial Map

Learning arises through the activity of large ensembles of cells, yet most of the data neuroscientists accumulate is at the level of individual neurons; we need models that can bridge this gap. We have taken spatial learning as our starting point, computationally modeling the activity of place cells using methods derived from algebraic topology, especially persistent homology. We previously showed that ensembles of hundreds of place cells could accurately encode topological information about different environments (“learn” the space) within certain values of place cell firing rate, place field size, and cell population; we called this parameter space the learning region. Here we advance the model both technically and conceptually. To make the model more physiological, we explored the effects of theta precession on spatial learning in our virtual ensembles. Theta precession, which is believed to influence learning and memory, did in fact enhance learning in our model, increasing both speed and the size of the learning region. Interestingly, theta precession also increased the number of spurious loops during simplicial complex formation. We next explored how downstream readout neurons might define co-firing by grouping together cells within different windows of time and thereby capturing different degrees of temporal overlap between spike trains. Our model's optimum coactivity window correlates well with experimental data, ranging from ∼150–200 msec. We further studied the relationship between learning time, window width, and theta precession. Our results validate our topological model for spatial learning and open new avenues for connecting data at the level of individual neurons to behavioral outcomes at the neuronal ensemble level. Finally, we analyzed the dynamics of simplicial complex formation and loop transience to propose that the simplicial complex provides a useful working description of the spatial learning process.

The Correlation Structure of Local Neuronal Networks Intrinsically Results from Recurrent Dynamics

Correlated neuronal activity is a natural consequence of network connectivity and shared inputs to pairs of neurons, but the task-dependent modulation of correlations in relation to behavior also hints at a functional role. Correlations influence the gain of postsynaptic neurons, the amount of information encoded in the population activity and decoded by readout neurons, and synaptic plasticity. Further, it affects the power and spatial reach of extracellular signals like the local-field potential. A theory of correlated neuronal activity accounting for recurrent connectivity as well as fluctuating external sources is currently lacking. In particular, it is unclear how the recently found mechanism of active decorrelation by negative feedback on the population level affects the network response to externally applied correlated stimuli. Here, we present such an extension of the theory of correlations in stochastic binary networks. We show that (1) for homogeneous external input, the structure of correlations is mainly determined by the local recurrent connectivity, (2) homogeneous external inputs provide an additive, unspecific contribution to the correlations, (3) inhibitory feedback effectively decorrelates neuronal activity, even if neurons receive identical external inputs, and (4) identical synaptic input statistics to excitatory and to inhibitory cells increases intrinsically generated fluctuations and pairwise correlations. We further demonstrate how the accuracy of mean-field predictions can be improved by self-consistently including correlations. As a byproduct, we show that the cancellation of correlations between the summed inputs to pairs of neurons does not originate from the fast tracking of external input, but from the suppression of fluctuations on the population level by the local network. This suppression is a necessary constraint, but not sufficient to determine the structure of correlations; specifically, the structure observed at finite network size differs from the prediction based on perfect tracking, even though perfect tracking implies suppression of population fluctuations.

Decoding synchronized oscillations within the brain: Phase-delayed inhibition provides a robust mechanism for creating a sharp synchrony filter

The widespread presence of synchronized neuronal oscillations within the brain suggests that a mechanism must exist that is capable of decoding such activity. Two realistic designs for such a decoder include: (1) a read-out neuron with a high spike threshold, or (2) a phase-delayed inhibition network motif. Despite requiring a more elaborate network architecture, phase-delayed inhibition has been observed in multiple systems, suggesting that it may provide inherent advantages over simply imposing a high spike threshold. In this work, we use a computational and mathematical approach to investigate the efficacy of the phase-delayed inhibition motif in detecting synchronized oscillations. We show that phase-delayed inhibition is capable of creating a synchrony detector with sharp synchrony filtering properties that depend critically on the time course of inputs. Additionally, we show that phase-delayed inhibition creates a synchrony filter that is far more robust than that created by a high spike threshold.

The importance of mixed selectivity in complex cognitive tasks.

Single-neuron activity in the prefrontal cortex (PFC) is tuned to mixtures of multiple task-related aspects. Such mixed selectivity is highly heterogeneous, seemingly disordered and therefore difficult to interpret. We analysed the neural activity recorded in monkeys during an object sequence memory task to identify a role of mixed selectivity in subserving the cognitive functions ascribed to the PFC. We show that mixed selectivity neurons encode distributed information about all task-relevant aspects. Each aspect can be decoded from the population of neurons even when single-cell selectivity to that aspect is eliminated. Moreover, mixed selectivity offers a significant computational advantage over specialized responses in terms of the repertoire of input-output functions implementable by readout neurons. This advantage originates from the highly diverse nonlinear selectivity to mixtures of task-relevant variables, a signature of high-dimensional neural representations. Crucially, this dimensionality is predictive of animal behaviour as it collapses in error trials. Our findings recommend a shift of focus for future studies from neurons that have easily interpretable response tuning to the widely observed, but rarely analysed, mixed selectivity neurons.

Emergence of Optimal Decoding of Population Codes Through STDP

The brain faces the problem of inferring reliable hidden causes from large populations of noisy neurons, for example, the direction of a moving object from spikes in area MT. It is known that a theoretically optimal likelihood decoding could be carried out by simple linear readout neurons if weights of synaptic connections were set to certain values that depend on the tuning functions of sensory neurons. We show here that such theoretically optimal readout weights emerge autonomously through STDP in conjunction with lateral inhibition between readout neurons. In particular, we identify a class of optimal STDP learning rules with homeostatic plasticity, for which the autonomous emergence of optimal readouts can be explained on the basis of a rigorous learning theory. This theory shows that the network motif we consider approximates expectation-maximization for creating internal generative models for hidden causes of high-dimensional spike inputs. Notably, we find that this optimal functionality can be well approximated by a variety of STDP rules beyond those predicted by theory. Furthermore, we show that this learning process is very stable and automatically adjusts weights to changes in the number of readout neurons, the tuning functions of sensory neurons, and the statistics of external stimuli.

Alternative Post-Processing on a CMOS Chip to Fabricate a Planar Microelectrode Array

We present an alternative post-processing on a CMOS chip to release a planar microelectrode array (pMEA) integrated with its signal readout circuit, which can be used for monitoring the neuronal activity of vestibular ganglion neurons in newborn Wistar strain rats. This chip is fabricated through a 0.6 μm CMOS standard process and it has 12 pMEA through a 4 × 3 electrodes matrix. The alternative CMOS post-process includes the development of masks to protect the readout circuit and the power supply pads. A wet etching process eliminates the aluminum located on the surface of the p+-type silicon. This silicon is used as transducer for recording the neuronal activity and as interface between the readout circuit and neurons. The readout circuit is composed of an amplifier and tunable bandpass filter, which is placed on a 0.015 mm2 silicon area. The tunable bandpass filter has a bandwidth of 98 kHz and a common mode rejection ratio (CMRR) of 87 dB. These characteristics of the readout circuit are appropriate for neuronal recording applications.