Plausible Neural Network Model Research Articles

A design for neural network model of continuous reading

Cognition and learning are exceedingly modeled as an associative activity of connectionist neural networks. However, only a few such models exist for continuous reading, which involves the delicate coordination of word recognition and eye movements. Moreover, these models are limited to only orthographic level of word processing with predetermined lexicons. Here, we present a conceptual design of a developmentally plausible neural network model of reading designed to simulate word learning, parafoveal preview activation of words, their later foveal word recognition including phonological decoding, and forward saccade length as a control mechanism for intake of new textual information. We will discuss the theoretical advancements of the design and avenues for future developments.

Multimodal acoustic-electric trigeminal nerve stimulation modulates conscious perception

Multimodal stimulation can reverse pathological neural activity and improve symptoms in neuropsychiatric diseases. Recent research shows that multimodal acoustic-electric trigeminal-nerve stimulation (TNS) (i.e., musical stimulation synchronized to electrical stimulation of the trigeminal nerve) can improve consciousness in patients with disorders of consciousness. However, the reliability and mechanism of this novel approach remain largely unknown. We explored the effects of multimodal acoustic-electric TNS in healthy human participants by assessing conscious perception before and after stimulation using behavioral and neural measures in tactile and auditory target-detection tasks. To explore the mechanisms underlying the putative effects of acoustic-electric stimulation, we fitted a biologically plausible neural network model to the neural data using dynamic causal modeling. We observed that (1) acoustic-electric stimulation improves conscious tactile perception without a concomitant change in auditory perception, (2) this improvement is caused by the interplay of the acoustic and electric stimulation rather than any of the unimodal stimulation alone, and (3) the effect of acoustic-electric stimulation on conscious perception correlates with inter-regional connection changes in a recurrent neural processing model. These results provide evidence that acoustic-electric TNS can promote conscious perception. Alterations in inter-regional cortical connections might be the mechanism by which acoustic-electric TNS achieves its consciousness benefits.

Inflexible Updating of the Self-Other Divide During a Social Context in Autism: Psychophysical, Electrophysiological, and Neural Network Modeling Evidence

BackgroundAutism spectrum disorder (ASD) affects many aspects of life, from social interactions to (multi)sensory processing. Similarly, the condition expresses at a variety of levels of description, from genetics to neural circuits and interpersonal behavior. We attempt to bridge between domains and levels of description by detailing the behavioral, electrophysiological, and putative neural network basis of peripersonal space (PPS) updating in ASD during a social context, given that the encoding of this space relies on appropriate multisensory integration, is malleable by social context, and is thought to delineate the boundary between the self and others. MethodsFifty (20 male/30 female) young adults, either diagnosed with ASD or age- and sex-matched individuals, took part in a visuotactile reaction time task indexing PPS, while high-density electroencephalography was continuously recorded. Neural network modeling was performed in silico. ResultsMultisensory psychophysics demonstrates that while PPS in neurotypical individuals shrinks in the presence of others—as to “give space”—this does not occur in ASD. Likewise, electroencephalography recordings suggest that multisensory integration is altered by social context in neurotypical individuals but not in individuals with ASD. Finally, a biologically plausible neural network model shows, as a proof of principle, that PPS updating may be inflexible in ASD owing to the altered excitatory/inhibitory balance that characterizes neural circuits in animal models of ASD. ConclusionsFindings are conceptually in line with recent statistical inference accounts, suggesting diminished flexibility in ASD, and further these observations by suggesting within an example relevant for social cognition that such inflexibility may be due to excitatory/inhibitory imbalances.

Complex interplay between spectral harmonicity and different types of cross-frequency couplings in nonlinear oscillators and biologically plausible neural network models.

Cross-frequency coupling (CFC) refers to the nonlinear interaction between oscillations in different frequency bands, and it is a rather ubiquitous phenomenon that has been observed in a variety of physical and biophysical systems. In particular, the coupling between the phase of slow oscillations and the amplitude of fast oscillations, referred as phase-amplitude coupling (PAC), has been intensively explored in the brain activity recorded from animals and humans. However, the interpretation of these CFC patterns remains challenging since harmonic spectral correlations characterizing nonsinusoidal oscillatory dynamics can act as a confounding factor. Specialized signal processing techniques are proposed to address the complex interplay between spectral harmonicity and different types of CFC, not restricted only to PAC. For this, we provide an in-depth characterization of the time locked index (TLI) as a tool aimed to efficiently quantify the harmonic content of noisy time series. It is shown that the proposed TLI measure is more robust and outperforms traditional phase coherence metrics (e.g., phase locking value, pairwise phase consistency) in several aspects. We found that a nonlinear oscillator under the effect of additive noise can produce spurious CFC with low spectral harmonic content. On the other hand, two coupled oscillatory dynamics with independent fundamental frequencies can produce true CFC with high spectral harmonic content via a rectification mechanism or other post-interaction nonlinear processing mechanisms. These results reveal a complex interplay between CFC and harmonicity emerging in the dynamics of biologically plausible neural network models and more generic nonlinear and parametric oscillators. We show that, contrary to what is usually assumed in the literature, the high harmonic content observed in nonsinusoidal oscillatory dynamics is neither a sufficient nor necessary condition to interpret the associated CFC patterns as epiphenomenal. There is mounting evidence suggesting that the combination of multimodal recordings, specialized signal processing techniques, and theoretical modeling is becoming a required step to completely understand CFC patterns observed in oscillatory rich dynamics of physical and biophysical systems.

Tuned normalization in perceptual decision-making circuits can explain seemingly suboptimal confidence behavior

Abstract Current dominant views hold that perceptual confidence reflects the probability that a decision is correct. Although these views have enjoyed some empirical support, recent behavioral results indicate that confidence and the probability of being correct can be dissociated. An alternative hypothesis suggests that confidence instead reflects the magnitude of evidence in favor of a decision while being relatively insensitive to the evidence opposing the decision. We considered how this alternative hypothesis might be biologically instantiated by developing a simple leaky competing accumulator neural network model incorporating a known property of sensory neurons: tuned normalization. The key idea of the model is that each accumulator neuron’s normalization ‘tuning’ dictates its contribution to perceptual decisions versus confidence judgments. We demonstrate that this biologically plausible model can account for several counterintuitive findings reported in the literature, where confidence and decision accuracy were shown to dissociate -- and that the differential contribution a neuron makes to decisions versus confidence judgments based on its normalization tuning is vital to capturing some of these effects. One critical prediction of the model is that systematic variability in normalization tuning exists not only in sensory cortices but also in the decision-making circuitry. We tested and validated this prediction in macaque superior colliculus (SC; a region implicated in decision-making). The confirmation of this novel prediction provides direct support for our model. These findings suggest that the brain has developed and implements this alternative, heuristic theory of perceptual confidence computation by capitalizing on the diversity of neural resources available. Significance The dominant view of perceptual confidence proposes that confidence optimally reflects the probability that a decision is correct. But recent empirical evidence suggests that perceptual confidence exhibits a suboptimal ‘confirmation bias’, just as in human decision-making in general. We tested how this ‘bias’ might be neurally implemented by building a biologically plausible neural network model, and showed that the ‘bias’ emerges when each neuron’s degree of divisive normalization dictates how it drives decisions versus confidence judgments. We confirmed the model’s biological substrate using electrophysiological recordings in monkeys. These results challenge the dominant model, suggesting that the brain instead capitalizes on the diversity of available machinery (i.e., neuronal resources) to implement heuristic -- not optimal -- strategies to compute subjective confidence.

Grounding Bayesian accounts of numerosity and variability effects in a similarity-based framework: the case of self-organising maps

ABSTRACTIn this paper, we propose self-organising maps as possible candidates to explain the psychological mechanisms underlying category generalisation. Self-organising maps are psychologically and biologically plausible neural network models that can learn after limited exposure to positive category examples, without any need of contrastive information. They reproduce human behaviour in category generalisation, in particular, the Numerosity and Variability effects, which are usually explained with Bayesian tools. Where category generalisation is concerned, self-organising maps deserve attention to bridge the gap between the computational level of analysis in Marr's hierarchy (where Bayesian models are often situated) and the algorithmic level of analysis in which plausible mechanisms are described.

Classification-Based Prediction of Effective Connectivity Between Timeseries With a Realistic Cortical Network Model.

Effective connectivity measures the pattern of causal interactions between brain regions. Traditionally, these patterns of causality are inferred from brain recordings using either non-parametric, i.e., model-free, or parametric, i.e., model-based, approaches. The latter approaches, when based on biophysically plausible models, have the advantage that they may facilitate the interpretation of causality in terms of underlying neural mechanisms. Recent biophysically plausible neural network models of recurrent microcircuits have shown the ability to reproduce well the characteristics of real neural activity and can be applied to model interacting cortical circuits. Unfortunately, however, it is challenging to invert these models in order to estimate effective connectivity from observed data. Here, we propose to use a classification-based method to approximate the result of such complex model inversion. The classifier predicts the pattern of causal interactions given a multivariate timeseries as input. The classifier is trained on a large number of pairs of multivariate timeseries and the respective pattern of causal interactions, which are generated by simulation from the neural network model. In simulated experiments, we show that the proposed method is much more accurate in detecting the causal structure of timeseries than current best practice methods. Additionally, we present further results to characterize the validity of the neural network model and the ability of the classifier to adapt to the generative model of the data.

Perception Coordination Network: A Framework for Online Multi-Modal Concept Acquisition and Binding

A biologically plausible neural network model named Perception Coordination Network (PCN) is proposed for online multi-modal concept acquisition and binding. It is a hierarchical structure inspired by the structure of the brain, and functionally divided into the primary sensory area (PSA), the primary sensory association area (SAA), and the higher order association area (HAA). The PSA processes many elementary features, e.g., colors, shapes, syllables, and basic flavors, etc. The SAA combines these elementary features to represent the unimodal concept of an object, e.g., the image, name and taste of an apple, etc. The HAA connects several primary sensory association areas like a function of synaesthesia, which means associating the image, name and taste of an object. PCN is able to continuously acquire and bind multi-modal concepts in an online way. Experimental results suggest that PCN can handle the multi-modal concept acquisition and binding problem effectively.

Familiarity Detection is an Intrinsic Property of Cortical Microcircuits with Bidirectional Synaptic Plasticity.

Humans instantly recognize a previously seen face as “familiar.” To deepen our understanding of familiarity-novelty detection, we simulated biologically plausible neural network models of generic cortical microcircuits consisting of spiking neurons with random recurrent synaptic connections. NMDA receptor (NMDAR)-dependent synaptic plasticity was implemented to allow for unsupervised learning and bidirectional modifications. Network spiking activity evoked by sensory inputs consisting of face images altered synaptic efficacy, which resulted in the network responding more strongly to a previously seen face than a novel face. Network size determined how many faces could be accurately recognized as familiar. When the simulated model became sufficiently complex in structure, multiple familiarity traces could be retained in the same network by forming partially-overlapping subnetworks that differ slightly from each other, thereby resulting in a high storage capacity. Fisher’s discriminant analysis was applied to identify critical neurons whose spiking activity predicted familiar input patterns. Intriguingly, as sensory exposure was prolonged, the selected critical neurons tended to appear at deeper layers of the network model, suggesting recruitment of additional circuits in the network for incremental information storage. We conclude that generic cortical microcircuits with bidirectional synaptic plasticity have an intrinsic ability to detect familiar inputs. This ability does not require a specialized wiring diagram or supervision and can therefore be expected to emerge naturally in developing cortical circuits.

Lateral connections synchronize population activity in a spiking neural network model of midbrain superior colliculus

Saccades are rapid and ballistic eye-head gaze shifts between points of interest in the visual field. They are crucial for gathering high-resolution visual information. The midbrain superior colliculus (SC) generates saccadic eye-movement commands for downstream oculomotor circuits. It contains an eye-centered, gaze-motor map that relates the location of a Gaussian-shaped neural population to the intended movement vector. The gaze-motor map mediates the spatiotemporal transformation for eye-head orienting gaze shifts to peripheral targets [1]. Electrophysiological recordings have shown that SC neurons exhibit some remarkable activity properties that depend on both their anatomical position and the resulting saccade trajectory [2]. Here, we propose a biologically plausible spiking neural network model that is constrained by the observed firing patterns of real SC neurons for visually evoked saccades. The functional two-dimensional network model reproduces the spike trains of single neurons in recorded SC populations for saccades with different amplitudes and directions. The network model consists of a 2D grid of neurons, representing the gaze-motor map. The adaptive integrate-and-fire neurons [3] portray the observed site-dependent bursting profiles of individual SC neurons through distinct intrinsic biophysical properties, whereas Mexican-hat shaped lateral connections ensure the observed synchronized population activity by a soft winner-takes-all mechanism. We argue that our model offers a basis for neuronal algorithms of spatiotemporal transformations and bio-inspired optimal control signal generators.

Modelling and analysing neural networks using a hybrid process algebra

Research involving artificial neural networks has tended to be driven towards efficient computation, especially in the domain of pattern recognition, or towards elucidating biological processes in the brain. Models have become more detailed as our understanding of the biology of the brain has increased, incorporating real-time behaviour of individual neurons interacting within complex system structures and dynamics. There are few examples of abstract and fully formal models of biologically plausible neural networks: in the neural networks literature models are often presented as a mixture of mathematical equations and natural language, supported by simulation code and associated experimental results. The informality often hides or obscures important aspects of a particular model, and leaves a large conceptual gap between the model descriptions and the usually low-level programming code used to simulate them.The main contribution of this paper is formally modelling and analysing a biologically plausible neural network model from the literature that exhibits complex neuron-level behaviour and network-level structure. To achieve this a modelling language ‘Pann’ is developed, based on the process algebras CSP and Hybridχ. It is designed to be convenient for mixing the behaviour of discrete events (such as a neuron spike) with mutable continuous and discrete variables (representing chemical properties of a neuron, for instance). Its behaviour is defined using an operational semantics, from which a set of general properties of the language is proved.The groundwork for the biological model is laid by first formalising some well-known concepts from the artificial neural networks domain, such as feedforward behaviour, backpropagation, and recurrent neural networks. The Pann model of a feedforward network, comprising a set of communicating processes representing individual neurons, is proved equivalent to the standard one-line calculation of feedforward behaviour.

Parallel temporal dynamics in hierarchical cognitive control

Cognitive control allows us to follow abstract rules in order to choose appropriate responses given our desired outcomes. Cognitive control is often conceptualized as a hierarchical decision process, wherein decisions made at higher, more abstract levels of control asymmetrically influence lower-level decisions. These influences could evolve sequentially across multiple levels of a hierarchical decision, consistent with much prior evidence for central bottlenecks and seriality in decision-making processes. However, here, we show that multiple levels of hierarchical cognitive control are processed primarily in parallel. Human participants selected responses to stimuli using a complex, multiply contingent (third order) rule structure. A response deadline procedure allowed assessment of the accuracy and timing of decisions made at each level of the hierarchy. In contrast to a serial decision process, error rates across levels of the decision mostly declined simultaneously and at identical rates, with only a slight tendency to complete the highest level decision first. Simulations with a biologically plausible neural network model demonstrate how such parallel processing could emerge from a previously developed hierarchically nested frontostriatal architecture. Our results support a parallel processing model of cognitive control, in which uncertainty on multiple levels of a decision is reduced simultaneously.

Modelling of double reset spiking neuron and Fitzhugh–Nagumo equations with coupling kernel functions

Abstract Creation of a computationally efficient and biologically plausible neuron and neural network models is an exceptionally important occupation, the results of which are much needed for the neuroscience in general or for the neurophysiology in partial. Besides, the applying neural network technological areas are always looking for novel improvements of the methodological approaches, algorithms, computational facilities. In this paper, a new (in some aspects) hybrid integrate-and-fire (HIF) neuron model, based on N-shaped Na + and K + ionic current–voltage characteristics and a double discrete reset mechanism, is proposed. In the results of computational experiments, the spike series from a single spiking to the burst and chaotic burst trains are demonstrated. In the other part of the paper, analogous FitzHugh–Nagumo neuron model as a generalized neural networks system with an expansion to an inclusion of a coupling kernel function has been considered. The mean field approximation of neuronal potentials and recovery currents inside a neuron cluster was used. The biologically more realistic nonlinear N-shaped Na + characteristic and kernel functions (one - or two-dimensional) were applied. A possibility to present the nonlinear integral differential equations with kernel functions under the Fourier transformation by partial differential equations was used that allowed us to overcome the analytical and numerical modeling difficulties. The equivalence of two kinds of solutions was additionally confirmed based on the error evaluation. The approach of the equivalent partial differential equations was successfully employed to solve the system with the heterogeneous synaptic functions. The analytical studies are corroborated by many numerical modeling experiments.

Learning of New Perceptual Groupings - A Biologically Plausible Recurrent Neural Network Model that Learns Contour Integration

Learning of New Perceptual Groupings - A Biologically Plausible Recurrent Neural Network Model that Learns Contour Integration

Dynamical Regimes in Neural Network Models of Matching Behavior

The matching law constitutes a quantitative description of choice behavior that is often observed in foraging tasks. According to the matching law, organisms distribute their behavior across available response alternatives in the same proportion that reinforcers are distributed across those alternatives. Recently a few biophysically plausible neural network models have been proposed to explain the matching behavior observed in the experiments. Here we study systematically the learning dynamics of these networks while performing a matching task on the concurrent variable interval (VI) schedule. We found that the model neural network can operate in one of three qualitatively different regimes depending on the parameters that characterize the synaptic dynamics and the reward schedule: (1) a matching behavior regime, in which the probability of choosing an option is roughly proportional to the baiting fractional probability of that option; (2) a perseverative regime, in which the network tends to make always the same decision; and (3) a tristable regime, in which the network can either perseverate or choose the two targets randomly approximately with the same probability. Different parameters of the synaptic dynamics lead to different types of deviations from the matching law, some of which have been observed experimentally. We show that the performance of the network depends on the number of stable states of each synapse and that bistable synapses perform close to optimal when the proper learning rate is chosen. Because our model provides a link between synaptic dynamics and qualitatively different behaviors, this work provides us with insight into the effects of neuromodulators on adaptive behaviors and psychiatric disorders.

Modeling the BOLD correlates of competitive neural dynamics

Winner-take-all models are commonly used to model decision-making tasks where one outcome must be selected from several competing options. Related random walk and diffusion models have been used to explain such processes and apply them to psychometric and neurophysiological data. Recent model-based fMRI studies have sought to find the neural correlates of decision-making processes. However, due to the fact that hemodynamic responses likely reflect synaptic rather than spiking activity, the expected BOLD signature of winner-take-all circuits is not clear. A powerful way to integrate data from neurophysiology and brain imaging is by developing biologically plausible neural network models constrained and testable by neural and behavioral data, and then using Synthetic Brain Imaging — transforming the output of simulations with the model to make predictions testable against neuroimaging data. We developed a biologically realistic spiking winner-take-all model comprised of coupled excitatory and inhibitory neural populations. We varied the difficulty of a decision-making task by adjusting the contrast, or relative strength of inputs representing two response options. Synthetic brain imaging was used to estimate the BOLD response of the model and analyze its peak as a function of input contrast. We performed a parameter space analysis to determine values for which the model performs the task accurately, and given accurate performance, the distribution of the input contrast–BOLD response relationship. This underscores the need for models grounded in neurophysiological data for brain imaging analyses which attempt to localize the neural correlates of cognitive processes based on predicted BOLD responses.

Transitory memory retrieval in a biologically plausible neural network model

A number of memory models have been proposed. These all have the basic structure that excitatory neurons are reciprocally connected by recurrent connections together with the connections with inhibitory neurons, which yields associative memory (i.e., pattern completion) and successive retrieval of memory. In most of the models, a simple mathematical model for a neuron in the form of a discrete map is adopted. It has not, however, been clarified whether behaviors like associative memory and successive retrieval of memory appear when a biologically plausible neuron model is used. In this paper, we propose a network model for associative memory and successive retrieval of memory based on Pinsky-Rinzel neurons. The state of pattern completion in associative memory can be observed with an appropriate balance of excitatory and inhibitory connection strengths. Increasing of the connection strength of inhibitory interneurons changes the state of memory retrieval from associative memory to successive retrieval of memory. We investigate this transition.

A Computational Model of the Development of Separate Representations of Facial Identity and Expression in the Primate Visual System

Experimental studies have provided evidence that the visual processing areas of the primate brain represent facial identity and facial expression within different subpopulations of neurons. For example, in non-human primates there is evidence that cells within the inferior temporal gyrus (TE) respond primarily to facial identity, while cells within the superior temporal sulcus (STS) respond to facial expression. More recently, it has been found that the orbitofrontal cortex (OFC) of non-human primates contains some cells that respond exclusively to changes in facial identity, while other cells respond exclusively to facial expression. How might the primate visual system develop physically separate representations of facial identity and expression given that the visual system is always exposed to simultaneous combinations of facial identity and expression during learning? In this paper, a biologically plausible neural network model, VisNet, of the ventral visual pathway is trained on a set of carefully-designed cartoon faces with different identities and expressions. The VisNet model architecture is composed of a hierarchical series of four Self-Organising Maps (SOMs), with associative learning in the feedforward synaptic connections between successive layers. During learning, the network develops separate clusters of cells that respond exclusively to either facial identity or facial expression. We interpret the performance of the network in terms of the learning properties of SOMs, which are able to exploit the statistical indendependence between facial identity and expression.

The influence of structure on the response properties of biologically plausible neural network models

Random sparse network architectures are intriguingly easy to construct, and, due to the lack of relevant structural studies, they have been difficult to disprove. Over the last decade, they have become the workhorse of computational studies of neuronal network dynamics. With the arrival of optical methods like channelrhodopsin and glutamate uncaging we can finally compare the predictions stemming from these architectures with responses measured in real cortical networks. Our model network is simulating L2/3 of a mouse barrel cortex column with three distinct neuronal populations: Excitatory pyramidal cells, fast-spiking inhibitory cells and non-fast-spiking inhibitory cells, modeled with biological plausible, cell-type specific adaptive exponential integrate and fire (AdEx) models [1]. For the connections between the three populations we used the probability of each connection and the distribution of synaptic weights, observed in biological measurements [2]. In parallel experiments and simulations, in which we stimulate groups of neurons and record the responses of the surrounding network, we can show that uniform random network models constructed from the statistics of pairwise recordings cannot capture the experimentally measured network responses. To improve the similarity between model and experiments, we investigate different ways to change the network architecture, for example by introducing correlations in the distribution of synaptic weights [3] or by modifying the in- and out-degree distributions of the network neurons. These changes are applied in a way that will still preserve the pairwise statistics in regards to the strengths and probabilities measured in biological experiments. In order to avoid an explosion of the number of usable parameters we are focusing on changes of the three most relevant types of connections, excitatory to excitatory, fast-spiking and non-fast-spiking, leading to a 12 dimensional parameter space. In order to visualize the high dimensional parameter space we rely on clutter based dimension reordering (CBDR) [4], a graphical technique used to build an intuitive representation of high dimensional data. This allows us to find not only the best network architecture in our parameter space but also to give an intuitive understanding of the impact of each parameter and of the interactions between parameters. The parameters with the highest impact on the response similarity between model and experiment are the in-degree and the input weight correlations in the connection from excitatory to fast-spiking neurons. Our work should contribute to a new, standard model architecture that will replace random networks as a tool to analyze biological network behavior.

Maximization of learning speed in motor cortex due to neuron redundancy

AbstractMany kinds of redundancies play important roles in the motor system. Many combinations of muscle activity can generate an identical movement. One of the functional and computational roles of this muscle redundancy is to control joint impedance for stabilizing movements against applied external perturbation. Many combinations of joint angles can yield an identical fingertip position. A functional and computational role of this kinematic redundancy is to ensure motor accuracy, if a joint angle is perturbed, by adjusting redundant ones. Neurons are overwhelmingly more numerous than muscles in our motor system. However, an open question remains as to what the functional and computational roles of this neuron redundancy are. To generate desired motor commands, our motor system adapts to changes in both the internal and external environment, caused by muscle fatigue, force fields, etc. Faster learning is more desirable than slower learning because accumulated error can be minimized. Our analysis on a neural network model, assuming visuomotor or force field adaptation, revealed that learning speed reaches its maximum value if the model includes sufficient neuron redundancy. We subsequently ran numerical simulations of more biologically plausible neural network models. The results suggest that neuron redundancy accelerates learning speed even if the models include recurrent connections, a two-layer structure, or a skewed bimodal distribution of preferred directions. Neuron redundancy also yields the equivalence between a learning rule of synaptic weights and a model for sensorimotor learning, which is based on the results of behavioral experiments. These results suggest that neuron redundancy contributes to maximizing learning speed and also that the learning curves observed in behavioral experiments are tuned to be as fast as possible.