Computational Neuroscience Models Research Articles

Model Virtues in Computational Cognitive Neuroscience.

There is an abundance of computational models in cognitive neuroscience. A framework for what is desirable in a model, what justifies the introduction of a new one, or what makes one better than another is lacking, however. In this article, we examine key qualities ("virtues") that are desirable in computational models, and how these are interrelated. To keep the scope of the article manageable, we focus on the field of cognitive control, where we identified six "model virtues": empirical accuracy, empirical scope, functional analysis, causal detail, biological plausibility, and psychological plausibility. We first illustrate their use in published work on Stroop modeling and then discuss what expert modelers in the field of cognitive control said about them in a series of qualitative interviews. We found that virtues are interrelated and that their value depends on the modeler's goals, in ways that are not typically acknowledged in the literature. We recommend that researchers make the reasons for their modeling choices more explicit in published work. Our work is meant as a first step. Although our focus here is on cognitive control, we hope that our findings will spark discussion of virtues in other fields as well.

Uncertainty quantification and sensitivity analysis of neuron models with ion concentration dynamics.

This paper provides a comprehensive and computationally efficient case study for uncertainty quantification (UQ) and global sensitivity analysis (GSA) in a neuron model incorporating ion concentration dynamics. We address how challenges with UQ and GSA in this context can be approached and solved, including challenges related to computational cost, parameters affecting the system's resting state, and the presence of both fast and slow dynamics. Specifically, we analyze the electrodiffusive neuron-extracellular-glia (edNEG) model, which captures electrical potentials, ion concentrations (Na+, K+, Ca2+, and Cl-), and volume changes across six compartments. Our methodology includes a UQ procedure assessing the model's reliability and susceptibility to input uncertainty and a variance-based GSA identifying the most influential input parameters. To mitigate computational costs, we employ surrogate modeling techniques, optimized using efficient numerical integration methods. We propose a strategy for isolating parameters affecting the resting state and analyze the edNEG model dynamics under both physiological and pathological conditions. The influence of uncertain parameters on model outputs, particularly during spiking dynamics, is systematically explored. Rapid dynamics of membrane potentials necessitate a focus on informative spiking features, while slower variations in ion concentrations allow a meaningful study at each time point. Our study offers valuable guidelines for future UQ and GSA investigations on neuron models with ion concentration dynamics, contributing to the broader application of such models in computational neuroscience.

High-performing neural network models of visual cortex benefit from high latent dimensionality.

Geometric descriptions of deep neural networks (DNNs) have the potential to uncover core representational principles of computational models in neuroscience. Here we examined the geometry of DNN models of visual cortex by quantifying the latent dimensionality of their natural image representations. A popular view holds that optimal DNNs compress their representations onto low-dimensional subspaces to achieve invariance and robustness, which suggests that better models of visual cortex should have lower dimensional geometries. Surprisingly, we found a strong trend in the opposite direction-neural networks with high-dimensional image subspaces tended to have better generalization performance when predicting cortical responses to held-out stimuli in both monkey electrophysiology and human fMRI data. Moreover, we found that high dimensionality was associated with better performance when learning new categories of stimuli, suggesting that higher dimensional representations are better suited to generalize beyond their training domains. These findings suggest a general principle whereby high-dimensional geometry confers computational benefits to DNN models of visual cortex.

Reconstructing computational system dynamics from neural data with recurrent neural networks.

Computational models in neuroscience usually take the form of systems of differential equations. The behaviour of such systems is the subject of dynamical systems theory. Dynamical systemstheory provides a powerful mathematical toolbox for analysing neurobiological processes and has been a mainstay of computational neuroscience for decades. Recently, recurrent neural networks (RNNs) have become a popular machine learning tool for studying the non-linear dynamics of neural and behavioural processes by emulating an underlying system of differential equations. RNNs have been routinely trained on similar behavioural tasks to those used for animal subjects to generate hypotheses about the underlying computational mechanisms. By contrast, RNNs can also be trained on the measured physiological and behavioural data, thereby directly inheriting their temporal and geometrical properties. In this way they become a formal surrogate for the experimentally probed system that can be further analysed, perturbed and simulated. This powerful approach is called dynamical system reconstruction. In this Perspective, we focus on recent trends in artificial intelligence and machine learning in this exciting and rapidly expanding field, which may be less well known in neuroscience. We discuss formal prerequisites, different model architectures and training approaches for RNN-based dynamical system reconstructions, ways to evaluate and validate model performance, how to interpret trained models in a neuroscience context, and current challenges.

Qualitatively Different Delay-Dependent Working Memory Distortions in People With Schizophrenia and Healthy Control Participants

BackgroundImpairments in working memory (WM) have been well documented in people with schizophrenia (PSZ). However, these quantitative WM impairments can often be explained by nonspecific factors, such as impaired goal maintenance. Here, we used a spatial orientation delayed response task to explore a qualitative difference in WM dynamics between PSZ and healthy control participants (HCs). More specifically, we took advantage of the discovery that WM representations may drift either toward or away from previous trial targets (serial dependence). We tested the hypothesis that WM representations would drift toward the previous trial target in HCs but away from the previous trial target in PSZ. MethodsWe assessed serial dependence in PSZ (n = 31) and HCs (n = 25) using orientation as the to-be-remembered feature and memory delays lasting from 0 to 8 seconds. Participants were asked to remember the orientation of a teardrop-shaped object and reproduce the orientation after a delay period of varying length. ResultsConsistent with prior studies, we found that current trial memory representations were less precise in PSZ than in HCs. We also found that WM for the current trial orientation drifted toward the previous trial orientation in HCs (representational attraction) but drifted away from the previous trial orientation in PSZ (representational repulsion). ConclusionsThese results demonstrate a qualitative difference in WM dynamics between PSZ and HCs that cannot be easily explained by nuisance factors such as reduced effort. Most computational neuroscience models also fail to explain these results because they maintain information solely by means of sustained neural firing, which does not extend across trials. The results suggest a fundamental difference between PSZ and HCs in longer-term memory mechanisms that persist across trials, such as short-term potentiation and neuronal adaptation.

Astromorphic Self-Repair of Neuromorphic Hardware Systems

While neuromorphic computing architectures based on Spiking Neural Networks (SNNs) are increasingly gaining interest as a pathway toward bio-plausible machine learning, attention is still focused on computational units like the neuron and synapse. Shifting from this neuro-synaptic perspective, this paper attempts to explore the self-repair role of glial cells, in particular, astrocytes. The work investigates stronger correlations with astrocyte computational neuroscience models to develop macro-models with a higher degree of bio-fidelity that accurately captures the dynamic behavior of the self-repair process. Hardware-software co-design analysis reveals that bio-morphic astrocytic regulation has the potential to self-repair hardware realistic faults in neuromorphic hardware systems with significantly better accuracy and repair convergence for unsupervised learning tasks on the MNIST and F-MNIST datasets. Our implementation source code and trained models are available at https://github.com/NeuroCompLab-psu/Astromorphic_Self_Repair.

Limits of entrainment of circadian neuronal networks.

Circadian rhythmicity lies at the center of various important physiological and behavioral processes in mammals, such as sleep, metabolism, homeostasis, mood changes, and more. Misalignment of intrinsic neuronal oscillations with the external day-night cycle can disrupt such processes and lead to numerous disorders. In this work, we computationally determine the limits of circadian synchronization to external light signals of different frequency, duty cycle, and simulated amplitude. Instead of modeling circadian dynamics with generic oscillator models (e.g., Kuramoto-type), we use a detailed computational neuroscience model, which integrates biomolecular dynamics, neuronal electrophysiology, and network effects. This allows us to investigate the effect of small drug molecules, such as Longdaysin, and connect our results with experimental findings. To combat the high dimensionality of such a detailed model, we employ a matrix-free approach, while our entire algorithmic pipeline enables numerical continuation and construction of bifurcation diagrams using only direct simulation. We, thus, computationally explore the effect of heterogeneity in the circadian neuronal network, as well as the effect of the corrective therapeutic intervention of Longdaysin. Last, we employ unsupervised learning to construct a data-driven embedding space for representing neuronal heterogeneity.

Biomechanical and Sensory Feedback Regularize the Behavior of Different Locomotor Central Pattern Generators.

This work presents an in-depth numerical investigation into a hypothesized two-layer central pattern generator (CPG) that controls mammalian walking and how different parameter choices might affect the stepping of a simulated neuromechanical model. Particular attention is paid to the functional role of features that have not received a great deal of attention in previous work: the weak cross-excitatory connectivity within the rhythm generator and the synapse strength between the two layers. Sensitivity evaluations of deafferented CPG models and the combined neuromechanical model are performed. Locomotion frequency is increased in two different ways for both models to investigate whether the model's stability can be predicted by trends in the CPG's phase response curves (PRCs). Our results show that the weak cross-excitatory connection can make the CPG more sensitive to perturbations and that increasing the synaptic strength between the two layers results in a trade-off between forced phase locking and the amount of phase delay that can exist between the two layers. Additionally, although the models exhibit these differences in behavior when disconnected from the biomechanical model, these differences seem to disappear with the full neuromechanical model and result in similar behavior despite a variety of parameter combinations. This indicates that the neural variables do not have to be fixed precisely for stable walking; the biomechanical entrainment and sensory feedback may cancel out the strengths of excitatory connectivity in the neural circuit and play a critical role in shaping locomotor behavior. Our results support the importance of including biomechanical models in the development of computational neuroscience models that control mammalian locomotion.

Generative Models of Brain Dynamics.

This review article gives a high-level overview of the approaches across different scales of organization and levels of abstraction. The studies covered in this paper include fundamental models in computational neuroscience, nonlinear dynamics, data-driven methods, as well as emergent practices. While not all of these models span the intersection of neuroscience, AI, and system dynamics, all of them do or can work in tandem as generative models, which, as we argue, provide superior properties for the analysis of neuroscientific data. We discuss the limitations and unique dynamical traits of brain data and the complementary need for hypothesis- and data-driven modeling. By way of conclusion, we present several hybrid generative models from recent literature in scientific machine learning, which can be efficiently deployed to yield interpretable models of neural dynamics.

Computational Neuroscience Approach to Psychiatry: A Review on Theory-driven Approaches.

Translating progress in neuroscience into clinical benefits for patients with psychiatric disorders is challenging because it involves the brain as the most complex organ and its interaction with a complex environment and condition. Dealing with such complexity requires powerful techniques. Computational neuroscience approach to psychiatry integrates multiple levels and types of simulation, analysis and computation according to the different types of computational models to enhance comprehending, prediction and treatment of psychiatric disorder. This approach comprises two approaches: theory-driven and data-driven. In this review, we focus on recent advances in theory-driven approaches that mathematically and mechanistically examine the relationships between disorder-related changes and behavior at different level of brain organization. We discuss recent progresses in computational neuroscience models that relate to psychiatry and show how principles of neural computational modeling can be employed to explain psychopathology.

Learning as filtering: Implications for spike-based plasticity.

Most normative models in computational neuroscience describe the task of learning as the optimisation of a cost function with respect to a set of parameters. However, learning as optimisation fails to account for a time-varying environment during the learning process and the resulting point estimate in parameter space does not account for uncertainty. Here, we frame learning as filtering, i.e., a principled method for including time and parameter uncertainty. We derive the filtering-based learning rule for a spiking neuronal network-the Synaptic Filter-and show its computational and biological relevance. For the computational relevance, we show that filtering improves the weight estimation performance compared to a gradient learning rule with optimal learning rate. The dynamics of the mean of the Synaptic Filter is consistent with spike-timing dependent plasticity (STDP) while the dynamics of the variance makes novel predictions regarding spike-timing dependent changes of EPSP variability. Moreover, the Synaptic Filter explains experimentally observed negative correlations between homo- and heterosynaptic plasticity.

Convolutional Neural Networks as a Model of the Visual System: Past, Present, and Future.

Convolutional neural networks (CNNs) were inspired by early findings in the study of biological vision. They have since become successful tools in computer vision and state-of-the-art models of both neural activity and behavior on visual tasks. This review highlights what, in the context of CNNs, it means to be a good model in computational neuroscience and the various ways models can provide insight. Specifically, it covers the origins of CNNs and the methods by which we validate them as models of biological vision. It then goes on to elaborate on what we can learn about biological vision by understanding and experimenting on CNNs and discusses emerging opportunities for the use of CNNs in vision research beyond basic object recognition.

Performance of biologically grounded models of the early visual system on standard object recognition tasks

Computational neuroscience models of vision and neural network models for object recognition are often framed by different research agendas. Computational neuroscience mainly aims at replicating experimental data, while (artificial) neural networks target high performance on classification tasks. However, we propose that models of vision should be validated on object recognition tasks. At some point, mechanisms of realistic neuro-computational models of the visual cortex have to convince in object recognition as well.In order to foster this idea, we report the recognition accuracy for two different neuro-computational models of the visual cortex on several object recognition datasets. The models were trained using unsupervised Hebbian learning rules on natural scene inputs for the emergence of receptive fields comparable to their biological counterpart. We assume that the emerged receptive fields result in a general codebook of features, which should be applicable to a variety of visual scenes.We report the performances on datasets with different levels of difficulty, ranging from the simple MNIST to the more complex CIFAR-10 or ETH-80. We found that both networks show good results on simple digit recognition, comparable with previously published biologically plausible models. We also observed that our deeper layer neurons provide for naturalistic datasets a better recognition codebook. As for most datasets, recognition results of biologically grounded models are not available yet, our results provide a broad basis of performance values to compare methodologically similar models.

A global framework for a systemic view of brain modeling

The brain is a complex system, due to the heterogeneity of its structure, the diversity of the functions in which it participates and to its reciprocal relationships with the body and the environment. A systemic description of the brain is presented here, as a contribution to developing a brain theory and as a general framework where specific models in computational neuroscience can be integrated and associated with global information flows and cognitive functions. In an enactive view, this framework integrates the fundamental organization of the brain in sensorimotor loops with the internal and the external worlds, answering four fundamental questions (what, why, where and how). Our survival-oriented definition of behavior gives a prominent role to pavlovian and instrumental conditioning, augmented during phylogeny by the specific contribution of other kinds of learning, related to semantic memory in the posterior cortex, episodic memory in the hippocampus and working memory in the frontal cortex. This framework highlights that responses can be prepared in different ways, from pavlovian reflexes and habitual behavior to deliberations for goal-directed planning and reasoning, and explains that these different kinds of responses coexist, collaborate and compete for the control of behavior. It also lays emphasis on the fact that cognition can be described as a dynamical system of interacting memories, some acting to provide information to others, to replace them when they are not efficient enough, or to help for their improvement. Describing the brain as an architecture of learning systems has also strong implications in Machine Learning. Our biologically informed view of pavlovian and instrumental conditioning can be very precious to revisit classical Reinforcement Learning and provide a basis to ensure really autonomous learning.

Increased repulsion of working memory representations in schizophrenia.

Computational neuroscience models propose that working memory (WM) involves recurrent excitatory feedback loops that maintain firing over time along with lateral inhibition that prevents the spreading of activity to other feature values. In behavioral paradigms, this lateral inhibition appears to cause a repulsion of WM representations away from each other and from other strong sources of input. Recent computational models of schizophrenia have proposed that reduction in the strength of inhibition relative to strength of excitation may underlie impaired cognition, and this leads to the prediction that repulsion effects should be reduced in people with schizophrenia spectrum disorders (PSZ) relative to healthy control subjects (HCS). We tested this hypothesis in 2 experiments measuring WM repulsion effects. In Experiment 1, 45 PSZ and 32 HCS remembered the location of a single object relative to a centrally presented visual landmark and reported this location after a short delay. The reported location was repelled away from the landmark in both groups, but this repulsion effect was increased rather than decreased in PSZ relative to HCS. In Experiment 2, 41 PSZ and 34 HCS remembered 2 sequentially presented orientations and reported each orientation after a short delay. The reported orientations were biased away from each other in both groups, and this repulsion effect was again more pronounced in PSZ than in HCS. Contrary to the widespread hypothesis of reduced inhibition in schizophrenia, we provide robust evidence from 2 experiments showing that the behavioral performance of PSZ exhibited an exaggeration rather than a reduction of competitive inhibition. (PsycInfo Database Record (c) 2020 APA, all rights reserved).

Low-dimensional firing-rate dynamics for populations of renewal-type spiking neurons.

The macroscopic dynamics of large populations of neurons can be mathematically analyzed using low-dimensional firing-rate or neural-mass models. However, these models fail to capture spike synchronization effects and nonstationary responses of the population activity to rapidly changing stimuli. Here we derive low-dimensional firing-rate models for homogeneous populations of neurons modeled as time-dependent renewal processes. The class of renewal neurons includes integrate-and-fire models driven by white noise and has been frequently used to model neuronal refractoriness and spike synchronization dynamics. The derivation is based on an eigenmode expansion of the associated refractory density equation, which generalizes previous spectral methods for Fokker-Planck equations to arbitrary renewal models. We find a simple relation between the eigenvalues characterizing the timescales of the firing rate dynamics and the Laplace transform of the interspike interval density, for which explicit expressions are available for many renewal models. Retaining only the first eigenmode already yields a reliable low-dimensional approximation of the firing-rate dynamics that captures spike synchronization effects and fast transient dynamics at stimulus onset. We explicitly demonstrate the validity of our model for a large homogeneous population of Poisson neurons with absolute refractoriness and other renewal models that admit an explicit analytical calculation of the eigenvalues. The eigenmode expansion presented here provides a systematic framework for alternative firing-rate models in computational neuroscience based on spiking neuron dynamics with refractoriness.

The SONATA data format for efficient description of large-scale network models.

Increasing availability of comprehensive experimental datasets and of high-performance computing resources are driving rapid growth in scale, complexity, and biological realism of computational models in neuroscience. To support construction and simulation, as well as sharing of such large-scale models, a broadly applicable, flexible, and high-performance data format is necessary. To address this need, we have developed the Scalable Open Network Architecture TemplAte (SONATA) data format. It is designed for memory and computational efficiency and works across multiple platforms. The format represents neuronal circuits and simulation inputs and outputs via standardized files and provides much flexibility for adding new conventions or extensions. SONATA is used in multiple modeling and visualization tools, and we also provide reference Application Programming Interfaces and model examples to catalyze further adoption. SONATA format is free and open for the community to use and build upon with the goal of enabling efficient model building, sharing, and reproducibility.

Expansion coding and computation in the cerebellum: 50 years after the Marr-Albus codon theory.

Fifty years ago, David Marr and James Albus proposed a computational model of cerebellar cortical function based on the pioneering circuit models described by John Eccles, Masao Ito and Janos Szentagothai. The Marr-Albus model remains one of the most enduring and influential models in computational neuroscience, despite apparent falsification of some of the original predictions. We re-examine the Marr-Albus model in the context of the modern theory of computational neural networks and in the context of expanded interpretations of the connectivity and function of cerebellar cortex within the full motor system. By doing so, we show that the original elements of the codon theory continue to make important predictions for cerebellar mechanism, and we show that evidence appearing to contradict the original model is based on an artificially narrow interpretation of cerebellar structure and motor function.

Memory-Efficient Synaptic Connectivity for Spike-Timing- Dependent Plasticity.

Spike-Timing-Dependent Plasticity (STDP) is a bio-inspired local incremental weight update rule commonly used for online learning in spike-based neuromorphic systems. In STDP, the intensity of long-term potentiation and depression in synaptic efficacy (weight) between neurons is expressed as a function of the relative timing between pre- and post-synaptic action potentials (spikes), while the polarity of change is dependent on the order (causality) of the spikes. Online STDP weight updates for causal and acausal relative spike times are activated at the onset of post- and pre-synaptic spike events, respectively, implying access to synaptic connectivity both in forward (pre-to-post) and reverse (post-to-pre) directions. Here we study the impact of different arrangements of synaptic connectivity tables on weight storage and STDP updates for large-scale neuromorphic systems. We analyze the memory efficiency for varying degrees of density in synaptic connectivity, ranging from crossbar arrays for full connectivity to pointer-based lookup for sparse connectivity. The study includes comparison of storage and access costs and efficiencies for each memory arrangement, along with a trade-off analysis of the benefits of each data structure depending on application requirements and budget. Finally, we present an alternative formulation of STDP via a delayed causal update mechanism that permits efficient weight access, requiring no more than forward connectivity lookup. We show functional equivalence of the delayed causal updates to the original STDP formulation, with substantial savings in storage and access costs and efficiencies for networks with sparse synaptic connectivity as typically encountered in large-scale models in computational neuroscience.

The Importance of Space and Time for Signal Processing in Neuromorphic Agents: The Challenge of Developing Low-Power, Autonomous Agents That Interact With the Environment

Artificial neural networks (ANNs) and computational neuroscience models have made tremendous progress, enabling us to achieve impressive results in artificial intelligence applications, such as image recognition, natural language processing, and autonomous driving. Despite this, biological neural systems consume orders of magnitude less energy than today's ANNs and are much more flexible and robust. This adaptivity and efficiency gap is partially explained by the computing substrate of biological neural processing systems that is fundamentally different from the way today?s computers are built. Biological systems use in-memory computing elements operating in a massively parallel way rather than time-multiplexed computing units that are reused in a sequential fashion. Moreover, the activity of biological neurons follows continuous-time dynamics in real, physical time instead of operating on discrete temporal cycles abstracted away from real time.