Large-scale Neural Network Simulations Research Articles

Collective dynamics and shot-noise-induced switching in a two-population neural network.

Neural mass models are a powerful tool for modeling of neural populations. Such models are often used as building blocks for the simulation of large-scale neural networks and the whole brain. Here, we carry out systematic bifurcation analysis of a neural mass model for the basic motif of various neural circuits, a system of two populations, an excitatory, and an inhibitory ones. We describe the scenarios for the emergence of complex collective behavior, including chaotic oscillations and multistability. We also compare the dynamics of the neural mass model and the exact microscopic system and show that their agreement may be far from perfect. The discrepancy can be interpreted as the action of the so-called shot noise originating from finite-size effects. This shot noise can lead to the blurring of the neural mass dynamics or even turn its attractors into metastable states between which the system switches recurrently.

Biophysically detailed forward modeling of the neural origin of EEG and MEG signals

Electroencephalography (EEG) and magnetoencephalography (MEG) are among the most important techniques for non-invasively studying cognition and disease in the human brain. These signals are known to originate from cortical neural activity, typically described in terms of current dipoles. While the link between cortical current dipoles and EEG/MEG signals is relatively well understood, surprisingly little is known about the link between different kinds of neural activity and the current dipoles themselves. Detailed biophysical modeling has played an important role in exploring the neural origin of intracranial electric signals, like extracellular spikes and local field potentials. However, this approach has not yet been taken full advantage of in the context of exploring the neural origin of the cortical current dipoles that are causing EEG/MEG signals.Here, we present a method for reducing arbitrary simulated neural activity to single current dipoles. We find that the method is applicable for calculating extracranial signals, but less suited for calculating intracranial electrocorticography (ECoG) signals. We demonstrate that this approach can serve as a powerful tool for investigating the neural origin of EEG/MEG signals. This is done through example studies of the single-neuron EEG contribution, the putative EEG contribution from calcium spikes, and from calculating EEG signals from large-scale neural network simulations. We also demonstrate how the simulated current dipoles can be used directly in combination with detailed head models, allowing for simulated EEG signals with an unprecedented level of biophysical details.In conclusion, this paper presents a framework for biophysically detailed modeling of EEG and MEG signals, which can be used to better our understanding of non-inasively measured neural activity in humans.

Application of the Quasi-Static Memdiode Model in Cross-Point Arrays for Large Dataset Pattern Recognition

We investigate the use and performance of the quasi-static memdiode model (QMM) when incorporated into large cross-point arrays intended for pattern classification tasks. Following Chua's memristive devices theory, the QMM comprises two equations, one equation for the electron transport based on the double-diode circuit with single series resistance and a second equation for the internal memory state of the device based on the so-called logistic hysteron or memory map. Ex-situ trained memdiodes with different MNIST-like databases are used to establish the synaptic weights linking the top and bottom wire networks. The role played by the memdiode electrical parameters, wire resistance and capacitance values, image pixelation, connection schemes, signal-to-noise ratio and device-to-device variability in the classification effectiveness are investigated. The confusion matrix is used to benchmark the system performance metrics. We show that the simplicity, accuracy and robustness of the memdiode model makes it a suitable candidate for the realistic simulation of large-scale neural networks with non-idealities.

Synchronization and FPGA realization of fractional-order Izhikevich neuron model

This paper generalizes the Izhikevich neuron model in the fractional-order domain for better modeling of neuron dynamics. Accurate and computationally efficient numerical techniques such as non-standard finite difference (NSFD) scheme is used to solve the neuron system in the fractional-order domain for different cases. Neuron synchronization plays an important role in the process of information exchange among coupled neurons. The general formula for the synchronization of different Izhikevich neurons is proposed. Also, the synchronization of two and three neurons are studied at different fractional orders. Furthermore, the fractional-order regular spiking neuron of Izhikevich model is implemented on Xilinx (XC5VLX30T) Virtex 5 FPGA kit using only combinational logic. FPGAs are known for their reconfigurability and parallelism which makes them suitable for large-scale neural network simulations.

An O(n) method of calculating Kendall correlations of spike trains.

The ability to record from increasingly large numbers of neurons, and the increasing attention being paid to large scale neural network simulations, demands computationally fast algorithms to compute relevant statistical measures. We present an O(n) algorithm for calculating the Kendall correlation of spike trains, a correlation measure that is becoming especially recognized as an important tool in neuroscience. We show that our method is around 50 times faster than the O (n ln n) method which is a current standard for quickly computing the Kendall correlation. In addition to providing a faster algorithm, we emphasize the role that taking the specific nature of spike trains had on reducing the run time. We imagine that there are many other useful algorithms that can be even more significantly sped up when taking this into consideration. A MATLAB function executing the method described here has been made freely available on-line.

Performance Comparison of the Digital Neuromorphic Hardware SpiNNaker and the Neural Network Simulation Software NEST for a Full-Scale Cortical Microcircuit Model.

The digital neuromorphic hardware SpiNNaker has been developed with the aim of enabling large-scale neural network simulations in real time and with low power consumption. Real-time performance is achieved with 1 ms integration time steps, and thus applies to neural networks for which faster time scales of the dynamics can be neglected. By slowing down the simulation, shorter integration time steps and hence faster time scales, which are often biologically relevant, can be incorporated. We here describe the first full-scale simulations of a cortical microcircuit with biological time scales on SpiNNaker. Since about half the synapses onto the neurons arise within the microcircuit, larger cortical circuits have only moderately more synapses per neuron. Therefore, the full-scale microcircuit paves the way for simulating cortical circuits of arbitrary size. With approximately 80, 000 neurons and 0.3 billion synapses, this model is the largest simulated on SpiNNaker to date. The scale-up is enabled by recent developments in the SpiNNaker software stack that allow simulations to be spread across multiple boards. Comparison with simulations using the NEST software on a high-performance cluster shows that both simulators can reach a similar accuracy, despite the fixed-point arithmetic of SpiNNaker, demonstrating the usability of SpiNNaker for computational neuroscience applications with biological time scales and large network size. The runtime and power consumption are also assessed for both simulators on the example of the cortical microcircuit model. To obtain an accuracy similar to that of NEST with 0.1 ms time steps, SpiNNaker requires a slowdown factor of around 20 compared to real time. The runtime for NEST saturates around 3 times real time using hybrid parallelization with MPI and multi-threading. However, achieving this runtime comes at the cost of increased power and energy consumption. The lowest total energy consumption for NEST is reached at around 144 parallel threads and 4.6 times slowdown. At this setting, NEST and SpiNNaker have a comparable energy consumption per synaptic event. Our results widen the application domain of SpiNNaker and help guide its development, showing that further optimizations such as synapse-centric network representation are necessary to enable real-time simulation of large biological neural networks.

Analytical modelling of temperature effects on an AMPA-type synapse

It was previously reported, that temperature may significantly influence neural dynamics on the different levels of brain function. Thus, in computational neuroscience, it would be useful to make models scalable for a wide range of various brain temperatures. However, lack of experimental data and an absence of temperature-dependent analytical models of synaptic conductance does not allow to include temperature effects at the multi-neuron modeling level. In this paper, we propose a first step to deal with this problem: A new analytical model of AMPA-type synaptic conductance, which is able to incorporate temperature effects in low-frequency stimulations. It was constructed based on Markov model description of AMPA receptor kinetics using the set of coupled ODEs. The closed-form solution for the set of differential equations was found using uncoupling assumption (introduced in the paper) with few simplifications motivated both from experimental data and from Monte Carlo simulation of synaptic transmission. The model may be used for computationally efficient and biologically accurate implementation of temperature effects on AMPA receptor conductance in large-scale neural network simulations. As a result, it may open a wide range of new possibilities for researching the influence of temperature on certain aspects of brain functioning.

Bidirectional Non-Filamentary RRAM as an Analog Neuromorphic Synapse, Part I: Al/Mo/Pr0.7Ca0.3MnO3 Material Improvements and Device Measurements

We report on material improvements to non-filamentary RRAM devices based on Pr0.7Ca0.3MnO3 by introducing an MoOx buffer layer together with a reactive Al electrode, and on device measurements designed to help gauge the performance of these devices as bidirectional analog synapses for on-chip acceleration of the backpropagation algorithm. Previous Al/PCMO devices exhibited degraded LRS retention due to the low activation energy for oxidation of the Al electrode, and Mo/PCMO devices showed low conductance contrast. To control the redox reaction at the metal/PCMO interface, we introduce a 4-nm interfacial layer of conducting MoOx as an oxygen buffer layer. Due to the controlled redox reaction within this Al/Mo/PCMO device, we observed improvements in both retention and conductance on/off ratio. We confirm bidirectional analog synapse characteristics and measure “jump-tables” suitable for large scale neural network simulations that attempt to capture complex and stochastic device behavior [see companion paper]. Finally, switching energy measurements are shown, illustrating a path for future device research toward smaller devices, shorter pulses and lower programming voltages.

Robby: A Neurorobotics Control Framework Using Spiking Neural Networks

The variety of neural models and robotic hardware has made simulation writing time-consuming and error prone, forcing thus scientists to spend a substantial amount of time on the implementation of their models. We developed a framework called Robby that allows the quick simulation of large-scale neural networks designed for robotic control by spiking neural networks. It provides both mechanism for robotic communication and tools for building and simulating neural controllers. We present the basic building blocks of Robby and a simple experiment to show its practical value.

Parallel Brain Simulator: A Multi-scale and Parallel Brain-Inspired Neural Network Modeling and Simulation Platform

The brain is naturally a parallel and distributed system. Reverse engineering a cognitive brain is considered to be a grand challenge. In this paper, we present the parallel brain simulator (PBS), a parallel and distributed platform for modeling the cognitive brain at multiple scales. Inspired by large-scale graph computation, PBS can be considered as a universal parallel execution engine, which is aimed at reducing the complexity of distributed programming and providing an easy to use programmable platform for computational neuroscientists and artificial intelligence researchers for modeling and simulation of large-scale neural networks. As illustrative examples and validations, three brain-inspired neural networks which are built on PBS are introduced, including the 1:1 human hippocampus network, the 1:1 mouse whole-brain network and the CASIA brain simulator built for cognitive robotics. We deploy PBS on both commodity clusters and supercomputers, and a scalable performance is achieved. In addition, we provide evaluations on the scalability and performance of both lumped synapse-based simulation and non-lumped synapse-based simulation with different data-graph distribution methods to show the effectiveness and usability of the PBS platform.

NeuroFlow: A General Purpose Spiking Neural Network Simulation Platform using Customizable Processors.

NeuroFlow is a scalable spiking neural network simulation platform for off-the-shelf high performance computing systems using customizable hardware processors such as Field-Programmable Gate Arrays (FPGAs). Unlike multi-core processors and application-specific integrated circuits, the processor architecture of NeuroFlow can be redesigned and reconfigured to suit a particular simulation to deliver optimized performance, such as the degree of parallelism to employ. The compilation process supports using PyNN, a simulator-independent neural network description language, to configure the processor. NeuroFlow supports a number of commonly used current or conductance based neuronal models such as integrate-and-fire and Izhikevich models, and the spike-timing-dependent plasticity (STDP) rule for learning. A 6-FPGA system can simulate a network of up to ~600,000 neurons and can achieve a real-time performance of 400,000 neurons. Using one FPGA, NeuroFlow delivers a speedup of up to 33.6 times the speed of an 8-core processor, or 2.83 times the speed of GPU-based platforms. With high flexibility and throughput, NeuroFlow provides a viable environment for large-scale neural network simulation.

NeuroXyce: a highly parallelized simulator for biologically realistic neural networks

Event Abstract Back to Event NeuroXyce: a highly parallelized simulator for biologically realistic neural networks Christina Warrender1*, James Aimone1, Corinne Teeter1 and Richard Schiek1 1 Sandia National Laboratories, United States The increasing availability of high performance computing platforms, either through supercomputers or cloud computing, offers tremendous potential to computational neuroscientists interested in simulating biologically realistic networks at large scales. Unfortunately, tools that take full advantage of these platforms have been slow to develop, and the parallelization of neural simulations represents a non-trivial amount of work. In current network simulators the parallelization scheme is often specified by the user. This specification can be quite arduous and often the user is uninformed of which scheme is optimal. This is noteworthy since parallelization techniques can substantially influence the run time of large-scale neural network simulations, and a poorly parallelized model may offer little or no advantage over conventional approaches. We have created a simulator capable of simulating multicompartment, branched neurons with ion channels by building on the previously existing Xyce parallel electronic circuit simulator (xyce.sandia.gov). NeuroXyce uses advanced parallel integration and solver methods, and automatically handles load balancing among multiple processors, removing this burden from the user. Here we demonstrate the scalability of NeuroXyce and compare the simulation run time and ease of use with other popular simulators (i.e. NEURON). Our simulation paradigm consists of a network of 80 percent excitatory neurons and 20 percent inhibitory neurons. Neurons have Hodgkin-Huxley sodium and potassium channel dynamics. The neurons are randomly connected with a probability of 0.02. The strength of the synapses scales depending on the size of the network (10,000 to 1,000,000 neurons). The excitatory connections simulate AMPA synapses and the inhibitory connections simulate GABA synapses. We measure simulation run time and network dynamics as the size of the network is increased. Keywords: Large scale modeling, neural networks, computational neuroscience, biological networks, Neural Simulation Conference: 5th INCF Congress of Neuroinformatics, Munich, Germany, 10 Sep - 12 Sep, 2012. Presentation Type: Poster Topic: Neuroinformatics Citation: Warrender C, Aimone J, Teeter C and Schiek R (2014). NeuroXyce: a highly parallelized simulator for biologically realistic neural networks. Front. Neuroinform. Conference Abstract: 5th INCF Congress of Neuroinformatics. doi: 10.3389/conf.fninf.2014.08.00105 Copyright: The abstracts in this collection have not been subject to any Frontiers peer review or checks, and are not endorsed by Frontiers. They are made available through the Frontiers publishing platform as a service to conference organizers and presenters. The copyright in the individual abstracts is owned by the author of each abstract or his/her employer unless otherwise stated. Each abstract, as well as the collection of abstracts, are published under a Creative Commons CC-BY 4.0 (attribution) licence (https://creativecommons.org/licenses/by/4.0/) and may thus be reproduced, translated, adapted and be the subject of derivative works provided the authors and Frontiers are attributed. For Frontiers’ terms and conditions please see https://www.frontiersin.org/legal/terms-and-conditions. Received: 21 Mar 2013; Published Online: 27 Feb 2014. * Correspondence: Dr. Christina Warrender, Sandia National Laboratories, unset, United States, cewarr@sandia.gov Login Required This action requires you to be registered with Frontiers and logged in. To register or login click here. Abstract Info Abstract The Authors in Frontiers Christina Warrender James Aimone Corinne Teeter Richard Schiek Google Christina Warrender James Aimone Corinne Teeter Richard Schiek Google Scholar Christina Warrender James Aimone Corinne Teeter Richard Schiek PubMed Christina Warrender James Aimone Corinne Teeter Richard Schiek Related Article in Frontiers Google Scholar PubMed Abstract Close Back to top Javascript is disabled. Please enable Javascript in your browser settings in order to see all the content on this page.

CPU-GPU hybrid platform for efficient spiking neural-network simulation

Nowadays, research in computational neuroscience is progressively demanding both detailed biologically-plausible neuron models and, at the same time, the simulation of large-scale neural networks in order to better understand the operation of specific nervous circuits of the central nervous system. To that aim, several neural simulators have been developed during last decades; these simulators have been conceived to either simulate detailed neuron models within small-scale neural networks (NEURON [1] and GENESIS [2]), or to simulate neuron models with low degree of biophysical detail within large-scale neural networks (Brian [3] and NEST [4]). In view of this situation, it would be desirable to go a step further in simulating neural networks and combine fast-and-simple neural models with detailed biologically-plausible neurons within large-scale neural networks. To achieve this goal, we have integrated in our generic neural simulator (EDLUT) [5] two simulation methods (time driven and event driven) with two different hardware processing architectures (CPU and GPU). (For an overview about current types of simulation strategies and algorithms the reader is referred to Brette [6]). It is well known that traditional CPU architectures present good performance when simulating small-scale neural network, however, they still present some drawbacks in case of large-scale simulations. To avoid CPU-based architecture limitations, we have enhanced our simulator efficiency by making it capable of running in a hybrid CPU-GPU platform. In this on-trend CPU-GPU platform, processors can operate conjointly to accomplish a real-time simulation taking full advantage of GPU high performance and CPU versatility. In this work we present how our neural network simulator EDLUT has been upgraded implementing most of those techniques referred in [5] making it capable of being configured in an hybrid event-and-time-driven simulating architecture (results show that using this architecture, up to 50 times higher speed-up values are obtained than using a stand-alone CPU approach). Once relatively-large-scale simulations can be achieved, the detailed biological neuron models to be simulated become fundamental. These neuron model simulations are intrinsically associated to their techniques of numerical integration. In other words, while numerical integration methods such as Euler and Runge-Kutta are computationally efficient and suitable for neuron models with non-stiff dynamics, more complex numerical integration methods, such as backward differentiation formula (BDF), are usually required for more complex neuron models. We have extended EDLUT with the ability of dealing with complex neural models and also with detailed network characteristics such as spike propagation delays. This point has been revealed as a key factor for learning laws based on spike-timing dependent plasticity. To sum up, we present here an event-and-time-driven neural network simulator in a hybrid CPU-GPU platform that can simultaneously use simple or complex integration methods.

Comparison of GPU- and CPU-implementations of mean-firing rate neural networks on parallel hardware

Modern parallel hardware such as multi-core processors (CPUs) and graphics processing units (GPUs) have a high computational power which can be greatly beneficial to the simulation of large-scale neural networks. Over the past years, a number of efforts have focused on developing parallel algorithms and simulators best suited for the simulation of spiking neural models. In this article, we aim at investigating the advantages and drawbacks of the CPU and GPU parallelization of mean-firing rate neurons, widely used in systems-level computational neuroscience. By comparing OpenMP, CUDA and OpenCL implementations towards a serial CPU implementation, we show that GPUs are better suited than CPUs for the simulation of very large networks, but that smaller networks would benefit more from an OpenMP implementation. As this performance strongly depends on data organization, we analyze the impact of various factors such as data structure, memory alignment and floating precision. We then discuss the suitability of the different hardware depending on the networks' size and connectivity, as random or sparse connectivities in mean-firing rate networks tend to break parallel performance on GPUs due to the violation of coalescence.

Streaming parallel GPU acceleration of large-scale filter-based spiking neural networks

The arrival of graphics processing (GPU) cards suitable for massively parallel computing promises affordable large-scale neural network simulation previously only available at supercomputing facilities. While the raw numbers suggest that GPUs may outperform CPUs by at least an order of magnitude, the challenge is to develop fine-grained parallel algorithms to fully exploit the particulars of GPUs. Computation in a neural network is inherently parallel and thus a natural match for GPU architectures: given inputs, the internal state for each neuron can be updated in parallel. We show that for filter-based spiking neurons, like the Spike Response Model, the additive nature of membrane potential dynamics enables additional update parallelism. This also reduces the accumulation of numerical errors when using single precision computation, the native precision of GPUs. We further show that optimizing simulation algorithms and data structures to the GPU's architecture has a large pay-off: for example, matching iterative neural updating to the memory architecture of the GPU speeds up this simulation step by a factor of three to five. With such optimizations, we can simulate in better-than-realtime plausible spiking neural networks of up to 50 000 neurons, processing over 35 million spiking events per second.

Analytical Integrate-and-Fire Neuron Models with Conductance-Based Dynamics and Realistic Postsynaptic Potential Time Course for Event-Driven Simulation Strategies

In a previous paper (Rudolph & Destexhe, 2006), we proposed various models, the gIF neuron models, of analytical integrate-and-fire (IF) neurons with conductance-based (COBA) dynamics for use in event-driven simulations. These models are based on an analytical approximation of the differential equation describing the IF neuron with exponential synaptic conductances and were successfully tested with respect to their response to random and oscillating inputs. Because they are analytical and mathematically simple, the gIF models are best suited for fast event-driven simulation strategies. However, the drawback of such models is they rely on a nonrealistic postsynaptic potential (PSP) time course, consisting of a discontinuous jump followed by a decay governed by the membrane time constant. Here, we address this limitation by conceiving an analytical approximation of the COBA IF neuron model with the full PSP time course. The subthreshold and suprathreshold response of this gIF4 model reproduces remarkably well the postsynaptic responses of the numerically solved passive membrane equation subject to conductance noise, while gaining at least two orders of magnitude in computational performance. Although the analytical structure of the gIF4 model is more complex than that of its predecessors due to the necessity of calculating future spike times, a simple and fast algorithmic implementation for use in large-scale neural network simulations is proposed.

Studies of stimulus parameters for seizure disruption using neural network simulations

A large scale neural network simulation with realistic cortical architecture has been undertaken to investigate the effects of external electrical stimulation on the propagation and evolution of ongoing seizure activity. This is an effort to explore the parameter space of stimulation variables to uncover promising avenues of research for this therapeutic modality. The model consists of an approximately 800 mum x 800 mum region of simulated cortex, and includes seven neuron classes organized by cortical layer, inhibitory or excitatory properties, and electrophysiological characteristics. The cell dynamics are governed by a modified version of the Hodgkin-Huxley equations in single compartment format. Axonal connections are patterned after histological data and published models of local cortical wiring. Stimulation induced action potentials take place at the axon initial segments, according to threshold requirements on the applied electric field distribution. Stimulation induced action potentials in horizontal axonal branches are also separately simulated. The calculations are performed on a 16 node distributed 32-bit processor system. Clear differences in seizure evolution are presented for stimulated versus the undisturbed rhythmic activity. Data is provided for frequency dependent stimulation effects demonstrating a plateau effect of stimulation efficacy as the applied frequency is increased from 60 to 200 Hz. Timing of the stimulation with respect to the underlying rhythmic activity demonstrates a phase dependent sensitivity. Electrode height and position effects are also presented. Using a dipole stimulation electrode arrangement, clear orientation effects of the dipole with respect to the model connectivity is also demonstrated. A sensitivity analysis of these results as a function of the stimulation threshold is also provided.

Towards cortex sized artificial neural systems

We propose, implement, and discuss an abstract model of the mammalian neocortex. This model is instantiated with a sparse recurrently connected neural network that has spiking leaky integrator units and continuous Hebbian learning. First we study the structure, modularization, and size of neocortex, and then we describe a generic computational model of the cortical circuitry. A characterizing feature of the model is that it is based on the modularization of neocortex into hypercolumns and minicolumns. Both a floating- and fixed-point arithmetic implementation of the model are presented along with simulation results. We conclude that an implementation on a cluster computer is not communication but computation bounded. A mouse and rat cortex sized version of our model executes in 44% and 23% of real-time respectively. Further, an instance of the model with 1.6 x 10(6) units and 2 x 10(11) connections performed noise reduction and pattern completion. These implementations represent the current frontier of large-scale abstract neural network simulations in terms of network size and running speed.

Programming the cerebellum

It is argued that large-scale neural network simulations of cerebellar cortex and nuclei, based on realistic compartmental models of me major cell populations, are necessary before the problem of motor learning in the cerebellum can be solved, [HOUK et al.; SIMPSON et al.]

Massively parallel architectures for large scale neural network simulations

A toroidal lattice architecture (TLA) and a planar lattice architecture (PLA) are proposed as massively parallel neurocomputer architectures for large-scale simulations. The performance of these architectures is almost proportional to the number of node processors, and they adopt the most efficient two-dimensional processor connections for WSI implementation. They also give a solution to the connectivity problem, the performance degradation caused by the data transmission bottleneck, and the load balancing problem for efficient parallel processing in large-scale neural network simulations. The general neuron model is defined. Implementation of the TLA with transputers is described. A Hopfield neural network and a multilayer perceptron have been implemented and applied to the traveling salesman problem and to identity mapping, respectively. Proof that the performance increases almost in proportion to the number of node processors is given.