Large-scale Network Simulations Research Articles

Performance Comparison of the Digital Neuromorphic Hardware SpiNNaker and the Neural Network Simulation Software NEST for a Full-Scale Cortical Microcircuit Model.

The digital neuromorphic hardware SpiNNaker has been developed with the aim of enabling large-scale neural network simulations in real time and with low power consumption. Real-time performance is achieved with 1 ms integration time steps, and thus applies to neural networks for which faster time scales of the dynamics can be neglected. By slowing down the simulation, shorter integration time steps and hence faster time scales, which are often biologically relevant, can be incorporated. We here describe the first full-scale simulations of a cortical microcircuit with biological time scales on SpiNNaker. Since about half the synapses onto the neurons arise within the microcircuit, larger cortical circuits have only moderately more synapses per neuron. Therefore, the full-scale microcircuit paves the way for simulating cortical circuits of arbitrary size. With approximately 80, 000 neurons and 0.3 billion synapses, this model is the largest simulated on SpiNNaker to date. The scale-up is enabled by recent developments in the SpiNNaker software stack that allow simulations to be spread across multiple boards. Comparison with simulations using the NEST software on a high-performance cluster shows that both simulators can reach a similar accuracy, despite the fixed-point arithmetic of SpiNNaker, demonstrating the usability of SpiNNaker for computational neuroscience applications with biological time scales and large network size. The runtime and power consumption are also assessed for both simulators on the example of the cortical microcircuit model. To obtain an accuracy similar to that of NEST with 0.1 ms time steps, SpiNNaker requires a slowdown factor of around 20 compared to real time. The runtime for NEST saturates around 3 times real time using hybrid parallelization with MPI and multi-threading. However, achieving this runtime comes at the cost of increased power and energy consumption. The lowest total energy consumption for NEST is reached at around 144 parallel threads and 4.6 times slowdown. At this setting, NEST and SpiNNaker have a comparable energy consumption per synaptic event. Our results widen the application domain of SpiNNaker and help guide its development, showing that further optimizations such as synapse-centric network representation are necessary to enable real-time simulation of large biological neural networks.

Analytical modelling of temperature effects on an AMPA-type synapse

It was previously reported, that temperature may significantly influence neural dynamics on the different levels of brain function. Thus, in computational neuroscience, it would be useful to make models scalable for a wide range of various brain temperatures. However, lack of experimental data and an absence of temperature-dependent analytical models of synaptic conductance does not allow to include temperature effects at the multi-neuron modeling level. In this paper, we propose a first step to deal with this problem: A new analytical model of AMPA-type synaptic conductance, which is able to incorporate temperature effects in low-frequency stimulations. It was constructed based on Markov model description of AMPA receptor kinetics using the set of coupled ODEs. The closed-form solution for the set of differential equations was found using uncoupling assumption (introduced in the paper) with few simplifications motivated both from experimental data and from Monte Carlo simulation of synaptic transmission. The model may be used for computationally efficient and biologically accurate implementation of temperature effects on AMPA receptor conductance in large-scale neural network simulations. As a result, it may open a wide range of new possibilities for researching the influence of temperature on certain aspects of brain functioning.

NeuroSpike viewer: A graphical environment for efficient control, communication and display of large-scale real-time simulation of Spiking Neural Networks on embedded systems

In recent years, the scientific community's interest for developing software or hardware tools capable of processing and displaying real-time simulations of large-scale Spiking Neural Networks (SNN) has been growing exponentially. Visualization of the simulations is crucial to observe and study the dynamics of the SNN, because these models involve high level of biological realism by encoding spatial-temporal information into action potentials or spikes like biological neurons do. Existing tools are based on supercomputers or customized neuromorphic architectures to compute large-scale SNN models. However, many of them are unsuitable to carry out real-time analysis of large scale SNN with minimal cost as large amount of complex data needs to be processed and displayed at the same time. This work intends to provide an efficient cost effective and power efficient emulation tool based on hardware-software hybrid to process, control, and display large-scale SNN network behavior in real time. The solution consists of a customized hardware architecture that computes the neural-synaptic parameters and a software environment that displays the same on the screen of a general purpose computer (Host). The data to be visualized is transferred to the host through high speed Ethernet link. The proposed solution tailors the use of the Ethernet link bandwidth to an optimum such that the customized neuromorphic architecture and the host computer duo can be used to visualize the dynamics of SNN network of any size in real-time. The hardware and software platforms are scalable in terms of number of neurons and synapses. This solution gives flexibility to the user to visualize and analyze SNN network of any size avoiding saturation of the high-speed serial Ethernet link while supporting the visualization of SNN dynamics with a good screen resolution.

Real-Time Neuromorphic System for Large-Scale Conductance-Based Spiking Neural Networks.

The investigation of the human intelligence, cognitive systems and functional complexity of human brain is significantly facilitated by high-performance computational platforms. In this paper, we present a real-time digital neuromorphic system for the simulation of large-scale conductance-based spiking neural networks (LaCSNN), which has the advantages of both high biological realism and large network scale. Using this system, a detailed large-scale cortico-basal ganglia-thalamocortical loop is simulated using a scalable 3-D network-on-chip (NoC) topology with six Altera Stratix III field-programmable gate arrays simulate 1 million neurons. Novel router architecture is presented to deal with the communication of multiple data flows in the multinuclei neural network, which has not been solved in previous NoC studies. At the single neuron level, cost-efficient conductance-based neuron models are proposed, resulting in the average utilization of 95% less memory resources and 100% less DSP resources for multiplier-less realization, which is the foundation of the large-scale realization. An analysis of the modified models is conducted, including investigation of bifurcation behaviors and ionic dynamics, demonstrating the required range of dynamics with a more reduced resource cost. The proposed LaCSNN system is shown to outperform the alternative state-of-the-art approaches previously used to implement the large-scale spiking neural network, and enables a broad range of potential applications due to its real-time computational power.

An FPGA-Based Massively Parallel Neuromorphic Cortex Simulator.

This paper presents a massively parallel and scalable neuromorphic cortex simulator designed for simulating large and structurally connected spiking neural networks, such as complex models of various areas of the cortex. The main novelty of this work is the abstraction of a neuromorphic architecture into clusters represented by minicolumns and hypercolumns, analogously to the fundamental structural units observed in neurobiology. Without this approach, simulating large-scale fully connected networks needs prohibitively large memory to store look-up tables for point-to-point connections. Instead, we use a novel architecture, based on the structural connectivity in the neocortex, such that all the required parameters and connections can be stored in on-chip memory. The cortex simulator can be easily reconfigured for simulating different neural networks without any change in hardware structure by programming the memory. A hierarchical communication scheme allows one neuron to have a fan-out of up to 200 k neurons. As a proof-of-concept, an implementation on one Altera Stratix V FPGA was able to simulate 20 million to 2.6 billion leaky-integrate-and-fire (LIF) neurons in real time. We verified the system by emulating a simplified auditory cortex (with 100 million neurons). This cortex simulator achieved a low power dissipation of 1.62 μW per neuron. With the advent of commercially available FPGA boards, our system offers an accessible and scalable tool for the design, real-time simulation, and analysis of large-scale spiking neural networks.

Neuromodulated Synaptic Plasticity on the SpiNNaker Neuromorphic System.

SpiNNaker is a digital neuromorphic architecture, designed specifically for the low power simulation of large-scale spiking neural networks at speeds close to biological real-time. Unlike other neuromorphic systems, SpiNNaker allows users to develop their own neuron and synapse models as well as specify arbitrary connectivity. As a result SpiNNaker has proved to be a powerful tool for studying different neuron models as well as synaptic plasticity—believed to be one of the main mechanisms behind learning and memory in the brain. A number of Spike-Timing-Dependent-Plasticity(STDP) rules have already been implemented on SpiNNaker and have been shown to be capable of solving various learning tasks in real-time. However, while STDP is an important biological theory of learning, it is a form of Hebbian or unsupervised learning and therefore does not explain behaviors that depend on feedback from the environment. Instead, learning rules based on neuromodulated STDP (three-factor learning rules) have been shown to be capable of solving reinforcement learning tasks in a biologically plausible manner. In this paper we demonstrate for the first time how a model of three-factor STDP, with the third-factor representing spikes from dopaminergic neurons, can be implemented on the SpiNNaker neuromorphic system. Using this learning rule we first show how reward and punishment signals can be delivered to a single synapse before going on to demonstrate it in a larger network which solves the credit assignment problem in a Pavlovian conditioning experiment. Because of its extra complexity, we find that our three-factor learning rule requires approximately 2× as much processing time as the existing SpiNNaker STDP learning rules. However, we show that it is still possible to run our Pavlovian conditioning model with up to 1 × 104 neurons in real-time, opening up new research opportunities for modeling behavioral learning on SpiNNaker.

Robust Modulation of Integrate-and-Fire Models

By controlling the state of neuronal populations, neuromodulators ultimately affect behavior. A key neuromodulation mechanism is the alteration of neuronal excitability via the modulation of ion channel expression. This type of neuromodulation is normally studied with conductance-based models, but those models are computationally challenging for large-scale network simulations needed in population studies. This article studies the modulation properties of the multiquadratic integrate-and-fire model, a generalization of the classical quadratic integrate-and-fire model. The model is shown to combine the computational economy of integrate-and-fire modeling and the physiological interpretability of conductance-based modeling. It is therefore a good candidate for affordable computational studies of neuromodulation in large networks.

SIM-Cumulus: An Academic Cloud for the Provisioning of Network-Simulation-as-a-Service (NSaaS)

Large-scale network simulations are resource and time intensive tasks due to a number of factors i.e., setup configuration, computation time, hardware, and energy cost. These factors ultimately force network researchers to scale-down the scope of experiments, either in terms of simulation entities involved or in abridging expected micro-level details. The Cloud technology facilitates researchers to address mentioned factors by the provisioning of pre-configured instances on shared infrastructure. In this paper, an academic Cloud architecture SIM-Cumulus targeting the research institutions is proposed. SIM-Cumulus provides the framework of virtual machine instances specifically configured for large-scale network simulations, with the aim of efficiency in terms of simulation execution time and energy cost. The performance of SIM-Cumulus is evaluated using large-scale wireless network simulations. Simulation results show that SIM-Cumulus is beneficial in three aspects i.e., 1) promotion of research within the domain of computer networks; 2) consumption of considerably fewer resources in terms of simulation elapsed time and usage cost; and 3) reduction of carbon emission leading toward sustainable IT development.

Harnessing the Power of HPC in Simulation and Optimization of Large Transportation Networks: Spatio-Temporal Traffic Management in the Greater Toronto Area

The significant growth of many urban areas comes at a cost of increasing demand for mobility and traffic congestion in large-scale urban environments. As congestion levels soar to unprecedented levels, and researchers and governments are challenged addressing the basic needs for transportation and mobility; solutions are becoming more complex and untraditional, creating a strong potential for optimization and simulation of large-scale transportation networks. In this paper, we present a generic traffic management framework for solving large-scale constraint optimization problems in advanced Intelligent Transportation systems (ITS) applications. The framework employs a distributed computing approach to enable replicating analysis of large-scale traffic simulation networks, providing a practical mechanism for solving complex optimization problems in transportation applications. We discuss employing the framework in two use cases, congestion pricing and emergency evacuation, targeting optimization of spatial and temporal traffic management.

Robby: A Neurorobotics Control Framework Using Spiking Neural Networks

The variety of neural models and robotic hardware has made simulation writing time-consuming and error prone, forcing thus scientists to spend a substantial amount of time on the implementation of their models. We developed a framework called Robby that allows the quick simulation of large-scale neural networks designed for robotic control by spiking neural networks. It provides both mechanism for robotic communication and tools for building and simulating neural controllers. We present the basic building blocks of Robby and a simple experiment to show its practical value.

New class of reduced computationally efficient neuronal models for large-scale simulations of brain dynamics

During slow-wave sleep, brain electrical activity is dominated by the slow (< 1Hz) electroencephalogram (EEG) oscillations characterized by the periodic transitions between active (or Up) and silent (or Down) states in the membrane voltage of the cortical and thalamic neurons. Sleep slow oscillation is believed to play critical role in consolidation of recent memories. Past computational studies, based on the Hodgkin-Huxley type neuronal models, revealed possible intracellular and network mechanisms of the neuronal activity during sleep, however, they failed to explore the large-scale cortical network dynamics depending on collective behavior in the large populations of neurons. In this new study, we developed a novel class of reduced discrete time spiking neuron models for large-scale network simulations of wake and sleep dynamics. In addition to the spiking mechanism, the new model implemented nonlinearities capturing effects of the leak current, the Ca2+ dependent K+ current and the persistent Na+ current that were found to be critical for transitions between Up and Down states of the slow oscillation. We applied the new model to study large-scale two-dimensional cortical network activity during slow-wave sleep. Our study explained traveling wave dynamics and characteristic synchronization properties of transitions between Up and Down states of the slow oscillation as observed in vivo in recordings from cats. We further predict a critical role of synaptic noise and slow adaptive currents for spike sequence replay as found during sleep related memory consolidation.

Fast Parallel Lightpath Re-Optimization for Space-Division Multiplexing Optical Networks Based on Time Synchronization

In recent years, driven by the rapid growth of global Internet traffic, optical networks are evolving to provide higher capacities. To enlarge transmission capacity, researchers focused on the dimension of space and proposed the concept of space-division multiplexing (SDM) by introducing multicore fiber and few-mode fiber into optical networks. In SDM networks, one of the main challenges is the crosstalk among SDM (core and mode) channels, which degrades signal quality. This degradation has a serious impact on routing and resource allocation. Previous studies have mostly been confined to the routing algorithms for crosstalk reduction but few focus on the re-optimization mechanisms in SDM networks, defined as re-routing the existing lightpaths to minimize the overall crosstalk without traffic disruption. In this paper, by analyzing the re-routing signaling processes of two straightforward serial re-optimization mechanisms, we find that the signaling interaction between node controllers and serial re-routing execution leads to long re-optimization times, which makes lightpath re-optimization impossible for practical use. In order to achieve fast lightpath re-optimization for crosstalk reduction in SDM networks, we propose a novel software-defined networking (SDN) based parallel lightpath re-optimization mechanism, enabled by high-precision time synchronization. The basic idea of our proposed parallel lightpath re-optimization mechanism is to make all the nodes execute the switching operations co-ordinately with the help of network time synchronization. Besides, we propose two key algorithms to achieve parallel re-optimization: a simulated annealing algorithm to determine the re-routing sequence of the lightpaths and a time arrangement algorithm to determine the exact time points of the switching operations at each node. Finally, we conduct a prototype experiment and large-scale network simulations to evaluate the performance of the proposed mechanism. The results demonstrate that our parallel re-optimization mechanism can reduce the overall crosstalk in a much shorter time by up to three orders of magnitude compared with the conventional serial mechanisms and thus is more practical for real applications.

A big data approach for clustering and calibration of link fundamental diagrams for large-scale network simulation applications

Existing methods for calibrating link fundamental diagrams (FDs) often focus on a limited number of links and use grouping strategies that are largely dependent on roadway physical attributes alone. In this study, we propose a big data-driven two-stage clustering framework to calibrate link FDs for freeway networks. The first stage captures, under normal traffic state, the variations of link FDs over multiple days based on which links are clustered in the second stage. Two methods, i.e. the standard k-means algorithm combined with hierarchical clustering and a modified hierarchical clustering based on the Fréchet distance, are applied in the first stage to obtain the FD parameter matrix for each link. The calibrated matrices are input into the second stage where the modified hierarchical clustering is re-employed asa static approach resulting in multiple clusters of links. To further consider the variations of link FDs, the static approach is extended by modifying the similarity measure through the principle component analysis (PCA). The resulting multivariate time-series clustering models the distributions of the FD parameters asa dynamic approach. The proposed framework is applied on the Melbourne freeway network using one-year worth of loop detector data. Results have shown that (a) similar roadway physical attributes do not necessarily result in similar link FDs, (b) the connectivity-based approach performs better in clustering link FDs as compared with the centroid-based approach, and (c) the proposed framework helps achieving a better understanding of the spatial distribution of links with similar FDs and the associated variations and distributions of the FD parameters.

PeerFlow: Secure Load Balancing in Tor

AbstractWe present PeerFlow, a system to securely load balance client traffic in Tor. Security in Tor requires that no adversary handle too much traffic. However, Tor relays are run by volunteers who cannot be trusted to report the relay bandwidths, which Tor clients use for load balancing. We show that existing methods to determine the bandwidths of Tor relays allow an adversary with little bandwidth to attack large amounts of client traffic. These methods include Tor’s current bandwidth-scanning system, TorFlow, and the peer-measurement system EigenSpeed. We present an improved design called PeerFlow that uses a peer-measurement process both to limit an adversary’s ability to increase his measured bandwidth and to improve accuracy. We show our system to be secure, fast, and efficient. We implement PeerFlow in Tor and demonstrate its speed and accuracy in large-scale network simulations.

Amazon Cloud Computing Platform EC2 and VANET Simulations

Network simulations are resource and time intensive tasks due to the involvement of a number of factors attributable to scalability with respect to computation time, cost, and energy. Academic clouds are employed for large-scale network simulations, which require extensive capacity and intelligent resource management. This paper explores the performance efficiency of cloud solution to facilitate network researchers by proposing the utilisation of network simulator configured virtual machines. The prototype framework is executed using Amazon elastic computing cloud (EC2) Windows' instances configured with OPNET simulator. Simulation results show three significant benefits i.e., i) for scalable network simulations, consumption of considerably fewer resources in terms of simulation elapsed time, hardware resources and usage costs; ii) reduction of carbon emission leading towards sustainable IT; iii) promotion of network research by making complicated issues like simulation scalability and setup (installation or configuration) of network simulators to facilitate researchers and students.

Large-scale dynamic transportation network simulation: A space-time-event parallel computing approach

This paper describes a computationally efficient parallel-computing framework for mesoscopic transportation simulation on large-scale networks. By introducing an overall data structure for mesoscopic dynamic transportation simulation, we discuss a set of implementation issues for enabling flexible parallel computing on a multi-core shared memory architecture. First, we embed an event-based simulation logic to implement a simplified kinematic wave model and reduce simulation overhead. Second, we present a space-time-event computing framework to decompose simulation steps to reduce communication overhead in parallel execution and an OpenMP-based space-time-processor implementation method that is used to automate task partition tasks. According to the spatial and temporal attributes, various types of simulation events are mapped to independent logical processes that can concurrently execute their procedures while maintaining good load balance. We propose a synchronous space-parallel simulation strategy to dynamically assign the logical processes to different threads. The proposed method is then applied to simulate large-scale, real-world networks to examine the computational efficiency under different numbers of CPU threads. Numerical experiments demonstrate that the implemented parallel computing algorithm can significantly improve the computational efficiency and it can reach up to a speedup of 10 on a workstation with 32 computing threads.

Dimension reduction in heterogeneous neural networks: Generalized Polynomial Chaos (gPC) and ANalysis-Of-VAriance (ANOVA)

We propose, and illustrate via a neural network example, two different approaches to coarse-graining large heterogeneous networks. Both approaches are inspired from, and use tools developed in, methods for uncertainty quantification in systems with multiple uncertain parameters - in our case, the parameters are heterogeneously distributed on the network nodes. The approach shows promise in accelerating large scale network simulations as well as coarse-grained fixed point, periodic solution and stability analysis. We also demonstrate that the approach can successfully deal with structural as well as intrinsic heterogeneities.

Simple Cortical and Thalamic Neuron Models for Digital Arithmetic Circuit Implementation.

Trade-off between reproducibility of neuronal activities and computational efficiency is one of crucial subjects in computational neuroscience and neuromorphic engineering. A wide variety of neuronal models have been studied from different viewpoints. The digital spiking silicon neuron (DSSN) model is a qualitative model that focuses on efficient implementation by digital arithmetic circuits. We expanded the DSSN model and found appropriate parameter sets with which it reproduces the dynamical behaviors of the ionic-conductance models of four classes of cortical and thalamic neurons. We first developed a four-variable model by reducing the number of variables in the ionic-conductance models and elucidated its mathematical structures using bifurcation analysis. Then, expanded DSSN models were constructed that reproduce these mathematical structures and capture the characteristic behavior of each neuron class. We confirmed that statistics of the neuronal spike sequences are similar in the DSSN and the ionic-conductance models. Computational cost of the DSSN model is larger than that of the recent sophisticated Integrate-and-Fire-based models, but smaller than the ionic-conductance models. This model is intended to provide another meeting point for above trade-off that satisfies the demand for large-scale neuronal network simulation with closer-to-biology models.

Research Challenges in Parallel and Distributed Simulation

The parallel and distributed simulation field has evolved and grown from its origins in the 1970s and 1980s and remains an active field of research to this day. A brief overview of research in the field is presented. Future research topics are explored including areas such as problem-driven simulation of large-scale systems and complex networks, exploitation of graphical processing unit hardware and cloud computing environments, predictive online simulation for system management and optimization, power and energy consumption in mobile platforms and data centers, and composition of heterogeneous simulations.

Parallel Brain Simulator: A Multi-scale and Parallel Brain-Inspired Neural Network Modeling and Simulation Platform

The brain is naturally a parallel and distributed system. Reverse engineering a cognitive brain is considered to be a grand challenge. In this paper, we present the parallel brain simulator (PBS), a parallel and distributed platform for modeling the cognitive brain at multiple scales. Inspired by large-scale graph computation, PBS can be considered as a universal parallel execution engine, which is aimed at reducing the complexity of distributed programming and providing an easy to use programmable platform for computational neuroscientists and artificial intelligence researchers for modeling and simulation of large-scale neural networks. As illustrative examples and validations, three brain-inspired neural networks which are built on PBS are introduced, including the 1:1 human hippocampus network, the 1:1 mouse whole-brain network and the CASIA brain simulator built for cognitive robotics. We deploy PBS on both commodity clusters and supercomputers, and a scalable performance is achieved. In addition, we provide evaluations on the scalability and performance of both lumped synapse-based simulation and non-lumped synapse-based simulation with different data-graph distribution methods to show the effectiveness and usability of the PBS platform.