Continuous Attractor Neural Networks Research Articles

Firing rate adaptation affords place cell theta sweeps, phase precession, and procession

Hippocampal place cells in freely moving rodents display both theta phase precession and procession, which is thought to play important roles in cognition, but the neural mechanism for producing theta phase shift remains largely unknown. Here, we show that firing rate adaptation within a continuous attractor neural network causes the neural activity bump to oscillate around the external input, resembling theta sweeps of decoded position during locomotion. These forward and backward sweeps naturally account for theta phase precession and procession of individual neurons, respectively. By tuning the adaptation strength, our model explains the difference between ‘bimodal cells’ showing interleaved phase precession and procession, and ‘unimodal cells’ in which phase precession predominates. Our model also explains the constant cycling of theta sweeps along different arms in a T-maze environment, the speed modulation of place cells’ firing frequency, and the continued phase shift after transient silencing of the hippocampus. We hope that this study will aid an understanding of the neural mechanism supporting theta phase coding in the brain.

Firing rate adaptation affords place cell theta sweeps, phase precession, and procession.

Hippocampal place cells in freely moving rodents display both theta phase precession and procession, which is thought to play important roles in cognition, but the neural mechanism for producing theta phase shift remains largely unknown. Here, we show that firing rate adaptation within a continuous attractor neural network causes the neural activity bump to oscillate around the external input, resembling theta sweeps of decoded position during locomotion. These forward and backward sweeps naturally account for theta phase precession and procession of individual neurons, respectively. By tuning the adaptation strength, our model explains the difference between 'bimodal cells' showing interleaved phase precession and procession, and 'unimodal cells' in which phase precession predominates. Our model also explains the constant cycling of theta sweeps along different arms in a T-maze environment, the speed modulation of place cells' firing frequency, and the continued phase shift after transient silencing of the hippocampus. We hope that this study will aid an understanding of the neural mechanism supporting theta phase coding in the brain.

The synaptic correlates of serial position effects in sequential working memory.

Sequential working memory (SWM), referring to the temporary storage and manipulation of information in order, plays a fundamental role in brain cognitive functions. The serial position effect refers to the phenomena that recall accuracy of an item is associated to the order of the item being presented. The neural mechanism underpinning the serial position effect remains unclear. The synaptic mechanism of working memory proposes that information is stored as hidden states in the form of facilitated neuronal synapse connections. Here, we build a continuous attractor neural network with synaptic short-term plasticity (STP) to explore the neural mechanism of the serial position effect. Using a delay recall task, our model reproduces the the experimental finding that as the maintenance period extends, the serial position effect transitions from the primacy to the recency effect. Using both numerical simulation and theoretical analysis, we show that the transition moment is determined by the parameters of STP and the interval between presented stimulus items. Our results highlight the pivotal role of STP in processing the order information in SWM.

Information content in continuous attractor neural networks is preserved in the presence of moderate disordered background connectivity.

Continuous attractor neural networks (CANN) form an appealing conceptual model for the storage of information in the brain. However a drawback of CANN is that they require finely tuned interactions. We here study the effect of quenched noise in the interactions on the coding of positional information within CANN. Using the replica method we compute the Fisher information for a network with position-dependent input and recurrent connections composed of a short-range (in space) and a disordered component. We find that the loss in positional information is small for not too large disorder strength, indicating that CANN have a regime in which the advantageous effects of local connectivity on information storage outweigh the detrimental ones. Furthermore, a substantial part of this information can be extracted with a simple linear readout.

Short-term postsynaptic plasticity facilitates predictive tracking in continuous attractors.

The N-methyl-D-aspartate receptor (NMDAR) plays a critical role in synaptic transmission and is associated with various neurological and psychiatric disorders. Recently, a novel form of postsynaptic plasticity known as NMDAR-based short-term postsynaptic plasticity (STPP) has been identified. It has been suggested that long-lasting glutamate binding to NMDAR allows for the retention of input information in brain slices up to 500 ms, leading to response facilitation. However, the impact of STPP on the dynamics of neuronal populations remains unexplored. In this study, we incorporated STPP into a continuous attractor neural network (CANN) model to investigate its effects on neural information encoding in populations of neurons. Unlike short-term facilitation, a form of presynaptic plasticity, the temporally enhanced synaptic efficacy resulting from STPP destabilizes the network state of the CANN by increasing its mobility. Our findings demonstrate that the inclusion of STPP in the CANN model enables the network state to predictively respond to a moving stimulus. This nontrivial dynamical effect facilitates the tracking of the anticipated stimulus, as the enhanced synaptic efficacy induced by STPP enhances the system's mobility. The discovered STPP-based mechanism for sensory prediction provides valuable insights into the potential development of brain-inspired computational algorithms for prediction. By elucidating the role of STPP in neural population dynamics, this study expands our understanding of the functional implications of NMDAR-related plasticity in information processing within the brain. The incorporation of STPP into a CANN model highlights its influence on the mobility and predictive capabilities of neural networks. These findings contribute to our knowledge of STPP-based mechanisms and their potential applications in developing computational algorithms for sensory prediction.

Bionic visual navigation model for enhanced template matching and loop closing in challenging lighting environments

In recent years, various computational models have been proposed in neuroscience, but only a few have been utilized in robot visual navigation through engineering approaches. Besides, these engineering methods lack environmental adaptability, especially in the aspect of visual place recognition in weak or uncontrollable light fluctuations. To address this issue and enhance the performance of visual template matching and map loop closure detection in challenging lighting environments, this paper proposed a bionic visual navigation model that combines two neural network architectures, the Pulse Coupled Neural Network (PCNN) and Continuous Attractor Neural Network (CANN). In our navigation model, the visual features of the environments are encoded as temporal information by the spiking model and input into the local view cells for visual template matching. Additionally, we utilize the pose cells network, which incorporates the similarity between current and previous templates, along with odometry data, to encode spatial information and store it as an experience map. To validate the effectiveness of the proposed model, we conducted evaluations on datasets collected within our library, campus, and an open-source office dataset. The experimental results reveal that our algorithm increases the F1-score of template matching by approximately 10.5%\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$ \\% $$\\end{document}, 35.8%\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$ \\% $$\\end{document}, 61.7%\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$ \\% $$\\end{document}, and 1.9%\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$ \\% $$\\end{document} in each dataset, compared to the conventional RatSLAM method. Furthermore, our algorithm generates a more accurate map that closely correlates with the real-world trajectory without compromising on computation time. The results suggest that our bionic visual navigation model is reliable for both standard and extreme lighting conditions.

Bimodular continuous attractor neural networks with static and moving stimuli.

We investigated the dynamical behaviors of bimodular continuous attractor neural networks, each processing a modality of sensory input and interacting with each other. We found that when bumps coexist in both modules, the position of each bump is shifted towards the other input when the intermodular couplings are excitatory and is shifted away when inhibitory. When one intermodular coupling is excitatory while another is moderately inhibitory, temporally modulated population spikes can be generated. On further increase of the inhibitory coupling, momentary spikes will emerge. In the regime of bump coexistence, bump heights are primarily strengthened by excitatory intermodular couplings, but there is a lesser weakening effect due to a bump being displaced from the direct input. When bimodular networks serve as decoders of multisensory integration, we extend the Bayesian framework to show that excitatory and inhibitory couplings encode attractive and repulsive priors, respectively. At low disparity, the bump positions decode the posterior means in the Bayesian framework, whereas at high disparity, multiple steady states exist. In the regime of multiple steady states, the less stable state can be accessed if the input causing the more stable state arrives after a sufficiently long delay. When one input is moving, the bump in the corresponding module is pinned when the moving stimulus is weak, unpinned at intermediate stimulus strength, and tracks the input at strong stimulus strength, and the stimulus strengths for these transitions increase with the velocity of the moving stimulus. These results are important to understanding multisensory integration of static and dynamic stimuli.

Unveiling Dynamic System Strategies for Multisensory Processing: From Neuronal Fixed-Criterion Integration to Population Bayesian Inference.

Multisensory processing is of vital importance for survival in the external world. Brain circuits can both integrate and separate visual and vestibular senses to infer self-motion and the motion of other objects. However, it is largely debated how multisensory brain regions process such multisensory information and whether they follow the Bayesian strategy in this process. Here, we combined macaque physiological recordings in the dorsal medial superior temporal area (MST-d) with modeling of synaptically coupled multilayer continuous attractor neural networks (CANNs) to study the underlying neuronal circuit mechanisms. In contrast to previous theoretical studies that focused on unisensory direction preference, our analysis showed that synaptic coupling induced cooperation and competition in the multisensory circuit and caused single MST-d neurons to switch between sensory integration or separation modes based on the fixed-criterion causal strategy, which is determined by the synaptic coupling strength. Furthermore, the prior of sensory reliability was represented by pooling diversified criteria at the MST-d population level, and the Bayesian strategy was achieved in downstream neurons whose causal inference flexibly changed with the prior. The CANN model also showed that synaptic input balance is the dynamic origin of neuronal direction preference formation and further explained the misalignment between direction preference and inference observed in previous studies. This work provides a computational framework for a new brain-inspired algorithm underlying multisensory computation.

A Positioning Method Based on Place Cells and Head-Direction Cells for Inertial/Visual Brain-Inspired Navigation System.

Mammals rely on vision and self-motion information in nature to distinguish directions and navigate accurately and stably. Inspired by the mammalian brain neurons to represent the spatial environment, the brain-inspired positioning method based on multi-sensors’ input is proposed to solve the problem of accurate navigation in the absence of satellite signals. In the research related to the application of brain-inspired engineering, it is not common to fuse various sensor information to improve positioning accuracy and decode navigation parameters from the encoded information of the brain-inspired model. Therefore, this paper establishes the head-direction cell model and the place cell model with application potential based on continuous attractor neural networks (CANNs) to encode visual and inertial input information, and then decodes the direction and position according to the population neuron firing response. The experimental results confirm that the brain-inspired navigation model integrates a variety of information, outputs more accurate and stable navigation parameters, and generates motion paths. The proposed model promotes the effective development of brain-inspired navigation research.

Neurodegenerative damage reduces firing coherence in a continuous attractor model of grid cells.

Grid cells in the dorsolateral band of the medial entorhinal cortex (dMEC) display strikingly regular periodic firing patterns on a lattice of positions in two-dimensional (2D) space. This helps animals to encode relative spatial location without reference to external cues. The dMEC is damaged in the early stages of Alzheimer's disease, which affects navigation ability of a disease victim, reducing the synaptic density of neurons in the network. Within an established two-dimensional continuous attractor neural network model of grid cell activity, we introduce neural sheet damage parametrized by radius and by the strength of the synaptic output for neurons in the damaged region. The mean proportionality of the grid field flow rate in the dMEC to the velocity of the model animal is maintained, but there is a broadened distribution of flow rates in the damaged case. This flow-rate-to-velocity proportionality is essential to establish coherent grid firing fields for individual grid cells for a roaming animal. When we examine the coherence of the grid cell firing field by studying Bragg peaks of the Fourier transformed lattice firing field intensity in both damaged and undamaged regions, we find that, for a wide range of damage radius and reduced synaptic strength, for undamaged model grid cells there is an incoherent firing field structure with only a single central peak. In the radius-damage plane this is adjacent to narrow bands of striped lattices (two additional Bragg peaks), which abut an orthorhombic pattern (four additional Bragg peaks), that abuts the undamaged hexagonal region (six additional Bragg peaks). Within the damaged region, grid cells show no Bragg peaks outside the central one, which shows reduced intensity with increasing damage, and outside the damaged region the central Bragg peak strength is largely unaffected. There is a reentrant region of normal grid firing fields for very large damage area. We anticipate that the modified grid cell behavior can be observed in noninvasive functional magnetic resonance imaging (fMRI) of the dMEC.

A brain-inspired localization system for the UAV based on navigation cells

Purpose To solve problems of low intelligence and poor robustness of traditional navigation systems, the purpose of this paper is to propose a brain-inspired localization method of the unmanned aerial vehicle (UAV). Design/methodology/approach First, the yaw angle of the UAV is obtained by modeling head direction cells with one-dimension continuous attractor neural network (1 D-CANN) and then inputs into 3D grid cells. After that, the motion information of the UAV is encoded as the firing of 3 D grid cells using 3 D-CANN. Finally, the current position of the UAV can be decoded from the neuron firing through the period-adic method. Findings Simulation results suggest that continuous yaw and position information can be generated from the conjunctive model of head direction cells and grid cells. Originality/value The proposed period-adic cell decoding method can provide a UAV with the 3 D position, which is more intelligent and robust than traditional navigation methods.

Temporally coherent perturbation of neural dynamics during retention alters human multi-item working memory.

Temporarily storing a list of items in working memory (WM), a fundamental ability in cognition, has been posited to rely on the temporal dynamics of multi-item neural representations during retention. However, the causal evidence, particularly in human subjects, is still lacking, let alone WM manipulation. Here, we develop a novel "dynamic perturbation" approach to manipulate the relative memory strength of WM items held in human brain, by presenting temporally correlated luminance sequences during retention to interfere with the multi-item neural dynamics. Six experiments on more than 150 subjects confirm the effectiveness of this WM manipulation approach. A computational model combining continuous attractor neural network (CANN) and short-term synaptic plasticity (STP) principles further reproduces all the empirical findings. The model reveals that the "dynamic perturbation" modifies the synaptic efficacies of WM items through STP principles, eventually leading to changes in their relative memory strengths. Our results support the causal role of temporal dynamics of neural network in mediating multi-item WM, and offer a promising, purely bottom-up approach to manipulate WM.

Neuromorphic Implementation of a Continuous Attractor Neural Network With Various Synaptic Dynamics

A continuous attractor neural network (CANN) configured with various synaptic dynamics can implement many brain cognitive functions. Corresponding neuromorphic implementation can be beneficial to carry out these cognitive computations in real-time. Herein, we present a neuromorphic implementation of the CANN of spiking neurons on the field-programmable gate array (FPGA), in which the synaptic strength of local excitation and global inhibition can be configured to achieve winner-take-all (WTA) competition. In contrast to previously reported neuromorphic CANN, in addition to fast-linear synapses, recurrent connections can also be implemented with slow-nonlinear synapses. This endows the proposed CANN on the FPGA with the capability of performing neural computations that require slow and nonlinear temporal dynamics, such as decision making and working memory. Circuit simulations on the hardware reproduced gradually ramping neural activities observed in decision-making experiments and multiple sustained activities observed during the delay in working memory experiments, especially the fade-out and merging of activity bumps in the latter task. Theoretical analysis illuminated the underlying mechanisms of decision making and working memory on the FPGA. The unstable two-bump steady state of the CANN is responsible for the two-alternative decision making, similar to the saddle-node structure in the reduced two-variable decision model. Meanwhile, the stable uniformly distributed N-bump steady states account for the multiple-item working memory, similar to the multistability in previous mean-field models. Furthermore, the asymmetrical N-bump unstable steady states exhibit attractor dynamics underlying the fade-out and merging of activity bumps. The consistency between simulations on the hardware and the theoretical analysis demonstrated that the neuromorphic CANN is endowed with attractor dynamics relating to slow and nonlinear neural computations and has the potential to implement sophisticated cognitive functions by configuring various synaptic dynamics. This implementation shows promise for integrating the cognitive platform due to its characteristics of various dynamics and flexible configurations.

A Bio-Inspired 3-D Neural Compass Based on Head Direction Cells

Head direction cells (HDCs) in mammals’ brains are thought to provide the sense of direction (function of a compass). They fire when the animal’s head points to a specific orientation. In this paper, a bio-inspired neural compass with 3 degrees of freedom (DOFs) which is consistent with neurophysiological characteristics of biological HDCs in three-dimensional (3-D) environments is proposed. The proposed model, which is based on continuous attractor neural network (CANN), is primarily driven by angular velocity of head turning to determine spatial orientations of unmanned vehicles in 3-D environments. By means of Hebbian learning, external perception information such as visual landmarks can be incorporated to calibrate cumulative errors of path integration based on self-motion information. Experimental results obtained by validating the model on a two-axis motion simulator are given. The neural system enables the simulator to track its spatial orientations in 3-D environments effectively.

Anticipative Tracking with the Short-Term Synaptic Plasticity of Spintronic Devices

Real-time tracking of high-speed objects in cognitive tasks is challenging in the present artificial intelligence techniques because the data processing and computation are time-consuming resulting in impeditive time delays. A brain-inspired continuous attractor neural network (CANN) can be used to track quickly moving targets, where the time delays are intrinsically compensated if the dynamical synapses in the network have the short-term plasticity. Here, we show that synapses with short-term depression can be realized by a magnetic tunnel junction, which perfectly reproduces the dynamics of the synaptic weight in a widely applied mathematical model. Then, these dynamical synapses are incorporated into one-dimensional and two-dimensional CANNs, which are demonstrated to have the ability to predict a moving object via micromagnetic simulations. This portable spintronics-based hardware for neuromorphic computing needs no training and is therefore very promising for the tracking technology for moving targets.

Precise spatial memory in local random networks.

Self-sustained, elevated neuronal activity persisting on timescales of 10s or longer is thought to be vital for aspects of working memory, including brain representations of real space. Continuous-attractor neural networks, one of the most well-known modeling frameworks for persistent activity, have been able to model crucial aspects of such spatial memory. These models tend to require highly structured or regular synaptic architectures. In contrast, we study numerical simulations of a geometrically embedded model with a local, but otherwise random, connectivity profile; imposing a global regulation of our system's mean firing rate produces localized, finely spaced discrete attractors that effectively span a two-dimensional manifold. We demonstrate how the set of attracting states can reliably encode a representation of the spatial locations at which the system receives external input, thereby accomplishing spatial memory via attractor dynamics without synaptic fine-tuning or regular structure. We then measure the network's storage capacity numerically and find that the statistics of retrievable positions are also equivalent to a full tiling of the plane, something hitherto achievable only with (approximately) translationally invariant synapses, and which may be of interest in modeling such biological phenomena as visuospatial working memory in two dimensions.

A Hybrid Compact Neural Architecture for Visual Place Recognition

State-of-the-art algorithms for visual place recognition, and related visual navigation systems, can be broadly split into two categories: computer-science-oriented models including deep learning or image retrieval-based techniques with minimal biological plausibility, and neuroscience-oriented dynamical networks that model temporal properties underlying spatial navigation in the brain. In this letter, we propose a new compact and high-performing place recognition model that bridges this divide for the first time. Our approach comprises two key neural models of these categories: (1) FlyNet, a compact, sparse two-layer neural network inspired by brain architectures of fruit flies, Drosophila melanogaster, and (2) a one-dimensional continuous attractor neural network (CANN). The resulting FlyNet+CANN network incorporates the compact pattern recognition capabilities of our FlyNet model with the powerful temporal filtering capabilities of an equally compact CANN, replicating entirely in a hybrid neural implementation the functionality that yields high performance in algorithmic localization approaches like SeqSLAM. We evaluate our model, and compare it to three state-of-the-art methods, on two benchmark real-world datasets with small viewpoint variations and extreme environmental changes - achieving 87% AUC results under day to night transitions compared to 60% for Multi-Process Fusion, 46% for LoST-X and 1% for SeqSLAM, while being 6.5, 310, and 1.5 times faster, respectively.

Memristor‐Based Biologically Plausible Memory Based on Discrete and Continuous Attractor Networks for Neuromorphic Systems

To approach an advanced neuromorphic system, a significant unsettled problem is how to realize biologically plausible memory structures that are dramatically different from classical computers. Herein, a physical system based on memristors is simulated to realize associative memory based on discrete attractor networks, which is essentially content‐based storage, and the influence of device characteristics on network performance is systematically studied. An in situ unsupervised learning method is applied to make greater use of array structure and competitions between neurons, demonstrating significant performance improvement in memory capacity and noise tolerance compared with existing supervised approaches. By extending to continuous attractor neural networks (CANNs), working memory is realized based on memristors for the first time via simulation, and the write and read noises in memristor arrays are found to have different impacts on the ability of CANN in maintaining dynamic information. This work lays a foundation for the construction of future advanced neuromorphic computing systems.

Can Grid Cell Ensembles Represent Multiple Spaces?

The way grid cells represent space in the rodent brain has been a striking discovery, with theoretical implications still unclear. Unlike hippocampal place cells, which are known to encode multiple, environment-dependent spatial maps, grid cells have been widely believed to encode space through a single low-dimensional manifold, in which coactivity relations between different neurons are preserved when the environment is changed. Does it have to be so? Here, we compute, using two alternative mathematical models, the storage capacity of a population of grid-like units, embedded in a continuous attractor neural network, for multiple spatial maps. We show that distinct representations of multiple environments can coexist, as existing models for grid cells have the potential to express several sets of hexagonal grid patterns, challenging the view of a universal grid map. This suggests that a population of grid cells can encode multiple noncongruent metric relationships, a feature that could in principle allow a grid-like code to represent environments with a variety of different geometries and possibly conceptual and cognitive spaces, which may be expected to entail such context-dependent metric relationships.

Stochastic neural fields as gradient dynamical systems.

Continuous attractor neural networks are used extensively to model a variety of experimentally observed coherent brain states, ranging from cortical waves of activity to stationary activity bumps. The latter are thought to play an important role in various forms of neural information processing, including population coding in primary visual cortex (V1) and working memory in prefrontal cortex. However, one limitation of continuous attractor networks is that the location of the peak of an activity bump (or wave) can diffuse due to intrinsic network noise. This reflects marginal stability of bump solutions with respect to the action of an underlying continuous symmetry group. Previous studies have used perturbation theory to derive an approximate stochastic differential equation for the location of the peak (phase) of the bump. Although this method captures the diffusive wandering of a bump solution, it ignores fluctuations in the amplitude of the bump. In this paper, we show how amplitude fluctuations can be analyzed by reducing the underlying stochastic neural field equation to a finite-dimensional stochastic gradient dynamical system that tracks the stochastic motion of both the amplitude and phase of bump solutions. This allows us to derive exact expressions for the steady-state probability density and its moments, which are then used to investigate two major issues: (i) the input-dependent suppression of neural variability and (ii) noise-induced transitions to bump extinction. We develop the theory by considering the particular example of a ring attractor network with SO(2) symmetry, which is the most common architecture used in attractor models of working memory and population tuning in V1. However, we also extend the analysis to a higher-dimensional spherical attractor network with SO(3) symmetry which has previously been proposed as a model of orientation and spatial frequency tuning in V1. We thus establish how a combination of stochastic analysis and group theoretic methods provides a powerful tool for investigating the effects of noise in continuous attractor networks.