Plasticity Rules Research Articles

Toward the biological model of the hippocampus as the successor representation agent

The hippocampus is an essential brain region for spatial memory and learning. Recently, a theoretical model of the hippocampus based on temporal difference (TD) learning has been published. Inspired by the successor representation (SR) learning algorithms, which decompose value function of TD learning into reward and state transition, they argued that the rate of firing of CA1 place cells in the hippocampus represents the probability of state transition. This theory, called predictive map theory, claims that the hippocampus representing space learns the probability of transition from the current state to the future state. The neural correlates of expecting the future state are the firing rates of the CA1 place cells. This explanation is plausible for the results recorded in behavioral experiments, but it is lacking the neurobiological implications.Modifying the SR learning algorithm added biological implications to the predictive map theory. Similar with the simultaneous needs of information of the current and future state in the SR learning algorithm, the CA1 place cells receive two inputs from CA3 and entorhinal cortex. Mathematical transformation showed that the SR learning algorithm is equivalent to the heterosynaptic plasticity rule. The heterosynaptic plasticity phenomena in CA1 were discussed and compared with the modified SR update rule. This study attempted to interpret the TD algorithm as the neurobiological mechanism occurring in place learning, and to integrate the neuroscience and artificial intelligence approaches in the field.

Hippocampal sharp wave-ripples and the associated sequence replay emerge from structured synaptic interactions in a network model of area CA3.

Hippocampal place cells are activated sequentially as an animal explores its environment. These activity sequences are internally recreated ('replayed'), either in the same or reversed order, during bursts of activity (sharp wave-ripples [SWRs]) that occur in sleep and awake rest. SWR-associated replay is thought to be critical for the creation and maintenance of long-term memory. In order to identify the cellular and network mechanisms of SWRs and replay, we constructed and simulated a data-driven model of area CA3 of the hippocampus. Our results show that the chain-like structure of recurrent excitatory interactions established during learning not only determines the content of replay, but is essential for the generation of the SWRs as well. We find that bidirectional replay requires the interplay of the experimentally confirmed, temporally symmetric plasticity rule, and cellular adaptation. Our model provides a unifying framework for diverse phenomena involving hippocampal plasticity, representations, and dynamics, and suggests that the structured neural codes induced by learning may have greater influence over cortical network states than previously appreciated.

New treatment for amblyopia based on rules of synaptic plasticity: a randomized clinical trial.

Amblyopia resulting from early deprivation of vision or defocus in one eye reflects an imbalance of input from the eyes to the visual cortex. We tested the hypothesis that asynchronous stimulation of the two eyes might induce synaptic plasticity and rebalance input. Experiments on normal adults showed that repetitive brief exposure of grating stimuli, with the onset of each stimulus delayed by 8.3 ms in one eye, results in a shift in perceptual eye dominance. Clinical studies (Clinical trial registration number: ChiCTR2100049130), using popular 3D movies with similar asynchrony between the two eyes (amblyopic eye stimulated first) to treat anisometropic amblyopia, established that just 10.5 h of conditioning over <3 weeks produced improvement that met criteria for successful treatment. The benefits of asynchronous conditioning accumulate over 20-30 45 min sessions, and are maintained for at least 2 years. Finally, we demonstrate that asynchronous binocular treatment alone is more effective than patching only. This novel treatment is popular with children and is some 50 times more efficient than patching alone.

Image and pattern reconstruction using differentiable plasticity

The brain is a substantial boon to humankind that adapts nature accordingly. The brain can learn and unlearn based on the situation. This singularity of human learning led to the research creating models using Artificial Intelligence (AI) to incorporate the brain’s behavior. The investigation opened up many new approaches to study AI with neural networks by adding new techniques to imitate the human brain’s functionalities. Many models can learn from experience like Recurrent Neural Network(RNN’s), Long Short Term Memory (LSTM) with the fixed network size. This paper describes the simple method of creating the model which will behave similar to the biological brain and recreates its differentiable plasticity to adopt the features of neural network connection. It also shows that applying plasticity and the Hebbian plastic connection rule can result in optimization in RNN. This new approach of reconstruction of images based on plastic neural network experiments shows that the above novel approach gives more optimized results than the traditionally used RNN techniques. In this paper, a proposal is made where models can memorize and reconstruct unseen sets of images by solving recurrent networks using plasticity rules.

Sensory coding and contrast invariance emerge from the control of plastic inhibition over emergent selectivity.

Visual stimuli are represented by a highly efficient code in the primary visual cortex, but the development of this code is still unclear. Two distinct factors control coding efficiency: Representational efficiency, which is determined by neuronal tuning diversity, and metabolic efficiency, which is influenced by neuronal gain. How these determinants of coding efficiency are shaped during development, supported by excitatory and inhibitory plasticity, is only partially understood. We investigate a fully plastic spiking network of the primary visual cortex, building on phenomenological plasticity rules. Our results suggest that inhibitory plasticity is key to the emergence of tuning diversity and accurate input encoding. We show that inhibitory feedback (random and specific) increases the metabolic efficiency by implementing a gain control mechanism. Interestingly, this led to the spontaneous emergence of contrast-invariant tuning curves. Our findings highlight that (1) interneuron plasticity is key to the development of tuning diversity and (2) that efficient sensory representations are an emergent property of the resulting network.

Toward Learning in Neuromorphic Circuits Based on Quantum Phase Slip Junctions.

We explore the use of superconducting quantum phase slip junctions (QPSJs), an electromagnetic dual to Josephson Junctions (JJs), in neuromorphic circuits. These small circuits could serve as the building blocks of neuromorphic circuits for machine learning applications because they exhibit desirable properties such as inherent ultra-low energy per operation, high speed, dense integration, negligible loss, and natural spiking responses. In addition, they have a relatively straight-forward micro/nano fabrication, which shows promise for implementation of an enormous number of lossless interconnections that are required to realize complex neuromorphic systems. We simulate QPSJ-only, as well as hybrid QPSJ + JJ circuits for application in neuromorphic circuits including artificial synapses and neurons, as well as fan-in and fan-out circuits. We also design and simulate learning circuits, where a simplified spike timing dependent plasticity rule is realized to provide potential learning mechanisms. We also take an alternative approach, which shows potential to overcome some of the expected challenges of QPSJ-based neuromorphic circuits, via QPSJ-based charge islands coupled together to generate non-linear charge dynamics that result in a large number of programmable weights or non-volatile memory states. Notably, we show that these weights are a function of the timing and frequency of the input spiking signals and can be programmed using a small number of DC voltage bias signals, therefore exhibiting spike-timing and rate dependent plasticity, which are mechanisms to realize learning in neuromorphic circuits.

Self-organization of a doubly asynchronous irregular network state for spikes and bursts.

Cortical pyramidal cells (PCs) have a specialized dendritic mechanism for the generation of bursts, suggesting that these events play a special role in cortical information processing. In vivo, bursts occur at a low, but consistent rate. Theory suggests that this network state increases the amount of information they convey. However, because burst activity relies on a threshold mechanism, it is rather sensitive to dendritic input levels. In spiking network models, network states in which bursts occur rarely are therefore typically not robust, but require fine-tuning. Here, we show that this issue can be solved by a homeostatic inhibitory plasticity rule in dendrite-targeting interneurons that is consistent with experimental data. The suggested learning rule can be combined with other forms of inhibitory plasticity to self-organize a network state in which both spikes and bursts occur asynchronously and irregularly at low rate. Finally, we show that this network state creates the network conditions for a recently suggested multiplexed code and thereby indeed increases the amount of information encoded in bursts.

Temporal Coding of Binary Patterns for Learning of Spiking Neuromorphic Systems Based on Nanocomposite Memristors

The metal/nanocomposite/metal (M/NC/M) memristive structures based on (Co40Fe40B20)x(LiNbO3)100–x have been studied. It has been shown that such memristors may change their conductance according to the bioinspired spike-timing-dependent plasticity (STDP) rules. Spiking neural network with 4 presynaptic inputs connected by memristor-synapses with a postsynaptic threshold neuron-integrator has been created, in which the images clustering with temporal coding has been implemented using the STDP rule. Thus, the fundamental possibility of using a temporal coding method, which is more effective than population-frequency coding, has been demonstrated for self-learning of spiking neuromorphic systems with synaptic weights based on nanocomposite memristors.

Evolving interpretable plasticity for spiking networks.

Continuous adaptation allows survival in an ever-changing world. Adjustments in the synaptic coupling strength between neurons are essential for this capability, setting us apart from simpler, hard-wired organisms. How these changes can be mathematically described at the phenomenological level, as so-called 'plasticity rules', is essential both for understanding biological information processing and for developing cognitively performant artificial systems. We suggest an automated approach for discovering biophysically plausible plasticity rules based on the definition of task families, associated performance measures and biophysical constraints. By evolving compact symbolic expressions, we ensure the discovered plasticity rules are amenable to intuitive understanding, fundamental for successful communication and human-guided generalization. We successfully apply our approach to typical learning scenarios and discover previously unknown mechanisms for learning efficiently from rewards, recover efficient gradient-descent methods for learning from target signals, and uncover various functionally equivalent STDP-like rules with tuned homeostatic mechanisms.

Plasticity of olfactory bulb inputs mediated by dendritic NMDA-spikes in rodent piriform cortex.

The piriform cortex (PCx) is essential for learning of odor information. The current view postulates that odor learning in the PCx is mainly due to plasticity in intracortical (IC) synapses, while odor information from the olfactory bulb carried via the lateral olfactory tract (LOT) is 'hardwired.' Here, we revisit this notion by studying location- and pathway-dependent plasticity rules. We find that in contrast to the prevailing view, synaptic and optogenetically activated LOT synapses undergo strong and robust long-term potentiation (LTP) mediated by only a few local NMDA-spikes delivered at theta frequency, while global spike timing-dependent plasticity (STDP) protocols failed to induce LTP in these distal synapses. In contrast, IC synapses in apical and basal dendrites undergo plasticity with both NMDA-spikes and STDP protocols but to a smaller extent compared with LOT synapses. These results are consistent with a self-potentiating mechanism of odor information via NMDA-spikes that can form branch-specific memory traces of odors that can further associate with contextual IC information via STDP mechanisms to provide cognitive and emotional value to odors.

A review of brain-like spiking neural network and its neuromorphic chip research

Under the current situation of the rapid development of brain-like artificial intelligence and the increasingly complex electromagnetic environment, the most bionic and anti-interference spiking neural network has shown great potential in computing speed, real-time information processing, and spatiotemporal data processing. Spiking neural network is the core part of brain-like artificial intelligence, which realizes brain-like computing by simulating the structure of biological neural network and the way of information transmission. This article first summarizes the advantages and disadvantages of the five models, and analyzes the characteristics of several network topologies. Then, it summarizes the spiking neural network algorithms. The unsupervised learning based on spike timing dependent plasticity (STDP) rules and four types of supervised learning algorithms are analyzed. Finally, the research on brain-like neuromorphic chips at home and abroad are reviewed. This paper aims to provide learning ideas and research directions for new colleagues in the field of spiking neural network.

Self-backpropagation of synaptic modifications elevates the efficiency of spiking and artificial neural networks.

Many synaptic plasticity rules found in natural circuits have not been incorporated into artificial neural networks (ANNs). We showed that incorporating a nonlocal feature of synaptic plasticity found in natural neural networks, whereby synaptic modification at output synapses of a neuron backpropagates to its input synapses made by upstream neurons, markedly reduced the computational cost without affecting the accuracy of spiking neural networks (SNNs) and ANNs in supervised learning for three benchmark tasks. For SNNs, synaptic modification at output neurons generated by spike timing–dependent plasticity was allowed to self-propagate to limited upstream synapses. For ANNs, modified synaptic weights via conventional backpropagation algorithm at output neurons self-backpropagated to limited upstream synapses. Such self-propagating plasticity may produce coordinated synaptic modifications across neuronal layers that reduce computational cost.

A CMOS-based neuromorphic device for seizure detection from LFP signals

Recent research provides examples of neuromorphic systems applied to process biological signals or to interface biological tissues. Usually, in such contexts, the neuromorphic system is used for automatic anomaly detection. The automation of the long-term monitoring of biological signals holds promise for lightening the burden placed on clinicians. At the same time, the adoption of such devices potentially allows processing to be performed locally, without the need to transfer data to an external processor. In turn, on-site signal analysis makes closed-loop intervention feasible, to correct the source of the anomalies. So far, the common approach has been to deploy the network of spiking neurons either on multi-core neuromorphic platforms or on programmable units (field programmable gate array). However, if the aim is to develop wearable or even chronically implantable devices, it is imperative to move in the direction of embedded solutions, tailored and optimized for the specific application. To this end, the present study proposes a neuromorphic device implemented in CMOS technology for the detection of epileptic seizures (ictal events) from local field potential (LFP) signals. The LFP data have been acquired by a multi-electrode array in a slice of mouse hippocampus-cortex. The system includes an analog-to-event converter (AEC) encoding the recorded signals into trains of spikes, and a small spiking neural network (SNN) of 2 × 1 neurons with online biological-plausible learning. The AEC yields two spike-trains: UP spikes that account for the positive slope signal and DOWN spikes that correspond to the negative slope signal. The synapses among the input and output layers are plastic and follow the spike-timing-dependent plasticity rule. Early results show that the SNN module is able to detect ictal events with a delay of 64.98 ± 30.92 ms, consuming 50 pW. The layout of the entire system occupies 528 µm × 278 µm.

From synapse to network: models of information storage and retrieval in neural circuits.

The mechanisms of information storage and retrieval in brain circuits are still the subject of debate. It is widely believed that information is stored at least in part through changes in synaptic connectivity in networks that encode this informationand that these changes lead in turn to modifications of network dynamics, such that the stored information can be retrieved at a later time. Here, we review recent progress in deriving synaptic plasticity rules from experimental dataand in understanding how plasticity rules affect the dynamics of recurrent networks. We show that the dynamics generated by such networks exhibit a large degree of diversity, depending on parameters, similar to experimental observations invivo during delayed response tasks.

SpikePropamine: Differentiable Plasticity in Spiking Neural Networks.

The adaptive changes in synaptic efficacy that occur between spiking neurons have been demonstrated to play a critical role in learning for biological neural networks. Despite this source of inspiration, many learning focused applications using Spiking Neural Networks (SNNs) retain static synaptic connections, preventing additional learning after the initial training period. Here, we introduce a framework for simultaneously learning the underlying fixed-weights and the rules governing the dynamics of synaptic plasticity and neuromodulated synaptic plasticity in SNNs through gradient descent. We further demonstrate the capabilities of this framework on a series of challenging benchmarks, learning the parameters of several plasticity rules including BCM, Oja's, and their respective set of neuromodulatory variants. The experimental results display that SNNs augmented with differentiable plasticity are sufficient for solving a set of challenging temporal learning tasks that a traditional SNN fails to solve, even in the presence of significant noise. These networks are also shown to be capable of producing locomotion on a high-dimensional robotic learning task, where near-minimal degradation in performance is observed in the presence of novel conditions not seen during the initial training period.

Calcium and Spike Timing-Dependent Plasticity.

Since its discovery, spike timing-dependent synaptic plasticity (STDP) has been thought to be a primary mechanism underlying the brain’s ability to learn and to form new memories. However, despite the enormous interest in both the experimental and theoretical neuroscience communities in activity-dependent plasticity, it is still unclear whether plasticity rules inferred from in vitro experiments apply to in vivo conditions. Among the multiple reasons why plasticity rules in vivo might differ significantly from in vitro studies is that extracellular calcium concentration use in most studies is higher than concentrations estimated in vivo. STDP, like many forms of long-term synaptic plasticity, strongly depends on intracellular calcium influx for its induction. Here, we discuss the importance of considering physiological levels of extracellular calcium concentration to study functional plasticity.

Three Musketeers: demonstration of multilevel memory, selector, and synaptic behaviors from an Ag-GeTe based chalcogenide material

Functional neuronal computing systems that support information diversification require high-density memory with selector devices to reduce leakage current in cross-point architectures, which drives us to develop a functional switching layer that operates as three distinct devices, namely non-volatile memory, selector, and synaptic devices, using a GeTe-based single material system. In this study, amorphous Ag-GeTe switching layers are engineered by doping with Te species to achieve either resistive switching (RS) or threshold switching properties. The Ag/Ag-GeTe/Ag memory device exhibits multilevel characteristics via a tunable compliance current approach. By comparison, Ag/Ag-GeTex/Ag selector device provides excellent selectivity (>106) with a very low OFF-current (∼10−11 A). The RS mechanism for memory and selector devices is interrogated by using conductive atomic force microscopy. Moreover, the Ag/Ag-GeTe/Ag RS device mimics a cohort of basic and complex synaptic plasticity properties, including potentiation-depression and four-spike time-dependent plasticity rules that include asymmetric Hebbian, asymmetric anti-Hebbian, symmetric Hebbian, and symmetric anti-Hebbian learning rules. The capability of the synaptic devices to detect image edges is demonstrated by using a convolution neural network. The present work showcases the multi-functionality of Ag-GeTe materials, which will likely emerge as a prominent candidate for high-density cross-point architecture-based neuromorphic computing systems.

Constitutive model with memory-surface shrinking rule for cyclic plasticity of structural steel

A constitutive model is proposed for both the cyclic hardening and softening of structural steel. A shrinking rule is incorporated into the memory surface for cyclic softening. The theoretical formulation of the proposed model is derived under cyclic loading with a gradually decreasing strain amplitude. The material constants are identified by numerically minimising the difference between the theoretical result and a single material test result using a genetic algorithm. The proposed model is validated by comparing numerical and material test results under various cyclic loadings for structural steel, where the model exhibits good ability to qualitatively and quantitatively evaluate the stress amplitude variation. The model is implemented in Abaqus FEA software, and the convergence rate of the consistent tangent operator is evaluated. FEA with the consistent tangent operator yields a quadratic convergence.

Evolving Plasticity for Autonomous Learning under Changing Environmental Conditions.

A fundamental aspect of learning in biological neural networks is the plasticity property which allows them to modify their configurations during their lifetime. Hebbian learning is a biologically plausible mechanism for modeling the plasticity property in artificial neural networks (ANNs), based on the local interactions of neurons. However, the emergence of a coherent global learning behavior from local Hebbian plasticity rules is not very well understood. The goal of this work is to discover interpretable local Hebbian learning rules that can provide autonomous global learning. To achieve this, we use a discrete representation to encode the learning rules in a finite search space. These rules are then used to perform synaptic changes, based on the local interactions of the neurons. We employ genetic algorithms to optimize these rules to allow learning on two separate tasks (a foraging and a prey-predator scenario) in online lifetime learning settings. The resulting evolved rules converged into a set of well-defined interpretable types, that are thoroughly discussed. Notably, the performance of these rules, while adapting the ANNs during the learning tasks, is comparable to that of offline learning methods such as hill climbing.

Robust rhythmogenesis via spike-timing-dependent plasticity.

Rhythmic activity has been observed in numerous animal species ranging from insects to humans, and in relation to a wide range of cognitive tasks. Various experimental and theoretical studies have investigated rhythmic activity. The theoretical efforts have mainly been focused on the neuronal dynamics, under the assumption that network connectivity satisfies certain fine-tuning conditions required to generate oscillations. However, it remains unclear how this fine-tuning is achieved. Here we investigated the hypothesis that spike-timing-dependent plasticity (STDP) can provide the underlying mechanism for tuning synaptic connectivity to generate rhythmic activity. We addressed this question in a modeling study. We examined STDP dynamics in the framework of a network of excitatory and inhibitory neuronal populations that has been suggested to underlie the generation of oscillations in the gamma range. Mean-field Fokker-Planck equations for the synaptic weight dynamics are derived in the limit of slow learning. We drew on this approximation to determine which types of STDP rules drive the system to exhibit rhythmic activity, and we demonstrate how the parameters that characterize the plasticity rule govern the rhythmic activity. Finally, we propose a mechanism that can ensure the robustness of self-developing processes in general, and for rhythmogenesis in particular.