Distinct dopaminergic spike-timing-dependent plasticity rules are suited to different functional roles

Various mathematical models have been formulated to describe the changes in synaptic strengths resulting from spike-timing-dependent plasticity (STDP). A subset of these models include a third factor, dopamine, which interacts with spike timing to contribute to plasticity at specific synapses, notably those from cortex to striatum at the input layer of the basal ganglia. Theoretical work to analyze these plasticity models has largely focused on abstract issues, such as the conditions under which they may promote synchronization and the weight distributions induced by inputs with simple correlation structures, rather than on scenarios associated with specific tasks, and has generally not considered dopamine-dependent forms of STDP. In this paper we introduce forms of dopamine-modulated STDP adapted from previously proposed plasticity rules. We then analyze, mathematically and with simulations, their performance in two biologically relevant scenarios. We test the ability of each of the three models to complete simple value estimation and action selection tasks, studying the learned weight distributions and corresponding task performance in each setting. Interestingly, we find that each plasticity rule is well suited to a subset of the scenarios studied but falls short in others. Different tasks may therefore require different forms of synaptic plasticity, yielding the prediction that the precise form of the STDP mechanism present may vary across regions of the striatum, and other brain areas impacted by dopamine, that are involved in distinct computational functions.

Synaptic plasticity is often explored as a form of unsupervised adaptation in cortical microcircuits to learn the structure of complex sensory inputs and thereby improve performance of classification and prediction. The question of whether the specific structure of the input patterns is encoded in the structure of neural networks has been largely neglected. Existing studies that have analyzed input-specific structural adaptation have used simplified, synthetic inputs in contrast to complex and noisy patterns found in real-world sensory data. In this work, input-specific structural changes are analyzed for three empirically derived models of plasticity applied to three temporal sensory classification tasks that include complex, real-world visual and auditory data. Two forms of spike-timing dependent plasticity (STDP) and the Bienenstock-Cooper-Munro (BCM) plasticity rule are used to adapt the recurrent network structure during the training process before performance is tested on the pattern recognition tasks. It is shown that synaptic adaptation is highly sensitive to specific classes of input pattern. However, plasticity does not improve the performance on sensory pattern recognition tasks, partly due to synaptic interference between consecutively presented input samples. The changes in synaptic strength produced by one stimulus are reversed by the presentation of another, thus largely preventing input-specific synaptic changes from being retained in the structure of the network. To solve the problem of interference, we suggest that models of plasticity be extended to restrict neural activity and synaptic modification to a subset of the neural circuit, which is increasingly found to be the case in experimental neuroscience.

Spike timing-dependent plasticity (STDP) experiments have shown that a synapse is strengthened when a presynaptic spike precedes a postsynaptic one and depressed vice versa. The canonical form of STDP has been shown to have an asymmetric shape with the peak long-term potentiation at +6 ms and the peak long-term depression at -5 ms. Experiments in hippocampal cultures with more complex stimuli such as triplets (one presynaptic spike combined with two postsynaptic spikes or one postsynaptic spike with two presynaptic spikes) have shown that pre-post-pre spike triplets result in no change in synaptic strength, whereas post-pre-post spike triplets lead to significant potentiation. The sign and magnitude of STDP have also been experimentally hypothesized to be modulated by inhibition. Recently, a computational study showed that the asymmetrical form of STDP in the CA1 pyramidal cell dendrite when two spikes interact switches to a symmetrical one in the presence of inhibition under certain conditions. In the present study, I investigate computationally how inhibition modulates STDP in the CA1 pyramidal neuron dendrite when it is driven by triplets. The model uses calcium as the postsynaptic signaling agent for STDP and is shown to be consistent with the experimental triplet observations in the absence of inhibition: simulated pre-post-pre spike triplets result in no change in synaptic strength, whereas simulated post-pre-post spike triplets lead to significant potentiation. When inhibition is bounded by the onset and offset of the triplet stimulation, then the strength of the synapse is decreased as the strength of inhibition increases. When inhibition arrives either few milliseconds before or at the onset of the last spike in the pre-post-pre triplet stimulation, then the synapse is potentiated. Variability in the frequency of inhibition (50 vs. 100 Hz) produces no change in synaptic strength. Finally, a 5% variation in model's calcium parameters (calcium thresholds) proves that the model's performance is robust.

Chapter 9 - Spike Timing-Dependent Plasticity

Binge alcohol drinking, a behavior characterized by rapid repeated alcohol intake, is most prevalent in young adults and is a risk factor for excessive alcohol consumption and alcohol dependence. Although the alteration of synaptic plasticity is thought to contribute to this behavior, there is currently little evidence that this is the case. We used drinking in the dark (DID) as a model of binge alcohol drinking to assess its effects on spike timing-dependent plasticity (STDP) in medium spiny neurons (MSNs) of the core nucleus accumbens (NAc) by combining patch-clamp recordings with calcium imaging and optogenetics. After 2 weeks of daily alcohol binges, synaptic plasticity was profoundly altered. STDP in MSNs expressing dopamine D1 receptors shifted from spike-timing-dependent long-term depression (tLTD), the predominant form of plasticity in naive male mice, to spike-timing-dependent long-term potentiation (tLTP) in DID mice, an effect that was totally reversed in the presence of 4 μm SCH23390, a dopamine D1 receptor antagonist. In MSNs presumably expressing dopamine D2 receptors, tLTP, the main form of plasticity in naive mice, was inhibited in DID mice. Interestingly, 1 μm sulpiride, a D2 receptor antagonist, restored tLTP. Although we observed no alterations of AMPA and NMDA receptor properties, we found that the AMPA/NMDA ratio increased at cortical and amygdaloid inputs but not at hippocampal inputs. Also, DID effects on STDP were accompanied by lower dendritic calcium transients. These data suggest that the role of dopamine in mediating the effects of binge alcohol drinking on synaptic plasticity of NAc MSNs differs markedly whether these neurons belong to the direct or indirect pathways.SIGNIFICANCE STATEMENT We examined the relationship between binge alcohol drinking and spike timing-dependent plasticity in nucleus accumbens (NAc) neurons. We found that repeated drinking bouts modulate differently synaptic plasticity in medium spiny neurons of the accumbens direct and indirect pathways. While timing-dependent long-term depression switches to long-term potentiation (LTP) in the former, timing-dependent LTP is inhibited in the latter. These effects are not accompanied by changes in AMPA and NMDA receptor properties at cortical, amygdaloid, and hippocampal synapses. Interestingly, dopamine D1 and D2 receptor antagonists have opposite effects on plasticity. Our data show that whether core NAc medium spiny neurons belong to the direct or indirect pathways determines the form of spike timing-dependent plasticity (STDP), the manner by which STDP responds to binge alcohol drinking, and its sensitivity to dopamine receptor antagonists.

Spike timing dependent plasticity (STDP) is a phenomenon in which the precise timing of spikes affects the sign and magnitude of changes in synaptic strength. STDP is often interpreted as the comprehensive learning rule for a synapse – the “first law” of synaptic plasticity. This interpretation is made explicit in theoretical models in which the total plasticity produced by complex spike patterns results from a superposition of the effects of all spike pairs. Although such models are appealing for their simplicity, they can fail dramatically. For example, the measured single-spike learning rule between hippocampal CA3 and CA1 pyramidal neurons does not predict the existence of long-term potentiation one of the best-known forms of synaptic plasticity. Layers of complexity have been added to the basic STDP model to repair predictive failures, but they have been outstripped by experimental data. We propose an alternate first law: neural activity triggers changes in key biochemical intermediates, which act as a more direct trigger of plasticity mechanisms. One particularly successful model uses intracellular calcium as the intermediate and can account for many observed properties of bidirectional plasticity. In this formulation, STDP is not itself the basis for explaining other forms of plasticity, but is instead a consequence of changes in the biochemical intermediate, calcium. Eventually a mechanism-based framework for learning rules should include other messengers, discrete change at individual synapses, spread of plasticity among neighboring synapses, and priming of hidden processes that change a synapse's susceptibility to future change. Mechanism-based models provide a rich framework for the computational representation of synaptic plasticity.

Since the discovery of place cells – single pyramidal neurons that encode spatial location – it has been hypothesized that the hippocampus may act as a cognitive map of known environments. This putative function has been extensively modeled using auto-associative networks, which utilize rate-coded synaptic plasticity rules in order to generate strong bi-directional connections between concurrently active place cells that encode for neighboring place fields. However, empirical studies using hippocampal cultures have demonstrated that the magnitude and direction of changes in synaptic strength can also be dictated by the relative timing of pre- and post-synaptic firing according to a spike-timing dependent plasticity (STDP) rule. Furthermore, electrophysiology studies have identified persistent “theta-coded” temporal correlations in place cell activity in vivo, characterized by phase precession of firing as the corresponding place field is traversed. It is not yet clear if STDP and theta-coded neural dynamics are compatible with cognitive map theory and previous rate-coded models of spatial learning in the hippocampus. Here, we demonstrate that an STDP rule based on empirical data obtained from the hippocampus can mediate rate-coded Hebbian learning when pre- and post-synaptic activity is stochastic and has no persistent sequence bias. We subsequently demonstrate that a spiking recurrent neural network that utilizes this STDP rule, alongside theta-coded neural activity, allows the rapid development of a cognitive map during directed or random exploration of an environment of overlapping place fields. Hence, we establish that STDP and phase precession are compatible with rate-coded models of cognitive map development.

A central hypothesis concerning brain functioning is that plasticity regulates the signal transfer function by modifying the efficacy of synaptic transmission. In the cerebellum, the granular layer has been shown to control the gain of signals transmitted through the mossy fiber pathway. Until now, the impact of plasticity on incoming activity patterns has been analyzed by combining electrophysiological recordings in acute cerebellar slices and computational modeling, unraveling a broad spectrum of different forms of synaptic plasticity in the granular layer, often accompanied by forms of intrinsic excitability changes. Here, we attempt to provide a brief overview of the most prominent forms of plasticity at the excitatory synapses formed by mossy fibers onto primary neuronal components (granule cells, Golgi cells and unipolar brush cells) in the granular layer. Specifically, we highlight the current understanding of the mechanisms and their functional implications for synaptic and intrinsic plasticity, providing valuable insights into how inputs are processed and reconfigured at the cerebellar input stage.

Spike timing is an important factor in the modification of synaptic strength. Various forms of spike timing-dependent plasticity (STDP) occur in the brains of diverse species, from insects to humans. In unimodal STDP, only LTP or LTD occurs at the synapse, regardless of which neuron spikes first; the magnitude of potentiation or depression increases as the time between presynaptic and postsynaptic spikes decreases. This from of STDP may promote developmental strengthening or weakening of early projections. In bidirectional Hebbian STDP, the magnitude and the sign (potentiation or depression) of plasticity depend, respectively, on the timing and the order of presynaptic and postsynaptic spikes. In the rodent barrel cortex, multiple forms of STDP appear sequentially during development, and they contribute to network formation, retraction, or fine-scale functional reorganization. Hebbian STDP appears at L4-L2/3 synapses starting at postnatal day (P) 15; the synapses exhibit unimodal "all-LTP STDP" before that age. The appearance of Hebbian STDP at L4-L2/3 synapses coincides with the maturation of parvalbumin-containing GABA interneurons in L4, which contributes to the generation of L4-before-L2/3 spiking in response to thalamic input by producing fast feedforward suppression of both L4 and L2/3 cells. After P15, L4-L2/3 STDP mediates fine-scale circuit refinement, essential for the critical period in the barrel cortex. In this review, we first briefly describe the relevance of STDP to map plasticity in the barrel cortex, then look over roles of distinct forms of STDP during development. Finally, we propose a hypothesis that explains the transition from network formation to the initiation of the critical period in the barrel cortex.

Synaptic plasticity provides the basis for most models of learning, memory and development in neural networks. The challenge for neuromorphic system designers is to find out efficient architectures to process accurately and speedily plasticity rules for a large number of synaptic connections. In this work, we propose a configurable architecture for real-time synaptic plasticity computation. Based on a dedicated plasticity processor, the architecture runs plasticity rules after a predefined configuration. As proof of concept, we implement on a commercial FPGA a biologically inspired form of spike timing-dependent plasticity (STDP) with complex time dependencies between pairs of pre- and post-synaptic spikes. Experimental results evaluate computation accuracy and speed as well as the number of synaptic connections we can process.

Rate-coded Hebbian learning, as characterized by the BCM formulation, is an established computational model of synaptic plasticity. Recently it has been demonstrated that changes in the strength of synapses in vivo can also depend explicitly on the relative timing of pre- and postsynaptic firing. Computational modeling of this spike-timing-dependent plasticity (STDP) has demonstrated that it can provide inherent stability or competition based on local synaptic variables. However, it has also been demonstrated that these properties rely on synaptic weights being either depressed or unchanged by an increase in mean stochastic firing rates, which directly contradicts empirical data. Several analytical studies have addressed this apparent dichotomy and identified conditions under which distinct and disparate STDP rules can be reconciled with rate-coded Hebbian learning. The aim of this research is to verify, unify, and expand on these previous findings by manipulating each element of a standard computational STDP model in turn. This allows us to identify the conditions under which this plasticity rule can replicate experimental data obtained using both rate and temporal stimulation protocols in a spiking recurrent neural network. Our results describe how the relative scale of mean synaptic weights and their dependence on stochastic pre- or postsynaptic firing rates can be manipulated by adjusting the exact profile of the asymmetric learning window and temporal restrictions on spike pair interactions respectively. These findings imply that previously disparate models of rate-coded autoassociative learning and temporally coded heteroassociative learning, mediated by symmetric and asymmetric connections respectively, can be implemented in a single network using a single plasticity rule. However, we also demonstrate that forms of STDP that can be reconciled with rate-coded Hebbian learning do not generate inherent synaptic competition, and thus some additional mechanism is required to guarantee long-term input-output selectivity.

Neurons show plasticity in neuronal and synaptic properties due to development and/or learning, affecting both the input levels to the neuron as well as the neural excitability. However, neurons have a limited dynamic range, i.e. the range over which they are sensitive to the input and are not in either a quiescent or a saturated activity state. This suggests neurons possess control mechanisms that match neural excitability and synaptic input levels. Recent experimental studies suggest that neurons indeed show a homeostatic scaling of excitability (HSE) by sensing activity levels and adapting the neural excitability via regulation of specific membrane conductance densities. The maintenance of sensitivity to synaptic input is also central to learning processes. In one form of learning it has been demonstrated that synaptic modification depends on the exact timing of presynaptic inputs and postsynaptic spikes. The performance of this spike timing dependent plasticity (STDP) is expected to be affected by a decrease in the sensitivity of the neuron to its input. At the one hand this suggests an important role for HSE in the functioning of STDP, at the other hand it leads to the question whether HSE could interfere with the learning of input patterns via STDP. Here, we address these issues by using both mathematical analysis and numerical simulations of a neuron that shows HSE and that receives input from synapses showing STDP. Based on experimental results, HSE is implemented as activity-dependent shifts of the input-output function. We use the multiplicative formulation of STDP in which the changes in synaptic strength depend on the synaptic strength itself. We show that while background input levels vary greatly, HSE keeps the neuron within its dynamic range and does not affect the synaptic weight distribution. HSE can also easily compensate for variations in the shape of the STDP learning window and maintain the sensitivity to correlations in the input. However, in neurons without HSE, the sensitivity to correlations in the input depends strongly on the various parameters. The effects of HSE are further explored by examining the neuron response to input patterns. We show that when neural excitability is controlled by HSE, STDP leads to changes of the synaptic weights as a function of the properties of the input pattern, i.e. the number of inputs forming the pattern and the strength of the correlation within the pattern. Learning of a pattern increases the probability of it generating a postsynaptic spike, depending on the properties of the pattern. HSE makes the effect of learning input patterns almost independent of the background levels. The results suggest HSE does not interfere with STDP and that HSE has a central role in maintaining the learning capabilities of the neuron in its highly plastic environment.

Synaptic plasticity is regarded as the major candidate mechanism for synaptic information storage and memory formation in the hippocampus. Mitogen-activated protein kinases have recently emerged as an important regulatory factor in many forms of synaptic plasticity and memory. As one of the subfamilies of mitogen-activated protein kinases, extracellular-regulated kinase is involved in the in vitro induction of long-term potentiation (LTP), whereas p38 mediates metabotropic glutamate receptor-dependent long-term depression (LTD) in vitro. Although c-Jun N-terminal kinase (JNK) has also been implicated in synaptic plasticity, the in vivo relevance of JNK activity to different forms of synaptic plasticity remains to be further explored. We investigated the effect of inhibition of JNK on different forms of synaptic plasticity in the dentate gyrus of freely behaving adult rats. Intracereboventricular application of c-Jun N-terminal protein kinase-inhibiting peptide (D-JNKI) (96 ng), a highly selective JNK inhibitor peptide, did not affect basal synaptic transmission but reduced neuronal excitability with a higher dose (192 ng). Application of D-JNKI, at a concentration that did not affect basal synaptic transmission, resulted in reduced specific phosphorylation of the JNK substrates postsynaptic density 95kD protein (PSD 95) and c-Jun, a significant enhancement of LTD and a facilitation of short-term depression into LTD. Both LTP and short-term potentiation were unaffected. An inhibition of depotentiation (recovery of LTP) occurred. These data suggest that suppression of JNK-dependent signalling may serve to enhance synaptic depression, and indirectly promote LTP through impairment of depotentiation.

The reinforcement learning hypothesis of dopamine function predicts that dopamine acts as a teaching signal by governing synaptic plasticity in the striatum. Induced changes in synaptic strength enable the cortico-striatal network to learn a mapping between situations and actions that lead to a reward. A review of the relevant neurophysiology of dopamine function in the cortico-striatal network and the machine reinforcement learning hypothesis reveals an apparent mismatch with recent electrophysiological studies. It was found that in addition to the well-described reward-related responses, a subpopulation of dopamine neurons also exhibits phasic responses to aversive stimuli or to cues predicting aversive stimuli. Obviously, actions that lead to aversive events should not be reinforced. However, published data suggest that the phasic responses of dopamine neurons to reward-related stimuli have a higher firing rate and have a longer duration than phasic responses of dopamine neurons to aversion-related stimuli. We propose that based on different dopamine concentrations, the target structures are able to decode reward-related dopamine from aversion-related dopamine responses. Thereby, the learning of actions in the basal-ganglia network integrates information about both costs and benefits. This hypothesis predicts that dopamine concentration should be a crucial parameter for plasticity rules at cortico-striatal synapses. Recent in vitro studies on cortico-striatal synaptic plasticity rules support a striatal action-learning scheme where during reward-related dopamine release dopamine-dependent forms of synaptic plasticity occur, while during aversion-related dopamine release the dopamine concentration only allows dopamine-independent forms of synaptic plasticity to occur.

The striatum is a major site of learning and memory formation for sensorimotor and cognitive association. One of the mechanisms used by the brain for memory storage is synaptic plasticity - the long-lasting, activity-dependent change in synaptic strength. All forms of synaptic plasticity require an elevation in intracellular calcium, and a common hypothesis is that the amplitude and duration of calcium transients can determine the direction of synaptic plasticity. The utility of this hypothesis in the striatum is unclear in part because dopamine is required for striatal plasticity and in part because of the diversity in stimulation protocols. To test whether calcium can predict plasticity direction, we developed a calcium-based plasticity rule using a spiny projection neuron model with sophisticated calcium dynamics including calcium diffusion, buffering and pump extrusion. We utilized three spike timing-dependent plasticity (STDP) induction protocols, in which postsynaptic potentials are paired with precisely timed action potentials and the timing of such pairing determines whether potentiation or depression will occur. Results show that despite the variation in calcium dynamics, a single, calcium-based plasticity rule, which explicitly considers duration of calcium elevations, can explain the direction of synaptic weight change for all three STDP protocols. Additional simulations show that the plasticity rule correctly predicts the NMDA receptor dependence of long-term potentiation and the L-type channel dependence of long-term depression. By utilizing realistic calcium dynamics, the model reveals mechanisms controlling synaptic plasticity direction, and shows that the dynamics of calcium, not just calcium amplitude, are crucial for synaptic plasticity.