Ramping Activity Research Articles

Competitive integration of time and reward explains value-sensitive foraging decisions and frontal cortex ramping dynamics.

Patch foraging presents a ubiquitous decision-making process in which animals decide when to abandon a resource patch of diminishing value to pursue an alternative. We developed a virtual foraging task in which mouse behavior varied systematically with patch value. Mouse behavior could be explained by a model integrating time and rewards antagonistically, scaled by a latent patience state. The model accounted for deviations from predictions of optimal foraging theory. Neural recordings throughout frontal areas revealed encoding of decision variables from the integrator model, most robustly in frontal cortex. Regression modeling followed by unsupervised clustering identified a subset of ramping neurons. These neurons' firing rates ramped up gradually (up to tens of seconds), were inhibited by rewards, and were better described as a continuous ramp than a discrete stepping process. Together, these results identify integration via frontal cortex ramping dynamics as a candidate mechanism for solving patch foraging problems.

Cognitive sequences in obsessive-compulsive disorder are supported by frontal cortex ramping activity and mediated by symptom severity.

Completing sequences is part of everyday life. Many such sequences can be considered abstract - that is, defined by a rule that governs the order but not the identity of individual steps (e.g., getting dressed for work). Over-engagement in ritualistic and repetitive behaviors seen in obsessive-compulsive disorder (OCD) suggests that abstract sequences may be disrupted in this disorder. Previous work has shown the necessity of the rostrolateral prefrontal cortex (RLPFC) for abstract sequence processing and that neural activity increases (ramps) in this region across sequences (Desrochers et al., 2015, 2019). Neurobiological models of the cortico-striatal-thalamo-cortical (CSTC) loops describe prefrontal circuitry connected to RLPFC and that is believed to be dysfunctional in OCD. As a potential extension of these models, we hypothesized that neural dynamics of RLPFC could be disrupted in OCD during abstract sequence engagement. We found that neural dynamics in RLPFC did not differ between OCD and healthy controls (HCs), but that increased ramping in pregenual anterior cingulate cortex (rACC), and superior frontal sulcus (SFS) dissociates these two groups in an abstract sequence paradigm. Further, we found that anxiety and depressive symptoms mediated the relationship between observed neural activity and behavioral differences observed in the task. This study highlights the importance of investigating ramping as a relevant neural dynamic during sequences and suggests expansion of current neurobiological models to include regions that support sequential behavior in OCD. Further, our results may point to novel regions to consider for neuromodulatory treatments of OCD in the future.

Slow ramping emerges from spontaneous fluctuations in spiking neural networks

The capacity to initiate actions endogenously is critical for goal-directed behavior. Spontaneous voluntary actions are typically preceded by slow-ramping activity in medial frontal cortex that begins around two seconds before movement, which may reflect spontaneous fluctuations that influence action timing. However, the mechanisms by which these slow ramping signals emerge from single-neuron and network dynamics remain poorly understood. Here, we developed a spiking neural-network model that produces spontaneous slow ramping activity in single neurons and population activity with onsets ~2 s before threshold crossings. A key prediction of our model is that neurons that ramp together have correlated firing patterns before ramping onset. We confirmed this model-derived hypothesis in a dataset of human single neuron recordings from medial frontal cortex. Our results suggest that slow ramping signals reflect bounded spontaneous fluctuations that emerge from quasi-winner-take-all dynamics in clustered networks that are temporally stabilized by slow-acting synapses.

Encoding of female mating dynamics by a hypothalamic line attractor

Females exhibit complex, dynamic behaviours during mating with variable sexual receptivity depending on hormonal status1–4. However, how their brains encode the dynamics of mating and receptivity remains largely unknown. The ventromedial hypothalamus, ventrolateral subdivision contains oestrogen receptor type 1-positive neurons that control mating receptivity in female mice5,6. Here, unsupervised dynamical system analysis of calcium imaging data from these neurons during mating uncovered a dimension with slow ramping activity, generating a line attractor in neural state space. Neural perturbations in behaving females demonstrated relaxation of population activity back into the attractor. During mating, population activity integrated male cues to ramp up along this attractor, peaking just before ejaculation. Activity in the attractor dimension was positively correlated with the degree of receptivity. Longitudinal imaging revealed that attractor dynamics appear and disappear across the oestrus cycle and are hormone dependent. These observations suggest that a hypothalamic line attractor encodes a persistent, escalating state of female sexual arousal or drive during mating. They also demonstrate that attractors can be reversibly modulated by hormonal status, on a timescale of days.

Integrator dynamics in the cortico-basal ganglia loop underlie flexible motor timing.

Flexible control of motor timing is crucial for behavior. Before volitional movement begins, the frontal cortex and striatum exhibit ramping spiking activity, with variable ramp slopes anticipating movement onsets. This activity in the cortico-basal ganglia loop may function as an adjustable 'timer,' triggering actions at the desired timing. However, because the frontal cortex and striatum share similar ramping dynamics and are both necessary for timing behaviors, distinguishing their individual roles in this timer function remains challenging. To address this, we conducted perturbation experiments combined with multi-regional electrophysiology in mice performing a flexible lick-timing task. Following transient silencing of the frontal cortex, cortical and striatal activity swiftly returned to pre-silencing levels and resumed ramping, leading to a shift in lick timing close to the silencing duration. Conversely, briefly inhibiting the striatum caused a gradual decrease in ramping activity in both regions, with ramping resuming from post-inhibition levels, shifting lick timing beyond the inhibition duration. Thus, inhibiting the frontal cortex and striatum effectively paused and rewound the timer, respectively. These findings suggest the striatum is a part of the network that temporally integrates input from the frontal cortex and generates ramping activity that regulates motor timing.

Encoding of self-initiated actions in axon terminals of the mesocortical pathway.

The initiation of goal-directed actions is a complex process involving the medial prefrontal cortex and dopaminergic inputs through the mesocortical pathway. However, it is unclear what information the mesocortical pathway conveys and how it impacts action initiation. In this study, we unveiled the indispensable role of mesocortical axon terminals in encoding the execution of movements in self-initiated actions. To investigate the role of mesocortical axon terminals in encoding the execution of movements in self-initiated actions. We designed a lever-press task in which mice internally determine the timing of the press, receiving a larger reward for longer waiting periods. Our study revealed that self-initiated actions depend on dopaminergic signaling mediated by D2 receptors, whereas sensory-triggered lever-press actions do not involve D2 signaling. Microprism-mediated two-photon calcium imaging further demonstrated ramping activity in mesocortical axon terminals approximately 0.5s before the self-initiated lever press. Remarkably, the ramping patterns remained consistent whether the mice responded to cues immediately for a smaller reward or held their response for a larger reward. We conclude that mesocortical dopamine axon terminals encode the timing of self-initiated actions, shedding light on a crucial aspect of the intricate neural mechanisms governing goal-directed behavior.

Neural Sequences and the Encoding of Time.

Converging experimental and computational evidence indicate that on the scale of seconds the brain encodes time through changing patterns of neural activity. Experimentally, two general forms of neural dynamic regimes that can encode time have been observed: neural population clocks and ramping activity. Neural population clocks provide a high-dimensional code to generate complex spatiotemporal output patterns, in which each neuron exhibits a nonlinear temporal profile. A prototypical example of neural population clocks are neural sequences, which have been observed across species, brain areas, and behavioral paradigms. Additionally, neural sequences emerge in artificial neural networks trained to solve time-dependent tasks. Here, we examine the role of neural sequences in the encoding of time, and how they may emerge in a biologically plausible manner. We conclude that neural sequences may represent a canonical computational regime to perform temporal computations.

Diverse Time Encoding Strategies Within the Medial Premotor Areas of the Primate.

The measurement of time in the subsecond scale is critical for many sophisticated behaviors, yet its neural underpinnings are largely unknown. Recent neurophysiological experiments from our laboratory have shown that the neural activity in the medial premotor areas (MPC) of macaques can represent different aspects of temporal processing. During single interval categorization, we found that preSMA encodes a subjective category limit by reaching a peak of activity at a time that divides the set of test intervals into short and long. We also observed neural signals associated with the category selected by the subjects and the reward outcomes of the perceptual decision. On the other hand, we have studied the behavioral and neurophysiological basis of rhythmic timing. First, we have shown in different tapping tasks that macaques are able to produce predictively and accurately intervals that are cued by auditory or visual metronomes or when intervals are produced internally without sensory guidance. In addition, we found that the rhythmic timing mechanism in MPC is governed by different layers of neural clocks. Next, the instantaneous activity of single cells shows ramping activity that encodes the elapsed or remaining time for a tapping movement. In addition, we found MPC neurons that build neural sequences, forming dynamic patterns of activation that flexibly cover all the produced interval depending on the tapping tempo. This rhythmic neural clock resets on every interval providing an internal representation of pulse. Furthermore, the MPC cells show mixed selectivity, encoding not only elapsed time, but also the tempo of the tapping and the serial order element in the rhythmic sequence. Hence, MPC can map different task parameters, including the passage of time, using different cell populations. Finally, the projection of the time varying activity of MPC hundreds of cells into a low dimensional state space showed circular neural trajectories whose geometry represented the internal pulse and the tapping tempo. Overall, these findings support the notion that MPC is part of the core timing mechanism for both single interval and rhythmic timing, using neural clocks with different encoding principles, probably to flexibly encode and mix the timing representation with other task parameters.

62 Neural Correlates of Cognitive Function in the Basal Ganglia

Objective:Timing, or the decision of when to act, is essential to mammalian behaviors from escaping predators to driving a car. It requires cognitive functions such as working memory for time-based rules and attention to the passing of time. Thus, it can be used as a proxy for higher order executive functions that are difficult to measure but are impaired in many neurological disorders. Therefore, insights from studies of interval timing, tasks which require estimating time intervals of several seconds, have great value for our understanding of human disease. Crucial to timing is the basal ganglia, which integrates cortical activity with midbrain dopamine signals and sends out signals to the spinal cord that regulate movement, motivation, and other behaviors. We have previously found that within the basal ganglia, medium spiny neurons of the striatum exhibit ramping activity in time-related tasks. In other words, they gradually increase or decrease firing frequency across a timed interval, and this is thought to encode time. Yet it is still unknown how the encoding of time is translated into time-based motor responses. To answer this question, we turned to the external globus pallidus (GPe) because it is a regulatory hub within the basal ganglia and is thus well positioned to regulate timing behavior. We sought to examine how the GPe functions in response to time-based demands.Participants and Methods:We recorded from neuronal ensembles using 16 channel electrode arrays implanted in the GPe of five mice while they performed an interval timing task called the switch interval timing task. Spike sorting was then used to identify signal from individual neurons.Results:Data were compiled from 43 neurons over several trials. Principal component analysis of neural firing activity was then conducted and revealed a downward ramping pattern in GPe neurons during interval timing trials. Data were then separated based on trials in which mice made correct decisions and those in which mice made a mistake. We found that when mice make correct timing decisions, there is downward ramping activity in the GPe, yet when mice make timing mistakes, this ramping pattern is lost.Conclusions:Our findings suggest that the GPe processes timing signals through ramping activity, before projecting to the output nuclei of the basal ganglia. This is a novel finding and contributes to a growing understanding of the temporal code of the basal ganglia. The full extent of this code is still unknown, but this insight contributes to a better understanding of how the globus pallidus represents cognition. If we can better explain the neural correlates of timing, we can use this knowledge to inform therapeutic interventions for basal ganglia dysfunction, which could have profound implications for diseases like Parkinson’s disease, which affects millions around the world.

Neural evidence of switch processes during semantic and phonetic foraging in human memory

Humans may retrieve words from memory by exploring and exploiting in "semantic space" similar to how nonhuman animals forage for resources in physical space. This has been studied using the verbal fluency test (VFT), in which participants generate words belonging to a semantic or phonetic category in a limited time. People produce bursts of related items during VFT, referred to as "clustering" and "switching." The strategic foraging model posits that cognitive search behavior is guided by a monitoring process which detects relevant declines in performance and then triggers the searcher to seek a new patch or cluster in memory after the current patch has been depleted. An alternative body of research proposes that this behavior can be explained by an undirected rather than strategic search process, such as random walks with or without random jumps to new parts of semantic space. This study contributes to this theoretical debate by testing for neural evidence of strategically timed switches during memory search. Thirty participants performed category and letter VFT during functional MRI. Responses were classified as cluster or switch events based on computational metrics of similarity and participant evaluations. Results showed greater hippocampal and posterior cerebellar activation during switching than clustering, even while controlling for interresponse times and linguistic distance. Furthermore, these regions exhibited ramping activity which increased during within-patch search leading up to switches. Findings support the strategic foraging model, clarifying how neural switch processes may guide memory search in a manner akin to foraging in patchy spatial environments.

Rat hippocampal CA1 region represents learning-related action and reward events with shorter latency than the lateral entorhinal cortex

The hippocampus and entorhinal cortex are deeply involved in learning and memory. However, little is known how ongoing events are processed in the hippocampal-entorhinal circuit. By recording from head-fixed rats during action-reward learning, here we show that the action and reward events are represented differently in the hippocampal CA1 region and lateral entorhinal cortex (LEC). Although diverse task-related activities developed after learning in both CA1 and LEC, phasic activities related to action and reward events differed in the timing of behavioral event representation. CA1 represented action and reward events almost instantaneously, whereas the superficial and deep layers of the LEC showed a delayed representation of the same events. Interestingly, we also found that ramping activity towards spontaneous action was correlated with waiting time in both regions and exceeded that in the motor cortex. Such functional activities observed in the entorhinal-hippocampal circuits may play a crucial role for animals in utilizing ongoing information to dynamically optimize their behaviors.

Multiplexing working memory and time in the trajectories of neural networks.

Working memory (WM) and timing are generally considered distinct cognitive functions, but similar neural signatures have been implicated in both. To explore the hypothesis that WM and timing may rely on shared neural mechanisms, we used psychophysical tasks that contained either task-irrelevant timing or WM components. In both cases, the task-irrelevant component influenced performance. We then developed recurrent neural network (RNN) simulations that revealed that cue-specific neural sequences, which multiplexed WM and time, emerged as the dominant regime that captured the behavioural findings. During training, RNN dynamics transitioned from low-dimensional ramps to high-dimensional neural sequences, and depending on task requirements, steady-state or ramping activity was also observed. Analysis of RNN structure revealed that neural sequences relied primarily on inhibitory connections, and could survive the deletion of all excitatory-to-excitatory connections. Our results indicate that in some instances WM is encoded in time-varying neural activity because of the importance of predicting when WM will be used.

Context-specific early recruitment of small motor units in the shoulder muscle reflects a reach movement plan.

Understanding how motor plans are transformed into appropriate patterns of muscle activity is a central question in motor control. While muscle activity during the delay period has not been reported using conventional EMG approaches, we isolated motor unit activity using a high-density surface EMG signal from the anterior deltoid muscle to test whether heterogeneity in motor units could reveal early preparatory activity. Consistent with our previous work (Rungta et al., 2021), we observed early selective recruitment of small-amplitude size motor units during the delay period for hand movements like early recruitment of small-amplitude motor units in neck muscles of non-human primates performing delayed saccade tasks. This early activity was spatially specific and increased with time and resembled an accumulation to threshold model that correlated with movement onset time. Such early recruitment of ramping motor units was observed at the single trial level as well. In contrast, no such recruitment of large amplitude size motor units, called non-rampers, was observed during the delay period. Instead, non-rampers became spatially specific and predicted movement onset time after the delay period. Interestingly, spatially specific delay period activity was only observed for hand movements but was absent for isometric force-driven cursor movements. Nonetheless, muscle activity was correlated with the time it took to initiate movements in both task conditions for non-rampers. Overall, our results reveal a novel heterogeneity in the EMG activity which allows the expression of early motor preparation via small amplitude size motor units which are differentially activated during initiation of movements.

Beyond Scalar Timing Theory: Integrating Neural Oscillators with Computational Accessibility in Memory

Abstract One of the major challenges for computational models of timing and time perception is to identify a neurobiological plausible implementation that predicts various behavioral properties, including the scalar property and retrospective timing. The available timing models primarily focus on the scalar property and prospective timing, while virtually ignoring the computational accessibility. Here, we first selectively review timing models based on ramping activity, oscillatory pattern, and time cells, and discuss potential challenges for the existing models. We then propose a multifrequency oscillatory model that offers computational accessibility, which could account for a much broader range of timing features, including both retrospective and prospective timing.

Neural population clocks: Encoding time in dynamic patterns of neural activity.

The ability to predict and prepare for near- and far-future events is among the most fundamental computations the brain performs. Because of the importance of time for prediction and sensorimotor processing, the brain has evolved multiple mechanisms to tell and encode time across scales ranging from microseconds to days and beyond. Converging experimental and computational data indicate that, on the scale of seconds, timing relies on diverse neural mechanisms distributed across different brain areas. Among the different encoding mechanisms on the scale of seconds, we distinguish between neural population clocks and ramping activity as distinct strategies to encode time. One instance of neural population clocks, neural sequences, represents in some ways an optimal and flexible dynamic regime for the encoding of time. Specifically, neural sequences comprise a high-dimensional representation that can be used by downstream areas to flexibly generate arbitrarily simple and complex output patterns using biologically plausible learning rules. We propose that high-level integration areas may use high-dimensional dynamics such as neural sequences to encode time, providing downstream areas information to build low-dimensional ramp-like activity that can drive movements and temporal expectation. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

Spatial representation by ramping activity of neurons in the retrohippocampal cortex

Neurons in the retrohippocampal cortices play crucial roles in spatial memory. Many retrohippocampal neurons have firing fields that are selectively active at specific locations, with memory for rewarded locations associated with reorganization of these firing fields. Whether this is the sole strategy for representing spatial memories is unclear. Here, we demonstrate that during a spatial memory task retrohippocampal neurons encode location through ramping activity that extends across segments of a linear track approaching and following a reward, with the rewarded location represented by offsets or switches in the slope of the ramping activity. Ramping representations could be maintained independently of trial outcome and cues marking the reward location, indicating that they result from recall of the track structure. When recorded in an open arena, neurons that generated ramping activity during the spatial memory task were more numerous than grid or border cells, with a majority showing spatial firing that did not meet criteria for classification as grid or border representations. Encoding of rewarded locations through offsets and switches in the slope of ramping activity also emerged in recurrent neural network models trained to solve a similar spatial memory task. Impaired performance of model networks following disruption of outputs from ramping neurons is consistent with this coding strategy supporting navigation to recalled locations of behavioral significance. Our results suggest that encoding of learned spaces by retrohippocampal networks employs both discrete firing fields and continuous ramping representations. We hypothesize that retrohippocampal ramping activity mediates readout of learned models for goal-directed navigation.

Hippocampal astrocytes encode reward location.

Astrocytic calcium dynamics has been implicated in the encoding of sensory information1-5, and modulation of calcium in astrocytes has been shown to affect behaviour6-10. However, longitudinal investigation of the real-time calcium activity of astrocytes in the hippocampus of awake mice is lacking. Here we used two-photon microscopy to chronically image CA1 astrocytes as mice ran in familiar or new virtual environments to obtain water rewards. We found that astrocytes exhibit persistent ramping activity towards the reward location in a familiar environment, but not in a new one. Shifting the reward location within a familiar environment also resulted in diminished ramping. After additional training, as the mice became familiar with the new context or new reward location, the ramping was re-established. Using linear decoders, we could predict the location of the mouse in a familiar environment from astrocyte activity alone. We could not do the same in a new environment, suggesting that the spatial modulation of astrocytic activity is experience dependent. Our results indicate that astrocytes can encode the expected reward location in spatial contexts, thereby extending their known computational abilities and their role in cognitive functions.

Ramping activity in the striatum

Control of the timing of behavior is thought to require the basal ganglia (BG) and BG pathologies impair performance in timing tasks. Temporal interval discrimination depends on the ramping activity of medium spiny neurons (MSN) in the main BG input structure, the striatum, but the underlying mechanisms driving this activity are unclear. Here, we combine an MSN dynamical network model with an action selection system applied to an interval discrimination task. We find that when network parameters are appropriate for the striatum so that slowly fluctuating marginally stable dynamics are intrinsically generated, up and down ramping populations naturally emerge which enable significantly above chance task performance. We show that emergent population activity is in very good agreement with empirical studies and discuss how MSN network dysfunction in disease may alter temporal perception.

Delay of gratification dissociates cognitive control and valuation brain regions in healthy young adults

Delay of gratification (DofG) refers to an inter-temporal choice phenomenon that is of great interest in many domains, including animal learning, cognitive development, economic decision-making, and executive control. Yet experimental tools for investigating DofG in human adults are almost non-existent, and as a consequence, very little is known regarding the brain basis of core DofG behaviors. Here, we utilize a novel DofG paradigm, adapted for use in neuroimaging contexts, to examine event-related changes in neural activity as healthy young adult participants made repeated choices to continue waiting for a delayed reward, rather than take an immediately available one of lesser value. On DofG trials, choose-to-wait events were associated with increased activation in fronto-parietal and cingulo-opercular regions associated with cognitive control. Activity in the right lateral prefrontal cortex (PFC) was also associated with individual variability in task performance and strategy. Fronto-parietal activity was clearly dissociable from that observed in ventromedial PFC, as this latter region exhibited a ramping-up pattern of activity during the waiting period prior to reward delivery. Ventromedial PFC ramping activity dynamics were further selective to DofG trials associated with increased future reward rate, consistent with the involvement of this region in subjective reward valuation that incorporates higher-order task structure. These results provide important initial validation of this experimental paradigm as a useful tool for investigating and isolating unique DofG neural mechanisms, which can now be utilized to study a wide-variety of populations and task factors.

Encoding time in neural dynamic regimes with distinct computational tradeoffs.

Converging evidence suggests the brain encodes time in dynamic patterns of neural activity, including neural sequences, ramping activity, and complex dynamics. Most temporal tasks, however, require more than just encoding time, and can have distinct computational requirements including the need to exhibit temporal scaling, generalize to novel contexts, or robustness to noise. It is not known how neural circuits can encode time and satisfy distinct computational requirements, nor is it known whether similar patterns of neural activity at the population level can exhibit dramatically different computational or generalization properties. To begin to answer these questions, we trained RNNs on two timing tasks based on behavioral studies. The tasks had different input structures but required producing identically timed output patterns. Using a novel framework we quantified whether RNNs encoded two intervals using either of three different timing strategies: scaling, absolute, or stimulus-specific dynamics. We found that similar neural dynamic patterns at the level of single intervals, could exhibit fundamentally different properties, including, generalization, the connectivity structure of the trained networks, and the contribution of excitatory and inhibitory neurons. Critically, depending on the task structure RNNs were better suited for generalization or robustness to noise. Further analysis revealed different connection patterns underlying the different regimes. Our results predict that apparently similar neural dynamic patterns at the population level (e.g., neural sequences) can exhibit fundamentally different computational properties in regards to their ability to generalize to novel stimuli and their robustness to noise—and that these differences are associated with differences in network connectivity and distinct contributions of excitatory and inhibitory neurons. We also predict that the task structure used in different experimental studies accounts for some of the experimentally observed variability in how networks encode time.