Reinforcement Cues Research Articles

Evidence for socially influenced and potentially actively coordinated cooperation by bumblebees.

Cooperation is common in animals, yet the specific mechanisms driving collaborative behaviour in different species remain unclear. We investigated the proximate mechanisms underlying the cooperative behaviour of bumblebees in two different tasks, where bees had to simultaneously push a block in an arena or a door at the end of a tunnel for access to reward. In both tasks, when their partner's entry into the arena/tunnel was delayed, bees took longer to first push the block/door compared with control bees that learned to push alone. In the tunnel task, just before gaining access to reward, bees were more likely to face towards their partner than expected by chance or compared with controls. These results show that bumblebees' cooperative behaviour is not simply a by-product of individual efforts but is socially influenced. We discuss how bees' turning behaviours, e.g. turning around before first reaching the door when their partner was delayed and turning back towards the door in response to seeing their partner heading towards the door, suggest the potential for active coordination. However, because these behaviours could also be interpreted as combined responses to social and secondary reinforcement cues, future studies are needed to help clarify whether bumblebees truly use active coordination.

Individual bitter-sensing neurons in Drosophila exhibit both ON and OFF responses that influence synaptic plasticity

The brain generates internal representations that translate sensory stimuli into appropriate behavior. In the taste system, different tastes activate distinct populations of sensory neurons. We investigated the temporal properties of taste responses in Drosophila and discovered that different types of taste sensory neurons show striking differences in their response dynamics. Strong responses to stimulus onset (ON responses) and offset (OFF responses) were observed in bitter-sensing neurons in the labellum, whereas bitter neurons in the leg and other classes of labellar taste neurons showed only an ON response. Individual labellar bitter neurons generate both ON and OFF responses through a cell-intrinsic mechanism that requires canonical bitter receptors. A single receptor complex likely generates both ON and OFF responses to a given bitter ligand. These ON and OFF responses in the periphery are propagated to dopaminergic neurons that mediate aversive learning, and the presence of the OFF response impacts synaptic plasticity when bitter is used as a reinforcement cue. These studies reveal previously unknown features of taste responses that impact neural circuit function and may be important for behavior. Moreover, these studies show that OFF responses can dramatically influence timing-based synaptic plasticity, which is thought to underlie associative learning.

Context-dependent representations of movement in Drosophila dopaminergic reinforcement pathways.

Dopamine plays a central role in motivating and modifying behavior, serving to invigorate current behavioral performance and guide future actions through learning. Here we examine how this single neuromodulator can contribute to such diverse forms of behavioral modulation. By recording from the dopaminergic reinforcement pathways of the Drosophila mushroom body during active odor navigation, we reveal how their ongoing motor-associated activity relates to goal-directed behavior. We find that dopaminergic neurons correlate with different behavioral variables depending on the specific navigational strategy of an animal, such that the activity of these neurons preferentially reflects the actions most relevant to odor pursuit. Furthermore, we show that these motor correlates are translated to ongoing dopamine release and acutely perturbing dopaminergic signaling alters the strength of odor tracking. Context-dependent representations of movement and reinforcement cues are thus multiplexed within the mushroom body dopaminergic pathways, enabling them to coordinately influence both ongoing and future behavior.

Visual instrumental learning in blindsight monkeys

Blindsight is the residual visuo-motor ability without subjective awareness observed after lesions of the primary visual cortex (V1). Various visual functions are retained, however, instrumental visual associative learning remains to be investigated. Here we examined the secondary reinforcing properties of visual cues presented to the hemianopic field of macaque monkeys with unilateral V1 lesions. Our aim was to test the potential role of visual pathways bypassing V1 in reinforcing visual instrumental learning. When learning the location of a hidden area in an oculomotor search task, conditioned visual cues presented to the lesion-affected hemifield operated as an effective secondary reinforcer. We noted that not only the hidden area location, but also the vector of the saccade entering the target area was reinforced. Importantly, when the visual reinforcement signal was presented in the lesion-affected field, the monkeys continued searching, as opposed to stopping when the cue was presented in the intact field. This suggests the monkeys were less confident that the target location had been discovered when the reinforcement cue was presented in the affected field. These results indicate that the visual signals mediated by the residual visual pathways after V1 lesions can access fundamental reinforcement mechanisms but with impaired visual awareness.

Moral clarity decreases as viewer age increases: a content analysis of the moral values and reinforcement cues depicted in popular U.S. children’s television

ABSTRACT Evidence suggests that exposure to narrative media content emphasizing the importance of moral intuitions can increase the extent to which children value those intuitions. This is especially the case when intuition exemplars are accompanied by clear social desirability cues (i.e., moral acts are rewarded/performed by heroes or immoral acts are punished/performed by villains). Less is known about the effects of exposure to ambiguous social desirability cues (i.e., moral acts are punished/performed by villains or immoral acts are rewarded/performed by heroes). In order to investigate the extent to which popular media content may serve as an effective moral educator for young viewers, this paper describes a content analysis designed to examine the frequency and clarity of moral intuition exemplars in popular children’s television shows (N = 30). Results demonstrated that moral clarity in content decreased as viewer age increased. That is, content popular among older children (ages 6–17) featured more ambiguous social desirability cues compared to content popular among younger children (ages 2–5). Content across age groups also featured a preponderance of care and fairness intuition exemplars. Findings are interpreted in line with extant research on moral complexity and narrative entertainment. Practical implications are considered. IMPACT SUMMARY a. Prior State of Knowledge Recent work has demonstrated that narrative media’s emphasis on moral intuitions can increase the importance audiences place on those intuitions. This work largely focuses on understanding the effects of moral exemplars that are clearly reinforced in narratives. b. Novel Contributions Using content analysis, the present study demonstrated that the clarity of narrative reinforcement cues decreases as the age group in which the content is popular increases. Specifically, moral exemplars in television shows popular among teens featured the most ambiguous reinforcement cues. c. Practical Implications For caregivers hoping to use media as a tool for inculcating certain moral values in children, these results suggest the importance of considering not only which moral values are exemplified in content, but also how clearly those values are reinforced.

Nucleus Basalis Links Sensory Stimuli with Delayed Reinforcement to Support Learning

Linking stimuli with delayed reinforcement requires neural circuits that can bridge extended temporal gaps. Auditory cortex (ACx) circuits reorganize to support auditory fear learning, but only when afferent sensory inputs temporally overlap with cholinergic reinforcement signals. Here we show that mouse ACx neurons rapidly reorganize to support learning, even when sensory and reinforcement cues are separated by a long gap. We found that cholinergic basal forebrain neurons bypass the temporal delay through multiplexed, short-latency encoding of sensory and reinforcement cues. At the initiation of learning, cholinergic neurons in Nucleus Basalis increase responses to conditioned sound frequencies and increase functional connectivity with ACx. By rapidly scaling up responses to sounds that predict reinforcement, cholinergic inputs jump the gap to align with bottom-up sensory traces and support associative cortical plasticity.

Impulsive personality dimensions are associated with altered behavioral performance and neural responses in the monetary incentive delay task.

Individual differences in dimensions of impulsivity personality including disinhibition and sensation seeking modulate approach responses to reinforcing stimuli, such as drugs and money. The current study examined the effects of monetary incentive on both behavioral performance and electrophysiological activity among individuals varying in disinhibition and sensation seeking. The monetary incentive delay (MID) task was completed under electroencephalogram (EEG) recording. Behavioral data showed that higher disinhibition and sensation-seeking were associated with lower performance accuracy. Event-related potential (ERP) data showed that high reinforcement cues elicited a larger late positive component (LPC) than other conditions among high disinhibition participants, indicating its strong emotional influence. Additionally, in the neutral incentive condition, the feedback-related negativity (FRN) elicited by correct outcomes was larger than that elicited by incorrect outcomes in the high disinhibition group only. This novel finding indicates that high disinhibition participants were less likely to expect correct outcomes compared to incorrect outcomes in the neutral incentive condition. Finally, the P3 component elicited by outcome presentation showed an interaction between two impulsivity dimensions; when disinhibition level was low, the P3 was larger among high than low sensation seeking participants.

Alterations in the functional neural circuitry supporting flexible choice behavior in autism spectrum disorders.

Restricted and repetitive behaviors, and a pronounced preference for behavioral and environmental consistency, are distinctive characteristics of autism spectrum disorder (ASD). Alterations in frontostriatal circuitry that supports flexible behavior might underlie this behavioral impairment. In an functional magnetic resonance imaging study of 17 individuals with ASD, and 23 age-, gender- and IQ-matched typically developing control participants, reversal learning tasks were used to assess behavioral flexibility as participants switched from one learned response choice to a different response choice when task contingencies changed. When choice outcome after reversal was uncertain, the ASD group demonstrated reduced activation in both frontal cortex and ventral striatum, in the absence of task performance differences. When the outcomes of novel responses were certain, there was no difference in brain activation between groups. Reduced activation in frontal cortex and ventral striatum suggest problems in decision-making and response planning, and in processing reinforcement cues, respectively. These processes, and their integration, are essential for flexible behavior. Alterations in these systems may therefore contribute to a rigid adherence to preferred behavioral patterns in individuals with an ASD. These findings provide an additional impetus for the use of reversal learning paradigms as a translational model for treatment development targeting the domain of restricted and repetitive behaviors in ASD.

How Does Awareness Modulate Goal-Directed and Stimulus-Driven Shifts of Attention Triggered by Value Learning?

In order to behave adaptively, attention can be directed in space either voluntarily (i.e., endogenously) according to strategic goals, or involuntarily (i.e., exogenously) through reflexive capture by salient or novel events. The emotional or motivational value of stimuli can also strongly influence attentional orienting. However, little is known about how reward-related effects compete or interact with endogenous and exogenous attention mechanisms, particularly outside of awareness. Here we developed a visual search paradigm to study subliminal value-based attentional orienting. We systematically manipulated goal-directed or stimulus-driven attentional orienting and examined whether an irrelevant, but previously rewarded stimulus could compete with both types of spatial attention during search. Critically, reward was learned without conscious awareness in a preceding phase where one among several visual symbols was consistently paired with a subliminal monetary reinforcement cue. Our results demonstrated that symbols previously associated with a monetary reward received higher attentional priority in the subsequent visual search task, even though these stimuli and reward were no longer task-relevant, and despite reward being unconsciously acquired. Thus, motivational processes operating independent of conscious awareness may provide powerful influences on mechanisms of attentional selection, which could mitigate both stimulus-driven and goal-directed shifts of attention.

A Context-Aware Application to Increase Elderly Users Compliance with Physical Rehabilitation Exercises at Home via Animatronic Biofeedback.

Biofeedback from physical rehabilitation exercises has proved to lead to faster recovery, better outcomes, and increased patient motivation. In addition, it allows the physical rehabilitation processes carried out at the clinic to be complemented with exercises performed at home. However, currently existing approaches rely mostly on audio and visual reinforcement cues, usually presented to the user on a computer screen or a mobile phone interface. Some users, such as elderly people, can experience difficulties to use and understand these interfaces, leading to non-compliance with the rehabilitation exercises. To overcome this barrier, latest biosignal technologies can be used to enhance the efficacy of the biofeedback, decreasing the complexity of the user interface. In this paper we propose and validate a context-aware framework for the use of animatronic biofeedback, as a way of potentially increasing the compliance of elderly users with physical rehabilitation exercises performed at home. In the scope of our work, animatronic biofeedback entails the use of pre-programmed actions on a robot that are triggered in response to certain changes detected in the users biomechanical or electrophysiological signals. We use electromyographic and accelerometer signals, collected in real time, to monitor the performance of the user while executing the exercises, and a mobile robot to provide animatronic reinforcement cues associated with their correct or incorrect execution. A context-aware application running on a smartphone aggregates the sensor data and controls the animatronic feedback. The acceptability of the animatronic biofeedback has been tested on a set of volunteer elderly users, and results suggest that the participants found the animatronic feedback engaging and of added value.

Attention-deficit/hyperactivity disorder, reward and punishment sensitivity and functional and dysfunctional impulsivity

Several theories of attention deficit/hyperactivity disorder (ADHD) have posited etiological roles for trait impulsivity and for the two motivational systems underlying sensitivity to reinforcement and punishment cues. It has been shown that both impulsivity and sensitivity to reward and punishment account for a significant portion of individual differences in ADHD. However, it is currently unclear if functional and dysfunctional impulsivity account for variance in ADHD symptoms incrementally above associations with reward and punishment sensitivity. Results from a sample of 123 (55% female) parents of children with ADHD revealed that dysfunctional impulsivity, sensitivity to reward and punishment had significant partial relationships with total ADHD symptoms; dysfunctional impulsivity, sensitivity to punishment and, to a lesser extent, sensitivity to reward, were related to inattention symptoms; and sensitivity to reward and dysfunctional impulsivity were related to hyperactivity-impulsivity symptoms. Results are discussed in terms of the recent theories of ADHD and the reinforcement sensitivity theory.

Short-Term Quetiapine Treatment Alters the Use of Reinforcement Signals during Risky Decision-Making and Promotes the Choice of Negative Expected Values in Healthy Adult Males

Effective decision-making can involve using environmental signals about the possible good and bad outcomes, and their probabilities, to select optimal actions. Problematic decision-making in psychiatric disorders, and particularly bipolar illness, may result from disrupted use of these reinforcement cues, leading to actions that reflect or precipitate pathological changes in mood. Previous experiments indicate that the processing of reinforcement cues while selecting between risky actions can be influenced by dopamine and serotonin activity. Quetiapine is an atypical antipsychotic agent with a complex pharmacology, including antagonist actions at 5-HT2A and, to a lesser extent, D2 receptors. Here, we investigated the effects of (short-term) treatment with quetiapine on the risky decision-making of healthy human adults. Twenty participants received 150 mg of quetiapine XL for 7 d, whereas 20 age- and IQ-matched participants received a placebo. On the eighth day, all participants completed a risky decision-making task that involved making a series of choices between two simultaneously presented gambles that differed in the magnitudes of their possible gains and losses, and the probabilities with which these outcomes were delivered. Quetiapine treatment was associated with a marked tendency to choose options with negative expected values compared with placebo treatment in male but not female participants. Our results demonstrate that antagonism of serotonin and dopamine receptor activity can alter the way individuals use information about gains and losses when selecting between risky actions, possibly reflecting gender-specific differences in risk attitudes. These effects may be beneficial by correcting decision-making biases that feature in mood disorders.

Pigeons’ use of cues in a repeated five-trial-sequence, single-reversal task

We studied behavioral flexibility, or the ability to modify one's behavior in accordance with the changing environment, in pigeons using a reversal-learning paradigm. In two experiments, each session consisted of a series of five-trial sequences involving a simple simultaneous color discrimination in which a reversal could occur during each sequence. The ideal strategy would be to start each sequence with a choice of S1 (the first correct stimulus) until it was no longer correct, and then to switch to S2 (the second correct stimulus), thus utilizing cues provided by local reinforcement (feedback from the preceding trial). In both experiments, subjects showed little evidence of using local reinforcement cues, but instead used the mean probabilities of reinforcement for S1 and S2 on each trial within each sequence. That is, subjects showed remarkably similar behavior, regardless of where (or, in Exp. 2, whether) a reversal occurred during a given sequence. Therefore, subjects appeared to be relatively insensitive to the consequences of responses (local feedback) and were not able to maximize reinforcement. The fact that pigeons did not use the more optimal feedback afforded by recent reinforcement contingencies to maximize their reinforcement has implications for their use of flexible response strategies under reversal-learning conditions.

A Dynamic Model of Guilt

Guilt is widely recognized as an important self-regulatory emotion, yet alternative theoretical accounts view guilt primarily as either a punishment cue or a pro-social motivator. Integrating these views, we propose that guilt functions dynamically to first provide a negative reinforcement cue associated with reduced approach motivation, which transforms into approach-motivated behavior when an opportunity for reparation presents itself. We tested this hypothesis in the context of racial prejudice. White subjects viewed a multiracial series of faces while cortical activity was recorded using electro-encephalography. Following bogus feedback indicating anti-Black responses, subjects reported elevated guilt, which was associated with changes in frontal cortical asymmetry indicating reduced approach motivation. When subjects were presented with an opportunity to engage in prejudice-reducing behavior, guilt predicted greater interest in prejudice reduction, which in turn was associated with an approach-related shift in frontal asymmetry. The results support a dynamic model in which guilt is associated with adaptive changes in motivation and behavior.

The Effects of Low Doses of Δ-9 Tetrahydrocannabinol on Reinforcement Processing in the Risky Decision-Making of Young Healthy Adults

Research suggests that risky decision-making is sensitive to neuromodulatory influences acting upon corticolimbic circuitry. However, while other evidence attests to effects of delta-9 tetrahydrocannabinol (THC) on the activity of reward pathways, relatively little is known about the possible involvement of cannabinoid activity in risky choice. In this experiment, we examined the effects of a single sublingual 5 mg dose of THC on a test of risky decision-making (requiring choices between simultaneously presented gambles differing in their magnitude of gains, magnitude of losses and the probability with which these outcomes were delivered). Tests of non-normative decision-making involving risk-aversion when deciding between gains and risk-seeking choices when deciding between losses were also included. In all, 15 healthy adults were administered 5 mg THC and placebo in a double-blind, placebo-controlled, within-subject, cross-over design. THC had three principal effects relative to placebo: (i) THC reduced choice of gambles with variable gains and losses, but increased choice of gambles with zero-expected value; (ii) THC reduced participants' attention towards losses when the probability of winning was low (and the probability of losing was high); and (iii) THC speeded participants' responses to gambles with large compared to small potential gains. These results suggest that THC mediates specific motivational processes and the processing of reinforcement cues during risky choice, perhaps reflecting altered CB1 receptor or catecholamine activity within corticolimbic pathways.

Elevated Impulsivity and Impaired Decision-Making in Abstinent Ecstasy (MDMA) Users Compared to Polydrug and Drug-Naïve Controls

Ecstasy (MDMA; 3,4-methylenedioxymethamphetamine) has a well-recognized neurotoxic effect on central serotonergic (5-HT) systems in animals, and there is some evidence of persistent serotonergic dysregulation in human ecstasy users. Serotonin is believed to mediate impulsive behavior and effective decision-making. Thus, the aim of the present study was to investigate impulsive behavior and decision-making in abstinent regular ecstasy users. Three groups were compared: 'ecstasy users' (recreational ecstasy users who reported modest use of illicit drugs other than cannabis), 'polydrug controls' (ecstasy naïve illicit drug users), and 'drug-naïve controls'. All participants completed personal details and general drug history questionnaires, the National Adult Reading Test, Matching Familiar Figures Test (MFF20), a risky decision-making task (RDMT), and the Card Arranging Reward Responsivity Objective Test (CARROT). The groups did not differ on the CARROT measure of responsiveness to financial incentive; however, the ecstasy group displayed significantly elevated MFF20 impulsivity, and showed reduced discrimination between magnitudes of prospective gains and losses when making risky decisions, compared to the 'polydrug' and 'drug-naïve' control groups. These findings may reflect a vulnerability of 5-HT systems in the orbital prefrontal cortex and interconnected corticolimbic circuitry to the cumulative neurotoxic effects of ecstasy and have clinical significance for regular ecstasy users. The combination of elevated impulsivity and impaired use of reinforcement cues in uncertain decision-making may comprise risk factors for continued drug abuse and everyday functioning.

Experimental results using force-feedback cueing in robot-assisted stroke therapy

Stroke is the leading cause of disability among adults in the United States. Behaviors such as learned nonuse hinder hemiplegic stroke survivors from the full use of both arms in activities of daily living. Active force-feedback cues, designed to restrain the use of the less-affected arm, were embedded into a meaningful driving simulation environment to create robot-assisted therapy device, driver's simulation environment for arm therapy (SEAT). The study hypothesized that force-feedback control mode could "motivate" stroke survivors to increase the productive use of their impaired arm throughout a bilateral steering task, by providing motivating feedback and reinforcement cues to reduce the overuse of the less-affected arm. Experimental results demonstrate that the force cues counteracted the tendency of hemiplegic subjects to produce counter-productive torques only during bilateral steering tasks (p < 0.05) that required the movement of their impaired arm in steering directions up and against gravity. Impaired arm activity was quantified in terms of torques due to the measured tangential forces on the split-steering wheel of driver's SEAT during bilateral steering. Results were verified using surface electromyograms recorded from key muscles in the impaired arm.

Hypnotic susceptibility and verbal conditioning.

Abstract 18 Ss highly susceptible to hypnosis and 18 Ss refractory to hypnosis were studied in a verbal conditioning task modeled after the one used by Taffel (1955). Results indicated that the highly susceptible group showed significantly greater conditioning than the low group. Awareness of the reinforcement contingency by S was not related to the learning task nor to hypnotic susceptibility. A measure of S's attitude toward the reinforcement cue during learning showed that the highly susceptible group had a more positive set toward the cue, whereas the low group tended to respond to it in a neutral or negative manner. Results were interpreted in terms of the theoretical nature of hypnotic susceptibility.

Regression in schizophrenics as a function of censure, reinforcement cues, and response habit hierarchies.

Abstract Three types of regressive behavior (temporal, nonspecific, and withdrawal) were experimentally defined. The effects of censure, response hierarchy, and reinforcement upon the three types of regression in good and poor premorbid schizophrenics were tested. Results indicated that censure increased only withdrawal regression in both good and poor premorbid Ss, but that poor premorbid Ss showed more of an immediate response deficit following censure. No differences in response hierarchy were found. Initial nonreinforcement of dominant responses increased both temporal and non-specific regression. The results are discussed in relation to theories which postulate increased drive as an important variable in schizophrenic regression.

Schedules of Reinforcement, Pretraining and Delay-Associated Cues in Runway Performance

The effects of distinctive cues on delayed-reward learning were investigated by pretraining 2 groups of rats in a Skinner box on either a CRF or a FR10 schedule. A light was paired with each bar press. Ss were then run in a straight-alley runway for 16 trials with a 30-sec. delay interval. A light was paired with the delay. A 2 × 3 analysis of variance (2 treatments × 3 blocks of 5 trials) indicated that the FR10 group ran significantly slower. The results were discussed in terms of Amsel's (1958) frustration hypothesis.