An optogenetic analogue of second-order reinforcement in Drosophila.

Christian König,Afshin Khalili,Bertram Gerber,Thomas Niewalda,Shiqiang Gao

doi:10.1098/rsbl.2019.0084

Abstract

In insects, odours are coded by the combinatorial activation of ascending pathways, including their third-order representation in mushroom body Kenyon cells. Kenyon cells also receive intersecting input from ascending and mostly dopaminergic reinforcement pathways. Indeed, in Drosophila, presenting an odour together with activation of the dopaminergic mushroom body input neuron PPL1-01 leads to a weakening of the synapse between Kenyon cells and the approach-promoting mushroom body output neuron MBON-11. As a result of such weakened approach tendencies, flies avoid the shock-predicting odour in a subsequent choice test. Thus, increased activity in PPL1-01 stands for punishment, whereas reduced activity in MBON-11 stands for predicted punishment. Given that punishment-predictors can themselves serve as punishments of second order, we tested whether presenting an odour together with the optogenetic silencing of MBON-11 would lead to learned odour avoidance, and found this to be the case. In turn, the optogenetic activation of MBON-11 together with odour presentation led to learned odour approach. Thus, manipulating activity in MBON-11 can be an analogue of predicted, second-order reinforcement.

Highlights

Animals and humans go to great lengths to obtain rewards, such as food and water, and to avoid punishment, such as bodily damage and pain
It targets premotor circuitry outside the mushroom bodies, and hetero-compartmental regions in the ipsi- and the contralateral mushroom body, and features a homo-compartmental and contralateral feedback loop onto the dopaminergic, punishing PPL1-01 neuron [13,17,25,26]. All of these regions could contribute to reinforcement through manipulation of MBON-11 activity, and we expressly do not draw a conclusion as to which of these regions is involved in these reinforcing effects
One scenario is that silencing MBON-11 lifts inhibition from PPL1-01, promotes PPL1-01 activity and exerts a punishing effect

Summary

Introduction

Animals and humans go to great lengths to obtain rewards, such as food and water, and to avoid punishment, such as bodily damage and pain. Essential to these processes is the learning of cues predictive of such actual or firstorder reinforcement. Predictive cues acquire learned valence but, once predictive relationships are established, can confer learned valence themselves; i.e. they can serve as second-order reinforcement [1 –3]. For example, learning that money can buy food establishes money as a second-order reward. Second-order conditioning may underlie chains of predictions and early anticipatory behaviour in humans and animals. The capacity for second-order conditioning is widely distributed across the animal kingdom, including insects [4,5,6,7], and is implemented in many computational models of associative learning [8]

Methods

Results

Conclusion