Abstract
Recent findings suggest that acetylcholine mediates uncertainty-seeking behaviors through its projection to dopamine neurons — another neuromodulatory system known for its major role in reinforcement learning and decision-making. In this paper, we propose a leaky-integrate-and-fire model of this mechanism. It implements a softmax-like selection with an uncertainty bonus by a cholinergic drive to dopaminergic neurons, which in turn influence synaptic currents of downstream neurons. The model is able to reproduce experimental data in two decision-making tasks. It also predicts that: (i) in the absence of cholinergic input, dopaminergic activity would not correlate with uncertainty, and that (ii) the adaptive advantage brought by the implemented uncertainty-seeking mechanism is most useful when sources of reward are not highly uncertain. Moreover, this modeling work allows us to propose novel experiments which might shed new light on the role of acetylcholine in both random and directed exploration. Overall, this study contributes to a more comprehensive understanding of the role of the cholinergic system and, in particular, its involvement in decision-making.
Highlights
Animals constantly face uncertainty due to noisy and incomplete information about the environment
The largest group of dopaminergic neurons is found in the ventral tegmental area (VTA) (Scatton et al, 1980)
The dopaminergic system seems to implement a series of mechanisms that reinforce and favor stimuli and actions that have been rewarding in the past, or that may be of interest in the future
Summary
Animals constantly face uncertainty due to noisy and incomplete information about the environment. Mice lacking the nicotinic acetylcholine receptors on the dopaminergic neurons in VTA showed less uncertainty-seeking behaviors and their decisions rather followed the standard softmax rule. Decision-making processes are generally modeled using competition mechanisms (Rumelhart & Zipser, 1985; Carpenter & Grossberg, 1988) Such mechanisms can constitute a neural implementation of the softmax rule. We propose a new version of this model using leaky-integrate-and-fire neurons and an additional uncertainty bonus We use this model, in comparison with three alternative models, to verify a set of hypotheses about how cholinergic projections to dopaminergic neurons in VTA mediate uncertainty-seeking. We perform additional simulations to assess the interest of such a mechanism for animals foraging in volatile environments These simulations suggest that ACh affects behavior by translating uncertainty into a source of motivation driving exploratory behaviors
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
