Abstract
Previous research has stressed the importance of uncertainty for controlling the speed of learning, and how such control depends on the learner inferring the noise properties of the environment, especially volatility: the speed of change. However, learning rates are jointly determined by the comparison between volatility and a second factor, moment-to-moment stochasticity. Yet much previous research has focused on simplified cases corresponding to estimation of either factor alone. Here, we introduce a learning model, in which both factors are learned simultaneously from experience, and use the model to simulate human and animal data across many seemingly disparate neuroscientific and behavioral phenomena. By considering the full problem of joint estimation, we highlight a set of previously unappreciated issues, arising from the mutual interdependence of inference about volatility and stochasticity. This interdependence complicates and enriches the interpretation of previous results, such as pathological learning in individuals with anxiety and following amygdala damage.
Highlights
Previous research has stressed the importance of uncertainty for controlling the speed of learning, and how such control depends on the learner inferring the noise properties of the environment, especially volatility: the speed of change
Experiments confirm that people adjust their learning rates in the predicted direction[24,25], and this behavior has been captured by a model that estimates stochasticity
From the perspective of a learner inferring volatility and stochasticity, these factors should compete or trade off against one another to best explain experienced noise. This means that any dysfunction or damage that impairs detection of stochasticity, should lead to a compensatory increase in inferred volatility, and vice versa: a classic pattern known in Bayesian inference as a failure of “explaining away.”. We argue that such compensatory tradeoffs may be apparent both in anxiety disorders and following damage to amygdala
Summary
Previous research has stressed the importance of uncertainty for controlling the speed of learning, and how such control depends on the learner inferring the noise properties of the environment, especially volatility: the speed of change. 1234567890():,; Among the successes of computational neuroscience is a level-spanning account of learning and conditioning, which has grounded biological plasticity mechanisms (error-driven updating) in terms of a normative analysis of the problem faced by the organism[1–5] These models recast learning as statistical inference: using experience to estimate the amount of some outcome (e.g., food) expected on average following some cue or action. This work has led to a strong argument that the brain’s mechanisms for tracking uncertainty, and the inference of the noise parameters that govern it, are crucial to healthy and disordered learning[26] These components seem individually well understood, in this article we argue that important insights are revealed by considering in greater detail the full problem facing the learner: simultaneously estimating both volatility and stochasticity during learning. By introducing a model that performs such joint estimation, and studying its behavior in reinforcement learning tasks, we show that because of the interrelationship between these variables, a full account of any of them interacts with the other in consequential ways
Sign-in/Register to view highlights & summary Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
