The effect of probability discounting on reward seeking: a three-dimensional perspective.

Yannick-Andrã© Breton,Peter Shizgal,Kent Conover

doi:10.3389/fnbeh.2014.00284

Yannick-Andrã© Breton, Peter Shizgal + Show 1 more

Open Access

https://doi.org/10.3389/fnbeh.2014.00284

Copy DOI

Abstract

Rats will work for electrical stimulation of the medial forebrain bundle. The rewarding effect arises from the volleys of action potentials fired by the stimulation and subsequent spatio-temporal integration of their post-synpatic impact. The proportion of time allocated to self-stimulation depends on the intensity of the rewarding effect as well as on other key determinants of decision-making, such as subjective opportunity costs and reward probability. We have proposed that a 3D model relating time allocation to the intensity and cost of reward can distinguish manipulations acting prior to the output of the spatio-temporal integrator from those acting at or beyond it. Here, we test this proposition by varying reward probability, a variable that influences the computation of payoff in the 3D model downstream from the output of the integrator. On riskless trials, reward was delivered on every occasion that the rat held down the lever for a cumulative duration called the “price,” whereas on risky trials, reward was delivered with probability 0.75 or 0.50. According to the model, the 3D structure relating time allocation to reward intensity and price is shifted leftward along the price axis by reductions in reward probability; the magnitude of the shift estimates the change in subjective probability. The predictions were borne out: reducing reward probability shifted the 3D structure systematically along the price axis while producing only small, inconsistent displacements along the pulse-frequency axis. The results confirm that the model can accurately distinguish manipulations acting at or beyond the spatio-temporal integrator and strengthen the conclusions of previous studies showing similar shifts following dopaminergic manipulations. Subjective and objective reward probabilities appeared indistinguishable over the range of 0.5 ≤ p ≤ 1.0.

Highlights

To forage successfully, an animal must trade off potential benefits, costs, and risks
Price-sensitivity exponent Brain stimulation reward Pulse frequency Firing frequency induced by F Shape parameter of the frequency-response function Firing frequency that produces a reward of half-maximal intensity Position parameter of the frequency-response function Reward-growth exponent Price Subjective reward probability Subjective reward intensity Maximum subjective reward intensity Subjective price Shape parameter of the subjective price function Subjective price at which the payoff from a maximal reward equals the payoff from alternate activities Minimum subjective price Time allocation to working for reward Maximum time allocation Minimum time allocation Parameter of subjective effort-cost function www.frontiersin.org
The Reward-Mountain Model has been used to infer the stage of neural processing at which such manipulations act to alter reward seeking

Summary

Introduction

To forage successfully, an animal must trade off potential benefits, costs, and risks. Benefits may be arrayed in terms of their kind (e.g., food, water, nesting material), amount, and quality (e.g., the concentration of nutrients). Costs include both the expenditure of energy to locate, procure, and handle a prey item and the time required to do so. Risks include the uncertainty that a given action will produce its intended result and the likelihood of encountering a predator. An influential account of foraging (Charnov, 1976) implicitly equips the animal with computational machinery that boils down multiple determinants so as to represent each available course of action by a single, continuously updated quantity, expressed in a common currency

Methods

Results

Conclusion