Abstract
We study Markov decision processes with Borel state spaces under quasi-hyperbolic discounting. This type of discounting nicely models human behaviour, which is time-inconsistent in the long run. The decision maker has preferences changing in time. Therefore, the standard approach based on the Bellman optimality principle fails. Within a dynamic game-theoretic framework, we prove the existence of randomised stationary Markov perfect equilibria for a large class of Markov decision processes with transitions having a density function. We also show that randomisation can be restricted to two actions in every state of the process. Moreover, we prove that under some conditions, this equilibrium can be replaced by a deterministic one. For models with countable state spaces, we establish the existence of deterministic Markov perfect equilibria. Many examples are given to illustrate our results, including a portfolio selection model with quasi-hyperbolic discounting.
Highlights
The discounted utility approach in dynamic decision making has been used since the beginning of modern economic theory; see e.g. Samuelson [59]
The strategy for the decision maker built from a Markov perfect equilibrium in the game is time-consistent, that is, no self has an incentive to change his best response to equilibrium strategies of the following selves
We have studied a fairly general class of time-inconsistent Markov decision processes with a Borel state space
Summary
The discounted utility approach in dynamic decision making has been used since the beginning of modern economic theory; see e.g. Samuelson [59]. Alj and Haurie [4] extended the finite state space model of Shapley to quasi-hyperbolic discounting They used an intergenerational dynamic game formulation of Phelps and Pollak [55] and proved that any finite horizon game has an equilibrium in Markovian strategies and each infinite horizon game has a stationary Markov perfect equilibrium. As already mentioned time-inconsistent preferences in various control models were recently studied by Björk and Murgoci [15], Björk et al [14], Christensen and Lindensjö [20], these papers, in contrast to our present work and works of Alj and Haurie [4], Jaskiewicz and Nowak [35], Nowak [51], examine neither stationary Markov perfect equilibria nor fixed points of best-response mappings. We consider Markov decision processes with a Borel state space and quasi-hyperbolic discounting and the Markov perfect equilibrium as a basic solution concept.
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Free) 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a call
