Machine Learning for Dynamic Incentive Problems

Abstract

We propose a generic computational framework for solving large-scale infinite-horizon, discrete-time dynamic incentive problems with persistent hidden types. First, we combine set-valued dynamic programming techniques with unsupervised machine learning to determine irregularly-shaped feasible sets. Second, we generate training data from these pre-computed feasible sets to recursively solve the dynamic incentive problem by applying supervised and reinforcement machine learning. Third, to speed up the time-to-solution by orders of magnitude, we propose a generic parallelization scheme for dynamic incentive problems that allows for an efficient use of contemporary high-performance computing hardware. This combination enables us to analyze models of complexity that were previously considered to be intractable. To demonstrate the broad applicability of our method, we solve two very different types of dynamic incentive models: first, an adverse selection model with many discrete types; second, a problem with infinitely many types and multiple state variables.