Solving nonlinear dynamic models by iterative dynamic programming

Abstract

The paper presents a discrete-time dynamic programming algorithm that is suitable to track nonlinearities in intertemporal optimization problems. In contrast to using linearization methods for solving intertemporal models, the proposed algorithm operates globally by iteratively computing the value function and the controls in feedback form on a grid. A conjecture of how the trajectories might behave is analytically obtained by letting the discount rate approach infinity. The dynamic found serves as a useful device for computing the trajectories for finite discount rates employing the algorithm. Commencing with a large step and grid size, and then pursuing time step and grid refinements allows for the replication of the nonlinear dynamics for various finite discount rates. As the time step and grid size shrink, the discretization errors vanish. The algorithm is applied to three economic examples. Two examples are of deterministic type; the third is stochastic. In the deterministic cases limit cycles are detected.