Policy Function Iteration Research Articles

Applications of Markov chain approximation methods to optimal control problems in economics

In this paper we explore some benefits of using the finite-state Markov chain approximation (MCA) method of Kushner and Dupuis (2001) to solve continuous-time optimal control problems in economics. We first show that the implicit finite-difference scheme of Achdou et al. (2022) amounts to a limiting form of the MCA method for a certain choice of approximating chains and policy function iteration for the resulting system of equations. We then illustrate that relative to the implicit finite-difference approach, using variations of modified policy function iteration to solve income fluctuation problems both with and without discrete choices can lead to an increase in the speed of convergence of more than an order of magnitude. Finally, we provide several consistent chain constructions for stationary portfolio problems with correlated state variables, and illustrate the flexibility of the MCA approach by using it to construct and compare two simple solution methods for a general equilibrium model with financial frictions.

Optimal investment strategies for pension funds with regulation-conform dynamic pension payment management in the absence of guarantees

In this article we consider the post-retirement phase optimization problem for a specific pension product in Germany that comes without guarantees. The continuous-time optimization problem is defined consisting of two specialties: first, we have a product-specific pension adjustment mechanism based on a certain capital coverage ratio which stipulates compulsory pension adjustments if the pension fund is underfunded or significantly overfunded. Second, due to the retiree’s fear of and aversion against pension reductions, we introduce a total wealth distribution to an investment portfolio and a buffer portfolio to lower the probability of future potential pension shortenings. The target functional in the optimization, that is to be maximized, is the client’s expected accumulated utility from the stochastic future pension cash flows. The optimization outcome is the optimal investment strategy in the proposed model. Due to the inherent complexity of the continuous-time framework, the discrete-time version of the optimization problem is considered and solved via the Bellman principle. In addition, for computational reasons, a policy function iteration algorithm is introduced to find a stationary solution to the problem in a computationally efficient and elegant fashion. A numerical case study on optimization and simulation completes the work with highlighting the benefits of the proposed model.

Optimal Investment Strategies for Pension Funds with Regulation-Conform Dynamic Pension Payment Management in the Absence of Guarantees

In this article we consider the post-retirement phase optimization problem for a specific pension product in Germany that comes without guarantees. The continuous-time optimization problem is defined consisting of two specialties: first, we have a product-specific pension adjustment mechanism based on a certain capital coverage ratio which stipulates compulsory pension adjustments if the pension fund is underfunded or significantly overfunded, and second, due to the retiree's fear of and aversion against pension reductions, we introduce a total wealth distribution to an investment portfolio and a buffer portfolio to lower the probability of future potential pension shortenings. Due to the inherent complexity of the continuous-time framework, the discrete-time version of the optimization problem is considered and solved via the Bellman principle. In addition, for computational reasons, a policy function iteration algorithm is introduced to find a stationary solution to the problem in a computationally efficient and elegant fashion. A numerical case study on optimization and simulation completes the work with highlighting the benefits of the proposed model.

Inference in dynamic discrete choice problems under local misspecification

Single‐agent dynamic discrete choice models are typically estimated using heavily parametrized econometric frameworks, making them susceptible to model misspecification. This paper investigates how misspecification affects the results of inference in these models. Specifically, we consider a local misspecification framework in which specification errors are assumed to vanish at an arbitrary and unknown rate with the sample size. Relative to global misspecification, the local misspecification analysis has two important advantages. First, it yields tractable and general results. Second, it allows us to focus on parameters with structural interpretation, instead of “pseudo‐true” parameters.We consider a general class of two‐step estimators based on the K‐stage sequential policy function iteration algorithm, where K denotes the number of iterations employed in the estimation. This class includes Hotz and Miller (1993)'s conditional choice probability estimator, Aguirregabiria and Mira (2002)'s pseudo‐likelihood estimator, and Pesendorfer and Schmidt‐Dengler (2008)'s asymptotic least squares estimator.We show that local misspecification can affect the asymptotic distribution and even the rate of convergence of these estimators. In principle, one might expect that the effect of the local misspecification could change with the number of iterations K. One of our main findings is that this is not the case, that is, the effect of local misspecification is invariant to K. In practice, this means that researchers cannot eliminate or even alleviate problems of model misspecification by choosing K. Single‐agent dynamic discrete choice models estimation inference misspecification local misspecification C13 C61 C73

The Market Resources Method for Solving Dynamic Optimization Problems

We introduce the market resources method (MRM) for solving dynamic optimization problems. MRM extends Carroll’s (2006) endogenous grid point method (EGM) for problems with more than one control variable using policy function iteration. The MRM algorithm is simple to implement and provides advantages in terms of speed and accuracy over Howard’s policy improvement algorithm. Codes are available.

Solving the income fluctuation problem with unbounded rewards

This paper studies the income fluctuation problem without imposing bounds on utility, assets, income or consumption. We prove that the Coleman operator is a contraction mapping over the natural class of candidate consumption policies when endowed with a metric that evaluates consumption differences in terms of marginal utility. We show that this metric is complete, and that the fixed point of the operator coincides with the unique optimal policy. As a consequence, even in this unbounded setting, policy function iteration always converges to the optimal policy at a geometric rate.

The zero lower bound: frequency, duration, and numerical convergence

AbstractWhen monetary policy faces a zero lower bound (ZLB) constraint on the nominal interest rate, a minimum state variable (MSV) solution may not exist even if the Taylor principle holds when the ZLB does not bind. This paper shows there is a clear tradeoff between the expected frequency and average duration of ZLB events along the boundary of the convergence region – the region of the parameter space where our policy function iteration algorithm converges to an MSV solution. We show this tradeoff with two alternative stochastic processes: one where monetary policy follows a 2-state Markov chain, which exogenously governs whether the ZLB binds, and the other where ZLB events are endogenous due to discount factor or technology shocks. We also show that small changes in the parameters of the stochastic processes cause meaningful differences in the decision rules and where the ZLB binds in the state space.

The Zero Lower Bound: Frequency, Duration, and Determinacy

When monetary policy faces a zero lower bound (ZLB) constraint on the nominal interest rate, a minimum state variable (MSV) solution may not exist even if the Taylor principle holds when the ZLB does not bind. This paper shows there is a clear tradeoff between the expected frequency and average duration of ZLB events along the boundary of the convergence region --- the region of the parameter space where our policy function iteration algorithm converges to an MSV solution. We show this tradeoff with two alternative stochastic processes: one where monetary policy follows a 2-state Markov chain, which exogenously governs whether the ZLB binds, and the other where ZLB events are endogenous due to discount factor or technology shocks. We also show that small changes in the parameters of the stochastic processes cause meaningful differences in the decision rules and where the ZLB binds in the state space.

Accuracy, Speed and Robustness of Policy Function Iteration

Policy function iteration methods for solving and analyzing dynamic stochastic general equilibrium models are powerful from a theoretical and computational perspective. Despite obvious theoretical appeal, significant startup costs and a reliance on grid-based methods have limited the use of policy function iteration as a solution algorithm. We reduce these costs by providing a user-friendly suite of MATLAB functions that introduce multi-core processing and Fortran via MATLAB's executable function. Within the class of policy function iteration methods, we advocate using time iteration with linear interpolation. We examine a canonical real business cycle model and a new Keynesian model that features regime switching in policy parameters, Epstein---Zin preferences, and monetary policy that occasionally hits the zero-lower bound on the nominal interest rate to highlight the attractiveness of our methodology. We compare our advocated approach to other familiar iteration and approximation methods, highlighting the tradeoffs between accuracy, speed and robustness.

Accuracy, Speed and Robustness of Policy Function Iteration

Policy function iteration methods for solving and analyzing dynamic, stochastic general equilibrium models are powerful from both a theoretical and computational perspective. Despite obvious theoretical appeal, significant startup costs and a reliance on a grid-based method have limited the use of policy function iteration as a solution algorithm. We reduce these costs by providing a user-friendly suite of MATLAB functions that introduce multi-core processing and Fortran via MATLAB's executable function. We demonstrate why policy function iteration is particularly useful in solving models with regime-dependent parameters, recursive preferences, and binding constraints. We examine a canonical real business cycle model and a new Keynesian model that features regime switching in policy parameters, Epstein-Zin preferences, and monetary policy that occasionally hits the zero-lower bound to highlight the attractiveness of our methodology. We compare our advocated approach to other familiar computational methods, highlighting the tradeoffs between accuracy and speed.

Dynamic Product Positioning in Differentiated Product Markets: The Effect of Fees for Musical Performance Rights on the Commercial Radio Industry

This article investigates how product positioning decisions in the radio industry would be affected if, as has recently been proposed, music stations have to pay fees for musical performance rights. A rich dynamic model, which captures many features of the industry such as multi-station ownership, economies of scope and both vertical and horizontal product differentiation, is formulated, estimated and (approximately) re-solved for different levels of fee, by applying the method of parametric policy function iteration in the context of a dynamic game. The estimated model predicts that fees could have substantial effects on product positioning. For example, if music stations have to pay 10% of their revenues as performance fees – which would be consistent with the fees currently paid by cable and satellite providers – the number of music stations would fall by 11% over a 15 year period. The article also considers how industry characteristics such as heterogeneous listener tastes, multi-station ownership and substantial repositioning costs affect the size and speed of adjustment.

Solving the Stochastic Growth Model by Policy-Function Iteration

This article describes a computer algorithm that solves the stochastic growth model by iterating on a fixed-point equation in the decision rule determining consumption as a function of the state variables. This algorithm does not discretize the state space, but rather it preserves the continuous domain of the capital stock and the productivity shock. The main advantage of this algorithm is that it is based on a Euler equation and thus it has a straightforward generalization to dynamic economies that cannot be solved by a central planner, such as a non-Pareto optimal competitive economy.

Solving the Stochastic Growth Model by Policy-Function Iteration

Solving the Stochastic Growth Model by Policy-Function Iteration