History-dependent Policies Research Articles

Volatile Stock Markets: Equilibrium Computation and Policy Analysis

This paper studies the benefits of leaning-against-the-wind policies in the presence of inefficient stock markets. It incorporates observed levels of volatility in stock returns into a standard general equilibrium model and finds large welfare gains from stabilizing policies. Stock prices in this model aggregate information about fundamentals which is dispersed in the economy but also reflect noise stemming from correlated distortions in beliefs. Dispersion of information allows arbitrarily small distortions in beliefs to generate large volatility in returns and renders arbitrage infeasible. The government cannot observe whether a given stock price movement originates from information or noise. As a normative result, stock return stabilizing policies lead to a higher level of consumption: a fall in the risk premium lowers the marginal product of capital and raises the capital stock and production. History-dependent policies may improve the information content of prices and result in even higher welfare gains. To solve the model, this paper develops a novel solution method for nonlinear models with dispersed information which can be applied to a large class of dynamic general equilibrium models.

Average optimality inequality for continuous-time Markov decision processes in Polish spaces

In this paper, we study the average optimality for continuous-time controlled jump Markov processes in general state and action spaces. The criterion to be minimized is the average expected costs. Both the transition rates and the cost rates are allowed to be unbounded. We propose another set of conditions under which we first establish one average optimality inequality by using the well-known “vanishing discounting factor approach”. Then, when the cost (or reward) rates are nonnegative (or nonpositive), from the average optimality inequality we prove the existence of an average optimal stationary policy in all randomized history dependent policies by using the Dynkin formula and the Tauberian theorem. Finally, when the cost (or reward) rates have neither upper nor lower bounds, we also prove the existence of an average optimal policy in all (deterministic) stationary policies by constructing a “new” cost (or reward) rate.

Another set of conditions for Markov decision processes with average sample-path costs

This paper deals with discrete-time Markov decision processes with average sample-path costs (ASPC) in Borel spaces. The costs may have neither upper nor lower bounds. We propose new conditions for the existence of ε-ASPC-optimal (deterministic) stationary policies in the class of all randomized history-dependent policies. Our conditions are weaker than those in the previous literature. Moreover, some sufficient conditions for the existence of ASPC optimal stationary policies are imposed on the primitive data of the model. In particular, the stochastic monotonicity condition in this paper has first been used to study the ASPC criterion. Also, the approach provided here is slightly different from the “optimality equation approach” widely used in the previous literature. On the other hand, under mild assumptions we show that average expected cost optimality and ASPC-optimality are equivalent. Finally, we use a controlled queueing system to illustrate our results.

Blackwell optimality in the class of all policies in Markov decision chains with a Borel state space and unbounded rewards

This paper is the second part of our study of Blackwell optimal policies in Markov decision chains with a Borel state space and unbounded rewards. We prove that a stationary policy is Blackwell optimal in the class of all history-dependent policies if it is Blackwell optimal in the class of stationary policies.

Semi-Markov and Jump Markov Controlled Models: Average Cost Criterion

Semi-Markov and Jump Markov Controlled Models: Average Cost Criterion