Long-run Optimality Research Articles

Nash Q-learning agents in Hotelling’s model: Reestablishing equilibrium

This paper examines adaptive agents’ behavior in a stochastic dynamic version of the Hotelling’s location model. We conduct an agent-based numerical simulation under the Hotelling’s setting with two agents who use the Nash Q-learning mechanism for adaptation. This allows exploring what alternations this technique brings compared to the original analytic solution of the famous static game-theoretic model with strong assumptions imposed on players. We discover that under the Nash Q-learning and quadratic consumer cost function, agents with high enough valuation of future profits learn behavior similar to aggressive market strategy. Both agents make similar products and lead a price war to eliminate their opponent from the market. This behavior closely resembles the Principle of Minimum Differentiation from Hotelling’s original paper with linear consumer costs. However, the quadratic consumer cost function would otherwise result in the maximum differentiation of production in the original model. Thus, the Principle of Minimum Differentiation can be justified based on repeated interactions of the agents and long-run optimization.

Optimal Long-Run Imperfect Maintenance With Asymptotic Virtual Age

We consider imperfect, periodic preventive maintenance of items with increasing failure rates, and minimal repairs in-between. The imperfect maintenance is performed in accordance with a Kijima II virtual age model. The obtained deterministic asymptotic value of virtual age defines the starting age of an item at the beginning of each cycle in the formulated and analyzed long-run optimization problem. A similar approach is used for the corresponding age-based preventive maintenance, where the starting age of the renewal-type cycle is already a random variable. Some meaningful examples are considered.

Long-Run Optimization Horizon, Durable Consumption Growth and Estimation of Risk Aversion

We present a durable consumption-based asset pricing model with Epstein-Zin preferences and the pricing kernel accommodating the long-run consumption risk. Consumption growth includes a small predictable component as in Bansal and Yaron (2004). The model is estimated with simple econometric techniques once we linearize it around a special case of the elasticity of inter- and intra-temporal substitution equal to one. After including the long-run consumption growth, the estimates of the model parameters become much more realistic and the fit closer to the data than for the case of the contemporaneous consumption growth. For example, the estimate of risk aversion falls from around 200 to 10, and the R2 increases from around 30% to 60%.

Natural carbon capture and storage (NCCS): Forests, land use and carbon accounting

The use of forests as natural carbon capture and storage sinks is considered by introducing carbon sequestration benefits’ accounting in a multi-vintage partial equilibrium land-use model, under different carbon price scenarios. The consequences to timber and land markets and to the profile of the carbon sequestration time path are examined in the short-run, long-run, and transition. Following IPCC, three carbon accounting methods are considered: the carbon flow, the ton-year crediting and the average storage. A full proof of long-run optimality of steady-state forest is provided. Numerical simulations are performed and results discussed illustrating the setup's potential.

The Role of Forests as Carbon Sinks: Land-Use and Carbon Accounting

The use of forests as carbon sinks is examined by introducing carbon sequestration benefits’ accounting in a multi-vintage land allocation model. Following the IPCC, three carbon accounting methods are considered. We compare the results in each case with those without carbon sequestration, as well as the performances of the ton-year and the average methods (second-best) to the carbon flow (first-best) concerning optimal land allocation between forestry and alternative uses, total carbon sequestered, timber production and social welfare. A full proof of long-run optimality of steady state forest is provided. Numerical simulations are performed and results discussed illustrating the setup’s potential.

Eliciting Risk and Time Preferences

We design experiments to jointly elicit risk and time preferences for the adult Danish population. Since subjects are generally risk averse, we find that joint elicitation provides estimates of discount rates that are significantly lower than those found in previous studies and more in line with what would be considered as a priori reasonable rates. The statistical specification relies on a theoretical framework that involves a latent trade-off between long-run optimization and short-run temptation. Estimation of this specification is undertaken using structural, maximum likelihood methods. Our main results based on exponential discounting are robust to alternative specifications such as hyperbolic discounting. These results have direct implications for attempts to elicit time preferences, as well as debates over the appropriate domain of the utility function when characterizing risk aversion and time consistency.

Stochastic optimal policies when the discount rate vanishes

Dutta [1991. What do discounted optima converge to? A theory of discount rate asymptotics in economic models. Journal of Economic Theory 55, 64–94] showed that long-run optimality of the limit of discounted optima when the discount rate vanishes is implied by a certain bound on the value function of the optimal program. We introduce a new method to verify this bound using coupling techniques.

Dynamic Non-Bayesian Decision Making

The model of a non-Bayesian agent who faces a repeated game with incomplete information against Nature is an appropriate tool for modeling general agent-environment interactions. In such a model the environment state (controlled by Nature) may change arbitrarily, and the feedback/reward function is initially unknown. The agent is not Bayesian, that is he does not form a prior probability neither on the state selection strategy of Nature, nor on his reward function. A policy for the agent is a function which assigns an action to every history of observations and actions. Two basic feedback structures are considered. In one of them -- the perfect monitoring case -- the agent is able to observe the previous environment state as part of his feedback, while in the other -- the imperfect monitoring case -- all that is available to the agent is the reward obtained. Both of these settings refer to partially observable processes, where the current environment state is unknown. Our main result refers to the competitive ratio criterion in the perfect monitoring case. We prove the existence of an efficient stochastic policy that ensures that the competitive ratio is obtained at almost all stages with an arbitrarily high probability, where efficiency is measured in terms of rate of convergence. It is further shown that such an optimal policy does not exist in the imperfect monitoring case. Moreover, it is proved that in the perfect monitoring case there does not exist a deterministic policy that satisfies our long run optimality criterion. In addition, we discuss the maxmin criterion and prove that a deterministic efficient optimal strategy does exist in the imperfect monitoring case under this criterion. Finally we show that our approach to long-run optimality can be viewed as qualitative, which distinguishes it from previous work in this area.

Dynamic Non Bayesian Decision Making

The model of a non-Bayesian agent who faces a repeated game with incomplete informationagainst Nature is an appropriate tool for modeling general agent- environment interactions. In such a modelthe environment state (controlled by Nature) may change arbitrarily and the reward function is initially unknown. The agent is non-bayesian, that is he does not form a prior probability neither on the state selection strategy of Nature, nor on his reward function. Two basic feedback structure are considered. In one of them- the perfect mopnitoring case- the agent is able to observe the previous environment state as part of his feedback, while in the other - the imperfect monitoring case- all that is available to the agent is the reward obtained. Both of these setting refer to partially observable processes, where the current environment state is unknown. Our main result refers to the competitive ratio criterion in the perfect monitoring case; We prove the existence of an efficient stochastic policy which ensures that the competitive ratio is obtained at almost all stages with an arbitrary high probability, where efficiency is measured in term of rate of convergence. It is further shown that such an optimal strategy does not exist in the imperfect monitoring case. Moreover, it is proved that in the perfect monitoring case there does not exist a deterministic policy that satisfy our long run optimality criterion. In addition we discuss the maxmin criterion and prove that a deterministic efficient optimal strategy does exist in the imperfect monitoring case under this criterion. Finally we show that our approach to long-run optimality can be vied as qualitative, which distinguishes it from previous work in this area.

Value of information on crop response function to soil salinity in a farm-level optimization model

The study investigates the value of additional information on the response function to soil salinity of a given crop (potatoes), with regard to a stochastic long-run optimization model for utilization of saline water in a single-farm framework. The analysis provides a conceptual and methodological framework for investigating the expected value of sample information ( evsi), as well as an efficient tool for empirical application. Although a few approximations have been used, the results provide an estimate of evsi and indicate the need for additional information.

On designing economic agents that behave like human agents

This paper explores the idea of constructing theoretical economic agents that behave like actual human agents and using them in neoclassical economic models. It does this in a repeated-choice setting by postulating “artificial agents” who use a learning algorithm calibrated against human learning data from psychological experiments. The resulting calibrated algorithm appears to replicate human learning behavior to a high degree and reproduces several “stylized facts” of learning. It can therefore be used to replace the idealized, perfectly rational agents in appropriate neoclassical models with “calibrated agents” that represent actual human behavior. The paper discusses the possibilities of using the algorithm to represent human learning in normal-form stage games and in more general neoclassical models in economics. It explores the likelihood of convergence to long-run optimality and to Nash behavior, and the “characteristic learning time” implicit in human adaptation in the economy.

Optimal pricing against a simple learning rule

The paper studies a repeated buyer-seller relationship under asymmetric information. Instead of assuming game-theoretic rationality, it is assumed that the buyer uses a nonstrategic learning rule to update his beliefs. The seller's optimal pricing policies against this rule are analyzed. It is shown that screening behavior emerges as a result of the seller's long-run optimization problem, although this is not a Nash equilibrium of the one-shot game. Moreover, despite its apparent naiveté the simple rule is optimal for the buyer in the sense that the seller cannot extract more from the buyer than the value of the stream of goods delivered.

Ex post tests for short-and long-run optimization

The purpose of this paper is to develop tests of long-run equilibrium models where the long-run is defined as the state where the observed levels of output and capital are consistent with optimizing behavior. We use the implied relationship between a restricted translog cost function and derived demand equations to provide a series of nested tests which can be interpreted as ex post tests for short- and long-run optimization behavior. A set of parameter restrictions on the translog system are specified for testing whether observable levels of capital and/or output of a firm correspond to the long-run optimal level. Our tests of the validity of long-run equilibrium specifications are based on data for 28 German industries for the period 1960-1981. For most of the industries the set of restrictions implied by long-run equilibrium had to be rejected.

Effluent regulation and long-run optimality

The long-run efficiency properties of regulatory instruments are examined in a multiple-input framework. The effluent tax and tradeable permit are shown to be efficient with free entry and exit of small firms. The across-the-board effluent standard results in excessive entry and excessive industry pollution.

A Minimal Role of Government in Achieving Optimal Growth

Paul Samuelson [5] [6] and Peter Diamond [2] have developed growth models in which the attempt by each economic unit to maximize its utility does not lead to a social optimum. It would be possible for all generations to consume more per capita, if economic growth were directed by an omniscient social planner. Their models are free from the usual sources of inefficiency encountered in price theory. They explicitly assume perfect competition, perfect foresight and no external economies or diseconomies. The purpose of this paper is to introduce government institutions which may have little foresight, which will not engage in central planning or use direct controls but whose existence will guarantee that the equilibrium solution will be optimal. I am interested in determining the minimal amount of government which is sufficient to produce long-run optimality. Our concern is with the Samuelson-Diamond model whose basic features may be briefly outlined. Two generations are alive at any time: a younger generation which works, and a retired generation. Population born at time t is (1 + n)t; and the participation rate is 100 per cent. of the younger generation. There are (1+ n)tl1 retired people who were born during the preceding period. The periods are of equal, but arbitrary, length. Savings of a worker are designed to maximize his lifetime utility; and his decision involves the substitution between future and present consumption, subject to a constraint. Let c1(t) be consumption of a worker during his youth and c2(t +1) his consumption during retirement one period later. Utility depends upon [c1(t), c2(t + 1)], the pattern of lifetime consumption. As an example, Diamond selected a utility function for the individual: