Nonstationary Transition Probabilities Research Articles

Causative matrix technique for deriving interim period transition probabilities in nonstationary markov process

AbstractThe Markov assumption that transition probabilities are assumed to be constant over entire periods has been applied in economic and social structures, for example, in the analysis of income and wage distributions. In many cases, however, nonstationary transition probabilities exist over different periods. Based on causative matrix technique, this study shows a binomial approximation for obtaining nonstationary interim transition probabilities under undisturbance when the first and the last transition matrices are known.

Nonstationary Markov Modeling: An Application to Wage-Influenced Industrial Relocation

A recently proposed technique for estimating nonstationary transition probabilities for a Markov process is developed for empirical implementation. Its use is then demonstrated by analyzing the process of industrial relocation for the U.S. apparel industries. Transition probabilities for twenty-one apparel industries and four time periods are estimated with aggregate frequency data and an embedded wage-adjustment model. The stationarity assumption of no wage-rate effect on the transition probabilities cannot be rejected.

An Empirical Analysis of the Optimal Advertising Policy

This paper determines an optimal policy for investment in advertising for a firm that wishes to maximize its discounted profits. To that end, an integrated approach consisting of model formulation, empirical investigation, and optimization is carried out. A model of market share response to advertising is formulated as a first-order Markov process, with nonstationary transition probabilities. These probabilities are assumed to be a function of the advertising goodwill accumulated by the firm and its competitors. The model as specified is nonlinear in its parameters, and nonlinear regression techniques are applied to estimate them. It is shown that this nonlinear form offers, via likelihood ratio tests, a unique opportunity for testing the model, and in a resulting empirical test, the model is found to be consistent with the data. Given these empirical findings, an optimal advertising policy is derived by the use of optimal control theory. The managerial implications of the recommended multi-period policy are examined, and the policy's sensitivity to managerial inputs and economic conditions is analyzed and illustrated.

First passage game

AbstractAppealing to the theory of stochastic games, a two‐person, zero‐sum first passage game, which may be viewed as a generalization of the first passage decision problem, is developed. In the first passage game, the players have stationary optional strategies and the values are unique and these can be computed using an algorithm for terminating stochastic games. It is also shown that the solution of a recurrence game is closely related to that of the first passage game. Finally, it is shown that a finite step stochastic game with nonstationary transition probabilities and payoffs can be converted to a first passage game whose solution yields a solution of the original finite step game. The first passage game so obtained has stationary transition probabilities and payoffs. Because of its special structure, the solution method reduces to a dynamic programming recursion in the context of games.

Estimation of nonstationary transition probabilities for agricultural firm size projection

Declining numbers of cotton warehouses in Oklahoma is of concern to the cotton industry. Increased competition at home and abroad and synthetic fibers are hypothesized to partially account for the decline.

Prediction and smoothing for partially observed Markov chains

Consider a continuous time Markov chain with stationary transition probabilities. A function of the state is observed. A regular conditional probability distribution for the trajectory of the chain, given observations up to time t, is obtained. This distribution also corresponds to a Markov chain, but the conditional chain has nonstationary transition probabilities. In particular, computation of the conditional distribution of the state at time s is discussed. For s > t, we have prediction (extrapolation), while s < t corresponds to smoothing (interpolation). Equations for the conditional state distribution are given on matrix form and as recursive differential equations with varying s or t. These differential equations are closely related to Kolmogorov's forward and backward equations. Markov chains with one observed and one unobserved component are treated as a special case. In an example, the conditional distribution of the change-point is derived for a Poisson process with a changing intensity, given observations of the Poisson process.

The Estimation of Nonstationary Transition Probabilities

The estimation of an underlying Markov process of constant transition probabilities which explains aggregated market share movements over time has received much attention from theorists and applied researchers. Summaries of the work to date are given in [Dent, Warren T., Richard Ballintine. 1971. A review of the estimation of transition probabilities in Markov chains. Australian J. Agricultural Econom. 15 and Lee, T. C., G. G. Judge, A. Zellner. 1970. Estimating the Parameters of the Markov Probability Model from Aggregate Time Series Data. North-Holland, Amsterdam.]. The problem of estimating a nonstationary process however has not been solved, although Hallberg [Hallberg, M. C. 1969. Projecting the size distribution of agricultural firms—an application to a Markov process with non-stationary transition probabilities. J. Farm Econom. 51.] has looked at the problem in the case of micro data, that is, when movements between periods are observable. Harary, Lipstein, and Styan [Harary, F., B. Lipstein, G. P. H. Styan. 1970. A matrix approach to nonstationary chains. Oper. Res. 18.] have investigated nonstationary chain properties in the case of aggregated data but have not looked at estimation. We present an attempt to formalize Harary, Lipstein, and Styan's model and to develop appropriate solution techniques for estimation of its unknown parameters. An iterative quadratic programming procedure is suggested.

The convergence of Markov chains with nonstationary transition probabilities and constant causative matrix

The convergence of Markov chains with nonstationary transition probabilities and constant causative matrix

State Substitution and the Estimation of Transition Probabilities

Variants of Markovian decision models are many. Of those concerned with the estimation of transition probabilities, few deal with the case of limited information in which only data on market shares are available. Where such investigations have been undertaken, the assumption of constant transition probabilities has been insurmountable. This paper offers a simple heuristic technique to estimate nonstationary transition probabilities, especially in the case where fluctuations in market shares are felt to reflect the effects of secondary considerations external to the transition probabilities as well as effects of the probabilities themselves. Dynamic and quadratic programming are the suggested tools of analysis.

A Note on Memoryless Rules for Controlling Sequential Control Processes

A Note on Memoryless Rules for Controlling Sequential Control Processes