Analysis of a Procedure for Controlling Discrete Stochastic Processes

Abstract

In this paper an analysis is made of a specific procedure for controlling one independent variable described by a discrete stochastic process. Operational or actual tolerances are specified for the variable, that is, a range of acceptable variation is given. The control procedure involves the making of corrective actions to maintain the process within the actual tolerance. To accomplish this, effective tolerance limits (which are smaller than the actual tolerance limits) are set. Corrective actions are then made whenever the process exceeds the effective tolerance limits. A method for setting the effective tolerance limits to minimize the expected equilibrium cost of control per trial of operation is given. The method uses the assumption that corrective actions can be considered as recurrent events. From the theory of recurrent events, expressions for the probabilities of making corrective actions can be developed for inclusion in the equation which relates the cost measure to tolerance setting. By differentiating the cost measure with respect to the effective tolerance setting, a minimum solution can be obtained. The equations for minimization are developed first for general stochastic processes. The recurrent-event method is then extended to processes not necessarily returned to initial conditions by corrective actions. Specific equations are developed which optimize the control of discrete homogeneous Markov Gaussian processes. An example is given in which the optimal control procedure is analyzed as a function of the parameters of such a process.