Instability of optimal stochastic control systems under parameter variations

Abstract

The deterioration of a linear optimal stochastic control scheme, designed under the assumptions of the certainty-equivalence principle (the optimal filter and controller, determined independently, combine to give a totally optimal system), is investigated when the parameters of the actual system do not coincide with the design values. This linear suboptimal stochastic system is described by a covariance matrix composed of covariances of the estimates of the state variables, the errors in the estimates of the state variables, and the correlation between these errors and the estimates. In particular, this paper is concerned with the covariance matrix resulting from a single state dynamical system and a scalar linear measurement function of both the state variable and the control variable (e.g., accelerometer measurements). A modeling error in the control variable coefficient of the measurement function may induce instability in the stochastic system with either unstable or stable dynamics. Furthermore, the absolute magnitude of the error in the control variable coefficient directly influences system stability, not the relative error. Thus, relatively small errors compared to the design value of this coefficient may be quite important. 4 LTHOUGH there are many studies on divergence of op-£^- tirnal filters, little attention has been given to the effect of modeling inaccuracies on optimal linear stochastic control systems. Here we extend Fitzgerald's1 investigation of Kalman filter divergence to optimal linear stochastic control systems. These systems are designed under the certainty equivalence principle2 which states that if the expected value of a quadratic function of the state and the control variables is to be minimized subject to linear dynamics, the optimal system is composed of an optimal filter in cascade with an optimal controller. This separation is possible because the estimate in the state is uncorrelated with the error in this estimate. If the parameters in the assumed model of the dynamics or the measurement device deviate from the parameters of the actual system, the estimate and the error in the estimate become correlated. The behavior of the system because of the gains based on an inaccurate model is studied by considering the coupled matrix covariance equation composed of the Covariances of the error in the estimate, the estimate, and the estimate with its error. Some of the characteristics of this linear matrix equation are studied through a scalar linear dynamic equation. The errors in system parameters enter into the 2X2 covariance equation in a dimensionless form allowing the following general results to be obtained: 1) The stochastic control system may be unstable when the nonoptimal filter and deterministic control systems individually are stable. 2) Instability occurs only when the error in the parameter exceeds a finite threshold value. 3) If the measurement is a linear function of the control variable as well as the state (e.g., accelerometer measurements) and there are errors in the coefficient of the control, then instability of the total system may occur for both stable and unstable dynamical systems. 4) The filter or control gains are not functions of the coefficient of the control variable in the measurement function. Consequently, Presented as Paper 70-36 at the AIAA 8th Aerospace Sciences