Discrete-time Stochastic Control Systems Research Articles

New method for optimal control and filtering of weakly coupled linear discrete stochastic systems

The algebraic regulator and filter Riccati equations of weakly coupled discrete-time stochastic linear control systems are completely and exactly decomposed into reduced-order continuous-time algebraic Riccati equations corresponding to the subsystems. That is, the exact solution of the global discrete algebraic Riccati equation is found in terms of the reduced-order subsystem nonsymmetric continuous-time algebraic Riccati equations. In addition, the optimal global Kaiman filter is decomposed into local optimal filters both driven by the system measurements and the system optimal control inputs. As a result, the optimal linear-quadratic Gaussian control problem for weakly coupled linear discrete systems takes decomposition and parallelism between subsystem filters and controllers.

Maximum principles for discrete-time nonlinear stochastic control systems

In this paper we treat discrete-time stochastic control systems. Using corresponding results for systems, which are linear with respect to the state variables, we derive under convexity assumptions optimality conditions in form of maximum principles

A hierarchical multiple model adaptive control of discrete-time stochastic systems for sensor and actuator uncertainties

A hierarchical multiple model adaptive control (MMAC) is described for discrete-time stochastic systems with unknown sensor and actuator parameters, where the decentralized structure consists of a central processor and of m local processors which do not communicate between each other. A major assumption in this study is that the central and any local stations have different knowledge of the hypotheses on the unknown parameters. This leads to a flexible design algorithm for passively adaptive control strategies. Furthermore, the coordinator algorithm in evaluating the global a posteriori probability is relatively simple to implement. The result is applied to the design problem of an instrument failure detection and identification (FDI) system.

Infinite horizon quadratic optimal control of a class of nonlinear stochastic systems

The author considers a class of discrete-time nonlinear stochastic control systems whose nonlinearities are described by statistical means. By introducing the proper mean-square stabilizability and detectability properties of such systems and giving a characterization of these properties in terms of system parameters, he examines the steady-state properties of quadratically optimal controllers. A robustness property of these controllers is pointed out. >

Theory of System Identification and Adaptive Control for Stochastic Systems

This paper presents a series of theoretical results on parameter identification and adaptive control for discrete-time stochastic feedback control systems. Without a use of persistent excitation condition the unknown coefficients of the system are consistently estimated by use of the extended least squares (ELS) and the stochastic gradient (SG) algorithms. Then a strong consistent estimate for orders of the system is also given by minimizing an information criterion designed for feedback control systems. In order to apply results of parameter estimation to adaptive control systems an attenuating excitation technique is introduced so that for strong consistency of the parameter estimate the conditions imposed on eigenvalues are replaced by the growth rate restriction on the input-output of the system. Further, optimal controls are designed respectively for tracking a given reference signal and for minimizing a quadratic loss function so that the parameter estimates are strongly consistent and, simultaneously, the performance indeces are minimized. When the system contains an unmodeled dynamics but the adaptive control is designed on the basis of the modeled part of the system a robust analysis is presented by estimating the error in parameter estimation and in control performances.

Adaptive policies for discrete-time stochastic control systems with unknown disturbance distribution

We introduce adaptive policies for discrete-time, infinite horizon, stochastic control systems x 1+1 = F( x 1, a 1, ξ 1, t =0, 1, …, with discounted reward criterion, where the disturbance process ξ 1 is a sequence of i.i.d. random elements with unknown distribution. These policies are shown to be asymptotically optimal and for each of them we obtain (almost surely) uniform approximations of the optimal reward function.

Optimal stochastic adaptive control with quadratic index

The discrete-time stochastic control system with complete observation is considered with quadratic loss function when the constant coefficients of the system are unknown. The parameter estimates given by the recursive least-squares method are used to define the feedback gain, and the adaptive control is taken to be either a linear-state feedback disturbed by a sequence of random vectors with variances tending to zero or only the disturbance without the feedback term in accordance with stopping times defined on the trajectory of the system. It is proved that the parameter estimates are strongly consistent and the loss function reaches its minimum, i.e. the adaptive control is optimal.

Adaptive Strategies for Certain Classes of Controlled Markov Processes

Previous article Next article Adaptive Strategies for Certain Classes of Controlled Markov ProcessesE. I. GordienkoE. I. Gordienkohttps://doi.org/10.1137/1129064PDFBibTexSections ToolsAdd to favoritesExport CitationTrack CitationsEmail SectionsAbout[1] V. N. Fomin, , A. L. Fradkov and , V. A. Yakubovich, Adaptive Control of Dynamic Objects, Nauka, Moscow, 1981, (In Russian.) 0522.93002 Google Scholar[2] V. G. Sragovich, Theory of Adaptive Systems, Nauka, 1976Moscow, (In Russian.) 0333.93005 Google Scholar[3] Yu. V. Popov, Adaptive systems for the control of certain classes of random processes of general type, Studies in the theory of adaptive systems (Russian), Vyčisl. Centr, Akad. Nauk SSSR, Moscow, 1976, 119–142, 223, (In Russian.) 58:33328 Google Scholar[4] G. A. Agasandyan, Adaptive system for homogeneous processes with continuous sets of states and controls, Theory Prob. Appl., 24 (1979), 515–528 0409.93030 Google Scholar[5] E. I. Gordienko, Adaptive optimal control of some Markov processes, Dokl. Akad. Nauk SSSR, 261 (1981), 271–275, (In Russian.) 83b:93041 0494.93027 Google Scholar[6] Ye. B. Dynkin and , A. A. Yushkevich, Controlled Markov processes, Grundlehren der Mathematischen Wissenschaften [Fundamental Principles of Mathematical Sciences], Vol. 235, Springer-Verlag, Berlin, 1979xvii+289 80k:90037 CrossrefGoogle Scholar[7] J. Doob, Stochastic processes, John Wiley & Sons Inc., New York, 1953viii+654 15,445b 0053.26802 Google Scholar[8] V. V. Kalashnikov, Qualitative Analysis of the Behavior of Complex Systems by the Method of Probe Functions, Nauka, Moscow, 1978, (In Russian.) 0451.93002 Google Scholar[9] L. G. Glubenko and , E. S. Shtatland, On controlled Markov processes with discrete timeTheory of Probability and Mathematical Statistics, Vol. 7, Naukova Dumka, Kiev, 1972, 51–64, (In Russian.) Google Scholar[10] V. V. Petrov, Sums of independent random variables, Springer-Verlag, New York, 1975x+346 52:9335 0322.60042 CrossrefGoogle Scholar[11] P. Ganssler and , W. Stute, Empirical processes: a survey of results for independent and identically distributed random variables, Ann. Probab., 7 (1979), 193–243 80d:60002 CrossrefGoogle Scholar[12] Patrick Billingsley and , Flemming Topsøe, Uniformity in weak convergence, Z. Wahrscheinlichkeitstheorie und Verw. Gebiete, 7 (1967), 1–16 35:326 0147.15701 CrossrefGoogle Scholar[13] A. N. Kolmogorov and , V. M. Tikhomirov, $\varepsilon$-entropy and $\varepsilon$-capacity of sets in function spaces, Uspehi Mat. Nauk, 14 (1959), 3–86 22:2890 Google Scholar[14] R. M. Dudley, The speed of mean Glivenko-Cantelli convergence, Ann. Math. Statist, 40 (1968), 40–50 38:5270 0184.41401 CrossrefGoogle Scholar Previous article Next article FiguresRelatedReferencesCited ByDetails Asymptotically Optimal Strategies for Adaptive Zero-Sum Discounted Markov GamesJ. Adolfo Minjárez-Sosa and Oscar Vega-AmayaSIAM Journal on Control and Optimization, Vol. 48, No. 3 | 15 April 2009AbstractPDF (230 KB)Empirical estimation in average Markov control processesApplied Mathematics Letters, Vol. 21, No. 5 | 1 May 2008 Cross Ref Average Optimality for Adaptive Markov Control Processes with Unbounded Costs and Unknown Disturbance DistributionMarkov Processes and Controlled Markov Chains | 1 Jan 2002 Cross Ref Approximation of average cost optimal policies for general Markov decision processes with unbounded costsMathematical Methods of Operations Research, Vol. 45, No. 2 | 1 Jun 1997 Cross Ref Recurrence conditions for Markov decision processes with Borel state space: A surveyAnnals of Operations Research, Vol. 28, No. 1 | 1 Dec 1991 Cross Ref Nonparametric estimation and adaptive control in a class of finite Markov decision chainsAnnals of Operations Research, Vol. 28, No. 1 | 1 Dec 1991 Cross Ref Density estimation and adaptive control of markov processes: Average and discounted criteriaActa Applicandae Mathematicae, Vol. 20, No. 3 | 1 Sep 1990 Cross Ref Nonparametric adaptive control of discrete-time partially observable stochastic systemsJournal of Mathematical Analysis and Applications, Vol. 137, No. 2 | 1 Feb 1989 Cross Ref Continuous dependence of stochastic control models on the noise distributionApplied Mathematics & Optimization, Vol. 17, No. 1 | 1 Jan 1988 Cross Ref Adaptive policies for discrete-time stochastic control systems with unknown disturbance distributionSystems & Control Letters, Vol. 9, No. 4 | 1 Oct 1987 Cross Ref Adaptive control of stochastic systems with unknown noise distribution--Discounted reward criterion1986 25th IEEE Conference on Decision and Control | 1 Dec 1986 Cross Ref Volume 29, Issue 3| 1985Theory of Probability & Its Applications427-645 History Submitted:12 April 1981Published online:17 July 2006 InformationCopyright © Society for Industrial and Applied MathematicsPDF Download Article & Publication DataArticle DOI:10.1137/1129064Article page range:pp. 504-518ISSN (print):0040-585XISSN (online):1095-7219Publisher:Society for Industrial and Applied Mathematics

Separation of estimation and control for decentralized stochastic control systems

A sufficient condition is given for a separation of estimation and control in decentralized discrete-time stochastic control systems. It is shown that the one-step delay sharing system is a special case satisfying this condition. A counterexample is given, however, to the conjecture that the separation will hold for the more-than-one-step delay sharing systems.

Separation of Estimation and Control for Decentralized Stochastic Control Systems

A sufficient condition is given for a separation of estimation and control in decentralized discrete-time stochastic control systems. It is shown that the one-step delay sharing system is a special case satisfying this condition. A counterexample is given, however, to the well-known conjecture that the separation will hold for the more-than-one-step delay sharing systems.

Decentralized Control of Discrete-Time Stochastic Systems

This paper deals with an optimal control problem of a discrete-time linear stochastic system with two controllers, each of which can make certain noisy measurements of the state of the system. These two controllers cooperate in trying to minimize the same performance index. Certain constraints are imposed on control structure, and so our problem is to determine the optimal values of the structure parameters of the controllers. Both the cases with and without information exchange between the two controllers are treated. Weak separability is introduced as an extended concept of the separation theorem and sufficient conditions for the problem to be weakly separable are obtained. Usefulness of the concept is illustrated by several numerical examples.

Optimal control of discrete-time stochastic systems

Optimal control of discrete-time stochastic systems