Control Of Stochastic Systems Research Articles

Compositional Construction of Safety Controllers for Networks of Continuous-Space POMDPs

In this article, we propose a compositional framework for the synthesis of safety controllers for networks of partially observable discrete-time stochastic control systems (also known as continuous-space partially observable Markov decision processes (POMDPs)). Given an estimator, we utilize a discretization-free approach to synthesize controllers ensuring safety specifications over finite-time horizons. The proposed framework is based on a notion of so-called <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">local control barrier functions</i> computed for subsystems in two different ways. In the first scheme, no prior knowledge of estimation accuracy is needed. The second framework utilizes a probability bound on the estimation accuracy using a notion of so-called <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">stochastic simulation functions</i> . In both proposed schemes, we derive sufficient small-gain-type conditions in order to compositionally construct control barrier functions for interconnected POMDPs using local barrier functions computed for subsystems. The constructed control barrier functions for the overall networks enable us to compute lower bounds on the probabilities that the interconnected POMDPs avoid certain unsafe regions in finite-time horizons. We demonstrate the effectiveness of our proposed approaches by applying them to an adaptive cruise control problem.

Relationships between the maximum principle and dynamic programming for infinite dimensional stochastic control systems

The Pontryagin type maximum principle and Bellman's dynamic programming principle serve as two of the most important tools in solving optimal control problems. There is a huge literature on the study of relationship between them. The main purpose of this paper is to investigate the relationship between the Pontryagin type maximum principle and the dynamic programming principle for control systems governed by stochastic evolution equations in infinite dimensional space, with the control variables entering into both the drift and the diffusion terms. To do so, we first prove the dynamic programming principle for those systems without employing the martingale solutions. Then we establish the desired relationships in both cases that value function associated is smooth or nonsmooth. For the nonsmooth case, in particular, by employing the relaxed transposition solution, we discover the connection between the superdifferentials and subdifferentials of value function and the first-order and second-order adjoint equations.

Using dynamic data reconciliation to improve the performance of PID feedback control systems with Gaussian/non-Gaussian distributed disturbance and measurement noise

For a stochastic PID feedback control system, the uncertainty of the working environment often leads to the unsatisfied performance of the system, which does not meet the profit requirements. The working environment generally includes external disturbance and measurement noise, etc. Gaussian distributed measurement noise and disturbances are widely considered while non-Gaussian distributed measurement noise and disturbances are rarely considered. In this paper, the performance degradation of Gaussian/non-Gaussian disturbances and measurement noise on a stochastic PID feedback system is considered and analyzed. An efficient method, dynamic data reconciliation (DDR) is developed to filter measurement noise and disturbances and improve the performance of the stochastic PID feedback control system. By utilizing model-based and measured information, DDR avoids time delays in output estimation. With the detailed theoretical analysis and simulation verification, the effectiveness of the proposed DDR technology on the stochastic PID feedback control system is verified. Compared with conventional exponential filters, DDR can achieve better control performance. The proposed DDR is also used for the control system of the DC–AC​ converter. The improved effect of DDR on the output quality is demonstrated by the results.

Tracking Control for Output Probability Density Function of Stochastic Systems Using FPD Method

Output probability density function (PDF) tracking control of stochastic systems has always been a challenging problem in both theoretical development and engineering practice. Focused on this challenge, this work proposes a novel stochastic control framework so that the output PDF can track a given time-varying PDF. Firstly, the output PDF is characterised by the weight dynamics following the B-spline model approximation. As a result, the PDF tracking problem is transferred to a state tracking problem for weight dynamics. In addition, the model error of the weight dynamics is described by the multiplicative noises to more effectively establish its stochastic dynamics. Moreover, to better reflect the practical applications in the real world, the given tracking target is set to be time-varying rather than static. Thus, an extended fully probabilistic design (FPD) is developed based on the conventional FPD to handle multiplicative noises and to track the time-varying references in a superior way. Finally, the proposed control framework is verified by a numerical example, and a comparison simulation with the linear-quadratic regulator (LQR) method is also included to illustrate the superiority of our proposed framework.

First order necessary condition for stochastic evolution control systems with random generators

&lt;p style='text-indent:20px;'&gt;The main purpose of this paper is to establish the first order necessary optimality condition for optimal control problems of stochastic evolution systems with random generators. For our controlled system, both the drift and the diffusion terms contain the control variable, and the control region is a nonempty closed subset of a separable Hilbert space. Compared with the existing works, the main difficulties here are to prove the well-posedness of the control and the adjoint systems. We obtain the well-posedness of the adjoint system in the sense of mild solutions and that of the control system by means of the stochastic transposition method. In addition, the variational analysis approach is employed to handle the nonconvexity of the control region when deriving our optimality condition.&lt;/p&gt;

Maximum principle for forward-backward partially observed optimal control of stochastic systems with delay

In this paper, we consider partially observed optimal control for forward-backward stochastic delay differential equations (FBSDDEs) where the control domain is non-convex and the control variable is allowed to enter into both diffusion and observation terms. We obtain a general stochastic maximum principle of these optimal control problems by using Girsanov?s theorem, the spike variational method and the filtering technique. We also derive the adjoint equations to the problem. Finally, we apply our results to study a linear-quadratic (LQ) optimal control with delay.

Direct Adaptive Control for Stochastic Systems with Risk-Sensitive Indices

We propose a direct adaptive control law based on the adaptive dynamic programming (ADP) algorithm for continuous-time stochastic linear systems with partially unknown system dynamics and infinite horizon quadratic risk-sensitive indices. A control design methodology is employed to iteratively solve the generalized algebraic Riccati equation by using the online information of the state and input, and to directly learn the optimal control law. We prove the convergence of the online ADP algorithm and show that the direct adaptive control law approximates the optimal control law as time goes on. Finally, a numerical simulation example is presented to demonstrate the effectiveness of our algorithm.

Robust H2 Analysis of Discrete-Time Linear Systems Characterized by Random Polytopes and Time-Varying Parameters

This paper deals with discrete-time linear stochastic systems characterized by random polytopes and time-varying parameters. Random polytopes and time-varying parameters are introduced so that they can represent a sort of temporal variations in the distributions of coefficient random matrices of the systems. An H2 norm is defined for associated stochastic systems, and H2 performance analysis is discussed. In particular, we derive a numerically tractable linear matrix inequality condition for such analysis, as an extension of our earlier results about H2 control for stochastic systems without random polytopes.

$ (s, S) $ Inventory policies for stochastic controlled system of Lindley-type with lost-sales

&lt;abstract&gt;&lt;p&gt;This paper presents a characterization of $ (s, S) $-inventory policies for Lindley systems with possibly unbounded costs, where the objective is to minimize the expected discounted total cost by ordering (production) strategies. Moreover, the existence of a subsequence of minimizers of the value iteration functions that converge to a $ (s, S) $ optimal inventory system policy is shown. A numerical example is given to illustrate the theory.&lt;/p&gt;&lt;/abstract&gt;

Constructing MDP Abstractions Using Data With Formal Guarantees

This letter is concerned with a data-driven technique for constructing finite Markov decision processes (MDPs) as finite abstractions of discrete-time stochastic control systems with <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">unknown</i> dynamics while providing formal closeness guarantees. The proposed scheme is based on notions of <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">stochastic bisimulation functions</i> (SBF) to capture the probabilistic distance between state trajectories of an unknown stochastic system and those of finite MDP. In our proposed setting, we first reformulate corresponding conditions of SBF as a robust convex program (RCP). We then propose a scenario convex program (SCP) associated to the original RCP by collecting a finite number of data from trajectories of the system. We ultimately construct an SBF between the data-driven finite MDP and the unknown stochastic system with a given confidence level by establishing a probabilistic relation between optimal values of the SCP and the RCP. We also propose two different approaches for the construction of finite MDPs from data. We illustrate the efficacy of our results over a <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">nonlinear</i> jet engine compressor with unknown dynamics. We construct a data-driven finite MDP as a suitable substitute of the original system to synthesize controllers maintaining the system in a safe set with some probability of satisfaction and a desirable confidence level.

Singular perturbations in stochastic optimal control with unbounded data

We study singular perturbations of a class of two-scale stochastic control systems with unbounded data. The assumptions are designed to cover some relaxation problems for deep neural networks. We construct effective Hamiltonian and initial data and prove the convergence of the value function to the solution of a limit (effective) Cauchy problem for a parabolic equation of HJB type. We use methods of probability, viscosity solutions and homogenization.

Recent Advances in Non-Gaussian Stochastic Systems Control Theory and Its Applications

Survey/review study Recent Advances in Non-Gaussian Stochastic Systems Control Theory and Its Applications Qichun Zhang 1, and Yuyang Zhou 2,* 1 Department of Computer Science, University of Bradford, Bradford BD7 1DP, United Kingdom 2 School of Computing, Engineering and the Built Environment, Edinburgh Napier University, Edinburgh EH10 5DT, United Kingdom * Correspondence: y.zhou@napier.ac.uk Received: 24 October 2022 Accepted: 18 November 2022 Published: 22 December 2022 Abstract: Non-Gaussian randomness widely exists in complex dynamical systems, in which the traditional mean-variance index cannot fully reflect the systematic characteristics. To improve the performance of control design subjected to non-Gaussian noises, stochastic distribution control (SDC) theory was proposed in the 1990s, where the output probability density function (PDF) has been investigated as an additional system variable. Following this framework, SDC has been extended to other research subjects in control systems such as filter design, fault diagnosis, and so on. It shows that SDC supplies an important solution to enhance the accuracy of system design, which is further beneficial to almost all the topics subject to non-Gaussian randomness. Meanwhile, the theoretical results of the SDC have been applied to several practical industrial applications. As data science raises based on the development of industrial artificial intelligence, SDC has been further developed recently focusing on data-driven design and multi-agent systems. To explore the new challenges with the evolution of SDC, e.g. unknown system models, unknown noise distributions, strong non-stationary transient dynamics, stability analysis and industrial applications, this survey summarises the most recent published results in the last 5 years of SDC work in terms of modelling, control, filtering, fault diagnosis, and industrial applications. Based on the technical analysis, potential future work is discussed in the end.

Attack‐parameter‐dependent state feedback controller design for stochastic systems with denial‐of‐service attacks

AbstractIn this article, we investigate the attack‐parameter‐dependent controller synthesis problem for a class of stochastic control system under aperiodic denial‐of‐service (DoS) attacks. Different from the classic DoS attack models, which are characterized by DoS frequency and DoS duration, a new type of time‐constrained DoS attack model that only restricts the upper and lower bounds of the sleeping and active periods of the DoS attacker is introduced. Considering the effect of such DoS attacks, a switching‐like stochastic system model is built. Then based on the attack‐parameter‐dependent time‐varying stochastic Lyapunov function method, a new mean square exponential stability criterion of the resulting stochastic system under the DoS attacks is formulated in the form of linear matrix inequalities. Also, the attack‐resilient controller is designed. Finally, taking an aircraft system as an example, the validity of the obtained theoretical results is verified.

Boundedness analysis of stochastic distributed delay-coupled systems on networks driven by G-Brownian motion

The present article is devoted to discuss the pth moment exponential ultimate boundedness for a class of stochastic distributed delay-coupled systems on networks driven by G-Brownian motion. By combining the method of proof by contradiction with Lyapunov method and graph theory, some sufficient condition for both the Lyapunov-type theorem and the coefficients-type theorem are proposed to ensure the pth moment exponential ultimate boundedness of the considered systems. As applications, the boundedness criteria are applied to stochastic distributed delay-coupled neural networks and stochastic distributed delay-coupled control systems in the framework of G-Brownian motion.

Adaptive speed tracking control for high speed trains under stochastic operation environments

This paper investigates the adaptive speed tracking control problem of high-speed trains (HSTs) in the presence of stochastic disturbances. Considering the complicated operation environments, the HST dynamics are firstly formulated into a stochastic control system, and then an improved minimum variance self-tuning regulator is proposed by incorporating with an attenuation factor and the Tchebycheff series, which is proven to be able to improve the robustness of the HST system to the parameter estimation uncertainties and the stochastic disturbances. The global stability of the closed-loop system is rigorously analyzed by establishing the logarithm law of the improved generalized minimum variance loss function. Moreover, a sufficient condition for the optimality of the closed-loop system is provided, which also guarantees the convergency of the proposed control method. The effectiveness of the proposed control approach is demonstrated through numerical simulations.

Event‐triggered control for stochastic systems with multiple delays

AbstractThis paper investigates the stabilization problem for continuous‐time stochastic systems with multiple delays under continuous event‐triggered mechanisms, of which both static case and dynamic case are considered individually. In order to avoid zeno phenomenon in every sample path, a suspension time after each successful execution is forced for our event‐triggered mechanisms, resulting in intermittent detection of system states. Under such control strategy, we deduce mean square exponential stability of stochastic systems with multiple delays by means of Hanalay‐type inequality and obtain a delay‐dependent‐based and less‐conservative stabilization criterion without involving the upper bound of time delays. Besides, a co‐design procedure is proposed for linear controller and event‐triggered mechanisms. In the end, an illustrative example is presented to show effectiveness of the proposed co‐design procedure and contrasts the system performance under static and dynamic event‐triggered mechanisms.

Recent Developments in Event-Triggered Control of Nonlinear Systems: An Overview

Event-triggered control is a most popular paradigm for transferring feedback information in an economical “as needed” manner. The study of event-triggered control can be traced back to the 1960s. Following significant advances on the topic of control over networks and the topic of nonlinear control systems over the last two decades, event-triggered control has quickly emerged as a major theoretical subject in control theory. Applications of event-triggered control are wide-spread ranging from embedded control systems and industrial control processes to unmanned systems and cyber-physical transportation networks. In this paper, we first review developments in the synthesis of event-triggered sampling mechanisms. Various event triggering mechanisms, such as static event trigger, dynamic event trigger, time-regularized event trigger, and event trigger with positive threshold offsets, are systematically discussed. Then, we study how to design a stabilizing controller that is robust with respect to the sampling errors. Finally, we review some recent results in the directions of self-triggered control, event-triggered tracking control and cooperative control, and event-triggered control of stochastic systems and partial differential equation systems. Practical applications of event-triggered control are also discussed.

A Modified Method of Successive Approximations for Stochastic Recursive Optimal Control Problems

Based on the stochastic maximum principle for the partially coupled forward-backward stochastic control system (FBSCS for short), a modified method of successive approximations (MSA for short) is established for stochastic recursive optimal control problems. The second-order adjoint processes are introduced in the augmented Hamiltonian minimization step since the control domain is not necessarily convex. Thanks to the theory of bounded mean oscillation martingales (BMO martingales for short), we give a delicate proof of the error estimate and then prove the convergence of the modified MSA algorithm. In a special case, we obtain a logarithmic convergence rate. When the control domain is convex and compact, a sufficient condition which makes the control returned from the MSA algorithm be a near-optimal control is given for a class of linear FBSCSs.

Uncertain Stochastic Optimal Control with Jump and Its Application in a Portfolio Game

This article describes a class of jump-uncertain stochastic control systems, and derives an Itô–Liu formula with jump. We characterize an optimal control law, that satisfies the Hamilton–Jacobi–Bellman equation with jump. Then, this paper deduces the optimal portfolio game under uncertain stochastic financial markets with jump. The information of players is symmetrical. The financial market is constituted of a risk-free asset and a risky asset whose price process is subjected to the jump-uncertain stochastic Black–Scholes model. The game is formulated by two utility maximization problems, each investor tries to maximize his relative utility, which is the weighted average of terminal wealth difference between his terminal wealth and that of his competitor. Finally, the explicit expressions of equilibrium investment strategies and value functions for the constant absolute risk-averse and constant relative risk-averse utility function are derived by using the dynamic programming principle.

The method of multi objective synthesis of stochastic robust control by multimass electromechanical systems under non-gausian random external disturbances

Aim. Development of the method of multi objective synthesis of stochastic robust control by multimass electromechanical systems to satisfy various requirements for the operation of such systems in various modes under non-gausian random external disturbances. Methodology. The problem of multi objective synthesis of stochastic robust control by multimass electromechanical systems to satisfy various requirements for the operation of such systems in various modes under non-gausian random external disturbances solved based on the choosing of weight matrices in the robust control goal vector.The calculation of the target vector is performed based on the solution of the zero-sum vector antagonistic game. The components of the game payoff vector are variable quality indicators that are applied to the system operation in various modes. The calculation of the components of payoff vector game are performed based on the simulation of the initial system closed by the synthesized stochastic controllers in various operating modes and under various external influences and variations in the parameters of the uncertainty of the initial plant. Results. The results of multi objective synthesis of stochastic robust two-mass electromechanical servo systems modes under non-gausian random external disturbances in which differences requirements for the operation of such systems in various modes were satisfied are given. Based on the results of modeling and experimental studies it is established, that with the help of synthesized robust nonlinear controllers, it is possible to improve of quality indicators of two-mass electromechanical servo system in comparison with the system with standard regulators. Originality. For the first time the method of multi objective synthesis of stochastic robust control by multimass electromechanical systems to satisfy various requirements for the operation of multimass systems in various modes is developed. Practical value. From the point of view of the practical implementation the possibility of solving the problem of multi objective synthesis of stochastic robust control systems to satisfy various requirements for the operation of multimass electromechanical systems in various modes is shown.