Control Of Stochastic Systems Research Articles

Reliability of nonlinear stochastic controlled systems considering the dynamics of sensors and actuators

A closed-loop controlled system usually consists of the main structure, sensors, and actuators. The dynamics of sensors and actuators may influence the motion of the main structure. This article presents an analytical study on the first-passage reliability of a nonlinear stochastic controlled system under the consideration of the dynamics of sensors and actuators. The coupled dynamic equations of the controlled systems with sensors and actuators are first given, which are further integrated into a controlled, randomly excited, dissipated Hamiltonian system. By applying the stochastic averaging method for quasi-Hamiltonian systems, a one-dimensional averaged differential equation for the Hamiltonian function is obtained. The backward Kolmogorov equation associated with the averaged equation is then derived for the first-passage reliability analysis, from which the approximate reliability function and probability density of first-passage time are obtained. The accuracy of the proposed procedure is demonstrated by an example. A comparative analysis of the reliability of the system with/without sensors and actuators is carried out, which indicates that ignoring sensors and actuators will make underestimation of the reliability of the closed-loop system with small time. However, when time increases, there appears the opposite trend. Our findings provide a reference for control strategy design.

A global maximum principle for stochastic optimal control problems with delay and applications

In this paper, an open problem is solved, for the stochastic optimal control problem with delay where the control domain is nonconvex and the diffusion term contains both control and its delayed term. Inspired by previous results about delayed stochastic control systems, Peng’s global stochastic maximum principle is generalized to the time delayed case. A special backward stochastic differential equation is introduced to deal with the cross terms, when applying the duality technique. Comparing with the classical result, the maximum condition contains an indicator function, which in fact is the characteristic of the stochastic optimal control problem with delay. Furthermore, to illustrate the applications of our theoretical results, three dynamic optimization problems are addressed based on the global maximum principle.

Optimal feedback control of stock prices under credit risk dynamics

In this paper we provide a stock price model that explicitly incorporates credit risk, under a stochastic optimal control system. The stock price model also incorporates the managerial control of credit risk through a control policy in the stochastic system. We provide explicit conditions on the existence of optimal feedback controls for the stock price model with credit risk. We prove the continuity of the value function, and then prove the dynamic programming principle for our system. Finally, we prove the Viscosity Solution of the Hamilton–Jacobi–Bellman equation. This paper is particularly relevant to industry, as the impact of credit risk upon stock prices has been prominent since the commencement of the Global Financial Crisis.

Quality estimation of stochastic control system using simulation methods

The stochastic control system is investigated. The article solves the problem of human-robot humanoid communication. The problem arises in the development of human behavior formation management systems. They describe human behavior in terms of probabilistic characteristics. Such control systems are stochastic. The management system of human behavior formation and its functioning is described quality assessment. The problem is solved by simulation modeling. The software implementing the method is described. The results of experimental research are presented.

Impossibility Results for Constrained Control of Stochastic Systems

Strictly unstable linear systems under additive and nonvanishing stochastic noise with unbounded supports are known to be impossible to stabilize by using deterministically constrained control inputs. In this paper, similar impossibility results are obtained for the scenarios where the control input is probabilistically constrained and the support of the noise distribution is not necessarily unbounded. In particular, control policies that have bounded time-averaged second moments are considered. It is shown that for such control policies, there are critical average moment bounds, below which second moment stabilization of a linear stochastic system is not possible, and moreover, second moment of the state diverges regardless of the choice of control policy and the initial state distribution. Nonnegative-definite Hermitian matrices are exploited to extract sufficient instability conditions that can be assessed by using the eigenstructure of the system matrix and the distribution of the noise. The results indicate that in certain networked control system settings with noise, designing stabilizing constrained controllers is an impossible task, if the probability of successful transmissions of control commands over the network is known to be too small in average.

Fault diagnosis and fault-tolerant control for T-S fuzzy time-varying delay stochastic distribution systems with actuator and sensor faults

ABSTRACT The problem of fault diagnosis (FD) and fault-tolerant control (FTC) for a class of nonlinear time-varying delay stochastic distribution control (SDC) systems with actuator and sensor faults is investigated in this paper. The Takagi–Sugeno (T-S) model is used to approximate the nonlinear dynamics. The B-spline function is used to approximate the output probability density function (PDF) of the system. According to the output equivalence principle and Laplace transformation, an augmented state vector is designed to solve the time-varying delay problem. An augmented state adaptive diagnosis observer is proposed to estimate the system state, actuator and sensor faults simultaneously. A new fault-tolerant control algorithm is designed based on the PI control strategy to compensate sensor fault and actuator faults simultaneously. The sensor fault is compensated through the information obtained by the observer. The PI controller can compensate for the influence of the actuator fault, and the output PDF of system can still track the desired PDF when fault occurs. Simulation results are provided to show the effectiveness of the proposed method.

A diffusion wavelets-based multiscale framework for inverse optimal control of stochastic systems

This work presents a multiscale framework to solve a class of inverse optimal control (IOC) problems in the context of robot motion planning and control in a complex environment. In order to handle complications resulting from a large decision space and complex environmental geometry, two key concepts are adopted: (a) a diffusion wavelet representation of the Markov chain for hierarchical abstraction of the state space; and (b) a desirability function-based representation of the Markov decision process (MDP) to efficiently calculate the optimal policy. An IOC problem constructed on a ‘abstract state’ is solved, which is much more tractable than using the original bases set; moreover, the solution can be obtained recursively in the ‘coarse to fine’ direction by utilizing the hierarchical structure of basis functions. The resulting multiscale plan is utilized to finally compute a continuous-time optimal control policy within a receding horizon implementation. Illustrative numerical experiments on a robot path control in a complex environment and on a quadrotor ball-catching task are presented to verify the proposed method.

Tensor Decomposition and High-Performance Computing for Solving High-Dimensional Stochastic Control System Numerically

The paper presents a numerical method for solving a class of high-dimensional stochastic control systems based on tensor decomposition and parallel computing. The HJB solution provides a globally optimal controller to the associated dynamical system. Variable substitution is used to simplify the nonlinear HJB equation. The curse of dimensionality is avoided by representing the HJB equation using separated representation. Alternating least squares (ALS) is used to reduced the separation rank. The experiment is conducted and the numerical solution is obtained. A high-performance algorithm is designed to reduce the separation rank in the parallel environment, solving the high-dimensional HJB equation with high efficiency.

Entropy-parametric control of stochastic system

The paper contains material about controlled methods of stochastic systems. The article considers the features of establishing control over a stochastic system, which are found when using parametric and informational criteria. It is proposed to use the potential of the Renyi entropy when building control over a complex multilevel system. The possibility of combining entropy and parametric criteria to increase the stability of control over the system is discussed.

Time Optimal Control of a Clarke Subdifferential Type Stochastic Evolution Inclusion in Hilbert Spaces

This paper investigates the time optimal control for a class of non-instantaneous impulsive Clarke subdifferential type stochastic evolution inclusions in Hilbert spaces. We focus first on the existence of mild solutions for these systems by using the measure of non-compactness and a fixed-point theorem wth the properties of Clarke subdifferential. Then, the existence of time optimal control of governed by stochastic control systems is also obtained. We do not assume that the evolution operator and the values of multi-valued map are compact. Finally, an example is given to illustrate the effectiveness of the results.

Reachability Analysis of Randomly Perturbed Hamiltonian Systems

Abstract In this paper, we revisit energy-based concepts of controllability and reformulate them for control-affine nonlinear systems perturbed by white noise. Specifically, we discuss the relation between controllability of deterministic systems and the corresponding stochastic control systems in the limit of small noise and in the case in which the target state is a measurable subset of the state space. We derive computable expression for hitting probabilities and mean first hitting times in terms of empirical Gramians, when the dynamics is given by a Hamiltonian system perturbed by dissipation and noise, and provide an easily computable expression for the corresponding controllability function as a function of a subset of the state variables.

RESILIENT HYBRID-TRIGGERED CONTROL FOR NETWORKED STOCHASTIC SYSTEMS UNDER DENIAL-OF-SERVICE ATTACKS

This paper investigates the stabilization problem for stochastic net-worked control systems under periodic denial-of-service (DoS) jammingattacks. First, the resilient hybrid-triggered communication scheme isdeveloped to reduce the network transmission data and improve theutilization efficiency, where a Bernoulli distribution is used to charac-terize the switching protocol between time-triggered scheme and event-triggered scheme. Then, a resilient hybrid-driven control protocol isdesigned, and a new switched stochastic system is constructed. Suffi-cient conditions of the mean-square exponential stability are derivedfor the underlying system under DoS attacks. Furthermore, a co-designscheme of the feedback gain and the hybrid-triggering parameter is ob-tained by solving linear matrix inequalities. Finally, a satellite controlsystem is employed to illustrate the virtue and applicability of the pro-posed approach.

RESILIENT HYBRID-TRIGGERED CONTROL FOR NETWORKED STOCHASTIC SYSTEMS UNDER DENIAL-OF-SERVICE ATTACKS

This paper investigates the stabilization problem for stochastic net-worked control systems under periodic denial-of-service (DoS) jammingattacks. First, the resilient hybrid-triggered communication scheme isdeveloped to reduce the network transmission data and improve theutilization efficiency, where a Bernoulli distribution is used to charac-terize the switching protocol between time-triggered scheme and event-triggered scheme. Then, a resilient hybrid-driven control protocol isdesigned, and a new switched stochastic system is constructed. Suffi-cient conditions of the mean-square exponential stability are derivedfor the underlying system under DoS attacks. Furthermore, a co-designscheme of the feedback gain and the hybrid-triggering parameter is ob-tained by solving linear matrix inequalities. Finally, a satellite controlsystem is employed to illustrate the virtue and applicability of the pro-posed approach.

Fault isolation and fault-tolerant control for Takagi-Sugeno fuzzy time-varying delay stochastic distribution systems

A fault isolation, estimation, and fault-tolerant control (FTC) scheme for nonlinear time-varying delay stochastic distribution control systems was presented in this paper. The Takagi-Sugeno fuzzy model was adopted to approach the nonlinear dynamics of time-varying delay systems. According to the output equivalence principle and Laplace transformation, an augmented state vector was given to solve the time-varying delay problem. When multiple actuator faults and interference occur simultaneously, fault detection, isolation and fault estimation was designed to obtained the fault information. To decouple faults and obtain the value and location information of the fault, the system was separated into two parts through the designed multiple conversion matrices, in which one subsystem was only affected by one actuator fault. This has simplified the design of fault isolation and estimation. A adaptive observer for fault estimation was given. Then, fault information such as the time, location, and size was determined. The observer gain matrices were calculated using linear matrix inequality (LMI). When a fault was detected and diagnosed, a FTC algorithm was devised using the proportional-integral control scheme to compensate the fault as much as possible. It has been shown that even if multiple faults actuator occurred simultaneously, the FTC controller still ensured the output probability density function of the system traced the desired probability density function when a fault occurred. Finally, the expected results were obtained through the simulation example, which confirmed the effectiveness of the method.

Robust Stochastic Observer-Based Attack-Tolerant Missile Guidance Control Design Under Malicious Actuator and Sensor Attacks

In this study, the robust H <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">∞</sub> stochastic observer-based attack-tolerant guidance control strategy is designed for the nonlinear stochastic missile guidance control system under the external disturbance and measurement noise as well as actuator attack signal and sensor attack signal. To simplify the attack signal estimation, a novel nonsingular smoothed dynamic model is introduced to efficiently describe the actuator and sensor attack signals. Consequently, the state/attack signal estimation can be easily achieved by using conventional Luenberger observer. Next, to attenuate the effect of external disturbance, measurement noise and approximation errors of attack signal on the missile guidance control system, the robust H <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">∞</sub> attack-tolerant guidance control performance is considered and the design condition is derived in terms of nonlinear Hamilton-Jacobi inequality (HJI) constrained problem. Since HJI cannot be easily solved analytically or numerically, the Takagi-Sugeno (T-S) fuzzy modelling method is utilized to facilitate the robust H <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">∞</sub> attack-tolerant guidance control strategy design. Thereafter, the H <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">∞</sub> observer-based attack-tolerant control design problem is transformed into linear matrix inequalities problem (LMIP) which can be solved very efficiently by using the convex optimization techniques. Simulation example, with the comparison between the proposed method and conventional robust missile guidance strategy, is given to illustrate the design procedures and validate the performance of the proposed method.

Data driven optimal control for stochastic systems with non-Gaussian disturbances

Data driven optimal control for stochastic systems with non-Gaussian disturbances

Data driven optimal control for stochastic systems with non-Gaussian disturbances

Data driven optimal control for stochastic systems with non-Gaussian disturbances

Optimal control of stochastic system with Fractional Brownian Motion.

In this paper, we introduce a class of stochastic harvesting population system with Fractional Brownian Motion (FBM), which is still unclear when the stochastic noise has the character of memorability. Stochastic optimal control problems with FBM can not be studied using classical methods, because FBM is neither a Markov pocess nor a semi-martingale. When the external environment impact on the system of FBM, the necessary and sufficient conditions for the optimization are offered through the stochastic maximum principle, Hamilton function and ItÔ formula in our work. To illustrate our study, we provide an example to demonstrate the obtained theoretical results, which is the expansion of certainty population system.

The difference and unity of irregular LQ control and standard LQ control and its solution

Irregular linear quadratic control (LQ, was called Singular LQ) has been a long-standing problem since 1970s. This paper will show that an irregular LQ control (deterministic) is solvable (for arbitrary initial value) if and only if the quadratic cost functional can be rewritten as a regular one by changing the terminal cost x′(T)Hx(T) to x′(T)[H + P1(T)]x(T), while the optimal controller can achieve P1(T)x(T) = 0 at the same time. In other words, the irregular controller (if exists) needs to do two things at the same time, one thing is to minimize the cost and the other is to achieve the terminal constraint P1(T)x(T) = 0, which clarifies the essential difference of irregular LQ from the standard LQ control where the controller is to minimize the cost only. With this breakthrough, we further study the irregular LQ control for stochastic systems with multiplicative noise. A sufficient solving condition and the optimal controller is presented based on Riccati equations.

Singular perturbations and optimal control of stochastic systems in infinite dimension: HJB equations and viscosity solutions

We study a stochastic optimal control problem for a two scale system driven by an infinite dimensional stochastic differential equation which consists of “slow” and “fast” components. We use the theory of viscosity solutions in Hilbert spaces to show that as the speed of the fast component goes to infinity, the value function of the optimal control problem converges to the viscosity solution of a reduced effective equation. We consider a rather general case where the evolution is given by an abstract semilinear stochastic differential equation with nonlinear dependence on the controls. The results of this paper generalize to the infinite dimensional case the finite dimensional results of Alvarez and Bardi [SIAM J. Control Optim.40(2001/02) 1159–1188] and complement the results in Hilbert spaces obtained recently in Guatteri and Tessitore [To appear in:Appl. Math. Optim.(2019)https://doi.org/10.1007/s00245-019-09577-y].