Discrete-time Stochastic Systems Research Articles

Event-triggered finite-time $$H_{2}/H_\infty $$ control for discrete-time stochastic mean-field systems

Event-triggered finite-time $$H_{2}/H_\infty $$ control for discrete-time stochastic mean-field systems

Almost sure exponential stability and stochastic stabilization of discrete-time stochastic systems with impulses

This paper considers the almost sure exponential stability and stochastic stabilization problems of discrete-time stochastic systems (DTSSs) with impulsive effects, where the average impulsive interval is taken into account. By using the average impulsive interval approach and the strong law of large numbers, we not only establish the criteria for almost sure exponential stability of general nonlinear discrete-time impulsive stochastic systems (DTISSs) but also design an exact method of a stochastic perturbation to stabilize a given unstable impulsive discrete-time systems. Adopting the average impulsive interval approach and the strong law of large numbers, we established a criterion for almost sure exponential stability of general nonlinear DTISSs. Furthermore, a method of stochastic perturbation has been developed to stabilize an unstable impulsive discrete-time system. Finally, two simulation examples demonstrate the effectiveness of the derived results.

Sampled-Data H∞ Dynamic Output-Feedback Control for NCSs With Successive Packet Losses and Stochastic Sampling.

The problem of sampled-data H∞ dynamic output-feedback control for networked control systems with successive packet losses (SPLs) and stochastic sampling is investigated in this article. The aim of using sampled-data control techniques is to alleviate network congestion. SPLs that occur in the sensor-to-controller (S-C) and controller-to-actuator (C-A) channels are modeled using a packet loss model. Additionally, it is assumed that stochastic sampling follows a Bernoulli distribution. A model is established to capture the stochastic characteristics of both the SPL model and stochastic sampling. This model is crucial as it allows us to determine the probability distribution of the sampling interval between successive update instants, which is essential for stability analysis. An exponential mean-square stability condition for the constructed equivalent discrete-time stochastic system, which also guarantees the prescribed H∞ performance, is established by incorporating probability theory. The desired controller is designed using a step-by-step synthesis approach, which may offer lower design conservatism compared to some existing methods. Finally, our designed approach using a networked F-404 engine system model is validated and its merits relative to existing results are discussed. The proposed method is finally validated by employing a networked model of the F-404 engine system. Furthermore, the advantages of our method are presented in comparison to previous results.

SVD-Based Parameter Identification of Discrete-Time Stochastic Systems with Unknown Exogenous Inputs

This paper addresses the problem of parameter identification for discrete-time stochastic systems with unknown exogenous inputs. These systems form an important class of dynamic stochastic system models used to describe objects and processes under a high level of a priori uncertainty, when it is not possible to make any assumptions about the evolution of the unknown input signal or its statistical properties. The main purpose of this paper is to construct a new SVD-based modification of the existing Gillijns and De Moor filtering algorithm for linear discrete-time stochastic systems with unknown exogenous inputs. Using the theoretical results obtained, we demonstrate how this modified algorithm can be applied to solve the problem of parameter identification. The results of our numerical experiments conducted in MATLAB confirm the effectiveness of the SVD-based parameter identification method that was developed, under conditions of unknown exogenous inputs, compared to maximum likelihood parameter identification when exogenous inputs are known.

On convergence of occupational measures sets of a discrete-time stochastic control system, with applications to averaging of hybrid systems

We establish that, under certain conditions, the set of random occupational measures generated by the state-control trajectories of a discrete-time stochastic system as well as the set of their mathematical expectations converge to a non-random, convex and compact set. We apply these results to the averaging a hybrid system with a slow continuous-time component and a fast discrete-time component. It is shown that the solutions of the hybrid system are approximated by the solutions of a differential inclusion. The novelty of our results is that we allow the state-control space of the fast component to be non-denumerable.

H∞ Controller Design for Networked Systems With Two-Channel Packet Dropouts and FDI Attacks.

In this article, the stochastic analysis and H∞ controller design problems of networked systems with packet dropouts and false data injection attacks are investigated. Different from the existing literature, we focus on the linear networked systems with external disturbances and both sensor-controller channel and controller-actuator channel are studied. First, we present a discrete-time modeling framework that leads to a stochastic closed-loop system with randomly varying parameters. To facilitate the analysis and H∞ control of resulting discrete-time stochastic closed-loop system, an equivalent yet analyzable stochastic augmented model is further constructed by matrix exponential computation. Based on this model, a stability condition is derived in the form of linear matrix inequality (LMI) with the aid of a reduced-order confluent Vandermonde matrix, Kronecker product operation, and law of total expectation. Specifically, the dimension of the LMI obtained in this article does not increase as the upper bound of consecutive packet dropouts does, which is also different from the existing literature. Subsequently, a desired H∞ controller is obtained such that the original discrete-time stochastic closed-loop system is exponentially mean-square stable with a prescribed H∞ performance. Finally, a numerical example and a direct current motor system are exploited to substantiate the effectiveness and practicability of the designed strategy.

Fixed time stability of discrete-time stochastic dynamical systems

In this paper, we address fixed time stability in probability of discrete-time stochastic dynamical systems. Unlike finite time stability in probability, wherein the finite time almost sure convergence behavior of the dynamical system depends on the system initial conditions, fixed time stability in probability involves finite time stability in probability for which the stochastic settling-time is guaranteed to be independent of the system initial conditions. More specifically, we develop Lyapunov theorems for fixed time stability in probability for Itô-type stationary nonlinear stochastic difference equations including a Lyapunov theorem that involves a Lyapunov difference satisfying an exponential inequality of the Lyapunov function that gives rise to a minimum bound on the average stochastic settling-time characterized by the primary and secondary branches of the Lambert W function.

Stability Analysis and Synthesis for Networked Systems With Noisy Sampling Intervals, Random Transmission Delays, and Packet Dropouts

In this article, we study the stability analysis and <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$H_{\infty }$</tex-math></inline-formula> controller synthesis problems for networked systems with random transmission delays and packet dropouts that are subject to noisy sampling intervals. The sampling errors and transmission delays are described by an independent and identically distributed sequence of continuous random variables and the transmission delays are assumed to be smaller than the noisy sampling intervals. First, a closed-loop discrete-time stochastic framework for networked system that allows investigating noisy sampling intervals, random transmission delays and packet dropouts simultaneously is established. To render the discrete-time stochastic system suitable for stability analysis and synthesis, an equivalent yet tractable stochastic augmented model is then reformulated with the help of matrix exponential computation. Based on this augmented model, a stability condition involving the expectation of a nonlinear and random coupling matrix is derived. By recurring to the law of total expectation and Kronecker product operation, the expectation operation of the coupling matrix subject to high nonlinearity and multiple randomness is decoupled. Subsequently, a controller algorithm is designed such that the exponential mean-square stability of the original discrete-time stochastic system with a prescribed <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$H_{\infty }$</tex-math></inline-formula> performance is guaranteed. Finally, an example is provided to illustrate the effectiveness of the controller design algorithm.

On the Kolmogorov constant explicit form in the theory of discrete-time stochastic branching systems

Abstract We consider a discrete-time population growth system called the Bienaymé–Galton–Watson stochastic branching system. We deal with a noncritical case, in which the per capita offspring mean $m\neq1$ . The famous Kolmogorov theorem asserts that the expectation of the population size in the subcritical case $m&lt;1$ on positive trajectories of the system asymptotically stabilizes and approaches ${1}/\mathcal{K}$ , where $\mathcal{K}$ is called the Kolmogorov constant. The paper is devoted to the search for an explicit expression of this constant depending on the structural parameters of the system. Our argumentation is essentially based on the basic lemma describing the asymptotic expansion of the probability-generating function of the number of individuals. We state this lemma for the noncritical case. Subsequently, we find an extended analogue of the Kolmogorov constant in the noncritical case. An important role in our discussion is also played by the asymptotic properties of transition probabilities of the Q-process and their convergence to invariant measures. Obtaining the explicit form of the extended Kolmogorov constant, we refine several limit theorems of the theory of noncritical branching systems, showing explicit leading terms in the asymptotic expansions.

Active Diagnosis of Incipient Actuator Faults for Stochastic Systems

In this paper, the active diagnosis problem of incipient actuator faults is investigated for a class of discrete-time stochastic systems. The input is exclusively designed for diagnosis purposes to optimally discriminate among multiple mode hypotheses that correspond to the multiple incipient actuator fault scenarios. Due to infeasibility in evaluating the misdiagnosis probability in the closed form, its new upper bound is derived as an alternative. It is proven that this upper bound is a tighter bound on the misdiagnosis probability for incipient actuator faults, compared with the widely used Bhattacharyya upper bound. For computation complexity reduction and diagnosis performance improvement purposes, a novel input design criterion is constructed for active fault diagnosis (AFD) based on the derived upper bound, under which the input design problem can be solved to global optimality. By injecting the optimal input, to fully utilize the diagnostically relevant information in the output data so as to draw a reliable diagnosis conclusion, the resulting measurement output sequence is exploited to make the AFD decision on the basis of the minimum misdiagnosis probability rule. Finally, experimental results on a four-tank system illustrate the effectiveness and superiority of the proposed AFD approach.

Robust Estimators for Missing Observations in Linear Discrete-Time Stochastic Systems with Uncertainties

As a first approach to estimating the signal and the state, Theorem 1 proposes recursive least-squares (RLS) Wiener fixed-point smoothing and filtering algorithms that are robust to missing measurements in linear discrete-time stochastic systems with uncertainties. The degraded quantity is given by multiplying the Bernoulli random variable by the degraded signal caused by the uncertainties in the system and observation matrices. The degraded quantity is observed with additional white observation noise. The probability that the degraded signal is present in the observation equation is assumed to be known. The design feature of the proposed robust estimators is the fitting of the degraded signal to a finite-order autoregressive (AR) model. Theorem 1 is transformed into Corollary 1, which expresses the covariance information in a semi-degenerate kernel form. The autocovariance function of the degraded state and the cross-covariance function between the nominal state and the degraded state is expressed in semi-degenerate kernel forms. Theorem 2 shows the robust RLS Wiener fixed-point and filtering algorithms for estimating the signal and state from degraded observations in the second method. The robust estimation algorithm of Theorem 2 has the advantage that, unlike Theorem 1 and the usual studies, it does not use information on the existence probability of the degraded signal. This is a unique feature of Theorem 2.

Parameter Identification of the Discrete-Time Stochastic Systems with Multiplicative and Additive Noises Using the UD-Based State Sensitivity Evaluation

The paper proposes a new method for solving the parameter identification problem for a class of discrete-time linear stochastic systems with multiplicative and additive noises using a numerical gradient-based optimization. The constructed method is based on the application of a covariance UD filter for the above systems and an original method for evaluating state sensitivities within the numerically stable, matrix-orthogonal MWGS transformation. In addition to the numerical stability of the proposed algorithm to machine roundoff errors due to the application of the MWGS-UD orthogonalization procedure at each step, the main advantage of the obtained results is the possibility of analytical calculation of derivatives at a given value of the identified parameter without the need to use finite-difference methods. Numerical experiments demonstrate how the obtained results can be applied to solve the parameter identification problem for the considered stochastic system model.

Closed-Loop Continuous-Time Subspace Identification with Prior Information

This paper presents a closed-loop continuous-time subspace identification method using prior information. Based on a rational inner function, a generalized orthonormal basis can be constructed, and the transformed noises have ergodicity features. The continuous-time stochastic system is converted into a discrete-time stochastic system by using generalized orthogonal basis functions. As is known to all, incorporating prior information into identification strategies can increase the precision of the identified model. To enhance the precision of the identification method, prior information is integrated through the use of constrained least squares, and principal component analysis is adopted to achieve the reliable estimate of the system. Moreover, the identification of open-loop models is the primary intent of the continuous-time system identification approaches. For closed-loop systems, the open-loop subspace identification methods may produce biased results. Principal component analysis, which reliably estimates closed-loop systems, provides a solution to this problem. The restricted least-squares method with an equality constraint is used to incorporate prior information into the impulse response following the principal component analysis. The input–output algebraic equation yielded an optimal multi-step-ahead predictor, and the equality constraints describe the prior information. The effectiveness of the proposed method is provided by numerical simulations.

Moment propagation of polynomial systems through Carleman linearization for probabilistic safety analysis

We develop a method to approximate the moments of a discrete-time stochastic polynomial system. Our method is built upon Carleman linearization with truncation. Specifically, we take a stochastic polynomial system with finitely many states and transform it into an infinite-dimensional system with linear deterministic dynamics, which describe the exact evolution of the moments of the original polynomial system. We then truncate this deterministic system to obtain a finite-dimensional linear system, and use it for moment approximation by iteratively propagating the moments along the finite-dimensional linear dynamics across time. We provide efficient online computation methods for this propagation scheme with several error bounds for the approximation. Our results also show that precise values of certain moments at a given time step can be obtained when the truncated system is sufficiently large. Furthermore, we investigate techniques to reduce the offline computation load using reduced Kronecker power. Based on the obtained approximate moments and their errors, we also provide hyperellipsoidal regions that are safe for some given probability bound. Those bounds allow us to conduct probabilistic safety analysis online through convex optimization. We demonstrate our results on a logistic map with stochastic dynamics and a vehicle dynamics subject to stochastic disturbance.

Observability and Detectability of Stochastic Labeled Graphs

In this paper, <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">observable stochastic graphs and detetectable stochastic graphs</i> are, respectively, defined with the detailed implementation for the observability and detectability of stochastic discrete-time and discrete-state dynamic systems. More specifically, they are generally two classes of vertex-colored and edge-labeled graphs rendering a walking agent therein to determine his initial and current positions, respectively, in probability one by measuring the color sequence of his traversed vertices. In the part of analysis, we follow the aforementioned two definitions and establish the corresponding polynomial-time verifying algorithms. Notably, the implicit formulas are also established to calculate the observability and detectability probability for any pairwise vertex pair. In the synthesis part, the minimal number of colors dyeing the vertices to make any considered graph stochastically observable and detectable is investigated, respectively. Our results indicate that the minimal coloring problem is NP-hard for observability in the stochastic situations but is solvable in polynomial time for detectability in the deterministic cases. The observability and detectability of stochastic finite-valued systems, assembling with the finite-cardinal state spaces, are validated as a compelling application of these two types of directed graphs, whereas the minimal sensors placement problems subject to observability and detectability problems are accordingly interpreted by the minimum set cover algorithm.

Parameter identification of the linear discrete-time stochastic systems with unknown exogenous inputs

The paper addresses a parameter identification problem for linear discrete-time stochastic systems with unknown exogenous inputs. Such systems are considered when solving practical problems related to the measurements processing in the case when it is impossible to do any assumptions about the evolution of unknown input signal or its statistical characteristics that can change over time. We consider a class of discrete time linear stochastic systems with unknown exogenous inputs, where an additional source of a priori uncertainty of the system model is introduced, namely, the unknown parameter, on the elements of which the system model matrices can depend. This formulation of the parameter identification problem under the conditions of unknown inputs and the presence of random noises describes a high degree of uncertainty of a discrete time linear stochastic system. We propose a novel solution to this problem based on the construction of a new instrumental identification criterion. Minimization of this criterion allows for evaluating the unknown system model parameters simultaneously with the estimating of the state vector and unknown exogenous inputs of the system. Numerical experiments confirm the validity and efficiency of the proposed parameter identification method.

Continuous-Time Subspace Identification with Prior Information Using Generalized Orthonormal Basis Functions

This paper presents a continuous-time subspace identification method utilizing prior information and generalized orthonormal basis functions. A generalized orthonormal basis is constructed by a rational inner function, and the transformed noises have ergodic properties. The lifting approach and the Hambo system transform are used to establish the equivalent nature of continuous and transformed discrete-time stochastic systems. The constrained least squares method is adopted to investigate the incorporation of prior knowledge in order to further increase the subspace identification algorithm’s accuracy. The input–output algebraic equation derives an optimal multistep forward predictor, and prior knowledge is expressed as equality constraints. In order to solve an optimization problem with equality constraints characterizing the prior knowledge, the proposed method reduces the computational burden. The effectiveness of the proposed method is provided by numerical simulations.

Data-driven verification and synthesis of stochastic systems via barrier certificates

In this work, we study verification and synthesis problems for safety specifications over unknown discrete-time stochastic systems. When a model of the system is available, barrier certificates have been successfully applied for ensuring the satisfaction of safety specifications. In this work, we formulate the computation of barrier certificates as a robust convex program (RCP). Solving the acquired RCP is hard in general because the model of the system that appears in one of the constraints of the RCP is unknown. We propose a data-driven approach that replaces the uncountable number of constraints in the RCP with a finite number of constraints by taking finitely many random samples from the trajectories of the system. We thus replace the original RCP with a scenario convex program (SCP) and show how to relate their optimizers. We guarantee that the solution of the SCP is a solution of the RCP with a priori guaranteed confidence when the number of samples is larger than a specific value. This provides a lower bound on the safety probability of the original unknown system together with a controller in the case of synthesis. We also discuss an extension of our verification approach to a case where the associated robust program is non-convex and show how a similar methodology can be applied. Finally, the applicability of our proposed approach is illustrated through three case studies.

Application of Migrating Optimization Algorithms in Problems of Optimal Control of Discrete-Time Stochastic Dynamical Systems

The problem of finding the optimal open-loop control for discrete-time stochastic dynamical systems is considered. It is assumed that the initial conditions and external influences are random. The average value of the Bolza functional defined on individual trajectories is minimized. It is proposed to solve the problem by means of classical and modified migrating optimization algorithms. The modification of the migrating algorithm consists of cloning the members of the initial population and choosing different strategies of migratory behavior for the main population and for populations formed by clones. At the final stage of the search for an extremum, an intensively clarifying migration cycle is implemented with the participation of three leaders of the populations participating in the search process. Problems of optimal control of bundles of trajectories of deterministic discrete dynamical systems, as well as individual trajectories, are considered as special cases. Seven model examples illustrating the performance of the proposed approach are solved.

Алгоритмы дискретной фильтрации на основе модифицированной взвешенной ортогонализации Грама –Шмидта для дискретных стохастических систем с мультипликативными и аддитивными шумами

Discrete-time stochastic systems with multiplicative and additive noises describe a wide class of mathematical models of complex systems, for example, industrial-technological, energy, economic, telecommunication, aerospace systems, etc. An important class of algorithms for processing measure ment information in complex systems are Kalman-type discrete-time filtering algorithms. Purpose. Construction of new discrete-time filtering algorithms for discrete-time linear systems with multiplicative and additive noises based on numerically stable modified weighted Gram – Schmidt orthogonalization (MWGS). Methodology. The methods of computational linear algebra were used, namely, the direct procedu re of MWGS-orthogonalization, the theory of Kalman filtering, methods of scientific programming in MATLAB. Findings. New LD-algorithms for discrete-time filtering in covariance and informational form for discrete-time stochastic systems with multiplicative and additive noises are constructed. The algorithms have an extended array form allowing updates for all necessary filter values. The method employs numerically stable modified weighted Gram – Schmidt orthogonalization. Algebraic equiva lence of LD-filters to Kalman-type covariance and informational algorithms for linear discrete time stochastic systems with multiplicative and additive noises is proved. The conducted numerical experiments have shown the effectiveness of the proposed algorithms using the example of solving the problem of parametric estimation of a model of almost rectilinear motion, as well as their savings in computation time compared to previously constructed UD filters. Value. New discrete-time filtering LD-algorithms can be used as a reliable computational alterna tive to the Kalman-type “standard algorithms” since they have the property being numerically stable to machine round-off errors due to the use of the MWGS orthogonalization computational procedure at each iteration of the algorithm. The results can be used to solve problems of measurement information processing in discrete-time systems with multiplicative and additive noise.