Linear Stochastic Systems Research Articles

Recovering the Graph Underlying Networked Dynamical Systems under Partial Observability: A Deep Learning Approach

We study the problem of graph structure identification, i.e., of recovering the graph of dependencies among time series. We model these time series data as components of the state of linear stochastic networked dynamical systems. We assume partial observability, where the state evolution of only a subset of nodes comprising the network is observed. We propose a new feature-based paradigm: to each pair of nodes, we compute a feature vector from the observed time series. We prove that these features are linearly separable, i.e., there exists a hyperplane that separates the cluster of features associated with connected pairs of nodes from those of disconnected pairs. This renders the features amenable to train a variety of classifiers to perform causal inference. In particular, we use these features to train Convolutional Neural Networks (CNNs). The resulting causal inference mechanism outperforms state-of-the-art counterparts w.r.t. sample-complexity. The trained CNNs generalize well over structurally distinct networks (dense or sparse) and noise-level profiles. Remarkably, they also generalize well to real-world networks while trained over a synthetic network -- namely, a particular realization of a random graph.

Maximum likelihood estimation for the reflected stochastic linear system with a large signal

Maximum likelihood estimation for the reflected stochastic linear system with a large signal

Sampling‐based model order reduction for stochastic differential equations driven by fractional Brownian motion

AbstractIn this paper, we study large‐scale linear fractional stochastic systems representing, e.g., spatially discretized stochastic partial differential equations (SPDEs) driven by fractional Brownian motion (fBm) with Hurst parameter H > ½. Such equations are more realistic in modeling real‐world phenomena in comparison to frameworks not capturing memory effects. To the best of our knowledge, dimension reduction schemes for such settings have not been studied so far.In this work, we investigate empirical reduced order methods that are either based on snapshots (e.g. proper orthogonal decomposition) or on approximated Gramians. In each case, dominant subspaces are learned from data. Such model reduction techniques are introduced and analyzed for stochastic systems with fractional noise and later applied to spatially discretized SPDEs driven by fBm in order to reduce the computational cost arising from both the high dimension of the considered stochastic system and a large number of required Monte Carlo runs.We validate our proposed techniques with numerical experiments for some large‐scale stochastic differential equations driven by fBm.

Optimal Linear-Quadratic Regulator for a Stochastic System under Mutually Inverse Time Preferences in the Cost

We investigate a long-run behavior of a linear stochastic system. It is assumed that the quadratic cost includes a time-varying function and its multiplicative inverse. Such a specification reflects the fact that time preferences used by agents to assess different types of losses evolve in opposite directions. We consider the case when priority is set for the losses associated with state deviations. The optimal control law is derived with respect to extended long-run average cost criteria. We provide conditions for the existence of an alternative control strategy, which is also optimal and is based on a solution of an algebraic Riccati equation.

Robust Kalman and Bayesian Set-Valued Filtering and Model Validation for Linear Stochastic Systems

Robust Kalman and Bayesian Set-Valued Filtering and Model Validation for Linear Stochastic Systems

Multi-Objective LQG Design with Primal-Dual Method

The objective of this paper is to investigate a multi-objective linear quadratic Gaussian (LQG) control problem. Specifically, we examine an optimal control problem that minimizes a quadratic cost over a finite time horizon for linear stochastic systems subject to control energy constraints. To tackle this problem, we propose an efficient bisection line search algorithm that outperforms other approaches such as semidefinite programming in terms of computational efficiency. The primary idea behind our algorithm is to use the Lagrangian function and Karush–Kuhn–Tucker (KKT) optimality conditions to address the constrained optimization problem. The bisection line search is employed to search for the Lagrange multiplier. Furthermore, we provide numerical examples to illustrate the efficacy of our proposed methods.

Reinforcement Learning for Adaptive Optimal Stationary Control of Linear Stochastic Systems

This article studies the adaptive optimal stationary control of continuous-time linear stochastic systems with both additive and multiplicative noises, using reinforcement learning techniques. Based on policy iteration, a novel off-policy reinforcement learning algorithm, named optimistic least-squares-based policy iteration, is proposed, which is able to find iteratively near-optimal policies of the adaptive optimal stationary control problem directly from input/state data without explicitly identifying any system matrices, starting from an initial admissible control policy. The solutions given by the proposed optimistic least-squares-based policy iteration are proved to converge to a small neighborhood of the optimal solution with probability one, under mild conditions. The application of the proposed algorithm to a triple inverted pendulum example validates its feasibility and effectiveness.

Фильтрация сигналов при скачкообразных мультипликативных помехах

The problem of constructing the approximately optimal Bayesian algorithm of filtering the output signal for a linear stochastic dynamical system is solved. Non-linear mixture of the output signal and of the multiplicative interference was measured. This interference is a continuous-valued random process with the unknown distribution law within the given limits of [c(1), c(2)], c(1) > 0, c(2) > 0, and in the frequency range of [0, ∆ω]. The filtering algorithm is synthesized by the approximately optimal signal estimation method based on the theory of systems with random jump structure and the method of two-time parametric approximation of the phase coordinates probability distribution. The method consists in approximate replacement of the unknown probability densities of phase coordinates by the known distribution laws with the unknown mathematical expectations and covariances determined as a result of solving the problem. The proposed approximate-optimal filtering algorithm is based on replacement of the continuous-valued multiplicative interference by a random jump process, i.e., Markov chain with two states c(1), c(2) and equal intensities of transitions from one state to another q′ = 2∆ω. Conditional probability densities of the output signal for fixed states of the Markov chain c(1), c(2) are approximated by gamma distributions that depend on mathematical expectations and variances of the x(t) signal at the fixed c(1), c(2) and measurements x(t) in a mixture with the Markov jump interference. An example of constructing an algorithm for evaluating the distance from an aircraft to the object based on object irradiance measurement by the infrared direction finder was considered. Irradiance was equal to the radiation strength ratio to the square of the distance. Radiation strength, i.e., multiplicative interference, was the random process with unknown continuous distribution in a limited range. The approximate-optimal filter was constructed for distance evaluation by replacing this process with a two-state Markov chain and approximating the distance probability density by the Rayleigh distribution

Filtered auxiliary model recursive generalized extended parameter estimation methods for Box–Jenkins systems by means of the filtering identification idea

AbstractFor equation‐error autoregressive moving average systems, that is, Box–Jenkins systems, this paper presents a filtered auxiliary model generalized extended stochastic gradient identification method, a filtered auxiliary model multi‐innovation generalized extended stochastic gradient identification method, a filtered auxiliary model recursive generalized extended gradient identification method, a filtered auxiliary model multi‐innovation recursive generalized extended gradient identification method, a filtered auxiliary model recursive generalized extended least squares identification method, and a filtered auxiliary model multi‐innovation recursive generalized extended least squares identification method by using the filtering identification idea and the auxiliary model identification idea. The proposed filtered auxiliary model recursive generalized extended identification methods can be generalized to other linear and nonlinear multivariable stochastic systems with colored noises.

Risk‐aware self‐triggered linear quadratic control

AbstractThis paper presents a self‐triggered control approach with the view of risks for discrete‐time linear stochastic systems. More specifically, under the assumption that the first two moments of the disturbance distribution are known, the paper considers the problem of designing a control law that reduces the use of communication and power resources by aperiodically sampling the states and updating control inputs, while achieving a given performance goal at a risk below a specified level. This problem is approached by using the worst‐case Conditional Value‐at‐Risk as a measure of risk, to quantify the tail behavior of the stochastic systems, thus allowing us to suppress large losses. A numerical example is provided to illustrate the performance of the proposed approach.

Delay-dependent Exponential Stability of Linear Stochastic Neutral Systems with General Delays Described by Stieltjes Integrals

Delay-dependent Exponential Stability of Linear Stochastic Neutral Systems with General Delays Described by Stieltjes Integrals

Optimal transmission scheduling for remote state estimation in CPSs with energy harvesting two-hop relay networks

This paper considers the optimal transmission power scheduling for accurate remote state estimation in cyber–physical systems (CPSs). The plant is modeled as a discrete-time stochastic linear system with sensor measurements transmitted to the remote estimator over an intelligent relay. A dynamic cooperative team is formulated in which the sensor and the relay sharing the same cost function are equipped with corresponding energy harvesters, and need to decide on the transmission power levels to minimize the average error covariance at the remote estimator. Since the nodes make simultaneous decisions to achieve a common goal, the sensor and the relay do not have access to the transmission power that will be selected by another node in advance, and hence they cannot build a value estimation of the cost function directly at each time point, which poses a challenge to determine the optimal transmission policies of both sides. To overcome this challenge, we provide a local-action value function and prove that both nodes select the transmission power with respect to the local-action value function without any loss of performance. In addition, a Markov decision process (MDP) and a multi-agent reinforcement learning algorithm are presented to obtain the optimal transmission power schedule over an infinite time horizon. Finally, simulation examples are provided to corroborate and illustrate the theoretical results.

Estimation of process noise variances from the measured output sequence with application to the empirical model of type 1 diabetes

This paper presents a novel method for estimating a priori unknown variances of the process noise affecting a general discrete-time stochastic state-space model under the assumption that the elements of the process noise vector are not correlated. To evaluate the proposed method in practice, only the system model and a sufficiently long sequence of output measurements are required, so there is no need for the system state to be measured or the state observer to be involved. In detail, the design of the method is based on the autocorrelation function of the stochastic output component, which is used to form the equivalent linear regression system with respect to the unknown vector of the process noise variances. Since the data-based estimate of the autocorrelation function has to be treated as uncertain, the generalized least squares method is proposed to solve the derived linear regression system, yielding the estimate with minimal variance. To verify the theory, the method is first validated within a case study that considers an artificial linear stochastic system, while this experiment was carried out under controlled statistical conditions. However, the intended target application domain of the method is the empirical model of type 1 diabetes, so another experiment based on virtual diabetic data obtained by simulating a complex physiology-based model that was subjected to input uncertainties was carried out.

Relations between entropy rate, entropy production and information geometry in linear stochastic systems

In this work, we investigate the relation between the concept of ‘information rate’, an information geometric method for measuring the speed of the time evolution of the statistical states of a stochastic process, and stochastic thermodynamics quantities like entropy rate and entropy production. Then, we propose the application of entropy rate and entropy production to different practical applications such as abrupt event detection, correlation analysis, and control engineering. Specifically, by utilising the Fokker–Planck equation of multi-variable linear stochastic processes described by Langevin equations, we calculate the exact value for information rate, entropy rate, and entropy production and derive various inequalities among them. Inspired by classical correlation coefficients and control techniques, we create entropic-informed correlation coefficients as abrupt event detection methods and information geometric cost functions as optimal thermodynamic control policies, respectively. The methods are analysed via the numerical simulations of common prototypical systems.

A New Event-Triggered Control Scheme for Stochastic Systems

This article studies event-triggered control of stochastic linear discrete-time systems with discounted quadratic cost functions. A new dynamic event-triggering condition is proposed, which has simultaneously stochastic and deterministic features. The designed event-triggered control system ensures the control performance to be within a desirable level relative to that using periodic time-triggered control, while discarding the unnecessary transmissions. By adjusting the parameters, the proposed event-triggering condition can be reduced to some existing ones in the literature. It is shown that the three features (dynamic, stochastic, and deterministic) are all helpful to further increase the average interevent times. Then, the criteria in terms of the parameters are presented to ensure mean-square stability of the closed-loop systems. Moreover, an improved version of the proposed event-triggering condition is given to enlarge the minimum interevent times. Finally, numerical simulations are given to illustrate the efficiency and feasibility of the proposed results.

Robust risk‐sensitive control

SummaryWe introduce arisk‐sensitivegeneralization of the mixed control problem for linear stochastic systems with additive noise. Two criteria ofexponential‐quadraticform are used to generalise the usual quadratic criteria. The solutions are found in a linear state‐feedback form for both the finite and the infinite horizon formulations in terms of coupled Riccati differential and algebraic equations. A change of measures for both criteria and completion of squares method is used to derive the solutions, and explicit sufficient conditions for the admissibility of controls are derived. An application to the problem ofrobustportfolio control in a market with random interest rate subject to a disturbance is also given.

Equation governing the probability density evolution of multi-dimensional linear fractional differential systems subject to Gaussian white noise

Stochastic fractional differential systems are important and useful in the mathematics, physics, and engineering fields. However, the determination of their probabilistic responses is difficult due to their non-Markovian property. The recently developed globally-evolving-based generalized density evolution equation (GE-GDEE), which is a unified partial differential equation (PDE) governing the transient probability density function (PDF) of a generic path-continuous process, including non-Markovian ones, provides a feasible tool to solve this problem. In the paper, the GE-GDEE for multi-dimensional linear fractional differential systems subject to Gaussian white noise is established. In particular, it is proved that in the GE-GDEE corresponding to the state-quantities of interest, the intrinsic drift coefficient is a time-varying linear function, and can be analytically determined. In this sense, an alternative low-dimensional equivalent linear integer-order differential system with exact closed-form coefficients for the original high-dimensional linear fractional differential system can be constructed such that their transient PDFs are identical. Specifically, for a multi-dimensional linear fractional differential system, if only one or two quantities are of interest, GE-GDEE is only in one or two dimensions, and the surrogate system would be a one- or two-dimensional linear integer-order system. Several examples are studied to assess the merit of the proposed method. Though presently the closed-form intrinsic drift coefficient is only available for linear stochastic fractional differential systems, the findings in the present paper provide a remarkable demonstration on the existence and eligibility of GE-GDEE for the case that the original high-dimensional system itself is non-Markovian, and provide insights for the physical-mechanism-informed determination of intrinsic drift and diffusion coefficients of GE-GDEE of more generic complex nonlinear systems.

Recursive Least-Squares Wiener Consensus Filter and Fixed-Point Smoother in Distributed Sensor Networks

Distributed Kalman filter (DKF) is classified into the information fusion Kalman filter (IFKF), i. e. the centralized Kalman filter (CKF), and the Kalman consensus filter (KCF) in distributed sensor networks. The KCF has the advantage to improve the estimate of the state at the sensor node uniformly by incorporating the information of the observations and the filtering estimates at the neighbor nodes. In the first devised KCF, a user adjusts the consensus gain. This paper designs the recursive least-squares (RLS) Wiener consensus filter and fixed-point smoother that do not need to be adjusted in linear discrete-time stochastic systems. In addition to the observation equation at the sensor node, an observation equation is introduced excessively. Here, the new observation is the sum of the filtering estimates of the signals at the neighbor nodes of the sensor node. Thus, it is interpreted that the RLS Wiener consensus estimators incorporate the information of the observations at the neighbor nodes indirectly because the observations are used in the calculations of the filtering estimates. A numerical simulation example shows that the proposed RLS Wiener consensus filter and fixed-point smoother are superior in estimation accuracy to the RLS Wiener estimators.

Detecting Nonlinear Interactions in Complex Systems: Application in Financial Markets

Emerging or diminishing nonlinear interactions in the evolution of a complex system may signal a possible structural change in its underlying mechanism. This type of structural break may exist in many applications, such as in climate and finance, and standard methods for change-point detection may not be sensitive to it. In this article, we present a novel scheme for detecting structural breaks through the occurrence or vanishing of nonlinear causal relationships in a complex system. A significance resampling test was developed for the null hypothesis (H0) of no nonlinear causal relationships using (a) an appropriate Gaussian instantaneous transform and vector autoregressive (VAR) process to generate the resampled multivariate time series consistent with H0; (b) the modelfree Granger causality measure of partial mutual information from mixed embedding (PMIME) to estimate all causal relationships; and (c) a characteristic of the network formed by PMIME as test statistic. The significance test was applied to sliding windows on the observed multivariate time series, and the change from rejection to no-rejection of H0, or the opposite, signaled a non-trivial change of the underlying dynamics of the observed complex system. Different network indices that capture different characteristics of the PMIME networks were used as test statistics. The test was evaluated on multiple synthetic complex and chaotic systems, as well as on linear and nonlinear stochastic systems, demonstrating that the proposed methodology is capable of detecting nonlinear causality. Furthermore, the scheme was applied to different records of financial indices regarding the global financial crisis of 2008, the two commodity crises of 2014 and 2020, the Brexit referendum of 2016, and the outbreak of COVID-19, accurately identifying the structural breaks at the identified times.

Convergence of adaptive MPC for linear stochastic systems

Convergence of adaptive MPC for linear stochastic systems