Linear State-space Model Research Articles

Bayesian multivariate nonlinear state space copula models

A novel flexible class of multivariate nonlinear non-Gaussian state space models, based on copulas, is proposed. Specifically, it is assumed that the observation equation and the state equation are defined by copula families that are not necessarily equal. Inference is performed within the Bayesian framework, using the Hamiltonian Monte Carlo method. Simulation studies show that the proposed copula-based approach is extremely flexible, since it is able to describe a wide range of dependence structures and, at the same time, allows us to deal with missing data. The application to atmospheric pollutant measurement data shows that the approach is suitable for accurate modeling and prediction of data dynamics in the presence of missing values. Comparison to a Gaussian linear state space model and to Bayesian additive regression trees shows the superior performance of the proposed model with respect to predictive accuracy.

Fine-grained Network Traffic Prediction from Coarse Data

ICT systems provide detailed information on computer network traffic. However, due to storage limitations, some of the information on past traffic is often only retained in an aggregated form. In this paper we show that Linear Gaussian State Space Models yield simple yet effective methods to make predictions based on time series at different aggregation levels. The models link coarse-grained and fine-grained time series to a single model that is able to provide fine-grained predictions. Our numerical experiments show up to 3.7 times improvement in expected mean absolute forecast error when forecasts are made using, instead of ignoring, additional coarse-grained observations. The forecasts are obtained in a Bayesian formulation of the model, which allows for provisioning of a traffic prediction service with highly informative priors obtained from coarse-grained historical data.

Real-time temperature distribution reconstruction via linear parameter-varying state-space model and Kalman filter in rack-based cooling data centers

Rapidly growing data centers (DCs) are facing challenges with energy consumption increase and thermal environmental security. Thus, this paper proposed a novel real-time temperature distribution prediction and reconstruction model for rack-based cooling DCs, in order to facilitate efficient energy consumption and thermal environment management. The parameter identification method coupling with inherent physical relationships was employed to obtain the linear parameter-varying state-space model, rapidly and approximately predicting the nonlinear thermal dynamic processes within rack-based cooling DC. Then, Kalman filter was adopted to reconstruct the temperature distribution in real-time based upon the developed linear parameter-varying state-space temperature prediction model and measurement information. This study comprehensively analyzed the temperature estimation performance of the proposed model under various model input biases, different sensor numbers and layouts. The results show that the input biases have significant impacts on the accuracy of the temperature prediction model, e.g., ±5% bias in IT workload resulting in an increase of 0.6 times in the mean absolute error (MAE) of the temperature prediction. The proposed temperature reconstruction model reveals excellent performance: compared to the original temperature prediction model, the MAE was decreased by about 5%, 10%, and 13% for reconstruction scenarios using one, two, and three sensors, respectively.

Bayesian Inference for State-Space Models With Student-t Mixture Distributions.

This article proposes a robust Bayesian inference approach for linear state-space models with nonstationary and heavy-tailed noise for robust state estimation. The predicted distribution is modeled as the hierarchical Student- t distribution, while the likelihood function is modified to the Student- t mixture distribution. By learning the corresponding parameters online, informative components of the Student- t mixture distribution are adapted to approximate the statistics of potential uncertainties. Then, the obstacle caused by the coupling of the updated parameters is eliminated by the variational Bayesian (VB) technique and fixed-point iterations. Discussions are provided to show the reasons for the achieved advantages analytically. Using the Newtonian tracking example and a three degree-of-freedom (DOF) hover system, we show that the proposed inference approach exhibits better performance compared with the existing method in the presence of modeling uncertainties and measurement outliers.

Process monitoring using recurrent Kalman variational auto-encoder for general complex dynamic processes

Recently, latent models have continued to find advantages in statistical process monitoring, especially with multifarious extensions coping with the nonlinearity and dynamics confronted by the pattern extraction. However, when later performing the famous T2 statistic on the patterns, the issue of non-Gaussianity has rarely been focused alongside, hence vitiating the control of Type-I and -II error in practice. In this paper, a model, in the name of recurrent Kalman variational auto-encoder (RKVAE), is theoretically derived for handling these properties simultaneously, and proposed for its use on monitoring general complex processes. The main idea is to build a monitoring panel on the basis of further reasoning the dynamics underlying the nonlinearly extracted Gaussian features. The model hierarchically consists of three parts: the vanilla VAE for encoding latent Gaussian features, a linear Gaussian state-space model (LGSSM) for temporally reasoning these features, and an RNN for recurrently updating the online LGSSM by fusing several candidate parameters. For the latter K in the name, it is the Kalman filter (KF) that facilitates the feature noise filtering and LGSSM identification. The model has, structurally, a good capacity to be extensible from both the depth and width when modelling complex processes. Finally, based on the explicit formula of LGSSM, a detection panel is designed, and two case studies are given to demonstrate its effectiveness in monitoring and relevant analysis.

Constrained temperature and relative humidity predictive control: Agricultural greenhouse case of study

The importance of Model Predictive Control (MPC) has significant applications in the agricultural industry, more specifically for greenhouse’s control tasks. However, the complexity of the greenhouse and its limited prior knowledge prevent an exact mathematical description of the system. Subspace methods provide a promising solution to this issue through their capacity to identify the system’s comportment using the fit between model output and observed data. In this paper, we introduce an application of Constrained Model Predictive Control (CMPC) for a greenhouse temperature and relative humidity. For this purpose, two Multi Input Single Output (MISO) systems, using Numerical Subspace State Space System Identification (N4SID) algorithm, are firstly suggested to identify the temperature and the relative humidity comportment to heating and ventilation actions. In this sense, linear state space models were adopted in order to evaluate the robustness of the control strategy. Once the system is identified, the MPC technique is applied for the temperature and the humidity regulation. Simulation results show that the regulation of the temperature and the relative humidity under constraints was guaranteed, both parameters respect the ranges 15 °C ≤ Tint ≤ 30 °C and 50 % ≤ Hint ≤ 70 % respectively. On the other hand, the control signals uf and uh applied to the fan and the heater, respect the hard constraints notion, the control signals for the fan and the heater did not exceed 0 ≤ uf ≤ 4.3 Volts and 0 ≤ uh ≤ 5 Volts, respectively, which proves the effectiveness of the MPC and the tracking tasks. Moreover, we show that with the proposed technique, using a new optimization toolbox, the computational complexity has been significantly reduced. The greenhouse in question is devoted to Schefflera Arboricola cultivation.

Numerical Gaussian Process Kalman Filtering for Spatiotemporal Systems

We present a novel Kalman filter for spatiotemporal systems called the numerical Gaussian process Kalman filter (NGPKF). Numerical Gaussian processes have recently been introduced as a physics informed machine learning method for simulating time-dependent partial differential equations without the need for spatial discretization while also providing uncertainty quantification of the simulation resulting from noisy initial data. We formulate numerical Gaussian processes as linear Gaussian state space models. This allows us to derive the recursive Kalman filter algorithm under the numerical Gaussian process state space model. Using two case studies, we show that the NGPKF is more accurate and robust, given enough measurements, than a spatial discretization based KF.

Applying and dissecting LSTM neural networks and regularized learning for dynamic inferential modeling

Deep learning models such as the long short-term memory (LSTM) network have been applied for dynamic inferential modeling. However, many studies apply LSTM as a black-box approach without examining the necessity and usefulness of the internal LSTM gates for inferential modeling. In this paper, we use LSTM as a state space realization and compare it with linear state space modeling and statistical learning methods, including N4SID, partial least squares, the Lasso, and support vector regression. Two case studies on an industrial 660 MW boiler and a debutanizer column process indicate that LSTM underperforms all other methods. LSTM is shown to be capable of outperforming linear methods for a simulated reactor process with severely excited nonlinearity in the data. By dissecting the sub-components of a simple LSTM model, the effectiveness of the LSTM gates and nonlinear activation functions is scrutinized.

Data-driven modeling for damping and positioning control of gantry crane

Flexible structural vibrations of large gantry cranes are a well-known issue. Several control strategies were proposed for this problem, which require a detailed dynamical model. Since the modeling of large-scaled structures, as well as of significant nonlinear friction effects is complicated and time consuming, a direct data-driven dynamic reconstruction based on the Koopman operator theory is proposed. Such identification results in a higher order linear state-space model, which reflects the nonlinear behavior in the broad dynamical range of the system and does not depend on a specific operating point. In this paper, a data-driven linear model identification is investigated for a simulation model, as well as for an experimental laboratory crane. The damping and positioning control dynamics based on the identified model were studied and it was shown, that the resulting performance is comparable to model-based nonlinear approaches.

Uncovering the Origins of Instability in Dynamical Systems: How Can the Attention Mechanism Help?

The behavior of the network and its stability are governed by both dynamics of the individual nodes, as well as their topological interconnections. The attention mechanism as an integral part of neural network models was initially designed for natural language processing (NLP) and, so far, has shown excellent performance in combining the dynamics of individual nodes and the coupling strengths between them within a network. Despite the undoubted impact of the attention mechanism, it is not yet clear why some nodes of a network obtain higher attention weights. To come up with more explainable solutions, we tried to look at the problem from a stability perspective. Based on stability theory, negative connections in a network can create feedback loops or other complex structures by allowing information to flow in the opposite direction. These structures play a critical role in the dynamics of a complex system and can contribute to abnormal synchronization, amplification, or suppression. We hypothesized that those nodes that are involved in organizing such structures could push the entire network into instability modes and therefore need more attention during analysis. To test this hypothesis, the attention mechanism, along with spectral and topological stability analyses, was performed on a real-world numerical problem, i.e., a linear Multi-Input Multi-Output state-space model of a piezoelectric tube actuator. The findings of our study suggest that the attention should be directed toward the collective behavior of imbalanced structures and polarity-driven structural instabilities within the network. The results demonstrated that the nodes receiving more attention cause more instability in the system. Our study provides a proof of concept to understand why perturbing some nodes of a network may cause dramatic changes in the network dynamics.

Data-driven worst case model predictive control algorithm for propylene distillation column under uncertainty of top composition

Composition soft sensors have wide application in the distillation process. In this study, considering the limitation of the prediction ability of the composition soft sensor, a data-driven worst case model predictive control of propylene distillation column is proposed to hedge against the uncertainty of top composition. Firstly, based on the compartmental method and the dynamic mechanism model, a linear state space model of the distillation column is constructed. Aiming at dealing with the uncertainty of top propylene content caused by composition soft sensor, the data-driven uncertainty set is constructed by the combination of principal component analysis and kernel density estimation based on the historical data. Then, the certainty equivalent, traditional worst case, data-driven worst case, set-point tracking and offset-free model predictive control algorithm are designed. Finally, a case study of composition control in a propylene distillation column is carried out. Compared with other strategies, the proposed algorithm ensures the quality of the product while achieving small quality surplus and low operating cost.

Grid following converters stability study and control enhancements using an improved test setup

To insure a stable and reliable operation of the existing and future grids, it is important to study the stability of the power electronic based converters’ controls which can replace synchronous generation. Most existing studies use a Thévenin equivalent model of the grid to test the grid following controls. The present article demonstrates the limits of these studies. An alternative setup is proposed and is shown to provide more insight into the underlying mechanisms of the grid following control behaviour in weak grids, using a linearized state space model. These insights are exploited to propose further improvements to the control. These improvements are shown to extend the stability of the control while reducing the interactions among converters.

Thrust Vector Controller Comparison for a Finless Rocket

The paper focuses on comparing applicability, tuning, and performance of different controllers implemented and tested on a finless rocket during its boost phase. The objective was to evaluate the advantages and disadvantages of each controller, such that the most appropriate one would then be developed and implemented in real-time in the finless rocket. The compared controllers were Linear Quadratic Regulator (LQR), Linear Quadratic Gaussian (LQG), and Proportional Integral Derivative (PID). To control the attitude of the rocket, emphasis is given to the Thrust Vector Control (TVC) component (sub-system) through the gimballing of the rocket engine. The launcher is commanded through the control input thrust gimbal angle δ, while the output parameter is expressed in terms of the pitch angle θ. After deriving a linearized state–space model, rocket stability is addressed before controller implementation and testing. The comparative study showed that both LQR and LQG track pitch angle changes rapidly, thus providing efficient closed-loop dynamic tracking. Tuning of the LQR controller, through the Q and R weighting matrices, illustrates how variations directly affect performance of the closed-loop system by varying the values of the feedback gain (K). The LQG controller provides a more realistic profile because, in general, not all variables are measurable and available for feedback. However, disturbances affecting the system are better handled and reduced with the PID controller, thus overcoming steady-state errors due to aerodynamic and model uncertainty. Overall controller performance is evaluated in terms of overshoot, settling and rise time, and steady-state error.

On Filtering and Smoothing Algorithms for Linear State-Space Models Having Quantized Output Data

The problem of state estimation of a linear, dynamical state-space system where the output is subject to quantization is challenging and important in different areas of research, such as control systems, communications, and power systems. There are a number of methods and algorithms to deal with this state estimation problem. However, there is no consensus in the control and estimation community on (1) which methods are more suitable for a particular application and why, and (2) how these methods compare in terms of accuracy, computational cost, and user friendliness. In this paper, we provide a comprehensive overview of the state-of-the-art algorithms to deal with state estimation subject to quantized measurements, and an exhaustive comparison among them. The comparison analysis is performed in terms of the accuracy of the state estimation, dimensionality issues, hyperparameter selection, user friendliness, and computational cost. We consider classical approaches and a new development in the literature to obtain the filtering and smoothing distributions of the state conditioned to quantized data. The classical approaches include the extended Kalman filter/smoother, the quantized Kalman filter/smoother, the unscented Kalman filter/smoother, and the sequential Monte Carlo sampling method, also called particle filter/smoother, with its most relevant variants. We also consider a new approach based on the Gaussian sum filter/smoother. Extensive numerical simulations—including a practical application—are presented in order to analyze the accuracy of the state estimation and the computational cost.

Predicting Nonlinear Modal Properties by Measuring Free Vibration Responses

AbstractIdentifying dynamical system models from measurements is a central challenge in the structural dynamics community. Nonlinear system identification, in particular, is a big challenge as there are many possible combinations of model structures, which requires expert knowledge to construct an appropriate model. Furthermore, traditional nonlinear system identification methods require a steady excitation input that is not always available in many practical applications. Recently, a technique referred to as the sparse identification of nonlinear dynamics (SINDy) algorithm was developed to discover mathematical models of general nonlinear systems. The SINDy method finds a generalized linear state-space model for an autonomous nonlinear system by analyzing the collected response data. In this work, the SINDy method is adapted and combined with the shooting method and the numerical continuation technique to form a system identification framework that can predict the nonlinear modal properties of mechanical oscillators. The proposed framework predicts the nonlinear normal modes (NNMs) of these systems by processing the noised data of the systems' free vibration response. In addition, the NNMs and the internal resonance of nonlinear systems at a high energy level can be captured using the proposed framework by processing the response data at a lower energy level. The proposed method is numerically demonstrated on a 2-degree-of-freedom mechanical oscillator. Furthermore, the effects of the measurement error and the excitation condition on NNM prediction are investigated. The NNM prediction framework presented in this paper is fairly general and is applicable to a variety of nonlinear systems.

Data-driven virtual sensor for online loads estimation of drivetrain of wind turbines

Data-driven approaches have gained interest recently in the field of wind energy. Data-driven online estimators have been investigated and demonstrated in several applications such as online loads estimation, wake center position estimations, online damage estimation. The present work demonstrates the application of machine learning algorithms to formulate an estimator of the internal loads acting on the bearings of the drivetrain of onshore wind turbines. The loads estimator is implemented as a linear state-space model that is augmented with a non-linear feed-forward neural network. The estimator infers the loads time series as a function of the standard measurements from the SCADA and condition monitoring systems (CMS). A formal analysis of the available data is carried out to define the structure of the virtual sensor regarding the order of the models, number of states, architecture of neural networks. Correlation coefficient of 98% in the time domain and matching of the frequency signature are achieved. Several applications are mentioned and discussed in this work such as online estimation of the forces for monitoring and model predictive control applications.

Measurement Difference Method: A Universal Tool for Noise Identification

This article deals with noise identification of a system described by the linear time-varying state-space model using correlation methods. In particular, the stress is laid on the measurement difference method (MDM) as a universal tool allowing estimation of moments and parameters of the state and measurement noises. The recent results are summarized in a common framework and the full (and weighted) MDM implementation is developed. This implementation provides unbiased and weakly consistent estimate of an arbitrary raw or central moment of the state and measurement noises. The performance of the method is shown in a numerical study.

Simple Factor Realized Stochastic Volatility Models

Abstract This paper considers the use of multiple noisy daily realized variance measures to extract a denoised latent variance process. The class of stochastic volatility models used for signal extraction has the important feature that they can be written as a linear state space model. As a result, prediction of the denoised latent variance and likelihood evaluation can be carried out efficiently using the Kalman filter. This is in contrast to stochastic models that jointly model the return and variance, which require computationally expensive nonlinear filtering for prediction and inference. The gain from using multiple noisy daily variance measures is examined empirically for the S&P 500 index using daily OHLC (open-high-low-close) data.

Time series clustering with an EM algorithm for mixtures of linear Gaussian state space models

In this paper, we consider the task of clustering a set of individual time series while modeling each cluster, that is, model-based time series clustering. The task requires a parametric model with sufficient flexibility to describe the dynamics in various time series. To address this problem, we propose a novel model-based time series clustering method with mixtures of linear Gaussian state space models, which have high flexibility. The proposed method uses a new expectation-maximization algorithm for the mixture model to estimate the model parameters, and determines the number of clusters using the Bayesian information criterion. Experiments on a simulated dataset demonstrate the effectiveness of the method in clustering, parameter estimation, and model selection. The method is applied to real datasets commonly used to evaluate time series clustering methods. Results showed that the proposed method produces clustering results that are as accurate or more accurate than those obtained using previous methods.

Design of intelligent nonlinear robust controller for hydro-turbine governing system based on state-dynamic-measurement hybrid feedback linearization method

Due to the stochastic grid loads and intermittent renewable energy sources in today’s power grid, hydro-generator and grid system stability is increasingly dependent on the hydro-turbine governing system (HTGS). For the HTGS with external disturbances and system uncertainties, an intelligent nonlinear robust controller (INRC) based on state-dynamic-measurement feedback linearization (SDMFL) is proposed to enhance the HTGS’s control performance in this paper. This strategy establishes a dynamic output function including actuator displacement, generator rotor angle, and turbine parameters. The relationship between dynamic output and control input is derived based on this function to establish the corresponding linear state-space model. Then, the mixed H2/H∞ robust controller based on the linear model is designed to suppress the effects of external disturbances and system uncertainties on the system performance. In addition, the controller performance is optimized by levy flight and chaos theory-based gravitational search algorithm (LCGSA), resulting in the INRC. Simulations are performed in MATLAB, and the results are compared with the nonlinear robust controller (NRC) and PID controller. The results show that the proposed controller INRC improves the control performance of nonlinear HTGS with smaller overshoot and shorter adjustment time in all cases.